Re: [Qemu-block] [Qemu-devel] [PATCH v2 6/7] iotests: allow pretty-print for qmp_log

[Vladimir Sementsov-Ogievskiy]

14.12.2018 23:51, John Snow wrote:
> 
> 
> On 12/13/18 8:09 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 13.12.2018 4:50, John Snow wrote:
>>> If iotests have lines exceeding >998 characters long, git doesn't
>>> want to send it plaintext to the list. We can solve this by allowing
>>> the iotests to use pretty printed QMP output that we can match against
>>> instead.
>>>
>>> As a bonus, it's much nicer for human eyes, too.
>>>
>>> Signed-off-by: John Snow 
>>> ---
>>>tests/qemu-iotests/iotests.py | 4 ++--
>>>1 file changed, 2 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
>>> index 9595429fea..dbbef4bad3 100644
>>> --- a/tests/qemu-iotests/iotests.py
>>> +++ b/tests/qemu-iotests/iotests.py
>>> @@ -447,12 +447,12 @@ class VM(qtest.QEMUQtestMachine):
>>>result.append(filter_qmp_event(ev))
>>>return result
>>>
>>> -def qmp_log(self, cmd, filters=[filter_testfiles], **kwargs):
>>> +def qmp_log(self, cmd, indent=None, filters=[filter_testfiles], 
>>> **kwargs):
>>>logmsg = '{"execute": "%s", "arguments": %s}' % \
>>>(cmd, json.dumps(kwargs, sort_keys=True))
>>>log(logmsg, filters)
>>
>> why not to prettify cmd json too? Just make fullobj = {"execute": cmd, 
>> "arguments": kwargs}, and prettify it.
>> (hmm, unfortunately, "execute" < "arguments", and they will be rearranged)
>>
> 
> It wasn't long enough to irritate me :)
> but we're here, so I'll do that too.
> 
>> with or without (as the patch don't aim to prettify everything):
>> Reviewed-by: Vladimir Sementsov-Ogievskiy 
>>
>>>result = self.qmp(cmd, **kwargs)
>>> -log(json.dumps(result, sort_keys=True), filters)
>>> +log(json.dumps(result, indent=indent, sort_keys=True), filters)
>>>return result
>>
>> hmm, doing the same thing as Eric (check specs), I see the following note:
>>
>>   > Note: Since the default item separator is ', ', the output might include 
>> trailing whitespace when indent is specified.
>>
>> And I remember a pain of trailing whitespaces in iotests on, may be, 
>> backporting, or something like this some time ago.
>> It's of course not about this patch, but I think it is a good idea to avoid 
>> trailing whitespaces in test output, at least
>> in common helpers. May be best place to fix is iotests.log() function
>>
> 
> Oh, good spot. I actually did run into this and wasn't aware of what
> caused it! I will change the default separator.

In this case output would be less pretty. I'd prefer just remove trailing 
whitespace as part of filtering process in log().

> 
>>>
>>>def run_job(self, job, auto_finalize=True, auto_dismiss=False):
>>>
>>
>>


-- 
Best regards,
Vladimir

Re: [Qemu-block] [PATCH v10 8/9] qcow2: skip writing zero buffers to empty COW areas

[Anton Nefedov]

On 14/12/2018 7:20 PM, Vladimir Sementsov-Ogievskiy wrote:
> 03.12.2018 13:14, Anton Nefedov wrote:
>> If COW areas of the newly allocated clusters are zeroes on the backing image,
>> efficient bdrv_write_zeroes(flags=BDRV_REQ_ALLOCATE) can be used on the whole
>> cluster instead of writing explicit zero buffers later in perform_cow().
>>
>> iotest 060:
>> write to the discarded cluster does not trigger COW anymore.
>> Use a backing image instead.
>>
> 
> [..]
> 
>> --- a/tests/qemu-iotests/060
>> +++ b/tests/qemu-iotests/060
>> @@ -150,27 +150,33 @@ $QEMU_IO -c "$OPEN_RO" -c "read -P 1 0 512" | 
>> _filter_qemu_io
>>echo
>>echo "=== Testing overlap while COW is in flight ==="
>>echo
>> +BACKING_IMG=$TEST_IMG.base
>> +TEST_IMG=$BACKING_IMG _make_test_img 1G
>> +
>> +$QEMU_IO -c 'write 64k 64k' "$BACKING_IMG" | _filter_qemu_io
>> +
>># compat=0.10 is required in order to make the following discard actually
>> -# unallocate the sector rather than make it a zero sector - we want COW, 
>> after
>> -# all.
>> -IMGOPTS='compat=0.10' _make_test_img 1G
>> +# unallocate the sector rather than make it a zero sector as we would like
>> +# to reuse it for another guest offset
>> +IMGOPTS='compat=0.10' _make_test_img -b "$BACKING_IMG" 1G
>># Write two clusters, the second one enforces creation of an L2 table 
>> after
>># the first data cluster.
>>$QEMU_IO -c 'write 0k 64k' -c 'write 512M 64k' "$TEST_IMG" | 
>> _filter_qemu_io
>> -# Discard the first cluster. This cluster will soon enough be reallocated 
>> and
>> -# used for COW.
>> +# Discard the first cluster. This cluster will soon enough be reallocated
>>$QEMU_IO -c 'discard 0k 64k' "$TEST_IMG" | _filter_qemu_io
>># Now, corrupt the image by marking the second L2 table cluster as free.
>>poke_file "$TEST_IMG" '131084' "\x00\x00" # 0x2000c
>> -# Start a write operation requiring COW on the image stopping it right 
>> before
>> -# doing the read; then, trigger the corruption prevention by writing 
>> anything to
>> -# any unallocated cluster, leading to an attempt to overwrite the second L2
>> +# Start a write operation requiring COW on the image;
>> +# this write will reuse the host offset released by a previous discard.
>> +# Stop it right before doing the read.
>> +# Then, trigger the corruption prevention by writing anything to
>> +# another unallocated cluster, leading to an attempt to overwrite the 
>> second L2
>># table. Finally, resume the COW write and see it fail (but not crash).
>>echo "open -o file.driver=blkdebug $TEST_IMG
>>break cow_read 0
>> -aio_write 0k 1k
>> +aio_write 64k 1k
>>wait_break 0
>> -write 64k 64k
>> +write 128k 64k
> 
> don't understand why you need these changes.
> 
> works for me, without them, if write to backing at 0 offset, of course.
> 
> As I understand, discard create unallocated holes in top qcow2 for old qcow2 
> version.
> 

Ok, so COW happens regardless if this guest offset has been discarded
before. These offset changes are indeed not needed. Just the backing
file.

[Qemu-block] [PATCH v5 02/18] xen: introduce new 'XenBus' and 'XenDevice' object hierarchy

[Paul Durrant]

This patch adds the basic boilerplate for a 'XenBus' object that will act
as a parent to 'XenDevice' PV backends.
A new 'XenBridge' object is also added to connect XenBus to the system bus.

The XenBus object is instantiated by a new xen_bus_init() function called
from the same sites as the legacy xen_be_init() function.

Subsequent patches will flesh-out the functionality of these objects.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: "Michael S. Tsirkin" 
Cc: Marcel Apfelbaum 
Cc: Paolo Bonzini 
Cc: Richard Henderson 
Cc: Eduardo Habkost 

v3:
 - Don't use local_err unless it is needed

v2:
 - Fix boilerplate
 - Make xen-bus hotplug capable
---
 hw/i386/xen/xen-hvm.c |   3 ++
 hw/xen/Makefile.objs  |   2 +-
 hw/xen/trace-events   |   6 +++
 hw/xen/xen-bus.c  | 127 ++
 hw/xenpv/xen_machine_pv.c |   3 ++
 include/hw/xen/xen-bus.h  |  55 
 6 files changed, 195 insertions(+), 1 deletion(-)
 create mode 100644 hw/xen/xen-bus.c
 create mode 100644 include/hw/xen/xen-bus.h

diff --git a/hw/i386/xen/xen-hvm.c b/hw/i386/xen/xen-hvm.c
index 1d63763..4497f75 100644
--- a/hw/i386/xen/xen-hvm.c
+++ b/hw/i386/xen/xen-hvm.c
@@ -17,6 +17,7 @@
 #include "hw/i386/apic-msidef.h"
 #include "hw/xen/xen_common.h"
 #include "hw/xen/xen-legacy-backend.h"
+#include "hw/xen/xen-bus.h"
 #include "qapi/error.h"
 #include "qapi/qapi-commands-misc.h"
 #include "qemu/error-report.h"
@@ -1479,6 +1480,8 @@ void xen_hvm_init(PCMachineState *pcms, MemoryRegion 
**ram_memory)
 QLIST_INIT(&state->dev_list);
 device_listener_register(&state->device_listener);
 
+xen_bus_init();
+
 /* Initialize backend core & drivers */
 if (xen_be_init() != 0) {
 error_report("xen backend core setup failed");
diff --git a/hw/xen/Makefile.objs b/hw/xen/Makefile.objs
index 3f64a44..d9d6d7b 100644
--- a/hw/xen/Makefile.objs
+++ b/hw/xen/Makefile.objs
@@ -1,5 +1,5 @@
 # xen backend driver support
-common-obj-$(CONFIG_XEN) += xen-legacy-backend.o xen_devconfig.o xen_pvdev.o 
xen-common.o
+common-obj-$(CONFIG_XEN) += xen-legacy-backend.o xen_devconfig.o xen_pvdev.o 
xen-common.o xen-bus.o
 
 obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen-host-pci-device.o
 obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen_pt.o xen_pt_config_init.o 
xen_pt_graphics.o xen_pt_msi.o
diff --git a/hw/xen/trace-events b/hw/xen/trace-events
index c7e7a3b..0172cd4 100644
--- a/hw/xen/trace-events
+++ b/hw/xen/trace-events
@@ -12,3 +12,9 @@ xen_unmap_portio_range(uint32_t id, uint64_t start_addr, 
uint64_t end_addr) "id:
 xen_map_pcidev(uint32_t id, uint8_t bus, uint8_t dev, uint8_t func) "id: %u 
bdf: %02x.%02x.%02x"
 xen_unmap_pcidev(uint32_t id, uint8_t bus, uint8_t dev, uint8_t func) "id: %u 
bdf: %02x.%02x.%02x"
 xen_domid_restrict(int err) "err: %u"
+
+# include/hw/xen/xen-bus.c
+xen_bus_realize(void) ""
+xen_bus_unrealize(void) ""
+xen_device_realize(const char *type) "type: %s"
+xen_device_unrealize(const char *type) "type: %s"
diff --git a/hw/xen/xen-bus.c b/hw/xen/xen-bus.c
new file mode 100644
index 000..bc04da7
--- /dev/null
+++ b/hw/xen/xen-bus.c
@@ -0,0 +1,127 @@
+/*
+ * Copyright (c) 2018  Citrix Systems Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/hw.h"
+#include "hw/sysbus.h"
+#include "hw/xen/xen-bus.h"
+#include "qapi/error.h"
+#include "trace.h"
+
+static void xen_bus_unrealize(BusState *bus, Error **errp)
+{
+trace_xen_bus_unrealize();
+}
+
+static void xen_bus_realize(BusState *bus, Error **errp)
+{
+trace_xen_bus_realize();
+}
+
+static void xen_bus_class_init(ObjectClass *class, void *data)
+{
+BusClass *bus_class = BUS_CLASS(class);
+
+bus_class->realize = xen_bus_realize;
+bus_class->unrealize = xen_bus_unrealize;
+}
+
+static const TypeInfo xen_bus_type_info = {
+.name = TYPE_XEN_BUS,
+.parent = TYPE_BUS,
+.instance_size = sizeof(XenBus),
+.class_size = sizeof(XenBusClass),
+.class_init = xen_bus_class_init,
+.interfaces = (InterfaceInfo[]) {
+{ TYPE_HOTPLUG_HANDLER },
+{ }
+},
+};
+
+static void xen_device_unrealize(DeviceState *dev, Error **errp)
+{
+XenDevice *xendev = XEN_DEVICE(dev);
+XenDeviceClass *xendev_class = XEN_DEVICE_GET_CLASS(xendev);
+const char *type = object_get_typename(OBJECT(xendev));
+
+trace_xen_device_unrealize(type);
+
+if (xendev_class->unrealize) {
+xendev_class->unrealize(xendev, errp);
+}
+}
+
+static void xen_device_realize(DeviceState *dev, Error **errp)
+{
+XenDevice *xendev = XEN_DEVICE(dev);
+XenDeviceClass *xendev_class = XEN_DEVICE_GET_CLASS(xendev);
+const char *type = object_get_typename(OBJECT(xendev));
+Error *local_err = NULL;
+
+trace_xen_device_realize(type);
+
+if (xendev_class->realize) {
+xendev_class->realize(

[Qemu-block] [PATCH v5 07/18] xen: add event channel interface for XenDevice-s

[Paul Durrant]

The legacy PV backend infrastructure provides functions to bind, unbind
and send notifications to event channnels. Similar functionality will be
required by XenDevice implementations so this patch adds the necessary
support.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Stefano Stabellini 

v2:
 - Added error pointers to notify and unbind
---
 hw/xen/xen-bus.c | 101 +++
 include/hw/xen/xen-bus.h |  18 +
 2 files changed, 119 insertions(+)

diff --git a/hw/xen/xen-bus.c b/hw/xen/xen-bus.c
index faa9fd3..9443f27 100644
--- a/hw/xen/xen-bus.c
+++ b/hw/xen/xen-bus.c
@@ -617,6 +617,81 @@ done:
 g_free(xengnttab_segs);
 }
 
+struct XenEventChannel {
+unsigned int local_port;
+XenEventHandler handler;
+void *opaque;
+Notifier notifier;
+};
+
+static void event_notify(Notifier *n, void *data)
+{
+XenEventChannel *channel = container_of(n, XenEventChannel, notifier);
+unsigned long port = (unsigned long)data;
+
+if (port == channel->local_port) {
+channel->handler(channel->opaque);
+}
+}
+
+XenEventChannel *xen_device_bind_event_channel(XenDevice *xendev,
+   unsigned int port,
+   XenEventHandler handler,
+   void *opaque, Error **errp)
+{
+XenEventChannel *channel = g_new0(XenEventChannel, 1);
+
+channel->local_port = xenevtchn_bind_interdomain(xendev->xeh,
+ xendev->frontend_id,
+ port);
+if (xendev->local_port < 0) {
+error_setg_errno(errp, errno, "xenevtchn_bind_interdomain failed");
+
+g_free(channel);
+return NULL;
+}
+
+channel->handler = handler;
+channel->opaque = opaque;
+channel->notifier.notify = event_notify;
+
+notifier_list_add(&xendev->event_notifiers, &channel->notifier);
+
+return channel;
+}
+
+void xen_device_notify_event_channel(XenDevice *xendev,
+ XenEventChannel *channel,
+ Error **errp)
+{
+if (!channel) {
+error_setg(errp, "bad channel");
+return;
+}
+
+if (xenevtchn_notify(xendev->xeh, channel->local_port) < 0) {
+error_setg_errno(errp, errno, "xenevtchn_notify failed");
+}
+}
+
+void xen_device_unbind_event_channel(XenDevice *xendev,
+ XenEventChannel *channel,
+ Error **errp)
+{
+if (!channel) {
+error_setg(errp, "bad channel");
+return;
+}
+
+notifier_remove(&channel->notifier);
+
+if (xenevtchn_unbind(xendev->xeh, channel->local_port) < 0) {
+error_setg_errno(errp, errno, "xenevtchn_unbind failed");
+}
+
+g_free(channel);
+}
+
 static void xen_device_unrealize(DeviceState *dev, Error **errp)
 {
 XenDevice *xendev = XEN_DEVICE(dev);
@@ -641,6 +716,12 @@ static void xen_device_unrealize(DeviceState *dev, Error 
**errp)
 xen_device_frontend_destroy(xendev);
 xen_device_backend_destroy(xendev);
 
+if (xendev->xeh) {
+qemu_set_fd_handler(xenevtchn_fd(xendev->xeh), NULL, NULL, NULL);
+xenevtchn_close(xendev->xeh);
+xendev->xeh = NULL;
+}
+
 if (xendev->xgth) {
 xengnttab_close(xendev->xgth);
 xendev->xgth = NULL;
@@ -657,6 +738,16 @@ static void xen_device_exit(Notifier *n, void *data)
 xen_device_unrealize(DEVICE(xendev), &error_abort);
 }
 
+static void xen_device_event(void *opaque)
+{
+XenDevice *xendev = opaque;
+unsigned long port = xenevtchn_pending(xendev->xeh);
+
+notifier_list_notify(&xendev->event_notifiers, (void *)port);
+
+xenevtchn_unmask(xendev->xeh, port);
+}
+
 static void xen_device_realize(DeviceState *dev, Error **errp)
 {
 XenDevice *xendev = XEN_DEVICE(dev);
@@ -697,6 +788,16 @@ static void xen_device_realize(DeviceState *dev, Error 
**errp)
 xendev->feature_grant_copy =
 (xengnttab_grant_copy(xendev->xgth, 0, NULL) == 0);
 
+xendev->xeh = xenevtchn_open(NULL, 0);
+if (!xendev->xeh) {
+error_setg_errno(errp, errno, "failed xenevtchn_open");
+goto unrealize;
+}
+
+notifier_list_init(&xendev->event_notifiers);
+qemu_set_fd_handler(xenevtchn_fd(xendev->xeh), xen_device_event, NULL,
+xendev);
+
 xen_device_backend_create(xendev, &local_err);
 if (local_err) {
 error_propagate(errp, local_err);
diff --git a/include/hw/xen/xen-bus.h b/include/hw/xen/xen-bus.h
index 63a09b6..f83a95c 100644
--- a/include/hw/xen/xen-bus.h
+++ b/include/hw/xen/xen-bus.h
@@ -26,6 +26,9 @@ typedef struct XenDevice {
 XenWatch *frontend_state_watch;
 xengnttab_handle *xgth;
 bool feature_grant_copy;
+xenevtchn_handle *xeh;
+xenevtchn_port_or_error_t local_port;
+ 

[Qemu-block] [PATCH v5 00/18] Xen PV backend 'qdevification'

[Paul Durrant]

This series introduces a new QOM compliant framework for Xen PV backends.
This is achieved by first moving the current non-compliant framework aside,
before building up a new framework incrementally.

This series was prompted by a thread [1] started by Kevin Wolf in response
to patches against xen_disk.c posted by Tim Smith. Therefore, alongside
the patches introducing the new framework, other patches build up a QOM
compliant replacement for 'xen_disk', called 'xen-qdisk'. Patch #16 swaps
this new device into place (having establisheda mechanism to auto-
instantiate devices that is compliant with existing Xen toolstacks in
patch #15) and patch #18 then removes the old xen_disk code.

Subsequent series will port other Xen PV backends across to the new
framework.

The series is also available as a repository branch [2] on xenbits.xen.org.

[1] https://lists.gnu.org/archive/html/qemu-devel/2018-11/msg00259.html
[2] 
http://xenbits.xen.org/gitweb/?p=people/pauldu/qemu.git;a=shortlog;h=refs/heads/qom30

Paul Durrant (18):
  xen: re-name XenDevice to XenLegacyDevice...
  xen: introduce new 'XenBus' and 'XenDevice' object hierarchy
  xen: introduce 'xen-block', 'xen-disk' and 'xen-cdrom'
  xen: create xenstore areas for XenDevice-s
  xen: add xenstore watcher infrastructure
  xen: add grant table interface for XenDevice-s
  xen: add event channel interface for XenDevice-s
  xen: duplicate xen_disk.c as basis of dataplane/xen-block.c
  xen: remove unnecessary code from dataplane/xen-block.c
  xen: add header and build dataplane/xen-block.c
  xen: remove 'XenBlkDev' and 'blkdev' names from dataplane/xen-block
  xen: remove 'ioreq' struct/varable/field names from
dataplane/xen-block.c
  xen: purge 'blk' and 'ioreq' from function names in
dataplane/xen-block.c
  xen: add implementations of xen-block connect and disconnect
functions...
  xen: add a mechanism to automatically create XenDevice-s...
  xen: automatically create XenBlockDevice-s
  MAINTAINERS: add myself as a Xen maintainer
  xen: remove the legacy 'xen_disk' backend

 MAINTAINERS |5 +-
 hw/9pfs/xen-9p-backend.c|   16 +-
 hw/block/Makefile.objs  |2 +-
 hw/block/dataplane/Makefile.objs|1 +
 hw/block/dataplane/xen-block.c  |  814 
 hw/block/dataplane/xen-block.h  |   29 +
 hw/block/trace-events   |   14 +
 hw/block/xen-block.c| 1007 +
 hw/block/xen_disk.c | 1011 -
 hw/char/xen_console.c   |   12 +-
 hw/display/xenfb.c  |   25 +-
 hw/i386/xen/xen-hvm.c   |5 +-
 hw/i386/xen/xen-mapcache.c  |2 +-
 hw/i386/xen/xen_platform.c  |2 +-
 hw/net/xen_nic.c|   14 +-
 hw/usb/xen-usb.c|   25 +-
 hw/xen/Makefile.objs|2 +-
 hw/xen/trace-events |   26 +
 hw/xen/xen-backend.c|  165 +
 hw/xen/xen-bus-helper.c |  184 ++
 hw/xen/xen-bus.c| 1197 +++
 hw/xen/xen-common.c |2 +-
 hw/xen/xen-legacy-backend.c |  853 +
 hw/xen/xen_backend.c|  845 -
 hw/xen/xen_devconfig.c  |2 +-
 hw/xen/xen_pt.c |2 +-
 hw/xen/xen_pt_config_init.c |2 +-
 hw/xen/xen_pt_graphics.c|2 +-
 hw/xen/xen_pt_msi.c |2 +-
 hw/xen/xen_pvdev.c  |   20 +-
 hw/xenpv/xen_domainbuild.c  |2 +-
 hw/xenpv/xen_machine_pv.c   |5 +-
 include/hw/xen/xen-backend.h|   39 ++
 include/hw/xen/xen-block.h  |   95 +++
 include/hw/xen/xen-bus-helper.h |   45 ++
 include/hw/xen/xen-bus.h|  138 
 include/hw/xen/xen-legacy-backend.h |  104 +++
 include/hw/xen/xen_backend.h|   99 ---
 include/hw/xen/xen_pvdev.h  |   38 +-
 include/qemu/module.h   |3 +
 40 files changed, 4814 insertions(+), 2042 deletions(-)
 create mode 100644 hw/block/dataplane/xen-block.c
 create mode 100644 hw/block/dataplane/xen-block.h
 create mode 100644 hw/block/xen-block.c
 delete mode 100644 hw/block/xen_disk.c
 create mode 100644 hw/xen/xen-backend.c
 create mode 100644 hw/xen/xen-bus-helper.c
 create mode 100644 hw/xen/xen-bus.c
 create mode 100644 hw/xen/xen-legacy-backend.c
 delete mode 100644 hw/xen/xen_backend.c
 create mode 100644 include/hw/xen/xen-backend.h
 create mode 100644 include/hw/xen/xen-block.h
 create mode 100644 include/hw/xen/xen-bus-helper.h
 create mode 100644 include/hw/xen/xen-bus.h
 create mode 100644 include/hw/xen/xen-legacy-backend.h
 delete mode 100644 include/hw/xen/xen_backend.h
---
Cc: Anthony Perard 
Cc: Eduardo Habkost 
Cc: Gerd Hoffmann 
Cc: Greg Kurz 
Cc: Jason Wang 
Cc: Kevin Wolf 
Cc: "Marc-André Lureau" 
Cc: Ma

[Qemu-block] [PATCH v5 01/18] xen: re-name XenDevice to XenLegacyDevice...

[Paul Durrant]

...and xen_backend.h to xen-legacy-backend.h

Rather than attempting to convert the existing backend infrastructure to
be QOM compliant (which would be hard to do in an incremental fashion),
subsequent patches will introduce a completely new framework for Xen PV
backends. Hence it is necessary to re-name parts of existing code to avoid
name clashes. The re-named 'legacy' infrastructure will be removed once all
backends have been ported to the new framework.

This patch is purely cosmetic. No functional change.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: Greg Kurz 
Cc: Kevin Wolf 
Cc: Max Reitz 
Cc: "Marc-André Lureau" 
Cc: Paolo Bonzini 
Cc: Richard Henderson 
Cc: Eduardo Habkost 
Cc: "Michael S. Tsirkin" 
Cc: Marcel Apfelbaum 
Cc: Jason Wang 
Cc: Gerd Hoffmann 
---
 hw/9pfs/xen-9p-backend.c|  16 +-
 hw/block/xen_disk.c |  24 +-
 hw/char/xen_console.c   |  12 +-
 hw/display/xenfb.c  |  25 +-
 hw/i386/xen/xen-hvm.c   |   2 +-
 hw/i386/xen/xen-mapcache.c  |   2 +-
 hw/i386/xen/xen_platform.c  |   2 +-
 hw/net/xen_nic.c|  14 +-
 hw/usb/xen-usb.c|  25 +-
 hw/xen/Makefile.objs|   2 +-
 hw/xen/xen-common.c |   2 +-
 hw/xen/xen-legacy-backend.c | 854 
 hw/xen/xen_backend.c| 845 ---
 hw/xen/xen_devconfig.c  |   2 +-
 hw/xen/xen_pt.c |   2 +-
 hw/xen/xen_pt_config_init.c |   2 +-
 hw/xen/xen_pt_graphics.c|   2 +-
 hw/xen/xen_pt_msi.c |   2 +-
 hw/xen/xen_pvdev.c  |  20 +-
 hw/xenpv/xen_domainbuild.c  |   2 +-
 hw/xenpv/xen_machine_pv.c   |   2 +-
 include/hw/xen/xen-legacy-backend.h | 104 +
 include/hw/xen/xen_backend.h|  99 -
 include/hw/xen/xen_pvdev.h  |  38 +-
 24 files changed, 1059 insertions(+), 1041 deletions(-)
 create mode 100644 hw/xen/xen-legacy-backend.c
 delete mode 100644 hw/xen/xen_backend.c
 create mode 100644 include/hw/xen/xen-legacy-backend.h
 delete mode 100644 include/hw/xen/xen_backend.h

diff --git a/hw/9pfs/xen-9p-backend.c b/hw/9pfs/xen-9p-backend.c
index 3f54a21..3859a06 100644
--- a/hw/9pfs/xen-9p-backend.c
+++ b/hw/9pfs/xen-9p-backend.c
@@ -12,7 +12,7 @@
 
 #include "hw/hw.h"
 #include "hw/9pfs/9p.h"
-#include "hw/xen/xen_backend.h"
+#include "hw/xen/xen-legacy-backend.h"
 #include "hw/9pfs/xen-9pfs.h"
 #include "qapi/error.h"
 #include "qemu/config-file.h"
@@ -45,7 +45,7 @@ typedef struct Xen9pfsRing {
 } Xen9pfsRing;
 
 typedef struct Xen9pfsDev {
-struct XenDevice xendev;  /* must be first */
+struct XenLegacyDevice xendev;  /* must be first */
 V9fsState state;
 char *path;
 char *security_model;
@@ -56,7 +56,7 @@ typedef struct Xen9pfsDev {
 Xen9pfsRing *rings;
 } Xen9pfsDev;
 
-static void xen_9pfs_disconnect(struct XenDevice *xendev);
+static void xen_9pfs_disconnect(struct XenLegacyDevice *xendev);
 
 static void xen_9pfs_in_sg(Xen9pfsRing *ring,
struct iovec *in_sg,
@@ -243,7 +243,7 @@ static const V9fsTransport xen_9p_transport = {
 .push_and_notify = xen_9pfs_push_and_notify,
 };
 
-static int xen_9pfs_init(struct XenDevice *xendev)
+static int xen_9pfs_init(struct XenLegacyDevice *xendev)
 {
 return 0;
 }
@@ -305,7 +305,7 @@ static void xen_9pfs_evtchn_event(void *opaque)
 qemu_bh_schedule(ring->bh);
 }
 
-static void xen_9pfs_disconnect(struct XenDevice *xendev)
+static void xen_9pfs_disconnect(struct XenLegacyDevice *xendev)
 {
 Xen9pfsDev *xen_9pdev = container_of(xendev, Xen9pfsDev, xendev);
 int i;
@@ -321,7 +321,7 @@ static void xen_9pfs_disconnect(struct XenDevice *xendev)
 }
 }
 
-static int xen_9pfs_free(struct XenDevice *xendev)
+static int xen_9pfs_free(struct XenLegacyDevice *xendev)
 {
 Xen9pfsDev *xen_9pdev = container_of(xendev, Xen9pfsDev, xendev);
 int i;
@@ -354,7 +354,7 @@ static int xen_9pfs_free(struct XenDevice *xendev)
 return 0;
 }
 
-static int xen_9pfs_connect(struct XenDevice *xendev)
+static int xen_9pfs_connect(struct XenLegacyDevice *xendev)
 {
 Error *err = NULL;
 int i;
@@ -467,7 +467,7 @@ out:
 return -1;
 }
 
-static void xen_9pfs_alloc(struct XenDevice *xendev)
+static void xen_9pfs_alloc(struct XenLegacyDevice *xendev)
 {
 xenstore_write_be_str(xendev, "versions", VERSIONS);
 xenstore_write_be_int(xendev, "max-rings", MAX_RINGS);
diff --git a/hw/block/xen_disk.c b/hw/block/xen_disk.c
index 36eff94..75fe55f 100644
--- a/hw/block/xen_disk.c
+++ b/hw/block/xen_disk.c
@@ -25,7 +25,7 @@
 #include 
 
 #include "hw/hw.h"
-#include "hw/xen/xen_backend.h"
+#include "hw/xen/xen-legacy-backend.h"
 #include "xen_blkif.h"
 #include "sysemu/blockdev.h"
 #include "sysemu/iothread.h"
@@ -63,7 +63,7 @@ struct ioreq {
 #define MAX_RING_PAGE_OR

[Qemu-block] [PATCH v5 06/18] xen: add grant table interface for XenDevice-s

[Paul Durrant]

The legacy PV backend infrastructure provides functions to map, unmap and
copy pages granted by frontends. Similar functionality will be required
by XenDevice implementations so this patch adds the necessary support.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Stefano Stabellini 
---
 hw/xen/xen-bus.c | 146 +++
 include/hw/xen/xen-bus.h |  25 
 2 files changed, 171 insertions(+)

diff --git a/hw/xen/xen-bus.c b/hw/xen/xen-bus.c
index 5e19592..faa9fd3 100644
--- a/hw/xen/xen-bus.c
+++ b/hw/xen/xen-bus.c
@@ -485,6 +485,138 @@ static void xen_device_frontend_destroy(XenDevice *xendev)
 }
 }
 
+void xen_device_set_max_grant_refs(XenDevice *xendev, unsigned int nr_refs,
+   Error **errp)
+{
+if (xengnttab_set_max_grants(xendev->xgth, nr_refs)) {
+error_setg_errno(errp, errno, "xengnttab_set_max_grants failed");
+}
+}
+
+void *xen_device_map_grant_refs(XenDevice *xendev, uint32_t *refs,
+unsigned int nr_refs, int prot,
+Error **errp)
+{
+void *map = xengnttab_map_domain_grant_refs(xendev->xgth, nr_refs,
+xendev->frontend_id, refs,
+prot);
+
+if (!map) {
+error_setg_errno(errp, errno,
+ "xengnttab_map_domain_grant_refs failed");
+}
+
+return map;
+}
+
+void xen_device_unmap_grant_refs(XenDevice *xendev, void *map,
+ unsigned int nr_refs, Error **errp)
+{
+if (xengnttab_unmap(xendev->xgth, map, nr_refs)) {
+error_setg_errno(errp, errno, "xengnttab_unmap failed");
+}
+}
+
+static void compat_copy_grant_refs(XenDevice *xendev, bool to_domain,
+   XenDeviceGrantCopySegment segs[],
+   unsigned int nr_segs, Error **errp)
+{
+uint32_t *refs = g_new(uint32_t, nr_segs);
+int prot = to_domain ? PROT_WRITE : PROT_READ;
+void *map;
+unsigned int i;
+
+for (i = 0; i < nr_segs; i++) {
+XenDeviceGrantCopySegment *seg = &segs[i];
+
+refs[i] = to_domain ? seg->dest.foreign.ref :
+seg->source.foreign.ref;
+}
+
+map = xengnttab_map_domain_grant_refs(xendev->xgth, nr_segs,
+  xendev->frontend_id, refs,
+  prot);
+if (!map) {
+error_setg_errno(errp, errno,
+ "xengnttab_map_domain_grant_refs failed");
+goto done;
+}
+
+for (i = 0; i < nr_segs; i++) {
+XenDeviceGrantCopySegment *seg = &segs[i];
+void *page = map + (i * XC_PAGE_SIZE);
+
+if (to_domain) {
+memcpy(page + seg->dest.foreign.offset, seg->source.virt,
+   seg->len);
+} else {
+memcpy(seg->dest.virt, page + seg->source.foreign.offset,
+   seg->len);
+}
+}
+
+if (xengnttab_unmap(xendev->xgth, map, nr_segs)) {
+error_setg_errno(errp, errno, "xengnttab_unmap failed");
+}
+
+done:
+g_free(refs);
+}
+
+void xen_device_copy_grant_refs(XenDevice *xendev, bool to_domain,
+XenDeviceGrantCopySegment segs[],
+unsigned int nr_segs, Error **errp)
+{
+xengnttab_grant_copy_segment_t *xengnttab_segs;
+unsigned int i;
+
+if (!xendev->feature_grant_copy) {
+compat_copy_grant_refs(xendev, to_domain, segs, nr_segs, errp);
+return;
+}
+
+xengnttab_segs = g_new0(xengnttab_grant_copy_segment_t, nr_segs);
+
+for (i = 0; i < nr_segs; i++) {
+XenDeviceGrantCopySegment *seg = &segs[i];
+xengnttab_grant_copy_segment_t *xengnttab_seg = &xengnttab_segs[i];
+
+if (to_domain) {
+xengnttab_seg->flags = GNTCOPY_dest_gref;
+xengnttab_seg->dest.foreign.domid = xendev->frontend_id;
+xengnttab_seg->dest.foreign.ref = seg->dest.foreign.ref;
+xengnttab_seg->dest.foreign.offset = seg->dest.foreign.offset;
+xengnttab_seg->source.virt = seg->source.virt;
+} else {
+xengnttab_seg->flags = GNTCOPY_source_gref;
+xengnttab_seg->source.foreign.domid = xendev->frontend_id;
+xengnttab_seg->source.foreign.ref = seg->source.foreign.ref;
+xengnttab_seg->source.foreign.offset =
+seg->source.foreign.offset;
+xengnttab_seg->dest.virt = seg->dest.virt;
+}
+
+xengnttab_seg->len = seg->len;
+}
+
+if (xengnttab_grant_copy(xendev->xgth, nr_segs, xengnttab_segs)) {
+error_setg_errno(errp, errno, "xengnttab_grant_copy failed");
+goto done;
+}
+
+for (i = 0; i < nr_segs; i++) {
+xengnttab_grant_copy_segment_t *xengnttab_seg = &xengnttab_segs[i];
+
+if (xengntta

[Qemu-block] [PATCH v5 03/18] xen: introduce 'xen-block', 'xen-disk' and 'xen-cdrom'

[Paul Durrant]

This patch adds new XenDevice-s: 'xen-disk' and 'xen-cdrom', both derived
from a common 'xen-block' parent type. These will eventually replace the
'xen_disk' (note the underscore rather than hyphen) legacy PV backend but
it is illustrative to build up the implementation incrementally, along with
the XenBus/XenDevice framework. Subsequent patches will therefore add to
these devices' implementation as new features are added to the framework.

After this patch has been applied it is possible to instantiate new
'xen-disk' or 'xen-cdrom' devices with a single 'vdev' parameter, which
accepts values adhering to the Xen VBD naming scheme [1]. For example, a
command-line instantiation of a xen-disk can be done with an argument
similar to the following:

-device xen-disk,vdev=hda

The implementation of the vdev parameter formulates the appropriate VBD
number for use in the PV protocol.

[1] https://xenbits.xen.org/docs/unstable/man/xen-vbd-interface.7.html

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Kevin Wolf 
Cc: Max Reitz 
Cc: Stefano Stabellini 

v5:
 - Introduce properties sub-structure (which will be useful later)
 - Keeping Anthony's R-b from v4 since change is trivial

v3:
 - Fix vdev parsing again, and vdev formatting this time
 - Remove an unnecessary local_err

v2:
 - Fix boilerplate
 - Fix vdev parsing
 - Change name from 'xen-qdisk' to 'xen-block', make abstract, and split
   off 'xen-disk' and 'xen-cdrom' as concrete sub-types
---
 MAINTAINERS|   2 +-
 hw/block/Makefile.objs |   1 +
 hw/block/trace-events  |   8 ++
 hw/block/xen-block.c   | 343 +
 include/hw/xen/xen-block.h |  73 ++
 5 files changed, 426 insertions(+), 1 deletion(-)
 create mode 100644 hw/block/xen-block.c
 create mode 100644 include/hw/xen/xen-block.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 63effdc..dd728c3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -403,7 +403,7 @@ F: hw/9pfs/xen-9p-backend.c
 F: hw/char/xen_console.c
 F: hw/display/xenfb.c
 F: hw/net/xen_nic.c
-F: hw/block/xen_*
+F: hw/block/xen*
 F: hw/xen/
 F: hw/xenpv/
 F: hw/i386/xen/
diff --git a/hw/block/Makefile.objs b/hw/block/Makefile.objs
index 53ce575..f34813a 100644
--- a/hw/block/Makefile.objs
+++ b/hw/block/Makefile.objs
@@ -4,6 +4,7 @@ common-obj-$(CONFIG_SSI_M25P80) += m25p80.o
 common-obj-$(CONFIG_NAND) += nand.o
 common-obj-$(CONFIG_PFLASH_CFI01) += pflash_cfi01.o
 common-obj-$(CONFIG_PFLASH_CFI02) += pflash_cfi02.o
+common-obj-$(CONFIG_XEN) += xen-block.o
 common-obj-$(CONFIG_XEN) += xen_disk.o
 common-obj-$(CONFIG_ECC) += ecc.o
 common-obj-$(CONFIG_ONENAND) += onenand.o
diff --git a/hw/block/trace-events b/hw/block/trace-events
index 335c092..4afbd62 100644
--- a/hw/block/trace-events
+++ b/hw/block/trace-events
@@ -127,3 +127,11 @@ xen_disk_init(char *name) "%s"
 xen_disk_connect(char *name) "%s"
 xen_disk_disconnect(char *name) "%s"
 xen_disk_free(char *name) "%s"
+
+# hw/block/xen-block.c
+xen_block_realize(const char *type, uint32_t disk, uint32_t partition) "%s 
d%up%u"
+xen_block_unrealize(const char *type, uint32_t disk, uint32_t partition) "%s 
d%up%u"
+xen_disk_realize(void) ""
+xen_disk_unrealize(void) ""
+xen_cdrom_realize(void) ""
+xen_cdrom_unrealize(void) ""
diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
new file mode 100644
index 000..8e78b6a
--- /dev/null
+++ b/hw/block/xen-block.c
@@ -0,0 +1,343 @@
+/*
+ * Copyright (c) 2018  Citrix Systems Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/cutils.h"
+#include "qapi/error.h"
+#include "qapi/visitor.h"
+#include "hw/hw.h"
+#include "hw/xen/xen-block.h"
+#include "trace.h"
+
+static void xen_block_unrealize(XenDevice *xendev, Error **errp)
+{
+XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
+XenBlockDeviceClass *blockdev_class =
+XEN_BLOCK_DEVICE_GET_CLASS(xendev);
+const char *type = object_get_typename(OBJECT(blockdev));
+XenBlockVdev *vdev = &blockdev->props.vdev;
+
+if (vdev->type == XEN_BLOCK_VDEV_TYPE_INVALID) {
+return;
+}
+
+trace_xen_block_unrealize(type, vdev->disk, vdev->partition);
+
+if (blockdev_class->unrealize) {
+blockdev_class->unrealize(blockdev, errp);
+}
+}
+
+static void xen_block_realize(XenDevice *xendev, Error **errp)
+{
+XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
+XenBlockDeviceClass *blockdev_class =
+XEN_BLOCK_DEVICE_GET_CLASS(xendev);
+const char *type = object_get_typename(OBJECT(blockdev));
+XenBlockVdev *vdev = &blockdev->props.vdev;
+Error *local_err = NULL;
+
+if (vdev->type == XEN_BLOCK_VDEV_TYPE_INVALID) {
+error_setg(errp, "vdev property not set");
+return;
+}
+
+trace_xen_block_realize(type, vdev->disk, vdev->partition);
+
+if (blockdev_class->realize) {
+blockde

[Qemu-block] [PATCH v5 09/18] xen: remove unnecessary code from dataplane/xen-block.c

[Paul Durrant]

Not all of the code duplicated from xen_disk.c is required as the basis for
the new dataplane implementation so this patch removes extraneous code,
along with the legacy #includes and calls to the legacy xen_pv_printf()
function. Error messages are changed to be reported using error_report().

NOTE: The code is still not yet built. Further transformations will be
  required to make it correctly interface to the new XenBus/XenDevice
  framework. They will be delivered in a subsequent patch.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: Stefan Hajnoczi 
Cc: Kevin Wolf 
Cc: Max Reitz 

v2:
 - Leave existing boilerplate alone, other than removing the now-incorrect
   description
---
 hw/block/dataplane/xen-block.c | 409 ++---
 1 file changed, 16 insertions(+), 393 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 9fae505..98f987d 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -1,6 +1,4 @@
 /*
- *  xen paravirt block device backend
- *
  *  (c) Gerd Hoffmann 
  *
  *  This program is free software; you can redistribute it and/or modify
@@ -19,26 +17,12 @@
  *  GNU GPL, version 2 or (at your option) any later version.
  */
 
-#include "qemu/osdep.h"
-#include "qemu/units.h"
-#include 
-#include 
-
-#include "hw/hw.h"
-#include "hw/xen/xen_backend.h"
-#include "xen_blkif.h"
-#include "sysemu/blockdev.h"
-#include "sysemu/iothread.h"
-#include "sysemu/block-backend.h"
-#include "qapi/error.h"
-#include "qapi/qmp/qdict.h"
-#include "qapi/qmp/qstring.h"
-#include "trace.h"
-
-/* - */
-
-#define BLOCK_SIZE  512
-#define IOCB_COUNT  (BLKIF_MAX_SEGMENTS_PER_REQUEST + 2)
+/*
+ * Copyright (c) 2018  Citrix Systems Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
 
 struct ioreq {
 blkif_request_t req;
@@ -101,8 +85,6 @@ struct XenBlkDev {
 AioContext  *ctx;
 };
 
-/* - */
-
 static void ioreq_reset(struct ioreq *ioreq)
 {
 memset(&ioreq->req, 0, sizeof(ioreq->req));
@@ -183,11 +165,6 @@ static int ioreq_parse(struct ioreq *ioreq)
 size_t len;
 int i;
 
-xen_pv_printf(
-xendev, 3,
-"op %d, nr %d, handle %d, id %" PRId64 ", sector %" PRId64 "\n",
-ioreq->req.operation, ioreq->req.nr_segments,
-ioreq->req.handle, ioreq->req.id, ioreq->req.sector_number);
 switch (ioreq->req.operation) {
 case BLKIF_OP_READ:
 break;
@@ -202,28 +179,27 @@ static int ioreq_parse(struct ioreq *ioreq)
 case BLKIF_OP_DISCARD:
 return 0;
 default:
-xen_pv_printf(xendev, 0, "error: unknown operation (%d)\n",
-  ioreq->req.operation);
+error_report("error: unknown operation (%d)", ioreq->req.operation);
 goto err;
 };
 
 if (ioreq->req.operation != BLKIF_OP_READ && blkdev->mode[0] != 'w') {
-xen_pv_printf(xendev, 0, "error: write req for ro device\n");
+error_report("error: write req for ro device");
 goto err;
 }
 
 ioreq->start = ioreq->req.sector_number * blkdev->file_blk;
 for (i = 0; i < ioreq->req.nr_segments; i++) {
 if (i == BLKIF_MAX_SEGMENTS_PER_REQUEST) {
-xen_pv_printf(xendev, 0, "error: nr_segments too big\n");
+error_report("error: nr_segments too big");
 goto err;
 }
 if (ioreq->req.seg[i].first_sect > ioreq->req.seg[i].last_sect) {
-xen_pv_printf(xendev, 0, "error: first > last sector\n");
+error_report("error: first > last sector");
 goto err;
 }
 if (ioreq->req.seg[i].last_sect * BLOCK_SIZE >= XC_PAGE_SIZE) {
-xen_pv_printf(xendev, 0, "error: page crossing\n");
+error_report("error: page crossing");
 goto err;
 }
 
@@ -232,7 +208,7 @@ static int ioreq_parse(struct ioreq *ioreq)
 ioreq->size += len;
 }
 if (ioreq->start + ioreq->size > blkdev->file_size) {
-xen_pv_printf(xendev, 0, "error: access beyond end of file\n");
+error_report("error: access beyond end of file");
 goto err;
 }
 return 0;
@@ -278,8 +254,7 @@ static int ioreq_grant_copy(struct ioreq *ioreq)
 rc = xen_be_copy_grant_refs(xendev, to_domain, segs, count);
 
 if (rc) {
-xen_pv_printf(xendev, 0,
-  "failed to copy data %d\n", rc);
+error_report("failed to copy data %d", rc);
 ioreq->aio_errors++;
 return -1;
 }
@@ -298,8 +273,9 @@ static void qemu_aio_complete(void *opaque, int ret)
 aio_context_acquire(blkdev->ctx);
 
 if (ret != 0) {
-xen_pv_printf(xendev, 0, "%s I/O error\n",
-  ioreq->req.operation == 

[Qemu-block] [PATCH v5 08/18] xen: duplicate xen_disk.c as basis of dataplane/xen-block.c

[Paul Durrant]

The new xen-block XenDevice implementation requires the same core
dataplane as the legacy xen_disk implementation it will eventually replace.
This patch therefore copies the legacy xen_disk.c source module into a new
dataplane/xen-block.c source module as the basis for the new dataplane and
adjusts the MAINTAINERS file accordingly.

NOTE: The duplicated code is not yet built. It is simply put into place by
  this patch (just fixing style violations) such that the
  modifications that will need to be made to the code are not
  conflated with code movement, thus making review harder.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: Stefan Hajnoczi 
Cc: Kevin Wolf 
Cc: Max Reitz 
---
 MAINTAINERS|1 +
 hw/block/dataplane/xen-block.c | 1019 
 2 files changed, 1020 insertions(+)
 create mode 100644 hw/block/dataplane/xen-block.c

diff --git a/MAINTAINERS b/MAINTAINERS
index dd728c3..ab62ad4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -404,6 +404,7 @@ F: hw/char/xen_console.c
 F: hw/display/xenfb.c
 F: hw/net/xen_nic.c
 F: hw/block/xen*
+F: hw/block/dataplane/xen*
 F: hw/xen/
 F: hw/xenpv/
 F: hw/i386/xen/
diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
new file mode 100644
index 000..9fae505
--- /dev/null
+++ b/hw/block/dataplane/xen-block.c
@@ -0,0 +1,1019 @@
+/*
+ *  xen paravirt block device backend
+ *
+ *  (c) Gerd Hoffmann 
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; under version 2 of the License.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License along
+ *  with this program; if not, see .
+ *
+ *  Contributions after 2012-01-13 are licensed under the terms of the
+ *  GNU GPL, version 2 or (at your option) any later version.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/units.h"
+#include 
+#include 
+
+#include "hw/hw.h"
+#include "hw/xen/xen_backend.h"
+#include "xen_blkif.h"
+#include "sysemu/blockdev.h"
+#include "sysemu/iothread.h"
+#include "sysemu/block-backend.h"
+#include "qapi/error.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/qmp/qstring.h"
+#include "trace.h"
+
+/* - */
+
+#define BLOCK_SIZE  512
+#define IOCB_COUNT  (BLKIF_MAX_SEGMENTS_PER_REQUEST + 2)
+
+struct ioreq {
+blkif_request_t req;
+int16_t status;
+
+/* parsed request */
+off_t   start;
+QEMUIOVectorv;
+void*buf;
+size_t  size;
+int presync;
+
+/* aio status */
+int aio_inflight;
+int aio_errors;
+
+struct XenBlkDev*blkdev;
+QLIST_ENTRY(ioreq)   list;
+BlockAcctCookie acct;
+};
+
+#define MAX_RING_PAGE_ORDER 4
+
+struct XenBlkDev {
+struct XenLegacyDevicexendev;  /* must be first */
+char*params;
+char*mode;
+char*type;
+char*dev;
+char*devtype;
+booldirectiosafe;
+const char  *fileproto;
+const char  *filename;
+unsigned intring_ref[1 << MAX_RING_PAGE_ORDER];
+unsigned intnr_ring_ref;
+void*sring;
+int64_t file_blk;
+int64_t file_size;
+int protocol;
+blkif_back_rings_t  rings;
+int more_work;
+
+/* request lists */
+QLIST_HEAD(inflight_head, ioreq) inflight;
+QLIST_HEAD(finished_head, ioreq) finished;
+QLIST_HEAD(freelist_head, ioreq) freelist;
+int requests_total;
+int requests_inflight;
+int requests_finished;
+unsigned intmax_requests;
+
+gbooleanfeature_discard;
+
+/* qemu block driver */
+DriveInfo   *dinfo;
+BlockBackend*blk;
+QEMUBH  *bh;
+
+IOThread*iothread;
+AioContext  *ctx;
+};
+
+/* - */
+
+static void ioreq_reset(struct ioreq *ioreq)
+{
+memset(&ioreq->req, 0, sizeof(ioreq->req));
+ioreq->status = 0;
+ioreq->start = 0;
+ioreq->buf = NULL;
+ioreq->size = 0;
+ioreq->presync = 0;
+
+ioreq->aio_inflight = 0;
+ioreq->aio_errors = 0;
+
+ioreq->blkdev = NULL;
+memset(&ioreq->list, 0, sizeof(ioreq->list));
+memset(&ioreq->acct, 0, sizeof(ioreq->acct))

[Qemu-block] [PATCH v5 04/18] xen: create xenstore areas for XenDevice-s

[Paul Durrant]

This patch adds a new source module, xen-bus-helper.c, which builds on
basic libxenstore primitives to provide functions to create (setting
permissions appropriately) and destroy xenstore areas, and functions to
'printf' and 'scanf' nodes therein. The main xen-bus code then uses
these primitives [1] to initialize and destroy the frontend and backend
areas for a XenDevice during realize and unrealize respectively.

The 'xen-block' implementation is extended with a 'get_name' method that
returns the VBD number. This number is required to 'name' the xenstore
areas.

NOTE: An exit handler is also added to make sure the xenstore areas are
  cleaned up if QEMU terminates without devices being unrealized.

[1] The 'scanf' functions are actually not yet needed, but they will be
needed by code delivered in subsequent patches.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: Kevin Wolf 
Cc: Max Reitz 

v5:
 - Re-base

v3:
 - Add transaction id parameters to xen-bus-helper functions
 - Not added Anthony's R-b because of change

v2:
 - Fix boilerplate
 - Add error pointers to all xs_node... helpers
 - Add GCC_FMT_ATTR to declarations of printf-like helpers
---
 hw/block/xen-block.c|   9 ++
 hw/xen/Makefile.objs|   2 +-
 hw/xen/trace-events |  12 +-
 hw/xen/xen-bus-helper.c | 150 +++
 hw/xen/xen-bus.c| 321 +++-
 include/hw/xen/xen-bus-helper.h |  39 +
 include/hw/xen/xen-bus.h|  12 ++
 7 files changed, 540 insertions(+), 5 deletions(-)
 create mode 100644 hw/xen/xen-bus-helper.c
 create mode 100644 include/hw/xen/xen-bus-helper.h

diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index 8e78b6a..d27a286 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -13,6 +13,14 @@
 #include "hw/xen/xen-block.h"
 #include "trace.h"
 
+static char *xen_block_get_name(XenDevice *xendev, Error **errp)
+{
+XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
+XenBlockVdev *vdev = &blockdev->props.vdev;
+
+return g_strdup_printf("%lu", vdev->number);
+}
+
 static void xen_block_unrealize(XenDevice *xendev, Error **errp)
 {
 XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
@@ -262,6 +270,7 @@ static void xen_block_class_init(ObjectClass *class, void 
*data)
 DeviceClass *dev_class = DEVICE_CLASS(class);
 XenDeviceClass *xendev_class = XEN_DEVICE_CLASS(class);
 
+xendev_class->get_name = xen_block_get_name;
 xendev_class->realize = xen_block_realize;
 xendev_class->unrealize = xen_block_unrealize;
 
diff --git a/hw/xen/Makefile.objs b/hw/xen/Makefile.objs
index d9d6d7b..77c0868 100644
--- a/hw/xen/Makefile.objs
+++ b/hw/xen/Makefile.objs
@@ -1,5 +1,5 @@
 # xen backend driver support
-common-obj-$(CONFIG_XEN) += xen-legacy-backend.o xen_devconfig.o xen_pvdev.o 
xen-common.o xen-bus.o
+common-obj-$(CONFIG_XEN) += xen-legacy-backend.o xen_devconfig.o xen_pvdev.o 
xen-common.o xen-bus.o xen-bus-helper.o
 
 obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen-host-pci-device.o
 obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen_pt.o xen_pt_config_init.o 
xen_pt_graphics.o xen_pt_msi.o
diff --git a/hw/xen/trace-events b/hw/xen/trace-events
index 0172cd4..75dc226 100644
--- a/hw/xen/trace-events
+++ b/hw/xen/trace-events
@@ -16,5 +16,13 @@ xen_domid_restrict(int err) "err: %u"
 # include/hw/xen/xen-bus.c
 xen_bus_realize(void) ""
 xen_bus_unrealize(void) ""
-xen_device_realize(const char *type) "type: %s"
-xen_device_unrealize(const char *type) "type: %s"
+xen_device_realize(const char *type, char *name) "type: %s name: %s"
+xen_device_unrealize(const char *type, char *name) "type: %s name: %s"
+xen_device_backend_state(const char *type, char *name, const char *state) 
"type: %s name: %s -> %s"
+xen_device_frontend_state(const char *type, char *name, const char *state) 
"type: %s name: %s -> %s"
+
+# include/hw/xen/xen-bus-helper.c
+xs_node_create(const char *node) "%s"
+xs_node_destroy(const char *node) "%s"
+xs_node_vprintf(char *path, char *value) "%s %s"
+xs_node_vscanf(char *path, char *value) "%s %s"
diff --git a/hw/xen/xen-bus-helper.c b/hw/xen/xen-bus-helper.c
new file mode 100644
index 000..15b3ad8
--- /dev/null
+++ b/hw/xen/xen-bus-helper.c
@@ -0,0 +1,150 @@
+/*
+ * Copyright (c) 2018  Citrix Systems Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/hw.h"
+#include "hw/sysbus.h"
+#include "hw/xen/xen.h"
+#include "hw/xen/xen-bus.h"
+#include "hw/xen/xen-bus-helper.h"
+#include "qapi/error.h"
+
+#include 
+
+struct xs_state {
+enum xenbus_state statenum;
+const char *statestr;
+};
+#define XS_STATE(state) { state, #state }
+
+static struct xs_state xs_state[] = {
+XS_STATE(XenbusStateUnknown),
+XS_STATE(XenbusStateInitialising),
+XS_STATE(XenbusStateInitWait),
+XS_STAT

[Qemu-block] [PATCH v5 05/18] xen: add xenstore watcher infrastructure

[Paul Durrant]

A Xen PV frontend communicates its state to the PV backend by writing to
the 'state' key in the frontend area in xenstore. It is therefore
necessary for a XenDevice implementation to be notified whenever the
value of this key changes.

This patch adds code to do this as follows:

- an 'fd handler' is registered on the libxenstore handle which will be
  triggered whenever a 'watch' event occurs
- primitives are added to xen-bus-helper to add or remove watch events
- a list of Notifier objects is added to XenBus to provide a mechanism
  to call the appropriate 'watch handler' when its associated event
  occurs

The xen-block implementation is extended with a 'frontend_changed' method,
which calls as-yet stub 'connect' and 'disconnect' functions when the
relevant frontend state transitions occur. A subsequent patch will supply
a full implementation for these functions.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Kevin Wolf 
Cc: Max Reitz 
Cc: Stefano Stabellini 

v5:
 - Re-base

v3:
 - Remove unnecessary instances of local_err

v2:
 - Don't crash when xen_block_disconnect() fails
 - Check xs_unwatch() for error
 - Add new_watch() and free_watch() utility functions
 - Use xs_check_watch() rather than xs_read_watch()
---
 hw/block/trace-events   |   2 +
 hw/block/xen-block.c|  70 +
 hw/xen/trace-events |   6 ++
 hw/xen/xen-bus-helper.c |  34 +++
 hw/xen/xen-bus.c| 211 +++-
 include/hw/xen/xen-bus-helper.h |   6 ++
 include/hw/xen/xen-bus.h|  15 +++
 7 files changed, 342 insertions(+), 2 deletions(-)

diff --git a/hw/block/trace-events b/hw/block/trace-events
index 4afbd62..89e2583 100644
--- a/hw/block/trace-events
+++ b/hw/block/trace-events
@@ -130,6 +130,8 @@ xen_disk_free(char *name) "%s"
 
 # hw/block/xen-block.c
 xen_block_realize(const char *type, uint32_t disk, uint32_t partition) "%s 
d%up%u"
+xen_block_connect(const char *type, uint32_t disk, uint32_t partition) "%s 
d%up%u"
+xen_block_disconnect(const char *type, uint32_t disk, uint32_t partition) "%s 
d%up%u"
 xen_block_unrealize(const char *type, uint32_t disk, uint32_t partition) "%s 
d%up%u"
 xen_disk_realize(void) ""
 xen_disk_unrealize(void) ""
diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index d27a286..3a963b0 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -21,6 +21,24 @@ static char *xen_block_get_name(XenDevice *xendev, Error 
**errp)
 return g_strdup_printf("%lu", vdev->number);
 }
 
+static void xen_block_disconnect(XenDevice *xendev, Error **errp)
+{
+XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
+const char *type = object_get_typename(OBJECT(blockdev));
+XenBlockVdev *vdev = &blockdev->props.vdev;
+
+trace_xen_block_disconnect(type, vdev->disk, vdev->partition);
+}
+
+static void xen_block_connect(XenDevice *xendev, Error **errp)
+{
+XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
+const char *type = object_get_typename(OBJECT(blockdev));
+XenBlockVdev *vdev = &blockdev->props.vdev;
+
+trace_xen_block_connect(type, vdev->disk, vdev->partition);
+}
+
 static void xen_block_unrealize(XenDevice *xendev, Error **errp)
 {
 XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
@@ -35,6 +53,9 @@ static void xen_block_unrealize(XenDevice *xendev, Error 
**errp)
 
 trace_xen_block_unrealize(type, vdev->disk, vdev->partition);
 
+/* Disconnect from the frontend in case this has not already happened */
+xen_block_disconnect(xendev, NULL);
+
 if (blockdev_class->unrealize) {
 blockdev_class->unrealize(blockdev, errp);
 }
@@ -64,6 +85,54 @@ static void xen_block_realize(XenDevice *xendev, Error 
**errp)
 }
 }
 
+static void xen_block_frontend_changed(XenDevice *xendev,
+   enum xenbus_state frontend_state,
+   Error **errp)
+{
+enum xenbus_state backend_state = xen_device_backend_get_state(xendev);
+Error *local_err = NULL;
+
+switch (frontend_state) {
+case XenbusStateInitialised:
+case XenbusStateConnected:
+if (backend_state == XenbusStateConnected) {
+break;
+}
+
+xen_block_disconnect(xendev, &local_err);
+if (local_err) {
+error_propagate(errp, local_err);
+break;
+}
+
+xen_block_connect(xendev, &local_err);
+if (local_err) {
+error_propagate(errp, local_err);
+break;
+}
+
+xen_device_backend_set_state(xendev, XenbusStateConnected);
+break;
+
+case XenbusStateClosing:
+xen_device_backend_set_state(xendev, XenbusStateClosing);
+break;
+
+case XenbusStateClosed:
+xen_block_disconnect(xendev, &local_err);
+if (local_err) {
+error_propagate(errp, local_err);
+break;
+}
+
+xen_device_backend_set_st

[Qemu-block] [PATCH v5 16/18] xen: automatically create XenBlockDevice-s

[Paul Durrant]

This patch adds create and destroy function for XenBlockDevice-s so that
they can be created automatically when the Xen toolstack instantiates a new
PV backend via xenstore. When the XenBlockDevice is created this way it is
also necessary to create a 'drive' which matches the configuration that the
Xen toolstack has written into xenstore. This is done by formulating the
parameters necessary for each 'blockdev' layer of the drive and then using
qmp_blockdev_add() to create the layers. Also, for compatibility with the
legacy 'xen_disk' implementation, an iothread is automatically created for
the new XenBlockDevice. This, like the driver layers, will be destroyed
after the XenBlockDevice is unrealized.

The legacy backend scan for 'qdisk' is removed by this patch, which makes
the 'xen_disk' code is redundant. The code will be removed by a subsequent
patch.

Signed-off-by: Paul Durrant 
---
Cc: Anthony Perard 
Cc: Kevin Wolf 
Cc: Max Reitz 
Cc: Stefano Stabellini 

v5:
 - Extensively re-worked to avoid using drive_new() and use
   qmp_blockdev_add() instead
 - Also use qmp_object_add() for IOThread
 - Dropped Anthony's R-b because of the code changes

v2:
 - Get rid of error_abort
 - Don't use qdev_init_nofail
 - Explain why file locking needs to be off
---
 hw/block/trace-events   |   4 +
 hw/block/xen-block.c| 419 
 hw/xen/xen-legacy-backend.c |   1 -
 include/hw/xen/xen-block.h  |  13 ++
 4 files changed, 436 insertions(+), 1 deletion(-)

diff --git a/hw/block/trace-events b/hw/block/trace-events
index 89e2583..55e5a55 100644
--- a/hw/block/trace-events
+++ b/hw/block/trace-events
@@ -137,3 +137,7 @@ xen_disk_realize(void) ""
 xen_disk_unrealize(void) ""
 xen_cdrom_realize(void) ""
 xen_cdrom_unrealize(void) ""
+xen_block_blockdev_add(char *str) "%s"
+xen_block_blockdev_del(const char *node_name) "%s"
+xen_block_device_create(unsigned int number) "%u"
+xen_block_device_destroy(unsigned int number) "%u"
diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index a7c37c1..5e69fa0 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -7,12 +7,20 @@
 
 #include "qemu/osdep.h"
 #include "qemu/cutils.h"
+#include "qemu/option.h"
 #include "qapi/error.h"
+#include "qapi/qapi-commands-block-core.h"
+#include "qapi/qapi-commands-misc.h"
+#include "qapi/qapi-visit-block-core.h"
 #include "qapi/visitor.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/qmp/qstring.h"
+#include "block/qdict.h"
 #include "hw/hw.h"
 #include "hw/xen/xen_common.h"
 #include "hw/block/xen_blkif.h"
 #include "hw/xen/xen-block.h"
+#include "hw/xen/xen-backend.h"
 #include "sysemu/blockdev.h"
 #include "sysemu/block-backend.h"
 #include "sysemu/iothread.h"
@@ -474,6 +482,7 @@ static void xen_block_class_init(ObjectClass *class, void 
*data)
 DeviceClass *dev_class = DEVICE_CLASS(class);
 XenDeviceClass *xendev_class = XEN_DEVICE_CLASS(class);
 
+xendev_class->backend = "qdisk";
 xendev_class->device = "vbd";
 xendev_class->get_name = xen_block_get_name;
 xendev_class->realize = xen_block_realize;
@@ -586,3 +595,413 @@ static void xen_block_register_types(void)
 }
 
 type_init(xen_block_register_types)
+
+static void xen_block_blockdev_del(const char *node_name, Error **errp)
+{
+trace_xen_block_blockdev_del(node_name);
+
+qmp_blockdev_del(node_name, errp);
+}
+
+static void add_item(const char *key, QObject *obj, void *opaque)
+{
+const char *val = qobject_get_try_str(obj);
+char **strp = opaque;
+char *str = *strp;
+
+*strp = (!str) ? g_strdup_printf("%s=%s", key, val) :
+ g_strdup_printf("%s %s=%s", str, key, val);
+g_free(str);
+}
+
+static char *xen_block_blockdev_add(const char *id, QDict *qdict,
+Error **errp)
+{
+const char *driver = qdict_get_try_str(qdict, "driver");
+BlockdevOptions *options = NULL;
+Error *local_err = NULL;
+char *str = NULL;
+char *node_name;
+Visitor *v;
+
+if (!driver) {
+error_setg(errp, "no 'driver' parameter");
+return NULL;
+}
+
+node_name = g_strdup_printf("%s-%s", id, driver);
+qdict_put_str(qdict, "node-name", node_name);
+
+qdict_iter(qdict, add_item, &str);
+
+trace_xen_block_blockdev_add(str);
+g_free(str);
+
+v = qobject_input_visitor_new_flat_confused(qdict, errp);
+if (!v) {
+goto fail;
+}
+
+visit_type_BlockdevOptions(v, NULL, &options, &local_err);
+visit_free(v);
+
+if (local_err) {
+error_propagate(errp, local_err);
+goto fail;
+}
+
+qmp_blockdev_add(options, &local_err);
+
+if (local_err) {
+error_propagate(errp, local_err);
+goto fail;
+}
+
+qapi_free_BlockdevOptions(options);
+
+return node_name;
+
+fail:
+if (options) {
+qapi_free_BlockdevOptions(options);
+}
+g_free(node_name);
+
+return NULL;
+}
+
+static void xen_block_drive_destroy(XenBlock

[Qemu-block] [PATCH v5 12/18] xen: remove 'ioreq' struct/varable/field names from dataplane/xen-block.c

[Paul Durrant]

This is a purely cosmetic patch that purges the name 'ioreq' from struct,
variable and field names. (This name has been problematic for a long time
as 'ioreq' is the name used for generic I/O requests coming from Xen).
The patch replaces 'struct ioreq' with a new 'XenBlockRequest' type and
'ioreq' field/variable names with 'request', and then does necessary
fix-up to adhere to coding style.

Function names are not modified by this patch. They will be dealt with in
a subsequent patch.

No functional change.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
---
Cc: Stefan Hajnoczi 
Cc: Stefano Stabellini 
Cc: Kevin Wolf 
Cc: Max Reitz 
---
 hw/block/dataplane/xen-block.c | 310 +
 1 file changed, 156 insertions(+), 154 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 6ecd160..426e83c 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -34,7 +34,7 @@
 #include "sysemu/iothread.h"
 #include "xen-block.h"
 
-struct ioreq {
+typedef struct XenBlockRequest {
 blkif_request_t req;
 int16_t status;
 off_t start;
@@ -45,9 +45,9 @@ struct ioreq {
 int aio_inflight;
 int aio_errors;
 XenBlockDataPlane *dataplane;
-QLIST_ENTRY(ioreq) list;
+QLIST_ENTRY(XenBlockRequest) list;
 BlockAcctCookie acct;
-};
+} XenBlockRequest;
 
 struct XenBlockDataPlane {
 XenDevice *xendev;
@@ -60,9 +60,9 @@ struct XenBlockDataPlane {
 int protocol;
 blkif_back_rings_t rings;
 int more_work;
-QLIST_HEAD(inflight_head, ioreq) inflight;
-QLIST_HEAD(finished_head, ioreq) finished;
-QLIST_HEAD(freelist_head, ioreq) freelist;
+QLIST_HEAD(inflight_head, XenBlockRequest) inflight;
+QLIST_HEAD(finished_head, XenBlockRequest) finished;
+QLIST_HEAD(freelist_head, XenBlockRequest) freelist;
 int requests_total;
 int requests_inflight;
 int requests_finished;
@@ -73,68 +73,68 @@ struct XenBlockDataPlane {
 AioContext *ctx;
 };
 
-static void ioreq_reset(struct ioreq *ioreq)
+static void ioreq_reset(XenBlockRequest *request)
 {
-memset(&ioreq->req, 0, sizeof(ioreq->req));
-ioreq->status = 0;
-ioreq->start = 0;
-ioreq->buf = NULL;
-ioreq->size = 0;
-ioreq->presync = 0;
+memset(&request->req, 0, sizeof(request->req));
+request->status = 0;
+request->start = 0;
+request->buf = NULL;
+request->size = 0;
+request->presync = 0;
 
-ioreq->aio_inflight = 0;
-ioreq->aio_errors = 0;
+request->aio_inflight = 0;
+request->aio_errors = 0;
 
-ioreq->dataplane = NULL;
-memset(&ioreq->list, 0, sizeof(ioreq->list));
-memset(&ioreq->acct, 0, sizeof(ioreq->acct));
+request->dataplane = NULL;
+memset(&request->list, 0, sizeof(request->list));
+memset(&request->acct, 0, sizeof(request->acct));
 
-qemu_iovec_reset(&ioreq->v);
+qemu_iovec_reset(&request->v);
 }
 
-static struct ioreq *ioreq_start(XenBlockDataPlane *dataplane)
+static XenBlockRequest *ioreq_start(XenBlockDataPlane *dataplane)
 {
-struct ioreq *ioreq = NULL;
+XenBlockRequest *request = NULL;
 
 if (QLIST_EMPTY(&dataplane->freelist)) {
 if (dataplane->requests_total >= dataplane->max_requests) {
 goto out;
 }
 /* allocate new struct */
-ioreq = g_malloc0(sizeof(*ioreq));
-ioreq->dataplane = dataplane;
+request = g_malloc0(sizeof(*request));
+request->dataplane = dataplane;
 dataplane->requests_total++;
-qemu_iovec_init(&ioreq->v, 1);
+qemu_iovec_init(&request->v, 1);
 } else {
 /* get one from freelist */
-ioreq = QLIST_FIRST(&dataplane->freelist);
-QLIST_REMOVE(ioreq, list);
+request = QLIST_FIRST(&dataplane->freelist);
+QLIST_REMOVE(request, list);
 }
-QLIST_INSERT_HEAD(&dataplane->inflight, ioreq, list);
+QLIST_INSERT_HEAD(&dataplane->inflight, request, list);
 dataplane->requests_inflight++;
 
 out:
-return ioreq;
+return request;
 }
 
-static void ioreq_finish(struct ioreq *ioreq)
+static void ioreq_finish(XenBlockRequest *request)
 {
-XenBlockDataPlane *dataplane = ioreq->dataplane;
+XenBlockDataPlane *dataplane = request->dataplane;
 
-QLIST_REMOVE(ioreq, list);
-QLIST_INSERT_HEAD(&dataplane->finished, ioreq, list);
+QLIST_REMOVE(request, list);
+QLIST_INSERT_HEAD(&dataplane->finished, request, list);
 dataplane->requests_inflight--;
 dataplane->requests_finished++;
 }
 
-static void ioreq_release(struct ioreq *ioreq, bool finish)
+static void ioreq_release(XenBlockRequest *request, bool finish)
 {
-XenBlockDataPlane *dataplane = ioreq->dataplane;
+XenBlockDataPlane *dataplane = request->dataplane;
 
-QLIST_REMOVE(ioreq, list);
-ioreq_reset(ioreq);
-ioreq->dataplane = dataplane;
-QLIST_INSERT_HEAD(&dataplane->freelist, ioreq, list);
+QLIST_REMOVE(request, list);
+io

[Qemu-block] [PATCH v5 15/18] xen: add a mechanism to automatically create XenDevice-s...

[Paul Durrant]

...that maintains compatibility with existing Xen toolstacks.

Xen toolstacks instantiate PV backends by simply writing information into
xenstore and expecting a backend implementation to be watching for this.

This patch adds a new 'xen-backend' module to allow individual XenDevice
implementations to register create and destroy functions. The creator
will be called when a tool-stack instantiates a new backend in this way,
and the destructor will then be called after the resulting XenDevice
object is unrealized.

To support this it is also necessary to add new watchers into the XenBus
implementation to handle enumeration of new backends and also destruction
of XenDevice-s when the toolstack sets the backend 'online' key to 0.

NOTE: This patch only adds the framework. A subsequent patch will add a
  creator function for xen-block devices.

Signed-off-by: Paul Durrant 
---
Cc: Anthony Perard 
Cc: Stefano Stabellini 
Cc: Anthony Perard 

v5:
 - Added destructor function, therefore Anthony's R-b is not carried
   from v4

v3:
 - Use a xenstore transaction in enumeration code to ensure consistent
   parameters
 - Not adding Anthony's R-b because of change

v2:
 - Sort out error paths and error reporting
---
 hw/xen/Makefile.objs |   2 +-
 hw/xen/trace-events  |   3 +
 hw/xen/xen-backend.c | 165 +++
 hw/xen/xen-bus.c | 164 +-
 include/hw/xen/xen-backend.h |  39 ++
 include/hw/xen/xen-bus.h |   1 +
 include/qemu/module.h|   3 +
 7 files changed, 375 insertions(+), 2 deletions(-)
 create mode 100644 hw/xen/xen-backend.c
 create mode 100644 include/hw/xen/xen-backend.h

diff --git a/hw/xen/Makefile.objs b/hw/xen/Makefile.objs
index 77c0868..84df60a 100644
--- a/hw/xen/Makefile.objs
+++ b/hw/xen/Makefile.objs
@@ -1,5 +1,5 @@
 # xen backend driver support
-common-obj-$(CONFIG_XEN) += xen-legacy-backend.o xen_devconfig.o xen_pvdev.o 
xen-common.o xen-bus.o xen-bus-helper.o
+common-obj-$(CONFIG_XEN) += xen-legacy-backend.o xen_devconfig.o xen_pvdev.o 
xen-common.o xen-bus.o xen-bus-helper.o xen-backend.o
 
 obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen-host-pci-device.o
 obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen_pt.o xen_pt_config_init.o 
xen_pt_graphics.o xen_pt_msi.o
diff --git a/hw/xen/trace-events b/hw/xen/trace-events
index d4651bd..f694462 100644
--- a/hw/xen/trace-events
+++ b/hw/xen/trace-events
@@ -16,6 +16,9 @@ xen_domid_restrict(int err) "err: %u"
 # include/hw/xen/xen-bus.c
 xen_bus_realize(void) ""
 xen_bus_unrealize(void) ""
+xen_bus_enumerate(void) ""
+xen_bus_type_enumerate(const char *type) "type: %s"
+xen_bus_backend_create(const char *type, const char *path) "type: %s path: %s"
 xen_bus_add_watch(const char *node, const char *key, char *token) "node: %s 
key: %s token: %s"
 xen_bus_remove_watch(const char *node, const char *key, char *token) "node: %s 
key: %s token: %s"
 xen_bus_watch(const char *token) "token: %s"
diff --git a/hw/xen/xen-backend.c b/hw/xen/xen-backend.c
new file mode 100644
index 000..da065f8
--- /dev/null
+++ b/hw/xen/xen-backend.c
@@ -0,0 +1,165 @@
+/*
+ * Copyright (c) 2018  Citrix Systems Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+#include "hw/xen/xen-backend.h"
+#include "hw/xen/xen-bus.h"
+
+typedef struct XenBackendImpl {
+const char *type;
+XenBackendDeviceCreate create;
+XenBackendDeviceDestroy destroy;
+} XenBackendImpl;
+
+struct XenBackendInstance {
+QLIST_ENTRY(XenBackendInstance) entry;
+const XenBackendImpl *impl;
+XenBus *xenbus;
+char *name;
+XenDevice *xendev;
+};
+
+static GHashTable *xen_backend_table_get(void)
+{
+static GHashTable *table;
+
+if (table == NULL) {
+table = g_hash_table_new(g_str_hash, g_str_equal);
+}
+
+return table;
+}
+
+static void xen_backend_table_add(XenBackendImpl *impl)
+{
+g_hash_table_insert(xen_backend_table_get(), (void *)impl->type, impl);
+}
+
+static const XenBackendImpl *xen_backend_table_lookup(const char *type)
+{
+return g_hash_table_lookup(xen_backend_table_get(), type);
+}
+
+void xen_backend_register(const XenBackendInfo *info)
+{
+XenBackendImpl *impl = g_new0(XenBackendImpl, 1);
+
+g_assert(info->type);
+
+if (xen_backend_table_lookup(info->type)) {
+error_report("attempt to register duplicate Xen backend type '%s'",
+ info->type);
+abort();
+}
+
+if (!info->create) {
+error_report("backend type '%s' has no creator", info->type);
+abort();
+}
+
+impl->type = info->type;
+impl->create = info->create;
+impl->destroy = info->destroy;
+
+xen_backend_table_add(impl);
+}
+
+static QLIST_HEAD(, XenBackendInstance) backend_list;
+
+static void xen_backend_

[Qemu-block] [PATCH v5 11/18] xen: remove 'XenBlkDev' and 'blkdev' names from dataplane/xen-block

[Paul Durrant]

This is a purely cosmetic patch that substitutes the old 'struct XenBlkDev'
name with 'XenBlockDataPlane' and 'blkdev' field/variable names with
'dataplane', and then does necessary fix-up to adhere to coding style.

No functional change.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: Stefan Hajnoczi 
Cc: Kevin Wolf 
Cc: Max Reitz 
---
 hw/block/dataplane/xen-block.c | 352 +
 hw/block/dataplane/xen-block.h |   2 +-
 2 files changed, 183 insertions(+), 171 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 20d16e7..6ecd160 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -44,12 +44,12 @@ struct ioreq {
 int presync;
 int aio_inflight;
 int aio_errors;
-struct XenBlkDev *blkdev;
+XenBlockDataPlane *dataplane;
 QLIST_ENTRY(ioreq) list;
 BlockAcctCookie acct;
 };
 
-struct XenBlkDev {
+struct XenBlockDataPlane {
 XenDevice *xendev;
 XenEventChannel *event_channel;
 unsigned int *ring_ref;
@@ -85,33 +85,33 @@ static void ioreq_reset(struct ioreq *ioreq)
 ioreq->aio_inflight = 0;
 ioreq->aio_errors = 0;
 
-ioreq->blkdev = NULL;
+ioreq->dataplane = NULL;
 memset(&ioreq->list, 0, sizeof(ioreq->list));
 memset(&ioreq->acct, 0, sizeof(ioreq->acct));
 
 qemu_iovec_reset(&ioreq->v);
 }
 
-static struct ioreq *ioreq_start(struct XenBlkDev *blkdev)
+static struct ioreq *ioreq_start(XenBlockDataPlane *dataplane)
 {
 struct ioreq *ioreq = NULL;
 
-if (QLIST_EMPTY(&blkdev->freelist)) {
-if (blkdev->requests_total >= blkdev->max_requests) {
+if (QLIST_EMPTY(&dataplane->freelist)) {
+if (dataplane->requests_total >= dataplane->max_requests) {
 goto out;
 }
 /* allocate new struct */
 ioreq = g_malloc0(sizeof(*ioreq));
-ioreq->blkdev = blkdev;
-blkdev->requests_total++;
+ioreq->dataplane = dataplane;
+dataplane->requests_total++;
 qemu_iovec_init(&ioreq->v, 1);
 } else {
 /* get one from freelist */
-ioreq = QLIST_FIRST(&blkdev->freelist);
+ioreq = QLIST_FIRST(&dataplane->freelist);
 QLIST_REMOVE(ioreq, list);
 }
-QLIST_INSERT_HEAD(&blkdev->inflight, ioreq, list);
-blkdev->requests_inflight++;
+QLIST_INSERT_HEAD(&dataplane->inflight, ioreq, list);
+dataplane->requests_inflight++;
 
 out:
 return ioreq;
@@ -119,26 +119,26 @@ out:
 
 static void ioreq_finish(struct ioreq *ioreq)
 {
-struct XenBlkDev *blkdev = ioreq->blkdev;
+XenBlockDataPlane *dataplane = ioreq->dataplane;
 
 QLIST_REMOVE(ioreq, list);
-QLIST_INSERT_HEAD(&blkdev->finished, ioreq, list);
-blkdev->requests_inflight--;
-blkdev->requests_finished++;
+QLIST_INSERT_HEAD(&dataplane->finished, ioreq, list);
+dataplane->requests_inflight--;
+dataplane->requests_finished++;
 }
 
 static void ioreq_release(struct ioreq *ioreq, bool finish)
 {
-struct XenBlkDev *blkdev = ioreq->blkdev;
+XenBlockDataPlane *dataplane = ioreq->dataplane;
 
 QLIST_REMOVE(ioreq, list);
 ioreq_reset(ioreq);
-ioreq->blkdev = blkdev;
-QLIST_INSERT_HEAD(&blkdev->freelist, ioreq, list);
+ioreq->dataplane = dataplane;
+QLIST_INSERT_HEAD(&dataplane->freelist, ioreq, list);
 if (finish) {
-blkdev->requests_finished--;
+dataplane->requests_finished--;
 } else {
-blkdev->requests_inflight--;
+dataplane->requests_inflight--;
 }
 }
 
@@ -148,7 +148,7 @@ static void ioreq_release(struct ioreq *ioreq, bool finish)
  */
 static int ioreq_parse(struct ioreq *ioreq)
 {
-struct XenBlkDev *blkdev = ioreq->blkdev;
+XenBlockDataPlane *dataplane = ioreq->dataplane;
 size_t len;
 int i;
 
@@ -171,12 +171,12 @@ static int ioreq_parse(struct ioreq *ioreq)
 };
 
 if (ioreq->req.operation != BLKIF_OP_READ &&
-blk_is_read_only(blkdev->blk)) {
+blk_is_read_only(dataplane->blk)) {
 error_report("error: write req for ro device");
 goto err;
 }
 
-ioreq->start = ioreq->req.sector_number * blkdev->file_blk;
+ioreq->start = ioreq->req.sector_number * dataplane->file_blk;
 for (i = 0; i < ioreq->req.nr_segments; i++) {
 if (i == BLKIF_MAX_SEGMENTS_PER_REQUEST) {
 error_report("error: nr_segments too big");
@@ -186,16 +186,16 @@ static int ioreq_parse(struct ioreq *ioreq)
 error_report("error: first > last sector");
 goto err;
 }
-if (ioreq->req.seg[i].last_sect * blkdev->file_blk >= XC_PAGE_SIZE) {
+if (ioreq->req.seg[i].last_sect * dataplane->file_blk >= XC_PAGE_SIZE) 
{
 error_report("error: page crossing");
 goto err;
 }
 
 len = (ioreq->req.seg[i].last_sect -
-   ioreq->req.seg[i].first_sect + 1) * blkdev->file_blk;
+   ioreq->r

[Qemu-block] [PATCH v5 17/18] MAINTAINERS: add myself as a Xen maintainer

[Paul Durrant]

I have made many significant contributions to the Xen code in QEMU,
particularly the recent patches introducing a new PV device framework.
I intend to make further significant contributions, porting other PV back-
ends to the new framework with the intent of eventually removing the
legacy code. It therefore seems reasonable that I become a maintainer of
the Xen code.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
Acked-by: Stefano Stabellini 
---
Cc: Paolo Bonzini 

v2:
 - Fix typo
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 9875581..e6bd441 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -396,6 +396,7 @@ Guest CPU Cores (Xen):
 X86
 M: Stefano Stabellini 
 M: Anthony Perard 
+M: Paul Durrant 
 L: xen-de...@lists.xenproject.org
 S: Supported
 F: */xen*
-- 
2.1.4

[Qemu-block] [PATCH v5 10/18] xen: add header and build dataplane/xen-block.c

[Paul Durrant]

This patch adds the transformations necessary to get dataplane/xen-block.c
to build against the new XenBus/XenDevice framework. MAINTAINERS is also
updated due to the introduction of dataplane/xen-block.h.

NOTE: Existing data structure names are retained for the moment. These will
  be modified by subsequent patches. A typedef for XenBlockDataPlane
  has been added to the header (based on the old struct XenBlkDev name
  for the moment) so that the old names don't need to leak out of the
  dataplane code.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Stefan Hajnoczi 
Cc: Kevin Wolf 
Cc: Max Reitz 
Cc: Stefano Stabellini 

v2:
 - Tidy up header inclusions
 - Get rid of error_fatal
---
 MAINTAINERS  |   1 +
 hw/block/dataplane/Makefile.objs |   1 +
 hw/block/dataplane/xen-block.c   | 356 ---
 hw/block/dataplane/xen-block.h   |  29 
 4 files changed, 287 insertions(+), 100 deletions(-)
 create mode 100644 hw/block/dataplane/xen-block.h

diff --git a/MAINTAINERS b/MAINTAINERS
index ab62ad4..9875581 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -408,6 +408,7 @@ F: hw/block/dataplane/xen*
 F: hw/xen/
 F: hw/xenpv/
 F: hw/i386/xen/
+F: include/hw/block/dataplane/xen*
 F: include/hw/xen/
 F: include/sysemu/xen-mapcache.h
 
diff --git a/hw/block/dataplane/Makefile.objs b/hw/block/dataplane/Makefile.objs
index e786f66..c6c68db 100644
--- a/hw/block/dataplane/Makefile.objs
+++ b/hw/block/dataplane/Makefile.objs
@@ -1 +1,2 @@
 obj-y += virtio-blk.o
+obj-$(CONFIG_XEN) += xen-block.o
diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 98f987d..20d16e7 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -24,65 +24,53 @@
  * See the COPYING file in the top-level directory.
  */
 
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+#include "hw/hw.h"
+#include "hw/xen/xen_common.h"
+#include "hw/block/xen_blkif.h"
+#include "sysemu/block-backend.h"
+#include "sysemu/iothread.h"
+#include "xen-block.h"
+
 struct ioreq {
-blkif_request_t req;
-int16_t status;
-
-/* parsed request */
-off_t   start;
-QEMUIOVectorv;
-void*buf;
-size_t  size;
-int presync;
-
-/* aio status */
-int aio_inflight;
-int aio_errors;
-
-struct XenBlkDev*blkdev;
-QLIST_ENTRY(ioreq)   list;
-BlockAcctCookie acct;
+blkif_request_t req;
+int16_t status;
+off_t start;
+QEMUIOVector v;
+void *buf;
+size_t size;
+int presync;
+int aio_inflight;
+int aio_errors;
+struct XenBlkDev *blkdev;
+QLIST_ENTRY(ioreq) list;
+BlockAcctCookie acct;
 };
 
-#define MAX_RING_PAGE_ORDER 4
-
 struct XenBlkDev {
-struct XenLegacyDevicexendev;  /* must be first */
-char*params;
-char*mode;
-char*type;
-char*dev;
-char*devtype;
-booldirectiosafe;
-const char  *fileproto;
-const char  *filename;
-unsigned intring_ref[1 << MAX_RING_PAGE_ORDER];
-unsigned intnr_ring_ref;
-void*sring;
-int64_t file_blk;
-int64_t file_size;
-int protocol;
-blkif_back_rings_t  rings;
-int more_work;
-
-/* request lists */
+XenDevice *xendev;
+XenEventChannel *event_channel;
+unsigned int *ring_ref;
+unsigned int nr_ring_ref;
+void *sring;
+int64_t file_blk;
+int64_t file_size;
+int protocol;
+blkif_back_rings_t rings;
+int more_work;
 QLIST_HEAD(inflight_head, ioreq) inflight;
 QLIST_HEAD(finished_head, ioreq) finished;
 QLIST_HEAD(freelist_head, ioreq) freelist;
-int requests_total;
-int requests_inflight;
-int requests_finished;
-unsigned intmax_requests;
-
-gbooleanfeature_discard;
-
-/* qemu block driver */
-DriveInfo   *dinfo;
-BlockBackend*blk;
-QEMUBH  *bh;
-
-IOThread*iothread;
-AioContext  *ctx;
+int requests_total;
+int requests_inflight;
+int requests_finished;
+unsigned int max_requests;
+BlockBackend *blk;
+QEMUBH *bh;
+IOThread *iothread;
+AioContext *ctx;
 };
 
 static void ioreq_reset(struct ioreq *ioreq)
@@ -161,7 +149,6 @@ static void ioreq_release(struct ioreq *ioreq, bool finish)
 static int ioreq_parse(struct ioreq *ioreq)
 {
 struct XenBlkDev *blkdev = ioreq->blkdev;
-struct XenLegacyDevice *xendev = &blkdev->xendev;
 size_t len;
 int i;
 
@@ -183,7 +170,8 @@ static int ioreq_parse(struct ioreq *ioreq)
 goto err;
 };
 
-if (ioreq-

[Qemu-block] [PATCH v5 13/18] xen: purge 'blk' and 'ioreq' from function names in dataplane/xen-block.c

[Paul Durrant]

This is a purely cosmetic patch that purges remaining use of 'blk' and
'ioreq' in local function names, and then makes sure all functions are
prefixed with 'xen_block_'.

No functional change.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: Stefan Hajnoczi 
Cc: Kevin Wolf 
Cc: Max Reitz 

v2:
 - Add 'xen_block_' prefix
---
 hw/block/dataplane/xen-block.c | 90 +-
 1 file changed, 46 insertions(+), 44 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 426e83c..8c451ae 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -73,7 +73,7 @@ struct XenBlockDataPlane {
 AioContext *ctx;
 };
 
-static void ioreq_reset(XenBlockRequest *request)
+static void reset_request(XenBlockRequest *request)
 {
 memset(&request->req, 0, sizeof(request->req));
 request->status = 0;
@@ -92,7 +92,7 @@ static void ioreq_reset(XenBlockRequest *request)
 qemu_iovec_reset(&request->v);
 }
 
-static XenBlockRequest *ioreq_start(XenBlockDataPlane *dataplane)
+static XenBlockRequest *xen_block_start_request(XenBlockDataPlane *dataplane)
 {
 XenBlockRequest *request = NULL;
 
@@ -117,7 +117,7 @@ out:
 return request;
 }
 
-static void ioreq_finish(XenBlockRequest *request)
+static void xen_block_finish_request(XenBlockRequest *request)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 
@@ -127,12 +127,12 @@ static void ioreq_finish(XenBlockRequest *request)
 dataplane->requests_finished++;
 }
 
-static void ioreq_release(XenBlockRequest *request, bool finish)
+static void xen_block_release_request(XenBlockRequest *request, bool finish)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 
 QLIST_REMOVE(request, list);
-ioreq_reset(request);
+reset_request(request);
 request->dataplane = dataplane;
 QLIST_INSERT_HEAD(&dataplane->freelist, request, list);
 if (finish) {
@@ -146,7 +146,7 @@ static void ioreq_release(XenBlockRequest *request, bool 
finish)
  * translate request into iovec + start offset
  * do sanity checks along the way
  */
-static int ioreq_parse(XenBlockRequest *request)
+static int xen_block_parse_request(XenBlockRequest *request)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 size_t len;
@@ -207,7 +207,7 @@ err:
 return -1;
 }
 
-static int ioreq_grant_copy(XenBlockRequest *request)
+static int xen_block_copy_request(XenBlockRequest *request)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 XenDevice *xendev = dataplane->xendev;
@@ -253,9 +253,9 @@ static int ioreq_grant_copy(XenBlockRequest *request)
 return 0;
 }
 
-static int ioreq_runio_qemu_aio(XenBlockRequest *request);
+static int xen_block_do_aio(XenBlockRequest *request);
 
-static void qemu_aio_complete(void *opaque, int ret)
+static void xen_block_complete_aio(void *opaque, int ret)
 {
 XenBlockRequest *request = opaque;
 XenBlockDataPlane *dataplane = request->dataplane;
@@ -272,7 +272,7 @@ static void qemu_aio_complete(void *opaque, int ret)
 request->aio_inflight--;
 if (request->presync) {
 request->presync = 0;
-ioreq_runio_qemu_aio(request);
+xen_block_do_aio(request);
 goto done;
 }
 if (request->aio_inflight > 0) {
@@ -283,7 +283,7 @@ static void qemu_aio_complete(void *opaque, int ret)
 case BLKIF_OP_READ:
 /* in case of failure request->aio_errors is increased */
 if (ret == 0) {
-ioreq_grant_copy(request);
+xen_block_copy_request(request);
 }
 qemu_vfree(request->buf);
 break;
@@ -299,7 +299,7 @@ static void qemu_aio_complete(void *opaque, int ret)
 }
 
 request->status = request->aio_errors ? BLKIF_RSP_ERROR : BLKIF_RSP_OKAY;
-ioreq_finish(request);
+xen_block_finish_request(request);
 
 switch (request->req.operation) {
 case BLKIF_OP_WRITE:
@@ -324,9 +324,9 @@ done:
 aio_context_release(dataplane->ctx);
 }
 
-static bool blk_split_discard(XenBlockRequest *request,
-  blkif_sector_t sector_number,
-  uint64_t nr_sectors)
+static bool xen_block_split_discard(XenBlockRequest *request,
+blkif_sector_t sector_number,
+uint64_t nr_sectors)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 int64_t byte_offset;
@@ -349,7 +349,7 @@ static bool blk_split_discard(XenBlockRequest *request,
 byte_chunk = byte_remaining > limit ? limit : byte_remaining;
 request->aio_inflight++;
 blk_aio_pdiscard(dataplane->blk, byte_offset, byte_chunk,
- qemu_aio_complete, request);
+ xen_block_complete_aio, request);
 byte_remaining -= byte_chunk;
 byte_offset += byte_chunk;
 } while (byte_remaining > 0);
@@ -357,7 +357,7 @@ static bool

[Qemu-block] [PATCH v5 18/18] xen: remove the legacy 'xen_disk' backend

[Paul Durrant]

This backend has now been replaced by the 'xen-qdisk' XenDevice.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
---
Cc: Kevin Wolf 
Cc: Max Reitz 
Cc: Stefano Stabellini 
---
 hw/block/Makefile.objs |1 -
 hw/block/xen_disk.c| 1011 
 2 files changed, 1012 deletions(-)
 delete mode 100644 hw/block/xen_disk.c

diff --git a/hw/block/Makefile.objs b/hw/block/Makefile.objs
index f34813a..e206b8e 100644
--- a/hw/block/Makefile.objs
+++ b/hw/block/Makefile.objs
@@ -5,7 +5,6 @@ common-obj-$(CONFIG_NAND) += nand.o
 common-obj-$(CONFIG_PFLASH_CFI01) += pflash_cfi01.o
 common-obj-$(CONFIG_PFLASH_CFI02) += pflash_cfi02.o
 common-obj-$(CONFIG_XEN) += xen-block.o
-common-obj-$(CONFIG_XEN) += xen_disk.o
 common-obj-$(CONFIG_ECC) += ecc.o
 common-obj-$(CONFIG_ONENAND) += onenand.o
 common-obj-$(CONFIG_NVME_PCI) += nvme.o
diff --git a/hw/block/xen_disk.c b/hw/block/xen_disk.c
deleted file mode 100644
index 75fe55f..000
--- a/hw/block/xen_disk.c
+++ /dev/null
@@ -1,1011 +0,0 @@
-/*
- *  xen paravirt block device backend
- *
- *  (c) Gerd Hoffmann 
- *
- *  This program is free software; you can redistribute it and/or modify
- *  it under the terms of the GNU General Public License as published by
- *  the Free Software Foundation; under version 2 of the License.
- *
- *  This program is distributed in the hope that it will be useful,
- *  but WITHOUT ANY WARRANTY; without even the implied warranty of
- *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- *  GNU General Public License for more details.
- *
- *  You should have received a copy of the GNU General Public License along
- *  with this program; if not, see .
- *
- *  Contributions after 2012-01-13 are licensed under the terms of the
- *  GNU GPL, version 2 or (at your option) any later version.
- */
-
-#include "qemu/osdep.h"
-#include "qemu/units.h"
-#include 
-#include 
-
-#include "hw/hw.h"
-#include "hw/xen/xen-legacy-backend.h"
-#include "xen_blkif.h"
-#include "sysemu/blockdev.h"
-#include "sysemu/iothread.h"
-#include "sysemu/block-backend.h"
-#include "qapi/error.h"
-#include "qapi/qmp/qdict.h"
-#include "qapi/qmp/qstring.h"
-#include "trace.h"
-
-/* - */
-
-#define BLOCK_SIZE  512
-#define IOCB_COUNT  (BLKIF_MAX_SEGMENTS_PER_REQUEST + 2)
-
-struct ioreq {
-blkif_request_t req;
-int16_t status;
-
-/* parsed request */
-off_t   start;
-QEMUIOVectorv;
-void*buf;
-size_t  size;
-int presync;
-
-/* aio status */
-int aio_inflight;
-int aio_errors;
-
-struct XenBlkDev*blkdev;
-QLIST_ENTRY(ioreq)   list;
-BlockAcctCookie acct;
-};
-
-#define MAX_RING_PAGE_ORDER 4
-
-struct XenBlkDev {
-struct XenLegacyDevicexendev;  /* must be first */
-char*params;
-char*mode;
-char*type;
-char*dev;
-char*devtype;
-booldirectiosafe;
-const char  *fileproto;
-const char  *filename;
-unsigned intring_ref[1 << MAX_RING_PAGE_ORDER];
-unsigned intnr_ring_ref;
-void*sring;
-int64_t file_blk;
-int64_t file_size;
-int protocol;
-blkif_back_rings_t  rings;
-int more_work;
-
-/* request lists */
-QLIST_HEAD(inflight_head, ioreq) inflight;
-QLIST_HEAD(finished_head, ioreq) finished;
-QLIST_HEAD(freelist_head, ioreq) freelist;
-int requests_total;
-int requests_inflight;
-int requests_finished;
-unsigned intmax_requests;
-
-gbooleanfeature_discard;
-
-/* qemu block driver */
-DriveInfo   *dinfo;
-BlockBackend*blk;
-QEMUBH  *bh;
-
-IOThread*iothread;
-AioContext  *ctx;
-};
-
-/* - */
-
-static void ioreq_reset(struct ioreq *ioreq)
-{
-memset(&ioreq->req, 0, sizeof(ioreq->req));
-ioreq->status = 0;
-ioreq->start = 0;
-ioreq->buf = NULL;
-ioreq->size = 0;
-ioreq->presync = 0;
-
-ioreq->aio_inflight = 0;
-ioreq->aio_errors = 0;
-
-ioreq->blkdev = NULL;
-memset(&ioreq->list, 0, sizeof(ioreq->list));
-memset(&ioreq->acct, 0, sizeof(ioreq->acct));
-
-qemu_iovec_reset(&ioreq->v);
-}
-
-static struct ioreq *ioreq_start(struct XenBlkDev *blkdev)
-{
-struct ioreq *ioreq = NULL;
-
-if (QLIST_EMPTY(&blkdev->freelist)) {
-if (blkdev->requests_total >= blkdev->max_requests) {
-goto out;
-}
-/* allocate new struct */
-ioreq = g_malloc0(sizeof(*ioreq));
-ioreq-

[Qemu-block] [PATCH v5 14/18] xen: add implementations of xen-block connect and disconnect functions...

[Paul Durrant]

...and wire in the dataplane.

This patch adds the remaining code to make the xen-block XenDevice
functional. The parameters that a block frontend expects to find are
populated in the backend xenstore area, and the 'ring-ref' and
'event-channel' values specified in the frontend xenstore area are
mapped/bound and used to set up the dataplane.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: Kevin Wolf 
Cc: Max Reitz 

v5:
 - Re-base

v3:
 - Add missing return statement to xen_block_realize()
 - Set device name to 'vbd' now the backend is functional
 - Add an unplug handler
 - Move backend watch code from subsequent patch since it's now needed
   to handle unplug

v2:
 - Tidy up header inclusions
 - Stop leaking ring_ref on error
 - Auto-create drive for CDRom devices
---
 hw/block/xen-block.c   | 166 
 hw/xen/trace-events|   3 +
 hw/xen/xen-bus.c   | 187 +++--
 include/hw/xen/xen-block.h |   9 +++
 include/hw/xen/xen-bus.h   |  14 +++-
 5 files changed, 353 insertions(+), 26 deletions(-)

diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index 3a963b0..a7c37c1 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -10,7 +10,13 @@
 #include "qapi/error.h"
 #include "qapi/visitor.h"
 #include "hw/hw.h"
+#include "hw/xen/xen_common.h"
+#include "hw/block/xen_blkif.h"
 #include "hw/xen/xen-block.h"
+#include "sysemu/blockdev.h"
+#include "sysemu/block-backend.h"
+#include "sysemu/iothread.h"
+#include "dataplane/xen-block.h"
 #include "trace.h"
 
 static char *xen_block_get_name(XenDevice *xendev, Error **errp)
@@ -28,6 +34,8 @@ static void xen_block_disconnect(XenDevice *xendev, Error 
**errp)
 XenBlockVdev *vdev = &blockdev->props.vdev;
 
 trace_xen_block_disconnect(type, vdev->disk, vdev->partition);
+
+xen_block_dataplane_stop(blockdev->dataplane);
 }
 
 static void xen_block_connect(XenDevice *xendev, Error **errp)
@@ -35,8 +43,72 @@ static void xen_block_connect(XenDevice *xendev, Error 
**errp)
 XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
 const char *type = object_get_typename(OBJECT(blockdev));
 XenBlockVdev *vdev = &blockdev->props.vdev;
+unsigned int order, nr_ring_ref, *ring_ref, event_channel, protocol;
+char *str;
 
 trace_xen_block_connect(type, vdev->disk, vdev->partition);
+
+if (xen_device_frontend_scanf(xendev, "ring-page-order", "%u",
+  &order) != 1) {
+nr_ring_ref = 1;
+ring_ref = g_new(unsigned int, nr_ring_ref);
+
+if (xen_device_frontend_scanf(xendev, "ring-ref", "%u",
+  &ring_ref[0]) != 1) {
+error_setg(errp, "failed to read ring-ref");
+g_free(ring_ref);
+return;
+}
+} else if (order <= blockdev->props.max_ring_page_order) {
+unsigned int i;
+
+nr_ring_ref = 1 << order;
+ring_ref = g_new(unsigned int, nr_ring_ref);
+
+for (i = 0; i < nr_ring_ref; i++) {
+const char *key = g_strdup_printf("ring-ref%u", i);
+
+if (xen_device_frontend_scanf(xendev, key, "%u",
+  &ring_ref[i]) != 1) {
+error_setg(errp, "failed to read %s", key);
+g_free((gpointer)key);
+g_free(ring_ref);
+return;
+}
+
+g_free((gpointer)key);
+}
+} else {
+error_setg(errp, "invalid ring-page-order (%d)", order);
+return;
+}
+
+if (xen_device_frontend_scanf(xendev, "event-channel", "%u",
+  &event_channel) != 1) {
+error_setg(errp, "failed to read event-channel");
+g_free(ring_ref);
+return;
+}
+
+if (xen_device_frontend_scanf(xendev, "protocol", "%ms",
+  &str) != 1) {
+protocol = BLKIF_PROTOCOL_NATIVE;
+} else {
+if (strcmp(str, XEN_IO_PROTO_ABI_X86_32) == 0) {
+protocol = BLKIF_PROTOCOL_X86_32;
+} else if (strcmp(str, XEN_IO_PROTO_ABI_X86_64) == 0) {
+protocol = BLKIF_PROTOCOL_X86_64;
+} else {
+protocol = BLKIF_PROTOCOL_NATIVE;
+}
+
+free(str);
+}
+
+xen_block_dataplane_start(blockdev->dataplane, ring_ref, nr_ring_ref,
+  event_channel, protocol, errp);
+
+g_free(ring_ref);
 }
 
 static void xen_block_unrealize(XenDevice *xendev, Error **errp)
@@ -56,6 +128,9 @@ static void xen_block_unrealize(XenDevice *xendev, Error 
**errp)
 /* Disconnect from the frontend in case this has not already happened */
 xen_block_disconnect(xendev, NULL);
 
+xen_block_dataplane_destroy(blockdev->dataplane);
+blockdev->dataplane = NULL;
+
 if (blockdev_class->unrealize) {
 blockdev_class->unrealize(blockdev, errp);
 }
@@ -68,6 +143,

Re: [Qemu-block] [Xen-devel] [PATCH v5 09/18] xen: remove unnecessary code from dataplane/xen-block.c

[Anthony PERARD]

On Mon, Dec 17, 2018 at 11:40:39AM +, Paul Durrant wrote:
> Not all of the code duplicated from xen_disk.c is required as the basis for
> the new dataplane implementation so this patch removes extraneous code,
> along with the legacy #includes and calls to the legacy xen_pv_printf()
> function. Error messages are changed to be reported using error_report().
> 
> NOTE: The code is still not yet built. Further transformations will be
>   required to make it correctly interface to the new XenBus/XenDevice
>   framework. They will be delivered in a subsequent patch.
> 
> Signed-off-by: Paul Durrant 
> Acked-by: Anthony Perard 
> ---
> Cc: Stefano Stabellini 
> Cc: Stefan Hajnoczi 
> Cc: Kevin Wolf 
> Cc: Max Reitz 
> 
> v2:
>  - Leave existing boilerplate alone, other than removing the now-incorrect
>description
> ---
>  hw/block/dataplane/xen-block.c | 409 
> ++---
>  1 file changed, 16 insertions(+), 393 deletions(-)
> 
> diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
> index 9fae505..98f987d 100644
> --- a/hw/block/dataplane/xen-block.c
> +++ b/hw/block/dataplane/xen-block.c
> @@ -1,6 +1,4 @@
>  /*
> - *  xen paravirt block device backend
> - *
>   *  (c) Gerd Hoffmann 
>   *
>   *  This program is free software; you can redistribute it and/or modify
> @@ -19,26 +17,12 @@
>   *  GNU GPL, version 2 or (at your option) any later version.
>   */
>  
> -#include "qemu/osdep.h"
> -#include "qemu/units.h"
> -#include 
> -#include 
> -
> -#include "hw/hw.h"
> -#include "hw/xen/xen_backend.h"
> -#include "xen_blkif.h"
> -#include "sysemu/blockdev.h"
> -#include "sysemu/iothread.h"
> -#include "sysemu/block-backend.h"
> -#include "qapi/error.h"
> -#include "qapi/qmp/qdict.h"
> -#include "qapi/qmp/qstring.h"
> -#include "trace.h"
> -
> -/* - */
> -
> -#define BLOCK_SIZE  512
> -#define IOCB_COUNT  (BLKIF_MAX_SEGMENTS_PER_REQUEST + 2)
> +/*
> + * Copyright (c) 2018  Citrix Systems Inc.
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + */

This patch obviously comes from v3 of the patch series. v4 was fine.
Please check comments on v1 v2 and v3.

Thanks,

-- 
Anthony PERARD

Re: [Qemu-block] [Xen-devel] [PATCH v5 09/18] xen: remove unnecessary code from dataplane/xen-block.c

[Paul Durrant]

> -Original Message-
> From: Anthony PERARD [mailto:anthony.per...@citrix.com]
> Sent: 17 December 2018 12:28
> To: Paul Durrant 
> Cc: qemu-de...@nongnu.org; qemu-block@nongnu.org; xen-
> de...@lists.xenproject.org; Kevin Wolf ; Stefano
> Stabellini ; Stefan Hajnoczi
> ; Max Reitz 
> Subject: Re: [Xen-devel] [PATCH v5 09/18] xen: remove unnecessary code
> from dataplane/xen-block.c
> 
> On Mon, Dec 17, 2018 at 11:40:39AM +, Paul Durrant wrote:
> > Not all of the code duplicated from xen_disk.c is required as the basis
> for
> > the new dataplane implementation so this patch removes extraneous code,
> > along with the legacy #includes and calls to the legacy xen_pv_printf()
> > function. Error messages are changed to be reported using
> error_report().
> >
> > NOTE: The code is still not yet built. Further transformations will be
> >   required to make it correctly interface to the new
> XenBus/XenDevice
> >   framework. They will be delivered in a subsequent patch.
> >
> > Signed-off-by: Paul Durrant 
> > Acked-by: Anthony Perard 
> > ---
> > Cc: Stefano Stabellini 
> > Cc: Stefan Hajnoczi 
> > Cc: Kevin Wolf 
> > Cc: Max Reitz 
> >
> > v2:
> >  - Leave existing boilerplate alone, other than removing the now-
> incorrect
> >description
> > ---
> >  hw/block/dataplane/xen-block.c | 409 ++
> ---
> >  1 file changed, 16 insertions(+), 393 deletions(-)
> >
> > diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-
> block.c
> > index 9fae505..98f987d 100644
> > --- a/hw/block/dataplane/xen-block.c
> > +++ b/hw/block/dataplane/xen-block.c
> > @@ -1,6 +1,4 @@
> >  /*
> > - *  xen paravirt block device backend
> > - *
> >   *  (c) Gerd Hoffmann 
> >   *
> >   *  This program is free software; you can redistribute it and/or
> modify
> > @@ -19,26 +17,12 @@
> >   *  GNU GPL, version 2 or (at your option) any later version.
> >   */
> >
> > -#include "qemu/osdep.h"
> > -#include "qemu/units.h"
> > -#include 
> > -#include 
> > -
> > -#include "hw/hw.h"
> > -#include "hw/xen/xen_backend.h"
> > -#include "xen_blkif.h"
> > -#include "sysemu/blockdev.h"
> > -#include "sysemu/iothread.h"
> > -#include "sysemu/block-backend.h"
> > -#include "qapi/error.h"
> > -#include "qapi/qmp/qdict.h"
> > -#include "qapi/qmp/qstring.h"
> > -#include "trace.h"
> > -
> > -/* - */
> > -
> > -#define BLOCK_SIZE  512
> > -#define IOCB_COUNT  (BLKIF_MAX_SEGMENTS_PER_REQUEST + 2)
> > +/*
> > + * Copyright (c) 2018  Citrix Systems Inc.
> > + *
> > + * This work is licensed under the terms of the GNU GPL, version 2 or
> later.
> > + * See the COPYING file in the top-level directory.
> > + */
> 
> This patch obviously comes from v3 of the patch series. v4 was fine.
> Please check comments on v1 v2 and v3.

Damn it. Somehow my local 'qom29' repo had diverged from xenbits. I'll replace 
that patch from there.

  Paul

> 
> Thanks,
> 
> --
> Anthony PERARD

Re: [Qemu-block] [PATCH] qemu: avoid memory leak while remove disk

[wangjian]

ok, I will send the patch by git send-email.

On 12/15/2018 4:22 AM, Michael S. Tsirkin wrote:

On Fri, Dec 07, 2018 at 09:53:06AM +0800, wangjian wrote:

Memset vhost_dev to zero in the vhost_dev_cleanup function.
This causes dev.vqs to be NULL, so that
vqs does not free up space when calling the g_free function.
This will result in a memory leak. But you can't release vqs
directly in the vhost_dev_cleanup function, because vhost_net
will also call this function, and vhost_net's vqs is assigned by array.
In order to solve this problem, we first save the pointer of vqs,
and release the space of vqs after vhost_dev_cleanup is called.

Signed-off-by: Jian Wang 

The patch does not seem to apply.
I suspect it was corrupted by your mailer, judging by
the fact that you also sent the patch in HTML format.

Can you please fix and repost?
Thanks!



---
  hw/block/vhost-user-blk.c | 7 +--
  hw/scsi/vhost-scsi.c  | 3 ++-
  hw/scsi/vhost-user-scsi.c | 3 ++-
  3 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index 1451940..c3af28f 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -250,6 +250,7 @@ static void vhost_user_blk_device_realize(DeviceState *dev, 
Error **errp)
  VirtIODevice *vdev = VIRTIO_DEVICE(dev);
  VHostUserBlk *s = VHOST_USER_BLK(vdev);
  VhostUserState *user;
+struct vhost_virtqueue *vqs = NULL;
  int i, ret;

  if (!s->chardev.chr) {
@@ -288,6 +289,7 @@ static void vhost_user_blk_device_realize(DeviceState *dev, 
Error **errp)
  s->dev.vqs = g_new(struct vhost_virtqueue, s->dev.nvqs);
  s->dev.vq_index = 0;
  s->dev.backend_features = 0;
+vqs = s->dev.vqs;

  vhost_dev_set_config_notifier(&s->dev, &blk_ops);

@@ -314,7 +316,7 @@ static void vhost_user_blk_device_realize(DeviceState *dev, 
Error **errp)
  vhost_err:
  vhost_dev_cleanup(&s->dev);
  virtio_err:
-g_free(s->dev.vqs);
+g_free(vqs);
  virtio_cleanup(vdev);

  vhost_user_cleanup(user);
@@ -326,10 +328,11 @@ static void vhost_user_blk_device_unrealize(DeviceState 
*dev, Error **errp)
  {
  VirtIODevice *vdev = VIRTIO_DEVICE(dev);
  VHostUserBlk *s = VHOST_USER_BLK(dev);
+struct vhost_virtqueue *vqs = s->dev.vqs;

  vhost_user_blk_set_status(vdev, 0);
  vhost_dev_cleanup(&s->dev);
-g_free(s->dev.vqs);
+g_free(vqs);
  virtio_cleanup(vdev);

  if (s->vhost_user) {
diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c
index 7f21b4f..61e2e57 100644
--- a/hw/scsi/vhost-scsi.c
+++ b/hw/scsi/vhost-scsi.c
@@ -215,6 +215,7 @@ static void vhost_scsi_unrealize(DeviceState *dev, Error 
**errp)
  {
  VirtIODevice *vdev = VIRTIO_DEVICE(dev);
  VHostSCSICommon *vsc = VHOST_SCSI_COMMON(dev);
+struct vhost_virtqueue *vqs = vsc->dev.vqs;

  migrate_del_blocker(vsc->migration_blocker);
  error_free(vsc->migration_blocker);
@@ -223,7 +224,7 @@ static void vhost_scsi_unrealize(DeviceState *dev, Error 
**errp)
  vhost_scsi_set_status(vdev, 0);

  vhost_dev_cleanup(&vsc->dev);
-g_free(vsc->dev.vqs);
+g_free(vqs);

  virtio_scsi_common_unrealize(dev, errp);
  }
diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c
index 2e1ba4a..6728878 100644
--- a/hw/scsi/vhost-user-scsi.c
+++ b/hw/scsi/vhost-user-scsi.c
@@ -121,12 +121,13 @@ static void vhost_user_scsi_unrealize(DeviceState *dev, 
Error **errp)
  VirtIODevice *vdev = VIRTIO_DEVICE(dev);
  VHostUserSCSI *s = VHOST_USER_SCSI(dev);
  VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s);
+struct vhost_virtqueue *vqs = vsc->dev.vqs;

  /* This will stop the vhost backend. */
  vhost_user_scsi_set_status(vdev, 0);

  vhost_dev_cleanup(&vsc->dev);
-g_free(vsc->dev.vqs);
+g_free(vqs);

  virtio_scsi_common_unrealize(dev, errp);

--
1.8.3.1


.

[Qemu-block] [PATCH v6 04/18] xen: create xenstore areas for XenDevice-s

[Paul Durrant]

This patch adds a new source module, xen-bus-helper.c, which builds on
basic libxenstore primitives to provide functions to create (setting
permissions appropriately) and destroy xenstore areas, and functions to
'printf' and 'scanf' nodes therein. The main xen-bus code then uses
these primitives [1] to initialize and destroy the frontend and backend
areas for a XenDevice during realize and unrealize respectively.

The 'xen-block' implementation is extended with a 'get_name' method that
returns the VBD number. This number is required to 'name' the xenstore
areas.

NOTE: An exit handler is also added to make sure the xenstore areas are
  cleaned up if QEMU terminates without devices being unrealized.

[1] The 'scanf' functions are actually not yet needed, but they will be
needed by code delivered in subsequent patches.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: Kevin Wolf 
Cc: Max Reitz 

v5:
 - Re-base

v3:
 - Add transaction id parameters to xen-bus-helper functions
 - Not added Anthony's R-b because of change

v2:
 - Fix boilerplate
 - Add error pointers to all xs_node... helpers
 - Add GCC_FMT_ATTR to declarations of printf-like helpers
---
 hw/block/xen-block.c|   9 +
 hw/xen/Makefile.objs|   2 +-
 hw/xen/trace-events |  12 +-
 hw/xen/xen-bus-helper.c | 150 +++
 hw/xen/xen-bus.c| 321 +++-
 include/hw/xen/xen-bus-helper.h |  39 
 include/hw/xen/xen-bus.h|  12 ++
 7 files changed, 540 insertions(+), 5 deletions(-)
 create mode 100644 hw/xen/xen-bus-helper.c
 create mode 100644 include/hw/xen/xen-bus-helper.h

diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index 8e78b6a034..d27a2865bc 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -13,6 +13,14 @@
 #include "hw/xen/xen-block.h"
 #include "trace.h"
 
+static char *xen_block_get_name(XenDevice *xendev, Error **errp)
+{
+XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
+XenBlockVdev *vdev = &blockdev->props.vdev;
+
+return g_strdup_printf("%lu", vdev->number);
+}
+
 static void xen_block_unrealize(XenDevice *xendev, Error **errp)
 {
 XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
@@ -262,6 +270,7 @@ static void xen_block_class_init(ObjectClass *class, void 
*data)
 DeviceClass *dev_class = DEVICE_CLASS(class);
 XenDeviceClass *xendev_class = XEN_DEVICE_CLASS(class);
 
+xendev_class->get_name = xen_block_get_name;
 xendev_class->realize = xen_block_realize;
 xendev_class->unrealize = xen_block_unrealize;
 
diff --git a/hw/xen/Makefile.objs b/hw/xen/Makefile.objs
index d9d6d7b4f9..77c0868190 100644
--- a/hw/xen/Makefile.objs
+++ b/hw/xen/Makefile.objs
@@ -1,5 +1,5 @@
 # xen backend driver support
-common-obj-$(CONFIG_XEN) += xen-legacy-backend.o xen_devconfig.o xen_pvdev.o 
xen-common.o xen-bus.o
+common-obj-$(CONFIG_XEN) += xen-legacy-backend.o xen_devconfig.o xen_pvdev.o 
xen-common.o xen-bus.o xen-bus-helper.o
 
 obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen-host-pci-device.o
 obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen_pt.o xen_pt_config_init.o 
xen_pt_graphics.o xen_pt_msi.o
diff --git a/hw/xen/trace-events b/hw/xen/trace-events
index 0172cd4e26..75dc226d75 100644
--- a/hw/xen/trace-events
+++ b/hw/xen/trace-events
@@ -16,5 +16,13 @@ xen_domid_restrict(int err) "err: %u"
 # include/hw/xen/xen-bus.c
 xen_bus_realize(void) ""
 xen_bus_unrealize(void) ""
-xen_device_realize(const char *type) "type: %s"
-xen_device_unrealize(const char *type) "type: %s"
+xen_device_realize(const char *type, char *name) "type: %s name: %s"
+xen_device_unrealize(const char *type, char *name) "type: %s name: %s"
+xen_device_backend_state(const char *type, char *name, const char *state) 
"type: %s name: %s -> %s"
+xen_device_frontend_state(const char *type, char *name, const char *state) 
"type: %s name: %s -> %s"
+
+# include/hw/xen/xen-bus-helper.c
+xs_node_create(const char *node) "%s"
+xs_node_destroy(const char *node) "%s"
+xs_node_vprintf(char *path, char *value) "%s %s"
+xs_node_vscanf(char *path, char *value) "%s %s"
diff --git a/hw/xen/xen-bus-helper.c b/hw/xen/xen-bus-helper.c
new file mode 100644
index 00..15b3ad8d78
--- /dev/null
+++ b/hw/xen/xen-bus-helper.c
@@ -0,0 +1,150 @@
+/*
+ * Copyright (c) 2018  Citrix Systems Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/hw.h"
+#include "hw/sysbus.h"
+#include "hw/xen/xen.h"
+#include "hw/xen/xen-bus.h"
+#include "hw/xen/xen-bus-helper.h"
+#include "qapi/error.h"
+
+#include 
+
+struct xs_state {
+enum xenbus_state statenum;
+const char *statestr;
+};
+#define XS_STATE(state) { state, #state }
+
+static struct xs_state xs_state[] = {
+XS_STATE(XenbusStateUnknown),
+XS_STATE(XenbusStateInitialising),
+XS_STATE(XenbusStateInitWait),
+ 

[Qemu-block] [PATCH v6 02/18] xen: introduce new 'XenBus' and 'XenDevice' object hierarchy

[Paul Durrant]

This patch adds the basic boilerplate for a 'XenBus' object that will act
as a parent to 'XenDevice' PV backends.
A new 'XenBridge' object is also added to connect XenBus to the system bus.

The XenBus object is instantiated by a new xen_bus_init() function called
from the same sites as the legacy xen_be_init() function.

Subsequent patches will flesh-out the functionality of these objects.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: "Michael S. Tsirkin" 
Cc: Marcel Apfelbaum 
Cc: Paolo Bonzini 
Cc: Richard Henderson 
Cc: Eduardo Habkost 

v3:
 - Don't use local_err unless it is needed

v2:
 - Fix boilerplate
 - Make xen-bus hotplug capable
---
 hw/i386/xen/xen-hvm.c |   3 +
 hw/xen/Makefile.objs  |   2 +-
 hw/xen/trace-events   |   6 ++
 hw/xen/xen-bus.c  | 127 ++
 hw/xenpv/xen_machine_pv.c |   3 +
 include/hw/xen/xen-bus.h  |  55 +
 6 files changed, 195 insertions(+), 1 deletion(-)
 create mode 100644 hw/xen/xen-bus.c
 create mode 100644 include/hw/xen/xen-bus.h

diff --git a/hw/i386/xen/xen-hvm.c b/hw/i386/xen/xen-hvm.c
index 1d637639c7..4497f751d2 100644
--- a/hw/i386/xen/xen-hvm.c
+++ b/hw/i386/xen/xen-hvm.c
@@ -17,6 +17,7 @@
 #include "hw/i386/apic-msidef.h"
 #include "hw/xen/xen_common.h"
 #include "hw/xen/xen-legacy-backend.h"
+#include "hw/xen/xen-bus.h"
 #include "qapi/error.h"
 #include "qapi/qapi-commands-misc.h"
 #include "qemu/error-report.h"
@@ -1479,6 +1480,8 @@ void xen_hvm_init(PCMachineState *pcms, MemoryRegion 
**ram_memory)
 QLIST_INIT(&state->dev_list);
 device_listener_register(&state->device_listener);
 
+xen_bus_init();
+
 /* Initialize backend core & drivers */
 if (xen_be_init() != 0) {
 error_report("xen backend core setup failed");
diff --git a/hw/xen/Makefile.objs b/hw/xen/Makefile.objs
index 3f64a44051..d9d6d7b4f9 100644
--- a/hw/xen/Makefile.objs
+++ b/hw/xen/Makefile.objs
@@ -1,5 +1,5 @@
 # xen backend driver support
-common-obj-$(CONFIG_XEN) += xen-legacy-backend.o xen_devconfig.o xen_pvdev.o 
xen-common.o
+common-obj-$(CONFIG_XEN) += xen-legacy-backend.o xen_devconfig.o xen_pvdev.o 
xen-common.o xen-bus.o
 
 obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen-host-pci-device.o
 obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen_pt.o xen_pt_config_init.o 
xen_pt_graphics.o xen_pt_msi.o
diff --git a/hw/xen/trace-events b/hw/xen/trace-events
index c7e7a3b523..0172cd4e26 100644
--- a/hw/xen/trace-events
+++ b/hw/xen/trace-events
@@ -12,3 +12,9 @@ xen_unmap_portio_range(uint32_t id, uint64_t start_addr, 
uint64_t end_addr) "id:
 xen_map_pcidev(uint32_t id, uint8_t bus, uint8_t dev, uint8_t func) "id: %u 
bdf: %02x.%02x.%02x"
 xen_unmap_pcidev(uint32_t id, uint8_t bus, uint8_t dev, uint8_t func) "id: %u 
bdf: %02x.%02x.%02x"
 xen_domid_restrict(int err) "err: %u"
+
+# include/hw/xen/xen-bus.c
+xen_bus_realize(void) ""
+xen_bus_unrealize(void) ""
+xen_device_realize(const char *type) "type: %s"
+xen_device_unrealize(const char *type) "type: %s"
diff --git a/hw/xen/xen-bus.c b/hw/xen/xen-bus.c
new file mode 100644
index 00..bc04da748b
--- /dev/null
+++ b/hw/xen/xen-bus.c
@@ -0,0 +1,127 @@
+/*
+ * Copyright (c) 2018  Citrix Systems Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/hw.h"
+#include "hw/sysbus.h"
+#include "hw/xen/xen-bus.h"
+#include "qapi/error.h"
+#include "trace.h"
+
+static void xen_bus_unrealize(BusState *bus, Error **errp)
+{
+trace_xen_bus_unrealize();
+}
+
+static void xen_bus_realize(BusState *bus, Error **errp)
+{
+trace_xen_bus_realize();
+}
+
+static void xen_bus_class_init(ObjectClass *class, void *data)
+{
+BusClass *bus_class = BUS_CLASS(class);
+
+bus_class->realize = xen_bus_realize;
+bus_class->unrealize = xen_bus_unrealize;
+}
+
+static const TypeInfo xen_bus_type_info = {
+.name = TYPE_XEN_BUS,
+.parent = TYPE_BUS,
+.instance_size = sizeof(XenBus),
+.class_size = sizeof(XenBusClass),
+.class_init = xen_bus_class_init,
+.interfaces = (InterfaceInfo[]) {
+{ TYPE_HOTPLUG_HANDLER },
+{ }
+},
+};
+
+static void xen_device_unrealize(DeviceState *dev, Error **errp)
+{
+XenDevice *xendev = XEN_DEVICE(dev);
+XenDeviceClass *xendev_class = XEN_DEVICE_GET_CLASS(xendev);
+const char *type = object_get_typename(OBJECT(xendev));
+
+trace_xen_device_unrealize(type);
+
+if (xendev_class->unrealize) {
+xendev_class->unrealize(xendev, errp);
+}
+}
+
+static void xen_device_realize(DeviceState *dev, Error **errp)
+{
+XenDevice *xendev = XEN_DEVICE(dev);
+XenDeviceClass *xendev_class = XEN_DEVICE_GET_CLASS(xendev);
+const char *type = object_get_typename(OBJECT(xendev));
+Error *local_err = NULL;
+
+trace_xen_device_realize(type);
+
+if (xendev_class->realize) {
+xendev_class

[Qemu-block] [PATCH v6 07/18] xen: add event channel interface for XenDevice-s

[Paul Durrant]

The legacy PV backend infrastructure provides functions to bind, unbind
and send notifications to event channnels. Similar functionality will be
required by XenDevice implementations so this patch adds the necessary
support.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Stefano Stabellini 

v2:
 - Added error pointers to notify and unbind
---
 hw/xen/xen-bus.c | 101 +++
 include/hw/xen/xen-bus.h |  18 +++
 2 files changed, 119 insertions(+)

diff --git a/hw/xen/xen-bus.c b/hw/xen/xen-bus.c
index faa9fd3577..9443f27d44 100644
--- a/hw/xen/xen-bus.c
+++ b/hw/xen/xen-bus.c
@@ -617,6 +617,81 @@ done:
 g_free(xengnttab_segs);
 }
 
+struct XenEventChannel {
+unsigned int local_port;
+XenEventHandler handler;
+void *opaque;
+Notifier notifier;
+};
+
+static void event_notify(Notifier *n, void *data)
+{
+XenEventChannel *channel = container_of(n, XenEventChannel, notifier);
+unsigned long port = (unsigned long)data;
+
+if (port == channel->local_port) {
+channel->handler(channel->opaque);
+}
+}
+
+XenEventChannel *xen_device_bind_event_channel(XenDevice *xendev,
+   unsigned int port,
+   XenEventHandler handler,
+   void *opaque, Error **errp)
+{
+XenEventChannel *channel = g_new0(XenEventChannel, 1);
+
+channel->local_port = xenevtchn_bind_interdomain(xendev->xeh,
+ xendev->frontend_id,
+ port);
+if (xendev->local_port < 0) {
+error_setg_errno(errp, errno, "xenevtchn_bind_interdomain failed");
+
+g_free(channel);
+return NULL;
+}
+
+channel->handler = handler;
+channel->opaque = opaque;
+channel->notifier.notify = event_notify;
+
+notifier_list_add(&xendev->event_notifiers, &channel->notifier);
+
+return channel;
+}
+
+void xen_device_notify_event_channel(XenDevice *xendev,
+ XenEventChannel *channel,
+ Error **errp)
+{
+if (!channel) {
+error_setg(errp, "bad channel");
+return;
+}
+
+if (xenevtchn_notify(xendev->xeh, channel->local_port) < 0) {
+error_setg_errno(errp, errno, "xenevtchn_notify failed");
+}
+}
+
+void xen_device_unbind_event_channel(XenDevice *xendev,
+ XenEventChannel *channel,
+ Error **errp)
+{
+if (!channel) {
+error_setg(errp, "bad channel");
+return;
+}
+
+notifier_remove(&channel->notifier);
+
+if (xenevtchn_unbind(xendev->xeh, channel->local_port) < 0) {
+error_setg_errno(errp, errno, "xenevtchn_unbind failed");
+}
+
+g_free(channel);
+}
+
 static void xen_device_unrealize(DeviceState *dev, Error **errp)
 {
 XenDevice *xendev = XEN_DEVICE(dev);
@@ -641,6 +716,12 @@ static void xen_device_unrealize(DeviceState *dev, Error 
**errp)
 xen_device_frontend_destroy(xendev);
 xen_device_backend_destroy(xendev);
 
+if (xendev->xeh) {
+qemu_set_fd_handler(xenevtchn_fd(xendev->xeh), NULL, NULL, NULL);
+xenevtchn_close(xendev->xeh);
+xendev->xeh = NULL;
+}
+
 if (xendev->xgth) {
 xengnttab_close(xendev->xgth);
 xendev->xgth = NULL;
@@ -657,6 +738,16 @@ static void xen_device_exit(Notifier *n, void *data)
 xen_device_unrealize(DEVICE(xendev), &error_abort);
 }
 
+static void xen_device_event(void *opaque)
+{
+XenDevice *xendev = opaque;
+unsigned long port = xenevtchn_pending(xendev->xeh);
+
+notifier_list_notify(&xendev->event_notifiers, (void *)port);
+
+xenevtchn_unmask(xendev->xeh, port);
+}
+
 static void xen_device_realize(DeviceState *dev, Error **errp)
 {
 XenDevice *xendev = XEN_DEVICE(dev);
@@ -697,6 +788,16 @@ static void xen_device_realize(DeviceState *dev, Error 
**errp)
 xendev->feature_grant_copy =
 (xengnttab_grant_copy(xendev->xgth, 0, NULL) == 0);
 
+xendev->xeh = xenevtchn_open(NULL, 0);
+if (!xendev->xeh) {
+error_setg_errno(errp, errno, "failed xenevtchn_open");
+goto unrealize;
+}
+
+notifier_list_init(&xendev->event_notifiers);
+qemu_set_fd_handler(xenevtchn_fd(xendev->xeh), xen_device_event, NULL,
+xendev);
+
 xen_device_backend_create(xendev, &local_err);
 if (local_err) {
 error_propagate(errp, local_err);
diff --git a/include/hw/xen/xen-bus.h b/include/hw/xen/xen-bus.h
index 63a09b67ee..f83a95cebd 100644
--- a/include/hw/xen/xen-bus.h
+++ b/include/hw/xen/xen-bus.h
@@ -26,6 +26,9 @@ typedef struct XenDevice {
 XenWatch *frontend_state_watch;
 xengnttab_handle *xgth;
 bool feature_grant_copy;
+xenevtchn_handle *xeh;
+xenevtchn_port_or_error_t local_port;

[Qemu-block] [PATCH v6 00/18] Xen PV backend 'qdevification'

[Paul Durrant]

This series introduces a new QOM compliant framework for Xen PV backends.
This is achieved by first moving the current non-compliant framework aside,
before building up a new framework incrementally.

This series was prompted by a thread [1] started by Kevin Wolf in response
to patches against xen_disk.c posted by Tim Smith. Therefore, alongside
the patches introducing the new framework, other patches build up a QOM
compliant replacement for 'xen_disk', called 'xen-qdisk'. Patch #16 swaps
this new device into place (having establisheda mechanism to auto-
instantiate devices that is compliant with existing Xen toolstacks in
patch #15) and patch #18 then removes the old xen_disk code.

Subsequent series will port other Xen PV backends across to the new
framework.

The series is also available as a repository branch [2] on xenbits.xen.org.

[1] https://lists.gnu.org/archive/html/qemu-devel/2018-11/msg00259.html
[2] 
http://xenbits.xen.org/gitweb/?p=people/pauldu/qemu.git;a=shortlog;h=refs/heads/qom31

Paul Durrant (18):
  xen: re-name XenDevice to XenLegacyDevice...
  xen: introduce new 'XenBus' and 'XenDevice' object hierarchy
  xen: introduce 'xen-block', 'xen-disk' and 'xen-cdrom'
  xen: create xenstore areas for XenDevice-s
  xen: add xenstore watcher infrastructure
  xen: add grant table interface for XenDevice-s
  xen: add event channel interface for XenDevice-s
  xen: duplicate xen_disk.c as basis of dataplane/xen-block.c
  xen: remove unnecessary code from dataplane/xen-block.c
  xen: add header and build dataplane/xen-block.c
  xen: remove 'XenBlkDev' and 'blkdev' names from dataplane/xen-block
  xen: remove 'ioreq' struct/varable/field names from
dataplane/xen-block.c
  xen: purge 'blk' and 'ioreq' from function names in
dataplane/xen-block.c
  xen: add implementations of xen-block connect and disconnect
functions...
  xen: add a mechanism to automatically create XenDevice-s...
  xen: automatically create XenBlockDevice-s
  MAINTAINERS: add myself as a Xen maintainer
  xen: remove the legacy 'xen_disk' backend

 MAINTAINERS   |5 +-
 hw/9pfs/xen-9p-backend.c  |   16 +-
 hw/block/Makefile.objs|2 +-
 hw/block/dataplane/Makefile.objs  |1 +
 hw/block/dataplane/xen-block.c|  808 +++
 hw/block/dataplane/xen-block.h|   29 +
 hw/block/trace-events |   14 +
 hw/block/xen-block.c  | 1007 ++
 hw/block/xen_disk.c   | 1011 --
 hw/char/xen_console.c |   12 +-
 hw/display/xenfb.c|   25 +-
 hw/i386/xen/xen-hvm.c |5 +-
 hw/i386/xen/xen-mapcache.c|2 +-
 hw/i386/xen/xen_platform.c|2 +-
 hw/net/xen_nic.c  |   14 +-
 hw/usb/xen-usb.c  |   25 +-
 hw/xen/Makefile.objs  |2 +-
 hw/xen/trace-events   |   26 +
 hw/xen/xen-backend.c  |  165 +++
 hw/xen/xen-bus-helper.c   |  184 +++
 hw/xen/xen-bus.c  | 1197 +
 hw/xen/xen-common.c   |2 +-
 .../{xen_backend.c => xen-legacy-backend.c}   |   80 +-
 hw/xen/xen_devconfig.c|2 +-
 hw/xen/xen_pt.c   |2 +-
 hw/xen/xen_pt_config_init.c   |2 +-
 hw/xen/xen_pt_graphics.c  |2 +-
 hw/xen/xen_pt_msi.c   |2 +-
 hw/xen/xen_pvdev.c|   20 +-
 hw/xenpv/xen_domainbuild.c|2 +-
 hw/xenpv/xen_machine_pv.c |5 +-
 include/hw/xen/xen-backend.h  |   39 +
 include/hw/xen/xen-block.h|   95 ++
 include/hw/xen/xen-bus-helper.h   |   45 +
 include/hw/xen/xen-bus.h  |  138 ++
 .../{xen_backend.h => xen-legacy-backend.h}   |   43 +-
 include/hw/xen/xen_pvdev.h|   38 +-
 include/qemu/module.h |3 +
 38 files changed, 3919 insertions(+), 1153 deletions(-)
 create mode 100644 hw/block/dataplane/xen-block.c
 create mode 100644 hw/block/dataplane/xen-block.h
 create mode 100644 hw/block/xen-block.c
 delete mode 100644 hw/block/xen_disk.c
 create mode 100644 hw/xen/xen-backend.c
 create mode 100644 hw/xen/xen-bus-helper.c
 create mode 100644 hw/xen/xen-bus.c
 rename hw/xen/{xen_backend.c => xen-legacy-backend.c} (89%)
 create mode 100644 include/hw/xen/xen-backend.h
 create mode 100644 include/hw/xen/xen-block.h
 create mode 100644 include/hw/xen/xen-bus-helper.h
 create mode 100644 include/hw/xen/xen-bus.h
 rename include/hw/xen/{xen_backend.h => xen-legacy-backend.h} (61%)
---
Cc: Anthony Perard 
Cc: Eduardo Hab

[Qemu-block] [PATCH v6 08/18] xen: duplicate xen_disk.c as basis of dataplane/xen-block.c

[Paul Durrant]

The new xen-block XenDevice implementation requires the same core
dataplane as the legacy xen_disk implementation it will eventually replace.
This patch therefore copies the legacy xen_disk.c source module into a new
dataplane/xen-block.c source module as the basis for the new dataplane and
adjusts the MAINTAINERS file accordingly.

NOTE: The duplicated code is not yet built. It is simply put into place by
  this patch (just fixing style violations) such that the
  modifications that will need to be made to the code are not
  conflated with code movement, thus making review harder.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: Stefan Hajnoczi 
Cc: Kevin Wolf 
Cc: Max Reitz 
---
 MAINTAINERS|1 +
 hw/block/dataplane/xen-block.c | 1019 
 2 files changed, 1020 insertions(+)
 create mode 100644 hw/block/dataplane/xen-block.c

diff --git a/MAINTAINERS b/MAINTAINERS
index dd728c3228..ab62ad44a8 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -404,6 +404,7 @@ F: hw/char/xen_console.c
 F: hw/display/xenfb.c
 F: hw/net/xen_nic.c
 F: hw/block/xen*
+F: hw/block/dataplane/xen*
 F: hw/xen/
 F: hw/xenpv/
 F: hw/i386/xen/
diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
new file mode 100644
index 00..9fae50534e
--- /dev/null
+++ b/hw/block/dataplane/xen-block.c
@@ -0,0 +1,1019 @@
+/*
+ *  xen paravirt block device backend
+ *
+ *  (c) Gerd Hoffmann 
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; under version 2 of the License.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License along
+ *  with this program; if not, see .
+ *
+ *  Contributions after 2012-01-13 are licensed under the terms of the
+ *  GNU GPL, version 2 or (at your option) any later version.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/units.h"
+#include 
+#include 
+
+#include "hw/hw.h"
+#include "hw/xen/xen_backend.h"
+#include "xen_blkif.h"
+#include "sysemu/blockdev.h"
+#include "sysemu/iothread.h"
+#include "sysemu/block-backend.h"
+#include "qapi/error.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/qmp/qstring.h"
+#include "trace.h"
+
+/* - */
+
+#define BLOCK_SIZE  512
+#define IOCB_COUNT  (BLKIF_MAX_SEGMENTS_PER_REQUEST + 2)
+
+struct ioreq {
+blkif_request_t req;
+int16_t status;
+
+/* parsed request */
+off_t   start;
+QEMUIOVectorv;
+void*buf;
+size_t  size;
+int presync;
+
+/* aio status */
+int aio_inflight;
+int aio_errors;
+
+struct XenBlkDev*blkdev;
+QLIST_ENTRY(ioreq)   list;
+BlockAcctCookie acct;
+};
+
+#define MAX_RING_PAGE_ORDER 4
+
+struct XenBlkDev {
+struct XenLegacyDevicexendev;  /* must be first */
+char*params;
+char*mode;
+char*type;
+char*dev;
+char*devtype;
+booldirectiosafe;
+const char  *fileproto;
+const char  *filename;
+unsigned intring_ref[1 << MAX_RING_PAGE_ORDER];
+unsigned intnr_ring_ref;
+void*sring;
+int64_t file_blk;
+int64_t file_size;
+int protocol;
+blkif_back_rings_t  rings;
+int more_work;
+
+/* request lists */
+QLIST_HEAD(inflight_head, ioreq) inflight;
+QLIST_HEAD(finished_head, ioreq) finished;
+QLIST_HEAD(freelist_head, ioreq) freelist;
+int requests_total;
+int requests_inflight;
+int requests_finished;
+unsigned intmax_requests;
+
+gbooleanfeature_discard;
+
+/* qemu block driver */
+DriveInfo   *dinfo;
+BlockBackend*blk;
+QEMUBH  *bh;
+
+IOThread*iothread;
+AioContext  *ctx;
+};
+
+/* - */
+
+static void ioreq_reset(struct ioreq *ioreq)
+{
+memset(&ioreq->req, 0, sizeof(ioreq->req));
+ioreq->status = 0;
+ioreq->start = 0;
+ioreq->buf = NULL;
+ioreq->size = 0;
+ioreq->presync = 0;
+
+ioreq->aio_inflight = 0;
+ioreq->aio_errors = 0;
+
+ioreq->blkdev = NULL;
+memset(&ioreq->list, 0, sizeof(ioreq->list));
+memset(&ioreq->acct, 0, sizeof(ioreq->ac

[Qemu-block] [PATCH v6 03/18] xen: introduce 'xen-block', 'xen-disk' and 'xen-cdrom'

[Paul Durrant]

This patch adds new XenDevice-s: 'xen-disk' and 'xen-cdrom', both derived
from a common 'xen-block' parent type. These will eventually replace the
'xen_disk' (note the underscore rather than hyphen) legacy PV backend but
it is illustrative to build up the implementation incrementally, along with
the XenBus/XenDevice framework. Subsequent patches will therefore add to
these devices' implementation as new features are added to the framework.

After this patch has been applied it is possible to instantiate new
'xen-disk' or 'xen-cdrom' devices with a single 'vdev' parameter, which
accepts values adhering to the Xen VBD naming scheme [1]. For example, a
command-line instantiation of a xen-disk can be done with an argument
similar to the following:

-device xen-disk,vdev=hda

The implementation of the vdev parameter formulates the appropriate VBD
number for use in the PV protocol.

[1] https://xenbits.xen.org/docs/unstable/man/xen-vbd-interface.7.html

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Kevin Wolf 
Cc: Max Reitz 
Cc: Stefano Stabellini 

v5:
 - Introduce properties sub-structure (which will be useful later)
 - Keeping Anthony's R-b from v4 since change is trivial

v3:
 - Fix vdev parsing again, and vdev formatting this time
 - Remove an unnecessary local_err

v2:
 - Fix boilerplate
 - Fix vdev parsing
 - Change name from 'xen-qdisk' to 'xen-block', make abstract, and split
   off 'xen-disk' and 'xen-cdrom' as concrete sub-types
---
 MAINTAINERS|   2 +-
 hw/block/Makefile.objs |   1 +
 hw/block/trace-events  |   8 +
 hw/block/xen-block.c   | 343 +
 include/hw/xen/xen-block.h |  73 
 5 files changed, 426 insertions(+), 1 deletion(-)
 create mode 100644 hw/block/xen-block.c
 create mode 100644 include/hw/xen/xen-block.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 63effdc473..dd728c3228 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -403,7 +403,7 @@ F: hw/9pfs/xen-9p-backend.c
 F: hw/char/xen_console.c
 F: hw/display/xenfb.c
 F: hw/net/xen_nic.c
-F: hw/block/xen_*
+F: hw/block/xen*
 F: hw/xen/
 F: hw/xenpv/
 F: hw/i386/xen/
diff --git a/hw/block/Makefile.objs b/hw/block/Makefile.objs
index 53ce5751ae..f34813a377 100644
--- a/hw/block/Makefile.objs
+++ b/hw/block/Makefile.objs
@@ -4,6 +4,7 @@ common-obj-$(CONFIG_SSI_M25P80) += m25p80.o
 common-obj-$(CONFIG_NAND) += nand.o
 common-obj-$(CONFIG_PFLASH_CFI01) += pflash_cfi01.o
 common-obj-$(CONFIG_PFLASH_CFI02) += pflash_cfi02.o
+common-obj-$(CONFIG_XEN) += xen-block.o
 common-obj-$(CONFIG_XEN) += xen_disk.o
 common-obj-$(CONFIG_ECC) += ecc.o
 common-obj-$(CONFIG_ONENAND) += onenand.o
diff --git a/hw/block/trace-events b/hw/block/trace-events
index 335c092450..4afbd62a88 100644
--- a/hw/block/trace-events
+++ b/hw/block/trace-events
@@ -127,3 +127,11 @@ xen_disk_init(char *name) "%s"
 xen_disk_connect(char *name) "%s"
 xen_disk_disconnect(char *name) "%s"
 xen_disk_free(char *name) "%s"
+
+# hw/block/xen-block.c
+xen_block_realize(const char *type, uint32_t disk, uint32_t partition) "%s 
d%up%u"
+xen_block_unrealize(const char *type, uint32_t disk, uint32_t partition) "%s 
d%up%u"
+xen_disk_realize(void) ""
+xen_disk_unrealize(void) ""
+xen_cdrom_realize(void) ""
+xen_cdrom_unrealize(void) ""
diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
new file mode 100644
index 00..8e78b6a034
--- /dev/null
+++ b/hw/block/xen-block.c
@@ -0,0 +1,343 @@
+/*
+ * Copyright (c) 2018  Citrix Systems Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/cutils.h"
+#include "qapi/error.h"
+#include "qapi/visitor.h"
+#include "hw/hw.h"
+#include "hw/xen/xen-block.h"
+#include "trace.h"
+
+static void xen_block_unrealize(XenDevice *xendev, Error **errp)
+{
+XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
+XenBlockDeviceClass *blockdev_class =
+XEN_BLOCK_DEVICE_GET_CLASS(xendev);
+const char *type = object_get_typename(OBJECT(blockdev));
+XenBlockVdev *vdev = &blockdev->props.vdev;
+
+if (vdev->type == XEN_BLOCK_VDEV_TYPE_INVALID) {
+return;
+}
+
+trace_xen_block_unrealize(type, vdev->disk, vdev->partition);
+
+if (blockdev_class->unrealize) {
+blockdev_class->unrealize(blockdev, errp);
+}
+}
+
+static void xen_block_realize(XenDevice *xendev, Error **errp)
+{
+XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
+XenBlockDeviceClass *blockdev_class =
+XEN_BLOCK_DEVICE_GET_CLASS(xendev);
+const char *type = object_get_typename(OBJECT(blockdev));
+XenBlockVdev *vdev = &blockdev->props.vdev;
+Error *local_err = NULL;
+
+if (vdev->type == XEN_BLOCK_VDEV_TYPE_INVALID) {
+error_setg(errp, "vdev property not set");
+return;
+}
+
+trace_xen_block_realize(type, vdev->disk, vdev->partition);
+
+if (blockdev_class->realize) {
+  

[Qemu-block] [PATCH v6 06/18] xen: add grant table interface for XenDevice-s

[Paul Durrant]

The legacy PV backend infrastructure provides functions to map, unmap and
copy pages granted by frontends. Similar functionality will be required
by XenDevice implementations so this patch adds the necessary support.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Stefano Stabellini 
---
 hw/xen/xen-bus.c | 146 +++
 include/hw/xen/xen-bus.h |  25 +++
 2 files changed, 171 insertions(+)

diff --git a/hw/xen/xen-bus.c b/hw/xen/xen-bus.c
index 5e19592190..faa9fd3577 100644
--- a/hw/xen/xen-bus.c
+++ b/hw/xen/xen-bus.c
@@ -485,6 +485,138 @@ static void xen_device_frontend_destroy(XenDevice *xendev)
 }
 }
 
+void xen_device_set_max_grant_refs(XenDevice *xendev, unsigned int nr_refs,
+   Error **errp)
+{
+if (xengnttab_set_max_grants(xendev->xgth, nr_refs)) {
+error_setg_errno(errp, errno, "xengnttab_set_max_grants failed");
+}
+}
+
+void *xen_device_map_grant_refs(XenDevice *xendev, uint32_t *refs,
+unsigned int nr_refs, int prot,
+Error **errp)
+{
+void *map = xengnttab_map_domain_grant_refs(xendev->xgth, nr_refs,
+xendev->frontend_id, refs,
+prot);
+
+if (!map) {
+error_setg_errno(errp, errno,
+ "xengnttab_map_domain_grant_refs failed");
+}
+
+return map;
+}
+
+void xen_device_unmap_grant_refs(XenDevice *xendev, void *map,
+ unsigned int nr_refs, Error **errp)
+{
+if (xengnttab_unmap(xendev->xgth, map, nr_refs)) {
+error_setg_errno(errp, errno, "xengnttab_unmap failed");
+}
+}
+
+static void compat_copy_grant_refs(XenDevice *xendev, bool to_domain,
+   XenDeviceGrantCopySegment segs[],
+   unsigned int nr_segs, Error **errp)
+{
+uint32_t *refs = g_new(uint32_t, nr_segs);
+int prot = to_domain ? PROT_WRITE : PROT_READ;
+void *map;
+unsigned int i;
+
+for (i = 0; i < nr_segs; i++) {
+XenDeviceGrantCopySegment *seg = &segs[i];
+
+refs[i] = to_domain ? seg->dest.foreign.ref :
+seg->source.foreign.ref;
+}
+
+map = xengnttab_map_domain_grant_refs(xendev->xgth, nr_segs,
+  xendev->frontend_id, refs,
+  prot);
+if (!map) {
+error_setg_errno(errp, errno,
+ "xengnttab_map_domain_grant_refs failed");
+goto done;
+}
+
+for (i = 0; i < nr_segs; i++) {
+XenDeviceGrantCopySegment *seg = &segs[i];
+void *page = map + (i * XC_PAGE_SIZE);
+
+if (to_domain) {
+memcpy(page + seg->dest.foreign.offset, seg->source.virt,
+   seg->len);
+} else {
+memcpy(seg->dest.virt, page + seg->source.foreign.offset,
+   seg->len);
+}
+}
+
+if (xengnttab_unmap(xendev->xgth, map, nr_segs)) {
+error_setg_errno(errp, errno, "xengnttab_unmap failed");
+}
+
+done:
+g_free(refs);
+}
+
+void xen_device_copy_grant_refs(XenDevice *xendev, bool to_domain,
+XenDeviceGrantCopySegment segs[],
+unsigned int nr_segs, Error **errp)
+{
+xengnttab_grant_copy_segment_t *xengnttab_segs;
+unsigned int i;
+
+if (!xendev->feature_grant_copy) {
+compat_copy_grant_refs(xendev, to_domain, segs, nr_segs, errp);
+return;
+}
+
+xengnttab_segs = g_new0(xengnttab_grant_copy_segment_t, nr_segs);
+
+for (i = 0; i < nr_segs; i++) {
+XenDeviceGrantCopySegment *seg = &segs[i];
+xengnttab_grant_copy_segment_t *xengnttab_seg = &xengnttab_segs[i];
+
+if (to_domain) {
+xengnttab_seg->flags = GNTCOPY_dest_gref;
+xengnttab_seg->dest.foreign.domid = xendev->frontend_id;
+xengnttab_seg->dest.foreign.ref = seg->dest.foreign.ref;
+xengnttab_seg->dest.foreign.offset = seg->dest.foreign.offset;
+xengnttab_seg->source.virt = seg->source.virt;
+} else {
+xengnttab_seg->flags = GNTCOPY_source_gref;
+xengnttab_seg->source.foreign.domid = xendev->frontend_id;
+xengnttab_seg->source.foreign.ref = seg->source.foreign.ref;
+xengnttab_seg->source.foreign.offset =
+seg->source.foreign.offset;
+xengnttab_seg->dest.virt = seg->dest.virt;
+}
+
+xengnttab_seg->len = seg->len;
+}
+
+if (xengnttab_grant_copy(xendev->xgth, nr_segs, xengnttab_segs)) {
+error_setg_errno(errp, errno, "xengnttab_grant_copy failed");
+goto done;
+}
+
+for (i = 0; i < nr_segs; i++) {
+xengnttab_grant_copy_segment_t *xengnttab_seg = &xengnttab_segs[i];
+
+if (xengnttab_s

[Qemu-block] [PATCH v6 05/18] xen: add xenstore watcher infrastructure

[Paul Durrant]

A Xen PV frontend communicates its state to the PV backend by writing to
the 'state' key in the frontend area in xenstore. It is therefore
necessary for a XenDevice implementation to be notified whenever the
value of this key changes.

This patch adds code to do this as follows:

- an 'fd handler' is registered on the libxenstore handle which will be
  triggered whenever a 'watch' event occurs
- primitives are added to xen-bus-helper to add or remove watch events
- a list of Notifier objects is added to XenBus to provide a mechanism
  to call the appropriate 'watch handler' when its associated event
  occurs

The xen-block implementation is extended with a 'frontend_changed' method,
which calls as-yet stub 'connect' and 'disconnect' functions when the
relevant frontend state transitions occur. A subsequent patch will supply
a full implementation for these functions.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Kevin Wolf 
Cc: Max Reitz 
Cc: Stefano Stabellini 

v5:
 - Re-base

v3:
 - Remove unnecessary instances of local_err

v2:
 - Don't crash when xen_block_disconnect() fails
 - Check xs_unwatch() for error
 - Add new_watch() and free_watch() utility functions
 - Use xs_check_watch() rather than xs_read_watch()
---
 hw/block/trace-events   |   2 +
 hw/block/xen-block.c|  70 +++
 hw/xen/trace-events |   6 +
 hw/xen/xen-bus-helper.c |  34 +
 hw/xen/xen-bus.c| 211 +++-
 include/hw/xen/xen-bus-helper.h |   6 +
 include/hw/xen/xen-bus.h|  15 +++
 7 files changed, 342 insertions(+), 2 deletions(-)

diff --git a/hw/block/trace-events b/hw/block/trace-events
index 4afbd62a88..89e258319c 100644
--- a/hw/block/trace-events
+++ b/hw/block/trace-events
@@ -130,6 +130,8 @@ xen_disk_free(char *name) "%s"
 
 # hw/block/xen-block.c
 xen_block_realize(const char *type, uint32_t disk, uint32_t partition) "%s 
d%up%u"
+xen_block_connect(const char *type, uint32_t disk, uint32_t partition) "%s 
d%up%u"
+xen_block_disconnect(const char *type, uint32_t disk, uint32_t partition) "%s 
d%up%u"
 xen_block_unrealize(const char *type, uint32_t disk, uint32_t partition) "%s 
d%up%u"
 xen_disk_realize(void) ""
 xen_disk_unrealize(void) ""
diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index d27a2865bc..3a963b0383 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -21,6 +21,24 @@ static char *xen_block_get_name(XenDevice *xendev, Error 
**errp)
 return g_strdup_printf("%lu", vdev->number);
 }
 
+static void xen_block_disconnect(XenDevice *xendev, Error **errp)
+{
+XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
+const char *type = object_get_typename(OBJECT(blockdev));
+XenBlockVdev *vdev = &blockdev->props.vdev;
+
+trace_xen_block_disconnect(type, vdev->disk, vdev->partition);
+}
+
+static void xen_block_connect(XenDevice *xendev, Error **errp)
+{
+XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
+const char *type = object_get_typename(OBJECT(blockdev));
+XenBlockVdev *vdev = &blockdev->props.vdev;
+
+trace_xen_block_connect(type, vdev->disk, vdev->partition);
+}
+
 static void xen_block_unrealize(XenDevice *xendev, Error **errp)
 {
 XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
@@ -35,6 +53,9 @@ static void xen_block_unrealize(XenDevice *xendev, Error 
**errp)
 
 trace_xen_block_unrealize(type, vdev->disk, vdev->partition);
 
+/* Disconnect from the frontend in case this has not already happened */
+xen_block_disconnect(xendev, NULL);
+
 if (blockdev_class->unrealize) {
 blockdev_class->unrealize(blockdev, errp);
 }
@@ -64,6 +85,54 @@ static void xen_block_realize(XenDevice *xendev, Error 
**errp)
 }
 }
 
+static void xen_block_frontend_changed(XenDevice *xendev,
+   enum xenbus_state frontend_state,
+   Error **errp)
+{
+enum xenbus_state backend_state = xen_device_backend_get_state(xendev);
+Error *local_err = NULL;
+
+switch (frontend_state) {
+case XenbusStateInitialised:
+case XenbusStateConnected:
+if (backend_state == XenbusStateConnected) {
+break;
+}
+
+xen_block_disconnect(xendev, &local_err);
+if (local_err) {
+error_propagate(errp, local_err);
+break;
+}
+
+xen_block_connect(xendev, &local_err);
+if (local_err) {
+error_propagate(errp, local_err);
+break;
+}
+
+xen_device_backend_set_state(xendev, XenbusStateConnected);
+break;
+
+case XenbusStateClosing:
+xen_device_backend_set_state(xendev, XenbusStateClosing);
+break;
+
+case XenbusStateClosed:
+xen_block_disconnect(xendev, &local_err);
+if (local_err) {
+error_propagate(errp, local_err);
+break;
+}
+
+xen_device_backend_set_stat

[Qemu-block] [PATCH v6 09/18] xen: remove unnecessary code from dataplane/xen-block.c

[Paul Durrant]

Not all of the code duplicated from xen_disk.c is required as the basis for
the new dataplane implementation so this patch removes extraneous code,
along with the legacy #includes and calls to the legacy xen_pv_printf()
function. Error messages are changed to be reported using error_report().

NOTE: The code is still not yet built. Further transformations will be
  required to make it correctly interface to the new XenBus/XenDevice
  framework. They will be delivered in a subsequent patch.

Signed-off-by: Paul Durrant 
---
Cc: Anthony Perard 
Cc: Stefano Stabellini 
Cc: Stefan Hajnoczi 
Cc: Kevin Wolf 
Cc: Max Reitz 

v6:
 - v5 was messed up so fix the boilerplate again
 - Drop Anthony's R-b so he can check it

v2:
 - Leave existing boilerplate alone, other than removing the now-incorrect
   description
---
 hw/block/dataplane/xen-block.c | 429 ++---
 1 file changed, 23 insertions(+), 406 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 9fae50534e..228472320a 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -1,45 +1,23 @@
 /*
- *  xen paravirt block device backend
+ * Copyright (c) 2018  Citrix Systems Inc.
+ * (c) Gerd Hoffmann 
  *
- *  (c) Gerd Hoffmann 
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; under version 2 of the License.
  *
- *  This program is free software; you can redistribute it and/or modify
- *  it under the terms of the GNU General Public License as published by
- *  the Free Software Foundation; under version 2 of the License.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
  *
- *  This program is distributed in the hope that it will be useful,
- *  but WITHOUT ANY WARRANTY; without even the implied warranty of
- *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- *  GNU General Public License for more details.
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see .
  *
- *  You should have received a copy of the GNU General Public License along
- *  with this program; if not, see .
- *
- *  Contributions after 2012-01-13 are licensed under the terms of the
- *  GNU GPL, version 2 or (at your option) any later version.
+ * Contributions after 2012-01-13 are licensed under the terms of the
+ * GNU GPL, version 2 or (at your option) any later version.
  */
 
-#include "qemu/osdep.h"
-#include "qemu/units.h"
-#include 
-#include 
-
-#include "hw/hw.h"
-#include "hw/xen/xen_backend.h"
-#include "xen_blkif.h"
-#include "sysemu/blockdev.h"
-#include "sysemu/iothread.h"
-#include "sysemu/block-backend.h"
-#include "qapi/error.h"
-#include "qapi/qmp/qdict.h"
-#include "qapi/qmp/qstring.h"
-#include "trace.h"
-
-/* - */
-
-#define BLOCK_SIZE  512
-#define IOCB_COUNT  (BLKIF_MAX_SEGMENTS_PER_REQUEST + 2)
-
 struct ioreq {
 blkif_request_t req;
 int16_t status;
@@ -101,8 +79,6 @@ struct XenBlkDev {
 AioContext  *ctx;
 };
 
-/* - */
-
 static void ioreq_reset(struct ioreq *ioreq)
 {
 memset(&ioreq->req, 0, sizeof(ioreq->req));
@@ -183,11 +159,6 @@ static int ioreq_parse(struct ioreq *ioreq)
 size_t len;
 int i;
 
-xen_pv_printf(
-xendev, 3,
-"op %d, nr %d, handle %d, id %" PRId64 ", sector %" PRId64 "\n",
-ioreq->req.operation, ioreq->req.nr_segments,
-ioreq->req.handle, ioreq->req.id, ioreq->req.sector_number);
 switch (ioreq->req.operation) {
 case BLKIF_OP_READ:
 break;
@@ -202,28 +173,27 @@ static int ioreq_parse(struct ioreq *ioreq)
 case BLKIF_OP_DISCARD:
 return 0;
 default:
-xen_pv_printf(xendev, 0, "error: unknown operation (%d)\n",
-  ioreq->req.operation);
+error_report("error: unknown operation (%d)", ioreq->req.operation);
 goto err;
 };
 
 if (ioreq->req.operation != BLKIF_OP_READ && blkdev->mode[0] != 'w') {
-xen_pv_printf(xendev, 0, "error: write req for ro device\n");
+error_report("error: write req for ro device");
 goto err;
 }
 
 ioreq->start = ioreq->req.sector_number * blkdev->file_blk;
 for (i = 0; i < ioreq->req.nr_segments; i++) {
 if (i == BLKIF_MAX_SEGMENTS_PER_REQUEST) {
-xen_pv_printf(xendev, 0, "error: nr_segments too big\n");
+error_report("error: nr_segments too big");
 goto err;
 }
 if (ioreq->req.seg[i].first_

[Qemu-block] [PATCH v6 01/18] xen: re-name XenDevice to XenLegacyDevice...

[Paul Durrant]

...and xen_backend.h to xen-legacy-backend.h

Rather than attempting to convert the existing backend infrastructure to
be QOM compliant (which would be hard to do in an incremental fashion),
subsequent patches will introduce a completely new framework for Xen PV
backends. Hence it is necessary to re-name parts of existing code to avoid
name clashes. The re-named 'legacy' infrastructure will be removed once all
backends have been ported to the new framework.

This patch is purely cosmetic. No functional change.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: Greg Kurz 
Cc: Kevin Wolf 
Cc: Max Reitz 
Cc: "Marc-André Lureau" 
Cc: Paolo Bonzini 
Cc: Richard Henderson 
Cc: Eduardo Habkost 
Cc: "Michael S. Tsirkin" 
Cc: Marcel Apfelbaum 
Cc: Jason Wang 
Cc: Gerd Hoffmann 
---
 hw/9pfs/xen-9p-backend.c  | 16 ++--
 hw/block/xen_disk.c   | 24 +++---
 hw/char/xen_console.c | 12 +--
 hw/display/xenfb.c| 25 +++---
 hw/i386/xen/xen-hvm.c |  2 +-
 hw/i386/xen/xen-mapcache.c|  2 +-
 hw/i386/xen/xen_platform.c|  2 +-
 hw/net/xen_nic.c  | 14 ++--
 hw/usb/xen-usb.c  | 25 +++---
 hw/xen/Makefile.objs  |  2 +-
 hw/xen/xen-common.c   |  2 +-
 .../{xen_backend.c => xen-legacy-backend.c}   | 79 +++
 hw/xen/xen_devconfig.c|  2 +-
 hw/xen/xen_pt.c   |  2 +-
 hw/xen/xen_pt_config_init.c   |  2 +-
 hw/xen/xen_pt_graphics.c  |  2 +-
 hw/xen/xen_pt_msi.c   |  2 +-
 hw/xen/xen_pvdev.c| 20 ++---
 hw/xenpv/xen_domainbuild.c|  2 +-
 hw/xenpv/xen_machine_pv.c |  2 +-
 .../{xen_backend.h => xen-legacy-backend.h}   | 43 +-
 include/hw/xen/xen_pvdev.h| 38 -
 22 files changed, 169 insertions(+), 151 deletions(-)
 rename hw/xen/{xen_backend.c => xen-legacy-backend.c} (89%)
 rename include/hw/xen/{xen_backend.h => xen-legacy-backend.h} (61%)

diff --git a/hw/9pfs/xen-9p-backend.c b/hw/9pfs/xen-9p-backend.c
index 3f54a21c76..3859a06fe7 100644
--- a/hw/9pfs/xen-9p-backend.c
+++ b/hw/9pfs/xen-9p-backend.c
@@ -12,7 +12,7 @@
 
 #include "hw/hw.h"
 #include "hw/9pfs/9p.h"
-#include "hw/xen/xen_backend.h"
+#include "hw/xen/xen-legacy-backend.h"
 #include "hw/9pfs/xen-9pfs.h"
 #include "qapi/error.h"
 #include "qemu/config-file.h"
@@ -45,7 +45,7 @@ typedef struct Xen9pfsRing {
 } Xen9pfsRing;
 
 typedef struct Xen9pfsDev {
-struct XenDevice xendev;  /* must be first */
+struct XenLegacyDevice xendev;  /* must be first */
 V9fsState state;
 char *path;
 char *security_model;
@@ -56,7 +56,7 @@ typedef struct Xen9pfsDev {
 Xen9pfsRing *rings;
 } Xen9pfsDev;
 
-static void xen_9pfs_disconnect(struct XenDevice *xendev);
+static void xen_9pfs_disconnect(struct XenLegacyDevice *xendev);
 
 static void xen_9pfs_in_sg(Xen9pfsRing *ring,
struct iovec *in_sg,
@@ -243,7 +243,7 @@ static const V9fsTransport xen_9p_transport = {
 .push_and_notify = xen_9pfs_push_and_notify,
 };
 
-static int xen_9pfs_init(struct XenDevice *xendev)
+static int xen_9pfs_init(struct XenLegacyDevice *xendev)
 {
 return 0;
 }
@@ -305,7 +305,7 @@ static void xen_9pfs_evtchn_event(void *opaque)
 qemu_bh_schedule(ring->bh);
 }
 
-static void xen_9pfs_disconnect(struct XenDevice *xendev)
+static void xen_9pfs_disconnect(struct XenLegacyDevice *xendev)
 {
 Xen9pfsDev *xen_9pdev = container_of(xendev, Xen9pfsDev, xendev);
 int i;
@@ -321,7 +321,7 @@ static void xen_9pfs_disconnect(struct XenDevice *xendev)
 }
 }
 
-static int xen_9pfs_free(struct XenDevice *xendev)
+static int xen_9pfs_free(struct XenLegacyDevice *xendev)
 {
 Xen9pfsDev *xen_9pdev = container_of(xendev, Xen9pfsDev, xendev);
 int i;
@@ -354,7 +354,7 @@ static int xen_9pfs_free(struct XenDevice *xendev)
 return 0;
 }
 
-static int xen_9pfs_connect(struct XenDevice *xendev)
+static int xen_9pfs_connect(struct XenLegacyDevice *xendev)
 {
 Error *err = NULL;
 int i;
@@ -467,7 +467,7 @@ out:
 return -1;
 }
 
-static void xen_9pfs_alloc(struct XenDevice *xendev)
+static void xen_9pfs_alloc(struct XenLegacyDevice *xendev)
 {
 xenstore_write_be_str(xendev, "versions", VERSIONS);
 xenstore_write_be_int(xendev, "max-rings", MAX_RINGS);
diff --git a/hw/block/xen_disk.c b/hw/block/xen_disk.c
index 36eff94f84..75fe55f2ae 100644
--- a/hw/block/xen_disk.c
+++ b/hw/block/xen_disk.c
@@ -25,7 +25,7 @@
 #include 
 
 #include "hw/hw.h"
-#include "hw/xen/xen_backend.h"
+#include "hw/xen/xen-legacy-backend.h"
 #include "xen_blkif.h"
 #include "sysemu/blockdev.h"
 #include "sysemu/iothread.h"
@@ -63,7 +63,7 @@ struct io

Re: [Qemu-block] [PATCH v2 0/8] qcow2: encryption threads

[Vladimir Sementsov-Ogievskiy]

11.12.2018 19:43, Vladimir Sementsov-Ogievskiy wrote:
> Hi all!
> 
> The series brings threads to qcow2 encryption/decryption path,
> like it is already done for compression.
> 
> v2: - multiple cipher inside QCryptoBlock instead of multiple
>blocks inside qcow2, as suggested by Daniel, and it is
>done in separate series
>  - use threaded encryption in do_perform_cow_encrypt() too
>  - some renaming and refactoring and simplifications
> (Sorry for not being very careful about change list, but v1 isn't
>   actually reviewed, as approach with multiple blocks was early
>   rejected by Denial)
> 
> The series now based on two queued for 4.0 series, which, in
> turn, may be applied in any order: "crypto threads" (Daniel's
> tree), and  "qcow2 decompress in threads", which is now in
> Kevin's block-next.
> 
> Based-on: <20181207161351.4380-1-vsement...@virtuozzo.com>
> ([PATCH v3 0/5] crypto threads)
> Based-on: git://repo.or.cz/qemu/kevin.git block-next
> (decompress in threads inside)

Both merged, so, now based on master.

-- 
Best regards,
Vladimir

[Qemu-block] [PATCH v6 12/18] xen: remove 'ioreq' struct/varable/field names from dataplane/xen-block.c

[Paul Durrant]

This is a purely cosmetic patch that purges the name 'ioreq' from struct,
variable and field names. (This name has been problematic for a long time
as 'ioreq' is the name used for generic I/O requests coming from Xen).
The patch replaces 'struct ioreq' with a new 'XenBlockRequest' type and
'ioreq' field/variable names with 'request', and then does necessary
fix-up to adhere to coding style.

Function names are not modified by this patch. They will be dealt with in
a subsequent patch.

No functional change.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
---
Cc: Stefan Hajnoczi 
Cc: Stefano Stabellini 
Cc: Kevin Wolf 
Cc: Max Reitz 
---
 hw/block/dataplane/xen-block.c | 310 +
 1 file changed, 156 insertions(+), 154 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 1ff464973c..6788bbf338 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -28,7 +28,7 @@
 #include "sysemu/iothread.h"
 #include "xen-block.h"
 
-struct ioreq {
+typedef struct XenBlockRequest {
 blkif_request_t req;
 int16_t status;
 off_t start;
@@ -39,9 +39,9 @@ struct ioreq {
 int aio_inflight;
 int aio_errors;
 XenBlockDataPlane *dataplane;
-QLIST_ENTRY(ioreq) list;
+QLIST_ENTRY(XenBlockRequest) list;
 BlockAcctCookie acct;
-};
+} XenBlockRequest;
 
 struct XenBlockDataPlane {
 XenDevice *xendev;
@@ -54,9 +54,9 @@ struct XenBlockDataPlane {
 int protocol;
 blkif_back_rings_t rings;
 int more_work;
-QLIST_HEAD(inflight_head, ioreq) inflight;
-QLIST_HEAD(finished_head, ioreq) finished;
-QLIST_HEAD(freelist_head, ioreq) freelist;
+QLIST_HEAD(inflight_head, XenBlockRequest) inflight;
+QLIST_HEAD(finished_head, XenBlockRequest) finished;
+QLIST_HEAD(freelist_head, XenBlockRequest) freelist;
 int requests_total;
 int requests_inflight;
 int requests_finished;
@@ -67,68 +67,68 @@ struct XenBlockDataPlane {
 AioContext *ctx;
 };
 
-static void ioreq_reset(struct ioreq *ioreq)
+static void ioreq_reset(XenBlockRequest *request)
 {
-memset(&ioreq->req, 0, sizeof(ioreq->req));
-ioreq->status = 0;
-ioreq->start = 0;
-ioreq->buf = NULL;
-ioreq->size = 0;
-ioreq->presync = 0;
+memset(&request->req, 0, sizeof(request->req));
+request->status = 0;
+request->start = 0;
+request->buf = NULL;
+request->size = 0;
+request->presync = 0;
 
-ioreq->aio_inflight = 0;
-ioreq->aio_errors = 0;
+request->aio_inflight = 0;
+request->aio_errors = 0;
 
-ioreq->dataplane = NULL;
-memset(&ioreq->list, 0, sizeof(ioreq->list));
-memset(&ioreq->acct, 0, sizeof(ioreq->acct));
+request->dataplane = NULL;
+memset(&request->list, 0, sizeof(request->list));
+memset(&request->acct, 0, sizeof(request->acct));
 
-qemu_iovec_reset(&ioreq->v);
+qemu_iovec_reset(&request->v);
 }
 
-static struct ioreq *ioreq_start(XenBlockDataPlane *dataplane)
+static XenBlockRequest *ioreq_start(XenBlockDataPlane *dataplane)
 {
-struct ioreq *ioreq = NULL;
+XenBlockRequest *request = NULL;
 
 if (QLIST_EMPTY(&dataplane->freelist)) {
 if (dataplane->requests_total >= dataplane->max_requests) {
 goto out;
 }
 /* allocate new struct */
-ioreq = g_malloc0(sizeof(*ioreq));
-ioreq->dataplane = dataplane;
+request = g_malloc0(sizeof(*request));
+request->dataplane = dataplane;
 dataplane->requests_total++;
-qemu_iovec_init(&ioreq->v, 1);
+qemu_iovec_init(&request->v, 1);
 } else {
 /* get one from freelist */
-ioreq = QLIST_FIRST(&dataplane->freelist);
-QLIST_REMOVE(ioreq, list);
+request = QLIST_FIRST(&dataplane->freelist);
+QLIST_REMOVE(request, list);
 }
-QLIST_INSERT_HEAD(&dataplane->inflight, ioreq, list);
+QLIST_INSERT_HEAD(&dataplane->inflight, request, list);
 dataplane->requests_inflight++;
 
 out:
-return ioreq;
+return request;
 }
 
-static void ioreq_finish(struct ioreq *ioreq)
+static void ioreq_finish(XenBlockRequest *request)
 {
-XenBlockDataPlane *dataplane = ioreq->dataplane;
+XenBlockDataPlane *dataplane = request->dataplane;
 
-QLIST_REMOVE(ioreq, list);
-QLIST_INSERT_HEAD(&dataplane->finished, ioreq, list);
+QLIST_REMOVE(request, list);
+QLIST_INSERT_HEAD(&dataplane->finished, request, list);
 dataplane->requests_inflight--;
 dataplane->requests_finished++;
 }
 
-static void ioreq_release(struct ioreq *ioreq, bool finish)
+static void ioreq_release(XenBlockRequest *request, bool finish)
 {
-XenBlockDataPlane *dataplane = ioreq->dataplane;
+XenBlockDataPlane *dataplane = request->dataplane;
 
-QLIST_REMOVE(ioreq, list);
-ioreq_reset(ioreq);
-ioreq->dataplane = dataplane;
-QLIST_INSERT_HEAD(&dataplane->freelist, ioreq, list);
+QLIST_REMOVE(request, list);
+iore

[Qemu-block] [PATCH v6 18/18] xen: remove the legacy 'xen_disk' backend

[Paul Durrant]

This backend has now been replaced by the 'xen-qdisk' XenDevice.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
---
Cc: Kevin Wolf 
Cc: Max Reitz 
Cc: Stefano Stabellini 
---
 hw/block/Makefile.objs |1 -
 hw/block/xen_disk.c| 1011 
 2 files changed, 1012 deletions(-)
 delete mode 100644 hw/block/xen_disk.c

diff --git a/hw/block/Makefile.objs b/hw/block/Makefile.objs
index f34813a377..e206b8e712 100644
--- a/hw/block/Makefile.objs
+++ b/hw/block/Makefile.objs
@@ -5,7 +5,6 @@ common-obj-$(CONFIG_NAND) += nand.o
 common-obj-$(CONFIG_PFLASH_CFI01) += pflash_cfi01.o
 common-obj-$(CONFIG_PFLASH_CFI02) += pflash_cfi02.o
 common-obj-$(CONFIG_XEN) += xen-block.o
-common-obj-$(CONFIG_XEN) += xen_disk.o
 common-obj-$(CONFIG_ECC) += ecc.o
 common-obj-$(CONFIG_ONENAND) += onenand.o
 common-obj-$(CONFIG_NVME_PCI) += nvme.o
diff --git a/hw/block/xen_disk.c b/hw/block/xen_disk.c
deleted file mode 100644
index 75fe55f2ae..00
--- a/hw/block/xen_disk.c
+++ /dev/null
@@ -1,1011 +0,0 @@
-/*
- *  xen paravirt block device backend
- *
- *  (c) Gerd Hoffmann 
- *
- *  This program is free software; you can redistribute it and/or modify
- *  it under the terms of the GNU General Public License as published by
- *  the Free Software Foundation; under version 2 of the License.
- *
- *  This program is distributed in the hope that it will be useful,
- *  but WITHOUT ANY WARRANTY; without even the implied warranty of
- *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- *  GNU General Public License for more details.
- *
- *  You should have received a copy of the GNU General Public License along
- *  with this program; if not, see .
- *
- *  Contributions after 2012-01-13 are licensed under the terms of the
- *  GNU GPL, version 2 or (at your option) any later version.
- */
-
-#include "qemu/osdep.h"
-#include "qemu/units.h"
-#include 
-#include 
-
-#include "hw/hw.h"
-#include "hw/xen/xen-legacy-backend.h"
-#include "xen_blkif.h"
-#include "sysemu/blockdev.h"
-#include "sysemu/iothread.h"
-#include "sysemu/block-backend.h"
-#include "qapi/error.h"
-#include "qapi/qmp/qdict.h"
-#include "qapi/qmp/qstring.h"
-#include "trace.h"
-
-/* - */
-
-#define BLOCK_SIZE  512
-#define IOCB_COUNT  (BLKIF_MAX_SEGMENTS_PER_REQUEST + 2)
-
-struct ioreq {
-blkif_request_t req;
-int16_t status;
-
-/* parsed request */
-off_t   start;
-QEMUIOVectorv;
-void*buf;
-size_t  size;
-int presync;
-
-/* aio status */
-int aio_inflight;
-int aio_errors;
-
-struct XenBlkDev*blkdev;
-QLIST_ENTRY(ioreq)   list;
-BlockAcctCookie acct;
-};
-
-#define MAX_RING_PAGE_ORDER 4
-
-struct XenBlkDev {
-struct XenLegacyDevicexendev;  /* must be first */
-char*params;
-char*mode;
-char*type;
-char*dev;
-char*devtype;
-booldirectiosafe;
-const char  *fileproto;
-const char  *filename;
-unsigned intring_ref[1 << MAX_RING_PAGE_ORDER];
-unsigned intnr_ring_ref;
-void*sring;
-int64_t file_blk;
-int64_t file_size;
-int protocol;
-blkif_back_rings_t  rings;
-int more_work;
-
-/* request lists */
-QLIST_HEAD(inflight_head, ioreq) inflight;
-QLIST_HEAD(finished_head, ioreq) finished;
-QLIST_HEAD(freelist_head, ioreq) freelist;
-int requests_total;
-int requests_inflight;
-int requests_finished;
-unsigned intmax_requests;
-
-gbooleanfeature_discard;
-
-/* qemu block driver */
-DriveInfo   *dinfo;
-BlockBackend*blk;
-QEMUBH  *bh;
-
-IOThread*iothread;
-AioContext  *ctx;
-};
-
-/* - */
-
-static void ioreq_reset(struct ioreq *ioreq)
-{
-memset(&ioreq->req, 0, sizeof(ioreq->req));
-ioreq->status = 0;
-ioreq->start = 0;
-ioreq->buf = NULL;
-ioreq->size = 0;
-ioreq->presync = 0;
-
-ioreq->aio_inflight = 0;
-ioreq->aio_errors = 0;
-
-ioreq->blkdev = NULL;
-memset(&ioreq->list, 0, sizeof(ioreq->list));
-memset(&ioreq->acct, 0, sizeof(ioreq->acct));
-
-qemu_iovec_reset(&ioreq->v);
-}
-
-static struct ioreq *ioreq_start(struct XenBlkDev *blkdev)
-{
-struct ioreq *ioreq = NULL;
-
-if (QLIST_EMPTY(&blkdev->freelist)) {
-if (blkdev->requests_total >= blkdev->max_requests) {
-goto out;
-}
-/* allocate new struct */
-ioreq = g_malloc0(sizeof(*ioreq));
-io

[Qemu-block] [PATCH v6 17/18] MAINTAINERS: add myself as a Xen maintainer

[Paul Durrant]

I have made many significant contributions to the Xen code in QEMU,
particularly the recent patches introducing a new PV device framework.
I intend to make further significant contributions, porting other PV back-
ends to the new framework with the intent of eventually removing the
legacy code. It therefore seems reasonable that I become a maintainer of
the Xen code.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
Acked-by: Stefano Stabellini 
---
Cc: Paolo Bonzini 

v2:
 - Fix typo
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 9875581df2..e6bd44189a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -396,6 +396,7 @@ Guest CPU Cores (Xen):
 X86
 M: Stefano Stabellini 
 M: Anthony Perard 
+M: Paul Durrant 
 L: xen-de...@lists.xenproject.org
 S: Supported
 F: */xen*
-- 
2.20.1.2.gb21ebb6

[Qemu-block] [PATCH v6 16/18] xen: automatically create XenBlockDevice-s

[Paul Durrant]

This patch adds create and destroy function for XenBlockDevice-s so that
they can be created automatically when the Xen toolstack instantiates a new
PV backend via xenstore. When the XenBlockDevice is created this way it is
also necessary to create a 'drive' which matches the configuration that the
Xen toolstack has written into xenstore. This is done by formulating the
parameters necessary for each 'blockdev' layer of the drive and then using
qmp_blockdev_add() to create the layers. Also, for compatibility with the
legacy 'xen_disk' implementation, an iothread is automatically created for
the new XenBlockDevice. This, like the driver layers, will be destroyed
after the XenBlockDevice is unrealized.

The legacy backend scan for 'qdisk' is removed by this patch, which makes
the 'xen_disk' code is redundant. The code will be removed by a subsequent
patch.

Signed-off-by: Paul Durrant 
---
Cc: Anthony Perard 
Cc: Kevin Wolf 
Cc: Max Reitz 
Cc: Stefano Stabellini 

v5:
 - Extensively re-worked to avoid using drive_new() and use
   qmp_blockdev_add() instead
 - Also use qmp_object_add() for IOThread
 - Dropped Anthony's R-b because of the code changes

v2:
 - Get rid of error_abort
 - Don't use qdev_init_nofail
 - Explain why file locking needs to be off
---
 hw/block/trace-events   |   4 +
 hw/block/xen-block.c| 419 
 hw/xen/xen-legacy-backend.c |   1 -
 include/hw/xen/xen-block.h  |  13 ++
 4 files changed, 436 insertions(+), 1 deletion(-)

diff --git a/hw/block/trace-events b/hw/block/trace-events
index 89e258319c..55e5a5500c 100644
--- a/hw/block/trace-events
+++ b/hw/block/trace-events
@@ -137,3 +137,7 @@ xen_disk_realize(void) ""
 xen_disk_unrealize(void) ""
 xen_cdrom_realize(void) ""
 xen_cdrom_unrealize(void) ""
+xen_block_blockdev_add(char *str) "%s"
+xen_block_blockdev_del(const char *node_name) "%s"
+xen_block_device_create(unsigned int number) "%u"
+xen_block_device_destroy(unsigned int number) "%u"
diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index a7c37c185a..5e69fa0412 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -7,12 +7,20 @@
 
 #include "qemu/osdep.h"
 #include "qemu/cutils.h"
+#include "qemu/option.h"
 #include "qapi/error.h"
+#include "qapi/qapi-commands-block-core.h"
+#include "qapi/qapi-commands-misc.h"
+#include "qapi/qapi-visit-block-core.h"
 #include "qapi/visitor.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/qmp/qstring.h"
+#include "block/qdict.h"
 #include "hw/hw.h"
 #include "hw/xen/xen_common.h"
 #include "hw/block/xen_blkif.h"
 #include "hw/xen/xen-block.h"
+#include "hw/xen/xen-backend.h"
 #include "sysemu/blockdev.h"
 #include "sysemu/block-backend.h"
 #include "sysemu/iothread.h"
@@ -474,6 +482,7 @@ static void xen_block_class_init(ObjectClass *class, void 
*data)
 DeviceClass *dev_class = DEVICE_CLASS(class);
 XenDeviceClass *xendev_class = XEN_DEVICE_CLASS(class);
 
+xendev_class->backend = "qdisk";
 xendev_class->device = "vbd";
 xendev_class->get_name = xen_block_get_name;
 xendev_class->realize = xen_block_realize;
@@ -586,3 +595,413 @@ static void xen_block_register_types(void)
 }
 
 type_init(xen_block_register_types)
+
+static void xen_block_blockdev_del(const char *node_name, Error **errp)
+{
+trace_xen_block_blockdev_del(node_name);
+
+qmp_blockdev_del(node_name, errp);
+}
+
+static void add_item(const char *key, QObject *obj, void *opaque)
+{
+const char *val = qobject_get_try_str(obj);
+char **strp = opaque;
+char *str = *strp;
+
+*strp = (!str) ? g_strdup_printf("%s=%s", key, val) :
+ g_strdup_printf("%s %s=%s", str, key, val);
+g_free(str);
+}
+
+static char *xen_block_blockdev_add(const char *id, QDict *qdict,
+Error **errp)
+{
+const char *driver = qdict_get_try_str(qdict, "driver");
+BlockdevOptions *options = NULL;
+Error *local_err = NULL;
+char *str = NULL;
+char *node_name;
+Visitor *v;
+
+if (!driver) {
+error_setg(errp, "no 'driver' parameter");
+return NULL;
+}
+
+node_name = g_strdup_printf("%s-%s", id, driver);
+qdict_put_str(qdict, "node-name", node_name);
+
+qdict_iter(qdict, add_item, &str);
+
+trace_xen_block_blockdev_add(str);
+g_free(str);
+
+v = qobject_input_visitor_new_flat_confused(qdict, errp);
+if (!v) {
+goto fail;
+}
+
+visit_type_BlockdevOptions(v, NULL, &options, &local_err);
+visit_free(v);
+
+if (local_err) {
+error_propagate(errp, local_err);
+goto fail;
+}
+
+qmp_blockdev_add(options, &local_err);
+
+if (local_err) {
+error_propagate(errp, local_err);
+goto fail;
+}
+
+qapi_free_BlockdevOptions(options);
+
+return node_name;
+
+fail:
+if (options) {
+qapi_free_BlockdevOptions(options);
+}
+g_free(node_name);
+
+return NULL;
+}
+
+static void xen_block_drive_destroy(XenB

[Qemu-block] [PATCH v6 14/18] xen: add implementations of xen-block connect and disconnect functions...

[Paul Durrant]

...and wire in the dataplane.

This patch adds the remaining code to make the xen-block XenDevice
functional. The parameters that a block frontend expects to find are
populated in the backend xenstore area, and the 'ring-ref' and
'event-channel' values specified in the frontend xenstore area are
mapped/bound and used to set up the dataplane.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: Kevin Wolf 
Cc: Max Reitz 

v5:
 - Re-base

v3:
 - Add missing return statement to xen_block_realize()
 - Set device name to 'vbd' now the backend is functional
 - Add an unplug handler
 - Move backend watch code from subsequent patch since it's now needed
   to handle unplug

v2:
 - Tidy up header inclusions
 - Stop leaking ring_ref on error
 - Auto-create drive for CDRom devices
---
 hw/block/xen-block.c   | 166 
 hw/xen/trace-events|   3 +
 hw/xen/xen-bus.c   | 187 -
 include/hw/xen/xen-block.h |   9 ++
 include/hw/xen/xen-bus.h   |  14 ++-
 5 files changed, 353 insertions(+), 26 deletions(-)

diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index 3a963b0383..a7c37c185a 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -10,7 +10,13 @@
 #include "qapi/error.h"
 #include "qapi/visitor.h"
 #include "hw/hw.h"
+#include "hw/xen/xen_common.h"
+#include "hw/block/xen_blkif.h"
 #include "hw/xen/xen-block.h"
+#include "sysemu/blockdev.h"
+#include "sysemu/block-backend.h"
+#include "sysemu/iothread.h"
+#include "dataplane/xen-block.h"
 #include "trace.h"
 
 static char *xen_block_get_name(XenDevice *xendev, Error **errp)
@@ -28,6 +34,8 @@ static void xen_block_disconnect(XenDevice *xendev, Error 
**errp)
 XenBlockVdev *vdev = &blockdev->props.vdev;
 
 trace_xen_block_disconnect(type, vdev->disk, vdev->partition);
+
+xen_block_dataplane_stop(blockdev->dataplane);
 }
 
 static void xen_block_connect(XenDevice *xendev, Error **errp)
@@ -35,8 +43,72 @@ static void xen_block_connect(XenDevice *xendev, Error 
**errp)
 XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
 const char *type = object_get_typename(OBJECT(blockdev));
 XenBlockVdev *vdev = &blockdev->props.vdev;
+unsigned int order, nr_ring_ref, *ring_ref, event_channel, protocol;
+char *str;
 
 trace_xen_block_connect(type, vdev->disk, vdev->partition);
+
+if (xen_device_frontend_scanf(xendev, "ring-page-order", "%u",
+  &order) != 1) {
+nr_ring_ref = 1;
+ring_ref = g_new(unsigned int, nr_ring_ref);
+
+if (xen_device_frontend_scanf(xendev, "ring-ref", "%u",
+  &ring_ref[0]) != 1) {
+error_setg(errp, "failed to read ring-ref");
+g_free(ring_ref);
+return;
+}
+} else if (order <= blockdev->props.max_ring_page_order) {
+unsigned int i;
+
+nr_ring_ref = 1 << order;
+ring_ref = g_new(unsigned int, nr_ring_ref);
+
+for (i = 0; i < nr_ring_ref; i++) {
+const char *key = g_strdup_printf("ring-ref%u", i);
+
+if (xen_device_frontend_scanf(xendev, key, "%u",
+  &ring_ref[i]) != 1) {
+error_setg(errp, "failed to read %s", key);
+g_free((gpointer)key);
+g_free(ring_ref);
+return;
+}
+
+g_free((gpointer)key);
+}
+} else {
+error_setg(errp, "invalid ring-page-order (%d)", order);
+return;
+}
+
+if (xen_device_frontend_scanf(xendev, "event-channel", "%u",
+  &event_channel) != 1) {
+error_setg(errp, "failed to read event-channel");
+g_free(ring_ref);
+return;
+}
+
+if (xen_device_frontend_scanf(xendev, "protocol", "%ms",
+  &str) != 1) {
+protocol = BLKIF_PROTOCOL_NATIVE;
+} else {
+if (strcmp(str, XEN_IO_PROTO_ABI_X86_32) == 0) {
+protocol = BLKIF_PROTOCOL_X86_32;
+} else if (strcmp(str, XEN_IO_PROTO_ABI_X86_64) == 0) {
+protocol = BLKIF_PROTOCOL_X86_64;
+} else {
+protocol = BLKIF_PROTOCOL_NATIVE;
+}
+
+free(str);
+}
+
+xen_block_dataplane_start(blockdev->dataplane, ring_ref, nr_ring_ref,
+  event_channel, protocol, errp);
+
+g_free(ring_ref);
 }
 
 static void xen_block_unrealize(XenDevice *xendev, Error **errp)
@@ -56,6 +128,9 @@ static void xen_block_unrealize(XenDevice *xendev, Error 
**errp)
 /* Disconnect from the frontend in case this has not already happened */
 xen_block_disconnect(xendev, NULL);
 
+xen_block_dataplane_destroy(blockdev->dataplane);
+blockdev->dataplane = NULL;
+
 if (blockdev_class->unrealize) {
 blockdev_class->unrealize(blockdev, errp);
 }
@@ -68,6 +143,7 @@ static 

[Qemu-block] [PATCH v6 10/18] xen: add header and build dataplane/xen-block.c

[Paul Durrant]

This patch adds the transformations necessary to get dataplane/xen-block.c
to build against the new XenBus/XenDevice framework. MAINTAINERS is also
updated due to the introduction of dataplane/xen-block.h.

NOTE: Existing data structure names are retained for the moment. These will
  be modified by subsequent patches. A typedef for XenBlockDataPlane
  has been added to the header (based on the old struct XenBlkDev name
  for the moment) so that the old names don't need to leak out of the
  dataplane code.

Signed-off-by: Paul Durrant 
Reviewed-by: Anthony Perard 
---
Cc: Stefan Hajnoczi 
Cc: Kevin Wolf 
Cc: Max Reitz 
Cc: Stefano Stabellini 

v2:
 - Tidy up header inclusions
 - Get rid of error_fatal
---
 MAINTAINERS  |   1 +
 hw/block/dataplane/Makefile.objs |   1 +
 hw/block/dataplane/xen-block.c   | 356 ++-
 hw/block/dataplane/xen-block.h   |  29 +++
 4 files changed, 287 insertions(+), 100 deletions(-)
 create mode 100644 hw/block/dataplane/xen-block.h

diff --git a/MAINTAINERS b/MAINTAINERS
index ab62ad44a8..9875581df2 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -408,6 +408,7 @@ F: hw/block/dataplane/xen*
 F: hw/xen/
 F: hw/xenpv/
 F: hw/i386/xen/
+F: include/hw/block/dataplane/xen*
 F: include/hw/xen/
 F: include/sysemu/xen-mapcache.h
 
diff --git a/hw/block/dataplane/Makefile.objs b/hw/block/dataplane/Makefile.objs
index e786f66421..c6c68dbc00 100644
--- a/hw/block/dataplane/Makefile.objs
+++ b/hw/block/dataplane/Makefile.objs
@@ -1 +1,2 @@
 obj-y += virtio-blk.o
+obj-$(CONFIG_XEN) += xen-block.o
diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 228472320a..ed2b91 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -18,65 +18,53 @@
  * GNU GPL, version 2 or (at your option) any later version.
  */
 
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+#include "hw/hw.h"
+#include "hw/xen/xen_common.h"
+#include "hw/block/xen_blkif.h"
+#include "sysemu/block-backend.h"
+#include "sysemu/iothread.h"
+#include "xen-block.h"
+
 struct ioreq {
-blkif_request_t req;
-int16_t status;
-
-/* parsed request */
-off_t   start;
-QEMUIOVectorv;
-void*buf;
-size_t  size;
-int presync;
-
-/* aio status */
-int aio_inflight;
-int aio_errors;
-
-struct XenBlkDev*blkdev;
-QLIST_ENTRY(ioreq)   list;
-BlockAcctCookie acct;
+blkif_request_t req;
+int16_t status;
+off_t start;
+QEMUIOVector v;
+void *buf;
+size_t size;
+int presync;
+int aio_inflight;
+int aio_errors;
+struct XenBlkDev *blkdev;
+QLIST_ENTRY(ioreq) list;
+BlockAcctCookie acct;
 };
 
-#define MAX_RING_PAGE_ORDER 4
-
 struct XenBlkDev {
-struct XenLegacyDevicexendev;  /* must be first */
-char*params;
-char*mode;
-char*type;
-char*dev;
-char*devtype;
-booldirectiosafe;
-const char  *fileproto;
-const char  *filename;
-unsigned intring_ref[1 << MAX_RING_PAGE_ORDER];
-unsigned intnr_ring_ref;
-void*sring;
-int64_t file_blk;
-int64_t file_size;
-int protocol;
-blkif_back_rings_t  rings;
-int more_work;
-
-/* request lists */
+XenDevice *xendev;
+XenEventChannel *event_channel;
+unsigned int *ring_ref;
+unsigned int nr_ring_ref;
+void *sring;
+int64_t file_blk;
+int64_t file_size;
+int protocol;
+blkif_back_rings_t rings;
+int more_work;
 QLIST_HEAD(inflight_head, ioreq) inflight;
 QLIST_HEAD(finished_head, ioreq) finished;
 QLIST_HEAD(freelist_head, ioreq) freelist;
-int requests_total;
-int requests_inflight;
-int requests_finished;
-unsigned intmax_requests;
-
-gbooleanfeature_discard;
-
-/* qemu block driver */
-DriveInfo   *dinfo;
-BlockBackend*blk;
-QEMUBH  *bh;
-
-IOThread*iothread;
-AioContext  *ctx;
+int requests_total;
+int requests_inflight;
+int requests_finished;
+unsigned int max_requests;
+BlockBackend *blk;
+QEMUBH *bh;
+IOThread *iothread;
+AioContext *ctx;
 };
 
 static void ioreq_reset(struct ioreq *ioreq)
@@ -155,7 +143,6 @@ static void ioreq_release(struct ioreq *ioreq, bool finish)
 static int ioreq_parse(struct ioreq *ioreq)
 {
 struct XenBlkDev *blkdev = ioreq->blkdev;
-struct XenLegacyDevice *xendev = &blkdev->xendev;
 size_t len;
 int i;
 
@@ -177,7 +164,8 @@ static int ioreq_parse(struct ioreq *ioreq)
 goto err;
 };

[Qemu-block] [PATCH v6 13/18] xen: purge 'blk' and 'ioreq' from function names in dataplane/xen-block.c

[Paul Durrant]

This is a purely cosmetic patch that purges remaining use of 'blk' and
'ioreq' in local function names, and then makes sure all functions are
prefixed with 'xen_block_'.

No functional change.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: Stefan Hajnoczi 
Cc: Kevin Wolf 
Cc: Max Reitz 

v2:
 - Add 'xen_block_' prefix
---
 hw/block/dataplane/xen-block.c | 90 +-
 1 file changed, 46 insertions(+), 44 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 6788bbf338..8e3965e171 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -67,7 +67,7 @@ struct XenBlockDataPlane {
 AioContext *ctx;
 };
 
-static void ioreq_reset(XenBlockRequest *request)
+static void reset_request(XenBlockRequest *request)
 {
 memset(&request->req, 0, sizeof(request->req));
 request->status = 0;
@@ -86,7 +86,7 @@ static void ioreq_reset(XenBlockRequest *request)
 qemu_iovec_reset(&request->v);
 }
 
-static XenBlockRequest *ioreq_start(XenBlockDataPlane *dataplane)
+static XenBlockRequest *xen_block_start_request(XenBlockDataPlane *dataplane)
 {
 XenBlockRequest *request = NULL;
 
@@ -111,7 +111,7 @@ out:
 return request;
 }
 
-static void ioreq_finish(XenBlockRequest *request)
+static void xen_block_finish_request(XenBlockRequest *request)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 
@@ -121,12 +121,12 @@ static void ioreq_finish(XenBlockRequest *request)
 dataplane->requests_finished++;
 }
 
-static void ioreq_release(XenBlockRequest *request, bool finish)
+static void xen_block_release_request(XenBlockRequest *request, bool finish)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 
 QLIST_REMOVE(request, list);
-ioreq_reset(request);
+reset_request(request);
 request->dataplane = dataplane;
 QLIST_INSERT_HEAD(&dataplane->freelist, request, list);
 if (finish) {
@@ -140,7 +140,7 @@ static void ioreq_release(XenBlockRequest *request, bool 
finish)
  * translate request into iovec + start offset
  * do sanity checks along the way
  */
-static int ioreq_parse(XenBlockRequest *request)
+static int xen_block_parse_request(XenBlockRequest *request)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 size_t len;
@@ -201,7 +201,7 @@ err:
 return -1;
 }
 
-static int ioreq_grant_copy(XenBlockRequest *request)
+static int xen_block_copy_request(XenBlockRequest *request)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 XenDevice *xendev = dataplane->xendev;
@@ -247,9 +247,9 @@ static int ioreq_grant_copy(XenBlockRequest *request)
 return 0;
 }
 
-static int ioreq_runio_qemu_aio(XenBlockRequest *request);
+static int xen_block_do_aio(XenBlockRequest *request);
 
-static void qemu_aio_complete(void *opaque, int ret)
+static void xen_block_complete_aio(void *opaque, int ret)
 {
 XenBlockRequest *request = opaque;
 XenBlockDataPlane *dataplane = request->dataplane;
@@ -266,7 +266,7 @@ static void qemu_aio_complete(void *opaque, int ret)
 request->aio_inflight--;
 if (request->presync) {
 request->presync = 0;
-ioreq_runio_qemu_aio(request);
+xen_block_do_aio(request);
 goto done;
 }
 if (request->aio_inflight > 0) {
@@ -277,7 +277,7 @@ static void qemu_aio_complete(void *opaque, int ret)
 case BLKIF_OP_READ:
 /* in case of failure request->aio_errors is increased */
 if (ret == 0) {
-ioreq_grant_copy(request);
+xen_block_copy_request(request);
 }
 qemu_vfree(request->buf);
 break;
@@ -293,7 +293,7 @@ static void qemu_aio_complete(void *opaque, int ret)
 }
 
 request->status = request->aio_errors ? BLKIF_RSP_ERROR : BLKIF_RSP_OKAY;
-ioreq_finish(request);
+xen_block_finish_request(request);
 
 switch (request->req.operation) {
 case BLKIF_OP_WRITE:
@@ -318,9 +318,9 @@ done:
 aio_context_release(dataplane->ctx);
 }
 
-static bool blk_split_discard(XenBlockRequest *request,
-  blkif_sector_t sector_number,
-  uint64_t nr_sectors)
+static bool xen_block_split_discard(XenBlockRequest *request,
+blkif_sector_t sector_number,
+uint64_t nr_sectors)
 {
 XenBlockDataPlane *dataplane = request->dataplane;
 int64_t byte_offset;
@@ -343,7 +343,7 @@ static bool blk_split_discard(XenBlockRequest *request,
 byte_chunk = byte_remaining > limit ? limit : byte_remaining;
 request->aio_inflight++;
 blk_aio_pdiscard(dataplane->blk, byte_offset, byte_chunk,
- qemu_aio_complete, request);
+ xen_block_complete_aio, request);
 byte_remaining -= byte_chunk;
 byte_offset += byte_chunk;
 } while (byte_remaining > 0);
@@ -351,7 +351,7 @@ static bool b

[Qemu-block] [PATCH v6 11/18] xen: remove 'XenBlkDev' and 'blkdev' names from dataplane/xen-block

[Paul Durrant]

This is a purely cosmetic patch that substitutes the old 'struct XenBlkDev'
name with 'XenBlockDataPlane' and 'blkdev' field/variable names with
'dataplane', and then does necessary fix-up to adhere to coding style.

No functional change.

Signed-off-by: Paul Durrant 
Acked-by: Anthony Perard 
---
Cc: Stefano Stabellini 
Cc: Stefan Hajnoczi 
Cc: Kevin Wolf 
Cc: Max Reitz 
---
 hw/block/dataplane/xen-block.c | 352 +
 hw/block/dataplane/xen-block.h |   2 +-
 2 files changed, 183 insertions(+), 171 deletions(-)

diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index ed2b91..1ff464973c 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -38,12 +38,12 @@ struct ioreq {
 int presync;
 int aio_inflight;
 int aio_errors;
-struct XenBlkDev *blkdev;
+XenBlockDataPlane *dataplane;
 QLIST_ENTRY(ioreq) list;
 BlockAcctCookie acct;
 };
 
-struct XenBlkDev {
+struct XenBlockDataPlane {
 XenDevice *xendev;
 XenEventChannel *event_channel;
 unsigned int *ring_ref;
@@ -79,33 +79,33 @@ static void ioreq_reset(struct ioreq *ioreq)
 ioreq->aio_inflight = 0;
 ioreq->aio_errors = 0;
 
-ioreq->blkdev = NULL;
+ioreq->dataplane = NULL;
 memset(&ioreq->list, 0, sizeof(ioreq->list));
 memset(&ioreq->acct, 0, sizeof(ioreq->acct));
 
 qemu_iovec_reset(&ioreq->v);
 }
 
-static struct ioreq *ioreq_start(struct XenBlkDev *blkdev)
+static struct ioreq *ioreq_start(XenBlockDataPlane *dataplane)
 {
 struct ioreq *ioreq = NULL;
 
-if (QLIST_EMPTY(&blkdev->freelist)) {
-if (blkdev->requests_total >= blkdev->max_requests) {
+if (QLIST_EMPTY(&dataplane->freelist)) {
+if (dataplane->requests_total >= dataplane->max_requests) {
 goto out;
 }
 /* allocate new struct */
 ioreq = g_malloc0(sizeof(*ioreq));
-ioreq->blkdev = blkdev;
-blkdev->requests_total++;
+ioreq->dataplane = dataplane;
+dataplane->requests_total++;
 qemu_iovec_init(&ioreq->v, 1);
 } else {
 /* get one from freelist */
-ioreq = QLIST_FIRST(&blkdev->freelist);
+ioreq = QLIST_FIRST(&dataplane->freelist);
 QLIST_REMOVE(ioreq, list);
 }
-QLIST_INSERT_HEAD(&blkdev->inflight, ioreq, list);
-blkdev->requests_inflight++;
+QLIST_INSERT_HEAD(&dataplane->inflight, ioreq, list);
+dataplane->requests_inflight++;
 
 out:
 return ioreq;
@@ -113,26 +113,26 @@ out:
 
 static void ioreq_finish(struct ioreq *ioreq)
 {
-struct XenBlkDev *blkdev = ioreq->blkdev;
+XenBlockDataPlane *dataplane = ioreq->dataplane;
 
 QLIST_REMOVE(ioreq, list);
-QLIST_INSERT_HEAD(&blkdev->finished, ioreq, list);
-blkdev->requests_inflight--;
-blkdev->requests_finished++;
+QLIST_INSERT_HEAD(&dataplane->finished, ioreq, list);
+dataplane->requests_inflight--;
+dataplane->requests_finished++;
 }
 
 static void ioreq_release(struct ioreq *ioreq, bool finish)
 {
-struct XenBlkDev *blkdev = ioreq->blkdev;
+XenBlockDataPlane *dataplane = ioreq->dataplane;
 
 QLIST_REMOVE(ioreq, list);
 ioreq_reset(ioreq);
-ioreq->blkdev = blkdev;
-QLIST_INSERT_HEAD(&blkdev->freelist, ioreq, list);
+ioreq->dataplane = dataplane;
+QLIST_INSERT_HEAD(&dataplane->freelist, ioreq, list);
 if (finish) {
-blkdev->requests_finished--;
+dataplane->requests_finished--;
 } else {
-blkdev->requests_inflight--;
+dataplane->requests_inflight--;
 }
 }
 
@@ -142,7 +142,7 @@ static void ioreq_release(struct ioreq *ioreq, bool finish)
  */
 static int ioreq_parse(struct ioreq *ioreq)
 {
-struct XenBlkDev *blkdev = ioreq->blkdev;
+XenBlockDataPlane *dataplane = ioreq->dataplane;
 size_t len;
 int i;
 
@@ -165,12 +165,12 @@ static int ioreq_parse(struct ioreq *ioreq)
 };
 
 if (ioreq->req.operation != BLKIF_OP_READ &&
-blk_is_read_only(blkdev->blk)) {
+blk_is_read_only(dataplane->blk)) {
 error_report("error: write req for ro device");
 goto err;
 }
 
-ioreq->start = ioreq->req.sector_number * blkdev->file_blk;
+ioreq->start = ioreq->req.sector_number * dataplane->file_blk;
 for (i = 0; i < ioreq->req.nr_segments; i++) {
 if (i == BLKIF_MAX_SEGMENTS_PER_REQUEST) {
 error_report("error: nr_segments too big");
@@ -180,16 +180,16 @@ static int ioreq_parse(struct ioreq *ioreq)
 error_report("error: first > last sector");
 goto err;
 }
-if (ioreq->req.seg[i].last_sect * blkdev->file_blk >= XC_PAGE_SIZE) {
+if (ioreq->req.seg[i].last_sect * dataplane->file_blk >= XC_PAGE_SIZE) 
{
 error_report("error: page crossing");
 goto err;
 }
 
 len = (ioreq->req.seg[i].last_sect -
-   ioreq->req.seg[i].first_sect + 1) * blkdev->file_blk;
+   ioreq->req

[Qemu-block] [PATCH v6 15/18] xen: add a mechanism to automatically create XenDevice-s...

[Paul Durrant]

...that maintains compatibility with existing Xen toolstacks.

Xen toolstacks instantiate PV backends by simply writing information into
xenstore and expecting a backend implementation to be watching for this.

This patch adds a new 'xen-backend' module to allow individual XenDevice
implementations to register create and destroy functions. The creator
will be called when a tool-stack instantiates a new backend in this way,
and the destructor will then be called after the resulting XenDevice
object is unrealized.

To support this it is also necessary to add new watchers into the XenBus
implementation to handle enumeration of new backends and also destruction
of XenDevice-s when the toolstack sets the backend 'online' key to 0.

NOTE: This patch only adds the framework. A subsequent patch will add a
  creator function for xen-block devices.

Signed-off-by: Paul Durrant 
---
Cc: Anthony Perard 
Cc: Stefano Stabellini 
Cc: Anthony Perard 

v5:
 - Added destructor function, therefore Anthony's R-b is not carried
   from v4

v3:
 - Use a xenstore transaction in enumeration code to ensure consistent
   parameters
 - Not adding Anthony's R-b because of change

v2:
 - Sort out error paths and error reporting
---
 hw/xen/Makefile.objs |   2 +-
 hw/xen/trace-events  |   3 +
 hw/xen/xen-backend.c | 165 +++
 hw/xen/xen-bus.c | 164 +-
 include/hw/xen/xen-backend.h |  39 +
 include/hw/xen/xen-bus.h |   1 +
 include/qemu/module.h|   3 +
 7 files changed, 375 insertions(+), 2 deletions(-)
 create mode 100644 hw/xen/xen-backend.c
 create mode 100644 include/hw/xen/xen-backend.h

diff --git a/hw/xen/Makefile.objs b/hw/xen/Makefile.objs
index 77c0868190..84df60a928 100644
--- a/hw/xen/Makefile.objs
+++ b/hw/xen/Makefile.objs
@@ -1,5 +1,5 @@
 # xen backend driver support
-common-obj-$(CONFIG_XEN) += xen-legacy-backend.o xen_devconfig.o xen_pvdev.o 
xen-common.o xen-bus.o xen-bus-helper.o
+common-obj-$(CONFIG_XEN) += xen-legacy-backend.o xen_devconfig.o xen_pvdev.o 
xen-common.o xen-bus.o xen-bus-helper.o xen-backend.o
 
 obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen-host-pci-device.o
 obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen_pt.o xen_pt_config_init.o 
xen_pt_graphics.o xen_pt_msi.o
diff --git a/hw/xen/trace-events b/hw/xen/trace-events
index d4651bdb30..f6944624b2 100644
--- a/hw/xen/trace-events
+++ b/hw/xen/trace-events
@@ -16,6 +16,9 @@ xen_domid_restrict(int err) "err: %u"
 # include/hw/xen/xen-bus.c
 xen_bus_realize(void) ""
 xen_bus_unrealize(void) ""
+xen_bus_enumerate(void) ""
+xen_bus_type_enumerate(const char *type) "type: %s"
+xen_bus_backend_create(const char *type, const char *path) "type: %s path: %s"
 xen_bus_add_watch(const char *node, const char *key, char *token) "node: %s 
key: %s token: %s"
 xen_bus_remove_watch(const char *node, const char *key, char *token) "node: %s 
key: %s token: %s"
 xen_bus_watch(const char *token) "token: %s"
diff --git a/hw/xen/xen-backend.c b/hw/xen/xen-backend.c
new file mode 100644
index 00..da065f81b7
--- /dev/null
+++ b/hw/xen/xen-backend.c
@@ -0,0 +1,165 @@
+/*
+ * Copyright (c) 2018  Citrix Systems Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+#include "hw/xen/xen-backend.h"
+#include "hw/xen/xen-bus.h"
+
+typedef struct XenBackendImpl {
+const char *type;
+XenBackendDeviceCreate create;
+XenBackendDeviceDestroy destroy;
+} XenBackendImpl;
+
+struct XenBackendInstance {
+QLIST_ENTRY(XenBackendInstance) entry;
+const XenBackendImpl *impl;
+XenBus *xenbus;
+char *name;
+XenDevice *xendev;
+};
+
+static GHashTable *xen_backend_table_get(void)
+{
+static GHashTable *table;
+
+if (table == NULL) {
+table = g_hash_table_new(g_str_hash, g_str_equal);
+}
+
+return table;
+}
+
+static void xen_backend_table_add(XenBackendImpl *impl)
+{
+g_hash_table_insert(xen_backend_table_get(), (void *)impl->type, impl);
+}
+
+static const XenBackendImpl *xen_backend_table_lookup(const char *type)
+{
+return g_hash_table_lookup(xen_backend_table_get(), type);
+}
+
+void xen_backend_register(const XenBackendInfo *info)
+{
+XenBackendImpl *impl = g_new0(XenBackendImpl, 1);
+
+g_assert(info->type);
+
+if (xen_backend_table_lookup(info->type)) {
+error_report("attempt to register duplicate Xen backend type '%s'",
+ info->type);
+abort();
+}
+
+if (!info->create) {
+error_report("backend type '%s' has no creator", info->type);
+abort();
+}
+
+impl->type = info->type;
+impl->create = info->create;
+impl->destroy = info->destroy;
+
+xen_backend_table_add(impl);
+}
+
+static QLIST_HEAD(, XenBackendInstance) backend_list;
+
+static void xen_backend

Re: [Qemu-block] [PATCH 2/2] avoid TABs in files that only contain a few

[Stefan Markovic]

On 13.12.18. 23:37, Paolo Bonzini wrote:
> Most files that have TABs only contain a handful of them.  Change
> them to spaces so that we don't confuse people.
>
> disas, standard-headers, linux-headers and libdecnumber are imported
> from other projects and probably should be exempted from the check.
> Outside those, after this patch the following files still contain both
> 8-space and TAB sequences at the beginning of the line.  Many of them
> have a majority of TABs, or were initially committed with all tabs.
>
>  bsd-user/i386/target_syscall.h
>  bsd-user/x86_64/target_syscall.h
>  crypto/aes.c
>  hw/audio/fmopl.c
>  hw/audio/fmopl.h
>  hw/block/tc58128.c
>  hw/display/cirrus_vga.c
>  hw/display/xenfb.c
>  hw/dma/etraxfs_dma.c
>  hw/intc/sh_intc.c
>  hw/misc/mst_fpga.c
>  hw/net/pcnet.c
>  hw/sh4/sh7750.c
>  hw/timer/m48t59.c
>  hw/timer/sh_timer.c
>  include/crypto/aes.h
>  include/disas/bfd.h
>  include/hw/sh4/sh.h
>  libdecnumber/decNumber.c
>  linux-headers/asm-generic/unistd.h
>  linux-headers/linux/kvm.h
>  linux-user/alpha/target_syscall.h
>  linux-user/arm/nwfpe/double_cpdo.c
>  linux-user/arm/nwfpe/fpa11_cpdt.c
>  linux-user/arm/nwfpe/fpa11_cprt.c
>  linux-user/arm/nwfpe/fpa11.h
>  linux-user/flat.h
>  linux-user/flatload.c
>  linux-user/i386/target_syscall.h
>  linux-user/ppc/target_syscall.h
>  linux-user/sparc/target_syscall.h
>  linux-user/syscall.c
>  linux-user/syscall_defs.h
>  linux-user/x86_64/target_syscall.h
>  slirp/cksum.c
>  slirp/if.c
>  slirp/ip.h
>  slirp/ip_icmp.c
>  slirp/ip_icmp.h
>  slirp/ip_input.c
>  slirp/ip_output.c
>  slirp/mbuf.c
>  slirp/misc.c
>  slirp/sbuf.c
>  slirp/socket.c
>  slirp/socket.h
>  slirp/tcp_input.c
>  slirp/tcpip.h
>  slirp/tcp_output.c
>  slirp/tcp_subr.c
>  slirp/tcp_timer.c
>  slirp/tftp.c
>  slirp/udp.c
>  slirp/udp.h
>  target/cris/cpu.h
>  target/cris/mmu.c
>  target/cris/op_helper.c
>  target/sh4/helper.c
>  target/sh4/op_helper.c
>  target/sh4/translate.c
>  tcg/sparc/tcg-target.inc.c
>  tests/tcg/cris/check_addo.c
>  tests/tcg/cris/check_moveq.c
>  tests/tcg/cris/check_swap.c
>  tests/tcg/multiarch/test-mmap.c
>  ui/vnc-enc-hextile-template.h
>  ui/vnc-enc-zywrle.h
>  util/envlist.c
>  util/readline.c
>
> The following have only TABs:
>
>  bsd-user/i386/target_signal.h
>  bsd-user/sparc64/target_signal.h
>  bsd-user/sparc64/target_syscall.h
>  bsd-user/sparc/target_signal.h
>  bsd-user/sparc/target_syscall.h
>  bsd-user/x86_64/target_signal.h
>  crypto/desrfb.c
>  hw/audio/intel-hda-defs.h
>  hw/core/uboot_image.h
>  hw/sh4/sh7750_regnames.c
>  hw/sh4/sh7750_regs.h
>  include/hw/cris/etraxfs_dma.h
>  linux-user/alpha/termbits.h
>  linux-user/arm/nwfpe/fpopcode.h
>  linux-user/arm/nwfpe/fpsr.h
>  linux-user/arm/syscall_nr.h
>  linux-user/arm/target_signal.h
>  linux-user/cris/target_signal.h
>  linux-user/i386/target_signal.h
>  linux-user/linux_loop.h
>  linux-user/m68k/target_signal.h
>  linux-user/microblaze/target_signal.h
>  linux-user/mips64/target_signal.h
>  linux-user/mips/target_signal.h
>  linux-user/mips/target_syscall.h
>  linux-user/mips/termbits.h
>  linux-user/ppc/target_signal.h
>  linux-user/sh4/target_signal.h
>  linux-user/sh4/termbits.h
>  linux-user/sparc64/target_syscall.h
>  linux-user/sparc/target_signal.h
>  linux-user/x86_64/target_signal.h
>  linux-user/x86_64/termbits.h
>  pc-bios/optionrom/optionrom.h
>  slirp/mbuf.h
>  slirp/misc.h
>  slirp/sbuf.h
>  slirp/tcp.h
>  slirp/tcp_timer.h
>  slirp/tcp_var.h
>  target/i386/svm.h
>  target/sparc/asi.h
>  target/xtensa/core-dc232b/xtensa-modules.inc.c
>  target/xtensa/core-dc233c/xtensa-modules.inc.c
>  target/xtensa/core-de212/core-isa.h
>  target/xtensa/core-de212/xtensa-modules.inc.c
>  target/xtensa/core-fsf/xtensa-modules.inc.c
>  target/xtensa/core-sample_controller/core-isa.h
>  target/xtensa/core-sample_controller/xtensa-modules.inc.c
>  target/xtensa/core-test_kc705_be/core-isa.h
>  target/xtensa/core-test_kc705_be/xtensa-modules.inc.c
>  tests/tcg/cris/check_abs.c
>  tests/tcg/cris/check_addc.c
>  tests/tcg/cris/check_addcm.c
>  tests/tcg/cris/check_addoq.c
>  tests/tcg/cris/check_bound.c
>  tests/tcg/cris/check_ftag.c
>  tests/tcg/cris/check_int64.c
>  tests/tcg/cris/check_lz.c
>  tests/tcg/cris/check_openpf5.c
>  tests/tcg/cris/check_sigalrm.c
>  tests/tcg/cris/crisutils.h
>  tests/tcg/cris/sys.c
>  tests/tcg/i386/test-i386-ssse3.c
>  ui/vgafont.h


MIPS parts:

Reviewed-by: Stefan Markovic 


> Signed-off-by: Paolo Bonzini 
> ---

Re: [Qemu-block] [PATCH v3 6/7] iotests: allow pretty-print for qmp_log

[Vladimir Sementsov-Ogievskiy]

15.12.2018 2:15, John Snow wrote:
> If iotests have lines exceeding >998 characters long, git doesn't
> want to send it plaintext to the list. We can solve this by allowing
> the iotests to use pretty printed QMP output that we can match against
> instead.
> 
> As a bonus, it's much nicer for human eyes, too.
> 
> Note that this changes the sort order for "command" and "arguments",
> so I restrict this reordering to occur only when the indent is specified.
> ---
>   tests/qemu-iotests/iotests.py | 17 +
>   1 file changed, 13 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
> index 9595429fea..ab5823c011 100644
> --- a/tests/qemu-iotests/iotests.py
> +++ b/tests/qemu-iotests/iotests.py
> @@ -447,12 +447,21 @@ class VM(qtest.QEMUQtestMachine):
>   result.append(filter_qmp_event(ev))
>   return result
>   
> -def qmp_log(self, cmd, filters=[filter_testfiles], **kwargs):
> -logmsg = '{"execute": "%s", "arguments": %s}' % \
> -(cmd, json.dumps(kwargs, sort_keys=True))
> +def qmp_log(self, cmd, indent=None, filters=[filter_testfiles], 
> **kwargs):
> +# Python < 3.4 needs to know not to add whitespace when 
> pretty-printing:
> +separators = (',', ': ') if indent is not None else (',', ': ')
> +if indent is not None:
> +fullcmd = { "execute": cmd,
> +"arguments": kwargs }
> +logmsg = json.dumps(fullcmd, indent=indent, 
> separators=separators,
> +sort_keys=True)
> +else:
> +logmsg = '{"execute": "%s", "arguments": %s}' % \
> +(cmd, json.dumps(kwargs, sort_keys=True))

definitely overlogic on None/is not None, but anyway, ',' separator leads to 
less
beautiful output. So, I think it is better to do just "re.sub(r'\s+\n', '\n', 
longmsg)"
on final message, here or in log(), or in both.

>   log(logmsg, filters)
>   result = self.qmp(cmd, **kwargs)
> -log(json.dumps(result, sort_keys=True), filters)
> +log(json.dumps(result, indent=indent, separators=separators,
> +   sort_keys=True), filters)
>   return result
>   
>   def run_job(self, job, auto_finalize=True, auto_dismiss=False):
> 


-- 
Best regards,
Vladimir

Re: [Qemu-block] [PATCH v3 5/7] iotests: add filter_generated_node_ids

[Vladimir Sementsov-Ogievskiy]

15.12.2018 2:15, John Snow wrote:
> To mimic the common filter of the same name, but for the python tests.
> 
> Signed-off-by: John Snow  '>'

and also, strange CC line in header, including "John Snow , 
John"

> Reviewed-by: Eric Blake 
> Reviewed-by: Vladimir Sementsov-Ogievskiy 
> ---
>   tests/qemu-iotests/iotests.py | 3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
> index a34e66813a..9595429fea 100644
> --- a/tests/qemu-iotests/iotests.py
> +++ b/tests/qemu-iotests/iotests.py
> @@ -239,6 +239,9 @@ def filter_testfiles(msg):
>   prefix = os.path.join(test_dir, "%s-" % (os.getpid()))
>   return msg.replace(prefix, 'TEST_DIR/PID-')
>   
> +def filter_generated_node_ids(msg):
> +return re.sub("#block[0-9]+", "NODE_NAME", msg)
> +
>   def filter_img_info(output, filename):
>   lines = []
>   for line in output.split('\n'):
> 


-- 
Best regards,
Vladimir

Re: [Qemu-block] Virtio-BLK/SCSI write requests and data payload checksums

[Stefan Hajnoczi]

On Sun, Dec 16, 2018 at 06:53:44PM +0100, Peter Lieven wrote:
> It turned out that for writes a bounce buffer is indeed always necessary. But 
> what I found out is that
> it seems that even for reads it happens that the OS (Windows in this case) 
> issues 2 read requests with
> the same buffer in parallel. Is it possible that this is a bug in the virtio 
> or has anybone ever seen this before?

Is it Windows resume-from-sleep?  I remember a race condition that
Microsoft fixed.

The other weird thing I've seen is scatter-gather lists where the same
buffer address is used multiple times within one request (this can go
wrong if the host splits up the request, causing races).

Stefan


signature.asc
Description: PGP signature

Re: [Qemu-block] [PATCH v2 04/22] qemu-nbd: Enhance man page

[Eric Blake]

On 12/15/18 8:13 AM, Richard W.M. Jones wrote:

On Sat, Dec 15, 2018 at 07:53:06AM -0600, Eric Blake wrote:

Document some useful qemu-nbd command lines. Mention some restrictions
on particular options, like -p being only for MBR images, or -c/-d
being Linux-only.  Update some text given the recent change to no
longer serve oldstyle protocol (missed in commit 7f7dfe2a).  Also,
consistently use trailing '.' in describing options.

Signed-off-by: Eric Blake 

---
v2: new patch
---
  qemu-nbd.texi | 85 +++
  1 file changed, 66 insertions(+), 19 deletions(-)

diff --git a/qemu-nbd.texi b/qemu-nbd.texi
index 9a84e81eed9..0e24c2801ee 100644
--- a/qemu-nbd.texi
+++ b/qemu-nbd.texi
@@ -8,7 +8,10 @@

  @c man begin DESCRIPTION

-Export a QEMU disk image using the NBD protocol.
+Provide access to various QEMU NBD features.  Most commonly used to
+export a QEMU disk image using the NBD protocol as a server, but can
+also be used (on Linux) to manage kernel bindings of a /dev/nbdX
+block device to a QEMU server.


This is only a minor quibble, but I thought the original text was a
good summary, and only needs additional paragraphs describing the more
minor use cases.  Thus the description would become by the end of the
patch series:

   DESCRIPTION

   Export a QEMU disk image using the NBD protocol.

   Other uses:
   * (On Linux) bind /dev/nbdX block device to a QEMU server.
   * As a client to query exports of a remote NBD server.


Seems reasonable.




+@c man begin EXAMPLES
+Start a server listening on port 10809 that exposes only the
+guest-visible contents of a qcow2 file, with no TLS encryption, and
+with the default export name (an empty string). The command will block
+until the first successful client disconnects:


TBH I'd always include the -t option in every example.  I don't
understand (except for backwards compatibility) why it isn't the
default since it's something I always trip over when using qemu-nbd.


I'd still like one example without -t, to call out specifically that it 
creates a one-shot server that goes away after the first client, but 
don't mind fixing the rest of the examples to use -t.


Using -e for read-only connections makes sense, using -e for writable 
exports is a bit more questionable - we _don't_ advertise the 
NBD_FLAG_CAN_MULTI_CONN which states that caches are kept consistent 
between simultaneous write connections, although maybe we should see if 
qemu-nbd can start promising multi-write consistency in future patches?


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-block] Virtio-BLK/SCSI write requests and data payload checksums

[Peter Lieven]

Am 17.12.18 um 15:48 schrieb Stefan Hajnoczi:

On Sun, Dec 16, 2018 at 06:53:44PM +0100, Peter Lieven wrote:

It turned out that for writes a bounce buffer is indeed always necessary. But 
what I found out is that
it seems that even for reads it happens that the OS (Windows in this case) 
issues 2 read requests with
the same buffer in parallel. Is it possible that this is a bug in the virtio or 
has anybone ever seen this before?

Is it Windows resume-from-sleep?  I remember a race condition that
Microsoft fixed.



Its a running guest, no resume. From the dumped Data it seems that its often 
(not always) log files.




The other weird thing I've seen is scatter-gather lists where the same
buffer address is used multiple times within one request (this can go
wrong if the host splits up the request, causing races).



The requests go to different LBAs often several 100MB apart from each other. 
The only thing I

can see it that the request size always seems to be 4096 byte.


I have not seen data corruption so far, but this at all seems very strange.

I only have observed this because the data digest checksum is wrong. After a 
retry it is correct.

When dumping the buffer addresses (in this case the address from the first 
buffer in the qiov)

that came from Qemu/the guest the address is the same.


Actually I don't know for sure that the address comes from the guest. In theory 
it could be that

the request from the guest was less than 4096 byte and Qemu just assigned a 
bounce buffer

to read the whole block and copied the relevant part to the guest. Can I see 
from the

address (0x7fa4354c) if it comes from Qemu or the guest?


Thanks,

Peter

Re: [Qemu-block] [PATCH v2 12/22] nbd/client: Improve error handling in nbd_negotiate_simple_meta_context()

[Eric Blake]

On 12/15/18 9:19 AM, Richard W.M. Jones wrote:

On Sat, Dec 15, 2018 at 07:53:14AM -0600, Eric Blake wrote:

Always allocate space for the reply returned by the server and
hoist the trace earlier, as it is more interesting to trace the
server's reply (even if it is unexpected) than parroting our
request only on success.  After all, skipping the allocation
for a wrong size was merely a micro-optimization that only
benefitted a broken server, rather than the common case of a
compliant server that meets our expectations.

Then turn the reply handling into a loop (even though we still
never iterate more than once), to make this code easier to use
when later patches do support multiple server replies.  This
changes the error message for a server with two replies (a
corner case we are unlikely to hit in practice) from:

Unexpected reply type 4 (meta context), expected 0 (ack)

to:

Server replied with more than one context

Signed-off-by: Eric Blake 

---
v2: split patch into easier-to-review pieces [Rich, Vladimir]
---
  nbd/client.c | 16 
  1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/nbd/client.c b/nbd/client.c
index bcccd5f555e..b6a85fc3ef8 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -684,10 +684,11 @@ static int nbd_negotiate_simple_meta_context(QIOChannel 
*ioc,
  return ret;
  }

-if (reply.type == NBD_REP_META_CONTEXT) {
+while (reply.type == NBD_REP_META_CONTEXT) {


I'm not sure I understand why this change is safe.

As far as I can see reply.type is only updated in the loop by
nbd_receive_option_reply, and that reads from the server, and so the
server might keep sending NBD_REP_META_CONTEXT packets (instead of the
expected NBD_REP_ACK), so it could now loop forever against a
malicious server?  (This is not taking into account any later patches)


The loop can execute at most twice:



Rich.


  char *name;

-if (reply.length != sizeof(info->context_id) + context_len) {
+if (reply.length <= sizeof(info->context_id) ||
+reply.length > NBD_MAX_BUFFER_SIZE) {
  error_setg(errp, "Failed to negotiate meta context '%s', server "
 "answered with unexpected length %" PRIu32, context,
 reply.length);
@@ -708,6 +709,15 @@ static int nbd_negotiate_simple_meta_context(QIOChannel 
*ioc,
  return -1;
  }
  name[reply.length] = '\0';
+trace_nbd_opt_meta_reply(name, info->context_id);
+
+if (received) {
+error_setg(errp, "Server replied with more than one context");
+g_free(name);
+nbd_send_opt_abort(ioc);
+return -1;
+}


If the server replies with a second context, we break the loop by 
complaining.


The old code accepted at most one context, by complaining if the 
server's second reply was not ACK; the new code accepts at most one 
context, by complaining if the server sent more than one context, so the 
net effect of killing the connection for a misbehaving server response 
to SET is unchanged.


However, your point about a misbehaving server providing an infinite 
stream of responses to NBD_OPT_LIST or NBD_OPT_LIST_META_CONTEXT is an 
interesting question, and may be worth asking upstream to see if the NBD 
protocol should be tweaked to document any boundaries at how many 
listings a server might send before a client should worry about the 
server being malicious.  (Does not affect this patch, but pre-existing 
when we call nbd_receive_list() for servers that lack NBD_OPT_GO, and 
does impact the later patches in this series that call 
NBD_OPT_LIST_META_CONTEXT for 'qemu-nbd --list').


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-block] [PATCH v2 12/22] nbd/client: Improve error handling in nbd_negotiate_simple_meta_context()

[Eric Blake]

On 12/15/18 9:19 AM, Richard W.M. Jones wrote:

On Sat, Dec 15, 2018 at 07:53:14AM -0600, Eric Blake wrote:

Always allocate space for the reply returned by the server and
hoist the trace earlier, as it is more interesting to trace the
server's reply (even if it is unexpected) than parroting our
request only on success.  After all, skipping the allocation
for a wrong size was merely a micro-optimization that only
benefitted a broken server, rather than the common case of a
compliant server that meets our expectations.

Then turn the reply handling into a loop (even though we still
never iterate more than once), to make this code easier to use
when later patches do support multiple server replies.  This
changes the error message for a server with two replies (a
corner case we are unlikely to hit in practice) from:

Unexpected reply type 4 (meta context), expected 0 (ack)

to:

Server replied with more than one context

Signed-off-by: Eric Blake 

---
v2: split patch into easier-to-review pieces [Rich, Vladimir]
---
  nbd/client.c | 16 
  1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/nbd/client.c b/nbd/client.c
index bcccd5f555e..b6a85fc3ef8 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -684,10 +684,11 @@ static int nbd_negotiate_simple_meta_context(QIOChannel 
*ioc,
  return ret;
  }

-if (reply.type == NBD_REP_META_CONTEXT) {
+while (reply.type == NBD_REP_META_CONTEXT) {


I'm not sure I understand why this change is safe.

As far as I can see reply.type is only updated in the loop by
nbd_receive_option_reply, and that reads from the server, and so the
server might keep sending NBD_REP_META_CONTEXT packets (instead of the
expected NBD_REP_ACK), so it could now loop forever against a
malicious server?  (This is not taking into account any later patches)


Hmm - now that I've already responded to why the conversion to a loop 
does not change this code, I'm now wondering if I even need this patch. 
In v1 of the series, both SET and LIST shared a common function, and 
since LIST needs the loop, converting SET to use a loop that exits early 
if it executes more than once was needed to make the two actions share a 
common entry point.  But since v2 uses different entry points (because 
it separated the common code into separate helper functions, leaving the 
SET entry point unchanged and adding a new LIST entry point), where only 
the LIST entry point actually has to loop, I might be able to just drop 
this patch entirely and still achieve the same effect.


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-block] [PATCH v2 13/22] nbd/client: Split out nbd_send_one_meta_context()

[Eric Blake]

On 12/15/18 9:22 AM, Richard W.M. Jones wrote:

On Sat, Dec 15, 2018 at 07:53:15AM -0600, Eric Blake wrote:

Refactor nbd_negotiate_simple_meta_context() to pull out the
code that can be reused to send a LIST request for 0 or 1 query.
No semantic change.

Signed-off-by: Eric Blake 

---
v2: split patch into easier-to-review pieces [Rich, Vladimir]
---
  nbd/client.c | 64 ++--
  nbd/trace-events |  2 +-
  2 files changed, 46 insertions(+), 20 deletions(-)



+uint32_t queries = !!query;


This initialization...



+uint32_t context_len = 0;
+uint32_t data_len;
+char *data;
+char *p;
+
+data_len = sizeof(export_len) + export_len + sizeof(queries);


...plus the fact that it is now sizeof(variable) instead of sizeof(type)...


@@ -651,26 +693,10 @@ static int nbd_negotiate_simple_meta_context(QIOChannel 
*ioc,
  NBDOptionReply reply;
  const char *context = info->x_dirty_bitmap ?: "base:allocation";
  bool received = false;
-uint32_t export_len = strlen(info->name);
-uint32_t context_len = strlen(context);
-uint32_t data_len = sizeof(export_len) + export_len +
-sizeof(uint32_t) + /* number of queries */


...was the reason that I dropped the comment here.  The comment made 
sense for explaining why a sizeof(type) was being injected into data_len,



-sizeof(context_len) + context_len;
-char *data = g_malloc(data_len);
-char *p = data;

-trace_nbd_opt_meta_request(context, info->name);
-stl_be_p(p, export_len);
-memcpy(p += sizeof(export_len), info->name, export_len);
-stl_be_p(p += export_len, 1);


...because the old code was hard-coding 1 query, while the new code uses 
the variable (whose value is the number of queries) rather than a 
hard-coding.





Although a straightforward refactoring, we still lost the comment
/* number of queries */.  I'd still perhaps like to see a bit more
explanation of the layout and reasoning behind the data buffer.


Perhaps a possible improvement would be to introduce a packed struct 
that matches the protocol layout, instead of piece-meal constructing the 
struct.  But I'm not sure how much effort to spend on this code.



But anyway ..

Reviewed-by: Richard W.M. Jones 


Thanks. I'm glad you forced me to split the v1 patch into more 
manageable pieces; I like how it turned out.


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-block] [PATCH v2 18/22] nbd/client: Add nbd_receive_export_list()

[Eric Blake]

On 12/15/18 9:42 AM, Richard W.M. Jones wrote:

On Sat, Dec 15, 2018 at 07:53:20AM -0600, Eric Blake wrote:

We want to be able to detect whether a given qemu NBD server is
exposing the right export(s) and dirty bitmaps, at least for
regression testing.  We could use 'nbd-client -l' from the upstream
NBD project to list exports, but it's annoying to rely on
out-of-tree binaries; furthermore, nbd-client doesn't necessarily
know about all of the qemu NBD extensions.  Thus, we plan on adding
a new mode to qemu-nbd that merely sniffs all possible information
from the server during handshake phase, then disconnects and dumps
the information.

This patch adds the low-level client code for grabbing the list
of exports.  It benefits from the recent refactoring patches, as
well as a minor tweak of changing nbd_opt_go() to nbd_opt_info_or_go(),
in order to share as much code as possible when it comes to doing
validation of server replies.  The resulting information is stored
in an array of NBDExportInfo which has been expanded to any
description string, along with a convenience function for freeing
the list.

Signed-off-by: Eric Blake 

---



+while (1) {
+char *name;
+char *desc;
+
+rc = nbd_receive_list(ioc, &name, &desc, errp);
+if (rc < 0) {
+goto out;
+} else if (rc == 0) {
+break;
+}
+array = g_renew(NBDExportInfo, array, ++count);
+memset(&array[count - 1], 0, sizeof(*array));
+array[count - 1].name = name;
+array[count - 1].description = desc;
+array[count - 1].structured_reply = result == 3;
+}


Do we care about limiting ‘count’ to some reasonable value here?


Maybe.



I tried to look at the protocol document to see if there's a limit on
the number of exports that a server can have, but if there is I cannot
find it.  I don't know how much we care about malicious NBD servers --
mostly I'm interested in malicious NBD clients :-)


You brought up the same question earlier in the series, and we already 
have a pre-existing repeat of the situation with NBD_OPT_LIST for 
servers that lack NBD_OPT_GO.


A client can always hang up on a server that it doesn't like; so maybe 
we could set our own limit - but what is a good limit?  An NBD server 
set up to export every file in a given directory, where that directory 
has over 1k files, doesn't seem too far-fetched.  Having 1000 contexts 
available in NBD_OPT_LIST_META_CONTEXT, on the other hand, is less likely.


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-block] [PATCH v3 7/7] iotests: add iotest 236 for testing bitmap merge

[Vladimir Sementsov-Ogievskiy]

15.12.2018 2:15, John Snow wrote:
> New interface, new smoke test.
> ---
>   tests/qemu-iotests/236 | 124 +++
>   tests/qemu-iotests/236.out | 200 +
>   tests/qemu-iotests/group   |   1 +
>   3 files changed, 325 insertions(+)
>   create mode 100755 tests/qemu-iotests/236
>   create mode 100644 tests/qemu-iotests/236.out
> 
> diff --git a/tests/qemu-iotests/236 b/tests/qemu-iotests/236
> new file mode 100755
> index 00..edf16c4ab1
> --- /dev/null
> +++ b/tests/qemu-iotests/236
> @@ -0,0 +1,124 @@
> +#!/usr/bin/env python
> +#
> +# Test bitmap merges.
> +#
> +# Copyright (c) 2018 John Snow for Red Hat, Inc.
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program.  If not, see .
> +#
> +# owner=js...@redhat.com
> +
> +import json
> +import iotests
> +from iotests import log
> +
> +iotests.verify_image_format(supported_fmts=['qcow2'])
> +

we hardcode patterns, so it's better to hardcode size here too:

size = 64 * 1024 * 1024

> +patterns = [("0x5d", "0", "64k"),
> +("0xd5", "1M","64k"),
> +("0xdc", "32M",   "64k"),
> +("0xcd", "0x3ff", "64k")]  # 64M - 64K
> +
> +overwrite = [("0xab", "0", "64k"), # Full overwrite
> + ("0xad", "0x00f8000", "64k"), # Partial-left (1M-32K)
> + ("0x1d", "0x2008000", "64k"), # Partial-right (32M+32K)
> + ("0xea", "0x3fe", "64k")] # Adjacent-left (64M - 128K)
> +
> +def query_bitmaps(vm):
> +res = vm.qmp("query-block")
> +return {device['device']: device['dirty-bitmaps'] for
> +device in res['return']}
> +
> +with iotests.FilePath('img') as img_path, \
> + iotests.VM() as vm:
> +
> +log('--- Preparing image & VM ---\n')
> +iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, img_path, '64M')

no needs for qemu_img_pipe, as you don't use its output, qemu_img() is enough 
and qemu_img_create() is better,
and the best I think is just:

vm.add_drive_raw('id=drive0,driver=null-co,size=' + str(size))

as qcow2 (and real disk in general) is actually unrelated to the test.


> +vm.add_drive(img_path)
> +vm.launch()
> +
> +log('--- Adding preliminary bitmaps A & B ---\n')
> +vm.qmp_log("block-dirty-bitmap-add", node="drive0", name="bitmapA")
> +vm.qmp_log("block-dirty-bitmap-add", node="drive0", name="bitmapB")
> +
> +# Dirties 4 clusters. count=262144
> +log('')
> +log('--- Emulating writes ---\n')
> +for p in patterns:
> +cmd = "write -P%s %s %s" % p
> +log(cmd)
> +log(vm.hmp_qemu_io("drive0", cmd))
> +
> +log(json.dumps(query_bitmaps(vm), indent=2, separators=(',', ': ')))

log can do json.dumps, so it's strange to do it here, and I don't like ',' 
separator, as I described



-- 
Best regards,
Vladimir

Re: [Qemu-block] [PATCH v3 6/7] iotests: allow pretty-print for qmp_log

[Vladimir Sementsov-Ogievskiy]

17.12.2018 17:26, Vladimir Sementsov-Ogievskiy wrote:
> 15.12.2018 2:15, John Snow wrote:
>> If iotests have lines exceeding >998 characters long, git doesn't
>> want to send it plaintext to the list. We can solve this by allowing
>> the iotests to use pretty printed QMP output that we can match against
>> instead.
>>
>> As a bonus, it's much nicer for human eyes, too.
>>
>> Note that this changes the sort order for "command" and "arguments",
>> so I restrict this reordering to occur only when the indent is specified.
>> ---
>>   tests/qemu-iotests/iotests.py | 17 +
>>   1 file changed, 13 insertions(+), 4 deletions(-)
>>
>> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
>> index 9595429fea..ab5823c011 100644
>> --- a/tests/qemu-iotests/iotests.py
>> +++ b/tests/qemu-iotests/iotests.py
>> @@ -447,12 +447,21 @@ class VM(qtest.QEMUQtestMachine):
>>   result.append(filter_qmp_event(ev))
>>   return result
>> -    def qmp_log(self, cmd, filters=[filter_testfiles], **kwargs):
>> -    logmsg = '{"execute": "%s", "arguments": %s}' % \
>> -    (cmd, json.dumps(kwargs, sort_keys=True))
>> +    def qmp_log(self, cmd, indent=None, filters=[filter_testfiles], 
>> **kwargs):
>> +    # Python < 3.4 needs to know not to add whitespace when 
>> pretty-printing:
>> +    separators = (',', ': ') if indent is not None else (',', ': ')
>> +    if indent is not None:
>> +    fullcmd = { "execute": cmd,
>> +    "arguments": kwargs }
>> +    logmsg = json.dumps(fullcmd, indent=indent, 
>> separators=separators,
>> +    sort_keys=True)
>> +    else:
>> +    logmsg = '{"execute": "%s", "arguments": %s}' % \
>> +    (cmd, json.dumps(kwargs, sort_keys=True))
> 
> definitely overlogic on None/is not None, but anyway, ',' separator leads to 
> less
> beautiful output. So, I think it is better to do just "re.sub(r'\s+\n', '\n', 
> longmsg)"
> on final message, here or in log(), or in both.

find myself composing counter-proposal patch, will send soon...

> 
>>   log(logmsg, filters)
>>   result = self.qmp(cmd, **kwargs)
>> -    log(json.dumps(result, sort_keys=True), filters)
>> +    log(json.dumps(result, indent=indent, separators=separators,
>> +   sort_keys=True), filters)
>>   return result
>>   def run_job(self, job, auto_finalize=True, auto_dismiss=False):
>>
> 
> 


-- 
Best regards,
Vladimir

[Qemu-block] [QEMU-devel][PATCH v3] aio-posix: Fix concurrent aio_poll/set_fd_handler.

[remy . noel]

From: Remy Noel 

It is possible for an io_poll callback to be concurrently executed along
with an aio_set_fd_handlers. This can cause all sorts of problems, like
a NULL callback or a bad opaque pointer.

This changes set_fd_handlers so that it no longer modify existing handlers
entries and instead, always insert those after having proper initialisation.

Also, we do not call aio_epoll_update for deleted handlers as this has
no impact whatsoever.

Signed-off-by: Remy Noel 
---
 util/aio-posix.c | 86 
 util/aio-win32.c | 67 -
 2 files changed, 79 insertions(+), 74 deletions(-)

diff --git a/util/aio-posix.c b/util/aio-posix.c
index 51c41ed3c9..d658cf3007 100644
--- a/util/aio-posix.c
+++ b/util/aio-posix.c
@@ -200,6 +200,31 @@ static AioHandler *find_aio_handler(AioContext *ctx, int 
fd)
 return NULL;
 }
 
+static bool aio_remove_fd_handler(AioContext *ctx, AioHandler *node)
+{
+/* If the GSource is in the process of being destroyed then
+ * g_source_remove_poll() causes an assertion failure.  Skip
+ * removal in that case, because glib cleans up its state during
+ * destruction anyway.
+ */
+if (!g_source_is_destroyed(&ctx->source)) {
+g_source_remove_poll(&ctx->source, &node->pfd);
+}
+
+/* If a read is in progress, just mark the node as deleted */
+if (qemu_lockcnt_count(&ctx->list_lock)) {
+node->deleted = 1;
+node->pfd.revents = 0;
+return false;
+}
+/* Otherwise, delete it for real.  We can't just mark it as
+ * deleted because deleted nodes are only cleaned up while
+ * no one is walking the handlers list.
+ */
+QLIST_REMOVE(node, node);
+return true;
+}
+
 void aio_set_fd_handler(AioContext *ctx,
 int fd,
 bool is_external,
@@ -209,6 +234,7 @@ void aio_set_fd_handler(AioContext *ctx,
 void *opaque)
 {
 AioHandler *node;
+AioHandler *new_node = NULL;
 bool is_new = false;
 bool deleted = false;
 int poll_disable_change;
@@ -223,50 +249,36 @@ void aio_set_fd_handler(AioContext *ctx,
 qemu_lockcnt_unlock(&ctx->list_lock);
 return;
 }
-
-/* If the GSource is in the process of being destroyed then
- * g_source_remove_poll() causes an assertion failure.  Skip
- * removal in that case, because glib cleans up its state during
- * destruction anyway.
- */
-if (!g_source_is_destroyed(&ctx->source)) {
-g_source_remove_poll(&ctx->source, &node->pfd);
-}
-
-/* If a read is in progress, just mark the node as deleted */
-if (qemu_lockcnt_count(&ctx->list_lock)) {
-node->deleted = 1;
-node->pfd.revents = 0;
-} else {
-/* Otherwise, delete it for real.  We can't just mark it as
- * deleted because deleted nodes are only cleaned up while
- * no one is walking the handlers list.
- */
-QLIST_REMOVE(node, node);
-deleted = true;
-}
 poll_disable_change = -!node->io_poll;
 } else {
 poll_disable_change = !io_poll - (node && !node->io_poll);
 if (node == NULL) {
-/* Alloc and insert if it's not already there */
-node = g_new0(AioHandler, 1);
-node->pfd.fd = fd;
-QLIST_INSERT_HEAD_RCU(&ctx->aio_handlers, node, node);
-
-g_source_add_poll(&ctx->source, &node->pfd);
 is_new = true;
 }
+/* Alloc and insert if it's not already there */
+new_node = g_new0(AioHandler, 1);
 
 /* Update handler with latest information */
-node->io_read = io_read;
-node->io_write = io_write;
-node->io_poll = io_poll;
-node->opaque = opaque;
-node->is_external = is_external;
+new_node->io_read = io_read;
+new_node->io_write = io_write;
+new_node->io_poll = io_poll;
+new_node->opaque = opaque;
+new_node->is_external = is_external;
+
+if (is_new) {
+new_node->pfd.fd = fd;
+} else {
+new_node->pfd = node->pfd;
+}
+g_source_add_poll(&ctx->source, &new_node->pfd);
+
+new_node->pfd.events = (io_read ? G_IO_IN | G_IO_HUP | G_IO_ERR : 0);
+new_node->pfd.events |= (io_write ? G_IO_OUT | G_IO_ERR : 0);
 
-node->pfd.events = (io_read ? G_IO_IN | G_IO_HUP | G_IO_ERR : 0);
-node->pfd.events |= (io_write ? G_IO_OUT | G_IO_ERR : 0);
+QLIST_INSERT_HEAD_RCU(&ctx->aio_handlers, new_node, node);
+}
+if (node) {
+deleted = aio_remove_fd_handler(ctx, node);
 }
 
 /* No need to order poll_disable_cnt writes against other updates;
@@ -278,7 +290,9 @@ void aio_set_fd_handler(AioContext *ctx,
 atomic_set(&ctx->poll_disable_cnt,
atomic

[Qemu-block] [PATCH] iotests.py: rework log and qmp_log

[Vladimir Sementsov-Ogievskiy]

There are several problems to be solved:

1. log() may try to apply filters (which should be
   str -> str functions) to objects, which is not correct. Let's
   instead do json.dumps first and then apply filters.

2. Let's drop trailing whitespaces, if there are any. Hopefully, now
   only two tests have trailing whitespaces, both bash-based: 130, 181.

3. qmp_log() do json.dumps() separately and call log(), when log() can
   print objects too. It's obviously a duplication, let's instead pass
   objects from qmp_log to log

4. qmp_log may produce lines exceeding 998 characters, which is
   unfriendly to git send-email. To fix it, let's use pretty json
   output, by setting indent parameter of json.dumps. Note that with
   that parameter, dumps may produce lines with trailing whitespaces,
   so [2.] becomes very necessary.
   It may be a good thing (keeping in mind, that putting large objects
   to log() will lead to the same problem), to prettify all json in
   tests output, but it would be very large patch, hard to review. So,
   let's keep qmp_log_depr for old behavior and don't prettify log()
   output by default.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/206|  4 ++--
 tests/qemu-iotests/207|  2 +-
 tests/qemu-iotests/210| 12 ++--
 tests/qemu-iotests/211|  4 ++--
 tests/qemu-iotests/212|  4 ++--
 tests/qemu-iotests/213|  4 ++--
 tests/qemu-iotests/iotests.py | 28 
 7 files changed, 35 insertions(+), 23 deletions(-)

diff --git a/tests/qemu-iotests/206 b/tests/qemu-iotests/206
index 128c334c7c..cc5b5098ce 100755
--- a/tests/qemu-iotests/206
+++ b/tests/qemu-iotests/206
@@ -26,7 +26,7 @@ from iotests import imgfmt
 iotests.verify_image_format(supported_fmts=['qcow2'])
 
 def blockdev_create(vm, options):
-result = vm.qmp_log('blockdev-create', job_id='job0', options=options)
+result = vm.qmp_log_depr('blockdev-create', job_id='job0', options=options)
 
 if 'return' in result:
 assert result['return'] == {}
@@ -52,7 +52,7 @@ with iotests.FilePath('t.qcow2') as disk_path, \
   'filename': disk_path,
   'size': 0 })
 
-vm.qmp_log('blockdev-add', driver='file', filename=disk_path,
+vm.qmp_log_depr('blockdev-add', driver='file', filename=disk_path,
node_name='imgfile')
 
 blockdev_create(vm, { 'driver': imgfmt,
diff --git a/tests/qemu-iotests/207 b/tests/qemu-iotests/207
index c617ee7453..f5b74e5bfb 100755
--- a/tests/qemu-iotests/207
+++ b/tests/qemu-iotests/207
@@ -31,7 +31,7 @@ def filter_hash(msg):
 return re.sub('"hash": "[0-9a-f]+"', '"hash": HASH', msg)
 
 def blockdev_create(vm, options):
-result = vm.qmp_log('blockdev-create', job_id='job0', options=options,
+result = vm.qmp_log_depr('blockdev-create', job_id='job0', options=options,
 filters=[iotests.filter_testfiles, filter_hash])
 
 if 'return' in result:
diff --git a/tests/qemu-iotests/210 b/tests/qemu-iotests/210
index d142841e2b..b06232861e 100755
--- a/tests/qemu-iotests/210
+++ b/tests/qemu-iotests/210
@@ -27,7 +27,7 @@ iotests.verify_image_format(supported_fmts=['luks'])
 iotests.verify_protocol(supported=['file'])
 
 def blockdev_create(vm, options):
-result = vm.qmp_log('blockdev-create', job_id='job0', options=options)
+result = vm.qmp_log_depr('blockdev-create', job_id='job0', options=options)
 
 if 'return' in result:
 assert result['return'] == {}
@@ -52,7 +52,7 @@ with iotests.FilePath('t.luks') as disk_path, \
   'filename': disk_path,
   'size': 0 })
 
-vm.qmp_log('blockdev-add', driver='file', filename=disk_path,
+vm.qmp_log_depr('blockdev-add', driver='file', filename=disk_path,
node_name='imgfile')
 
 blockdev_create(vm, { 'driver': imgfmt,
@@ -170,10 +170,10 @@ with iotests.FilePath('t.luks') as disk_path, \
 
 
vm.add_blockdev('driver=luks,file=node0,key-secret=keysec0,node-name=node1')
 vm.launch()
-vm.qmp_log('block_resize', node_name='node1', size=9223372036854775296)
-vm.qmp_log('block_resize', node_name='node1', size=9223372036854775808)
-vm.qmp_log('block_resize', node_name='node1', size=18446744073709551104)
-vm.qmp_log('block_resize', node_name='node1', size=-9223372036854775808)
+vm.qmp_log_depr('block_resize', node_name='node1', 
size=9223372036854775296)
+vm.qmp_log_depr('block_resize', node_name='node1', 
size=9223372036854775808)
+vm.qmp_log_depr('block_resize', node_name='node1', 
size=18446744073709551104)
+vm.qmp_log_depr('block_resize', node_name='node1', 
size=-9223372036854775808)
 vm.shutdown()
 
 # TODO Proper support for images to be used with imgopts and/or protocols
diff --git a/tests/qemu-iotests/211 b/tests/qemu-iotests/211
index 7b7985db6c..c791499659 100755
--- a/tests/qemu-iotests/211
+++ b/tests/qemu-iotests/211
@@

Re: [Qemu-block] [PATCH v3 6/7] iotests: allow pretty-print for qmp_log

[John Snow]

On 12/17/18 11:37 AM, Vladimir Sementsov-Ogievskiy wrote:
> 17.12.2018 17:26, Vladimir Sementsov-Ogievskiy wrote:
>> 15.12.2018 2:15, John Snow wrote:
>>> If iotests have lines exceeding >998 characters long, git doesn't
>>> want to send it plaintext to the list. We can solve this by allowing
>>> the iotests to use pretty printed QMP output that we can match against
>>> instead.
>>>
>>> As a bonus, it's much nicer for human eyes, too.
>>>
>>> Note that this changes the sort order for "command" and "arguments",
>>> so I restrict this reordering to occur only when the indent is specified.
>>> ---
>>>   tests/qemu-iotests/iotests.py | 17 +
>>>   1 file changed, 13 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
>>> index 9595429fea..ab5823c011 100644
>>> --- a/tests/qemu-iotests/iotests.py
>>> +++ b/tests/qemu-iotests/iotests.py
>>> @@ -447,12 +447,21 @@ class VM(qtest.QEMUQtestMachine):
>>>   result.append(filter_qmp_event(ev))
>>>   return result
>>> -    def qmp_log(self, cmd, filters=[filter_testfiles], **kwargs):
>>> -    logmsg = '{"execute": "%s", "arguments": %s}' % \
>>> -    (cmd, json.dumps(kwargs, sort_keys=True))
>>> +    def qmp_log(self, cmd, indent=None, filters=[filter_testfiles], 
>>> **kwargs):
>>> +    # Python < 3.4 needs to know not to add whitespace when 
>>> pretty-printing:
>>> +    separators = (',', ': ') if indent is not None else (',', ': ')
>>> +    if indent is not None:
>>> +    fullcmd = { "execute": cmd,
>>> +    "arguments": kwargs }
>>> +    logmsg = json.dumps(fullcmd, indent=indent, 
>>> separators=separators,
>>> +    sort_keys=True)
>>> +    else:
>>> +    logmsg = '{"execute": "%s", "arguments": %s}' % \
>>> +    (cmd, json.dumps(kwargs, sort_keys=True))
>>
>> definitely overlogic on None/is not None, but anyway, ',' separator leads to 
>> less
>> beautiful output. So, I think it is better to do just "re.sub(r'\s+\n', 
>> '\n', longmsg)"
>> on final message, here or in log(), or in both.

I can't use "if indent" because 0 is a valid indentation, and I decided
to keep the "correct" order for execute/arguments if no indentation is
requested.

How is ',' less beautiful? When pretty-printing output ',' only occurs
at the end of a line anyway. I don't understand the critique.

> 
> find myself composing counter-proposal patch, will send soon...
> 
>>
>>>   log(logmsg, filters)
>>>   result = self.qmp(cmd, **kwargs)
>>> -    log(json.dumps(result, sort_keys=True), filters)
>>> +    log(json.dumps(result, indent=indent, separators=separators,
>>> +   sort_keys=True), filters)
>>>   return result
>>>   def run_job(self, job, auto_finalize=True, auto_dismiss=False):
>>>
>>
>>
> 
> 

Re: [Qemu-block] [Qemu-devel] [PATCH v3 0/7] bitmaps: remove x- prefix from QMP api

[John Snow]

On 12/14/18 6:15 PM, John Snow wrote:
> Touch up a few last things and remove the x- prefix.
> 
> V3:
>  - Reworked qmp_log to pretty-print the outgoing command, too [Vladimir]
>  - Modified test to log only bitmap information [Vladimir]
>  - Test disable/enable transaction toggle [Eric]
> 
> Note that the filter I added is now unused, but I think we will want it
> and it's small enough, so I'm going to check it in anyway. If you disagree,
> I'll just drop the patch instead.
> 
> --js
> 
> John Snow (7):
>   blockdev: abort transactions in reverse order
>   blockdev: n-ary bitmap merge
>   block: remove 'x' prefix from experimental bitmap APIs
>   iotests.py: don't abort if IMGKEYSECRET is undefined
>   iotests: add filter_generated_node_ids
>   iotests: allow pretty-print for qmp_log
>   iotests: add iotest 236 for testing bitmap merge
> 
>  blockdev.c|  96 +---
>  qapi/block-core.json  |  56 +-
>  qapi/transaction.json |  12 +-
>  tests/qemu-iotests/223|   4 +-
>  tests/qemu-iotests/236| 124 +
>  tests/qemu-iotests/236.out| 200 ++
>  tests/qemu-iotests/group  |   1 +
>  tests/qemu-iotests/iotests.py |  22 +++-
>  8 files changed, 436 insertions(+), 79 deletions(-)
>  create mode 100755 tests/qemu-iotests/236
>  create mode 100644 tests/qemu-iotests/236.out
> 

Thanks, I'm staging patches 1-5 and I'll send the PR once we get to the
bottom of patches 6 and 7, just to keep volume on the list down.

--js

Re: [Qemu-block] [PATCH v3 7/7] iotests: add iotest 236 for testing bitmap merge

[John Snow]

On 12/17/18 10:48 AM, Vladimir Sementsov-Ogievskiy wrote:
> 15.12.2018 2:15, John Snow wrote:
>> New interface, new smoke test.
>> ---
>>   tests/qemu-iotests/236 | 124 +++
>>   tests/qemu-iotests/236.out | 200 +
>>   tests/qemu-iotests/group   |   1 +
>>   3 files changed, 325 insertions(+)
>>   create mode 100755 tests/qemu-iotests/236
>>   create mode 100644 tests/qemu-iotests/236.out
>>
>> diff --git a/tests/qemu-iotests/236 b/tests/qemu-iotests/236
>> new file mode 100755
>> index 00..edf16c4ab1
>> --- /dev/null
>> +++ b/tests/qemu-iotests/236
>> @@ -0,0 +1,124 @@
>> +#!/usr/bin/env python
>> +#
>> +# Test bitmap merges.
>> +#
>> +# Copyright (c) 2018 John Snow for Red Hat, Inc.
>> +#
>> +# This program is free software; you can redistribute it and/or modify
>> +# it under the terms of the GNU General Public License as published by
>> +# the Free Software Foundation; either version 2 of the License, or
>> +# (at your option) any later version.
>> +#
>> +# This program is distributed in the hope that it will be useful,
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> +# GNU General Public License for more details.
>> +#
>> +# You should have received a copy of the GNU General Public License
>> +# along with this program.  If not, see .
>> +#
>> +# owner=js...@redhat.com
>> +
>> +import json
>> +import iotests
>> +from iotests import log
>> +
>> +iotests.verify_image_format(supported_fmts=['qcow2'])
>> +
> 
> we hardcode patterns, so it's better to hardcode size here too:
> 
> size = 64 * 1024 * 1024

If you insist.

> 
>> +patterns = [("0x5d", "0", "64k"),
>> +("0xd5", "1M","64k"),
>> +("0xdc", "32M",   "64k"),
>> +("0xcd", "0x3ff", "64k")]  # 64M - 64K
>> +
>> +overwrite = [("0xab", "0", "64k"), # Full overwrite
>> + ("0xad", "0x00f8000", "64k"), # Partial-left (1M-32K)
>> + ("0x1d", "0x2008000", "64k"), # Partial-right (32M+32K)
>> + ("0xea", "0x3fe", "64k")] # Adjacent-left (64M - 128K)
>> +
>> +def query_bitmaps(vm):
>> +res = vm.qmp("query-block")
>> +return {device['device']: device['dirty-bitmaps'] for
>> +device in res['return']}
>> +
>> +with iotests.FilePath('img') as img_path, \
>> + iotests.VM() as vm:
>> +
>> +log('--- Preparing image & VM ---\n')
>> +iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, img_path, '64M')
> 
> no needs for qemu_img_pipe, as you don't use its output, qemu_img() is enough 
> and qemu_img_create() is better,
> and the best I think is just:
> 

More cargo cult copy and paste.

> vm.add_drive_raw('id=drive0,driver=null-co,size=' + str(size))
> 
> as qcow2 (and real disk in general) is actually unrelated to the test.
> 

I think I'll leave the image creation in just so that if you run the
test under different formats it'll actually do something vaguely
different, just in case.

Actually, it did highlight how weird VPC is again and I changed the rest
as a result to accommodate image formats that round their disk sizes.

> 
>> +vm.add_drive(img_path)
>> +vm.launch()
>> +
>> +log('--- Adding preliminary bitmaps A & B ---\n')
>> +vm.qmp_log("block-dirty-bitmap-add", node="drive0", name="bitmapA")
>> +vm.qmp_log("block-dirty-bitmap-add", node="drive0", name="bitmapB")
>> +
>> +# Dirties 4 clusters. count=262144
>> +log('')
>> +log('--- Emulating writes ---\n')
>> +for p in patterns:
>> +cmd = "write -P%s %s %s" % p
>> +log(cmd)
>> +log(vm.hmp_qemu_io("drive0", cmd))
>> +
>> +log(json.dumps(query_bitmaps(vm), indent=2, separators=(',', ': ')))
> 
> log can do json.dumps, so it's strange to do it here, and I don't like ',' 
> separator, as I described

Because it tries to filter the rich object before it converts, as you've
noticed. I think I'll take a page from your book and try to fix log() to
be better.

As for not liking the ',' separator, it should be identical to your
approach because it only removes the space when pretty printing is
enabled. Can you show me what this approach does that you find to be
ugly and how it's different from your regex approach?

--js

[Qemu-block] [PATCH v12 00/31] block: Fix some filename generation issues

[Max Reitz]

Once more, I’ll spare both me and you another iteration of the cover
letter, so I direct you to the previous version’s cover letter (which
will direct you further):

http://lists.nongnu.org/archive/html/qemu-block/2018-10/msg00229.html

There are only minor changes in this version.  Provided no major
complaints arise, I intend to merge this series in the next month.


git-backport-diff against v11:

Key:
[] : patches are identical
[] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/31:[0015] [FC] 'block: Use bdrv_refresh_filename() to pull'
002/31:[] [--] 'block: Use children list in bdrv_refresh_filename'
003/31:[] [--] 'block: Skip implicit nodes for filename info'
004/31:[] [--] 'block: Add BDS.auto_backing_file'
005/31:[] [--] 'block: Respect backing bs in bdrv_refresh_filename'
006/31:[] [--] 'iotests.py: Add filter_imgfmt()'
007/31:[] [--] 'iotests.py: Add node_info()'
008/31:[0072] [FC] 'iotests: Add test for backing file overrides'
009/31:[] [--] 'block: Make path_combine() return the path'
010/31:[] [-C] 'block: bdrv_get_full_backing_filename_from_...'s ret. val.'
011/31:[] [--] 'block: bdrv_get_full_backing_filename's ret. val.'
012/31:[] [--] 'block: Add bdrv_make_absolute_filename()'
013/31:[] [--] 'block: Fix bdrv_find_backing_image()'
014/31:[] [--] 'block: Add bdrv_dirname()'
015/31:[] [--] 'blkverify: Make bdrv_dirname() return NULL'
016/31:[] [--] 'quorum: Make bdrv_dirname() return NULL'
017/31:[] [--] 'block/nbd: Make bdrv_dirname() return NULL'
018/31:[] [--] 'block/nfs: Implement bdrv_dirname()'
019/31:[] [--] 'block: Use bdrv_dirname() for relative filenames'
020/31:[] [--] 'iotests: Add quorum case to test 110'
021/31:[] [--] 'block: Add strong_runtime_opts to BlockDriver'
022/31:[] [--] 'block: Add BlockDriver.bdrv_gather_child_options'
023/31:[] [--] 'block: Generically refresh runtime options'
024/31:[] [--] 'block: Purify .bdrv_refresh_filename()'
025/31:[] [--] 'block: Do not copy exact_filename from format file'
026/31:[] [--] 'block/nvme: Fix bdrv_refresh_filename()'
027/31:[] [--] 'block/curl: Harmonize option defaults'
028/31:[] [--] 'block/curl: Implement bdrv_refresh_filename()'
029/31:[] [--] 'block/null: Generate filename even with latency-ns'
030/31:[] [--] 'block: BDS options may lack the "driver" option'
031/31:[0024] [FC] 'iotests: Test json:{} filenames of internal BDSs'


v12:
- Patch 1:
  - Due to rebase, one hunk was dropped (vpc has one instance of
"bs->filename" less)
  - I grepped through everything ("->filename") again and noticed I did
not refresh the filename before qemu-img.c's accesses to
bs->filename.  Now it probably isn't necessary to do this (because
we don't use complex graph features there, and even more so no graph
modifications can occur at runtime), but then again I believe it
doesn't hurt to refresh the filename even here.
So there are four additional hunks in qemu-img.c now.
- Patches 8 and 31:
  Had to adjust the reference output due to the Python2/3 compatibility
  patches (kept the R-b, because the changes are obvious, I believe)


Max Reitz (31):
  block: Use bdrv_refresh_filename() to pull
  block: Use children list in bdrv_refresh_filename
  block: Skip implicit nodes for filename info
  block: Add BDS.auto_backing_file
  block: Respect backing bs in bdrv_refresh_filename
  iotests.py: Add filter_imgfmt()
  iotests.py: Add node_info()
  iotests: Add test for backing file overrides
  block: Make path_combine() return the path
  block: bdrv_get_full_backing_filename_from_...'s ret. val.
  block: bdrv_get_full_backing_filename's ret. val.
  block: Add bdrv_make_absolute_filename()
  block: Fix bdrv_find_backing_image()
  block: Add bdrv_dirname()
  blkverify: Make bdrv_dirname() return NULL
  quorum: Make bdrv_dirname() return NULL
  block/nbd: Make bdrv_dirname() return NULL
  block/nfs: Implement bdrv_dirname()
  block: Use bdrv_dirname() for relative filenames
  iotests: Add quorum case to test 110
  block: Add strong_runtime_opts to BlockDriver
  block: Add BlockDriver.bdrv_gather_child_options
  block: Generically refresh runtime options
  block: Purify .bdrv_refresh_filename()
  block: Do not copy exact_filename from format file
  block/nvme: Fix bdrv_refresh_filename()
  block/curl: Harmonize option defaults
  block/curl: Implement bdrv_refresh_filename()
  block/null: Generate filename even with latency-ns
  block: BDS options may lack the "driver" option
  iotests: Test json:{} filenames of internal BDSs

 include/block/block.h |  16 +-
 include/block/block_int.h |  48 +++-
 block.c   | 503 ++
 block/blkdebug.c  |  70 ++---
 block/blklogwrites.c  |  34 +--
 block/blkve

[Qemu-block] [PATCH v12 04/31] block: Add BDS.auto_backing_file

[Max Reitz]

If the backing file is overridden, this most probably does change the
guest-visible data of a BDS.  Therefore, we will need to consider this
in bdrv_refresh_filename().

To see whether it has been overridden, we might want to compare
bs->backing_file and bs->backing->bs->filename.  However,
bs->backing_file is changed by bdrv_set_backing_hd() (which is just used
to change the backing child at runtime, without modifying the image
header), so bs->backing_file most of the time simply contains a copy of
bs->backing->bs->filename anyway, so it is useless for such a
comparison.

This patch adds an auto_backing_file BDS field which contains the
backing file path as indicated by the image header, which is not changed
by bdrv_set_backing_hd().

Because of bdrv_refresh_filename() magic, however, a BDS's filename may
differ from what has been specified during bdrv_open().  Then, the
comparison between bs->auto_backing_file and bs->backing->bs->filename
may fail even though bs->backing was opened from bs->auto_backing_file.
To mitigate this, we can copy the real BDS's filename (after the whole
bdrv_open() and bdrv_refresh_filename() process) into
bs->auto_backing_file, if we know the former has been opened based on
the latter.  This is only possible if no options modifying the backing
file's behavior have been specified, though.  To simplify things, this
patch only copies the filename from the backing file if no options have
been specified for it at all.

Furthermore, there are cases where an overlay is created by qemu which
already contains a BDS's filename (e.g. in blockdev-snapshot-sync).  We
do not need to worry about updating the overlay's bs->auto_backing_file
there, because we actually wrote a post-bdrv_refresh_filename() filename
into the image header.

So all in all, there will be false negatives where (as of a future
patch) bdrv_refresh_filename() will assume that the backing file differs
from what was specified in the image header, even though it really does
not.  However, these cases should be limited to where (1) the user
actually did override something in the backing chain (e.g. by specifying
options for the backing file), or (2) the user executed a QMP command to
change some node's backing file (e.g. change-backing-file or
block-commit with @backing-file given) where the given filename does not
happen to coincide with qemu's idea of the backing BDS's filename.

Then again, (1) really is limited to -drive.  With -blockdev or
blockdev-add, you have to adhere to the schema, so a user cannot give
partial "unimportant" options (e.g. by just setting backing.node-name
and leaving the rest to the image header).  Therefore, trying to fix
this would mean trying to fix something for -drive only.

To improve on (2), we would need a full infrastructure to "canonicalize"
an arbitrary filename (+ options), so it can be compared against
another.  That seems a bit over the top, considering that filenames
nowadays are there mostly for the user's entertainment.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
---
 include/block/block_int.h |  4 
 block.c   | 19 +++
 block/qcow.c  |  7 +--
 block/qcow2.c | 10 +++---
 block/qed.c   |  7 +--
 block/vmdk.c  |  6 --
 6 files changed, 44 insertions(+), 9 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index f605622216..93cd669a35 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -696,6 +696,10 @@ struct BlockDriverState {
 char filename[PATH_MAX];
 char backing_file[PATH_MAX]; /* if non zero, the image is a diff of
 this file image */
+/* The backing filename indicated by the image header; if we ever
+ * open this file, then this is replaced by the resulting BDS's
+ * filename (i.e. after a bdrv_refresh_filename() run). */
+char auto_backing_file[PATH_MAX];
 char backing_format[16]; /* if non-zero and backing_file exists */
 
 QDict *full_open_options;
diff --git a/block.c b/block.c
index 589d43e18a..c384355073 100644
--- a/block.c
+++ b/block.c
@@ -2330,6 +2330,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict 
*parent_options,
 char *bdref_key_dot;
 const char *reference = NULL;
 int ret = 0;
+bool implicit_backing = false;
 BlockDriverState *backing_hd;
 QDict *options;
 QDict *tmp_parent_options = NULL;
@@ -2365,6 +2366,16 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict 
*parent_options,
 qobject_unref(options);
 goto free_exit;
 } else {
+if (qdict_size(options) == 0) {
+/* If the user specifies options that do not modify the
+ * backing file's behavior, we might still consider it the
+ * implicit backing file.  But it's easier this way, and
+ * just specifying some of the backing BDS's options is
+

[Qemu-block] [PATCH v12 03/31] block: Skip implicit nodes for filename info

[Max Reitz]

bdrv_refresh_filename() should simply skip all implicit nodes.  They are
supposed to be invisible to the user, so they should not appear in
filename information.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
---
 block.c | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/block.c b/block.c
index 6c6ccb1749..589d43e18a 100644
--- a/block.c
+++ b/block.c
@@ -5331,6 +5331,20 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 bdrv_refresh_filename(child->bs);
 }
 
+if (bs->implicit) {
+/* For implicit nodes, just copy everything from the single child */
+child = QLIST_FIRST(&bs->children);
+assert(QLIST_NEXT(child, next) == NULL);
+
+pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
+child->bs->exact_filename);
+pstrcpy(bs->filename, sizeof(bs->filename), child->bs->filename);
+
+bs->full_open_options = qobject_ref(child->bs->full_open_options);
+
+return;
+}
+
 if (drv->bdrv_refresh_filename) {
 /* Obsolete information is of no use here, so drop the old file name
  * information before refreshing it */
-- 
2.19.2

[Qemu-block] [PATCH v12 01/31] block: Use bdrv_refresh_filename() to pull

[Max Reitz]

Before this patch, bdrv_refresh_filename() is used in a pushing manner:
Whenever the BDS graph is modified, the parents of the modified edges
are supposed to be updated (recursively upwards).  However, that is
nonviable, considering that we want child changes not to concern
parents.

Also, in the long run we want a pull model anyway: Here, we would have a
bdrv_filename() function which returns a BDS's filename, freshly
constructed.

This patch is an intermediate step.  It adds bdrv_refresh_filename()
calls before every place a BDS.filename value is used.  The only
exceptions are protocol drivers that use their own filename, which
clearly would not profit from refreshing that filename before.

Also, bdrv_get_encrypted_filename() is removed along the way (as a user
of BDS.filename), since it is completely unused.

In turn, all of the calls to bdrv_refresh_filename() before this patch
are removed, because we no longer have to call this function on graph
changes.

Signed-off-by: Max Reitz 
---
 include/block/block.h |  1 -
 block.c   | 31 +++
 block/qapi.c  |  4 
 block/raw-format.c|  1 +
 block/replication.c   |  2 --
 block/vhdx-log.c  |  1 +
 block/vmdk.c  |  6 ++
 blockdev.c|  8 
 qemu-img.c| 11 +--
 9 files changed, 44 insertions(+), 21 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index f70a843b72..1e54c8f505 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -483,7 +483,6 @@ void bdrv_round_to_clusters(BlockDriverState *bs,
 int64_t *cluster_offset,
 int64_t *cluster_bytes);
 
-const char *bdrv_get_encrypted_filename(BlockDriverState *bs);
 void bdrv_get_backing_filename(BlockDriverState *bs,
char *filename, int filename_size);
 void bdrv_get_full_backing_filename(BlockDriverState *bs,
diff --git a/block.c b/block.c
index 4f5ff2cc12..2f29c56536 100644
--- a/block.c
+++ b/block.c
@@ -323,8 +323,11 @@ void bdrv_get_full_backing_filename_from_filename(const 
char *backed,
 void bdrv_get_full_backing_filename(BlockDriverState *bs, char *dest, size_t 
sz,
 Error **errp)
 {
-char *backed = bs->exact_filename[0] ? bs->exact_filename : bs->filename;
+char *backed;
 
+bdrv_refresh_filename(bs);
+
+backed = bs->exact_filename[0] ? bs->exact_filename : bs->filename;
 bdrv_get_full_backing_filename_from_filename(backed, bs->backing_file,
  dest, sz, errp);
 }
@@ -1004,6 +1007,8 @@ static void bdrv_backing_attach(BdrvChild *c)
"node is used as backing hd of '%s'",
bdrv_get_device_or_node_name(parent));
 
+bdrv_refresh_filename(backing_hd);
+
 parent->open_flags &= ~BDRV_O_NO_BACKING;
 pstrcpy(parent->backing_file, sizeof(parent->backing_file),
 backing_hd->filename);
@@ -1413,6 +1418,7 @@ static int bdrv_open_common(BlockDriverState *bs, 
BlockBackend *file,
 }
 
 if (file != NULL) {
+bdrv_refresh_filename(blk_bs(file));
 filename = blk_bs(file)->filename;
 } else {
 /*
@@ -2303,8 +2309,6 @@ void bdrv_set_backing_hd(BlockDriverState *bs, 
BlockDriverState *backing_hd,
 bdrv_unref(backing_hd);
 }
 
-bdrv_refresh_filename(bs);
-
 out:
 bdrv_refresh_limits(bs, NULL);
 }
@@ -2833,8 +2837,6 @@ static BlockDriverState *bdrv_open_inherit(const char 
*filename,
 g_free(child_key_dot);
 }
 
-bdrv_refresh_filename(bs);
-
 /* Check if any unknown options were used */
 if (qdict_size(options) != 0) {
 const QDictEntry *entry = qdict_first(options);
@@ -3279,6 +3281,7 @@ int bdrv_reopen_prepare(BDRVReopenState *reopen_state, 
BlockReopenQueue *queue,
 if (local_err != NULL) {
 error_propagate(errp, local_err);
 } else {
+bdrv_refresh_filename(reopen_state->bs);
 error_setg(errp, "failed while preparing to reopen image '%s'",
reopen_state->bs->filename);
 }
@@ -3874,7 +3877,10 @@ int bdrv_drop_intermediate(BlockDriverState *top, 
BlockDriverState *base,
 /* success - we can delete the intermediate states, and link top->base */
 /* TODO Check graph modification op blockers (BLK_PERM_GRAPH_MOD) once
  * we've figured out how they should work. */
-backing_file_str = backing_file_str ? backing_file_str : base->filename;
+if (!backing_file_str) {
+bdrv_refresh_filename(base);
+backing_file_str = base->filename;
+}
 
 QLIST_FOREACH_SAFE(c, &top->parents, next_parent, next) {
 /* Check whether we are allowed to switch c from top to base */
@@ -4274,16 +4280,6 @@ bool bdrv_can_write_zeroes_with_unmap(BlockDriverState 
*bs)
 return bs->supported_zero_flags & BDRV_REQ_MAY_UNMAP;
 }
 
-const cha

[Qemu-block] [PATCH v12 02/31] block: Use children list in bdrv_refresh_filename

[Max Reitz]

bdrv_refresh_filename() should invoke itself recursively on all
children, not just on file.

With that change, we can remove the manual invocations in blkverify,
quorum, commit, mirror, and blklogwrites.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
---
 block.c  | 9 +
 block/blklogwrites.c | 3 ---
 block/blkverify.c| 3 ---
 block/commit.c   | 1 -
 block/mirror.c   | 1 -
 block/quorum.c   | 1 -
 6 files changed, 5 insertions(+), 13 deletions(-)

diff --git a/block.c b/block.c
index 2f29c56536..6c6ccb1749 100644
--- a/block.c
+++ b/block.c
@@ -5318,16 +5318,17 @@ static bool append_open_options(QDict *d, 
BlockDriverState *bs)
 void bdrv_refresh_filename(BlockDriverState *bs)
 {
 BlockDriver *drv = bs->drv;
+BdrvChild *child;
 QDict *opts;
 
 if (!drv) {
 return;
 }
 
-/* This BDS's file name will most probably depend on its file's name, so
- * refresh that first */
-if (bs->file) {
-bdrv_refresh_filename(bs->file->bs);
+/* This BDS's file name may depend on any of its children's file names, so
+ * refresh those first */
+QLIST_FOREACH(child, &bs->children, next) {
+bdrv_refresh_filename(child->bs);
 }
 
 if (drv->bdrv_refresh_filename) {
diff --git a/block/blklogwrites.c b/block/blklogwrites.c
index ff98cd5533..8f1b589d54 100644
--- a/block/blklogwrites.c
+++ b/block/blklogwrites.c
@@ -285,9 +285,6 @@ static void 
blk_log_writes_refresh_filename(BlockDriverState *bs,
 {
 BDRVBlkLogWritesState *s = bs->opaque;
 
-/* bs->file->bs has already been refreshed */
-bdrv_refresh_filename(s->log_file->bs);
-
 if (bs->file->bs->full_open_options
 && s->log_file->bs->full_open_options)
 {
diff --git a/block/blkverify.c b/block/blkverify.c
index 89bf4386e3..035d77b64a 100644
--- a/block/blkverify.c
+++ b/block/blkverify.c
@@ -285,9 +285,6 @@ static void blkverify_refresh_filename(BlockDriverState 
*bs, QDict *options)
 {
 BDRVBlkverifyState *s = bs->opaque;
 
-/* bs->file->bs has already been refreshed */
-bdrv_refresh_filename(s->test_file->bs);
-
 if (bs->file->bs->full_open_options
 && s->test_file->bs->full_open_options)
 {
diff --git a/block/commit.c b/block/commit.c
index 53148e610b..093b1505de 100644
--- a/block/commit.c
+++ b/block/commit.c
@@ -232,7 +232,6 @@ static int coroutine_fn 
bdrv_commit_top_preadv(BlockDriverState *bs,
 
 static void bdrv_commit_top_refresh_filename(BlockDriverState *bs, QDict *opts)
 {
-bdrv_refresh_filename(bs->backing->bs);
 pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
 bs->backing->bs->filename);
 }
diff --git a/block/mirror.c b/block/mirror.c
index ab59ad77e8..c965b2d3e1 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -1439,7 +1439,6 @@ static void 
bdrv_mirror_top_refresh_filename(BlockDriverState *bs, QDict *opts)
  * bdrv_set_backing_hd */
 return;
 }
-bdrv_refresh_filename(bs->backing->bs);
 pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
 bs->backing->bs->filename);
 }
diff --git a/block/quorum.c b/block/quorum.c
index 16b3c8067c..cf9d7c16c2 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -1073,7 +1073,6 @@ static void quorum_refresh_filename(BlockDriverState *bs, 
QDict *options)
 int i;
 
 for (i = 0; i < s->num_children; i++) {
-bdrv_refresh_filename(s->children[i]->bs);
 if (!s->children[i]->bs->full_open_options) {
 return;
 }
-- 
2.19.2

[Qemu-block] [PATCH v12 05/31] block: Respect backing bs in bdrv_refresh_filename

[Max Reitz]

Basically, bdrv_refresh_filename() should respect all children of a
BlockDriverState. However, generally those children are driver-specific,
so this function cannot handle the general case. On the other hand,
there are only few drivers which use other children than @file and
@backing (that being vmdk, quorum, and blkverify).

Most block drivers only use @file and/or @backing (if they use any
children at all). Both can be implemented directly in
bdrv_refresh_filename.

The user overriding the file's filename is already handled, however, the
user overriding the backing file is not. If this is done, opening the
BDS with the plain filename of its file will not be correct, so we may
not set bs->exact_filename in that case.

iotest 051 contains test cases for overriding the backing file, and so
its output changes with this patch applied.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 block.c   | 38 ++-
 tests/qemu-iotests/051.out|  8 
 tests/qemu-iotests/051.pc.out |  8 
 3 files changed, 45 insertions(+), 9 deletions(-)

diff --git a/block.c b/block.c
index c384355073..22f91fb3ee 100644
--- a/block.c
+++ b/block.c
@@ -5322,6 +5322,21 @@ static bool append_open_options(QDict *d, 
BlockDriverState *bs)
 return found_any;
 }
 
+/* Note: This function may return false positives; it may return true
+ * even if opening the backing file specified by bs's image header
+ * would result in exactly bs->backing. */
+static bool bdrv_backing_overridden(BlockDriverState *bs)
+{
+if (bs->backing) {
+return strcmp(bs->auto_backing_file,
+  bs->backing->bs->filename);
+} else {
+/* No backing BDS, so if the image header reports any backing
+ * file, it must have been suppressed */
+return bs->auto_backing_file[0] != '\0';
+}
+}
+
 /* Updates the following BDS fields:
  *  - exact_filename: A filename which may be used for opening a block device
  *which (mostly) equals the given BDS (even without any
@@ -5339,6 +5354,7 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 BlockDriver *drv = bs->drv;
 BdrvChild *child;
 QDict *opts;
+bool backing_overridden;
 
 if (!drv) {
 return;
@@ -5364,6 +5380,16 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 return;
 }
 
+backing_overridden = bdrv_backing_overridden(bs);
+
+if (bs->open_flags & BDRV_O_NO_IO) {
+/* Without I/O, the backing file does not change anything.
+ * Therefore, in such a case (primarily qemu-img), we can
+ * pretend the backing file has not been overridden even if
+ * it technically has been. */
+backing_overridden = false;
+}
+
 if (drv->bdrv_refresh_filename) {
 /* Obsolete information is of no use here, so drop the old file name
  * information before refreshing it */
@@ -5389,6 +5415,7 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 
 opts = qdict_new();
 has_open_options = append_open_options(opts, bs);
+has_open_options |= backing_overridden;
 
 /* If no specific options have been given for this BDS, the filename of
  * the underlying file should suffice for this one as well */
@@ -5400,11 +5427,20 @@ void bdrv_refresh_filename(BlockDriverState *bs)
  * file BDS. The full options QDict of that file BDS should somehow
  * contain a representation of the filename, therefore the following
  * suffices without querying the (exact_)filename of this BDS. */
-if (bs->file->bs->full_open_options) {
+if (bs->file->bs->full_open_options &&
+(!bs->backing || bs->backing->bs->full_open_options))
+{
 qdict_put_str(opts, "driver", drv->format_name);
 qdict_put(opts, "file",
   qobject_ref(bs->file->bs->full_open_options));
 
+if (bs->backing) {
+qdict_put(opts, "backing",
+  qobject_ref(bs->backing->bs->full_open_options));
+} else if (backing_overridden) {
+qdict_put_null(opts, "backing");
+}
+
 bs->full_open_options = opts;
 } else {
 qobject_unref(opts);
diff --git a/tests/qemu-iotests/051.out b/tests/qemu-iotests/051.out
index 793af2ab96..b900935fbc 100644
--- a/tests/qemu-iotests/051.out
+++ b/tests/qemu-iotests/051.out
@@ -82,7 +82,7 @@ QEMU X.Y.Z monitor - type 'help' for more information
 Testing: -drive 
file=TEST_DIR/t.qcow2,driver=qcow2,backing.file.filename=TEST_DIR/t.qcow2.orig,if=none,id=drive0
 -nodefaults
 QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) info block
-drive0 (NODE_NAME): TEST_DIR/t.qcow2 (qcow2)
+drive0 (NODE_NAME): json:{"backing": {"driver": "qcow2", "file": {"driver": 
"file", "filename": "TEST_DIR/t.qcow2.orig"}}, "driver": "qcow2", "file": 
{"driver": "file", "filename": "T

[Qemu-block] [PATCH v12 07/31] iotests.py: Add node_info()

[Max Reitz]

This function queries a node; since we cannot do that right now, it
executes query-named-block-nodes and returns the matching node's object.

Signed-off-by: Max Reitz 
Reviewed-by: John Snow 
Reviewed-by: Alberto Garcia 
Reviewed-by: Eric Blake 
---
 tests/qemu-iotests/iotests.py | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 0c1b8ad272..ad9e9abad1 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -474,6 +474,13 @@ class VM(qtest.QEMUQtestMachine):
 else:
 iotests.log(ev)
 
+def node_info(self, node_name):
+nodes = self.qmp('query-named-block-nodes')
+for x in nodes['return']:
+if x['node-name'] == node_name:
+return x
+return None
+
 
 index_re = re.compile(r'([^\[]+)\[([^\]]+)\]')
 
-- 
2.19.2

[Qemu-block] [PATCH v12 06/31] iotests.py: Add filter_imgfmt()

[Max Reitz]

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
---
 tests/qemu-iotests/iotests.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index d537538ba0..0c1b8ad272 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -251,6 +251,9 @@ def filter_img_info(output, filename):
 lines.append(line)
 return '\n'.join(lines)
 
+def filter_imgfmt(msg):
+return msg.replace(imgfmt, 'IMGFMT')
+
 def log(msg, filters=[]):
 for flt in filters:
 msg = flt(msg)
-- 
2.19.2

[Qemu-block] [PATCH v12 08/31] iotests: Add test for backing file overrides

[Max Reitz]

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 tests/qemu-iotests/228 | 235 +
 tests/qemu-iotests/228.out |  84 +
 tests/qemu-iotests/group   |   1 +
 3 files changed, 320 insertions(+)
 create mode 100755 tests/qemu-iotests/228
 create mode 100644 tests/qemu-iotests/228.out

diff --git a/tests/qemu-iotests/228 b/tests/qemu-iotests/228
new file mode 100755
index 00..a2200efba5
--- /dev/null
+++ b/tests/qemu-iotests/228
@@ -0,0 +1,235 @@
+#!/usr/bin/env python
+#
+# Test for when a backing file is considered overridden (thus, a
+# json:{} filename is generated for the overlay) and when it is not
+#
+# Copyright (C) 2018 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+# Creator/Owner: Max Reitz 
+
+import iotests
+from iotests import log, qemu_img, filter_testfiles, filter_imgfmt
+
+# Need backing file and change-backing-file support
+iotests.verify_image_format(supported_fmts=['qcow2', 'qed'])
+iotests.verify_platform(['linux'])
+
+
+def log_node_info(node):
+log('')
+
+log('bs->filename: ' + node['image']['filename'],
+filters=[filter_testfiles, filter_imgfmt])
+log('bs->backing_file: ' + node['backing_file'],
+filters=[filter_testfiles, filter_imgfmt])
+
+if 'backing-image' in node['image']:
+log('bs->backing->bs->filename: ' +
+node['image']['backing-image']['filename'],
+filters=[filter_testfiles, filter_imgfmt])
+else:
+log('bs->backing: (none)')
+
+log('')
+
+
+with iotests.FilePath('base.img') as base_img_path, \
+ iotests.FilePath('top.img') as top_img_path, \
+ iotests.VM() as vm:
+
+assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0
+# Choose a funny way to describe the backing filename
+assert qemu_img('create', '-f', iotests.imgfmt, '-b',
+'file:' + base_img_path, top_img_path) == 0
+
+vm.launch()
+
+log('--- Implicit backing file ---')
+log('')
+
+vm.qmp_log('blockdev-add',
+node_name='node0',
+driver=iotests.imgfmt,
+file={
+'driver': 'file',
+'filename': top_img_path
+},
+filters=[filter_testfiles, filter_imgfmt])
+
+# Filename should be plain, and the backing filename should not
+# contain the "file:" prefix
+log_node_info(vm.node_info('node0'))
+
+vm.qmp_log('blockdev-del', node_name='node0')
+
+log('')
+log('--- change-backing-file ---')
+log('')
+
+vm.qmp_log('blockdev-add',
+   node_name='node0',
+   driver=iotests.imgfmt,
+   file={
+   'driver': 'file',
+   'filename': top_img_path
+   },
+   filters=[filter_testfiles, filter_imgfmt])
+
+# Changing the backing file to a qemu-reported filename should
+# result in qemu accepting the corresponding BDS as the implicit
+# backing BDS (and thus not generate a json:{} filename).
+# So, first, query the backing filename.
+
+backing_filename = \
+vm.node_info('node0')['image']['backing-image']['filename']
+
+# Next, change the backing file to something different
+
+vm.qmp_log('change-backing-file',
+   image_node_name='node0',
+   device='node0',
+   backing_file='null-co://')
+
+# Now, verify that we get a json:{} filename
+# (Image header says "null-co://", actual backing file still is
+# base_img_path)
+
+log_node_info(vm.node_info('node0'))
+
+# Change it back
+# (To get header and backing file in sync)
+
+vm.qmp_log('change-backing-file',
+   image_node_name='node0',
+   device='node0',
+   backing_file=backing_filename)
+
+# And verify that we get our original results
+
+log_node_info(vm.node_info('node0'))
+
+# Finally, try a "file:" prefix.  While this is actually what we
+# originally had in the image header, qemu will not reopen the
+# backing file here, so it cannot verify that this filename
+# "resolves" to the actual backing BDS's filename and will thus
+# consider both to be different.
+# (This may be fixed in the future.)
+
+vm.qmp_log('change-backing-file',
+   image_node_n

[Qemu-block] [PATCH v12 10/31] block: bdrv_get_full_backing_filename_from_...'s ret. val.

[Max Reitz]

Make bdrv_get_full_backing_filename_from_filename() return an allocated
string instead of placing the result in a caller-provided buffer.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 include/block/block.h |  7 +++---
 block.c   | 53 ++-
 block/vmdk.c  |  9 
 qemu-img.c| 12 --
 4 files changed, 48 insertions(+), 33 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index c69b08bef9..9dd573351b 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -487,10 +487,9 @@ void bdrv_get_backing_filename(BlockDriverState *bs,
char *filename, int filename_size);
 void bdrv_get_full_backing_filename(BlockDriverState *bs,
 char *dest, size_t sz, Error **errp);
-void bdrv_get_full_backing_filename_from_filename(const char *backed,
-  const char *backing,
-  char *dest, size_t sz,
-  Error **errp);
+char *bdrv_get_full_backing_filename_from_filename(const char *backed,
+   const char *backing,
+   Error **errp);
 
 int path_has_protocol(const char *path);
 int path_is_absolute(const char *path);
diff --git a/block.c b/block.c
index 933a43c989..0cd4412725 100644
--- a/block.c
+++ b/block.c
@@ -312,20 +312,29 @@ fail:
 return -EACCES;
 }
 
-void bdrv_get_full_backing_filename_from_filename(const char *backed,
-  const char *backing,
-  char *dest, size_t sz,
-  Error **errp)
+/*
+ * If @backing is empty, this function returns NULL without setting
+ * @errp.  In all other cases, NULL will only be returned with @errp
+ * set.
+ *
+ * Therefore, a return value of NULL without @errp set means that
+ * there is no backing file; if @errp is set, there is one but its
+ * absolute filename cannot be generated.
+ */
+char *bdrv_get_full_backing_filename_from_filename(const char *backed,
+   const char *backing,
+   Error **errp)
 {
-if (backing[0] == '\0' || path_has_protocol(backing) ||
-path_is_absolute(backing))
-{
-pstrcpy(dest, sz, backing);
+if (backing[0] == '\0') {
+return NULL;
+} else if (path_has_protocol(backing) || path_is_absolute(backing)) {
+return g_strdup(backing);
 } else if (backed[0] == '\0' || strstart(backed, "json:", NULL)) {
 error_setg(errp, "Cannot use relative backing file names for '%s'",
backed);
+return NULL;
 } else {
-path_combine_deprecated(dest, sz, backed, backing);
+return path_combine(backed, backing);
 }
 }
 
@@ -333,12 +342,24 @@ void bdrv_get_full_backing_filename(BlockDriverState *bs, 
char *dest, size_t sz,
 Error **errp)
 {
 char *backed;
+char *full_name;
+Error *local_error = NULL;
 
 bdrv_refresh_filename(bs);
 
 backed = bs->exact_filename[0] ? bs->exact_filename : bs->filename;
-bdrv_get_full_backing_filename_from_filename(backed, bs->backing_file,
- dest, sz, errp);
+
+full_name = bdrv_get_full_backing_filename_from_filename(backed,
+ bs->backing_file,
+ &local_error);
+if (full_name) {
+pstrcpy(dest, sz, full_name);
+g_free(full_name);
+} else if (local_error) {
+error_propagate(errp, local_error);
+} else if (sz > 0) {
+*dest = '\0';
+}
 }
 
 void bdrv_register(BlockDriver *bdrv)
@@ -4965,17 +4986,17 @@ void bdrv_img_create(const char *filename, const char 
*fmt,
 size = qemu_opt_get_size(opts, BLOCK_OPT_SIZE, img_size);
 if (backing_file && !(flags & BDRV_O_NO_BACKING)) {
 BlockDriverState *bs;
-char *full_backing = g_new0(char, PATH_MAX);
+char *full_backing;
 int back_flags;
 QDict *backing_options = NULL;
 
-bdrv_get_full_backing_filename_from_filename(filename, backing_file,
- full_backing, PATH_MAX,
- &local_err);
+full_backing =
+bdrv_get_full_backing_filename_from_filename(filename, 
backing_file,
+ &local_err);
 if (local_err) {
-g_free(full_backing);
 goto out;
 }
+assert(full_backing);
 
 /* backing files always opened read

[Qemu-block] [PATCH v12 11/31] block: bdrv_get_full_backing_filename's ret. val.

[Max Reitz]

Make bdrv_get_full_backing_filename() return an allocated string instead
of placing the result in a caller-provided buffer.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 include/block/block.h |  3 +--
 block.c   | 48 +++
 block/qapi.c  | 12 ++-
 3 files changed, 20 insertions(+), 43 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index 9dd573351b..a1e1b59e9c 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -485,8 +485,7 @@ void bdrv_round_to_clusters(BlockDriverState *bs,
 
 void bdrv_get_backing_filename(BlockDriverState *bs,
char *filename, int filename_size);
-void bdrv_get_full_backing_filename(BlockDriverState *bs,
-char *dest, size_t sz, Error **errp);
+char *bdrv_get_full_backing_filename(BlockDriverState *bs, Error **errp);
 char *bdrv_get_full_backing_filename_from_filename(const char *backed,
const char *backing,
Error **errp);
diff --git a/block.c b/block.c
index 0cd4412725..615ffa7f40 100644
--- a/block.c
+++ b/block.c
@@ -338,28 +338,16 @@ char *bdrv_get_full_backing_filename_from_filename(const 
char *backed,
 }
 }
 
-void bdrv_get_full_backing_filename(BlockDriverState *bs, char *dest, size_t 
sz,
-Error **errp)
+char *bdrv_get_full_backing_filename(BlockDriverState *bs, Error **errp)
 {
 char *backed;
-char *full_name;
-Error *local_error = NULL;
 
 bdrv_refresh_filename(bs);
 
 backed = bs->exact_filename[0] ? bs->exact_filename : bs->filename;
-
-full_name = bdrv_get_full_backing_filename_from_filename(backed,
- bs->backing_file,
- &local_error);
-if (full_name) {
-pstrcpy(dest, sz, full_name);
-g_free(full_name);
-} else if (local_error) {
-error_propagate(errp, local_error);
-} else if (sz > 0) {
-*dest = '\0';
-}
+return bdrv_get_full_backing_filename_from_filename(backed,
+bs->backing_file,
+errp);
 }
 
 void bdrv_register(BlockDriver *bdrv)
@@ -2356,7 +2344,7 @@ out:
 int bdrv_open_backing_file(BlockDriverState *bs, QDict *parent_options,
const char *bdref_key, Error **errp)
 {
-char *backing_filename = g_malloc0(PATH_MAX);
+char *backing_filename = NULL;
 char *bdref_key_dot;
 const char *reference = NULL;
 int ret = 0;
@@ -2391,7 +2379,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict 
*parent_options,
  */
 reference = qdict_get_try_str(parent_options, bdref_key);
 if (reference || qdict_haskey(options, "file.filename")) {
-backing_filename[0] = '\0';
+/* keep backing_filename NULL */
 } else if (bs->backing_file[0] == '\0' && qdict_size(options) == 0) {
 qobject_unref(options);
 goto free_exit;
@@ -2406,8 +2394,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict 
*parent_options,
 implicit_backing = !strcmp(bs->auto_backing_file, 
bs->backing_file);
 }
 
-bdrv_get_full_backing_filename(bs, backing_filename, PATH_MAX,
-   &local_err);
+backing_filename = bdrv_get_full_backing_filename(bs, &local_err);
 if (local_err) {
 ret = -EINVAL;
 error_propagate(errp, local_err);
@@ -2428,9 +2415,8 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict 
*parent_options,
 qdict_put_str(options, "driver", bs->backing_format);
 }
 
-backing_hd = bdrv_open_inherit(*backing_filename ? backing_filename : NULL,
-   reference, options, 0, bs, &child_backing,
-   errp);
+backing_hd = bdrv_open_inherit(backing_filename, reference, options, 0, bs,
+   &child_backing, errp);
 if (!backing_hd) {
 bs->open_flags |= BDRV_O_NO_BACKING;
 error_prepend(errp, "Could not open backing file: ");
@@ -4436,7 +4422,6 @@ BlockDriverState 
*bdrv_find_backing_image(BlockDriverState *bs,
 int is_protocol = 0;
 BlockDriverState *curr_bs = NULL;
 BlockDriverState *retval = NULL;
-Error *local_error = NULL;
 
 if (!bs || !bs->drv || !backing_file) {
 return NULL;
@@ -4456,21 +4441,22 @@ BlockDriverState 
*bdrv_find_backing_image(BlockDriverState *bs,
 /* If either of the filename paths is actually a protocol, then
  * compare unmodified paths; otherwise make paths relative */
 if (is_protocol || path_has_protocol(curr_bs->backing_file)) {
+char *backing_file_full_ret;
+
   

[Qemu-block] [PATCH v12 09/31] block: Make path_combine() return the path

[Max Reitz]

Besides being safe for arbitrary path lengths, after some follow-up
patches all callers will want a freshly allocated buffer anyway.

In the meantime, path_combine_deprecated() is added which has the same
interface as path_combine() had before this patch. All callers to that
function will be converted in follow-up patches.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
Reviewed-by: Kevin Wolf 
---
 include/block/block.h |  4 +-
 block.c   | 85 ---
 block/vmdk.c  |  3 +-
 3 files changed, 49 insertions(+), 43 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index 1e54c8f505..c69b08bef9 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -494,9 +494,7 @@ void bdrv_get_full_backing_filename_from_filename(const 
char *backed,
 
 int path_has_protocol(const char *path);
 int path_is_absolute(const char *path);
-void path_combine(char *dest, int dest_size,
-  const char *base_path,
-  const char *filename);
+char *path_combine(const char *base_path, const char *filename);
 
 int bdrv_readv_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos);
 int bdrv_writev_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos);
diff --git a/block.c b/block.c
index 22f91fb3ee..933a43c989 100644
--- a/block.c
+++ b/block.c
@@ -152,53 +152,62 @@ int path_is_absolute(const char *path)
 #endif
 }
 
-/* if filename is absolute, just copy it to dest. Otherwise, build a
+/* if filename is absolute, just return its duplicate. Otherwise, build a
path to it by considering it is relative to base_path. URL are
supported. */
-void path_combine(char *dest, int dest_size,
-  const char *base_path,
-  const char *filename)
+char *path_combine(const char *base_path, const char *filename)
 {
+const char *protocol_stripped = NULL;
 const char *p, *p1;
+char *result;
 int len;
 
-if (dest_size <= 0)
-return;
 if (path_is_absolute(filename)) {
-pstrcpy(dest, dest_size, filename);
-} else {
-const char *protocol_stripped = NULL;
+return g_strdup(filename);
+}
 
-if (path_has_protocol(base_path)) {
-protocol_stripped = strchr(base_path, ':');
-if (protocol_stripped) {
-protocol_stripped++;
-}
+if (path_has_protocol(base_path)) {
+protocol_stripped = strchr(base_path, ':');
+if (protocol_stripped) {
+protocol_stripped++;
 }
-p = protocol_stripped ?: base_path;
+}
+p = protocol_stripped ?: base_path;
 
-p1 = strrchr(base_path, '/');
+p1 = strrchr(base_path, '/');
 #ifdef _WIN32
-{
-const char *p2;
-p2 = strrchr(base_path, '\\');
-if (!p1 || p2 > p1)
-p1 = p2;
+{
+const char *p2;
+p2 = strrchr(base_path, '\\');
+if (!p1 || p2 > p1) {
+p1 = p2;
 }
+}
 #endif
-if (p1)
-p1++;
-else
-p1 = base_path;
-if (p1 > p)
-p = p1;
-len = p - base_path;
-if (len > dest_size - 1)
-len = dest_size - 1;
-memcpy(dest, base_path, len);
-dest[len] = '\0';
-pstrcat(dest, dest_size, filename);
+if (p1) {
+p1++;
+} else {
+p1 = base_path;
+}
+if (p1 > p) {
+p = p1;
 }
+len = p - base_path;
+
+result = g_malloc(len + strlen(filename) + 1);
+memcpy(result, base_path, len);
+strcpy(result + len, filename);
+
+return result;
+}
+
+static void path_combine_deprecated(char *dest, int dest_size,
+const char *base_path,
+const char *filename)
+{
+char *combined = path_combine(base_path, filename);
+pstrcpy(dest, dest_size, combined);
+g_free(combined);
 }
 
 /*
@@ -316,7 +325,7 @@ void bdrv_get_full_backing_filename_from_filename(const 
char *backed,
 error_setg(errp, "Cannot use relative backing file names for '%s'",
backed);
 } else {
-path_combine(dest, sz, backed, backing);
+path_combine_deprecated(dest, sz, backed, backing);
 }
 }
 
@@ -4445,8 +4454,8 @@ BlockDriverState 
*bdrv_find_backing_image(BlockDriverState *bs,
 } else {
 /* If not an absolute filename path, make it relative to the 
current
  * image's filename path */
-path_combine(filename_tmp, PATH_MAX, curr_bs->filename,
- backing_file);
+path_combine_deprecated(filename_tmp, PATH_MAX, curr_bs->filename,
+backing_file);
 
 /* We are going to compare absolute pathnames */
 if (!realpath(filename_tmp, filename_full)) {
@@ -4455,8 +4464,8 @@ BlockDriverState 
*bdrv_find_backing_image(Bl

[Qemu-block] [PATCH v12 12/31] block: Add bdrv_make_absolute_filename()

[Max Reitz]

This is a general function for making a filename that is relative to a
certain BDS absolute.

It calls bdrv_get_full_backing_filename_from_filename() for now, but
that will be changed in a follow-up patch.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 block.c | 27 ---
 1 file changed, 20 insertions(+), 7 deletions(-)

diff --git a/block.c b/block.c
index 615ffa7f40..08b89b86fd 100644
--- a/block.c
+++ b/block.c
@@ -338,16 +338,29 @@ char *bdrv_get_full_backing_filename_from_filename(const 
char *backed,
 }
 }
 
-char *bdrv_get_full_backing_filename(BlockDriverState *bs, Error **errp)
+/*
+ * If @filename is empty or NULL, this function returns NULL without
+ * setting @errp.  In all other cases, NULL will only be returned with
+ * @errp set.
+ */
+static char *bdrv_make_absolute_filename(BlockDriverState *relative_to,
+ const char *filename, Error **errp)
 {
-char *backed;
+char *bs_filename;
 
-bdrv_refresh_filename(bs);
+bdrv_refresh_filename(relative_to);
+
+bs_filename = relative_to->exact_filename[0]
+  ? relative_to->exact_filename
+  : relative_to->filename;
 
-backed = bs->exact_filename[0] ? bs->exact_filename : bs->filename;
-return bdrv_get_full_backing_filename_from_filename(backed,
-bs->backing_file,
-errp);
+return bdrv_get_full_backing_filename_from_filename(bs_filename,
+filename ?: "", errp);
+}
+
+char *bdrv_get_full_backing_filename(BlockDriverState *bs, Error **errp)
+{
+return bdrv_make_absolute_filename(bs, bs->backing_file, errp);
 }
 
 void bdrv_register(BlockDriver *bdrv)
-- 
2.19.2

[Qemu-block] [PATCH v12 15/31] blkverify: Make bdrv_dirname() return NULL

[Max Reitz]

blkverify's BDSs have a file BDS, but we do not want this to be
preferred over the raw node. There is no way to decide between the two
(and not really a reason to, either), so just return NULL in blkverify's
implementation of bdrv_dirname().

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
---
 block/blkverify.c | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/block/blkverify.c b/block/blkverify.c
index 035d77b64a..3c7d4c8729 100644
--- a/block/blkverify.c
+++ b/block/blkverify.c
@@ -313,6 +313,15 @@ static void blkverify_refresh_filename(BlockDriverState 
*bs, QDict *options)
 }
 }
 
+static char *blkverify_dirname(BlockDriverState *bs, Error **errp)
+{
+/* In general, there are two BDSs with different dirnames below this one;
+ * so there is no unique dirname we could return (unless both are equal by
+ * chance). Therefore, to be consistent, just always return NULL. */
+error_setg(errp, "Cannot generate a base directory for blkverify nodes");
+return NULL;
+}
+
 static BlockDriver bdrv_blkverify = {
 .format_name  = "blkverify",
 .protocol_name= "blkverify",
@@ -324,6 +333,7 @@ static BlockDriver bdrv_blkverify = {
 .bdrv_child_perm  = bdrv_filter_default_perms,
 .bdrv_getlength   = blkverify_getlength,
 .bdrv_refresh_filename= blkverify_refresh_filename,
+.bdrv_dirname = blkverify_dirname,
 
 .bdrv_co_preadv   = blkverify_co_preadv,
 .bdrv_co_pwritev  = blkverify_co_pwritev,
-- 
2.19.2

[Qemu-block] [PATCH v12 14/31] block: Add bdrv_dirname()

[Max Reitz]

This function may be implemented by block drivers to derive a directory
name from a BDS. Concatenating this g_free()-able string with a relative
filename must result in a valid (not necessarily existing) filename, so
this is a function that should generally be not implemented by format
drivers, because this is protocol-specific.

If a BDS's driver does not implement this function, bdrv_dirname() will
fall through to the BDS's file if it exists. If it does not, the
exact_filename field will be used to generate a directory name.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 include/block/block.h |  1 +
 include/block/block_int.h |  7 +++
 block.c   | 27 +++
 3 files changed, 35 insertions(+)

diff --git a/include/block/block.h b/include/block/block.h
index a1e1b59e9c..2e33535465 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -489,6 +489,7 @@ char *bdrv_get_full_backing_filename(BlockDriverState *bs, 
Error **errp);
 char *bdrv_get_full_backing_filename_from_filename(const char *backed,
const char *backing,
Error **errp);
+char *bdrv_dirname(BlockDriverState *bs, Error **errp);
 
 int path_has_protocol(const char *path);
 int path_is_absolute(const char *path);
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 93cd669a35..d65f80a013 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -141,6 +141,13 @@ struct BlockDriver {
 
 void (*bdrv_refresh_filename)(BlockDriverState *bs, QDict *options);
 
+/*
+ * Returns an allocated string which is the directory name of this BDS: It
+ * will be used to make relative filenames absolute by prepending this
+ * function's return value to them.
+ */
+char *(*bdrv_dirname)(BlockDriverState *bs, Error **errp);
+
 /* aio */
 BlockAIOCB *(*bdrv_aio_preadv)(BlockDriverState *bs,
 uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags,
diff --git a/block.c b/block.c
index ccddf4858a..3257bc2c9d 100644
--- a/block.c
+++ b/block.c
@@ -5497,6 +5497,33 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 }
 }
 
+char *bdrv_dirname(BlockDriverState *bs, Error **errp)
+{
+BlockDriver *drv = bs->drv;
+
+if (!drv) {
+error_setg(errp, "Node '%s' is ejected", bs->node_name);
+return NULL;
+}
+
+if (drv->bdrv_dirname) {
+return drv->bdrv_dirname(bs, errp);
+}
+
+if (bs->file) {
+return bdrv_dirname(bs->file->bs, errp);
+}
+
+bdrv_refresh_filename(bs);
+if (bs->exact_filename[0] != '\0') {
+return path_combine(bs->exact_filename, "");
+}
+
+error_setg(errp, "Cannot generate a base directory for %s nodes",
+   drv->format_name);
+return NULL;
+}
+
 /*
  * Hot add/remove a BDS's child. So the user can take a child offline when
  * it is broken and take a new child online
-- 
2.19.2

[Qemu-block] [PATCH v12 13/31] block: Fix bdrv_find_backing_image()

[Max Reitz]

bdrv_find_backing_image() should use bdrv_get_full_backing_filename() or
bdrv_make_absolute_filename() instead of trying to do what those
functions do by itself.

path_combine_deprecated() can now be dropped, so let's do that.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 block.c | 33 ++---
 1 file changed, 10 insertions(+), 23 deletions(-)

diff --git a/block.c b/block.c
index 08b89b86fd..ccddf4858a 100644
--- a/block.c
+++ b/block.c
@@ -201,15 +201,6 @@ char *path_combine(const char *base_path, const char 
*filename)
 return result;
 }
 
-static void path_combine_deprecated(char *dest, int dest_size,
-const char *base_path,
-const char *filename)
-{
-char *combined = path_combine(base_path, filename);
-pstrcpy(dest, dest_size, combined);
-g_free(combined);
-}
-
 /*
  * Helper function for bdrv_parse_filename() implementations to remove optional
  * protocol prefixes (especially "file:") from a filename and for putting the
@@ -4442,13 +4433,9 @@ BlockDriverState 
*bdrv_find_backing_image(BlockDriverState *bs,
 
 filename_full = g_malloc(PATH_MAX);
 backing_file_full = g_malloc(PATH_MAX);
-filename_tmp  = g_malloc(PATH_MAX);
 
 is_protocol = path_has_protocol(backing_file);
 
-/* This will recursively refresh everything in the backing chain */
-bdrv_refresh_filename(bs);
-
 for (curr_bs = bs; curr_bs->backing; curr_bs = curr_bs->backing->bs) {
 
 /* If either of the filename paths is actually a protocol, then
@@ -4474,22 +4461,23 @@ BlockDriverState 
*bdrv_find_backing_image(BlockDriverState *bs,
 } else {
 /* If not an absolute filename path, make it relative to the 
current
  * image's filename path */
-path_combine_deprecated(filename_tmp, PATH_MAX, curr_bs->filename,
-backing_file);
-
-/* We are going to compare absolute pathnames */
-if (!realpath(filename_tmp, filename_full)) {
+filename_tmp = bdrv_make_absolute_filename(curr_bs, backing_file,
+   NULL);
+/* We are going to compare canonicalized absolute pathnames */
+if (!filename_tmp || !realpath(filename_tmp, filename_full)) {
+g_free(filename_tmp);
 continue;
 }
+g_free(filename_tmp);
 
 /* We need to make sure the backing filename we are comparing 
against
  * is relative to the current image filename (or absolute) */
-path_combine_deprecated(filename_tmp, PATH_MAX, curr_bs->filename,
-curr_bs->backing_file);
-
-if (!realpath(filename_tmp, backing_file_full)) {
+filename_tmp = bdrv_get_full_backing_filename(curr_bs, NULL);
+if (!filename_tmp || !realpath(filename_tmp, backing_file_full)) {
+g_free(filename_tmp);
 continue;
 }
+g_free(filename_tmp);
 
 if (strcmp(backing_file_full, filename_full) == 0) {
 retval = curr_bs->backing->bs;
@@ -4500,7 +4488,6 @@ BlockDriverState 
*bdrv_find_backing_image(BlockDriverState *bs,
 
 g_free(filename_full);
 g_free(backing_file_full);
-g_free(filename_tmp);
 return retval;
 }
 
-- 
2.19.2

[Qemu-block] [PATCH v12 18/31] block/nfs: Implement bdrv_dirname()

[Max Reitz]

While the basic idea is obvious and could be handled by the default
bdrv_dirname() implementation, we cannot generate a directory name if
the gid or uid are set, so we have to explicitly return NULL in those
cases.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 block/nfs.c | 15 +++
 1 file changed, 15 insertions(+)

diff --git a/block/nfs.c b/block/nfs.c
index eab1a2c408..19ee07c321 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -855,6 +855,20 @@ static void nfs_refresh_filename(BlockDriverState *bs, 
QDict *options)
 bs->full_open_options = opts;
 }
 
+static char *nfs_dirname(BlockDriverState *bs, Error **errp)
+{
+NFSClient *client = bs->opaque;
+
+if (client->uid || client->gid) {
+bdrv_refresh_filename(bs);
+error_setg(errp, "Cannot generate a base directory for NFS node '%s'",
+   bs->filename);
+return NULL;
+}
+
+return g_strdup_printf("nfs://%s%s/", client->server->host, client->path);
+}
+
 #ifdef LIBNFS_FEATURE_PAGECACHE
 static void coroutine_fn nfs_co_invalidate_cache(BlockDriverState *bs,
  Error **errp)
@@ -889,6 +903,7 @@ static BlockDriver bdrv_nfs = {
 .bdrv_detach_aio_context= nfs_detach_aio_context,
 .bdrv_attach_aio_context= nfs_attach_aio_context,
 .bdrv_refresh_filename  = nfs_refresh_filename,
+.bdrv_dirname   = nfs_dirname,
 
 #ifdef LIBNFS_FEATURE_PAGECACHE
 .bdrv_co_invalidate_cache   = nfs_co_invalidate_cache,
-- 
2.19.2

[Qemu-block] [PATCH v12 16/31] quorum: Make bdrv_dirname() return NULL

[Max Reitz]

While the common implementation for bdrv_dirname() should return NULL
for quorum BDSs already (because they do not have a file node and their
exact_filename field should be empty), there is no reason not to make
that explicit.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Reviewed-by: Alberto Garcia 
---
 block/quorum.c | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/block/quorum.c b/block/quorum.c
index cf9d7c16c2..a890f21e85 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -1094,6 +1094,16 @@ static void quorum_refresh_filename(BlockDriverState 
*bs, QDict *options)
 bs->full_open_options = opts;
 }
 
+static char *quorum_dirname(BlockDriverState *bs, Error **errp)
+{
+/* In general, there are multiple BDSs with different dirnames below this
+ * one; so there is no unique dirname we could return (unless all are equal
+ * by chance, or there is only one). Therefore, to be consistent, just
+ * always return NULL. */
+error_setg(errp, "Cannot generate a base directory for quorum nodes");
+return NULL;
+}
+
 static BlockDriver bdrv_quorum = {
 .format_name= "quorum",
 
@@ -1102,6 +1112,7 @@ static BlockDriver bdrv_quorum = {
 .bdrv_open  = quorum_open,
 .bdrv_close = quorum_close,
 .bdrv_refresh_filename  = quorum_refresh_filename,
+.bdrv_dirname   = quorum_dirname,
 
 .bdrv_co_flush_to_disk  = quorum_co_flush,
 
-- 
2.19.2

[Qemu-block] [PATCH v12 17/31] block/nbd: Make bdrv_dirname() return NULL

[Max Reitz]

The generic bdrv_dirname() implementation would be able to generate some
form of directory name for many NBD nodes, but it would be always wrong.
Therefore, we have to explicitly make it an error (until NBD has some
form of specification for export paths, if it ever will).

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
Reviewed-by: Eric Blake 
---
 block/nbd.c | 13 +
 1 file changed, 13 insertions(+)

diff --git a/block/nbd.c b/block/nbd.c
index e87699fb73..bca127c8f5 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -564,6 +564,16 @@ static void nbd_refresh_filename(BlockDriverState *bs, 
QDict *options)
 bs->full_open_options = opts;
 }
 
+static char *nbd_dirname(BlockDriverState *bs, Error **errp)
+{
+/* The generic bdrv_dirname() implementation is able to work out some
+ * directory name for NBD nodes, but that would be wrong. So far there is 
no
+ * specification for how "export paths" would work, so NBD does not have
+ * directory names. */
+error_setg(errp, "Cannot generate a base directory for NBD nodes");
+return NULL;
+}
+
 static BlockDriver bdrv_nbd = {
 .format_name= "nbd",
 .protocol_name  = "nbd",
@@ -582,6 +592,7 @@ static BlockDriver bdrv_nbd = {
 .bdrv_attach_aio_context= nbd_attach_aio_context,
 .bdrv_refresh_filename  = nbd_refresh_filename,
 .bdrv_co_block_status   = nbd_client_co_block_status,
+.bdrv_dirname   = nbd_dirname,
 };
 
 static BlockDriver bdrv_nbd_tcp = {
@@ -602,6 +613,7 @@ static BlockDriver bdrv_nbd_tcp = {
 .bdrv_attach_aio_context= nbd_attach_aio_context,
 .bdrv_refresh_filename  = nbd_refresh_filename,
 .bdrv_co_block_status   = nbd_client_co_block_status,
+.bdrv_dirname   = nbd_dirname,
 };
 
 static BlockDriver bdrv_nbd_unix = {
@@ -622,6 +634,7 @@ static BlockDriver bdrv_nbd_unix = {
 .bdrv_attach_aio_context= nbd_attach_aio_context,
 .bdrv_refresh_filename  = nbd_refresh_filename,
 .bdrv_co_block_status   = nbd_client_co_block_status,
+.bdrv_dirname   = nbd_dirname,
 };
 
 static void bdrv_nbd_init(void)
-- 
2.19.2

[Qemu-block] [PATCH v12 21/31] block: Add strong_runtime_opts to BlockDriver

[Max Reitz]

This new field can be set by block drivers to list the runtime options
they accept that may influence the contents of the respective BDS. As of
a follow-up patch, this list will be used by the common
bdrv_refresh_filename() implementation to decide which options to put
into BDS.full_open_options (and consequently whether a JSON filename has
to be created), thus freeing the drivers of having to implement that
logic themselves.

Additionally, this patch adds the field to all of the block drivers that
need it and sets it accordingly.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 include/block/block_int.h |  7 +++
 block/blkdebug.c  | 16 
 block/blklogwrites.c  |  8 
 block/crypto.c|  8 
 block/curl.c  | 21 +
 block/gluster.c   | 19 +++
 block/iscsi.c | 18 ++
 block/nbd.c   | 14 ++
 block/nfs.c   | 11 +++
 block/null.c  |  9 +
 block/nvme.c  |  8 
 block/qcow.c  |  7 +++
 block/qcow2.c |  7 +++
 block/quorum.c| 11 +++
 block/raw-format.c| 10 +-
 block/rbd.c   | 14 ++
 block/replication.c   |  8 
 block/sheepdog.c  | 12 
 block/ssh.c   | 12 
 block/throttle.c  |  7 +++
 block/vpc.c   |  7 +++
 block/vvfat.c | 12 
 block/vxhs.c  | 11 +++
 23 files changed, 256 insertions(+), 1 deletion(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index d65f80a013..40f00aa44e 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -516,6 +516,13 @@ struct BlockDriver {
 void (*bdrv_register_buf)(BlockDriverState *bs, void *host, size_t size);
 void (*bdrv_unregister_buf)(BlockDriverState *bs, void *host);
 QLIST_ENTRY(BlockDriver) list;
+
+/* Pointer to a NULL-terminated array of names of strong options
+ * that can be specified for bdrv_open(). A strong option is one
+ * that changes the data of a BDS.
+ * If this pointer is NULL, the array is considered empty.
+ * "filename" and "driver" are always considered strong. */
+const char *const *strong_runtime_opts;
 };
 
 typedef struct BlockLimits {
diff --git a/block/blkdebug.c b/block/blkdebug.c
index 0759452925..71b4275b98 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -888,6 +888,20 @@ static int blkdebug_reopen_prepare(BDRVReopenState 
*reopen_state,
 return 0;
 }
 
+static const char *const blkdebug_strong_runtime_opts[] = {
+"config",
+"inject-error.",
+"set-state.",
+"align",
+"max-transfer",
+"opt-write-zero",
+"max-write-zero",
+"opt-discard",
+"max-discard",
+
+NULL
+};
+
 static BlockDriver bdrv_blkdebug = {
 .format_name= "blkdebug",
 .protocol_name  = "blkdebug",
@@ -917,6 +931,8 @@ static BlockDriver bdrv_blkdebug = {
 = blkdebug_debug_remove_breakpoint,
 .bdrv_debug_resume  = blkdebug_debug_resume,
 .bdrv_debug_is_suspended= blkdebug_debug_is_suspended,
+
+.strong_runtime_opts= blkdebug_strong_runtime_opts,
 };
 
 static void bdrv_blkdebug_init(void)
diff --git a/block/blklogwrites.c b/block/blklogwrites.c
index 8f1b589d54..1c8e200326 100644
--- a/block/blklogwrites.c
+++ b/block/blklogwrites.c
@@ -518,6 +518,13 @@ blk_log_writes_co_pdiscard(BlockDriverState *bs, int64_t 
offset, int count)
  LOG_DISCARD_FLAG, false);
 }
 
+static const char *const blk_log_writes_strong_runtime_opts[] = {
+"log-append",
+"log-sector-size",
+
+NULL
+};
+
 static BlockDriver bdrv_blk_log_writes = {
 .format_name= "blklogwrites",
 .instance_size  = sizeof(BDRVBlkLogWritesState),
@@ -537,6 +544,7 @@ static BlockDriver bdrv_blk_log_writes = {
 .bdrv_co_block_status   = bdrv_co_block_status_from_file,
 
 .is_filter  = true,
+.strong_runtime_opts= blk_log_writes_strong_runtime_opts,
 };
 
 static void bdrv_blk_log_writes_init(void)
diff --git a/block/crypto.c b/block/crypto.c
index f0a5f6b987..d9bfd1084a 100644
--- a/block/crypto.c
+++ b/block/crypto.c
@@ -622,6 +622,12 @@ block_crypto_get_specific_info_luks(BlockDriverState *bs)
 return spec_info;
 }
 
+static const char *const block_crypto_strong_runtime_opts[] = {
+BLOCK_CRYPTO_OPT_LUKS_KEY_SECRET,
+
+NULL
+};
+
 BlockDriver bdrv_crypto_luks = {
 .format_name= "luks",
 .instance_size  = sizeof(BlockCrypto),
@@ -643,6 +649,8 @@ BlockDriver bdrv_crypto_luks = {
 .bdrv_getlength = block_crypto_getlength,
 .bdrv_get_info  = block_crypto_get_info_luks,
 .bdrv_get_specific_info = block_crypto_get_specific_info_luks,
+

[Qemu-block] [PATCH v12 19/31] block: Use bdrv_dirname() for relative filenames

[Max Reitz]

bdrv_get_full_backing_filename_from_filename() breaks down when it comes
to JSON filenames. Using bdrv_dirname() as the basis is better because
since we have BDS, we can descend through the BDS tree to the protocol
layer, which gives us a greater probability of finding a non-JSON name;
also, bdrv_dirname() is more correct as it allows block drivers to
override the generation of that directory name in a protocol-specific
way.

We still need to keep bdrv_get_full_backing_filename_from_filename(),
though, because it has valid callers which need it during image creation
when no BDS is available yet.

This makes a test case in qemu-iotest 110, which was supposed to fail,
work. That is actually good, but we need to change the reference output
(and the comment in 110) accordingly.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 block.c| 20 +---
 tests/qemu-iotests/110 |  3 ++-
 tests/qemu-iotests/110.out |  2 +-
 3 files changed, 16 insertions(+), 9 deletions(-)

diff --git a/block.c b/block.c
index 3257bc2c9d..9f1756b1f4 100644
--- a/block.c
+++ b/block.c
@@ -337,16 +337,22 @@ char *bdrv_get_full_backing_filename_from_filename(const 
char *backed,
 static char *bdrv_make_absolute_filename(BlockDriverState *relative_to,
  const char *filename, Error **errp)
 {
-char *bs_filename;
+char *dir, *full_name;
 
-bdrv_refresh_filename(relative_to);
+if (!filename || filename[0] == '\0') {
+return NULL;
+} else if (path_has_protocol(filename) || path_is_absolute(filename)) {
+return g_strdup(filename);
+}
 
-bs_filename = relative_to->exact_filename[0]
-  ? relative_to->exact_filename
-  : relative_to->filename;
+dir = bdrv_dirname(relative_to, errp);
+if (!dir) {
+return NULL;
+}
 
-return bdrv_get_full_backing_filename_from_filename(bs_filename,
-filename ?: "", errp);
+full_name = g_strconcat(dir, filename, NULL);
+g_free(dir);
+return full_name;
 }
 
 char *bdrv_get_full_backing_filename(BlockDriverState *bs, Error **errp)
diff --git a/tests/qemu-iotests/110 b/tests/qemu-iotests/110
index b64b3b215a..3e9d72d302 100755
--- a/tests/qemu-iotests/110
+++ b/tests/qemu-iotests/110
@@ -60,7 +60,8 @@ echo '=== Non-reconstructable filename ==='
 echo
 
 # Across blkdebug without a config file, you cannot reconstruct filenames, so
-# qemu is incapable of knowing the directory of the top image
+# qemu is incapable of knowing the directory of the top image from the filename
+# alone. However, using bdrv_dirname(), it should still work.
 TEST_IMG="json:{
 'driver': '$IMGFMT',
 'file': {
diff --git a/tests/qemu-iotests/110.out b/tests/qemu-iotests/110.out
index b3584ff87f..5370bc1d26 100644
--- a/tests/qemu-iotests/110.out
+++ b/tests/qemu-iotests/110.out
@@ -14,7 +14,7 @@ backing file: t.IMGFMT.base (actual path: 
TEST_DIR/t.IMGFMT.base)
 image: json:{"driver": "IMGFMT", "file": {"set-state.0.event": "read_aio", 
"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": 
"blkdebug", "set-state.0.new_state": 42}}
 file format: IMGFMT
 virtual size: 64M (67108864 bytes)
-backing file: t.IMGFMT.base (cannot determine actual path)
+backing file: t.IMGFMT.base (actual path: TEST_DIR/t.IMGFMT.base)
 
 === Backing name is always relative to the backed image ===
 
-- 
2.19.2

[Qemu-block] [PATCH v12 24/31] block: Purify .bdrv_refresh_filename()

[Max Reitz]

Currently, BlockDriver.bdrv_refresh_filename() is supposed to both
refresh the filename (BDS.exact_filename) and set BDS.full_open_options.
Now that we have generic code in the central bdrv_refresh_filename() for
creating BDS.full_open_options, we can drop the latter part from all
BlockDriver.bdrv_refresh_filename() implementations.

This also means that we can drop all of the existing default code for
this from the global bdrv_refresh_filename() itself.

Furthermore, we now have to call BlockDriver.bdrv_refresh_filename()
after having set BDS.full_open_options, because the block driver's
implementation should now be allowed to depend on BDS.full_open_options
being set correctly.

Finally, with this patch we can drop the @options parameter from
BlockDriver.bdrv_refresh_filename(); also, add a comment on this
function's purpose in block/block_int.h while touching its interface.

This completely obsoletes blklogwrite's implementation of
.bdrv_refresh_filename().

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 include/block/block_int.h |   6 +-
 block.c   | 131 +++---
 block/blkdebug.c  |  54 ++--
 block/blklogwrites.c  |  23 ---
 block/blkverify.c |  16 +
 block/commit.c|   2 +-
 block/mirror.c|   2 +-
 block/nbd.c   |  23 +--
 block/nfs.c   |  36 +--
 block/null.c  |  22 ---
 block/nvme.c  |  22 ---
 block/quorum.c|  30 -
 12 files changed, 80 insertions(+), 287 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 8df23ab79c..620581d236 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -139,7 +139,11 @@ struct BlockDriver {
 Error **errp);
 int (*bdrv_make_empty)(BlockDriverState *bs);
 
-void (*bdrv_refresh_filename)(BlockDriverState *bs, QDict *options);
+/*
+ * Refreshes the bs->exact_filename field. If that is impossible,
+ * bs->exact_filename has to be left empty.
+ */
+void (*bdrv_refresh_filename)(BlockDriverState *bs);
 
 /*
  * Gathers the open options for all children into @target.
diff --git a/block.c b/block.c
index 53cb14a5ed..c8399a9036 100644
--- a/block.c
+++ b/block.c
@@ -5403,33 +5403,6 @@ static bool append_strong_runtime_options(QDict *d, 
BlockDriverState *bs)
 return found_any;
 }
 
-static bool append_open_options(QDict *d, BlockDriverState *bs)
-{
-const QDictEntry *entry;
-QemuOptDesc *desc;
-bool found_any = false;
-
-for (entry = qdict_first(bs->options); entry;
- entry = qdict_next(bs->options, entry))
-{
-/* Exclude all non-driver-specific options */
-for (desc = bdrv_runtime_opts.desc; desc->name; desc++) {
-if (!strcmp(qdict_entry_key(entry), desc->name)) {
-break;
-}
-}
-if (desc->name) {
-continue;
-}
-
-qdict_put_obj(d, qdict_entry_key(entry),
-  qobject_ref(qdict_entry_value(entry)));
-found_any = true;
-}
-
-return found_any;
-}
-
 /* Note: This function may return false positives; it may return true
  * even if opening the backing file specified by bs's image header
  * would result in exactly bs->backing. */
@@ -5463,6 +5436,8 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 BdrvChild *child;
 QDict *opts;
 bool backing_overridden;
+bool generate_json_filename; /* Whether our default implementation should
+fill exact_filename (false) or not (true) 
*/
 
 if (!drv) {
 return;
@@ -5498,90 +5473,10 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 backing_overridden = false;
 }
 
-if (drv->bdrv_refresh_filename) {
-/* Obsolete information is of no use here, so drop the old file name
- * information before refreshing it */
-bs->exact_filename[0] = '\0';
-if (bs->full_open_options) {
-qobject_unref(bs->full_open_options);
-bs->full_open_options = NULL;
-}
-
-opts = qdict_new();
-append_open_options(opts, bs);
-drv->bdrv_refresh_filename(bs, opts);
-qobject_unref(opts);
-} else if (bs->file) {
-/* Try to reconstruct valid information from the underlying file */
-bool has_open_options;
-
-bs->exact_filename[0] = '\0';
-if (bs->full_open_options) {
-qobject_unref(bs->full_open_options);
-bs->full_open_options = NULL;
-}
-
-opts = qdict_new();
-has_open_options = append_open_options(opts, bs);
-has_open_options |= backing_overridden;
-
-/* If no specific options have been given for this BDS, the filename of
- * the underlying file should suffice for this one as well */
-if (b

[Qemu-block] [PATCH v12 23/31] block: Generically refresh runtime options

[Max Reitz]

Instead of having every block driver which implements
bdrv_refresh_filename() copy all of the strong runtime options over to
bs->full_open_options, implement this process generically in
bdrv_refresh_filename().

This patch only adds this new generic implementation, it does not remove
the old functionality. This is done in a follow-up patch.

With this patch, some superfluous information (that should never have
been there) may be removed from some JSON filenames, as can be seen in
the change to iotest 110's reference output.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 block.c| 116 -
 tests/qemu-iotests/110.out |   2 +-
 2 files changed, 116 insertions(+), 2 deletions(-)

diff --git a/block.c b/block.c
index 9f1756b1f4..53cb14a5ed 100644
--- a/block.c
+++ b/block.c
@@ -5317,6 +5317,92 @@ out:
 return to_replace_bs;
 }
 
+/**
+ * Iterates through the list of runtime option keys that are said to
+ * be "strong" for a BDS.  An option is called "strong" if it changes
+ * a BDS's data.  For example, the null block driver's "size" and
+ * "read-zeroes" options are strong, but its "latency-ns" option is
+ * not.
+ *
+ * If a key returned by this function ends with a dot, all options
+ * starting with that prefix are strong.
+ */
+static const char *const *strong_options(BlockDriverState *bs,
+ const char *const *curopt)
+{
+static const char *const global_options[] = {
+"driver", "filename", NULL
+};
+
+if (!curopt) {
+return &global_options[0];
+}
+
+curopt++;
+if (curopt == &global_options[ARRAY_SIZE(global_options) - 1] && bs->drv) {
+curopt = bs->drv->strong_runtime_opts;
+}
+
+return (curopt && *curopt) ? curopt : NULL;
+}
+
+/**
+ * Copies all strong runtime options from bs->options to the given
+ * QDict.  The set of strong option keys is determined by invoking
+ * strong_options().
+ *
+ * Returns true iff any strong option was present in bs->options (and
+ * thus copied to the target QDict) with the exception of "filename"
+ * and "driver".  The caller is expected to use this value to decide
+ * whether the existence of strong options prevents the generation of
+ * a plain filename.
+ */
+static bool append_strong_runtime_options(QDict *d, BlockDriverState *bs)
+{
+bool found_any = false;
+const char *const *option_name = NULL;
+
+if (!bs->drv) {
+return false;
+}
+
+while ((option_name = strong_options(bs, option_name))) {
+bool option_given = false;
+
+assert(strlen(*option_name) > 0);
+if ((*option_name)[strlen(*option_name) - 1] != '.') {
+QObject *entry = qdict_get(bs->options, *option_name);
+if (!entry) {
+continue;
+}
+
+qdict_put_obj(d, *option_name, qobject_ref(entry));
+option_given = true;
+} else {
+const QDictEntry *entry;
+for (entry = qdict_first(bs->options); entry;
+ entry = qdict_next(bs->options, entry))
+{
+if (strstart(qdict_entry_key(entry), *option_name, NULL)) {
+qdict_put_obj(d, qdict_entry_key(entry),
+  qobject_ref(qdict_entry_value(entry)));
+option_given = true;
+}
+}
+}
+
+/* While "driver" and "filename" need to be included in a JSON 
filename,
+ * their existence does not prohibit generation of a plain filename. */
+if (!found_any && option_given &&
+strcmp(*option_name, "driver") && strcmp(*option_name, "filename"))
+{
+found_any = true;
+}
+}
+
+return found_any;
+}
+
 static bool append_open_options(QDict *d, BlockDriverState *bs)
 {
 const QDictEntry *entry;
@@ -5493,9 +5579,37 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 bs->full_open_options = opts;
 }
 
+/* Gather the options QDict */
+opts = qdict_new();
+append_strong_runtime_options(opts, bs);
+
+if (drv->bdrv_gather_child_options) {
+/* Some block drivers may not want to present all of their children's
+ * options, or name them differently from BdrvChild.name */
+drv->bdrv_gather_child_options(bs, opts, backing_overridden);
+} else {
+QLIST_FOREACH(child, &bs->children, next) {
+if (child->role == &child_backing && !backing_overridden) {
+/* We can skip the backing BDS if it has not been overridden */
+continue;
+}
+
+qdict_put(opts, child->name,
+  qobject_ref(child->bs->full_open_options));
+}
+
+if (backing_overridden && !bs->backing) {
+/* Force no backing file */
+qdict_put_null(opts, "backing");
+}
+}
+
+qobject_unref(bs->full_open_options);
+  

[Qemu-block] [PATCH v12 20/31] iotests: Add quorum case to test 110

[Max Reitz]

Test 110 tests relative backing filenames for complex BDS trees.  Now
that the originally supposedly failing test passes, let us add a new
failing test: Quorum can never work automatically (without detecting
whether all child nodes have the same base directory, but that would be
rather inconsistent behavior).

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 tests/qemu-iotests/110 | 26 ++
 tests/qemu-iotests/110.out |  7 +++
 2 files changed, 33 insertions(+)

diff --git a/tests/qemu-iotests/110 b/tests/qemu-iotests/110
index 3e9d72d302..185ad5437e 100755
--- a/tests/qemu-iotests/110
+++ b/tests/qemu-iotests/110
@@ -29,6 +29,7 @@ status=1  # failure is the default!
 _cleanup()
 {
_cleanup_test_img
+rm -f "$TEST_IMG.copy"
 }
 trap "_cleanup; exit \$status" 0 1 2 3 15
 
@@ -86,6 +87,31 @@ echo
 # omit the image size; it should work anyway
 _make_test_img -b "$TEST_IMG_REL.base"
 
+echo
+echo '=== Nodes without a common directory ==='
+echo
+
+cp "$TEST_IMG" "$TEST_IMG.copy"
+
+# Should inform us that the actual path of the backing file cannot be 
determined
+TEST_IMG="json:{
+'driver': '$IMGFMT',
+'file': {
+'driver': 'quorum',
+'vote-threshold': 1,
+'children': [
+{
+'driver': 'file',
+'filename': '$TEST_IMG'
+},
+{
+'driver': 'file',
+'filename': '$TEST_IMG.copy'
+}
+]
+}
+}" _img_info | _filter_img_info
+
 
 # success, all done
 echo '*** done'
diff --git a/tests/qemu-iotests/110.out b/tests/qemu-iotests/110.out
index 5370bc1d26..1d0b2475cc 100644
--- a/tests/qemu-iotests/110.out
+++ b/tests/qemu-iotests/110.out
@@ -19,4 +19,11 @@ backing file: t.IMGFMT.base (actual path: 
TEST_DIR/t.IMGFMT.base)
 === Backing name is always relative to the backed image ===
 
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 
backing_file=t.IMGFMT.base
+
+=== Nodes without a common directory ===
+
+image: json:{"driver": "IMGFMT", "file": {"children": [{"driver": "file", 
"filename": "TEST_DIR/t.IMGFMT"}, {"driver": "file", "filename": 
"TEST_DIR/t.IMGFMT.copy"}], "driver": "quorum", "blkverify": false, 
"rewrite-corrupted": false, "vote-threshold": 1}}
+file format: IMGFMT
+virtual size: 64M (67108864 bytes)
+backing file: t.IMGFMT.base (cannot determine actual path)
 *** done
-- 
2.19.2

[Qemu-block] [PATCH v12 22/31] block: Add BlockDriver.bdrv_gather_child_options

[Max Reitz]

Some follow-up patches will rework the way bs->full_open_options is
refreshed in bdrv_refresh_filename(). The new implementation will remove
the need for the block drivers' bdrv_refresh_filename() implementations
to set bs->full_open_options; instead, it will be generic and use static
information from each block driver.

However, by implementing bdrv_gather_child_options(), block drivers will
still be able to override the way the full_open_options of their
children are incorporated into their own.

We need to implement this function for VMDK because we have to prevent
the generic implementation from gathering the options of all children:
It is not possible to specify options for the extents through the
runtime options.

For quorum, the child names that would be used by the generic
implementation and the ones that we actually (currently) want to use
differ. See quorum_gather_child_options() for more information.

Note that both of these are cases which are not ideal: In case of VMDK
it would probably be nice to be able to specify options for all extents.
In case of quorum, the current runtime option structure is simply broken
and needs to be fixed (but that is left for another patch).

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 include/block/block_int.h | 24 +++
 block/quorum.c| 40 +++
 block/vmdk.c  | 19 +++
 3 files changed, 83 insertions(+)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 40f00aa44e..8df23ab79c 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -141,6 +141,30 @@ struct BlockDriver {
 
 void (*bdrv_refresh_filename)(BlockDriverState *bs, QDict *options);
 
+/*
+ * Gathers the open options for all children into @target.
+ * A simple format driver (without backing file support) might
+ * implement this function like this:
+ *
+ * QINCREF(bs->file->bs->full_open_options);
+ * qdict_put(target, "file", bs->file->bs->full_open_options);
+ *
+ * If not specified, the generic implementation will simply put
+ * all children's options under their respective name.
+ *
+ * @backing_overridden is true when bs->backing seems not to be
+ * the child that would result from opening bs->backing_file.
+ * Therefore, if it is true, the backing child's options should be
+ * gathered; otherwise, there is no need since the backing child
+ * is the one implied by the image header.
+ *
+ * Note that ideally this function would not be needed.  Every
+ * block driver which implements it is probably doing something
+ * shady regarding its runtime option structure.
+ */
+void (*bdrv_gather_child_options)(BlockDriverState *bs, QDict *target,
+  bool backing_overridden);
+
 /*
  * Returns an allocated string which is the directory name of this BDS: It
  * will be used to make relative filenames absolute by prepending this
diff --git a/block/quorum.c b/block/quorum.c
index 1af6458dc4..3984f0aa4f 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -1094,6 +1094,45 @@ static void quorum_refresh_filename(BlockDriverState 
*bs, QDict *options)
 bs->full_open_options = opts;
 }
 
+static void quorum_gather_child_options(BlockDriverState *bs, QDict *target,
+bool backing_overridden)
+{
+BDRVQuorumState *s = bs->opaque;
+QList *children_list;
+int i;
+
+/*
+ * The generic implementation for gathering child options in
+ * bdrv_refresh_filename() would use the names of the children
+ * as specified for bdrv_open_child() or bdrv_attach_child(),
+ * which is "children.%u" with %u being a value
+ * (s->next_child_index) that is incremented each time a new child
+ * is added (and never decremented).  Since children can be
+ * deleted at runtime, there may be gaps in that enumeration.
+ * When creating a new quorum BDS and specifying the children for
+ * it through runtime options, the enumeration used there may not
+ * have any gaps, though.
+ *
+ * Therefore, we have to create a new gap-less enumeration here
+ * (which we can achieve by simply putting all of the children's
+ * full_open_options into a QList).
+ *
+ * XXX: Note that there are issues with the current child option
+ *  structure quorum uses (such as the fact that children do
+ *  not really have unique permanent names).  Therefore, this
+ *  is going to have to change in the future and ideally we
+ *  want quorum to be covered by the generic implementation.
+ */
+
+children_list = qlist_new();
+qdict_put(target, "children", children_list);
+
+for (i = 0; i < s->num_children; i++) {
+qlist_append(children_list,
+ qobject_ref(s->children[i]->bs->full_open_options));
+  

[Qemu-block] [PATCH v12 26/31] block/nvme: Fix bdrv_refresh_filename()

[Max Reitz]

Currently, nvme's bdrv_refresh_filename() is an exact copy of null's
implementation.  However, for null, "null-co://" and "null-aio://" are
indeed valid filenames -- for nvme, they are not, as a device address is
still required.

The correct implementation should generate a filename of the form
"nvme://[PCI address]/[namespace]" (as the comment above
nvme_parse_filename() describes).

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 block/nvme.c | 23 +--
 1 file changed, 9 insertions(+), 14 deletions(-)

diff --git a/block/nvme.c b/block/nvme.c
index 51d100cca4..4e5972d9a1 100644
--- a/block/nvme.c
+++ b/block/nvme.c
@@ -111,6 +111,9 @@ typedef struct {
 
 /* Total size of mapped qiov, accessed under dma_map_lock */
 int dma_map_count;
+
+/* PCI address (required for nvme_refresh_filename()) */
+char *device;
 } BDRVNVMeState;
 
 #define NVME_BLOCK_OPT_DEVICE "device"
@@ -556,6 +559,7 @@ static int nvme_init(BlockDriverState *bs, const char 
*device, int namespace,
 
 qemu_co_mutex_init(&s->dma_map_lock);
 qemu_co_queue_init(&s->dma_flush_queue);
+s->device = g_strdup(device);
 s->nsid = namespace;
 s->aio_context = bdrv_get_aio_context(bs);
 ret = event_notifier_init(&s->irq_notifier, 0);
@@ -728,6 +732,8 @@ static void nvme_close(BlockDriverState *bs)
 event_notifier_cleanup(&s->irq_notifier);
 qemu_vfio_pci_unmap_bar(s->vfio, 0, (void *)s->regs, 0, NVME_BAR_SIZE);
 qemu_vfio_close(s->vfio);
+
+g_free(s->device);
 }
 
 static int nvme_file_open(BlockDriverState *bs, QDict *options, int flags,
@@ -1058,21 +1064,10 @@ static int nvme_reopen_prepare(BDRVReopenState 
*reopen_state,
 
 static void nvme_refresh_filename(BlockDriverState *bs)
 {
-const QDictEntry *e;
-
-for (e = qdict_first(bs->full_open_options); e;
- e = qdict_next(bs->full_open_options, e))
-{
-/* These options can be ignored */
-if (strcmp(qdict_entry_key(e), "filename") &&
-strcmp(qdict_entry_key(e), "driver"))
-{
-return;
-}
-}
+BDRVNVMeState *s = bs->opaque;
 
-snprintf(bs->exact_filename, sizeof(bs->exact_filename), "%s://",
- bs->drv->format_name);
+snprintf(bs->exact_filename, sizeof(bs->exact_filename), "nvme://%s/%i",
+ s->device, s->nsid);
 }
 
 static void nvme_refresh_limits(BlockDriverState *bs, Error **errp)
-- 
2.19.2

[Qemu-block] [PATCH v12 25/31] block: Do not copy exact_filename from format file

[Max Reitz]

If a format BDS's file BDS is in turn a format BDS, we cannot simply use
the same filename, because when opening a BDS tree based on a filename
alone, qemu will create only one format node on top of one protocol node
(disregarding a potential backing file).

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 block.c | 18 +++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/block.c b/block.c
index c8399a9036..44bebfc80c 100644
--- a/block.c
+++ b/block.c
@@ -5513,9 +5513,21 @@ void bdrv_refresh_filename(BlockDriverState *bs)
 
 bs->exact_filename[0] = '\0';
 
-/* If no specific options have been given for this BDS, the filename of
- * the underlying file should suffice for this one as well */
-if (bs->file->bs->exact_filename[0] && !generate_json_filename) {
+/*
+ * We can use the underlying file's filename if:
+ * - it has a filename,
+ * - the file is a protocol BDS, and
+ * - opening that file (as this BDS's format) will automatically create
+ *   the BDS tree we have right now, that is:
+ *   - the user did not significantly change this BDS's behavior with
+ * some explicit (strong) options
+ *   - no non-file child of this BDS has been overridden by the user
+ *   Both of these conditions are represented by 
generate_json_filename.
+ */
+if (bs->file->bs->exact_filename[0] &&
+bs->file->bs->drv->bdrv_file_open &&
+!generate_json_filename)
+{
 strcpy(bs->exact_filename, bs->file->bs->exact_filename);
 }
 }
-- 
2.19.2

[Qemu-block] [PATCH v12 27/31] block/curl: Harmonize option defaults

[Max Reitz]

Both of the defaults we currently have in the curl driver are named
based on a slightly different schema, let's unify that and call both
CURL_BLOCK_OPT_${NAME}_DEFAULT.

While at it, we can add a macro for the third option for which a default
exists, namely "sslverify".

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 block/curl.c | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index 97c63ad3ac..02f9d9d692 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -73,8 +73,6 @@ static CURLMcode __curl_multi_socket_action(CURLM 
*multi_handle,
 
 #define CURL_NUM_STATES 8
 #define CURL_NUM_ACB8
-#define READ_AHEAD_DEFAULT (256 * 1024)
-#define CURL_TIMEOUT_DEFAULT 5
 #define CURL_TIMEOUT_MAX 1
 
 #define CURL_BLOCK_OPT_URL   "url"
@@ -88,6 +86,10 @@ static CURLMcode __curl_multi_socket_action(CURLM 
*multi_handle,
 #define CURL_BLOCK_OPT_PROXY_USERNAME "proxy-username"
 #define CURL_BLOCK_OPT_PROXY_PASSWORD_SECRET "proxy-password-secret"
 
+#define CURL_BLOCK_OPT_READAHEAD_DEFAULT (256 * 1024)
+#define CURL_BLOCK_OPT_SSLVERIFY_DEFAULT true
+#define CURL_BLOCK_OPT_TIMEOUT_DEFAULT 5
+
 struct BDRVCURLState;
 
 static bool libcurl_initialized;
@@ -708,7 +710,7 @@ static int curl_open(BlockDriverState *bs, QDict *options, 
int flags,
 }
 
 s->readahead_size = qemu_opt_get_size(opts, CURL_BLOCK_OPT_READAHEAD,
-  READ_AHEAD_DEFAULT);
+  CURL_BLOCK_OPT_READAHEAD_DEFAULT);
 if ((s->readahead_size & 0x1ff) != 0) {
 error_setg(errp, "HTTP_READAHEAD_SIZE %zd is not a multiple of 512",
s->readahead_size);
@@ -716,13 +718,14 @@ static int curl_open(BlockDriverState *bs, QDict 
*options, int flags,
 }
 
 s->timeout = qemu_opt_get_number(opts, CURL_BLOCK_OPT_TIMEOUT,
- CURL_TIMEOUT_DEFAULT);
+ CURL_BLOCK_OPT_TIMEOUT_DEFAULT);
 if (s->timeout > CURL_TIMEOUT_MAX) {
 error_setg(errp, "timeout parameter is too large or negative");
 goto out_noclean;
 }
 
-s->sslverify = qemu_opt_get_bool(opts, CURL_BLOCK_OPT_SSLVERIFY, true);
+s->sslverify = qemu_opt_get_bool(opts, CURL_BLOCK_OPT_SSLVERIFY,
+ CURL_BLOCK_OPT_SSLVERIFY_DEFAULT);
 
 cookie = qemu_opt_get(opts, CURL_BLOCK_OPT_COOKIE);
 cookie_secret = qemu_opt_get(opts, CURL_BLOCK_OPT_COOKIE_SECRET);
-- 
2.19.2

[Qemu-block] [PATCH v12 28/31] block/curl: Implement bdrv_refresh_filename()

[Max Reitz]

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 block/curl.c | 21 +
 1 file changed, 21 insertions(+)

diff --git a/block/curl.c b/block/curl.c
index 02f9d9d692..1ccc2e92e1 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -963,6 +963,23 @@ static int64_t curl_getlength(BlockDriverState *bs)
 return s->len;
 }
 
+static void curl_refresh_filename(BlockDriverState *bs)
+{
+BDRVCURLState *s = bs->opaque;
+
+/* "readahead" and "timeout" do not change the guest-visible data,
+ * so ignore them */
+if (s->sslverify != CURL_BLOCK_OPT_SSLVERIFY_DEFAULT ||
+s->cookie || s->username || s->password || s->proxyusername ||
+s->proxypassword)
+{
+return;
+}
+
+pstrcpy(bs->exact_filename, sizeof(bs->exact_filename), s->url);
+}
+
+
 static const char *const curl_strong_runtime_opts[] = {
 CURL_BLOCK_OPT_URL,
 CURL_BLOCK_OPT_SSLVERIFY,
@@ -991,6 +1008,7 @@ static BlockDriver bdrv_http = {
 .bdrv_detach_aio_context= curl_detach_aio_context,
 .bdrv_attach_aio_context= curl_attach_aio_context,
 
+.bdrv_refresh_filename  = curl_refresh_filename,
 .strong_runtime_opts= curl_strong_runtime_opts,
 };
 
@@ -1009,6 +1027,7 @@ static BlockDriver bdrv_https = {
 .bdrv_detach_aio_context= curl_detach_aio_context,
 .bdrv_attach_aio_context= curl_attach_aio_context,
 
+.bdrv_refresh_filename  = curl_refresh_filename,
 .strong_runtime_opts= curl_strong_runtime_opts,
 };
 
@@ -1027,6 +1046,7 @@ static BlockDriver bdrv_ftp = {
 .bdrv_detach_aio_context= curl_detach_aio_context,
 .bdrv_attach_aio_context= curl_attach_aio_context,
 
+.bdrv_refresh_filename  = curl_refresh_filename,
 .strong_runtime_opts= curl_strong_runtime_opts,
 };
 
@@ -1045,6 +1065,7 @@ static BlockDriver bdrv_ftps = {
 .bdrv_detach_aio_context= curl_detach_aio_context,
 .bdrv_attach_aio_context= curl_attach_aio_context,
 
+.bdrv_refresh_filename  = curl_refresh_filename,
 .strong_runtime_opts= curl_strong_runtime_opts,
 };
 
-- 
2.19.2

[Qemu-block] [PATCH v12 31/31] iotests: Test json:{} filenames of internal BDSs

[Max Reitz]

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 tests/qemu-iotests/224 | 139 +
 tests/qemu-iotests/224.out |  18 +
 tests/qemu-iotests/group   |   1 +
 3 files changed, 158 insertions(+)
 create mode 100755 tests/qemu-iotests/224
 create mode 100644 tests/qemu-iotests/224.out

diff --git a/tests/qemu-iotests/224 b/tests/qemu-iotests/224
new file mode 100755
index 00..ef652ad488
--- /dev/null
+++ b/tests/qemu-iotests/224
@@ -0,0 +1,139 @@
+#!/usr/bin/env python
+#
+# Test json:{} filenames with qemu-internal BDSs
+# (the one of commit, to be precise)
+#
+# Copyright (C) 2018 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+# Creator/Owner: Max Reitz 
+
+import iotests
+from iotests import log, qemu_img, qemu_io_silent, filter_testfiles, \
+filter_imgfmt
+import json
+
+# Need backing file support (for arbitrary backing formats)
+iotests.verify_image_format(supported_fmts=['qcow2', 'qcow', 'qed'])
+iotests.verify_platform(['linux'])
+
+
+# There are two variations of this test:
+# (1) We do not set filter_node_name.  In that case, the commit_top
+# driver should not appear anywhere.
+# (2) We do set filter_node_name.  In that case, it should appear.
+#
+# This for loop executes both.
+for filter_node_name in False, True:
+log('')
+log('--- filter_node_name: %s ---' % filter_node_name)
+log('')
+
+with iotests.FilePath('base.img') as base_img_path, \
+ iotests.FilePath('mid.img') as mid_img_path, \
+ iotests.FilePath('top.img') as top_img_path, \
+ iotests.VM() as vm:
+
+assert qemu_img('create', '-f', iotests.imgfmt,
+base_img_path, '64M') == 0
+assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path,
+mid_img_path) == 0
+assert qemu_img('create', '-f', iotests.imgfmt, '-b', mid_img_path,
+top_img_path) == 0
+
+# Something to commit
+assert qemu_io_silent(mid_img_path, '-c', 'write -P 1 0 1M') == 0
+
+vm.launch()
+
+# Change the bottom-most image's backing file (to null-co://)
+# to enforce json:{} filenames
+vm.qmp_log('blockdev-add',
+node_name='top',
+driver=iotests.imgfmt,
+file={
+'driver': 'file',
+'filename': top_img_path
+},
+backing={
+'node-name': 'mid',
+'driver': iotests.imgfmt,
+'file': {
+'driver': 'file',
+'filename': mid_img_path
+},
+'backing': {
+'node-name': 'base',
+'driver': iotests.imgfmt,
+'file': {
+'driver': 'file',
+'filename': base_img_path
+},
+'backing': {
+'driver': 'null-co'
+}
+}
+},
+filters=[filter_testfiles, filter_imgfmt])
+
+# As long as block-commit does not accept node names, we have to
+# get our mid/base filenames here
+mid_name = vm.node_info('mid')['image']['filename']
+base_name = vm.node_info('base')['image']['filename']
+
+assert mid_name[:5] == 'json:'
+assert base_name[:5] == 'json:'
+
+# Start the block job
+if filter_node_name:
+vm.qmp_log('block-commit',
+job_id='commit',
+device='top',
+filter_node_name='filter_node',
+top=mid_name,
+base=base_name,
+speed=1,
+filters=[filter_testfiles, filter_imgfmt])
+else:
+vm.qmp_log('block-commit',
+job_id='commit',
+device='top',
+top=mid_name,
+base=base_name,
+speed=1,
+filters=[filter_testfi

[Qemu-block] [PATCH v12 30/31] block: BDS options may lack the "driver" option

[Max Reitz]

When BDSs are created by qemu itself (e.g. as filters in block jobs),
they may not have a "driver" option in their options QDict.  When
generating a json:{} filename, however, it must always be present.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 block.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/block.c b/block.c
index 44bebfc80c..b75276183b 100644
--- a/block.c
+++ b/block.c
@@ -5400,6 +5400,12 @@ static bool append_strong_runtime_options(QDict *d, 
BlockDriverState *bs)
 }
 }
 
+if (!qdict_haskey(d, "driver")) {
+/* Drivers created with bdrv_new_open_driver() may not have a
+ * @driver option.  Add it here. */
+qdict_put_str(d, "driver", bs->drv->format_name);
+}
+
 return found_any;
 }
 
-- 
2.19.2

[Qemu-block] [PATCH v12 29/31] block/null: Generate filename even with latency-ns

[Max Reitz]

While we cannot represent the latency-ns option in a filename, it is not
a strong option so not being able to should not stop us from generating
a filename nonetheless.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
---
 block/null.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/block/null.c b/block/null.c
index 1c56a0ef01..a322929478 100644
--- a/block/null.c
+++ b/block/null.c
@@ -248,7 +248,8 @@ static void null_refresh_filename(BlockDriverState *bs)
 {
 /* These options can be ignored */
 if (strcmp(qdict_entry_key(e), "filename") &&
-strcmp(qdict_entry_key(e), "driver"))
+strcmp(qdict_entry_key(e), "driver") &&
+strcmp(qdict_entry_key(e), NULL_OPT_LATENCY))
 {
 return;
 }
-- 
2.19.2

Re: [Qemu-block] Virtio-BLK/SCSI write requests and data payload checksums

[Michael S. Tsirkin]

On Mon, Dec 17, 2018 at 04:19:53PM +0100, Peter Lieven wrote:
> Am 17.12.18 um 15:48 schrieb Stefan Hajnoczi:
> > On Sun, Dec 16, 2018 at 06:53:44PM +0100, Peter Lieven wrote:
> > > It turned out that for writes a bounce buffer is indeed always necessary. 
> > > But what I found out is that
> > > it seems that even for reads it happens that the OS (Windows in this 
> > > case) issues 2 read requests with
> > > the same buffer in parallel. Is it possible that this is a bug in the 
> > > virtio or has anybone ever seen this before?
> > Is it Windows resume-from-sleep?  I remember a race condition that
> > Microsoft fixed.
> 
> 
> Its a running guest, no resume. From the dumped Data it seems that its often 
> (not always) log files.
> 
> 
> > 
> > The other weird thing I've seen is scatter-gather lists where the same
> > buffer address is used multiple times within one request (this can go
> > wrong if the host splits up the request, causing races).
> 
> 
> The requests go to different LBAs often several 100MB apart from each other. 
> The only thing I
> 
> can see it that the request size always seems to be 4096 byte.
> 
> 
> I have not seen data corruption so far, but this at all seems very strange.
> 
> I only have observed this because the data digest checksum is wrong. After a 
> retry it is correct.
> 
> When dumping the buffer addresses (in this case the address from the first 
> buffer in the qiov)
> 
> that came from Qemu/the guest the address is the same.
> 
> 
> Actually I don't know for sure that the address comes from the guest. In 
> theory it could be that
> 
> the request from the guest was less than 4096 byte and Qemu just assigned a 
> bounce buffer
> 
> to read the whole block and copied the relevant part to the guest. Can I see 
> from the
> 
> address (0x7fa4354c) if it comes from Qemu or the guest?
> 
> 
> Thanks,
> 
> Peter
> 

+cc some windows driver developers.


-- 
MST

Re: [Qemu-block] sheepdog build warning

[Michael S. Tsirkin]

On Mon, Dec 17, 2018 at 11:03:11PM -0500, Michael S. Tsirkin wrote:
> mingw32 build on fedora fails with this warning:
> 
> /scm/qemu/block/sheepdog.c: In function 'find_vdi_name':
> /scm/qemu/block/sheepdog.c:1239:5: error: 'strncpy' specified bound 256 
> equals destination size [-Werror=stringop-truncation]
>  strncpy(buf + SD_MAX_VDI_LEN, tag, SD_MAX_VDI_TAG_LEN);
>  ^~
> cc1: all warnings being treated as errors
> make: *** [/scm/qemu/rules.mak:69: block/sheepdog.o] Error 1
> make: *** Waiting for unfinished jobs
> 
> 
> Reading the code one sees it's working as intended:
> 
> 
> static int find_vdi_name(BDRVSheepdogState *s, const char *filename,
>  uint32_t snapid, const char *tag, uint32_t *vid,
>  bool lock, Error **errp)
> {
> int ret, fd;
> SheepdogVdiReq hdr;
> SheepdogVdiRsp *rsp = (SheepdogVdiRsp *)&hdr;
> unsigned int wlen, rlen = 0;
> char buf[SD_MAX_VDI_LEN + SD_MAX_VDI_TAG_LEN];
> 
> fd = connect_to_sdog(s, errp);
> if (fd < 0) {
> return fd;
> }
> 
> /* This pair of strncpy calls ensures that the buffer is zero-filled,
>  * which is desirable since we'll soon be sending those bytes, and
>  * don't want the send_req to read uninitialized data.
>  */
> strncpy(buf, filename, SD_MAX_VDI_LEN);
> strncpy(buf + SD_MAX_VDI_LEN, tag, SD_MAX_VDI_TAG_LEN);
> 
> 
> .
> 
> }
> 
> 
> 
> so this seems to be the case of GCC developers deciding that
> strncpy is simply a bad API and a correct use of it should be
> warned against.
> 
> I propose either
> 
> 1. simply adding
> 
> #pragma GCC diagnostic ignored "-Wstringop-truncation"
> 
> in osdep.
> 
> 2. adding an inline wrapper with said pragma in there.
> 
> 3. -Wno-stringop-truncation is the makefile
> 
> Thoughts?
> 
> -- 


So here's approach 2. However I note that a newer gcc 8.2.1
does not give this warning. Maybe detect at configure time
and suppress (option 3)?



diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index 3bf48bcdec..64d8258529 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -134,6 +134,22 @@ extern int daemon(int, int);
 #define assert(x)  g_assert(x)
 #endif
 
+/*
+ * GCC 8.0 declared war on strncpy. Admittedly it's a tricky interface
+ * with unintuitive semantics, but we use it widely.
+ */
+#if defined(__GNUC__) && __GNUC__ >= 8
+static inline void qemu_strncpy(char *to, const char *from, int n)
+{
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wstringop-truncation"
+strncpy(to, from, n);
+#pragma GCC diagnostic pop
+}
+
+#define strncpy qemu_strncpy
+#endif
+
 /*
  * According to waitpid man page:
  * WCOREDUMP

[Qemu-block] sheepdog build warning

[Michael S. Tsirkin]

mingw32 build on fedora fails with this warning:

/scm/qemu/block/sheepdog.c: In function 'find_vdi_name':
/scm/qemu/block/sheepdog.c:1239:5: error: 'strncpy' specified bound 256 equals 
destination size [-Werror=stringop-truncation]
 strncpy(buf + SD_MAX_VDI_LEN, tag, SD_MAX_VDI_TAG_LEN);
 ^~
cc1: all warnings being treated as errors
make: *** [/scm/qemu/rules.mak:69: block/sheepdog.o] Error 1
make: *** Waiting for unfinished jobs


Reading the code one sees it's working as intended:


static int find_vdi_name(BDRVSheepdogState *s, const char *filename,
 uint32_t snapid, const char *tag, uint32_t *vid,
 bool lock, Error **errp)
{
int ret, fd;
SheepdogVdiReq hdr;
SheepdogVdiRsp *rsp = (SheepdogVdiRsp *)&hdr;
unsigned int wlen, rlen = 0;
char buf[SD_MAX_VDI_LEN + SD_MAX_VDI_TAG_LEN];

fd = connect_to_sdog(s, errp);
if (fd < 0) {
return fd;
}

/* This pair of strncpy calls ensures that the buffer is zero-filled,
 * which is desirable since we'll soon be sending those bytes, and
 * don't want the send_req to read uninitialized data.
 */
strncpy(buf, filename, SD_MAX_VDI_LEN);
strncpy(buf + SD_MAX_VDI_LEN, tag, SD_MAX_VDI_TAG_LEN);


.

}



so this seems to be the case of GCC developers deciding that
strncpy is simply a bad API and a correct use of it should be
warned against.

I propose either

1. simply adding

#pragma GCC diagnostic ignored "-Wstringop-truncation"

in osdep.

2. adding an inline wrapper with said pragma in there.

3. -Wno-stringop-truncation is the makefile

Thoughts?

-- 
MST

[Qemu-block] [PATCH v11 04/10] block: introduce BDRV_REQ_ALLOCATE flag

[Anton Nefedov]

The flag is supposed to indicate that the region of the disk image has
to be sufficiently allocated so it reads as zeroes.

The call with the flag set must return -ENOTSUP if allocation cannot
be done efficiently.
This has to be made sure of by both
  - the drivers that support the flag
  - and the common block layer (so it will not fall back to any slowpath
(like writing zero buffers) in case the driver does not support
the flag).

Signed-off-by: Anton Nefedov 
---
 include/block/block.h | 10 +-
 include/block/block_int.h |  3 ++-
 block/io.c| 14 +-
 3 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index f70a843b72..643d32f4b8 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -83,8 +83,16 @@ typedef enum {
  */
 BDRV_REQ_SERIALISING= 0x80,
 
+/*
+ * The BDRV_REQ_ALLOCATE flag is used to indicate that the driver has to
+ * efficiently allocate the space so it reads as zeroes, or return an 
error.
+ * If this flag is set then BDRV_REQ_ZERO_WRITE must also be set.
+ * This flag cannot be set together with BDRV_REQ_MAY_UNMAP.
+ */
+BDRV_REQ_ALLOCATE   = 0x100,
+
 /* Mask of valid flags */
-BDRV_REQ_MASK   = 0xff,
+BDRV_REQ_MASK   = 0x1ff,
 } BdrvRequestFlags;
 
 typedef struct BlockSizes {
diff --git a/include/block/block_int.h b/include/block/block_int.h
index f605622216..833129d912 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -724,7 +724,8 @@ struct BlockDriverState {
  * their children. */
 unsigned int supported_write_flags;
 /* Flags honored during pwrite_zeroes (so far: BDRV_REQ_FUA,
- * BDRV_REQ_MAY_UNMAP, BDRV_REQ_WRITE_UNCHANGED) */
+ * BDRV_REQ_MAY_UNMAP, BDRV_REQ_WRITE_UNCHANGED, BDRV_REQ_ALLOCATE)
+ */
 unsigned int supported_zero_flags;
 
 /* the following member gives a name to every node on the bs graph. */
diff --git a/block/io.c b/block/io.c
index bd9d688f8b..66006a089d 100644
--- a/block/io.c
+++ b/block/io.c
@@ -1534,7 +1534,7 @@ static int coroutine_fn 
bdrv_co_do_pwrite_zeroes(BlockDriverState *bs,
 assert(!bs->supported_zero_flags);
 }
 
-if (ret == -ENOTSUP) {
+if (ret == -ENOTSUP && !(flags & BDRV_REQ_ALLOCATE)) {
 /* Fall back to bounce buffer if write zeroes is unsupported */
 BdrvRequestFlags write_flags = flags & ~BDRV_REQ_ZERO_WRITE;
 
@@ -1773,6 +1773,9 @@ static int coroutine_fn bdrv_co_do_zero_pwritev(BdrvChild 
*child,
 
 assert(flags & BDRV_REQ_ZERO_WRITE);
 if (head_padding_bytes || tail_padding_bytes) {
+if (flags & BDRV_REQ_ALLOCATE) {
+return -ENOTSUP;
+}
 buf = qemu_blockalign(bs, align);
 iov = (struct iovec) {
 .iov_base   = buf,
@@ -1858,6 +1861,9 @@ int coroutine_fn bdrv_co_pwritev(BdrvChild *child,
 bool use_local_qiov = false;
 int ret;
 
+assert(!((flags & BDRV_REQ_ALLOCATE) && (flags & BDRV_REQ_MAY_UNMAP)));
+assert(!((flags & BDRV_REQ_ALLOCATE) && !(flags & BDRV_REQ_ZERO_WRITE)));
+
 trace_bdrv_co_pwritev(child->bs, offset, bytes, flags);
 
 if (!bs->drv) {
@@ -1980,6 +1986,12 @@ int coroutine_fn bdrv_co_pwrite_zeroes(BdrvChild *child, 
int64_t offset,
 {
 trace_bdrv_co_pwrite_zeroes(child->bs, offset, bytes, flags);
 
+if ((flags & BDRV_REQ_ALLOCATE) &&
+!(child->bs->supported_zero_flags & BDRV_REQ_ALLOCATE))
+{
+return -ENOTSUP;
+}
+
 if (!(child->bs->open_flags & BDRV_O_UNMAP)) {
 flags &= ~BDRV_REQ_MAY_UNMAP;
 }
-- 
2.17.1

[Qemu-block] [PATCH v11 01/10] mirror: inherit supported write/zero flags

[Anton Nefedov]

Signed-off-by: Anton Nefedov 
Reviewed-by: Alberto Garcia 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
---
 block/mirror.c | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/block/mirror.c b/block/mirror.c
index ab59ad77e8..be52c9be9c 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -1529,8 +1529,12 @@ static void mirror_start_job(const char *job_id, 
BlockDriverState *bs,
 mirror_top_bs->implicit = true;
 }
 mirror_top_bs->total_sectors = bs->total_sectors;
-mirror_top_bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED;
-mirror_top_bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED;
+mirror_top_bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
+(BDRV_REQ_FUA & bs->supported_write_flags);
+mirror_top_bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED |
+((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP)
+ & bs->supported_zero_flags);
+
 bs_opaque = g_new0(MirrorBDSOpaque, 1);
 mirror_top_bs->opaque = bs_opaque;
 bdrv_set_aio_context(mirror_top_bs, bdrv_get_aio_context(bs));
-- 
2.17.1

[Qemu-block] [PATCH v11 06/10] file-posix: reset fallocate-related flags without CONFIG_FALLOCATE*

[Anton Nefedov]

these flags currently affect nothing without CONFIG_FALLOCATE*, so it's
not a bug. Fixing it makes possible to adjust supported zero flag
BDRV_REQ_ALLOCATE regardless of configuration (in the following patch).

Signed-off-by: Anton Nefedov 
---
 block/file-posix.c | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/block/file-posix.c b/block/file-posix.c
index d8f0b93752..a65e464cbc 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -1488,9 +1488,7 @@ static ssize_t 
handle_aiocb_write_zeroes_block(RawPosixAIOData *aiocb)
 static int handle_aiocb_write_zeroes(void *opaque)
 {
 RawPosixAIOData *aiocb = opaque;
-#if defined(CONFIG_FALLOCATE) || defined(CONFIG_XFS)
 BDRVRawState *s = aiocb->bs->opaque;
-#endif
 #ifdef CONFIG_FALLOCATE
 int64_t len;
 #endif
@@ -1514,6 +1512,8 @@ static int handle_aiocb_write_zeroes(void *opaque)
 }
 s->has_write_zeroes = false;
 }
+#else
+s->has_write_zeroes = false;
 #endif
 
 #ifdef CONFIG_FALLOCATE_PUNCH_HOLE
@@ -1533,6 +1533,8 @@ static int handle_aiocb_write_zeroes(void *opaque)
 s->has_discard = false;
 }
 }
+#else
+s->has_discard = false;
 #endif
 
 #ifdef CONFIG_FALLOCATE
@@ -1546,6 +1548,8 @@ static int handle_aiocb_write_zeroes(void *opaque)
 }
 s->has_fallocate = false;
 }
+#else
+s->has_fallocate = false;
 #endif
 
 return -ENOTSUP;
-- 
2.17.1