date:20180221

On 21/02/2018 01:05, Thomas Huth wrote:
> On 20.02.2018 19:37, Paolo Bonzini wrote:
>> On 20/02/2018 18:40, Thomas Huth wrote:
>>> "-net" is clearly a legacy option. Yet we still use it in almost all
>>> examples in the qemu documentation, and many other spots in the network
>>> chapter. We should make it less prominent that users are not lured into
>>> using it so often anymore. So instead of starting the network chapter with
>>> "-net nic" and documenting "-net " below "-netdev "
>>> everywhere, all the "-net" related documentation is now moved to the end
>>> of the chapter. And the examples are changed to use the "--device" and
>>> "--netdev" options instead of "-net nic -net ".
>>
>> Do we want to change them to "-nic" instead?  The proof is in the
>> pudding, they say, :) and "-nic" is way easier to learn than "-device
>> -netdev".
> 
> While -nic is easier to use than -netdev, I don't think that we should
> put the focus in our main qemu-doc on -nic instead of -netdev. -nic is a
> convenience option, while -netdev is the "architected" way to configure
> network devices. We first should document how to do it "right", and
> teach the user to proper distinguish between emulated guest hardware and
> host network backend (with the old -net command, a lot of people seemed
> to have mixed that up IIRC), and then finally explain -nic on top of it.

Heh, that's a philosophy question regarding the organization of the
whole manual.  Currently the "architected" way is pretty much confined
to docs/qdev-device-use.txt.  The manual is full of uses of -drive or
-hda, and I think it makes sense because honestly that's what users use.
 I should have explained this in the previous message, sorry.

>> And maybe we *should* go the extra mile and deprecate "-net" altogether.
>>  The only case where the newer syntax is a bit more uncomfortable is for
>> "-net nic -net nic -net tap|user", which however does work with "-nic
>> hubport -nic hubport -netdev tap|user,id=x -netdev hubport,netdev=x".
> 
> I'd be glad to add such a deprecation patch to this series - I just
> thought it might have been too early so far, but if you feel confident
> that we can mark it as deprecated, I can spin a v3 with such a patch on
> top...

I can't deny it's going to be a lng deprecation.  But we have to
start somewhere, and -nic is a great start.

I think you should send v3 with the minimal changes required to accept
these patches, and then leave the rest to a separate submission, but of
course you don't have to do it that way.

Thanks,

Paolo

Re: [Qemu-devel] [PATCH] configure: fix sanitizers' test program to mend ASan detection

2018-02-21 Thread Marc-André Lureau

Hi

On Wed, Feb 21, 2018 at 2:03 AM, Emilio G. Cota  wrote:
> Commit 218bb57 ("build-sys: check static linking of UBSAN", 2018-02-13)
> adds a small test program to check whether ubsan works even when
> configuring with --static. This added program is used to
> detect all sanitizers, which breaks ASan's detection since the
> compilation fails with -fsanitize=address, at least with gcc 5.4.0
> and 7.2.0:
>
>   qemu-conf.c: In function ‘main’:
>   qemu-conf.c:3:20: error: integer overflow in expression [-Werror=overflow]
>  return INT32_MIN / -1;
>   ^
>   cc1: all warnings being treated as errors


I sent a patch a few days ago:
"[PATCH 1/6] build-sys: fix -fsanitize=address check"

>
> Fix it by:
>
> - Changing the test program to one that incurs undefined behaviour that isn't
>   detected at compile-time, even with -fsanitize=address.
>
> - To be extra safe (since compilers might evolve and eventually figure out
>   the UB at compile-time), use this newly-added test only when checking
>   for UBSan; use the skeleton otherwise.
>
> Signed-off-by: Emilio G. Cota 
> ---
>  configure | 20 
>  1 file changed, 12 insertions(+), 8 deletions(-)
>
> diff --git a/configure b/configure
> index 913e148..56f647f 100755
> --- a/configure
> +++ b/configure
> @@ -5306,13 +5306,13 @@ fi
>  ##
>  # checks for sanitizers
>
> -# we could use a simple skeleton for flags checks, but this also
> -# detect the static linking issue of ubsan, see also:
> +# Use this program to detect the static linking issue of ubsan; see
>  # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84285
>  cat > $TMPC << EOF
> -#include 
> -int main(void) {
> -  return INT32_MIN / -1;
> +int main(int argc, char **argv) {
> +  int k = 0x7fff;
> +  k += argc;
> +  return 0;
>  }
>  EOF
>
> @@ -5322,12 +5322,16 @@ have_asan_iface_h=no
>  have_asan_iface_fiber=no
>
>  if test "$sanitizers" = "yes" ; then
> +  if compile_prog "$CPU_CFLAGS -Werror -fsanitize=undefined" ""; then
> +have_ubsan=yes
> +  fi
> +
> +  # Use the skeleton for all other sanitizer checks
> +  write_c_skeleton
> +
>if compile_prog "$CPU_CFLAGS -Werror -fsanitize=address" ""; then
>have_asan=yes
>fi
> -  if compile_prog "$CPU_CFLAGS -Werror -fsanitize=undefined" ""; then
> -  have_ubsan=yes
> -  fi
>
>if check_include "sanitizer/asan_interface.h" ; then
>have_asan_iface_h=yes
> --
> 2.7.4
>

[Qemu-devel] SSD virtio-scsi passthrough

2018-02-21 Thread Nitin Gupta

Dear Qemu Team

Please let me know the qemu command for doing ssd virtio-scsi passthrough .
i am able to do the pass through with virsh .but same command when i am
trying with qemu , VM is not coming up

Please let me know how can i proceed further . any help will be appreciated

Regards
nitin

Re: [Qemu-devel] [PATCH v4 4/5] usb-mtp: Introduce write support for MTP objects

2018-02-21 Thread Daniel P . Berrangé

On Tue, Feb 20, 2018 at 05:59:03PM -0500, Bandan Das wrote:
> Allow write operations on behalf of the initiator. The
> precursor to write is the sending of the write metadata
> that consists of the ObjectInfo dataset. This patch introduces
> a flag that is set when the responder is ready to receive
> write data based on a previous SendObjectInfo operation by
> the initiator (The SendObjectInfo implementation is in a
> later patch)
> 
> Signed-off-by: Bandan Das 
> ---
>  hw/usb/dev-mtp.c | 152 
> ++-
>  1 file changed, 150 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/usb/dev-mtp.c b/hw/usb/dev-mtp.c
> index 5ef77f3e9f..9b51708614 100644
> --- a/hw/usb/dev-mtp.c
> +++ b/hw/usb/dev-mtp.c
> @@ -47,6 +47,7 @@ enum mtp_code {
>  CMD_GET_OBJECT_INFO= 0x1008,
>  CMD_GET_OBJECT = 0x1009,
>  CMD_DELETE_OBJECT  = 0x100b,
> +CMD_SEND_OBJECT= 0x100d,
>  CMD_GET_PARTIAL_OBJECT = 0x101b,
>  CMD_GET_OBJECT_PROPS_SUPPORTED = 0x9801,
>  CMD_GET_OBJECT_PROP_DESC   = 0x9802,
> @@ -63,9 +64,11 @@ enum mtp_code {
>  RES_INVALID_STORAGE_ID = 0x2008,
>  RES_INVALID_OBJECT_HANDLE  = 0x2009,
>  RES_INVALID_OBJECT_FORMAT_CODE = 0x200b,
> +RES_STORE_FULL = 0x200c,
>  RES_STORE_READ_ONLY= 0x200e,
>  RES_PARTIAL_DELETE = 0x2012,
>  RES_SPEC_BY_FORMAT_UNSUPPORTED = 0x2014,
> +RES_INVALID_OBJECTINFO = 0x2015,
>  RES_INVALID_PARENT_OBJECT  = 0x201a,
>  RES_INVALID_PARAMETER  = 0x201d,
>  RES_SESSION_ALREADY_OPEN   = 0x201e,
> @@ -183,6 +186,14 @@ struct MTPState {
>  int  inotifyfd;
>  QTAILQ_HEAD(events, MTPMonEntry) events;
>  #endif
> +/* Responder is expecting a write operation */
> +bool write_pending;
> +struct {
> +uint32_t parent_handle;
> +uint16_t format;
> +uint32_t size;
> +char *filename;
> +} dataset;
>  };
>  
>  #define TYPE_USB_MTP "usb-mtp"
> @@ -804,6 +815,7 @@ static MTPData *usb_mtp_get_device_info(MTPState *s, 
> MTPControl *c)
>  CMD_GET_OBJECT_HANDLES,
>  CMD_GET_OBJECT_INFO,
>  CMD_DELETE_OBJECT,
> +CMD_SEND_OBJECT,

Seems we should not advertize this for readonly devices.

>  CMD_GET_OBJECT,
>  CMD_GET_PARTIAL_OBJECT,
>  CMD_GET_OBJECT_PROPS_SUPPORTED,
> @@ -1378,6 +1390,14 @@ static void usb_mtp_command(MTPState *s, MTPControl *c)
>  nres = 1;
>  res0 = data_in->length;
>  break;
> +case CMD_SEND_OBJECT:
> +if (!s->write_pending) {
> +usb_mtp_queue_result(s, RES_INVALID_OBJECTINFO,
> + c->trans, 0, 0, 0, 0);
> +return;
> +}
> +s->data_out = usb_mtp_data_alloc(c);
> +return;
>  case CMD_GET_OBJECT_PROPS_SUPPORTED:
>  if (c->argv[0] != FMT_UNDEFINED_OBJECT &&
>  c->argv[0] != FMT_ASSOCIATION) {
> @@ -1472,12 +1492,126 @@ static void usb_mtp_cancel_packet(USBDevice *dev, 
> USBPacket *p)
>  fprintf(stderr, "%s\n", __func__);
>  }
>  
> +static void usb_mtp_write_data(MTPState *s)
> +{
> +MTPData *d = s->data_out;
> +MTPObject *parent =
> +usb_mtp_object_lookup(s, s->dataset.parent_handle);
> +char *path = NULL;
> +int rc = -1;
> +mode_t mask = 0644;
> +
> +assert(d != NULL);
> +


Somewhere in here should surely be validating the "readonly" flag.

> +if (parent == NULL || !s->write_pending) {
> +usb_mtp_queue_result(s, RES_INVALID_OBJECTINFO, d->trans,
> + 0, 0, 0, 0);
> +return;
> +}
> +

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH v4 5/5] usb-mtp: Advertise SendObjectInfo for write support

2018-02-21 Thread Daniel P . Berrangé

On Tue, Feb 20, 2018 at 05:59:04PM -0500, Bandan Das wrote:
> This patch implements a dummy ObjectInfo structure so that
> it's easy to typecast the incoming data. If the metadata is
> valid, write_pending is set. Also, the incoming filename
> is utf-16, so, instead of depending on external libraries, just
> implement a simple function to get the filename
> 
> Signed-off-by: Bandan Das 
> ---
>  hw/usb/dev-mtp.c | 132 
> ++-
>  1 file changed, 130 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/usb/dev-mtp.c b/hw/usb/dev-mtp.c
> index 9b51708614..086296f415 100644
> --- a/hw/usb/dev-mtp.c
> +++ b/hw/usb/dev-mtp.c
> @@ -47,6 +47,7 @@ enum mtp_code {
>  CMD_GET_OBJECT_INFO= 0x1008,
>  CMD_GET_OBJECT = 0x1009,
>  CMD_DELETE_OBJECT  = 0x100b,
> +CMD_SEND_OBJECT_INFO   = 0x100c,
>  CMD_SEND_OBJECT= 0x100d,
>  CMD_GET_PARTIAL_OBJECT = 0x101b,
>  CMD_GET_OBJECT_PROPS_SUPPORTED = 0x9801,
> @@ -67,8 +68,10 @@ enum mtp_code {
>  RES_STORE_FULL = 0x200c,
>  RES_STORE_READ_ONLY= 0x200e,
>  RES_PARTIAL_DELETE = 0x2012,
> +RES_STORE_NOT_AVAILABLE= 0x2013,
>  RES_SPEC_BY_FORMAT_UNSUPPORTED = 0x2014,
>  RES_INVALID_OBJECTINFO = 0x2015,
> +RES_DESTINATION_UNSUPPORTED= 0x2020,
>  RES_INVALID_PARENT_OBJECT  = 0x201a,
>  RES_INVALID_PARAMETER  = 0x201d,
>  RES_SESSION_ALREADY_OPEN   = 0x201e,
> @@ -196,6 +199,34 @@ struct MTPState {
>  } dataset;
>  };
>  
> +/*
> + * ObjectInfo dataset received from initiator
> + * Fields we don't care about are ignored
> + */
> +typedef struct {
> +uint32_t storage_id; /*unused*/
> +uint16_t format;
> +uint16_t protection_status; /*unused*/
> +uint32_t size;
> +uint16_t thumb_format; /*unused*/
> +uint32_t thumb_comp_sz; /*unused*/
> +uint32_t thumb_pix_width; /*unused*/
> +uint32_t thumb_pix_height; /*unused*/
> +uint32_t image_pix_width; /*unused*/
> +uint32_t image_pix_height; /*unused*/
> +uint32_t image_bit_depth; /*unused*/
> +uint32_t parent; /*unused*/
> +uint16_t assoc_type;
> +uint32_t assoc_desc;
> +uint32_t seq_no; /*unused*/
> +uint8_t length; /*part of filename field*/
> +uint16_t filename[0];
> +char date_created[0]; /*unused*/
> +char date_modified[0]; /*unused*/
> +char keywords[0]; /*unused*/
> +/* string and other data follows */
> +} QEMU_PACKED ObjectInfo;
> +
>  #define TYPE_USB_MTP "usb-mtp"
>  #define USB_MTP(obj) OBJECT_CHECK(MTPState, (obj), TYPE_USB_MTP)
>  
> @@ -437,7 +468,6 @@ static MTPObject *usb_mtp_add_child(MTPState *s, 
> MTPObject *o,
>  return child;
>  }
>  
> -#ifdef CONFIG_INOTIFY1
>  static MTPObject *usb_mtp_object_lookup_name(MTPObject *parent,
>   char *name, int len)
>  {
> @@ -452,6 +482,7 @@ static MTPObject *usb_mtp_object_lookup_name(MTPObject 
> *parent,
>  return NULL;
>  }
>  
> +#ifdef CONFIG_INOTIFY1
>  static MTPObject *usb_mtp_object_lookup_wd(MTPState *s, int wd)
>  {
>  MTPObject *iter;
> @@ -815,6 +846,7 @@ static MTPData *usb_mtp_get_device_info(MTPState *s, 
> MTPControl *c)
>  CMD_GET_OBJECT_HANDLES,
>  CMD_GET_OBJECT_INFO,
>  CMD_DELETE_OBJECT,
> +CMD_SEND_OBJECT_INFO,

Same question about filtering this out for read-only devices,
and somewhere else in this patch validating it too.

>  CMD_SEND_OBJECT,
>  CMD_GET_OBJECT,
>  CMD_GET_PARTIAL_OBJECT,
> @@ -1243,7 +1275,7 @@ static void usb_mtp_object_delete(MTPState *s, uint32_t 
> handle,
>  static void usb_mtp_command(MTPState *s, MTPControl *c)
>  {
>  MTPData *data_in = NULL;
> -MTPObject *o;
> +MTPObject *o = NULL;
>  uint32_t nres = 0, res0 = 0;
>  
>  /* sanity checks */
> @@ -1390,6 +1422,37 @@ static void usb_mtp_command(MTPState *s, MTPControl *c)
>  nres = 1;
>  res0 = data_in->length;
>  break;
> +case CMD_SEND_OBJECT_INFO:
> +/* First parameter points to storage id or is 0 */
> +if (c->argv[0] && (c->argv[0] != QEMU_STORAGE_ID)) {
> +usb_mtp_queue_result(s, RES_STORE_NOT_AVAILABLE, c->trans,
> + 0, 0, 0, 0);
> +} else if (c->argv[1] && !c->argv[0]) {
> +/* If second parameter is specified, first must also be 
> specified */
> +usb_mtp_queue_result(s, RES_DESTINATION_UNSUPPORTED, c->trans,
> + 0, 0, 0, 0);
> +} else {
> +uint32_t handle = c->argv[1];
> +if (handle == 0x || handle == 0) {
> +/* root object */
> +o = QTAILQ_FIRST(&s->objects);
> +} else {
> +o = usb_mtp_object_lookup(s, handle);
> +}
> +if (o == NULL)

Re: [Qemu-devel] [PATCH v4 3/5] usb-mtp: Support delete of mtp objects

2018-02-21 Thread Daniel P . Berrangé

On Tue, Feb 20, 2018 at 05:59:02PM -0500, Bandan Das wrote:
> Write of existing objects by the initiator is acheived by
> making a temporary buffer with the new changes, deleting the
> old file and then writing a new file with the same name.
> 
> Also, add a "readonly" property which needs to be set to false
> for deletion to work.
> 
> Signed-off-by: Bandan Das 
> ---
>  hw/usb/dev-mtp.c | 123 
> +++
>  1 file changed, 123 insertions(+)
> 
> diff --git a/hw/usb/dev-mtp.c b/hw/usb/dev-mtp.c
> index 63f8f3b90b..5ef77f3e9f 100644
> --- a/hw/usb/dev-mtp.c
> +++ b/hw/usb/dev-mtp.c
> @@ -46,6 +46,7 @@ enum mtp_code {
>  CMD_GET_OBJECT_HANDLES = 0x1007,
>  CMD_GET_OBJECT_INFO= 0x1008,
>  CMD_GET_OBJECT = 0x1009,
> +CMD_DELETE_OBJECT  = 0x100b,
>  CMD_GET_PARTIAL_OBJECT = 0x101b,
>  CMD_GET_OBJECT_PROPS_SUPPORTED = 0x9801,
>  CMD_GET_OBJECT_PROP_DESC   = 0x9802,
> @@ -62,6 +63,8 @@ enum mtp_code {
>  RES_INVALID_STORAGE_ID = 0x2008,
>  RES_INVALID_OBJECT_HANDLE  = 0x2009,
>  RES_INVALID_OBJECT_FORMAT_CODE = 0x200b,
> +RES_STORE_READ_ONLY= 0x200e,
> +RES_PARTIAL_DELETE = 0x2012,
>  RES_SPEC_BY_FORMAT_UNSUPPORTED = 0x2014,
>  RES_INVALID_PARENT_OBJECT  = 0x201a,
>  RES_INVALID_PARAMETER  = 0x201d,
> @@ -172,6 +175,7 @@ struct MTPState {
>  MTPControl   *result;
>  uint32_t session;
>  uint32_t next_handle;
> +bool readonly;
>  
>  QTAILQ_HEAD(, MTPObject) objects;
>  #ifdef CONFIG_INOTIFY1
> @@ -799,6 +803,7 @@ static MTPData *usb_mtp_get_device_info(MTPState *s, 
> MTPControl *c)
>  CMD_GET_NUM_OBJECTS,
>  CMD_GET_OBJECT_HANDLES,
>  CMD_GET_OBJECT_INFO,
> +CMD_DELETE_OBJECT,

Should we not advertize this in the first place if the device is readonly.

>  CMD_GET_OBJECT,
>  CMD_GET_PARTIAL_OBJECT,
>  CMD_GET_OBJECT_PROPS_SUPPORTED,
> @@ -1113,6 +1118,116 @@ static MTPData 
> *usb_mtp_get_object_prop_value(MTPState *s, MTPControl *c,
>  return d;
>  }
>  
> +/* Return correct return code for a delete event */
> +enum {
> +ALL_DELETE,
> +PARTIAL_DELETE,
> +READ_ONLY,
> +};
> +
> +/* Assumes that children, if any, have been already freed */
> +static void usb_mtp_object_free_one(MTPState *s, MTPObject *o)
> +{
> +#ifndef CONFIG_INOTIFY1
> +assert(o->nchildren == 0);
> +QTAILQ_REMOVE(&s->objects, o, next);
> +g_free(o->name);
> +g_free(o->path);
> +g_free(o);
> +#endif
> +}
> +
> +static int usb_mtp_deletefn(MTPState *s, MTPObject *o, uint32_t trans)
> +{
> +MTPObject *iter, *iter2;
> +bool partial_delete = false;
> +bool success = false;
> +
> +/*
> + * TODO: Add support for Protection Status
> + */
> +
> +QLIST_FOREACH(iter, &o->children, list) {
> +if (iter->format == FMT_ASSOCIATION) {
> +QLIST_FOREACH(iter2, &iter->children, list) {
> +usb_mtp_deletefn(s, iter2, trans);
> +}
> +}
> +}
> +
> +if (o->format == FMT_UNDEFINED_OBJECT) {
> +if (remove(o->path)) {
> +partial_delete = true;
> +} else {
> +usb_mtp_object_free_one(s, o);
> +success = true;
> +}
> +}
> +
> +if (o->format == FMT_ASSOCIATION) {
> +if (rmdir(o->path)) {
> +partial_delete = true;
> +} else {
> +usb_mtp_object_free_one(s, o);
> +success = true;
> +}
> +}
> +
> +if (success && partial_delete) {
> +return PARTIAL_DELETE;
> +}
> +if (!success && partial_delete) {
> +return READ_ONLY;
> +}
> +return ALL_DELETE;
> +}
> +
> +static void usb_mtp_object_delete(MTPState *s, uint32_t handle,
> +  uint32_t format_code, uint32_t trans)
> +{
> +MTPObject *o;
> +int ret;
> +
> +/* Return error if store is read-only */
> +if (!FLAG_SET(s, MTP_FLAG_WRITABLE)) {
> +usb_mtp_queue_result(s, RES_STORE_READ_ONLY,
> + trans, 0, 0, 0, 0);
> +return;
> +}
> +
> +if (format_code != 0) {
> +usb_mtp_queue_result(s, RES_SPEC_BY_FORMAT_UNSUPPORTED,
> + trans, 0, 0, 0, 0);
> +return;
> +}
> +
> +if (handle == 0xFFF) {
> +o = QTAILQ_FIRST(&s->objects);
> +} else {
> +o = usb_mtp_object_lookup(s, handle);
> +}
> +if (o == NULL) {
> +usb_mtp_queue_result(s, RES_INVALID_OBJECT_HANDLE,
> + trans, 0, 0, 0, 0);
> +return;
> +}
> +
> +ret = usb_mtp_deletefn(s, o, trans);
> +if (ret == PARTIAL_DELETE) {
> +usb_mtp_queue_result(s, RES_PARTIAL_DELETE,
> + trans, 0, 0, 0, 0);
> +return;
> +} else if (ret == RE

Re: [Qemu-devel] [PATCH 1/2] qcow2: Prefer byte-based calls into bs->file

On Tue 20 Feb 2018 11:24:58 PM CET, Eric Blake wrote:
> diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
> index d46b69d7f34..3fefeb3dc50 100644
> --- a/block/qcow2-refcount.c
> +++ b/block/qcow2-refcount.c
> @@ -2310,8 +2310,8 @@ write_refblocks:
>  on_disk_refblock = (void *)((char *) *refcount_table +
>  refblock_index * s->cluster_size);
>
> -ret = bdrv_write(bs->file, refblock_offset / BDRV_SECTOR_SIZE,
> - on_disk_refblock, s->cluster_sectors);
> +ret = bdrv_pwrite(bs->file, refblock_offset,
> + on_disk_refblock, s->cluster_size);

It looks like the second line is not properly indented. I think you can
also move on_disk_refblock to the previous line.

Otherwise the patch looks good.

Reviewed-by: Alberto Garcia 

Berto

Re: [Qemu-devel] [RFC PATCH qemu] qmp: Add qom-list-properties to list QOM object properties

On 21/02/2018 04:36, Alexey Kardashevskiy wrote:
> On 19/01/18 16:09, Alexey Kardashevskiy wrote:
>> There is already 'device-list-properties' which does most of the job,
>> however it does not handle everything returned by qom-list-types such
>> as machines as they inherit directly from TYPE_OBJECT and not TYPE_DEVICE.
>>
>> This adds a new qom-list-properties command which prints properties
>> of a specific class and its instance. It is pretty much a simplified copy
>> of the device-list-properties handler.
>>
>> Since it creates an object instance, device properties should appear
>> in the output as they are copied to QOM properties at the instance_init
>> hook.
>>
>> Signed-off-by: Alexey Kardashevskiy 
> 
> So is it ack or nack for the patch? Whose area is this? Thanks,

I think Markus, but I can queue it too because he's on leave.  Can you
please resubmit it?

Paolo

Re: [Qemu-devel] [PATCH 2/2] qcow2: Avoid memory over-allocation on compressed images

On Tue 20 Feb 2018 11:24:59 PM CET, Eric Blake wrote:

I was also preparing a patch to change this, but you arrived first :-)

> So, it's time to cut back on the waste.  A compressed cluster
> will NEVER occupy more than an uncompressed cluster (okay, gzip
> DOES document that because the compression stream adds metadata,
> and because of the pigeonhole principle, there are worst case
> scenarios where attempts to compress will actually inflate an
> image - but in those cases, we would just write the cluster
> uncompressed instead of inflating it).  And as that is a smaller
> amount of memory, we can get by with the simpler g_malloc.

> -if (!s->cluster_cache) {
> -s->cluster_cache = g_malloc(s->cluster_size);
> +assert(!s->cluster_cache);
> +s->cluster_data = g_try_malloc(s->cluster_size);
> +s->cluster_cache = g_try_malloc(s->cluster_size);

There's a few things here:

- QEMU won't write compressed data if the size is >= s->cluster_size
  (there's an explicit check for that in qcow2_co_pwritev_compressed())

- The size field of the compressed cluster descriptor *does* allow
  larger sizes, so you can't simply read csize bytes into
  s->cluster_data becuase you could cause a buffer overflow.

- Solution a: check that csize < s->cluster_size and return an error if
  it's not. However! although QEMU won't produce an image with a
  compressed cluster that is larger than the uncompressed one, the qcow2
  on-disk format in principle allows for that, so arguably we should
  accept it.

- Solution b: the width of the 'compressed cluster size' field is
  (cluster_bits - 8), that's (cluster_size / 256) sectors. Since the
  size of each sector is 512 bytes, the maximum possible size that the
  field can store is (cluster_size * 2) bytes. So allocate that amount
  of space for s->cluster_data, read the data as it is on disk and let
  the decompression code return an error if the data is indeed
  corrupted or it doesn't fit in the output buffer.

Berto

Re: [Qemu-devel] [qemu-web PATCH] Add a blog post documenting Spectre/Meltdown options for QEMU 2.11.1

On 16/02/2018 12:57, Dr. David Alan Gilbert wrote:
> It's a bit weird to end up pointing to them to a lkml post;
> Paolo: Any ideas on anything better to say?

IBRS is really that much slower than retpolines on pre-Skylake machines,
so it's a hard call to tell people to use it.  It looks like Intel is
going to add Yet Another Bit to clean up the Skylake mess; in the
meanwhile I would just use retpolines even on Skylake and newer
machines, because the Spectre attacks are much more theoretical than
Meltdown, but it's good to point out the issue for the paranoid folks.

Paolo

[Qemu-devel] [PATCH v3 5/7] net: Remove the deprecated way of dumping network packets

"-net dump" has been marked as deprecated since QEMU v2.10, since it
only works with the deprecated 'vlan' parameter (or hubs). Network
dumping should be done with "-object filter-dump" nowadays instead.
Since nobody complained so far about the deprecation message, let's
finally get rid of "-net dump" now.

Signed-off-by: Thomas Huth 
---
 net/dump.c  | 102 ++--
 net/net.c   |   9 +
 qapi/net.json   |  29 
 qemu-doc.texi   |   6 
 qemu-options.hx |   8 -
 5 files changed, 9 insertions(+), 145 deletions(-)

diff --git a/net/dump.c b/net/dump.c
index 15df9a4..f16c354 100644
--- a/net/dump.c
+++ b/net/dump.c
@@ -109,7 +109,7 @@ static int net_dump_state_init(DumpState *s, const char 
*filename,
 
 fd = open(filename, O_CREAT | O_TRUNC | O_WRONLY | O_BINARY, 0644);
 if (fd < 0) {
-error_setg_errno(errp, errno, "-net dump: can't open %s", filename);
+error_setg_errno(errp, errno, "net dump: can't open %s", filename);
 return -1;
 }
 
@@ -122,7 +122,7 @@ static int net_dump_state_init(DumpState *s, const char 
*filename,
 hdr.linktype = 1;
 
 if (write(fd, &hdr, sizeof(hdr)) < sizeof(hdr)) {
-error_setg_errno(errp, errno, "-net dump write error");
+error_setg_errno(errp, errno, "net dump write error");
 close(fd);
 return -1;
 }
@@ -136,104 +136,6 @@ static int net_dump_state_init(DumpState *s, const char 
*filename,
 return 0;
 }
 
-/* Dumping via VLAN netclient */
-
-struct DumpNetClient {
-NetClientState nc;
-DumpState ds;
-};
-typedef struct DumpNetClient DumpNetClient;
-
-static ssize_t dumpclient_receive(NetClientState *nc, const uint8_t *buf,
-  size_t size)
-{
-DumpNetClient *dc = DO_UPCAST(DumpNetClient, nc, nc);
-struct iovec iov = {
-.iov_base = (void *)buf,
-.iov_len = size
-};
-
-return dump_receive_iov(&dc->ds, &iov, 1);
-}
-
-static ssize_t dumpclient_receive_iov(NetClientState *nc,
-  const struct iovec *iov, int cnt)
-{
-DumpNetClient *dc = DO_UPCAST(DumpNetClient, nc, nc);
-
-return dump_receive_iov(&dc->ds, iov, cnt);
-}
-
-static void dumpclient_cleanup(NetClientState *nc)
-{
-DumpNetClient *dc = DO_UPCAST(DumpNetClient, nc, nc);
-
-dump_cleanup(&dc->ds);
-}
-
-static NetClientInfo net_dump_info = {
-.type = NET_CLIENT_DRIVER_DUMP,
-.size = sizeof(DumpNetClient),
-.receive = dumpclient_receive,
-.receive_iov = dumpclient_receive_iov,
-.cleanup = dumpclient_cleanup,
-};
-
-int net_init_dump(const Netdev *netdev, const char *name,
-  NetClientState *peer, Error **errp)
-{
-int len, rc;
-const char *file;
-char def_file[128];
-const NetdevDumpOptions *dump;
-NetClientState *nc;
-DumpNetClient *dnc;
-
-assert(netdev->type == NET_CLIENT_DRIVER_DUMP);
-dump = &netdev->u.dump;
-
-assert(peer);
-
-error_report("'-net dump' is deprecated. "
- "Please use '-object filter-dump' instead.");
-
-if (dump->has_file) {
-file = dump->file;
-} else {
-int id;
-int ret;
-
-ret = net_hub_id_for_client(peer, &id);
-assert(ret == 0); /* peer must be on a hub */
-
-snprintf(def_file, sizeof(def_file), "qemu-vlan%d.pcap", id);
-file = def_file;
-}
-
-if (dump->has_len) {
-if (dump->len > INT_MAX) {
-error_setg(errp, "invalid length: %"PRIu64, dump->len);
-return -1;
-}
-len = dump->len;
-} else {
-len = 65536;
-}
-
-nc = qemu_new_net_client(&net_dump_info, peer, "dump", name);
-snprintf(nc->info_str, sizeof(nc->info_str),
- "dump to %s (len=%d)", file, len);
-
-dnc = DO_UPCAST(DumpNetClient, nc, nc);
-rc = net_dump_state_init(&dnc->ds, file, len, errp);
-if (rc) {
-qemu_del_net_client(nc);
-}
-return rc;
-}
-
-/* Dumping via filter */
-
 #define TYPE_FILTER_DUMP "filter-dump"
 
 #define FILTER_DUMP(obj) \
diff --git a/net/net.c b/net/net.c
index dd80f1b..cbd553d 100644
--- a/net/net.c
+++ b/net/net.c
@@ -63,7 +63,6 @@ static QTAILQ_HEAD(, NetClientState) net_clients;
 const char *host_net_devices[] = {
 "tap",
 "socket",
-"dump",
 #ifdef CONFIG_NET_BRIDGE
 "bridge",
 #endif
@@ -967,7 +966,6 @@ static int (* const 
net_client_init_fun[NET_CLIENT_DRIVER__MAX])(
 #ifdef CONFIG_NETMAP
 [NET_CLIENT_DRIVER_NETMAP]= net_init_netmap,
 #endif
-[NET_CLIENT_DRIVER_DUMP]  = net_init_dump,
 #ifdef CONFIG_NET_BRIDGE
 [NET_CLIENT_DRIVER_BRIDGE]= net_init_bridge,
 #endif
@@ -993,8 +991,7 @@ static int net_client_init1(const void *object, bool 
is_netdev, Error **errp)
 netdev = object;
 name = netdev->id;
 
-if (netdev->type == NET_CLIENT_DRIVER_DUMP ||
-netdev->type == NET_CLIENT_DRI

[Qemu-devel] [PATCH v3 0/7] Improvements and clean-ups related to -net

"-net" is a legacy option that often causes confusion and
misconfigurations for the users, since most people are not aware
of the underlying "vlan" (i.e. hub) concept that is used for this
parameter. The prefered way of configuring your network stack is
to use "--netdev" instead, which gives you a clean 1:1 connection
between your emulated guest hardware and the host network backend.

However, there are two reasons why we could not completely deprecate
"-net" yet:

1) Convenience:
In some cases, it's more convenient to use "-net" instead of "--netdev",
e.g. if you just want to have a "tap" network connection, it's faster
to type "-net nic -net tap" instead of "--device e1000,netdev=n1
--netdev tap,id=n1".

2) On-board NICs:
Currently the "-net nic" parameter is the only way to configure on-
board NICs on certain (embedded) machines via the nd_table[] array.

So beside some generic clean-ups and removal of code that has been
marked as deprecated since QEMU 2.10 already, this patch series intro-
duces a new parameter "--nic" (in patch 7) which should be able to re-
place "-net" in the long run completely: This new convenience parameter
can be used to configure the default/on-board guest HW together with a
host network backend in a very compact way. To configure a tap backend
for the default NIC, you just have to type "--nic tap" here for example.

Note that "-net" itself is not marked as deprecated yet - that is
subject to a later patch, since it likely needs some additional
discussion first (or we might rather want to wait for some more
releases first until --nic has been properly established instead)...

v3:
- Changed the comments in qapi/net.json in patch 5/7 according
  to the suggestion from Eric. Dropped the Reviewed-bys from this
  patch since the text changed a bit now.
- Dropped the final patch 8 with the updates to qemu-doc about
  making -net less prominent. I'll resubmit that patch together
  with a patch that deprecates "-net" in a separate patch series
  instead (which will likely both need some more discussion first).

v2:
- Renamed "-n" to "--nic" (suggested by Paolo)
- Improved the QAPI comment about the removal of 'dump (suggested by Eric)
- exit(0) instead of exit(1) after showing the "-netdev help" (Eric)
- Improved the documenation for hubport a little bit (suggested by Paolo)

Thomas Huth (7):
  net: Move error reporting from net_init_client/netdev to the calling
site
  net: List available netdevs with "-netdev help"
  net: Only show vhost-user in the help text if CONFIG_POSIX is defined
  net: Make net_client_init() static
  net: Remove the deprecated way of dumping network packets
  net: Remove the deprecated 'host_net_add' and 'host_net_remove' HMP
commands
  net: Add a new convenience option "--nic" to configure
default/on-board NICs

 hmp-commands.hx |  30 --
 hmp.h   |   3 -
 include/net/net.h   |   4 +-
 include/sysemu/sysemu.h |   1 +
 monitor.c   |  61 
 net/dump.c  | 102 +
 net/net.c   | 239 +++-
 qapi/net.json   |  29 ++
 qemu-doc.texi   |  16 
 qemu-options.hx |  48 +++---
 tests/test-hmp.c|   2 -
 vl.c|  10 +-
 12 files changed, 170 insertions(+), 375 deletions(-)

-- 
1.8.3.1

[Qemu-devel] [PATCH v3 1/7] net: Move error reporting from net_init_client/netdev to the calling site

It looks strange that net_init_client() and net_init_netdev() both
take an "Error **errp" parameter, but then do the error reporting
with "error_report_err(local_err)" on their own. Let's move the
error reporting to the calling site instead to simplify this code
a little bit.

Reviewed-by: Eric Blake 
Reviewed-by: Paolo Bonzini 
Signed-off-by: Thomas Huth 
---
 include/net/net.h |  2 +-
 net/net.c | 29 +
 vl.c  |  3 ++-
 3 files changed, 8 insertions(+), 26 deletions(-)

diff --git a/include/net/net.h b/include/net/net.h
index 3fc48e4..bdd4d9f 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -206,7 +206,7 @@ extern const char *legacy_bootp_filename;
 
 int net_client_init(QemuOpts *opts, bool is_netdev, Error **errp);
 int net_client_parse(QemuOptsList *opts_list, const char *str);
-int net_init_clients(void);
+int net_init_clients(Error **errp);
 void net_check_clients(void);
 void net_cleanup(void);
 void hmp_host_net_add(Monitor *mon, const QDict *qdict);
diff --git a/net/net.c b/net/net.c
index 7d42925..e213a61 100644
--- a/net/net.c
+++ b/net/net.c
@@ -1520,46 +1520,27 @@ void net_check_clients(void)
 
 static int net_init_client(void *dummy, QemuOpts *opts, Error **errp)
 {
-Error *local_err = NULL;
-
-net_client_init(opts, false, &local_err);
-if (local_err) {
-error_report_err(local_err);
-return -1;
-}
-
-return 0;
+return net_client_init(opts, false, errp);
 }
 
 static int net_init_netdev(void *dummy, QemuOpts *opts, Error **errp)
 {
-Error *local_err = NULL;
-int ret;
-
-ret = net_client_init(opts, true, &local_err);
-if (local_err) {
-error_report_err(local_err);
-return -1;
-}
-
-return ret;
+return net_client_init(opts, true, errp);
 }
 
-int net_init_clients(void)
+int net_init_clients(Error **errp)
 {
-QemuOptsList *net = qemu_find_opts("net");
-
 net_change_state_entry =
 qemu_add_vm_change_state_handler(net_vm_change_state_handler, NULL);
 
 QTAILQ_INIT(&net_clients);
 
 if (qemu_opts_foreach(qemu_find_opts("netdev"),
-  net_init_netdev, NULL, NULL)) {
+  net_init_netdev, NULL, errp)) {
 return -1;
 }
 
-if (qemu_opts_foreach(net, net_init_client, NULL, NULL)) {
+if (qemu_opts_foreach(qemu_find_opts("net"), net_init_client, NULL, errp)) 
{
 return -1;
 }
 
diff --git a/vl.c b/vl.c
index 9e7235d..698b681 100644
--- a/vl.c
+++ b/vl.c
@@ -4476,7 +4476,8 @@ int main(int argc, char **argv, char **envp)
 
 colo_info_init();
 
-if (net_init_clients() < 0) {
+if (net_init_clients(&err) < 0) {
+error_report_err(err);
 exit(1);
 }
 
-- 
1.8.3.1

[Qemu-devel] [PATCH v3 6/7] net: Remove the deprecated 'host_net_add' and 'host_net_remove' HMP commands

They are deprecated since QEMU v2.10, and so far nobody complained that
these commands are still necessary for any reason - and since you can use
'netdev_add' and 'netdev_remove' instead, there also should not be any
real reason. Since they are also standing in the way for the upcoming
'vlan' clean-up, it's now time to remove them.

Reviewed-by: Paolo Bonzini 
Signed-off-by: Thomas Huth 
---
 hmp-commands.hx  | 30 --
 hmp.h|  3 --
 monitor.c| 61 
 net/net.c| 94 
 qemu-doc.texi| 10 --
 tests/test-hmp.c |  2 --
 6 files changed, 200 deletions(-)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index d26eb41..964eb51 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1291,36 +1291,6 @@ Inject PCIe AER error
 ETEXI
 
 {
-.name   = "host_net_add",
-.args_type  = "device:s,opts:s?",
-.params = "tap|user|socket|vde|netmap|bridge|vhost-user|dump 
[options]",
-.help   = "add host VLAN client (deprecated, use netdev_add 
instead)",
-.cmd= hmp_host_net_add,
-.command_completion = host_net_add_completion,
-},
-
-STEXI
-@item host_net_add
-@findex host_net_add
-Add host VLAN client. Deprecated, please use @code{netdev_add} instead.
-ETEXI
-
-{
-.name   = "host_net_remove",
-.args_type  = "vlan_id:i,device:s",
-.params = "vlan_id name",
-.help   = "remove host VLAN client (deprecated, use netdev_del 
instead)",
-.cmd= hmp_host_net_remove,
-.command_completion = host_net_remove_completion,
-},
-
-STEXI
-@item host_net_remove
-@findex host_net_remove
-Remove host VLAN client. Deprecated, please use @code{netdev_del} instead.
-ETEXI
-
-{
 .name   = "netdev_add",
 .args_type  = "netdev:O",
 .params = 
"[user|tap|socket|vde|bridge|hubport|netmap|vhost-user],id=str[,prop=value][,...]",
diff --git a/hmp.h b/hmp.h
index 1143db4..b897338 100644
--- a/hmp.h
+++ b/hmp.h
@@ -132,9 +132,6 @@ void migrate_set_capability_completion(ReadLineState *rs, 
int nb_args,
const char *str);
 void migrate_set_parameter_completion(ReadLineState *rs, int nb_args,
   const char *str);
-void host_net_add_completion(ReadLineState *rs, int nb_args, const char *str);
-void host_net_remove_completion(ReadLineState *rs, int nb_args,
-const char *str);
 void delvm_completion(ReadLineState *rs, int nb_args, const char *str);
 void loadvm_completion(ReadLineState *rs, int nb_args, const char *str);
 void hmp_rocker(Monitor *mon, const QDict *qdict);
diff --git a/monitor.c b/monitor.c
index 373bb8d..a4f1f28 100644
--- a/monitor.c
+++ b/monitor.c
@@ -3574,67 +3574,6 @@ void migrate_set_parameter_completion(ReadLineState *rs, 
int nb_args,
 }
 }
 
-void host_net_add_completion(ReadLineState *rs, int nb_args, const char *str)
-{
-int i;
-size_t len;
-if (nb_args != 2) {
-return;
-}
-len = strlen(str);
-readline_set_completion_index(rs, len);
-for (i = 0; host_net_devices[i]; i++) {
-if (!strncmp(host_net_devices[i], str, len)) {
-readline_add_completion(rs, host_net_devices[i]);
-}
-}
-}
-
-void host_net_remove_completion(ReadLineState *rs, int nb_args, const char 
*str)
-{
-NetClientState *ncs[MAX_QUEUE_NUM];
-int count, i, len;
-
-len = strlen(str);
-readline_set_completion_index(rs, len);
-if (nb_args == 2) {
-count = qemu_find_net_clients_except(NULL, ncs,
- NET_CLIENT_DRIVER_NONE,
- MAX_QUEUE_NUM);
-for (i = 0; i < MIN(count, MAX_QUEUE_NUM); i++) {
-int id;
-char name[16];
-
-if (net_hub_id_for_client(ncs[i], &id)) {
-continue;
-}
-snprintf(name, sizeof(name), "%d", id);
-if (!strncmp(str, name, len)) {
-readline_add_completion(rs, name);
-}
-}
-return;
-} else if (nb_args == 3) {
-count = qemu_find_net_clients_except(NULL, ncs,
- NET_CLIENT_DRIVER_NIC,
- MAX_QUEUE_NUM);
-for (i = 0; i < MIN(count, MAX_QUEUE_NUM); i++) {
-int id;
-const char *name;
-
-if (ncs[i]->info->type == NET_CLIENT_DRIVER_HUBPORT ||
-net_hub_id_for_client(ncs[i], &id)) {
-continue;
-}
-name = ncs[i]->name;
-if (!strncmp(str, name, len)) {
-readline_add_completion(rs, name);
-}
-}
-return;
-}
-}
-
 static void vm_completion(ReadLineState *rs, const char *str)
 {
 size_

[Qemu-devel] [PATCH v3 2/7] net: List available netdevs with "-netdev help"

Other options like "-chardev" or "-device" feature a nice help text
with the available devices when being called with "help" or "?".
Since it is quite useful, especially if you want to see which network
backends have been compiled into the QEMU binary, let's provide such
a help text for "-netdev", too.

Reviewed-by: Paolo Bonzini 
Reviewed-by: Eric Blake 
Signed-off-by: Thomas Huth 
---
 net/net.c | 37 -
 1 file changed, 36 insertions(+), 1 deletion(-)

diff --git a/net/net.c b/net/net.c
index e213a61..cf07e15 100644
--- a/net/net.c
+++ b/net/net.c
@@ -1086,6 +1086,38 @@ static int net_client_init1(const void *object, bool 
is_netdev, Error **errp)
 return 0;
 }
 
+static void show_netdevs(void)
+{
+int idx;
+const char *available_netdevs[] = {
+"socket",
+"hubport",
+"tap",
+#ifdef CONFIG_SLIRP
+"user",
+#endif
+#ifdef CONFIG_L2TPV3
+"l2tpv3",
+#endif
+#ifdef CONFIG_VDE
+"vde",
+#endif
+#ifdef CONFIG_NET_BRIDGE
+"bridge",
+#endif
+#ifdef CONFIG_NETMAP
+"netmap",
+#endif
+#ifdef CONFIG_POSIX
+"vhost-user",
+#endif
+};
+
+printf("Available netdev backend types:\n");
+for (idx = 0; idx < ARRAY_SIZE(available_netdevs); idx++) {
+puts(available_netdevs[idx]);
+}
+}
 
 int net_client_init(QemuOpts *opts, bool is_netdev, Error **errp)
 {
@@ -1094,7 +1126,10 @@ int net_client_init(QemuOpts *opts, bool is_netdev, 
Error **errp)
 int ret = -1;
 Visitor *v = opts_visitor_new(opts);
 
-{
+if (is_netdev && is_help_option(qemu_opt_get(opts, "type"))) {
+show_netdevs();
+exit(0);
+} else {
 /* Parse convenience option format ip6-net=fec0::0[/64] */
 const char *ip6_net = qemu_opt_get(opts, "ipv6-net");
 
-- 
1.8.3.1

[Qemu-devel] [PATCH v3 3/7] net: Only show vhost-user in the help text if CONFIG_POSIX is defined

According to net/Makefile.objs we only link in the vhost-user code
if CONFIG_POSIX has been set. So the help screen should also only
show this information if CONFIG_POSIX has been defined.

Reviewed-by: Paolo Bonzini 
Signed-off-by: Thomas Huth 
---
 qemu-options.hx | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/qemu-options.hx b/qemu-options.hx
index 8ccd5dc..9ae49a0 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -1998,8 +1998,10 @@ DEF("netdev", HAS_ARG, QEMU_OPTION_netdev,
 "VALE port (created on the fly) called 'name' ('nmname' is 
name of the \n"
 "netmap device, defaults to '/dev/netmap')\n"
 #endif
+#ifdef CONFIG_POSIX
 "-netdev vhost-user,id=str,chardev=dev[,vhostforce=on|off]\n"
 "configure a vhost-user network, backed by a chardev 
'dev'\n"
+#endif
 "-netdev hubport,id=str,hubid=n[,netdev=nd]\n"
 "configure a hub port on QEMU VLAN 'n'\n", QEMU_ARCH_ALL)
 DEF("net", HAS_ARG, QEMU_OPTION_net,
-- 
1.8.3.1

[Qemu-devel] [PATCH v3 4/7] net: Make net_client_init() static

The function is only used within net.c, so there's no need that
this is a global function.

While we're at it, also remove the unused prototype compute_mcast_idx()
(the function has been removed in commit d9caeb09b107e91122d10ba4a08a).

Reviewed-by: Paolo Bonzini 
Signed-off-by: Thomas Huth 
---
 include/net/net.h | 2 --
 net/net.c | 2 +-
 2 files changed, 1 insertion(+), 3 deletions(-)

diff --git a/include/net/net.h b/include/net/net.h
index bdd4d9f..cd1708c 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -204,7 +204,6 @@ extern const char *host_net_devices[];
 extern const char *legacy_tftp_prefix;
 extern const char *legacy_bootp_filename;
 
-int net_client_init(QemuOpts *opts, bool is_netdev, Error **errp);
 int net_client_parse(QemuOptsList *opts_list, const char *str);
 int net_init_clients(Error **errp);
 void net_check_clients(void);
@@ -228,7 +227,6 @@ void qdev_set_nic_properties(DeviceState *dev, NICInfo *nd);
 #define POLYNOMIAL_LE 0xedb88320
 uint32_t net_crc32(const uint8_t *p, int len);
 uint32_t net_crc32_le(const uint8_t *p, int len);
-unsigned compute_mcast_idx(const uint8_t *ep);
 
 #define vmstate_offset_macaddr(_state, _field)   \
 vmstate_offset_array(_state, _field.a, uint8_t,\
diff --git a/net/net.c b/net/net.c
index cf07e15..dd80f1b 100644
--- a/net/net.c
+++ b/net/net.c
@@ -1119,7 +1119,7 @@ static void show_netdevs(void)
 }
 }
 
-int net_client_init(QemuOpts *opts, bool is_netdev, Error **errp)
+static int net_client_init(QemuOpts *opts, bool is_netdev, Error **errp)
 {
 void *object = NULL;
 Error *err = NULL;
-- 
1.8.3.1

[Qemu-devel] [PATCH v3 7/7] net: Add a new convenience option "--nic" to configure default/on-board NICs

The legacy "-net" option can be quite confusing for the users since most
people do not expect to get a "vlan" hub between their emulated guest
hardware and the host backend. But so far, we are also not able to get
rid of "-net" completely, since it is the only way to configure on-board
NICs that can not be instantiated via "-device" yet. It's also a little
bit shorter to type "-net nic -net tap" instead of "-device xyz,netdev=n1
-netdev tap,id=n1".

So what we need is a new convenience option that is shorter to type than
the full -device + -netdev stuff, and which can be used to configure the
on-board NICs that can not be handled via -device yet. Thus this patch now
provides such a new option "--nic": It adds an entry in the nd_table to
configure a on-board / default NIC, creates a host backend and connects
the two directly, without a confusing "vlan" hub inbetween.

Reviewed-by: Paolo Bonzini 
Signed-off-by: Thomas Huth 
---
 include/sysemu/sysemu.h |  1 +
 net/net.c   | 78 +
 qemu-options.hx | 40 +
 vl.c|  7 +
 4 files changed, 120 insertions(+), 6 deletions(-)

diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 77bb3da..66f0761 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -197,6 +197,7 @@ extern QemuOptsList bdrv_runtime_opts;
 extern QemuOptsList qemu_chardev_opts;
 extern QemuOptsList qemu_device_opts;
 extern QemuOptsList qemu_netdev_opts;
+extern QemuOptsList qemu_nic_opts;
 extern QemuOptsList qemu_net_opts;
 extern QemuOptsList qemu_global_opts;
 extern QemuOptsList qemu_mon_opts;
diff --git a/net/net.c b/net/net.c
index 2d05808..0bab269 100644
--- a/net/net.c
+++ b/net/net.c
@@ -1462,6 +1462,67 @@ static int net_init_netdev(void *dummy, QemuOpts *opts, 
Error **errp)
 return net_client_init(opts, true, errp);
 }
 
+/* For the convenience "--nic" parameter */
+static int net_param_nic(void *dummy, QemuOpts *opts, Error **errp)
+{
+char *mac, *nd_id;
+int idx, ret;
+NICInfo *ni;
+const char *type;
+
+type = qemu_opt_get(opts, "type");
+if (type && g_str_equal(type, "none")) {
+return 0;/* Nothing to do, default_net is cleared in vl.c */
+}
+
+idx = nic_get_free_idx();
+if (idx == -1 || nb_nics >= MAX_NICS) {
+error_setg(errp, "no more on-board/default NIC slots available");
+return -1;
+}
+
+if (!type) {
+qemu_opt_set(opts, "type", "user", &error_abort);
+}
+
+ni = &nd_table[idx];
+memset(ni, 0, sizeof(*ni));
+ni->model = qemu_opt_get_del(opts, "model");
+
+/* Create an ID if the user did not specify one */
+nd_id = g_strdup(qemu_opts_id(opts));
+if (!nd_id) {
+nd_id = g_strdup_printf("__org.qemu.nic%i\n", idx);
+qemu_opts_set_id(opts, nd_id);
+}
+
+/* Handle MAC address */
+mac = qemu_opt_get_del(opts, "mac");
+if (mac) {
+ret = net_parse_macaddr(ni->macaddr.a, mac);
+g_free(mac);
+if (ret) {
+error_setg(errp, "invalid syntax for ethernet address");
+return -1;
+}
+if (is_multicast_ether_addr(ni->macaddr.a)) {
+error_setg(errp, "NIC cannot have multicast MAC address");
+return -1;
+}
+}
+qemu_macaddr_default_if_unset(&ni->macaddr);
+
+ret = net_client_init(opts, true, errp);
+if (ret == 0) {
+ni->netdev = qemu_find_netdev(nd_id);
+ni->used = true;
+nb_nics++;
+}
+
+g_free(nd_id);
+return ret;
+}
+
 int net_init_clients(Error **errp)
 {
 net_change_state_entry =
@@ -1474,6 +1535,10 @@ int net_init_clients(Error **errp)
 return -1;
 }
 
+if (qemu_opts_foreach(qemu_find_opts("nic"), net_param_nic, NULL, errp)) {
+return -1;
+}
+
 if (qemu_opts_foreach(qemu_find_opts("net"), net_init_client, NULL, errp)) 
{
 return -1;
 }
@@ -1549,6 +1614,19 @@ QemuOptsList qemu_netdev_opts = {
 },
 };
 
+QemuOptsList qemu_nic_opts = {
+.name = "nic",
+.implied_opt_name = "type",
+.head = QTAILQ_HEAD_INITIALIZER(qemu_nic_opts.head),
+.desc = {
+/*
+ * no elements => accept any params
+ * validation will happen later
+ */
+{ /* end of list */ }
+},
+};
+
 QemuOptsList qemu_net_opts = {
 .name = "net",
 .implied_opt_name = "type",
diff --git a/qemu-options.hx b/qemu-options.hx
index a9249b6..399905e 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -2004,13 +2004,34 @@ DEF("netdev", HAS_ARG, QEMU_OPTION_netdev,
 #endif
 "-netdev hubport,id=str,hubid=n[,netdev=nd]\n"
 "configure a hub port on QEMU VLAN 'n'\n", QEMU_ARCH_ALL)
+DEF("nic", HAS_ARG, QEMU_OPTION_nic,
+"--nic [tap|bridge|"
+#ifdef CONFIG_SLIRP
+"user|"
+#endif
+#ifdef __linux__
+"l2tpv3|"
+#endif
+#ifdef CONFIG_VDE
+"vde|"
+#endif
+#ifdef CONFIG_NETMAP
+

Re: [Qemu-devel] [PATCH 11/27] block: x-blockdev-create QMP command

Am 15.02.2018 um 20:58 hat Eric Blake geschrieben:
> On 02/08/2018 01:23 PM, Kevin Wolf wrote:
> > This adds a synchronous x-blockdev-create QMP command that can create
> > qcow2 images on a given node name.
> > 
> > We don't want to block while creating an image, so this is not the final
> > interface in all aspects, but BlockdevCreateOptionsQcow2 and
> > .bdrv_co_create() are what they actually might look like in the end. In
> > any case, this should be good enough to test whether we interpret
> > BlockdevCreateOptions as we should.
> > 
> > Signed-off-by: Kevin Wolf 
> > ---
> 
> > @@ -3442,6 +3442,18 @@
> > } }
> >   ##
> > +# @x-blockdev-create:
> > +#
> > +# Create an image format on a given node.
> > +# TODO Replace with something asynchronous (block job?)
> 
> We've learned our lesson - don't commit to the final name on an interface
> that we have not yet experimented with.
> 
> > +#
> > +# Since: 2.12
> > +##
> > +{ 'command': 'x-blockdev-create',
> > +  'data': 'BlockdevCreateOptions',
> > +  'boxed': true }
> > +
> 
> Lots of code packed in that little description ;)
> 
> 
> > +++ b/include/block/block_int.h
> > @@ -130,6 +130,8 @@ struct BlockDriver {
> >   int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
> > Error **errp);
> >   void (*bdrv_close)(BlockDriverState *bs);
> > +int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,
> > +   Error **errp);
> 
> I know we haven't been very good in the past, but can you add a comment here
> on the contract that drivers are supposed to obey when implementing this
> callback?

Anything specific you want to see here?

Essentially the meaning of BlockdevCreateOptions depends on the driver
and is documented in the QAPI schema, how Error works is common
knowledge, and I don't see much else to explain here.

I mean, I can add something like "Creates an image. See the QAPI
documentation for BlockdevCreateOptions for details." if you think this
is useful. But is it?

> > +/* Call callback if it exists */
> > +if (!drv->bdrv_co_create) {
> > +error_setg(errp, "Driver does not support blockdev-create");
> 
> Should this error message refer to 'x-blockdev-create' in the short term?

Hm, it would be more correct. On the other hand, I'm almost sure we'd
forget to rename it back when we remove the x- prefix...

Kevin

Re: [Qemu-devel] [PATCH 19/19] mps2-an505: New board model: MPS2 with AN505 Cortex-M33 FPGA image

2018-02-21 Thread Igor Mammedov

On Tue, 20 Feb 2018 18:03:25 +
Peter Maydell  wrote:

> Define a new board model for the MPS2 with an AN505 FPGA image
> containing a Cortex-M33. Since the FPGA images for TrustZone
> cores (AN505, and the similar AN519 for Cortex-M23) have a
> significantly different layout of devices to the non-TrustZone
> images, we use a new source file rather than shoehorning them
> into the existing mps2.c.
> 
> Signed-off-by: Peter Maydell 
> ---
>  hw/arm/Makefile.objs |   1 +
>  hw/arm/mps2-tz.c | 504 
> +++
>  2 files changed, 505 insertions(+)
>  create mode 100644 hw/arm/mps2-tz.c
> 
> diff --git a/hw/arm/Makefile.objs b/hw/arm/Makefile.objs
> index 79cd30bb92..232258160a 100644
> --- a/hw/arm/Makefile.objs
> +++ b/hw/arm/Makefile.objs
> @@ -19,5 +19,6 @@ obj-$(CONFIG_FSL_IMX31) += fsl-imx31.o kzm.o
>  obj-$(CONFIG_FSL_IMX6) += fsl-imx6.o sabrelite.o
>  obj-$(CONFIG_ASPEED_SOC) += aspeed_soc.o aspeed.o
>  obj-$(CONFIG_MPS2) += mps2.o
> +obj-$(CONFIG_MPS2) += mps2-tz.o
>  obj-$(CONFIG_MSF2) += msf2-soc.o msf2-som.o
>  obj-$(CONFIG_IOTKIT) += iotkit.o
> diff --git a/hw/arm/mps2-tz.c b/hw/arm/mps2-tz.c
> new file mode 100644
> index 00..ff414c649c
> --- /dev/null
> +++ b/hw/arm/mps2-tz.c
> @@ -0,0 +1,504 @@
> +/*
> + * ARM V2M MPS2 board emulation, trustzone aware FPGA images
> + *
> + * Copyright (c) 2017 Linaro Limited
> + * Written by Peter Maydell
> + *
> + *  This program is free software; you can redistribute it and/or modify
> + *  it under the terms of the GNU General Public License version 2 or
> + *  (at your option) any later version.
> + */
> +
> +/* The MPS2 and MPS2+ dev boards are FPGA based (the 2+ has a bigger
> + * FPGA but is otherwise the same as the 2). Since the CPU itself
> + * and most of the devices are in the FPGA, the details of the board
> + * as seen by the guest depend significantly on the FPGA image.
> + * This source file covers the following FPGA images, for TrustZone cores:
> + *  "mps2-an505" -- Cortex-M33 as documented in ARM Application Note AN505
> + *
> + * Links to the TRM for the board itself and to the various Application
> + * Notes which document the FPGA images can be found here:
> + * 
> https://developer.arm.com/products/system-design/development-boards/fpga-prototyping-boards/mps2
> + *
> + * Board TRM:
> + * 
> http://infocenter.arm.com/help/topic/com.arm.doc.100112_0200_06_en/versatile_express_cortex_m_prototyping_systems_v2m_mps2_and_v2m_mps2plus_technical_reference_100112_0200_06_en.pdf
> + * Application Note AN505:
> + * http://infocenter.arm.com/help/topic/com.arm.doc.dai0505b/index.html
> + *
> + * The AN505 defers to the Cortex-M33 processor ARMv8M IoT Kit FVP User Guide
> + * (ARM ECM0601256) for the details of some of the device layout:
> + *   
> http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ecm0601256/index.html
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qapi/error.h"
> +#include "qemu/error-report.h"
> +#include "hw/arm/arm.h"
> +#include "hw/arm/armv7m.h"
> +#include "hw/or-irq.h"
> +#include "hw/boards.h"
> +#include "exec/address-spaces.h"
> +#include "sysemu/sysemu.h"
> +#include "hw/misc/unimp.h"
> +#include "hw/char/cmsdk-apb-uart.h"
> +#include "hw/timer/cmsdk-apb-timer.h"
> +#include "hw/misc/mps2-scc.h"
> +#include "hw/misc/mps2-fpgaio.h"
> +#include "hw/arm/iotkit.h"
> +#include "hw/devices.h"
> +#include "net/net.h"
> +#include "hw/core/split-irq.h"
> +
> +typedef enum MPS2TZFPGAType {
> +FPGA_AN505,
> +} MPS2TZFPGAType;
> +
> +typedef struct {
> +MachineClass parent;
> +MPS2TZFPGAType fpga_type;
> +const char *cpu_model;
I don't see it used in this patch, should it be removed?


> +uint32_t scc_id;
> +} MPS2TZMachineClass;
> +
[...]

Re: [Qemu-devel] [PATCH v3 0/7] Improvements and clean-ups related to -net

2018-02-21 Thread Mark Cave-Ayland

On 21/02/18 10:18, Thomas Huth wrote:

"-net" is a legacy option that often causes confusion and
misconfigurations for the users, since most people are not aware
of the underlying "vlan" (i.e. hub) concept that is used for this
parameter. The prefered way of configuring your network stack is
to use "--netdev" instead, which gives you a clean 1:1 connection
between your emulated guest hardware and the host network backend.

However, there are two reasons why we could not completely deprecate
"-net" yet:

1) Convenience:
In some cases, it's more convenient to use "-net" instead of "--netdev",
e.g. if you just want to have a "tap" network connection, it's faster
to type "-net nic -net tap" instead of "--device e1000,netdev=n1
--netdev tap,id=n1".

2) On-board NICs:
Currently the "-net nic" parameter is the only way to configure on-
board NICs on certain (embedded) machines via the nd_table[] array.

So beside some generic clean-ups and removal of code that has been
marked as deprecated since QEMU 2.10 already, this patch series intro-
duces a new parameter "--nic" (in patch 7) which should be able to re-
place "-net" in the long run completely: This new convenience parameter
can be used to configure the default/on-board guest HW together with a
host network backend in a very compact way. To configure a tap backend
for the default NIC, you just have to type "--nic tap" here for example.

Hi Thomas,

This is a great improvement for configuring on-board NICs! I do have a
couple of questions about your patchset based on personal experience:

1) Does the new -nic syntax support multiple on-board NICs? I remember
seeing this on some of the ARM boards I was studying when trying to
implement something similar for SPARC.

2) Is it possible to provide a convenient wrapper function to handle the
logic related to determining whether a specified NIC is on-board or not?

For example take a look at
https://git.qemu.org/?p=qemu.git;a=blob;f=hw/sparc64/sun4u.c;h=da28ab9413efdbe0bf0e1d3bf2b545577b83d88a;hb=a6e0344fa0e09413324835ae122c4cadd7890231#l596
where we have a nice dance to check whether the device specified is
on-board or not, and if it isn't to plug it in as an extra PCI device
instead. The complication here is that nd_tables[] is populated by
default (and also with -net nic -net user) but not when someone
specifies a single card via -netdev.

From memory this was derived from the code in
https://git.qemu.org/?p=qemu.git;a=blob;f=hw/arm/realview.c;h=87cd1e583cd20b6d8a2beeef1cba6977496d4477;hb=a6e0344fa0e09413324835ae122c4cadd7890231#l259
if having another example helps?

ATB,

Mark.

[Qemu-devel] [PULL 05/22] include/fpu/softfloat: implement float16_abs helper

This will be required when expanding the MINMAX() macro for 16
bit/half-precision operations.

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Peter Maydell 

diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h
index f3b9008f78..1d34f2c3eb 100644
--- a/include/fpu/softfloat.h
+++ b/include/fpu/softfloat.h
@@ -265,6 +265,13 @@ static inline int float16_is_zero_or_denormal(float16 a)
 return (float16_val(a) & 0x7c00) == 0;
 }
 
+static inline float16 float16_abs(float16 a)
+{
+/* Note that abs does *not* handle NaN specially, nor does
+ * it flush denormal inputs to zero.
+ */
+return make_float16(float16_val(a) & 0x7fff);
+}
 /*
 | The pattern for a default generated half-precision NaN.
 **/
-- 
2.15.1

[Qemu-devel] [PULL 01/22] fpu/softfloat: implement float16_squash_input_denormal

This will be required when expanding the MINMAX() macro for 16
bit/half-precision operations.

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 
Reviewed-by: Peter Maydell 

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 433c5dad2d..3a4ab1355f 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -3488,6 +3488,21 @@ static float16 roundAndPackFloat16(flag zSign, int zExp,
 return packFloat16(zSign, zExp, zSig >> 13);
 }
 
+/*
+| If `a' is denormal and we are in flush-to-zero mode then set the
+| input-denormal exception and return zero. Otherwise just return the value.
+**/
+float16 float16_squash_input_denormal(float16 a, float_status *status)
+{
+if (status->flush_inputs_to_zero) {
+if (extractFloat16Exp(a) == 0 && extractFloat16Frac(a) != 0) {
+float_raise(float_flag_input_denormal, status);
+return make_float16(float16_val(a) & 0x8000);
+}
+}
+return a;
+}
+
 static void normalizeFloat16Subnormal(uint32_t aSig, int *zExpPtr,
   uint32_t *zSigPtr)
 {
diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h
index 0f96a0edd1..d5e99667b6 100644
--- a/include/fpu/softfloat.h
+++ b/include/fpu/softfloat.h
@@ -277,6 +277,7 @@ void float_raise(uint8_t flags, float_status *status);
 | If `a' is denormal and we are in flush-to-zero mode then set the
 | input-denormal exception and return zero. Otherwise just return the value.
 **/
+float16 float16_squash_input_denormal(float16 a, float_status *status);
 float32 float32_squash_input_denormal(float32 a, float_status *status);
 float64 float64_squash_input_denormal(float64 a, float_status *status);
 
-- 
2.15.1

[Qemu-devel] [PULL 02/22] include/fpu/softfloat: remove USE_SOFTFLOAT_STRUCT_TYPES

It's not actively built and when enabled things fail to compile. I'm
not sure the type-checking is really helping here. Seeing as we "own"
our softfloat now lets remove the cruft.

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 

diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h
index d5e99667b6..52af1412de 100644
--- a/include/fpu/softfloat.h
+++ b/include/fpu/softfloat.h
@@ -103,32 +103,6 @@ enum {
 /*
 | Software IEC/IEEE floating-point types.
 **/
-/* Use structures for soft-float types.  This prevents accidentally mixing
-   them with native int/float types.  A sufficiently clever compiler and
-   sane ABI should be able to see though these structs.  However
-   x86/gcc 3.x seems to struggle a bit, so leave them disabled by default.  */
-//#define USE_SOFTFLOAT_STRUCT_TYPES
-#ifdef USE_SOFTFLOAT_STRUCT_TYPES
-typedef struct {
-uint16_t v;
-} float16;
-#define float16_val(x) (((float16)(x)).v)
-#define make_float16(x) __extension__ ({ float16 f16_val = {x}; f16_val; })
-#define const_float16(x) { x }
-typedef struct {
-uint32_t v;
-} float32;
-/* The cast ensures an error if the wrong type is passed.  */
-#define float32_val(x) (((float32)(x)).v)
-#define make_float32(x) __extension__ ({ float32 f32_val = {x}; f32_val; })
-#define const_float32(x) { x }
-typedef struct {
-uint64_t v;
-} float64;
-#define float64_val(x) (((float64)(x)).v)
-#define make_float64(x) __extension__ ({ float64 f64_val = {x}; f64_val; })
-#define const_float64(x) { x }
-#else
 typedef uint16_t float16;
 typedef uint32_t float32;
 typedef uint64_t float64;
@@ -141,7 +115,6 @@ typedef uint64_t float64;
 #define const_float16(x) (x)
 #define const_float32(x) (x)
 #define const_float64(x) (x)
-#endif
 typedef struct {
 uint64_t low;
 uint16_t high;
-- 
2.15.1

[Qemu-devel] [PULL 10/22] fpu/softfloat: move the extract functions to the top of the file

This is pure code-motion during re-factoring as the helpers will be
needed earlier.

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 
Reviewed-by: Peter Maydell 

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 3a4ab1355f..297e48f5c9 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -132,6 +132,60 @@ static inline flag extractFloat16Sign(float16 a)
 return float16_val(a)>>15;
 }
 
+/*
+| Returns the fraction bits of the single-precision floating-point value `a'.
+**/
+
+static inline uint32_t extractFloat32Frac(float32 a)
+{
+return float32_val(a) & 0x007F;
+}
+
+/*
+| Returns the exponent bits of the single-precision floating-point value `a'.
+**/
+
+static inline int extractFloat32Exp(float32 a)
+{
+return (float32_val(a) >> 23) & 0xFF;
+}
+
+/*
+| Returns the sign bit of the single-precision floating-point value `a'.
+**/
+
+static inline flag extractFloat32Sign(float32 a)
+{
+return float32_val(a) >> 31;
+}
+
+/*
+| Returns the fraction bits of the double-precision floating-point value `a'.
+**/
+
+static inline uint64_t extractFloat64Frac(float64 a)
+{
+return float64_val(a) & LIT64(0x000F);
+}
+
+/*
+| Returns the exponent bits of the double-precision floating-point value `a'.
+**/
+
+static inline int extractFloat64Exp(float64 a)
+{
+return (float64_val(a) >> 52) & 0x7FF;
+}
+
+/*
+| Returns the sign bit of the double-precision floating-point value `a'.
+**/
+
+static inline flag extractFloat64Sign(float64 a)
+{
+return float64_val(a) >> 63;
+}
+
 /*
 | Takes a 64-bit fixed-point value `absZ' with binary point between bits 6
 | and 7, and returns the properly rounded 32-bit integer corresponding to the
@@ -299,39 +353,6 @@ static int64_t roundAndPackUint64(flag zSign, uint64_t 
absZ0,
 return absZ0;
 }
 
-/*
-| Returns the fraction bits of the single-precision floating-point value `a'.
-**/
-
-static inline uint32_t extractFloat32Frac( float32 a )
-{
-
-return float32_val(a) & 0x007F;
-
-}
-
-/*
-| Returns the exponent bits of the single-precision floating-point value `a'.
-**/
-
-static inline int extractFloat32Exp(float32 a)
-{
-
-return ( float32_val(a)>>23 ) & 0xFF;
-
-}
-
-/*
-| Returns the sign bit of the single-precision floating-point value `a'.
-**/
-
-static inline flag extractFloat32Sign( float32 a )
-{
-
-return float32_val(a)>>31;
-
-}
-
 /*
 | If `a' is denormal and we are in flush-to-zero mode then set the
 | input-denormal exception and return zero. Otherwise just return the value.
@@ -492,39 +513,6 @@ static float32
 
 }
 
-/*
-| Returns the fraction bits of the double-precision floating-point value `a'.
-**/
-
-static inline uint64_t extractFloat64Frac( float64 a )
-{
-
-return float64_val(a) & LIT64( 0x000F );
-
-}
-
-/*
-| Returns the exponent bits of the double-precision floating-point value `a'.
-**/
-
-static inline int extractFloat64Exp(float64 a)
-{
-
-return ( float64_val(a)>>52 ) & 0x7FF;
-
-}
-
-/*
-| Returns the sign bit of the double-precision floating-point value `a'.
-*--

[Qemu-devel] [PULL 03/22] fpu/softfloat-types: new header to prevent excessive re-builds

The main culprit here is bswap.h which pulled in softfloat.h so it
could use the types in its CPU_Float* and ldfl/stfql functions. As
bswap.h is very widely included this added a compile dependency every
time we touch softfloat.h. Move the typedefs for each float type into
their own file so we don't re-build the world every time we tweak the
main softfloat.h header.

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 

diff --git a/include/fpu/softfloat-types.h b/include/fpu/softfloat-types.h
new file mode 100644
index 00..8210a94ea1
--- /dev/null
+++ b/include/fpu/softfloat-types.h
@@ -0,0 +1,115 @@
+/*
+ * QEMU float support
+ *
+ * The code in this source file is derived from release 2a of the SoftFloat
+ * IEC/IEEE Floating-point Arithmetic Package. Those parts of the code (and
+ * some later contributions) are provided under that license, as detailed 
below.
+ * It has subsequently been modified by contributors to the QEMU Project,
+ * so some portions are provided under:
+ *  the SoftFloat-2a license
+ *  the BSD license
+ *  GPL-v2-or-later
+ *
+ * This header holds definitions for code that might be dealing with
+ * softfloat types but not need access to the actual library functions.
+ */
+/*
+===
+This C header file is part of the SoftFloat IEC/IEEE Floating-point
+Arithmetic Package, Release 2a.
+
+Written by John R. Hauser.  This work was made possible in part by the
+International Computer Science Institute, located at Suite 600, 1947 Center
+Street, Berkeley, California 94704.  Funding was partially provided by the
+National Science Foundation under grant MIP-9311980.  The original version
+of this code was written as part of a project to build a fixed-point vector
+processor in collaboration with the University of California at Berkeley,
+overseen by Profs. Nelson Morgan and John Wawrzynek.  More information
+is available through the Web page `http://HTTP.CS.Berkeley.EDU/~jhauser/
+arithmetic/SoftFloat.html'.
+
+THIS SOFTWARE IS DISTRIBUTED AS IS, FOR FREE.  Although reasonable effort
+has been made to avoid it, THIS SOFTWARE MAY CONTAIN FAULTS THAT WILL AT
+TIMES RESULT IN INCORRECT BEHAVIOR.  USE OF THIS SOFTWARE IS RESTRICTED TO
+PERSONS AND ORGANIZATIONS WHO CAN AND WILL TAKE FULL RESPONSIBILITY FOR ANY
+AND ALL LOSSES, COSTS, OR OTHER PROBLEMS ARISING FROM ITS USE.
+
+Derivative works are acceptable, even for commercial purposes, so long as
+(1) they include prominent notice that the work is derivative, and (2) they
+include prominent notice akin to these four paragraphs for those parts of
+this code that are retained.
+
+===
+*/
+
+/* BSD licensing:
+ * Copyright (c) 2006, Fabrice Bellard
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright notice,
+ * this list of conditions and the following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above copyright notice,
+ * this list of conditions and the following disclaimer in the documentation
+ * and/or other materials provided with the distribution.
+ *
+ * 3. Neither the name of the copyright holder nor the names of its 
contributors
+ * may be used to endorse or promote products derived from this software 
without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/* Portions of this work are licensed under the terms of the GNU GPL,
+ * version 2 or later. See the COPYING file in the top-level directory.
+ */
+
+#ifndef SOFTFLOAT_TYPES_H
+#define SOFTFLOAT_TYPES_H
+
+/*
+ * Software IEC/IEEE floating-point types.
+ */
+
+typedef uint16_t float16;
+typedef uint32_t float32;
+typedef uint64_t float64;
+#define float16_val(x) (x)
+#define float32_val(x) (x)
+#define float64_val(x) (x)
+#define make_float16(x) (x)
+#define make_float32(x) (x)
+#define make_float64(x) (x)
+#define const_float16(x) (x)
+#define const_float32(x) (x)
+#define const_float64(x) (x)
+typedef

[Qemu-devel] [PULL 06/22] include/fpu/softfloat: implement float16_chs helper

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 
Reviewed-by: Peter Maydell 

diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h
index 1d34f2c3eb..f75aa59100 100644
--- a/include/fpu/softfloat.h
+++ b/include/fpu/softfloat.h
@@ -272,6 +272,15 @@ static inline float16 float16_abs(float16 a)
  */
 return make_float16(float16_val(a) & 0x7fff);
 }
+
+static inline float16 float16_chs(float16 a)
+{
+/* Note that chs does *not* handle NaN specially, nor does
+ * it flush denormal inputs to zero.
+ */
+return make_float16(float16_val(a) ^ 0x8000);
+}
+
 /*
 | The pattern for a default generated half-precision NaN.
 **/
-- 
2.15.1

Re: [Qemu-devel] [PATCH] tests/boot-serial-test: Fix problem with timeout due to dropped characters

On 16/02/2018 07:12, Thomas Huth wrote:
> Commit 92b540dac9fc3a5 introduce a counter to handle the timeouts in a
> better way. But in case ccnt reaches 512, the current read character is
> ignored - and if that character is part of the string that we are looking
> for, the test fails to match the string.
> 
> Almost all of the tests look for a string within the first 512 bytes of
> firmware output, so the problem never triggered there. But the hppa test
> that has been added recently looks for a longer string at the very end of
> a long output, thus there's a chance that we miss a character there so
> that the test fails unexpectedly. Fix it by *not* reading and dropping a
> character if the counter reaches 512.
> 
> Fixes: 92b540dac9fc3a572c7342edd0b073000f5a6abf
> Signed-off-by: Thomas Huth 
> ---
>  @Peter: Since this fixes the problem with running "make check", could
>  you maybe apply this directly to the master branch? Thanks, and sorry
>  for the inconvenience!
> 
>  tests/boot-serial-test.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/boot-serial-test.c b/tests/boot-serial-test.c
> index ea87a80..696f7a3 100644
> --- a/tests/boot-serial-test.c
> +++ b/tests/boot-serial-test.c
> @@ -101,7 +101,7 @@ static void check_guest_output(const testdef_t *test, int 
> fd)
>  /* Poll serial output... Wait at most 60 seconds */
>  for (i = 0; i < 6000; ++i) {
>  ccnt = 0;
> -while ((nbr = read(fd, &ch, 1)) == 1 && ccnt++ < 512) {
> +while (ccnt++ < 512 && (nbr = read(fd, &ch, 1)) == 1) {
>  if (ch == test->expect[pos]) {
>  pos += 1;
>  if (test->expect[pos] == '\0') {
> 

Queued in the meanwhile, thanks.

Paolo

[Qemu-devel] [PULL 07/22] include/fpu/softfloat: implement float16_set_sign helper

Signed-off-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Richard Henderson 

diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h
index f75aa59100..59c06ef192 100644
--- a/include/fpu/softfloat.h
+++ b/include/fpu/softfloat.h
@@ -281,6 +281,11 @@ static inline float16 float16_chs(float16 a)
 return make_float16(float16_val(a) ^ 0x8000);
 }
 
+static inline float16 float16_set_sign(float16 a, int sign)
+{
+return make_float16((float16_val(a) & 0x7fff) | (sign << 15));
+}
+
 /*
 | The pattern for a default generated half-precision NaN.
 **/
-- 
2.15.1

[Qemu-devel] [PULL 00/22] re-factor softfloat and add fp16 functions

The following changes since commit a6e0344fa0e09413324835ae122c4cadd7890231:

  Merge remote-tracking branch 'remotes/kraxel/tags/ui-20180220-pull-request' 
into staging (2018-02-20 14:05:00 +)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-softfloat-refactor-210218-1

for you to fetch changes up to c13bb2da9eedfbc5886c8048df1bc1114b285fb0:

  fpu/softfloat: re-factor sqrt (2018-02-21 10:21:54 +)


This is the re-factor of softfloat:

  - shared common code path float16/32/64
  - well commented and easy to follow code
  - added a bunch of float16 support

While some operations are slower the key ones exercised by the
floating point dbt-bench are the same: https://i.imgur.com/oXNJNql.png


Alex Bennée (22):
  fpu/softfloat: implement float16_squash_input_denormal
  include/fpu/softfloat: remove USE_SOFTFLOAT_STRUCT_TYPES
  fpu/softfloat-types: new header to prevent excessive re-builds
  target/*/cpu.h: remove softfloat.h
  include/fpu/softfloat: implement float16_abs helper
  include/fpu/softfloat: implement float16_chs helper
  include/fpu/softfloat: implement float16_set_sign helper
  include/fpu/softfloat: add some float16 constants
  fpu/softfloat: improve comments on ARM NaN propagation
  fpu/softfloat: move the extract functions to the top of the file
  fpu/softfloat: define decompose structures
  fpu/softfloat: re-factor add/sub
  fpu/softfloat: re-factor mul
  fpu/softfloat: re-factor div
  fpu/softfloat: re-factor muladd
  fpu/softfloat: re-factor round_to_int
  fpu/softfloat: re-factor float to int/uint
  fpu/softfloat: re-factor int/uint to float
  fpu/softfloat: re-factor scalbn
  fpu/softfloat: re-factor minmax
  fpu/softfloat: re-factor compare
  fpu/softfloat: re-factor sqrt

 fpu/softfloat-macros.h  |   48 +
 fpu/softfloat-specialize.h  |  109 +-
 fpu/softfloat.c | 4550 ---
 include/fpu/softfloat-types.h   |  179 ++
 include/fpu/softfloat.h |  202 +-
 include/qemu/bswap.h|2 +-
 target/alpha/cpu.h  |2 -
 target/arm/cpu.c|1 +
 target/arm/cpu.h|2 -
 target/arm/helper-a64.c |1 +
 target/arm/helper.c |1 +
 target/arm/neon_helper.c|1 +
 target/hppa/cpu.c   |1 +
 target/hppa/cpu.h   |1 -
 target/hppa/op_helper.c |2 +-
 target/i386/cpu.h   |4 -
 target/i386/fpu_helper.c|1 +
 target/m68k/cpu.c   |2 +-
 target/m68k/cpu.h   |1 -
 target/m68k/fpu_helper.c|1 +
 target/m68k/helper.c|1 +
 target/m68k/translate.c |2 +
 target/microblaze/cpu.c |1 +
 target/microblaze/cpu.h |2 +-
 target/microblaze/op_helper.c   |1 +
 target/moxie/cpu.h  |1 -
 target/nios2/cpu.h  |1 -
 target/openrisc/cpu.h   |1 -
 target/openrisc/fpu_helper.c|1 +
 target/ppc/cpu.h|1 -
 target/ppc/fpu_helper.c |1 +
 target/ppc/int_helper.c |1 +
 target/ppc/translate_init.c |1 +
 target/s390x/cpu.c  |1 +
 target/s390x/cpu.h  |2 -
 target/s390x/fpu_helper.c   |1 +
 target/sh4/cpu.c|1 +
 target/sh4/cpu.h|2 -
 target/sh4/op_helper.c  |1 +
 target/sparc/cpu.h  |2 -
 target/sparc/fop_helper.c   |1 +
 target/tricore/cpu.h|1 -
 target/tricore/fpu_helper.c |1 +
 target/tricore/helper.c |1 +
 target/unicore32/cpu.c  |1 +
 target/unicore32/cpu.h  |1 -
 target/unicore32/ucf64_helper.c |1 +
 target/xtensa/cpu.h |1 -
 target/xtensa/op_helper.c   |1 +
 49 files changed, 2204 insertions(+), 2941 deletions(-)
 create mode 100644 include/fpu/softfloat-types.h


-- 
2.15.1

Re: [Qemu-devel] Call for GSoC & Outreachy 2018 mentors & project ideas

2018-02-21 Thread Stefan Hajnoczi

On Tue, Feb 20, 2018 at 11:13 AM, Paolo Bonzini  wrote:
> On 20/02/2018 11:36, Stefan Hajnoczi wrote:
>> === Multi-CPU cluster support for GDB server in QEMU ===
>>
>> There are many examples in modern computing where multiple CPU
>> clusters are grouped together in a single SoC. This is common in the
>> ARM world especially. There are numerous examples such as ARM's
>> big.LITTLE implementations and Xilinx's 4xA53s and 2xR5s on the ZynqMP
>> SoC. The goal of this task is to add support to the GDB server to
>> allow users to debug across these clusters.
>>
>> This is another step towards single binary QEMU as well.
>>
>>  Detailed description of the project.
>>
>> Xilinx has an out of tree implementation that can be used as a
>> starting point. Work will need to be done on top of this to prepare it
>> for upstream submission and to ensure the implementation is more
>> generic.
>>
>> This will mostly involve extending GDB server to tell GDB about
>> different architectures and then allow the user to swap between them.
>>
>> The Xilinx implementation can be seen here:
>> https://github.com/Xilinx/qemu/blob/master/gdbstub.c
>> There has been some steps in preparing the work to go upstream, which
>> can be seen here:
>> https://github.com/Xilinx/qemu/tree/mainline/alistair/gdb
>
> I agree this is interesting.

Thanks!  I have added it to the wiki:

https://wiki.qemu.org/Google_Summer_of_Code_2018#Multi-CPU_cluster_support_for_GDB_server_in_QEMU

Stefan

[Qemu-devel] [PULL 19/22] fpu/softfloat: re-factor scalbn

This is one of the simpler manipulations you could make to a floating
point number.

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 4313d3a602..a07b8556b0 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -1663,6 +1663,39 @@ float64 uint16_to_float64(uint16_t a, float_status 
*status)
 return uint64_to_float64(a, status);
 }
 
+/* Multiply A by 2 raised to the power N.  */
+static FloatParts scalbn_decomposed(FloatParts a, int n, float_status *s)
+{
+if (unlikely(is_nan(a.cls))) {
+return return_nan(a, s);
+}
+if (a.cls == float_class_normal) {
+a.exp += n;
+}
+return a;
+}
+
+float16 float16_scalbn(float16 a, int n, float_status *status)
+{
+FloatParts pa = float16_unpack_canonical(a, status);
+FloatParts pr = scalbn_decomposed(pa, n, status);
+return float16_round_pack_canonical(pr, status);
+}
+
+float32 float32_scalbn(float32 a, int n, float_status *status)
+{
+FloatParts pa = float32_unpack_canonical(a, status);
+FloatParts pr = scalbn_decomposed(pa, n, status);
+return float32_round_pack_canonical(pr, status);
+}
+
+float64 float64_scalbn(float64 a, int n, float_status *status)
+{
+FloatParts pa = float64_unpack_canonical(a, status);
+FloatParts pr = scalbn_decomposed(pa, n, status);
+return float64_round_pack_canonical(pr, status);
+}
+
 /*
 | Takes a 64-bit fixed-point value `absZ' with binary point between bits 6
 | and 7, and returns the properly rounded 32-bit integer corresponding to the
@@ -6986,79 +7019,6 @@ MINMAX(32)
 MINMAX(64)
 
 
-/* Multiply A by 2 raised to the power N.  */
-float32 float32_scalbn(float32 a, int n, float_status *status)
-{
-flag aSign;
-int16_t aExp;
-uint32_t aSig;
-
-a = float32_squash_input_denormal(a, status);
-aSig = extractFloat32Frac( a );
-aExp = extractFloat32Exp( a );
-aSign = extractFloat32Sign( a );
-
-if ( aExp == 0xFF ) {
-if ( aSig ) {
-return propagateFloat32NaN(a, a, status);
-}
-return a;
-}
-if (aExp != 0) {
-aSig |= 0x0080;
-} else if (aSig == 0) {
-return a;
-} else {
-aExp++;
-}
-
-if (n > 0x200) {
-n = 0x200;
-} else if (n < -0x200) {
-n = -0x200;
-}
-
-aExp += n - 1;
-aSig <<= 7;
-return normalizeRoundAndPackFloat32(aSign, aExp, aSig, status);
-}
-
-float64 float64_scalbn(float64 a, int n, float_status *status)
-{
-flag aSign;
-int16_t aExp;
-uint64_t aSig;
-
-a = float64_squash_input_denormal(a, status);
-aSig = extractFloat64Frac( a );
-aExp = extractFloat64Exp( a );
-aSign = extractFloat64Sign( a );
-
-if ( aExp == 0x7FF ) {
-if ( aSig ) {
-return propagateFloat64NaN(a, a, status);
-}
-return a;
-}
-if (aExp != 0) {
-aSig |= LIT64( 0x0010 );
-} else if (aSig == 0) {
-return a;
-} else {
-aExp++;
-}
-
-if (n > 0x1000) {
-n = 0x1000;
-} else if (n < -0x1000) {
-n = -0x1000;
-}
-
-aExp += n - 1;
-aSig <<= 10;
-return normalizeRoundAndPackFloat64(aSign, aExp, aSig, status);
-}
-
 floatx80 floatx80_scalbn(floatx80 a, int n, float_status *status)
 {
 flag aSign;
diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h
index 3e6fdd756a..52621e0b79 100644
--- a/include/fpu/softfloat.h
+++ b/include/fpu/softfloat.h
@@ -244,6 +244,7 @@ float16 float16_sub(float16, float16, float_status *status);
 float16 float16_mul(float16, float16, float_status *status);
 float16 float16_muladd(float16, float16, float16, int, float_status *status);
 float16 float16_div(float16, float16, float_status *status);
+float16 float16_scalbn(float16, int, float_status *status);
 
 int float16_is_quiet_nan(float16, float_status *status);
 int float16_is_signaling_nan(float16, float_status *status);
-- 
2.15.1

[Qemu-devel] [PULL 04/22] target/*/cpu.h: remove softfloat.h

As cpu.h is another typically widely included file which doesn't need
full access to the softfloat API we can remove the includes from here
as well. Where they do need types it's typically for float_status and
the rounding modes so we move that to softfloat-types.h as well.

As a result of not having softfloat in every cpu.h call we now need to
add it to various helpers that do need the full softfloat.h
definitions.

Signed-off-by: Alex Bennée 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Richard Henderson 
[For PPC parts]
Acked-by: David Gibson 

diff --git a/include/fpu/softfloat-types.h b/include/fpu/softfloat-types.h
index 8210a94ea1..4e378cb612 100644
--- a/include/fpu/softfloat-types.h
+++ b/include/fpu/softfloat-types.h
@@ -80,6 +80,12 @@ this code that are retained.
 #ifndef SOFTFLOAT_TYPES_H
 #define SOFTFLOAT_TYPES_H
 
+/* This 'flag' type must be able to hold at least 0 and 1. It should
+ * probably be replaced with 'bool' but the uses would need to be audited
+ * to check that they weren't accidentally relying on it being a larger type.
+ */
+typedef uint8_t flag;
+
 /*
  * Software IEC/IEEE floating-point types.
  */
@@ -112,4 +118,62 @@ typedef struct {
 #define make_float128(high_, low_) ((float128) { .high = high_, .low = low_ })
 #define make_float128_init(high_, low_) { .high = high_, .low = low_ }
 
+/*
+ * Software IEC/IEEE floating-point underflow tininess-detection mode.
+ */
+
+enum {
+float_tininess_after_rounding  = 0,
+float_tininess_before_rounding = 1
+};
+
+/*
+ *Software IEC/IEEE floating-point rounding mode.
+ */
+
+enum {
+float_round_nearest_even = 0,
+float_round_down = 1,
+float_round_up   = 2,
+float_round_to_zero  = 3,
+float_round_ties_away= 4,
+/* Not an IEEE rounding mode: round to the closest odd mantissa value */
+float_round_to_odd   = 5,
+};
+
+/*
+ * Software IEC/IEEE floating-point exception flags.
+ */
+
+enum {
+float_flag_invalid   =  1,
+float_flag_divbyzero =  4,
+float_flag_overflow  =  8,
+float_flag_underflow = 16,
+float_flag_inexact   = 32,
+float_flag_input_denormal = 64,
+float_flag_output_denormal = 128
+};
+
+
+/*
+ * Floating Point Status. Individual architectures may maintain
+ * several versions of float_status for different functions. The
+ * correct status for the operation is then passed by reference to
+ * most of the softfloat functions.
+ */
+
+typedef struct float_status {
+signed char float_detect_tininess;
+signed char float_rounding_mode;
+uint8_t float_exception_flags;
+signed char floatx80_rounding_precision;
+/* should denormalised results go to zero and set the inexact flag? */
+flag flush_to_zero;
+/* should denormalised inputs go to zero and set the input_denormal flag? 
*/
+flag flush_inputs_to_zero;
+flag default_nan_mode;
+flag snan_bit_is_one;
+} float_status;
+
 #endif /* SOFTFLOAT_TYPES_H */
diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h
index 4e16e22e58..f3b9008f78 100644
--- a/include/fpu/softfloat.h
+++ b/include/fpu/softfloat.h
@@ -82,12 +82,6 @@ this code that are retained.
 #ifndef SOFTFLOAT_H
 #define SOFTFLOAT_H
 
-/* This 'flag' type must be able to hold at least 0 and 1. It should
- * probably be replaced with 'bool' but the uses would need to be audited
- * to check that they weren't accidentally relying on it being a larger type.
- */
-typedef uint8_t flag;
-
 #define LIT64( a ) a##LL
 
 /*
@@ -102,53 +96,6 @@ enum {
 
 #include "fpu/softfloat-types.h"
 
-/*
-| Software IEC/IEEE floating-point underflow tininess-detection mode.
-**/
-enum {
-float_tininess_after_rounding  = 0,
-float_tininess_before_rounding = 1
-};
-
-/*
-| Software IEC/IEEE floating-point rounding mode.
-**/
-enum {
-float_round_nearest_even = 0,
-float_round_down = 1,
-float_round_up   = 2,
-float_round_to_zero  = 3,
-float_round_ties_away= 4,
-/* Not an IEEE rounding mode: round to the closest odd mantissa value */
-float_round_to_odd   = 5,
-};
-
-/*
-| Software IEC/IEEE floating-point exception flags.
-**/
-enum {
-float_flag_invalid   =  1,
-float_flag_divbyzero =  4,
-float_flag_overflow  =  8,
-float_flag_underflow = 16,
-float_flag_inexact   = 32,
-float_flag_input_denormal = 64,
-float_flag_output_denormal = 128
-};
-
-typedef struct float_status {
-signed char fl

[Qemu-devel] [PULL 08/22] include/fpu/softfloat: add some float16 constants

This defines the same set of common constants for float 16 as defined
for 32 and 64 bit floats. These are often used by target helper
functions. I've also removed constants that are not used by anybody.

Signed-off-by: Alex Bennée 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Richard Henderson 

diff --git a/include/fpu/softfloat.h b/include/fpu/softfloat.h
index 59c06ef192..23824a3000 100644
--- a/include/fpu/softfloat.h
+++ b/include/fpu/softfloat.h
@@ -286,6 +286,11 @@ static inline float16 float16_set_sign(float16 a, int sign)
 return make_float16((float16_val(a) & 0x7fff) | (sign << 15));
 }
 
+#define float16_zero make_float16(0)
+#define float16_one make_float16(0x3c00)
+#define float16_half make_float16(0x3800)
+#define float16_infinity make_float16(0x7c00)
+
 /*
 | The pattern for a default generated half-precision NaN.
 **/
@@ -392,8 +397,6 @@ static inline float32 float32_set_sign(float32 a, int sign)
 
 #define float32_zero make_float32(0)
 #define float32_one make_float32(0x3f80)
-#define float32_ln2 make_float32(0x3f317218)
-#define float32_pi make_float32(0x40490fdb)
 #define float32_half make_float32(0x3f00)
 #define float32_infinity make_float32(0x7f80)
 
@@ -506,7 +509,6 @@ static inline float64 float64_set_sign(float64 a, int sign)
 #define float64_zero make_float64(0)
 #define float64_one make_float64(0x3ff0LL)
 #define float64_ln2 make_float64(0x3fe62e42fefa39efLL)
-#define float64_pi make_float64(0x400921fb54442d18LL)
 #define float64_half make_float64(0x3fe0LL)
 #define float64_infinity make_float64(0x7ff0LL)
 
-- 
2.15.1

Re: [Qemu-devel] [PATCH v4 4/5] usb-mtp: Introduce write support for MTP objects

> > +static void usb_mtp_write_data(MTPState *s)
> > +{
> > +MTPData *d = s->data_out;
> > +MTPObject *parent =
> > +usb_mtp_object_lookup(s, s->dataset.parent_handle);
> > +char *path = NULL;
> > +int rc = -1;
> > +mode_t mask = 0644;
> > +
> > +assert(d != NULL);
> > +
> 
> 
> Somewhere in here should surely be validating the "readonly" flag.
> 
> > +if (parent == NULL || !s->write_pending) {

Does happens here.  With a readonly device write_pending should
never be true.

> > +usb_mtp_queue_result(s, RES_INVALID_OBJECTINFO, d->trans,
> > + 0, 0, 0, 0);
> > +return;
> > +}

But adding an "assert(!readonly)" here as double-check surely doesn't hurt.

cheers,
  Gerd

[Qemu-devel] [PULL 09/22] fpu/softfloat: improve comments on ARM NaN propagation

Mention the pseudo-code fragment from which this is based.

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index de2c5d5702..4be0fb21ba 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -445,9 +445,10 @@ static float32 commonNaNToFloat32(commonNaNT a, 
float_status *status)
 
 #if defined(TARGET_ARM)
 static int pickNaN(flag aIsQNaN, flag aIsSNaN, flag bIsQNaN, flag bIsSNaN,
-flag aIsLargerSignificand)
+   flag aIsLargerSignificand)
 {
-/* ARM mandated NaN propagation rules: take the first of:
+/* ARM mandated NaN propagation rules (see FPProcessNaNs()), take
+ * the first of:
  *  1. A if it is signaling
  *  2. B if it is signaling
  *  3. A (quiet)
-- 
2.15.1

[Qemu-devel] [PULL 16/22] fpu/softfloat: re-factor round_to_int

We can now add float16_round_to_int and use the common round_decomposed and
canonicalize functions to have a single implementation for
float16/32/64 round_to_int functions.

Signed-off-by: Alex Bennée 
Signed-off-by: Richard Henderson 
Reviewed-by: Peter Maydell 

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index ae4ba6de51..616c6cef07 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -560,6 +560,25 @@ static bool is_qnan(FloatClass c)
 return c == float_class_qnan;
 }
 
+static FloatParts return_nan(FloatParts a, float_status *s)
+{
+switch (a.cls) {
+case float_class_snan:
+s->float_exception_flags |= float_flag_invalid;
+a.cls = float_class_msnan;
+/* fall through */
+case float_class_qnan:
+if (s->default_nan_mode) {
+a.cls = float_class_dnan;
+}
+break;
+
+default:
+g_assert_not_reached();
+}
+return a;
+}
+
 static FloatParts pick_nan(FloatParts a, FloatParts b, float_status *s)
 {
 if (is_snan(a.cls) || is_snan(b.cls)) {
@@ -1175,6 +1194,132 @@ float64 float64_div(float64 a, float64 b, float_status 
*status)
 return float64_round_pack_canonical(pr, status);
 }
 
+/*
+ * Rounds the floating-point value `a' to an integer, and returns the
+ * result as a floating-point value. The operation is performed
+ * according to the IEC/IEEE Standard for Binary Floating-Point
+ * Arithmetic.
+ */
+
+static FloatParts round_to_int(FloatParts a, int rounding_mode, float_status 
*s)
+{
+if (is_nan(a.cls)) {
+return return_nan(a, s);
+}
+
+switch (a.cls) {
+case float_class_zero:
+case float_class_inf:
+case float_class_qnan:
+/* already "integral" */
+break;
+case float_class_normal:
+if (a.exp >= DECOMPOSED_BINARY_POINT) {
+/* already integral */
+break;
+}
+if (a.exp < 0) {
+bool one;
+/* all fractional */
+s->float_exception_flags |= float_flag_inexact;
+switch (rounding_mode) {
+case float_round_nearest_even:
+one = a.exp == -1 && a.frac > DECOMPOSED_IMPLICIT_BIT;
+break;
+case float_round_ties_away:
+one = a.exp == -1 && a.frac >= DECOMPOSED_IMPLICIT_BIT;
+break;
+case float_round_to_zero:
+one = false;
+break;
+case float_round_up:
+one = !a.sign;
+break;
+case float_round_down:
+one = a.sign;
+break;
+default:
+g_assert_not_reached();
+}
+
+if (one) {
+a.frac = DECOMPOSED_IMPLICIT_BIT;
+a.exp = 0;
+} else {
+a.cls = float_class_zero;
+}
+} else {
+uint64_t frac_lsb = DECOMPOSED_IMPLICIT_BIT >> a.exp;
+uint64_t frac_lsbm1 = frac_lsb >> 1;
+uint64_t rnd_even_mask = (frac_lsb - 1) | frac_lsb;
+uint64_t rnd_mask = rnd_even_mask >> 1;
+uint64_t inc;
+
+switch (rounding_mode) {
+case float_round_nearest_even:
+inc = ((a.frac & rnd_even_mask) != frac_lsbm1 ? frac_lsbm1 : 
0);
+break;
+case float_round_ties_away:
+inc = frac_lsbm1;
+break;
+case float_round_to_zero:
+inc = 0;
+break;
+case float_round_up:
+inc = a.sign ? 0 : rnd_mask;
+break;
+case float_round_down:
+inc = a.sign ? rnd_mask : 0;
+break;
+default:
+g_assert_not_reached();
+}
+
+if (a.frac & rnd_mask) {
+s->float_exception_flags |= float_flag_inexact;
+a.frac += inc;
+a.frac &= ~rnd_mask;
+if (a.frac & DECOMPOSED_OVERFLOW_BIT) {
+a.frac >>= 1;
+a.exp++;
+}
+}
+}
+break;
+default:
+g_assert_not_reached();
+}
+return a;
+}
+
+float16 float16_round_to_int(float16 a, float_status *s)
+{
+FloatParts pa = float16_unpack_canonical(a, s);
+FloatParts pr = round_to_int(pa, s->float_rounding_mode, s);
+return float16_round_pack_canonical(pr, s);
+}
+
+float32 float32_round_to_int(float32 a, float_status *s)
+{
+FloatParts pa = float32_unpack_canonical(a, s);
+FloatParts pr = round_to_int(pa, s->float_rounding_mode, s);
+return float32_round_pack_canonical(pr, s);
+}
+
+float64 float64_round_to_int(float64 a, float_status *s)
+{
+FloatParts pa = float64_unpack_canonical(a, s);
+FloatParts pr = round_to_int(pa, s->float_rounding_mode, s);
+return float64_round_pack_canonical(pr, s);
+}
+
+float64 float64_trunc_to_int(float64 a

[Qemu-devel] [PULL 12/22] fpu/softfloat: re-factor add/sub

We can now add float16_add/sub and use the common decompose and
canonicalize functions to have a single implementation for
float16/32/64 add and sub functions.

Signed-off-by: Alex Bennée 
Signed-off-by: Richard Henderson 
Reviewed-by: Philippe Mathieu-Daudé 

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 568d95..2190e7de56 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -83,6 +83,7 @@ this code that are retained.
  * target-dependent and needs the TARGET_* macros.
  */
 #include "qemu/osdep.h"
+#include "qemu/bitops.h"
 #include "fpu/softfloat.h"
 
 /* We only need stdlib for abort() */
@@ -270,6 +271,470 @@ static const FloatFmt float64_params = {
 FLOAT_PARAMS(11, 52)
 };
 
+/* Unpack a float to parts, but do not canonicalize.  */
+static inline FloatParts unpack_raw(FloatFmt fmt, uint64_t raw)
+{
+const int sign_pos = fmt.frac_size + fmt.exp_size;
+
+return (FloatParts) {
+.cls = float_class_unclassified,
+.sign = extract64(raw, sign_pos, 1),
+.exp = extract64(raw, fmt.frac_size, fmt.exp_size),
+.frac = extract64(raw, 0, fmt.frac_size),
+};
+}
+
+static inline FloatParts float16_unpack_raw(float16 f)
+{
+return unpack_raw(float16_params, f);
+}
+
+static inline FloatParts float32_unpack_raw(float32 f)
+{
+return unpack_raw(float32_params, f);
+}
+
+static inline FloatParts float64_unpack_raw(float64 f)
+{
+return unpack_raw(float64_params, f);
+}
+
+/* Pack a float from parts, but do not canonicalize.  */
+static inline uint64_t pack_raw(FloatFmt fmt, FloatParts p)
+{
+const int sign_pos = fmt.frac_size + fmt.exp_size;
+uint64_t ret = deposit64(p.frac, fmt.frac_size, fmt.exp_size, p.exp);
+return deposit64(ret, sign_pos, 1, p.sign);
+}
+
+static inline float16 float16_pack_raw(FloatParts p)
+{
+return make_float16(pack_raw(float16_params, p));
+}
+
+static inline float32 float32_pack_raw(FloatParts p)
+{
+return make_float32(pack_raw(float32_params, p));
+}
+
+static inline float64 float64_pack_raw(FloatParts p)
+{
+return make_float64(pack_raw(float64_params, p));
+}
+
+/* Canonicalize EXP and FRAC, setting CLS.  */
+static FloatParts canonicalize(FloatParts part, const FloatFmt *parm,
+   float_status *status)
+{
+if (part.exp == parm->exp_max) {
+if (part.frac == 0) {
+part.cls = float_class_inf;
+} else {
+#ifdef NO_SIGNALING_NANS
+part.cls = float_class_qnan;
+#else
+int64_t msb = part.frac << (parm->frac_shift + 2);
+if ((msb < 0) == status->snan_bit_is_one) {
+part.cls = float_class_snan;
+} else {
+part.cls = float_class_qnan;
+}
+#endif
+}
+} else if (part.exp == 0) {
+if (likely(part.frac == 0)) {
+part.cls = float_class_zero;
+} else if (status->flush_inputs_to_zero) {
+float_raise(float_flag_input_denormal, status);
+part.cls = float_class_zero;
+part.frac = 0;
+} else {
+int shift = clz64(part.frac) - 1;
+part.cls = float_class_normal;
+part.exp = parm->frac_shift - parm->exp_bias - shift + 1;
+part.frac <<= shift;
+}
+} else {
+part.cls = float_class_normal;
+part.exp -= parm->exp_bias;
+part.frac = DECOMPOSED_IMPLICIT_BIT + (part.frac << parm->frac_shift);
+}
+return part;
+}
+
+/* Round and uncanonicalize a floating-point number by parts. There
+ * are FRAC_SHIFT bits that may require rounding at the bottom of the
+ * fraction; these bits will be removed. The exponent will be biased
+ * by EXP_BIAS and must be bounded by [EXP_MAX-1, 0].
+ */
+
+static FloatParts round_canonical(FloatParts p, float_status *s,
+  const FloatFmt *parm)
+{
+const uint64_t frac_lsbm1 = parm->frac_lsbm1;
+const uint64_t round_mask = parm->round_mask;
+const uint64_t roundeven_mask = parm->roundeven_mask;
+const int exp_max = parm->exp_max;
+const int frac_shift = parm->frac_shift;
+uint64_t frac, inc;
+int exp, flags = 0;
+bool overflow_norm;
+
+frac = p.frac;
+exp = p.exp;
+
+switch (p.cls) {
+case float_class_normal:
+switch (s->float_rounding_mode) {
+case float_round_nearest_even:
+overflow_norm = false;
+inc = ((frac & roundeven_mask) != frac_lsbm1 ? frac_lsbm1 : 0);
+break;
+case float_round_ties_away:
+overflow_norm = false;
+inc = frac_lsbm1;
+break;
+case float_round_to_zero:
+overflow_norm = true;
+inc = 0;
+break;
+case float_round_up:
+inc = p.sign ? 0 : round_mask;
+overflow_norm = p.sign;
+break;
+case float_round_down:
+inc = p.sign ? round_mask : 0;
+overflow_norm = !p.sign;
+

[Qemu-devel] [PULL 13/22] fpu/softfloat: re-factor mul

We can now add float16_mul and use the common decompose and
canonicalize functions to have a single implementation for
float16/32/64 versions.

Signed-off-by: Alex Bennée 
Signed-off-by: Richard Henderson 
Reviewed-by: Peter Maydell 

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 2190e7de56..6d29e1a103 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -735,6 +735,87 @@ float64 __attribute__((flatten)) float64_sub(float64 a, 
float64 b,
 return float64_round_pack_canonical(pr, status);
 }
 
+/*
+ * Returns the result of multiplying the floating-point values `a' and
+ * `b'. The operation is performed according to the IEC/IEEE Standard
+ * for Binary Floating-Point Arithmetic.
+ */
+
+static FloatParts mul_floats(FloatParts a, FloatParts b, float_status *s)
+{
+bool sign = a.sign ^ b.sign;
+
+if (a.cls == float_class_normal && b.cls == float_class_normal) {
+uint64_t hi, lo;
+int exp = a.exp + b.exp;
+
+mul64To128(a.frac, b.frac, &hi, &lo);
+shift128RightJamming(hi, lo, DECOMPOSED_BINARY_POINT, &hi, &lo);
+if (lo & DECOMPOSED_OVERFLOW_BIT) {
+shift64RightJamming(lo, 1, &lo);
+exp += 1;
+}
+
+/* Re-use a */
+a.exp = exp;
+a.sign = sign;
+a.frac = lo;
+return a;
+}
+/* handle all the NaN cases */
+if (is_nan(a.cls) || is_nan(b.cls)) {
+return pick_nan(a, b, s);
+}
+/* Inf * Zero == NaN */
+if ((a.cls == float_class_inf && b.cls == float_class_zero) ||
+(a.cls == float_class_zero && b.cls == float_class_inf)) {
+s->float_exception_flags |= float_flag_invalid;
+a.cls = float_class_dnan;
+a.sign = sign;
+return a;
+}
+/* Multiply by 0 or Inf */
+if (a.cls == float_class_inf || a.cls == float_class_zero) {
+a.sign = sign;
+return a;
+}
+if (b.cls == float_class_inf || b.cls == float_class_zero) {
+b.sign = sign;
+return b;
+}
+g_assert_not_reached();
+}
+
+float16 __attribute__((flatten)) float16_mul(float16 a, float16 b,
+ float_status *status)
+{
+FloatParts pa = float16_unpack_canonical(a, status);
+FloatParts pb = float16_unpack_canonical(b, status);
+FloatParts pr = mul_floats(pa, pb, status);
+
+return float16_round_pack_canonical(pr, status);
+}
+
+float32 __attribute__((flatten)) float32_mul(float32 a, float32 b,
+ float_status *status)
+{
+FloatParts pa = float32_unpack_canonical(a, status);
+FloatParts pb = float32_unpack_canonical(b, status);
+FloatParts pr = mul_floats(pa, pb, status);
+
+return float32_round_pack_canonical(pr, status);
+}
+
+float64 __attribute__((flatten)) float64_mul(float64 a, float64 b,
+ float_status *status)
+{
+FloatParts pa = float64_unpack_canonical(a, status);
+FloatParts pb = float64_unpack_canonical(b, status);
+FloatParts pr = mul_floats(pa, pb, status);
+
+return float64_round_pack_canonical(pr, status);
+}
+
 /*
 | Takes a 64-bit fixed-point value `absZ' with binary point between bits 6
 | and 7, and returns the properly rounded 32-bit integer corresponding to the
@@ -2546,70 +2627,6 @@ float32 float32_round_to_int(float32 a, float_status 
*status)
 
 }
 
-/*
-| Returns the result of multiplying the single-precision floating-point values
-| `a' and `b'.  The operation is performed according to the IEC/IEEE Standard
-| for Binary Floating-Point Arithmetic.
-**/
-
-float32 float32_mul(float32 a, float32 b, float_status *status)
-{
-flag aSign, bSign, zSign;
-int aExp, bExp, zExp;
-uint32_t aSig, bSig;
-uint64_t zSig64;
-uint32_t zSig;
-
-a = float32_squash_input_denormal(a, status);
-b = float32_squash_input_denormal(b, status);
-
-aSig = extractFloat32Frac( a );
-aExp = extractFloat32Exp( a );
-aSign = extractFloat32Sign( a );
-bSig = extractFloat32Frac( b );
-bExp = extractFloat32Exp( b );
-bSign = extractFloat32Sign( b );
-zSign = aSign ^ bSign;
-if ( aExp == 0xFF ) {
-if ( aSig || ( ( bExp == 0xFF ) && bSig ) ) {
-return propagateFloat32NaN(a, b, status);
-}
-if ( ( bExp | bSig ) == 0 ) {
-float_raise(float_flag_invalid, status);
-return float32_default_nan(status);
-}
-return packFloat32( zSign, 0xFF, 0 );
-}
-if ( bExp == 0xFF ) {
-if (bSig) {
-return propagateFloat32NaN(a, b, status);
-}
-if ( ( aExp | aSig ) == 0 ) {
-float_raise(float_flag_invalid, status);
-return float32_default_nan(status);
-

[Qemu-devel] [PULL 21/22] fpu/softfloat: re-factor compare

The compare function was already expanded from a macro. I keep the
macro expansion but move most of the logic into a compare_decomposed.

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 7a53c36da4..4bc425d7e4 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -1783,6 +1783,86 @@ MINMAX(64, maxnummag, false, true, true)
 
 #undef MINMAX
 
+/* Floating point compare */
+static int compare_floats(FloatParts a, FloatParts b, bool is_quiet,
+  float_status *s)
+{
+if (is_nan(a.cls) || is_nan(b.cls)) {
+if (!is_quiet ||
+a.cls == float_class_snan ||
+b.cls == float_class_snan) {
+s->float_exception_flags |= float_flag_invalid;
+}
+return float_relation_unordered;
+}
+
+if (a.cls == float_class_zero) {
+if (b.cls == float_class_zero) {
+return float_relation_equal;
+}
+return b.sign ? float_relation_greater : float_relation_less;
+} else if (b.cls == float_class_zero) {
+return a.sign ? float_relation_less : float_relation_greater;
+}
+
+/* The only really important thing about infinity is its sign. If
+ * both are infinities the sign marks the smallest of the two.
+ */
+if (a.cls == float_class_inf) {
+if ((b.cls == float_class_inf) && (a.sign == b.sign)) {
+return float_relation_equal;
+}
+return a.sign ? float_relation_less : float_relation_greater;
+} else if (b.cls == float_class_inf) {
+return b.sign ? float_relation_greater : float_relation_less;
+}
+
+if (a.sign != b.sign) {
+return a.sign ? float_relation_less : float_relation_greater;
+}
+
+if (a.exp == b.exp) {
+if (a.frac == b.frac) {
+return float_relation_equal;
+}
+if (a.sign) {
+return a.frac > b.frac ?
+float_relation_less : float_relation_greater;
+} else {
+return a.frac > b.frac ?
+float_relation_greater : float_relation_less;
+}
+} else {
+if (a.sign) {
+return a.exp > b.exp ? float_relation_less : 
float_relation_greater;
+} else {
+return a.exp > b.exp ? float_relation_greater : 
float_relation_less;
+}
+}
+}
+
+#define COMPARE(sz) \
+int float ## sz ## _compare(float ## sz a, float ## sz b,   \
+float_status *s)\
+{   \
+FloatParts pa = float ## sz ## _unpack_canonical(a, s); \
+FloatParts pb = float ## sz ## _unpack_canonical(b, s); \
+return compare_floats(pa, pb, false, s);\
+}   \
+int float ## sz ## _compare_quiet(float ## sz a, float ## sz b, \
+  float_status *s)  \
+{   \
+FloatParts pa = float ## sz ## _unpack_canonical(a, s); \
+FloatParts pb = float ## sz ## _unpack_canonical(b, s); \
+return compare_floats(pa, pb, true, s); \
+}
+
+COMPARE(16)
+COMPARE(32)
+COMPARE(64)
+
+#undef COMPARE
+
 /* Multiply A by 2 raised to the power N.  */
 static FloatParts scalbn_decomposed(FloatParts a, int n, float_status *s)
 {
@@ -6884,60 +6964,6 @@ int float128_unordered_quiet(float128 a, float128 b, 
float_status *status)
 return 0;
 }
 
-#define COMPARE(s, nan_exp)  \
-static inline int float ## s ## _compare_internal(float ## s a, float ## s b,\
-  int is_quiet, float_status *status)\
-{\
-flag aSign, bSign;   \
-uint ## s ## _t av, bv;  \
-a = float ## s ## _squash_input_denormal(a, status); \
-b = float ## s ## _squash_input_denormal(b, status); \
- \
-if (( ( extractFloat ## s ## Exp( a ) == nan_exp ) &&\
- extractFloat ## s ## Frac( a ) ) || \
-( ( extractFloat ## s ## Exp( b ) == nan_exp ) &&\
-  extractFloat ## s ## Frac( b ) )) {\
-if (!is_quiet || \
-float ## s ## _is_signaling_nan(a, status) ||  \
-float ## s ## _is_signaling_nan(b, status)) {

[Qemu-devel] [PULL 11/22] fpu/softfloat: define decompose structures

These structures pave the way for generic softfloat helper routines
that will operate on fully decomposed numbers.

Signed-off-by: Alex Bennée 
Signed-off-by: Richard Henderson 
Reviewed-by: Philippe Mathieu-Daudé 

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 297e48f5c9..568d95 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -83,7 +83,6 @@ this code that are retained.
  * target-dependent and needs the TARGET_* macros.
  */
 #include "qemu/osdep.h"
-
 #include "fpu/softfloat.h"
 
 /* We only need stdlib for abort() */
@@ -186,6 +185,91 @@ static inline flag extractFloat64Sign(float64 a)
 return float64_val(a) >> 63;
 }
 
+/*
+ * Classify a floating point number. Everything above float_class_qnan
+ * is a NaN so cls >= float_class_qnan is any NaN.
+ */
+
+typedef enum __attribute__ ((__packed__)) {
+float_class_unclassified,
+float_class_zero,
+float_class_normal,
+float_class_inf,
+float_class_qnan,  /* all NaNs from here */
+float_class_snan,
+float_class_dnan,
+float_class_msnan, /* maybe silenced */
+} FloatClass;
+
+/*
+ * Structure holding all of the decomposed parts of a float. The
+ * exponent is unbiased and the fraction is normalized. All
+ * calculations are done with a 64 bit fraction and then rounded as
+ * appropriate for the final format.
+ *
+ * Thanks to the packed FloatClass a decent compiler should be able to
+ * fit the whole structure into registers and avoid using the stack
+ * for parameter passing.
+ */
+
+typedef struct {
+uint64_t frac;
+int32_t  exp;
+FloatClass cls;
+bool sign;
+} FloatParts;
+
+#define DECOMPOSED_BINARY_POINT(64 - 2)
+#define DECOMPOSED_IMPLICIT_BIT(1ull << DECOMPOSED_BINARY_POINT)
+#define DECOMPOSED_OVERFLOW_BIT(DECOMPOSED_IMPLICIT_BIT << 1)
+
+/* Structure holding all of the relevant parameters for a format.
+ *   exp_size: the size of the exponent field
+ *   exp_bias: the offset applied to the exponent field
+ *   exp_max: the maximum normalised exponent
+ *   frac_size: the size of the fraction field
+ *   frac_shift: shift to normalise the fraction with DECOMPOSED_BINARY_POINT
+ * The following are computed based the size of fraction
+ *   frac_lsb: least significant bit of fraction
+ *   fram_lsbm1: the bit bellow the least significant bit (for rounding)
+ *   round_mask/roundeven_mask: masks used for rounding
+ */
+typedef struct {
+int exp_size;
+int exp_bias;
+int exp_max;
+int frac_size;
+int frac_shift;
+uint64_t frac_lsb;
+uint64_t frac_lsbm1;
+uint64_t round_mask;
+uint64_t roundeven_mask;
+} FloatFmt;
+
+/* Expand fields based on the size of exponent and fraction */
+#define FLOAT_PARAMS(E, F)   \
+.exp_size   = E, \
+.exp_bias   = ((1 << E) - 1) >> 1,   \
+.exp_max= (1 << E) - 1,  \
+.frac_size  = F, \
+.frac_shift = DECOMPOSED_BINARY_POINT - F,   \
+.frac_lsb   = 1ull << (DECOMPOSED_BINARY_POINT - F), \
+.frac_lsbm1 = 1ull << ((DECOMPOSED_BINARY_POINT - F) - 1),   \
+.round_mask = (1ull << (DECOMPOSED_BINARY_POINT - F)) - 1,   \
+.roundeven_mask = (2ull << (DECOMPOSED_BINARY_POINT - F)) - 1
+
+static const FloatFmt float16_params = {
+FLOAT_PARAMS(5, 10)
+};
+
+static const FloatFmt float32_params = {
+FLOAT_PARAMS(8, 23)
+};
+
+static const FloatFmt float64_params = {
+FLOAT_PARAMS(11, 52)
+};
+
 /*
 | Takes a 64-bit fixed-point value `absZ' with binary point between bits 6
 | and 7, and returns the properly rounded 32-bit integer corresponding to the
-- 
2.15.1

Re: [Qemu-devel] [PATCH] tests/boot-serial-test: Fix problem with timeout due to dropped characters

On 21.02.2018 12:05, Paolo Bonzini wrote:
> On 16/02/2018 07:12, Thomas Huth wrote:
>> Commit 92b540dac9fc3a5 introduce a counter to handle the timeouts in a
>> better way. But in case ccnt reaches 512, the current read character is
>> ignored - and if that character is part of the string that we are looking
>> for, the test fails to match the string.
>>
>> Almost all of the tests look for a string within the first 512 bytes of
>> firmware output, so the problem never triggered there. But the hppa test
>> that has been added recently looks for a longer string at the very end of
>> a long output, thus there's a chance that we miss a character there so
>> that the test fails unexpectedly. Fix it by *not* reading and dropping a
>> character if the counter reaches 512.
>>
>> Fixes: 92b540dac9fc3a572c7342edd0b073000f5a6abf
>> Signed-off-by: Thomas Huth 
>> ---
>>  @Peter: Since this fixes the problem with running "make check", could
>>  you maybe apply this directly to the master branch? Thanks, and sorry
>>  for the inconvenience!
>>
>>  tests/boot-serial-test.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/tests/boot-serial-test.c b/tests/boot-serial-test.c
>> index ea87a80..696f7a3 100644
>> --- a/tests/boot-serial-test.c
>> +++ b/tests/boot-serial-test.c
>> @@ -101,7 +101,7 @@ static void check_guest_output(const testdef_t *test, 
>> int fd)
>>  /* Poll serial output... Wait at most 60 seconds */
>>  for (i = 0; i < 6000; ++i) {
>>  ccnt = 0;
>> -while ((nbr = read(fd, &ch, 1)) == 1 && ccnt++ < 512) {
>> +while (ccnt++ < 512 && (nbr = read(fd, &ch, 1)) == 1) {
>>  if (ch == test->expect[pos]) {
>>  pos += 1;
>>  if (test->expect[pos] == '\0') {
>>
> 
> Queued in the meanwhile, thanks.

Thanks, but Peter already applied it to fix the "make check" failures:

https://git.qemu.org/?p=qemu.git;a=commit;h=5e5432b766c424a5d1

 Thomas

[Qemu-devel] [PULL 20/22] fpu/softfloat: re-factor minmax

Let's do the same re-factor treatment for minmax functions. I still
use the MACRO trick to expand but now all the checking code is common.

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index a07b8556b0..7a53c36da4 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -1663,6 +1663,126 @@ float64 uint16_to_float64(uint16_t a, float_status 
*status)
 return uint64_to_float64(a, status);
 }
 
+/* Float Min/Max */
+/* min() and max() functions. These can't be implemented as
+ * 'compare and pick one input' because that would mishandle
+ * NaNs and +0 vs -0.
+ *
+ * minnum() and maxnum() functions. These are similar to the min()
+ * and max() functions but if one of the arguments is a QNaN and
+ * the other is numerical then the numerical argument is returned.
+ * SNaNs will get quietened before being returned.
+ * minnum() and maxnum correspond to the IEEE 754-2008 minNum()
+ * and maxNum() operations. min() and max() are the typical min/max
+ * semantics provided by many CPUs which predate that specification.
+ *
+ * minnummag() and maxnummag() functions correspond to minNumMag()
+ * and minNumMag() from the IEEE-754 2008.
+ */
+static FloatParts minmax_floats(FloatParts a, FloatParts b, bool ismin,
+bool ieee, bool ismag, float_status *s)
+{
+if (unlikely(is_nan(a.cls) || is_nan(b.cls))) {
+if (ieee) {
+/* Takes two floating-point values `a' and `b', one of
+ * which is a NaN, and returns the appropriate NaN
+ * result. If either `a' or `b' is a signaling NaN,
+ * the invalid exception is raised.
+ */
+if (is_snan(a.cls) || is_snan(b.cls)) {
+return pick_nan(a, b, s);
+} else if (is_nan(a.cls) && !is_nan(b.cls)) {
+return b;
+} else if (is_nan(b.cls) && !is_nan(a.cls)) {
+return a;
+}
+}
+return pick_nan(a, b, s);
+} else {
+int a_exp, b_exp;
+bool a_sign, b_sign;
+
+switch (a.cls) {
+case float_class_normal:
+a_exp = a.exp;
+break;
+case float_class_inf:
+a_exp = INT_MAX;
+break;
+case float_class_zero:
+a_exp = INT_MIN;
+break;
+default:
+g_assert_not_reached();
+break;
+}
+switch (b.cls) {
+case float_class_normal:
+b_exp = b.exp;
+break;
+case float_class_inf:
+b_exp = INT_MAX;
+break;
+case float_class_zero:
+b_exp = INT_MIN;
+break;
+default:
+g_assert_not_reached();
+break;
+}
+
+a_sign = a.sign;
+b_sign = b.sign;
+if (ismag) {
+a_sign = b_sign = 0;
+}
+
+if (a_sign == b_sign) {
+bool a_less = a_exp < b_exp;
+if (a_exp == b_exp) {
+a_less = a.frac < b.frac;
+}
+return a_sign ^ a_less ^ ismin ? b : a;
+} else {
+return a_sign ^ ismin ? b : a;
+}
+}
+}
+
+#define MINMAX(sz, name, ismin, isiee, ismag)   \
+float ## sz float ## sz ## _ ## name(float ## sz a, float ## sz b,  \
+ float_status *s)   \
+{   \
+FloatParts pa = float ## sz ## _unpack_canonical(a, s); \
+FloatParts pb = float ## sz ## _unpack_canonical(b, s); \
+FloatParts pr = minmax_floats(pa, pb, ismin, isiee, ismag, s);  \
+\
+return float ## sz ## _round_pack_canonical(pr, s); \
+}
+
+MINMAX(16, min, true, false, false)
+MINMAX(16, minnum, true, true, false)
+MINMAX(16, minnummag, true, true, true)
+MINMAX(16, max, false, false, false)
+MINMAX(16, maxnum, false, true, false)
+MINMAX(16, maxnummag, false, true, true)
+
+MINMAX(32, min, true, false, false)
+MINMAX(32, minnum, true, true, false)
+MINMAX(32, minnummag, true, true, true)
+MINMAX(32, max, false, false, false)
+MINMAX(32, maxnum, false, true, false)
+MINMAX(32, maxnummag, false, true, true)
+
+MINMAX(64, min, true, false, false)
+MINMAX(64, minnum, true, true, false)
+MINMAX(64, minnummag, true, true, true)
+MINMAX(64, max, false, false, false)
+MINMAX(64, maxnum, false, true, false)
+MINMAX(64, maxnummag, false, true, true)
+
+#undef MINMAX
+
 /* Multiply A by 2 raised to the power N.  */
 static FloatParts scalbn_decomposed(FloatParts a, int n, float_status *s)
 {
@@ -6912,113 +7032,6 @@ int float128_compare_quiet(float128 a, float128 b, 
float_status *status)
 return float128_compare_internal(a, b, 1, status);
 }
 
-/* min() and max() functions. These can't be implemented as

[Qemu-devel] [PULL 14/22] fpu/softfloat: re-factor div

We can now add float16_div and use the common decompose and
canonicalize functions to have a single implementation for
float16/32/64 versions.

Signed-off-by: Alex Bennée 
Signed-off-by: Richard Henderson 
Reviewed-by: Peter Maydell 

diff --git a/fpu/softfloat-macros.h b/fpu/softfloat-macros.h
index 9cc6158cb4..c45a23193e 100644
--- a/fpu/softfloat-macros.h
+++ b/fpu/softfloat-macros.h
@@ -625,6 +625,54 @@ static uint64_t estimateDiv128To64( uint64_t a0, uint64_t 
a1, uint64_t b )
 
 }
 
+/* From the GNU Multi Precision Library - longlong.h __udiv_qrnnd
+ * (https://gmplib.org/repo/gmp/file/tip/longlong.h)
+ *
+ * Licensed under the GPLv2/LGPLv3
+ */
+static uint64_t div128To64(uint64_t n0, uint64_t n1, uint64_t d)
+{
+uint64_t d0, d1, q0, q1, r1, r0, m;
+
+d0 = (uint32_t)d;
+d1 = d >> 32;
+
+r1 = n1 % d1;
+q1 = n1 / d1;
+m = q1 * d0;
+r1 = (r1 << 32) | (n0 >> 32);
+if (r1 < m) {
+q1 -= 1;
+r1 += d;
+if (r1 >= d) {
+if (r1 < m) {
+q1 -= 1;
+r1 += d;
+}
+}
+}
+r1 -= m;
+
+r0 = r1 % d1;
+q0 = r1 / d1;
+m = q0 * d0;
+r0 = (r0 << 32) | (uint32_t)n0;
+if (r0 < m) {
+q0 -= 1;
+r0 += d;
+if (r0 >= d) {
+if (r0 < m) {
+q0 -= 1;
+r0 += d;
+}
+}
+}
+r0 -= m;
+
+/* Return remainder in LSB */
+return (q1 << 32) | q0 | (r0 != 0);
+}
+
 /*
 | Returns an approximation to the square root of the 32-bit significand given
 | by `a'.  Considered as an integer, `a' must be at least 2^31.  If bit 0 of
diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 6d29e1a103..4a859b2721 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -816,6 +816,94 @@ float64 __attribute__((flatten)) float64_mul(float64 a, 
float64 b,
 return float64_round_pack_canonical(pr, status);
 }
 
+/*
+ * Returns the result of dividing the floating-point value `a' by the
+ * corresponding value `b'. The operation is performed according to
+ * the IEC/IEEE Standard for Binary Floating-Point Arithmetic.
+ */
+
+static FloatParts div_floats(FloatParts a, FloatParts b, float_status *s)
+{
+bool sign = a.sign ^ b.sign;
+
+if (a.cls == float_class_normal && b.cls == float_class_normal) {
+uint64_t temp_lo, temp_hi;
+int exp = a.exp - b.exp;
+if (a.frac < b.frac) {
+exp -= 1;
+shortShift128Left(0, a.frac, DECOMPOSED_BINARY_POINT + 1,
+  &temp_hi, &temp_lo);
+} else {
+shortShift128Left(0, a.frac, DECOMPOSED_BINARY_POINT,
+  &temp_hi, &temp_lo);
+}
+/* LSB of quot is set if inexact which roundandpack will use
+ * to set flags. Yet again we re-use a for the result */
+a.frac = div128To64(temp_lo, temp_hi, b.frac);
+a.sign = sign;
+a.exp = exp;
+return a;
+}
+/* handle all the NaN cases */
+if (is_nan(a.cls) || is_nan(b.cls)) {
+return pick_nan(a, b, s);
+}
+/* 0/0 or Inf/Inf */
+if (a.cls == b.cls
+&&
+(a.cls == float_class_inf || a.cls == float_class_zero)) {
+s->float_exception_flags |= float_flag_invalid;
+a.cls = float_class_dnan;
+return a;
+}
+/* Div 0 => Inf */
+if (b.cls == float_class_zero) {
+s->float_exception_flags |= float_flag_divbyzero;
+a.cls = float_class_inf;
+a.sign = sign;
+return a;
+}
+/* Inf / x or 0 / x */
+if (a.cls == float_class_inf || a.cls == float_class_zero) {
+a.sign = sign;
+return a;
+}
+/* Div by Inf */
+if (b.cls == float_class_inf) {
+a.cls = float_class_zero;
+a.sign = sign;
+return a;
+}
+g_assert_not_reached();
+}
+
+float16 float16_div(float16 a, float16 b, float_status *status)
+{
+FloatParts pa = float16_unpack_canonical(a, status);
+FloatParts pb = float16_unpack_canonical(b, status);
+FloatParts pr = div_floats(pa, pb, status);
+
+return float16_round_pack_canonical(pr, status);
+}
+
+float32 float32_div(float32 a, float32 b, float_status *status)
+{
+FloatParts pa = float32_unpack_canonical(a, status);
+FloatParts pb = float32_unpack_canonical(b, status);
+FloatParts pr = div_floats(pa, pb, status);
+
+return float32_round_pack_canonical(pr, status);
+}
+
+float64 float64_div(float64 a, float64 b, float_status *status)
+{
+FloatParts pa = float64_unpack_canonical(a, status);
+FloatParts pb = float64_unpack_canonical(b, status);
+FloatParts pr = div_floats(pa, pb, status);
+
+return float64_round_pack_canonical(pr, status);
+}
+
 /*
 | Takes a 64-bit fixed-point value `absZ' with binary point between bits

[Qemu-devel] [PULL 22/22] fpu/softfloat: re-factor sqrt

This is a little bit of a departure from softfloat's original approach
as we skip the estimate step in favour of a straight iteration. There
is a minor optimisation to avoid calculating more bits of precision
than we need however this still brings a performance drop, especially
for float64 operations.

Suggested-by: Richard Henderson 
Signed-off-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Reviewed-by: Richard Henderson 

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 4bc425d7e4..e7fb0d357a 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -1896,6 +1896,102 @@ float64 float64_scalbn(float64 a, int n, float_status 
*status)
 return float64_round_pack_canonical(pr, status);
 }
 
+/*
+ * Square Root
+ *
+ * The old softfloat code did an approximation step before zeroing in
+ * on the final result. However for simpleness we just compute the
+ * square root by iterating down from the implicit bit to enough extra
+ * bits to ensure we get a correctly rounded result.
+ *
+ * This does mean however the calculation is slower than before,
+ * especially for 64 bit floats.
+ */
+
+static FloatParts sqrt_float(FloatParts a, float_status *s, const FloatFmt *p)
+{
+uint64_t a_frac, r_frac, s_frac;
+int bit, last_bit;
+
+if (is_nan(a.cls)) {
+return return_nan(a, s);
+}
+if (a.cls == float_class_zero) {
+return a;  /* sqrt(+-0) = +-0 */
+}
+if (a.sign) {
+s->float_exception_flags |= float_flag_invalid;
+a.cls = float_class_dnan;
+return a;
+}
+if (a.cls == float_class_inf) {
+return a;  /* sqrt(+inf) = +inf */
+}
+
+assert(a.cls == float_class_normal);
+
+/* We need two overflow bits at the top. Adding room for that is a
+ * right shift. If the exponent is odd, we can discard the low bit
+ * by multiplying the fraction by 2; that's a left shift. Combine
+ * those and we shift right if the exponent is even.
+ */
+a_frac = a.frac;
+if (!(a.exp & 1)) {
+a_frac >>= 1;
+}
+a.exp >>= 1;
+
+/* Bit-by-bit computation of sqrt.  */
+r_frac = 0;
+s_frac = 0;
+
+/* Iterate from implicit bit down to the 3 extra bits to compute a
+ * properly rounded result. Remember we've inserted one more bit
+ * at the top, so these positions are one less.
+ */
+bit = DECOMPOSED_BINARY_POINT - 1;
+last_bit = MAX(p->frac_shift - 4, 0);
+do {
+uint64_t q = 1ULL << bit;
+uint64_t t_frac = s_frac + q;
+if (t_frac <= a_frac) {
+s_frac = t_frac + q;
+a_frac -= t_frac;
+r_frac += q;
+}
+a_frac <<= 1;
+} while (--bit >= last_bit);
+
+/* Undo the right shift done above. If there is any remaining
+ * fraction, the result is inexact. Set the sticky bit.
+ */
+a.frac = (r_frac << 1) + (a_frac != 0);
+
+return a;
+}
+
+float16 __attribute__((flatten)) float16_sqrt(float16 a, float_status *status)
+{
+FloatParts pa = float16_unpack_canonical(a, status);
+FloatParts pr = sqrt_float(pa, status, &float16_params);
+return float16_round_pack_canonical(pr, status);
+}
+
+float32 __attribute__((flatten)) float32_sqrt(float32 a, float_status *status)
+{
+FloatParts pa = float32_unpack_canonical(a, status);
+FloatParts pr = sqrt_float(pa, status, &float32_params);
+return float32_round_pack_canonical(pr, status);
+}
+
+float64 __attribute__((flatten)) float64_sqrt(float64 a, float_status *status)
+{
+FloatParts pa = float64_unpack_canonical(a, status);
+FloatParts pr = sqrt_float(pa, status, &float64_params);
+return float64_round_pack_canonical(pr, status);
+}
+
+
 /*
 | Takes a 64-bit fixed-point value `absZ' with binary point between bits 6
 | and 7, and returns the properly rounded 32-bit integer corresponding to the
@@ -3303,62 +3399,6 @@ float32 float32_rem(float32 a, float32 b, float_status 
*status)
 }
 
 
-/*
-| Returns the square root of the single-precision floating-point value `a'.
-| The operation is performed according to the IEC/IEEE Standard for Binary
-| Floating-Point Arithmetic.
-**/
-
-float32 float32_sqrt(float32 a, float_status *status)
-{
-flag aSign;
-int aExp, zExp;
-uint32_t aSig, zSig;
-uint64_t rem, term;
-a = float32_squash_input_denormal(a, status);
-
-aSig = extractFloat32Frac( a );
-aExp = extractFloat32Exp( a );
-aSign = extractFloat32Sign( a );
-if ( aExp == 0xFF ) {
-if (aSig) {
-return propagateFloat32NaN(a, float32_zero, status);
-}
-if ( ! aSign ) return a;
-float_raise(float_flag_invalid, status);
-return float32_default_nan(status);
-}
-if ( aSign ) {
-if ( ( aExp | aSig ) == 0 ) return

[Qemu-devel] QEMU & KVM are participating in Outreachy May-August 2018!

2018-02-21 Thread Stefan Hajnoczi

QEMU & KVM are participating in the Outreachy open source internship
program that offers 12-week, paid, remote work internships for
individuals from underrepresented groups in open source.

The application period is from February 12 to March 22 2018.  The
internships themselves run May-August.

Our project ideas page is currently being populated and will have more
projects over the coming days:
https://www.outreachy.org/2018-may-august/communities/qemu/

Let me know if you have questions or drop by #qemu-outreachy on
irc.oftc.net to chat with mentors!

Stefan

[Qemu-devel] [PULL 17/22] fpu/softfloat: re-factor float to int/uint

We share the common int64/uint64_pack_decomposed function across all
the helpers and simply limit the final result depending on the final
size.

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 616c6cef07..da0c43c0e7 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -1320,6 +1320,186 @@ float64 float64_trunc_to_int(float64 a, float_status *s)
 return float64_round_pack_canonical(pr, s);
 }
 
+/*
+ * Returns the result of converting the floating-point value `a' to
+ * the two's complement integer format. The conversion is performed
+ * according to the IEC/IEEE Standard for Binary Floating-Point
+ * Arithmetic---which means in particular that the conversion is
+ * rounded according to the current rounding mode. If `a' is a NaN,
+ * the largest positive integer is returned. Otherwise, if the
+ * conversion overflows, the largest integer with the same sign as `a'
+ * is returned.
+*/
+
+static int64_t round_to_int_and_pack(FloatParts in, int rmode,
+ int64_t min, int64_t max,
+ float_status *s)
+{
+uint64_t r;
+int orig_flags = get_float_exception_flags(s);
+FloatParts p = round_to_int(in, rmode, s);
+
+switch (p.cls) {
+case float_class_snan:
+case float_class_qnan:
+return max;
+case float_class_inf:
+return p.sign ? min : max;
+case float_class_zero:
+return 0;
+case float_class_normal:
+if (p.exp < DECOMPOSED_BINARY_POINT) {
+r = p.frac >> (DECOMPOSED_BINARY_POINT - p.exp);
+} else if (p.exp - DECOMPOSED_BINARY_POINT < 2) {
+r = p.frac << (p.exp - DECOMPOSED_BINARY_POINT);
+} else {
+r = UINT64_MAX;
+}
+if (p.sign) {
+if (r < -(uint64_t) min) {
+return -r;
+} else {
+s->float_exception_flags = orig_flags | float_flag_invalid;
+return min;
+}
+} else {
+if (r < max) {
+return r;
+} else {
+s->float_exception_flags = orig_flags | float_flag_invalid;
+return max;
+}
+}
+default:
+g_assert_not_reached();
+}
+}
+
+#define FLOAT_TO_INT(fsz, isz)  \
+int ## isz ## _t float ## fsz ## _to_int ## isz(float ## fsz a, \
+float_status *s)\
+{   \
+FloatParts p = float ## fsz ## _unpack_canonical(a, s); \
+return round_to_int_and_pack(p, s->float_rounding_mode, \
+ INT ## isz ## _MIN, INT ## isz ## _MAX,\
+ s);\
+}   \
+\
+int ## isz ## _t float ## fsz ## _to_int ## isz ## _round_to_zero   \
+ (float ## fsz a, float_status *s)  \
+{   \
+FloatParts p = float ## fsz ## _unpack_canonical(a, s); \
+return round_to_int_and_pack(p, float_round_to_zero,\
+ INT ## isz ## _MIN, INT ## isz ## _MAX,\
+ s);\
+}
+
+FLOAT_TO_INT(16, 16)
+FLOAT_TO_INT(16, 32)
+FLOAT_TO_INT(16, 64)
+
+FLOAT_TO_INT(32, 16)
+FLOAT_TO_INT(32, 32)
+FLOAT_TO_INT(32, 64)
+
+FLOAT_TO_INT(64, 16)
+FLOAT_TO_INT(64, 32)
+FLOAT_TO_INT(64, 64)
+
+#undef FLOAT_TO_INT
+
+/*
+ *  Returns the result of converting the floating-point value `a' to
+ *  the unsigned integer format. The conversion is performed according
+ *  to the IEC/IEEE Standard for Binary Floating-Point
+ *  Arithmetic---which means in particular that the conversion is
+ *  rounded according to the current rounding mode. If `a' is a NaN,
+ *  the largest unsigned integer is returned. Otherwise, if the
+ *  conversion overflows, the largest unsigned integer is returned. If
+ *  the 'a' is negative, the result is rounded and zero is returned;
+ *  values that do not round to zero will raise the inexact exception
+ *  flag.
+ */
+
+static uint64_t round_to_uint_and_pack(FloatParts in, int rmode, uint64_t max,
+   float_status *s)
+{
+int orig_flags = get_float_exception_flags(s);
+FloatParts p = round_to_int(in, rmode, s);
+
+switch (p.cls) {
+case float_class_snan:
+case float_class_qnan:
+s->float_exception_flags = orig_flags | float_flag_invalid;
+return max;
+case float_class_inf:
+return p.sign ? 0 : max;
+case float_class_zero:
+return 0;
+case float_c

[Qemu-devel] [PULL 15/22] fpu/softfloat: re-factor muladd

We can now add float16_muladd and use the common decompose and
canonicalize functions to have a single implementation for
float16/32/64 muladd functions.

Signed-off-by: Alex Bennée 
Signed-off-by: Richard Henderson 
Reviewed-by: Peter Maydell 

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index 4be0fb21ba..e81ca001e1 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -729,58 +729,6 @@ static float32 propagateFloat32NaN(float32 a, float32 b, 
float_status *status)
 }
 }
 
-/*
-| Takes three single-precision floating-point values `a', `b' and `c', one of
-| which is a NaN, and returns the appropriate NaN result.  If any of  `a',
-| `b' or `c' is a signaling NaN, the invalid exception is raised.
-| The input infzero indicates whether a*b was 0*inf or inf*0 (in which case
-| obviously c is a NaN, and whether to propagate c or some other NaN is
-| implementation defined).
-**/
-
-static float32 propagateFloat32MulAddNaN(float32 a, float32 b,
- float32 c, flag infzero,
- float_status *status)
-{
-flag aIsQuietNaN, aIsSignalingNaN, bIsQuietNaN, bIsSignalingNaN,
-cIsQuietNaN, cIsSignalingNaN;
-int which;
-
-aIsQuietNaN = float32_is_quiet_nan(a, status);
-aIsSignalingNaN = float32_is_signaling_nan(a, status);
-bIsQuietNaN = float32_is_quiet_nan(b, status);
-bIsSignalingNaN = float32_is_signaling_nan(b, status);
-cIsQuietNaN = float32_is_quiet_nan(c, status);
-cIsSignalingNaN = float32_is_signaling_nan(c, status);
-
-if (aIsSignalingNaN | bIsSignalingNaN | cIsSignalingNaN) {
-float_raise(float_flag_invalid, status);
-}
-
-which = pickNaNMulAdd(aIsQuietNaN, aIsSignalingNaN,
-  bIsQuietNaN, bIsSignalingNaN,
-  cIsQuietNaN, cIsSignalingNaN, infzero, status);
-
-if (status->default_nan_mode) {
-/* Note that this check is after pickNaNMulAdd so that function
- * has an opportunity to set the Invalid flag.
- */
-return float32_default_nan(status);
-}
-
-switch (which) {
-case 0:
-return float32_maybe_silence_nan(a, status);
-case 1:
-return float32_maybe_silence_nan(b, status);
-case 2:
-return float32_maybe_silence_nan(c, status);
-case 3:
-default:
-return float32_default_nan(status);
-}
-}
-
 #ifdef NO_SIGNALING_NANS
 int float64_is_quiet_nan(float64 a_, float_status *status)
 {
@@ -936,58 +884,6 @@ static float64 propagateFloat64NaN(float64 a, float64 b, 
float_status *status)
 }
 }
 
-/*
-| Takes three double-precision floating-point values `a', `b' and `c', one of
-| which is a NaN, and returns the appropriate NaN result.  If any of  `a',
-| `b' or `c' is a signaling NaN, the invalid exception is raised.
-| The input infzero indicates whether a*b was 0*inf or inf*0 (in which case
-| obviously c is a NaN, and whether to propagate c or some other NaN is
-| implementation defined).
-**/
-
-static float64 propagateFloat64MulAddNaN(float64 a, float64 b,
- float64 c, flag infzero,
- float_status *status)
-{
-flag aIsQuietNaN, aIsSignalingNaN, bIsQuietNaN, bIsSignalingNaN,
-cIsQuietNaN, cIsSignalingNaN;
-int which;
-
-aIsQuietNaN = float64_is_quiet_nan(a, status);
-aIsSignalingNaN = float64_is_signaling_nan(a, status);
-bIsQuietNaN = float64_is_quiet_nan(b, status);
-bIsSignalingNaN = float64_is_signaling_nan(b, status);
-cIsQuietNaN = float64_is_quiet_nan(c, status);
-cIsSignalingNaN = float64_is_signaling_nan(c, status);
-
-if (aIsSignalingNaN | bIsSignalingNaN | cIsSignalingNaN) {
-float_raise(float_flag_invalid, status);
-}
-
-which = pickNaNMulAdd(aIsQuietNaN, aIsSignalingNaN,
-  bIsQuietNaN, bIsSignalingNaN,
-  cIsQuietNaN, cIsSignalingNaN, infzero, status);
-
-if (status->default_nan_mode) {
-/* Note that this check is after pickNaNMulAdd so that function
- * has an opportunity to set the Invalid flag.
- */
-return float64_default_nan(status);
-}
-
-switch (which) {
-case 0:
-return float64_maybe_silence_nan(a, status);
-case 1:
-return float64_maybe_silence_nan(b, status);
-case 2:
-return float64_maybe_silence_nan(c, status);
-case 3:
-default:
-return float64_default_nan(status);
-}
-}
-
 #ifdef NO_SIGNALING_NANS
 int floatx80_is_quiet_nan(floatx80 a_, float_status *status)
 {
diff --git a/fpu

Re: [Qemu-devel] [PATCH] migration: do not transfer ram during bulk storage migration

2018-02-21 Thread Stefan Hajnoczi

On Tue, Feb 20, 2018 at 04:10:03PM +0100, Peter Lieven wrote:
> this patch makes the bulk phase of a block migration to take
> place before we start transferring ram. As the bulk block migration
> can take a long time its pointless to transfer ram during that phase.
> 
> Signed-off-by: Peter Lieven 
> ---
>  migration/ram.c | 8 
>  1 file changed, 8 insertions(+)

This makes sense to me:

Reviewed-by: Stefan Hajnoczi 


signature.asc
Description: PGP signature

[Qemu-devel] [PULL 18/22] fpu/softfloat: re-factor int/uint to float

These are considerably simpler as the lower order integers can just
use the higher order conversion function. As the decomposed fractional
part is a full 64 bit rounding and inexact handling comes from the
pack functions.

Signed-off-by: Alex Bennée 
Reviewed-by: Richard Henderson 

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index da0c43c0e7..4313d3a602 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -1500,6 +1500,169 @@ FLOAT_TO_UINT(64, 64)
 
 #undef FLOAT_TO_UINT
 
+/*
+ * Integer to float conversions
+ *
+ * Returns the result of converting the two's complement integer `a'
+ * to the floating-point format. The conversion is performed according
+ * to the IEC/IEEE Standard for Binary Floating-Point Arithmetic.
+ */
+
+static FloatParts int_to_float(int64_t a, float_status *status)
+{
+FloatParts r;
+if (a == 0) {
+r.cls = float_class_zero;
+r.sign = false;
+} else if (a == (1ULL << 63)) {
+r.cls = float_class_normal;
+r.sign = true;
+r.frac = DECOMPOSED_IMPLICIT_BIT;
+r.exp = 63;
+} else {
+uint64_t f;
+if (a < 0) {
+f = -a;
+r.sign = true;
+} else {
+f = a;
+r.sign = false;
+}
+int shift = clz64(f) - 1;
+r.cls = float_class_normal;
+r.exp = (DECOMPOSED_BINARY_POINT - shift);
+r.frac = f << shift;
+}
+
+return r;
+}
+
+float16 int64_to_float16(int64_t a, float_status *status)
+{
+FloatParts pa = int_to_float(a, status);
+return float16_round_pack_canonical(pa, status);
+}
+
+float16 int32_to_float16(int32_t a, float_status *status)
+{
+return int64_to_float16(a, status);
+}
+
+float16 int16_to_float16(int16_t a, float_status *status)
+{
+return int64_to_float16(a, status);
+}
+
+float32 int64_to_float32(int64_t a, float_status *status)
+{
+FloatParts pa = int_to_float(a, status);
+return float32_round_pack_canonical(pa, status);
+}
+
+float32 int32_to_float32(int32_t a, float_status *status)
+{
+return int64_to_float32(a, status);
+}
+
+float32 int16_to_float32(int16_t a, float_status *status)
+{
+return int64_to_float32(a, status);
+}
+
+float64 int64_to_float64(int64_t a, float_status *status)
+{
+FloatParts pa = int_to_float(a, status);
+return float64_round_pack_canonical(pa, status);
+}
+
+float64 int32_to_float64(int32_t a, float_status *status)
+{
+return int64_to_float64(a, status);
+}
+
+float64 int16_to_float64(int16_t a, float_status *status)
+{
+return int64_to_float64(a, status);
+}
+
+
+/*
+ * Unsigned Integer to float conversions
+ *
+ * Returns the result of converting the unsigned integer `a' to the
+ * floating-point format. The conversion is performed according to the
+ * IEC/IEEE Standard for Binary Floating-Point Arithmetic.
+ */
+
+static FloatParts uint_to_float(uint64_t a, float_status *status)
+{
+FloatParts r = { .sign = false};
+
+if (a == 0) {
+r.cls = float_class_zero;
+} else {
+int spare_bits = clz64(a) - 1;
+r.cls = float_class_normal;
+r.exp = DECOMPOSED_BINARY_POINT - spare_bits;
+if (spare_bits < 0) {
+shift64RightJamming(a, -spare_bits, &a);
+r.frac = a;
+} else {
+r.frac = a << spare_bits;
+}
+}
+
+return r;
+}
+
+float16 uint64_to_float16(uint64_t a, float_status *status)
+{
+FloatParts pa = uint_to_float(a, status);
+return float16_round_pack_canonical(pa, status);
+}
+
+float16 uint32_to_float16(uint32_t a, float_status *status)
+{
+return uint64_to_float16(a, status);
+}
+
+float16 uint16_to_float16(uint16_t a, float_status *status)
+{
+return uint64_to_float16(a, status);
+}
+
+float32 uint64_to_float32(uint64_t a, float_status *status)
+{
+FloatParts pa = uint_to_float(a, status);
+return float32_round_pack_canonical(pa, status);
+}
+
+float32 uint32_to_float32(uint32_t a, float_status *status)
+{
+return uint64_to_float32(a, status);
+}
+
+float32 uint16_to_float32(uint16_t a, float_status *status)
+{
+return uint64_to_float32(a, status);
+}
+
+float64 uint64_to_float64(uint64_t a, float_status *status)
+{
+FloatParts pa = uint_to_float(a, status);
+return float64_round_pack_canonical(pa, status);
+}
+
+float64 uint32_to_float64(uint32_t a, float_status *status)
+{
+return uint64_to_float64(a, status);
+}
+
+float64 uint16_to_float64(uint16_t a, float_status *status)
+{
+return uint64_to_float64(a, status);
+}
+
 /*
 | Takes a 64-bit fixed-point value `absZ' with binary point between bits 6
 | and 7, and returns the properly rounded 32-bit integer corresponding to the
@@ -2591,43 +2754,6 @@ static float128 normalizeRoundAndPackFloat128(flag 
zSign, int32_t zExp,
 
 }
 
-/*
-| Returns the result of converting the 32-b

Re: [Qemu-devel] [PATCH v3 0/7] Improvements and clean-ups related to -net

On 21.02.2018 11:41, Mark Cave-Ayland wrote:
> On 21/02/18 10:18, Thomas Huth wrote:
> 
>> "-net" is a legacy option that often causes confusion and
>> misconfigurations for the users, since most people are not aware
>> of the underlying "vlan" (i.e. hub) concept that is used for this
>> parameter. The prefered way of configuring your network stack is
>> to use "--netdev" instead, which gives you a clean 1:1 connection
>> between your emulated guest hardware and the host network backend.
>>
>> However, there are two reasons why we could not completely deprecate
>> "-net" yet:
>>
>> 1) Convenience:
>> In some cases, it's more convenient to use "-net" instead of "--netdev",
>> e.g. if you just want to have a "tap" network connection, it's faster
>> to type "-net nic -net tap" instead of "--device e1000,netdev=n1
>> --netdev tap,id=n1".
>>
>> 2) On-board NICs:
>> Currently the "-net nic" parameter is the only way to configure on-
>> board NICs on certain (embedded) machines via the nd_table[] array.
>>
>> So beside some generic clean-ups and removal of code that has been
>> marked as deprecated since QEMU 2.10 already, this patch series intro-
>> duces a new parameter "--nic" (in patch 7) which should be able to re-
>> place "-net" in the long run completely: This new convenience parameter
>> can be used to configure the default/on-board guest HW together with a
>> host network backend in a very compact way. To configure a tap backend
>> for the default NIC, you just have to type "--nic tap" here for example.
> 
> Hi Thomas,
> 
> This is a great improvement for configuring on-board NICs! I do have a
> couple of questions about your patchset based on personal experience:
> 
> 1) Does the new -nic syntax support multiple on-board NICs? I remember
> seeing this on some of the ARM boards I was studying when trying to
> implement something similar for SPARC.

Yes, that's possible. If you specify the "--nic" parameter multiple
times, it populates multiple entries in the nd_table[] array, so boards
can pick up multiple NICs from there. For example hw/arm/xlnx-zynqmp.c
initializes up to 4 (XLNX_ZYNQMP_NUM_GEMS) on-board NICs this way.

> 2) Is it possible to provide a convenient wrapper function to handle the
> logic related to determining whether a specified NIC is on-board or not?
> 
> For example take a look at
> https://git.qemu.org/?p=qemu.git;a=blob;f=hw/sparc64/sun4u.c;h=da28ab9413efdbe0bf0e1d3bf2b545577b83d88a;hb=a6e0344fa0e09413324835ae122c4cadd7890231#l596
> where we have a nice dance to check whether the device specified is
> on-board or not, and if it isn't to plug it in as an extra PCI device
> instead. The complication here is that nd_tables[] is populated by
> default (and also with -net nic -net user) but not when someone
> specifies a single card via -netdev.
> 
> From memory this was derived from the code in
> https://git.qemu.org/?p=qemu.git;a=blob;f=hw/arm/realview.c;h=87cd1e583cd20b6d8a2beeef1cba6977496d4477;hb=a6e0344fa0e09413324835ae122c4cadd7890231#l259
> if having another example helps?

Do you have something in mind how the interface to such a function
should look like?
Not sure whether there is a easy, magic solution for all boards ...
there are multiple problems, e.g. that you can normally not specify
on-board NICs via --device yet...
Anyway, that would be subject to another clean-up patch series - this
series here only focuses on the net/* files, not on the files in the hw/
folder.

 Thomas

Re: [Qemu-devel] [PATCH v3 0/7] Improvements and clean-ups related to -net

On 21/02/2018 11:41, Mark Cave-Ayland wrote:
> 1) Does the new -nic syntax support multiple on-board NICs? I remember
> seeing this on some of the ARM boards I was studying when trying to
> implement something similar for SPARC.

Yes, but they will be in different subnets if you do "-nic user -nic
user".  If you want to put them on the same trunk, what you want is
(cut-and-pasted from an offlist email from Thomas):

 -netdev user,id=slirp \
 -netdev hubport,id=port,netdev=slirp,hubid=0 \
 -nic hubport,hubid=0 \
 -nic hubport,hubid=0

We could make id and hubid optional (id was already optional in -net so
the logic is there already, see assign_name in net/net.c), giving the
much nicer:

 -netdev user,id=slirp \
 -netdev hubport,netdev=slirp \
 -nic hubport \
 -nic hubport

Thanks,

Paolo

> 2) Is it possible to provide a convenient wrapper function to handle the
> logic related to determining whether a specified NIC is on-board or not?

Re: [Qemu-devel] [qemu-s390x] [PATCH 1/1] 390x/cpumodel: document S390FeatDef.bit not applicable

2018-02-21 Thread Halil Pasic



On 02/20/2018 05:25 PM, Cornelia Huck wrote:
> On Tue, 20 Feb 2018 17:08:52 +0100
> David Hildenbrand  wrote:
> 
>> On 20.02.2018 17:07, Cornelia Huck wrote:
>>> On Tue, 20 Feb 2018 17:04:19 +0100
>>> Christian Borntraeger  wrote:
>>>   
 On 02/20/2018 04:55 PM, David Hildenbrand wrote:  
> On 20.02.2018 16:53, Cornelia Huck wrote:
>> On Tue, 20 Feb 2018 16:07:13 +0100
>> Halil Pasic  wrote:
>>
>>> The 'bit' field of the 'S390FeatDef' structure is not applicable to all
>>> it's instances. Currently a this field is not applicable, and remains   
>>>  
>>
>> s/it's/its/
>>
>> s/a this/this/
>>
>>> unused, iff the feature is of type S390_FEAT_TYPE_MISC. Having the 
>>> value 0
>>> specified for multiple such feature definition  was a little confusing,
>>> as it's a perfectly legit bit value, and as usually the value of the bit
>>> field is ought to be unique for each feature.
>>>
>>> Let's document this, and hopefully reduce the potential for confusion.
>>>
>>> Signed-off-by: Halil Pasic 
>>> ---
>>>
>>> Hi!
>>>
>>> This may be an overkill. A comment where the misc features
>>> are defined would do to, but I think this is nicer. So
>>> I decided to try it with this approach first.
>>
>> Is there likely to be anything else than FEAT_MISC _not_ using .bit? If
>> not, would it be better to at a comment to the FEAT_MISC definition?
>
> Doubt it right now. I would sign the "overkill" part :)

 I can cconfirm that this code caused some questions and it took me some
 minutes to remember why 0 and 0 was ok. So I certainly want to have a 
 comment
 of some form.
  
>>>
>>> I'd prefer a comment about FEAT_MISC usage rather than a magic value.
>>>   
>>
>> We can also add FEAT_INIT_MISC. And add a comment in the initializer.
>>
> 
> That's what I like best.
> 

OK, seems we have a winner: I will redo this with
#define FEAT_INIT_MISC(_name, _desc) \  
FEAT_INIT(_name, S390_FEAT_TYPE_MISC, 0, _desc)

Everybody thanks for the comments.

Regards,
Halil

[Qemu-devel] [PATCH 00/12] ui: build sdl, gtk and curses as modules

This patch series adds a registry for user interfaces (aka displays),
adds support for user interface modules and allows to build sdl, gtk
and curses as modules.  Especially gtk cuts down the number of shared
libraries qemu links against by a significant amount.

Note one: Modules are disabled by default, so configure with
  --enable-modules to test this.

Note two: Qemu build system doesn't rebuild object files when
  the compiler flags change.  You might see build failures
  when enabling modules without "make clean" because of this,
  due to non-modular object files being built without -fPIC.

Gerd Hoffmann (12):
  console: add qemu display registry, add gtk
  sdl: switch over to new display registry
  cocoa: switch over to new display registry
  curses: switch over to new display registry
  egl-headless: switch over to new display registry
  console: add and use qemu_display_find_default
  console: add ui module loading support
  configure: add X11 vars to config-host.mak
  configure: opengl doesn't depend on x11
  sdl: build as ui module
  gtk: build as ui module
  curses: build as ui module

 configure | 29 +++-
 Makefile.objs |  1 +
 include/qemu/module.h |  1 +
 include/ui/console.h  | 75 ---
 ui/console.c  | 59 
 ui/curses.c   | 14 +-
 ui/egl-headless.c | 20 +-
 ui/gtk.c  | 17 ++--
 ui/sdl.c  | 24 +
 ui/sdl2.c | 17 ++--
 vl.c  | 74 --
 ui/Makefile.objs  | 31 +
 ui/cocoa.m| 14 +-
 13 files changed, 208 insertions(+), 168 deletions(-)

-- 
2.9.3

[Qemu-devel] [PATCH 02/12] sdl: switch over to new display registry

Signed-off-by: Gerd Hoffmann 
---
 include/ui/console.h | 19 ---
 ui/sdl.c | 24 +---
 ui/sdl2.c| 17 +++--
 vl.c | 15 +--
 4 files changed, 29 insertions(+), 46 deletions(-)

diff --git a/include/ui/console.h b/include/ui/console.h
index ce3589aadd..82bbea0242 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -441,25 +441,6 @@ void qemu_display_register(QemuDisplay *ui);
 void qemu_display_early_init(DisplayOptions *opts);
 void qemu_display_init(DisplayState *ds, DisplayOptions *opts);
 
-/* sdl.c */
-#ifdef CONFIG_SDL
-void sdl_display_early_init(DisplayOptions *opts);
-void sdl_display_init(DisplayState *ds, DisplayOptions *opts);
-#else
-static inline void sdl_display_early_init(DisplayOptions *opts)
-{
-/* This must never be called if CONFIG_SDL is disabled */
-error_report("SDL support is disabled");
-abort();
-}
-static inline void sdl_display_init(DisplayState *ds, DisplayOptions *opts)
-{
-/* This must never be called if CONFIG_SDL is disabled */
-error_report("SDL support is disabled");
-abort();
-}
-#endif
-
 /* cocoa.m */
 #ifdef CONFIG_COCOA
 void cocoa_display_init(DisplayState *ds, DisplayOptions *opts);
diff --git a/ui/sdl.c b/ui/sdl.c
index 963cdf77a7..153cbc6d1c 100644
--- a/ui/sdl.c
+++ b/ui/sdl.c
@@ -897,17 +897,7 @@ static const DisplayChangeListenerOps dcl_ops = {
 .dpy_cursor_define= sdl_mouse_define,
 };
 
-void sdl_display_early_init(DisplayOptions *opts)
-{
-if (opts->has_gl && opts->gl) {
-fprintf(stderr,
-"SDL1 display code has no opengl support.\n"
-"Please recompile qemu with SDL2, using\n"
-"./configure --enable-sdl --with-sdlabi=2.0\n");
-}
-}
-
-void sdl_display_init(DisplayState *ds, DisplayOptions *o)
+static void sdl1_display_init(DisplayState *ds, DisplayOptions *o)
 {
 int flags;
 uint8_t data = 0;
@@ -1019,3 +1009,15 @@ void sdl_display_init(DisplayState *ds, DisplayOptions 
*o)
 
 atexit(sdl_cleanup);
 }
+
+static QemuDisplay qemu_display_sdl1 = {
+.type   = DISPLAY_TYPE_SDL,
+.init   = sdl1_display_init,
+};
+
+static void register_sdl1(void)
+{
+qemu_display_register(&qemu_display_sdl1);
+}
+
+type_init(register_sdl1);
diff --git a/ui/sdl2.c b/ui/sdl2.c
index 6e96a4a24c..f17d039650 100644
--- a/ui/sdl2.c
+++ b/ui/sdl2.c
@@ -759,7 +759,7 @@ static const DisplayChangeListenerOps dcl_gl_ops = {
 };
 #endif
 
-void sdl_display_early_init(DisplayOptions *o)
+static void sdl2_display_early_init(DisplayOptions *o)
 {
 assert(o->type == DISPLAY_TYPE_SDL);
 if (o->has_gl && o->gl) {
@@ -769,7 +769,7 @@ void sdl_display_early_init(DisplayOptions *o)
 }
 }
 
-void sdl_display_init(DisplayState *ds, DisplayOptions *o)
+static void sdl2_display_init(DisplayState *ds, DisplayOptions *o)
 {
 int flags;
 uint8_t data = 0;
@@ -869,3 +869,16 @@ void sdl_display_init(DisplayState *ds, DisplayOptions *o)
 
 atexit(sdl_cleanup);
 }
+
+static QemuDisplay qemu_display_sdl2 = {
+.type   = DISPLAY_TYPE_SDL,
+.early_init = sdl2_display_early_init,
+.init   = sdl2_display_init,
+};
+
+static void register_sdl1(void)
+{
+qemu_display_register(&qemu_display_sdl2);
+}
+
+type_init(register_sdl1);
diff --git a/vl.c b/vl.c
index 70458ba708..45900ba7e6 100644
--- a/vl.c
+++ b/vl.c
@@ -2085,7 +2085,6 @@ static void parse_display(const char *p)
 const char *opts;
 
 if (strstart(p, "sdl", &opts)) {
-#ifdef CONFIG_SDL
 dpy.type = DISPLAY_TYPE_SDL;
 while (*opts) {
 const char *nextopt;
@@ -2146,10 +2145,6 @@ static void parse_display(const char *p)
 }
 opts = nextopt;
 }
-#else
-error_report("SDL support is disabled");
-exit(1);
-#endif
 } else if (strstart(p, "vnc", &opts)) {
 if (*opts == '=') {
 vnc_parse(opts + 1, &error_fatal);
@@ -4327,12 +4322,7 @@ int main(int argc, char **argv, char **envp)
  "ignoring option");
 }
 
-if (dpy.type == DISPLAY_TYPE_SDL) {
-sdl_display_early_init(&dpy);
-} else {
-qemu_display_early_init(&dpy);
-}
-
+qemu_display_early_init(&dpy);
 qemu_console_early_init();
 
 if (dpy.has_gl && dpy.gl && display_opengl == 0) {
@@ -4664,9 +4654,6 @@ int main(int argc, char **argv, char **envp)
 case DISPLAY_TYPE_CURSES:
 curses_display_init(ds, &dpy);
 break;
-case DISPLAY_TYPE_SDL:
-sdl_display_init(ds, &dpy);
-break;
 case DISPLAY_TYPE_COCOA:
 cocoa_display_init(ds, &dpy);
 break;
-- 
2.9.3

[Qemu-devel] [PATCH 03/12] cocoa: switch over to new display registry

Signed-off-by: Gerd Hoffmann 
---
 include/ui/console.h | 12 
 vl.c |  3 ---
 ui/cocoa.m   | 14 +-
 3 files changed, 13 insertions(+), 16 deletions(-)

diff --git a/include/ui/console.h b/include/ui/console.h
index 82bbea0242..b97d9ccae4 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -441,18 +441,6 @@ void qemu_display_register(QemuDisplay *ui);
 void qemu_display_early_init(DisplayOptions *opts);
 void qemu_display_init(DisplayState *ds, DisplayOptions *opts);
 
-/* cocoa.m */
-#ifdef CONFIG_COCOA
-void cocoa_display_init(DisplayState *ds, DisplayOptions *opts);
-#else
-static inline void cocoa_display_init(DisplayState *ds, DisplayOptions *opts)
-{
-/* This must never be called if CONFIG_COCOA is disabled */
-error_report("Cocoa support is disabled");
-abort();
-}
-#endif
-
 /* vnc.c */
 void vnc_display_init(const char *id);
 void vnc_display_open(const char *id, Error **errp);
diff --git a/vl.c b/vl.c
index 45900ba7e6..2c3cb4651c 100644
--- a/vl.c
+++ b/vl.c
@@ -4654,9 +4654,6 @@ int main(int argc, char **argv, char **envp)
 case DISPLAY_TYPE_CURSES:
 curses_display_init(ds, &dpy);
 break;
-case DISPLAY_TYPE_COCOA:
-cocoa_display_init(ds, &dpy);
-break;
 default:
 qemu_display_init(ds, &dpy);
 break;
diff --git a/ui/cocoa.m b/ui/cocoa.m
index 90d9aa57ea..8b0dce90cb 100644
--- a/ui/cocoa.m
+++ b/ui/cocoa.m
@@ -1683,7 +1683,7 @@ static void addRemovableDevicesMenuItems(void)
 qapi_free_BlockInfoList(pointerToFree);
 }
 
-void cocoa_display_init(DisplayState *ds, DisplayOptions *opts)
+static void cocoa_display_init(DisplayState *ds, DisplayOptions *opts)
 {
 COCOA_DEBUG("qemu_cocoa: cocoa_display_init\n");
 
@@ -1713,3 +1713,15 @@ void cocoa_display_init(DisplayState *ds, DisplayOptions 
*opts)
  */
 addRemovableDevicesMenuItems();
 }
+
+static QemuDisplay qemu_display_cocoa = {
+.type   = DISPLAY_TYPE_COCOA,
+.init   = cocoa_display_init,
+};
+
+static void register_cocoa(void)
+{
+qemu_display_register(&qemu_display_cocoa);
+}
+
+type_init(register_cocoa);
-- 
2.9.3

[Qemu-devel] [PATCH 06/12] console: add and use qemu_display_find_default

Using the new display registry instead of #ifdefs in vl.c.

Signed-off-by: Gerd Hoffmann 
---
 include/ui/console.h |  1 +
 ui/console.c | 19 +++
 vl.c | 15 +--
 3 files changed, 25 insertions(+), 10 deletions(-)

diff --git a/include/ui/console.h b/include/ui/console.h
index 1832c7eccf..a0d3330056 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -438,6 +438,7 @@ struct QemuDisplay {
 };
 
 void qemu_display_register(QemuDisplay *ui);
+bool qemu_display_find_default(DisplayOptions *opts);
 void qemu_display_early_init(DisplayOptions *opts);
 void qemu_display_init(DisplayState *ds, DisplayOptions *opts);
 
diff --git a/ui/console.c b/ui/console.c
index 8e55a05108..5a63e9dfa2 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -2178,6 +2178,25 @@ void qemu_display_register(QemuDisplay *ui)
 dpys[ui->type] = ui;
 }
 
+bool qemu_display_find_default(DisplayOptions *opts)
+{
+static DisplayType prio[] = {
+DISPLAY_TYPE_GTK,
+DISPLAY_TYPE_SDL,
+DISPLAY_TYPE_COCOA
+};
+int i;
+
+for (i = 0; i < ARRAY_SIZE(prio); i++) {
+if (dpys[prio[i]] == NULL) {
+continue;
+}
+opts->type = prio[i];
+return true;
+}
+return false;
+}
+
 void qemu_display_early_init(DisplayOptions *opts)
 {
 assert(opts->type < DISPLAY_TYPE__MAX);
diff --git a/vl.c b/vl.c
index 47c953f8dc..59e56593f8 100644
--- a/vl.c
+++ b/vl.c
@@ -4285,17 +4285,12 @@ int main(int argc, char **argv, char **envp)
 }
 #endif
 if (dpy.type == DISPLAY_TYPE_DEFAULT && !display_remote) {
-#if defined(CONFIG_GTK)
-dpy.type = DISPLAY_TYPE_GTK;
-#elif defined(CONFIG_SDL)
-dpy.type = DISPLAY_TYPE_SDL;
-#elif defined(CONFIG_COCOA)
-dpy.type = DISPLAY_TYPE_COCOA;
-#elif defined(CONFIG_VNC)
-vnc_parse("localhost:0,to=99,id=default", &error_abort);
-#else
-dpy.type = DISPLAY_TYPE_NONE;
+if (!qemu_display_find_default(&dpy)) {
+dpy.type = DISPLAY_TYPE_NONE;
+#if defined(CONFIG_VNC)
+vnc_parse("localhost:0,to=99,id=default", &error_abort);
 #endif
+}
 }
 if (dpy.type == DISPLAY_TYPE_DEFAULT) {
 dpy.type = DISPLAY_TYPE_NONE;
-- 
2.9.3

[Qemu-devel] [PATCH 01/12] console: add qemu display registry, add gtk

Add a registry for user interfaces.  Add qemu_display_init and
qemu_display_early_init helper functions for display initialization.

Hook up gtk ui as first user.

Signed-off-by: Gerd Hoffmann 
---
 include/ui/console.h | 32 
 ui/console.c | 34 ++
 ui/gtk.c | 17 +++--
 vl.c | 18 ++
 4 files changed, 67 insertions(+), 34 deletions(-)

diff --git a/include/ui/console.h b/include/ui/console.h
index e6b1227bef..ce3589aadd 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -429,6 +429,18 @@ void surface_gl_setup_viewport(QemuGLShader *gls,
int ww, int wh);
 #endif
 
+typedef struct QemuDisplay QemuDisplay;
+
+struct QemuDisplay {
+DisplayType type;
+void (*early_init)(DisplayOptions *opts);
+void (*init)(DisplayState *ds, DisplayOptions *opts);
+};
+
+void qemu_display_register(QemuDisplay *ui);
+void qemu_display_early_init(DisplayOptions *opts);
+void qemu_display_init(DisplayState *ds, DisplayOptions *opts);
+
 /* sdl.c */
 #ifdef CONFIG_SDL
 void sdl_display_early_init(DisplayOptions *opts);
@@ -484,26 +496,6 @@ static inline void curses_display_init(DisplayState *ds, 
DisplayOptions *opts)
 /* input.c */
 int index_from_key(const char *key, size_t key_length);
 
-/* gtk.c */
-#ifdef CONFIG_GTK
-void early_gtk_display_init(DisplayOptions *opts);
-void gtk_display_init(DisplayState *ds, DisplayOptions *opts);
-#else
-static inline void gtk_display_init(DisplayState *ds, DisplayOptions *opts)
-{
-/* This must never be called if CONFIG_GTK is disabled */
-error_report("GTK support is disabled");
-abort();
-}
-
-static inline void early_gtk_display_init(DisplayOptions *opts)
-{
-/* This must never be called if CONFIG_GTK is disabled */
-error_report("GTK support is disabled");
-abort();
-}
-#endif
-
 /* egl-headless.c */
 void egl_headless_init(DisplayOptions *opts);
 
diff --git a/ui/console.c b/ui/console.c
index 36584d039e..8e55a05108 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -2170,6 +2170,40 @@ PixelFormat qemu_default_pixelformat(int bpp)
 return pf;
 }
 
+static QemuDisplay *dpys[DISPLAY_TYPE__MAX];
+
+void qemu_display_register(QemuDisplay *ui)
+{
+assert(ui->type < DISPLAY_TYPE__MAX);
+dpys[ui->type] = ui;
+}
+
+void qemu_display_early_init(DisplayOptions *opts)
+{
+assert(opts->type < DISPLAY_TYPE__MAX);
+if (opts->type == DISPLAY_TYPE_NONE) {
+return;
+}
+if (dpys[opts->type] == NULL) {
+error_report("Display '%s' is not available.",
+ DisplayType_lookup.array[opts->type]);
+exit(1);
+}
+if (dpys[opts->type]->early_init) {
+dpys[opts->type]->early_init(opts);
+}
+}
+
+void qemu_display_init(DisplayState *ds, DisplayOptions *opts)
+{
+assert(opts->type < DISPLAY_TYPE__MAX);
+if (opts->type == DISPLAY_TYPE_NONE) {
+return;
+}
+assert(dpys[opts->type] != NULL);
+dpys[opts->type]->init(ds, opts);
+}
+
 void qemu_chr_parse_vc(QemuOpts *opts, ChardevBackend *backend, Error **errp)
 {
 int val;
diff --git a/ui/gtk.c b/ui/gtk.c
index ab646b70e1..c63408e036 100644
--- a/ui/gtk.c
+++ b/ui/gtk.c
@@ -2297,7 +2297,7 @@ static void gd_create_menus(GtkDisplayState *s)
 
 static gboolean gtkinit;
 
-void gtk_display_init(DisplayState *ds, DisplayOptions *opts)
+static void gtk_display_init(DisplayState *ds, DisplayOptions *opts)
 {
 VirtualConsole *vc;
 
@@ -2407,7 +2407,7 @@ void gtk_display_init(DisplayState *ds, DisplayOptions 
*opts)
 }
 }
 
-void early_gtk_display_init(DisplayOptions *opts)
+static void early_gtk_display_init(DisplayOptions *opts)
 {
 /* The QEMU code relies on the assumption that it's always run in
  * the C locale. Therefore it is not prepared to deal with
@@ -2450,3 +2450,16 @@ void early_gtk_display_init(DisplayOptions *opts)
 type_register(&char_gd_vc_type_info);
 #endif
 }
+
+static QemuDisplay qemu_display_gtk = {
+.type   = DISPLAY_TYPE_GTK,
+.early_init = early_gtk_display_init,
+.init   = gtk_display_init,
+};
+
+static void register_gtk(void)
+{
+qemu_display_register(&qemu_display_gtk);
+}
+
+type_init(register_gtk);
diff --git a/vl.c b/vl.c
index 9e7235df6d..70458ba708 100644
--- a/vl.c
+++ b/vl.c
@@ -2173,7 +2173,6 @@ static void parse_display(const char *p)
 exit(1);
 #endif
 } else if (strstart(p, "gtk", &opts)) {
-#ifdef CONFIG_GTK
 dpy.type = DISPLAY_TYPE_GTK;
 while (*opts) {
 const char *nextopt;
@@ -2205,10 +2204,6 @@ static void parse_display(const char *p)
 }
 opts = nextopt;
 }
-#else
-error_report("GTK support is disabled");
-exit(1);
-#endif
 } else if (strstart(p, "none", &opts)) {
 dpy.type = DISPLAY_TYPE_NONE;
 } else {
@@ -4318,6 +4313,9 @@ int main(int argc, char **argv, char **envp)

[Qemu-devel] [PATCH 09/12] configure: opengl doesn't depend on x11

So remove x11 from pkg-config check and don't
add x11 cflags/libs to opengl cflags/libs.

Signed-off-by: Gerd Hoffmann 
---
 configure | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/configure b/configure
index 2934a4bcff..66c210026a 100755
--- a/configure
+++ b/configure
@@ -3767,9 +3767,9 @@ libs_softmmu="$libs_softmmu $fdt_libs"
 
 if test "$opengl" != "no" ; then
   opengl_pkgs="epoxy libdrm gbm"
-  if $pkg_config $opengl_pkgs x11; then
-opengl_cflags="$($pkg_config --cflags $opengl_pkgs) $x11_cflags"
-opengl_libs="$($pkg_config --libs $opengl_pkgs) $x11_libs"
+  if $pkg_config $opengl_pkgs; then
+opengl_cflags="$($pkg_config --cflags $opengl_pkgs)"
+opengl_libs="$($pkg_config --libs $opengl_pkgs)"
 opengl=yes
 if test "$gtk" = "yes" && $pkg_config --exists "$gtkpackage >= 3.16"; then
 gtk_gl="yes"
-- 
2.9.3

[Qemu-devel] [PATCH 10/12] sdl: build as ui module

Shared library dependencies dropped from qemu-system-*:

libSDL2-2.0.so.0 => /lib64/libSDL2-2.0.so.0

Signed-off-by: Gerd Hoffmann 
---
 configure| 2 +-
 Makefile.objs| 1 +
 ui/Makefile.objs | 3 ++-
 3 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/configure b/configure
index 66c210026a..fe80940bdc 100755
--- a/configure
+++ b/configure
@@ -6030,7 +6030,7 @@ if test "$have_x11" = "yes" -a "$need_x11" = "yes"; then
   echo "X11_LIBS=$x11_libs" >> $config_host_mak
 fi
 if test "$sdl" = "yes" ; then
-  echo "CONFIG_SDL=y" >> $config_host_mak
+  echo "CONFIG_SDL=m" >> $config_host_mak
   echo "CONFIG_SDLABI=$sdlabi" >> $config_host_mak
   echo "SDL_CFLAGS=$sdl_cflags" >> $config_host_mak
   echo "SDL_LIBS=$sdl_libs" >> $config_host_mak
diff --git a/Makefile.objs b/Makefile.objs
index 5dc134818c..57ca6d908b 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -56,6 +56,7 @@ common-obj-y += hw/
 common-obj-y += replay/
 
 common-obj-y += ui/
+common-obj-m += ui/
 common-obj-y += bt-host.o bt-vhci.o
 bt-host.o-cflags := $(BLUEZ_CFLAGS)
 
diff --git a/ui/Makefile.objs b/ui/Makefile.objs
index 9b725b07aa..ef4bd83fde 100644
--- a/ui/Makefile.objs
+++ b/ui/Makefile.objs
@@ -11,7 +11,6 @@ common-obj-y += keymaps.o console.o cursor.o qemu-pixman.o
 common-obj-y += input.o input-keymap.o input-legacy.o
 common-obj-$(CONFIG_LINUX) += input-linux.o
 common-obj-$(CONFIG_SPICE) += spice-core.o spice-input.o spice-display.o
-common-obj-$(CONFIG_SDL) += sdl.mo
 common-obj-$(CONFIG_COCOA) += cocoa.o
 common-obj-$(CONFIG_CURSES) += curses.o
 common-obj-$(CONFIG_VNC) += $(vnc-obj-y)
@@ -22,6 +21,8 @@ common-obj-$(CONFIG_X11) += x_keymap.o
 x_keymap.o-cflags := $(X11_CFLAGS)
 x_keymap.o-libs := $(X11_LIBS)
 
+# ui-sdl module
+common-obj-$(CONFIG_SDL) += sdl.mo
 ifeq ($(CONFIG_SDLABI),1.2)
 sdl.mo-objs := sdl.o sdl_zoom.o
 endif
-- 
2.9.3

[Qemu-devel] [PATCH 08/12] configure: add X11 vars to config-host.mak

Simplifies handling the X11 dependency,
also makes ui/Makefile.objs more readable.

Signed-off-by: Gerd Hoffmann 
---
 configure| 10 --
 ui/Makefile.objs |  5 -
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/configure b/configure
index ed45a3c4dd..2934a4bcff 100755
--- a/configure
+++ b/configure
@@ -2508,9 +2508,8 @@ fi
 
 ##
 # X11 probe
-x11_cflags=
-x11_libs=-lX11
 if $pkg_config --exists "x11"; then
+have_x11=yes
 x11_cflags=$($pkg_config --cflags x11)
 x11_libs=$($pkg_config --libs x11)
 fi
@@ -2543,6 +2542,7 @@ if test "$gtk" != "no"; then
 gtk_libs=$($pkg_config --libs $gtkpackage)
 gtk_version=$($pkg_config --modversion $gtkpackage)
 if $pkg_config --exists "$gtkx11package >= $gtkversion"; then
+need_x11=yes
 gtk_cflags="$gtk_cflags $x11_cflags"
 gtk_libs="$gtk_libs $x11_libs"
 fi
@@ -2911,6 +2911,7 @@ if test "$sdl" = "yes" ; then
 int main(void) { return 0; }
 EOF
   if compile_prog "$sdl_cflags $x11_cflags" "$sdl_libs $x11_libs" ; then
+need_x11=yes
 sdl_cflags="$sdl_cflags $x11_cflags"
 sdl_libs="$sdl_libs $x11_libs"
   fi
@@ -6023,6 +6024,11 @@ if test "$modules" = "yes"; then
   echo "CONFIG_STAMP=_$( (echo $qemu_version; echo $pkgversion; cat $0) | 
$shacmd - | cut -f1 -d\ )" >> $config_host_mak
   echo "CONFIG_MODULES=y" >> $config_host_mak
 fi
+if test "$have_x11" = "yes" -a "$need_x11" = "yes"; then
+  echo "CONFIG_X11=y" >> $config_host_mak
+  echo "X11_CFLAGS=$x11_cflags" >> $config_host_mak
+  echo "X11_LIBS=$x11_libs" >> $config_host_mak
+fi
 if test "$sdl" = "yes" ; then
   echo "CONFIG_SDL=y" >> $config_host_mak
   echo "CONFIG_SDLABI=$sdlabi" >> $config_host_mak
diff --git a/ui/Makefile.objs b/ui/Makefile.objs
index ced7d91a63..9b725b07aa 100644
--- a/ui/Makefile.objs
+++ b/ui/Makefile.objs
@@ -17,7 +17,10 @@ common-obj-$(CONFIG_CURSES) += curses.o
 common-obj-$(CONFIG_VNC) += $(vnc-obj-y)
 common-obj-$(call lnot,$(CONFIG_VNC)) += vnc-stubs.o
 common-obj-$(CONFIG_GTK) += gtk.o
-common-obj-$(if $(CONFIG_WIN32),n,$(if $(CONFIG_SDL),y,$(CONFIG_GTK))) += 
x_keymap.o
+
+common-obj-$(CONFIG_X11) += x_keymap.o
+x_keymap.o-cflags := $(X11_CFLAGS)
+x_keymap.o-libs := $(X11_LIBS)
 
 ifeq ($(CONFIG_SDLABI),1.2)
 sdl.mo-objs := sdl.o sdl_zoom.o
-- 
2.9.3

[Qemu-devel] [PATCH 12/12] curses: build as ui module

Also drop curses libs from libs_softmmu.  Add CURSES_{CFLAGS,LIBS}
variables so we can use them for linking the curses module.

Shared library dependencies dropped from qemu-system-*:

libncursesw.so.5 => /lib64/libncursesw.so.5
libtinfo.so.5 => /lib64/libtinfo.so.5

Signed-off-by: Gerd Hoffmann 
---
 configure| 6 +++---
 ui/Makefile.objs | 6 +-
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/configure b/configure
index cfab17275f..4d0100ec7e 100755
--- a/configure
+++ b/configure
@@ -3268,8 +3268,6 @@ EOF
   unset IFS
   if compile_prog "$curses_inc" "$curses_lib" ; then
 curses_found=yes
-QEMU_CFLAGS="$curses_inc $QEMU_CFLAGS"
-libs_softmmu="$curses_lib $libs_softmmu"
 break
   fi
 done
@@ -6037,7 +6035,9 @@ if test "$cocoa" = "yes" ; then
   echo "CONFIG_COCOA=y" >> $config_host_mak
 fi
 if test "$curses" = "yes" ; then
-  echo "CONFIG_CURSES=y" >> $config_host_mak
+  echo "CONFIG_CURSES=m" >> $config_host_mak
+  echo "CURSES_CFLAGS=$curses_inc" >> $config_host_mak
+  echo "CURSES_LIBS=$curses_lib" >> $config_host_mak
 fi
 if test "$pipe2" = "yes" ; then
   echo "CONFIG_PIPE2=y" >> $config_host_mak
diff --git a/ui/Makefile.objs b/ui/Makefile.objs
index 9a0e8a94f1..dcd54a5287 100644
--- a/ui/Makefile.objs
+++ b/ui/Makefile.objs
@@ -12,7 +12,6 @@ common-obj-y += input.o input-keymap.o input-legacy.o
 common-obj-$(CONFIG_LINUX) += input-linux.o
 common-obj-$(CONFIG_SPICE) += spice-core.o spice-input.o spice-display.o
 common-obj-$(CONFIG_COCOA) += cocoa.o
-common-obj-$(CONFIG_CURSES) += curses.o
 common-obj-$(CONFIG_VNC) += $(vnc-obj-y)
 common-obj-$(call lnot,$(CONFIG_VNC)) += vnc-stubs.o
 
@@ -40,6 +39,11 @@ gtk.mo-objs := gtk.o
 gtk.mo-cflags := $(GTK_CFLAGS) $(VTE_CFLAGS)
 gtk.mo-libs := $(GTK_LIBS) $(VTE_LIBS)
 
+common-obj-$(CONFIG_CURSES) += curses.mo
+curses.mo-objs := curses.o
+curses.mo-cflags := $(CURSES_CFLAGS)
+curses.mo-libs := $(CURSES_LIBS)
+
 ifeq ($(CONFIG_OPENGL),y)
 common-obj-y += shader.o
 common-obj-y += console-gl.o
-- 
2.9.3

[Qemu-devel] [PATCH 04/12] curses: switch over to new display registry

Signed-off-by: Gerd Hoffmann 
---
 include/ui/console.h | 12 
 ui/curses.c  | 14 +-
 vl.c | 17 ++---
 3 files changed, 15 insertions(+), 28 deletions(-)

diff --git a/include/ui/console.h b/include/ui/console.h
index b97d9ccae4..4794c98c9a 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -450,18 +450,6 @@ int vnc_display_pw_expire(const char *id, time_t expires);
 QemuOpts *vnc_parse(const char *str, Error **errp);
 int vnc_init_func(void *opaque, QemuOpts *opts, Error **errp);
 
-/* curses.c */
-#ifdef CONFIG_CURSES
-void curses_display_init(DisplayState *ds, DisplayOptions *opts);
-#else
-static inline void curses_display_init(DisplayState *ds, DisplayOptions *opts)
-{
-/* This must never be called if CONFIG_CURSES is disabled */
-error_report("curses support is disabled");
-abort();
-}
-#endif
-
 /* input.c */
 int index_from_key(const char *key, size_t key_length);
 
diff --git a/ui/curses.c b/ui/curses.c
index 479b77bd03..d55e6d74a1 100644
--- a/ui/curses.c
+++ b/ui/curses.c
@@ -434,7 +434,7 @@ static const DisplayChangeListenerOps dcl_ops = {
 .dpy_text_cursor = curses_cursor_position,
 };
 
-void curses_display_init(DisplayState *ds, DisplayOptions *opts)
+static void curses_display_init(DisplayState *ds, DisplayOptions *opts)
 {
 #ifndef _WIN32
 if (!isatty(1)) {
@@ -455,3 +455,15 @@ void curses_display_init(DisplayState *ds, DisplayOptions 
*opts)
 
 invalidate = 1;
 }
+
+static QemuDisplay qemu_display_curses = {
+.type   = DISPLAY_TYPE_CURSES,
+.init   = curses_display_init,
+};
+
+static void register_curses(void)
+{
+qemu_display_register(&qemu_display_curses);
+}
+
+type_init(register_curses);
diff --git a/vl.c b/vl.c
index 2c3cb4651c..2b4af34fbb 100644
--- a/vl.c
+++ b/vl.c
@@ -2161,12 +2161,7 @@ static void parse_display(const char *p)
 exit(1);
 #endif
 } else if (strstart(p, "curses", &opts)) {
-#ifdef CONFIG_CURSES
 dpy.type = DISPLAY_TYPE_CURSES;
-#else
-error_report("curses support is disabled");
-exit(1);
-#endif
 } else if (strstart(p, "gtk", &opts)) {
 dpy.type = DISPLAY_TYPE_GTK;
 while (*opts) {
@@ -4647,17 +4642,9 @@ int main(int argc, char **argv, char **envp)
 qemu_register_reset(restore_boot_order, g_strdup(boot_order));
 }
 
-ds = init_displaystate();
-
 /* init local displays */
-switch (dpy.type) {
-case DISPLAY_TYPE_CURSES:
-curses_display_init(ds, &dpy);
-break;
-default:
-qemu_display_init(ds, &dpy);
-break;
-}
+ds = init_displaystate();
+qemu_display_init(ds, &dpy);
 
 /* must be after terminal init, SDL library changes signal handlers */
 os_setup_signal_handling();
-- 
2.9.3

[Qemu-devel] [PATCH 11/12] gtk: build as ui module

Also drop gtk and vte libs from libs_softmmu, so the libs are not
pulled in unless the gtk module actually gets loaded.

Shared library dependencies dropped from qemu-system-*:

libEGL.so.1 => /lib64/libEGL.so.1
libGL.so.1 => /lib64/libGL.so.1
libXcomposite.so.1 => /lib64/libXcomposite.so.1
libXcursor.so.1 => /lib64/libXcursor.so.1
libXdamage.so.1 => /lib64/libXdamage.so.1
libXfixes.so.3 => /lib64/libXfixes.so.3
libXinerama.so.1 => /lib64/libXinerama.so.1
libXrandr.so.2 => /lib64/libXrandr.so.2
libXrender.so.1 => /lib64/libXrender.so.1
libXxf86vm.so.1 => /lib64/libXxf86vm.so.1
libatk-1.0.so.0 => /lib64/libatk-1.0.so.0
libatk-bridge-2.0.so.0 => /lib64/libatk-bridge-2.0.so.0
libatspi.so.0 => /lib64/libatspi.so.0
libcairo-gobject.so.2 => /lib64/libcairo-gobject.so.2
libcairo.so.2 => /lib64/libcairo.so.2
libfontconfig.so.1 => /lib64/libfontconfig.so.1
libfreetype.so.6 => /lib64/libfreetype.so.6
libgdk-3.so.0 => /lib64/libgdk-3.so.0
libgdk_pixbuf-2.0.so.0 => /lib64/libgdk_pixbuf-2.0.so.0
libglapi.so.0 => /lib64/libglapi.so.0
libgraphite2.so.3 => /lib64/libgraphite2.so.3
libgtk-3.so.0 => /lib64/libgtk-3.so.0
libharfbuzz.so.0 => /lib64/libharfbuzz.so.0
libpango-1.0.so.0 => /lib64/libpango-1.0.so.0
libpangocairo-1.0.so.0 => /lib64/libpangocairo-1.0.so.0
libpangoft2-1.0.so.0 => /lib64/libpangoft2-1.0.so.0
libpcre2-8.so.0 => /lib64/libpcre2-8.so.0
libthai.so.0 => /lib64/libthai.so.0
libvte-2.91.so.0 => /lib64/libvte-2.91.so.0
libwayland-cursor.so.0 => /lib64/libwayland-cursor.so.0
libwayland-egl.so.1 => /lib64/libwayland-egl.so.1
libxcb-dri2.so.0 => /lib64/libxcb-dri2.so.0
libxcb-dri3.so.0 => /lib64/libxcb-dri3.so.0
libxcb-glx.so.0 => /lib64/libxcb-glx.so.0
libxcb-present.so.0 => /lib64/libxcb-present.so.0
libxcb-render.so.0 => /lib64/libxcb-render.so.0
libxcb-shm.so.0 => /lib64/libxcb-shm.so.0
libxcb-sync.so.1 => /lib64/libxcb-sync.so.1
libxcb-xfixes.so.0 => /lib64/libxcb-xfixes.so.0
libxkbcommon.so.0 => /lib64/libxkbcommon.so.0
libxshmfence.so.1 => /lib64/libxshmfence.so.1

Signed-off-by: Gerd Hoffmann 
---
 configure|  5 ++---
 ui/Makefile.objs | 17 +
 2 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/configure b/configure
index fe80940bdc..cfab17275f 100755
--- a/configure
+++ b/configure
@@ -2546,7 +2546,6 @@ if test "$gtk" != "no"; then
 gtk_cflags="$gtk_cflags $x11_cflags"
 gtk_libs="$gtk_libs $x11_libs"
 fi
-libs_softmmu="$gtk_libs $libs_softmmu"
 gtk="yes"
 elif test "$gtk" = "yes"; then
 feature_not_found "gtk" "Install gtk3-devel"
@@ -2796,7 +2795,6 @@ if test "$vte" != "no"; then
 vte_cflags=$($pkg_config --cflags $vtepackage)
 vte_libs=$($pkg_config --libs $vtepackage)
 vteversion=$($pkg_config --modversion $vtepackage)
-libs_softmmu="$vte_libs $libs_softmmu"
 vte="yes"
 elif test "$vte" = "yes"; then
 if test "$gtkabi" = "3.0"; then
@@ -6136,7 +6134,7 @@ if test "$glib_subprocess" = "yes" ; then
   echo "CONFIG_HAS_GLIB_SUBPROCESS_TESTS=y" >> $config_host_mak
 fi
 if test "$gtk" = "yes" ; then
-  echo "CONFIG_GTK=y" >> $config_host_mak
+  echo "CONFIG_GTK=m" >> $config_host_mak
   echo "CONFIG_GTKABI=$gtkabi" >> $config_host_mak
   echo "GTK_CFLAGS=$gtk_cflags" >> $config_host_mak
   echo "GTK_LIBS=$gtk_libs" >> $config_host_mak
@@ -6187,6 +6185,7 @@ fi
 if test "$vte" = "yes" ; then
   echo "CONFIG_VTE=y" >> $config_host_mak
   echo "VTE_CFLAGS=$vte_cflags" >> $config_host_mak
+  echo "VTE_LIBS=$vte_libs" >> $config_host_mak
 fi
 if test "$virglrenderer" = "yes" ; then
   echo "CONFIG_VIRGL=y" >> $config_host_mak
diff --git a/ui/Makefile.objs b/ui/Makefile.objs
index ef4bd83fde..9a0e8a94f1 100644
--- a/ui/Makefile.objs
+++ b/ui/Makefile.objs
@@ -15,7 +15,6 @@ common-obj-$(CONFIG_COCOA) += cocoa.o
 common-obj-$(CONFIG_CURSES) += curses.o
 common-obj-$(CONFIG_VNC) += $(vnc-obj-y)
 common-obj-$(call lnot,$(CONFIG_VNC)) += vnc-stubs.o
-common-obj-$(CONFIG_GTK) += gtk.o
 
 common-obj-$(CONFIG_X11) += x_keymap.o
 x_keymap.o-cflags := $(X11_CFLAGS)
@@ -35,6 +34,12 @@ endif
 sdl.mo-cflags := $(SDL_CFLAGS)
 sdl.mo-libs := $(SDL_LIBS)
 
+# ui-gtk module
+common-obj-$(CONFIG_GTK) += gtk.mo
+gtk.mo-objs := gtk.o
+gtk.mo-cflags := $(GTK_CFLAGS) $(VTE_CFLAGS)
+gtk.mo-libs := $(GTK_LIBS) $(VTE_LIBS)
+
 ifeq ($(CONFIG_OPENGL),y)
 common-obj-y += shader.o
 common-obj-y += console-gl.o
@@ -42,17 +47,13 @@ common-obj-y += egl-helpers.o
 common-obj-y += egl-context.o
 common-obj-$(CONFIG_OPENGL_DMABUF) += egl-headless.o
 ifeq ($(CONFIG_GTK_GL),y)
-common-obj-$(CONFIG_GTK) += gtk-gl-area.o
+gtk.mo-objs += gtk-gl-area.o
 else
-common-obj-$(CONFIG_GTK) += gtk-egl.o
+gtk.mo-objs += gtk-egl.o
+gtk.mo-libs += $(OPENGL_LIBS)
 endif
 endif
 
-gtk.o-cflags := $(GTK_CFLAGS) $(VTE_CFLAGS)
-gtk-egl.o-cflags := $(GTK_CFLAGS) $(VTE_CFLAGS)
-gtk-gl-area.o-cflags := $(GTK_CFLAGS) $(VTE_CFLAGS)
-
-gtk-egl.o-libs += $(OPENGL_LIBS)
 shader.o-libs += $(OPENGL_LIBS)
 console-gl.o-libs += $(OPE

[Qemu-devel] [PATCH 07/12] console: add ui module loading support

If a requested user interface is not available, try loading it as
module, simliar to block layer modules.  Needed to keep things working
when followup patches start to build user interfaces as modules.

Signed-off-by: Gerd Hoffmann 
---
 include/qemu/module.h | 1 +
 ui/console.c  | 6 ++
 2 files changed, 7 insertions(+)

diff --git a/include/qemu/module.h b/include/qemu/module.h
index 56dd218205..9fea75aaeb 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -53,6 +53,7 @@ typedef enum {
 #define trace_init(function) module_init(function, MODULE_INIT_TRACE)
 
 #define block_module_load_one(lib) module_load_one("block-", lib)
+#define ui_module_load_one(lib) module_load_one("ui-", lib)
 
 void register_module_init(void (*fn)(void), module_init_type type);
 void register_dso_module_init(void (*fn)(void), module_init_type type);
diff --git a/ui/console.c b/ui/console.c
index 5a63e9dfa2..dd663b9127 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -2189,6 +2189,9 @@ bool qemu_display_find_default(DisplayOptions *opts)
 
 for (i = 0; i < ARRAY_SIZE(prio); i++) {
 if (dpys[prio[i]] == NULL) {
+ui_module_load_one(DisplayType_lookup.array[prio[i]]);
+}
+if (dpys[prio[i]] == NULL) {
 continue;
 }
 opts->type = prio[i];
@@ -2204,6 +2207,9 @@ void qemu_display_early_init(DisplayOptions *opts)
 return;
 }
 if (dpys[opts->type] == NULL) {
+ui_module_load_one(DisplayType_lookup.array[opts->type]);
+}
+if (dpys[opts->type] == NULL) {
 error_report("Display '%s' is not available.",
  DisplayType_lookup.array[opts->type]);
 exit(1);
-- 
2.9.3

[Qemu-devel] [PATCH 05/12] egl-headless: switch over to new display registry

Signed-off-by: Gerd Hoffmann 
---
 include/ui/console.h |  3 ---
 ui/egl-headless.c| 20 +++-
 vl.c | 12 
 3 files changed, 19 insertions(+), 16 deletions(-)

diff --git a/include/ui/console.h b/include/ui/console.h
index 4794c98c9a..1832c7eccf 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -453,7 +453,4 @@ int vnc_init_func(void *opaque, QemuOpts *opts, Error 
**errp);
 /* input.c */
 int index_from_key(const char *key, size_t key_length);
 
-/* egl-headless.c */
-void egl_headless_init(DisplayOptions *opts);
-
 #endif
diff --git a/ui/egl-headless.c b/ui/egl-headless.c
index 38b3766548..655ef4eecb 100644
--- a/ui/egl-headless.c
+++ b/ui/egl-headless.c
@@ -154,7 +154,12 @@ static const DisplayChangeListenerOps egl_ops = {
 .dpy_gl_update   = egl_scanout_flush,
 };
 
-void egl_headless_init(DisplayOptions *opts)
+static void early_egl_headless_init(DisplayOptions *opts)
+{
+display_opengl = 1;
+}
+
+static void egl_headless_init(DisplayState *ds, DisplayOptions *opts)
 {
 QemuConsole *con;
 egl_dpy *edpy;
@@ -178,3 +183,16 @@ void egl_headless_init(DisplayOptions *opts)
 register_displaychangelistener(&edpy->dcl);
 }
 }
+
+static QemuDisplay qemu_display_egl = {
+.type   = DISPLAY_TYPE_EGL_HEADLESS,
+.early_init = early_egl_headless_init,
+.init   = egl_headless_init,
+};
+
+static void register_egl(void)
+{
+qemu_display_register(&qemu_display_egl);
+}
+
+type_init(register_egl);
diff --git a/vl.c b/vl.c
index 2b4af34fbb..47c953f8dc 100644
--- a/vl.c
+++ b/vl.c
@@ -2153,13 +2153,7 @@ static void parse_display(const char *p)
 exit(1);
 }
 } else if (strstart(p, "egl-headless", &opts)) {
-#ifdef CONFIG_OPENGL_DMABUF
-display_opengl = 1;
 dpy.type = DISPLAY_TYPE_EGL_HEADLESS;
-#else
-error_report("egl support is disabled");
-exit(1);
-#endif
 } else if (strstart(p, "curses", &opts)) {
 dpy.type = DISPLAY_TYPE_CURSES;
 } else if (strstart(p, "gtk", &opts)) {
@@ -4659,12 +4653,6 @@ int main(int argc, char **argv, char **envp)
 qemu_spice_display_init();
 }
 
-#ifdef CONFIG_OPENGL_DMABUF
-if (dpy.type == DISPLAY_TYPE_EGL_HEADLESS) {
-egl_headless_init(&dpy);
-}
-#endif
-
 if (foreach_device_config(DEV_GDB, gdbserver_start) < 0) {
 exit(1);
 }
-- 
2.9.3

Re: [Qemu-devel] [PATCH v2] specs/qcow2: Fix documentation of the compressed cluster descriptor

On Tue 20 Feb 2018 08:40:43 PM CET, Eric Blake wrote:
>>   Compressed Clusters Descriptor (x = 62 - (cluster_bits - 8)):
>
> I'm looking at how this works for different cluster sizes.  If we have
> 512-byte clusters, x is 61, and we DON'T have the 'number sectors'
> field at all!

Well, you can definitely have compressed images with 512-byte clusters.

So I think he have just found one more mistake in the documentation :)

 (x = 62 - (cluster_bits - 8)):

Bit  0 -  x:Host cluster offset.
   x+1 - 61:Number of 512-byte sectors

That's not how it works, it's rather [0, x-1], [x, 61]. For 512-byte
clusters x is 61 and we have 1 bit for the number of sectors, allowing
one or two sectors.

If you have a compressed image with 512-byte clusters you can also see
that since the compressed data is not aligned, some compressed clusters
span two different sectors (as expected).

That means that nb_csectors in the L2 entry is two (1+1), which is the
maximum allowed in this case, so that makes sense. And since the size of
our clusters is also 512 bytes, nb_csectors is twice the cluster size,
so we need s->cluster_data to be (cluster_size * 2) bytes (minus one,
strictly speaking).

> If we ever allowed a compressed cluster to spill across two host
> clusters, it would cause mayhem in trying to track refcounts and other
> things.

I haven't checked how this works in practice but it seems to work fine.
Note that those clusters are read-only so that surely makes things
easier.

Berto

Re: [Qemu-devel] [PATCH 0/2] Firmware blob and git submodule for Sam460ex

2018-02-21 Thread Peter Maydell

On 20 February 2018 at 20:44, Emilio G. Cota  wrote:
> On Tue, Feb 20, 2018 at 18:31:17 +, Peter Maydell wrote:
>> On 20 February 2018 at 18:10, BALATON Zoltan  wrote:
>> > I've created a git repo for the Sam460ex u-boot sources and this adds
>> > that as a submodule and a separate patch to add the binary built from
>> > these sources. Feel free to keep this as two patches, squash them into
>> > one patch or take the git repo and commit the content under the QEMU
>> > repo and use that as a submodule as you see fit (or let me know if any
>> > changes are needed for these patches).
>> >
>> > BALATON Zoltan (2):
>> >   roms: Added git submodule for u-boot-sam460 (firmware for sam460ex)
>> >   pc-bios: Added u-boot-sam460 firmware binary
>>
>> We already have a submodule for u-boot. Is it not possible to
>> build this bios blob from those upstream u-boot sources?
>
> This is discussed in the following thread:
>   Re: [Qemu-ppc] [PATCH v3 2/2] ppc: Add aCube Sam460ex board
>   http://lists.gnu.org/archive/html/qemu-ppc/2018-02/msg00268.html

If upstream u-boot have abandoned the board support I'm not very
enthusiastic about our taking it on :-(

thanks
-- PMM

[Qemu-devel] [PATCH v2 01/36] block/qapi: Introduce BlockdevCreateOptions

This creates a BlockdevCreateOptions union type that will contain all of
the options for image creation. We'll start out with an empty struct
type BlockdevCreateNotSupported for all drivers.

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 qapi/block-core.json | 62 
 1 file changed, 62 insertions(+)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 5c5921bfb7..d256cefc79 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3359,6 +3359,68 @@
 { 'command': 'blockdev-del', 'data': { 'node-name': 'str' } }
 
 ##
+# @BlockdevCreateNotSupported:
+#
+# This is used for all drivers that don't support creating images.
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockdevCreateNotSupported', 'data': {}}
+
+##
+# @BlockdevCreateOptions:
+#
+# Options for creating an image format on a given node.
+#
+# @driver   block driver to create the image format
+#
+# Since: 2.12
+##
+{ 'union': 'BlockdevCreateOptions',
+  'base': {
+  'driver': 'BlockdevDriver' },
+  'discriminator': 'driver',
+  'data': {
+  'blkdebug':   'BlockdevCreateNotSupported',
+  'blkverify':  'BlockdevCreateNotSupported',
+  'bochs':  'BlockdevCreateNotSupported',
+  'cloop':  'BlockdevCreateNotSupported',
+  'dmg':'BlockdevCreateNotSupported',
+  'file':   'BlockdevCreateNotSupported',
+  'ftp':'BlockdevCreateNotSupported',
+  'ftps':   'BlockdevCreateNotSupported',
+  'gluster':'BlockdevCreateNotSupported',
+  'host_cdrom': 'BlockdevCreateNotSupported',
+  'host_device':'BlockdevCreateNotSupported',
+  'http':   'BlockdevCreateNotSupported',
+  'https':  'BlockdevCreateNotSupported',
+  'iscsi':  'BlockdevCreateNotSupported',
+  'luks':   'BlockdevCreateNotSupported',
+  'nbd':'BlockdevCreateNotSupported',
+  'nfs':'BlockdevCreateNotSupported',
+  'null-aio':   'BlockdevCreateNotSupported',
+  'null-co':'BlockdevCreateNotSupported',
+  'nvme':   'BlockdevCreateNotSupported',
+  'parallels':  'BlockdevCreateNotSupported',
+  'qcow2':  'BlockdevCreateNotSupported',
+  'qcow':   'BlockdevCreateNotSupported',
+  'qed':'BlockdevCreateNotSupported',
+  'quorum': 'BlockdevCreateNotSupported',
+  'raw':'BlockdevCreateNotSupported',
+  'rbd':'BlockdevCreateNotSupported',
+  'replication':'BlockdevCreateNotSupported',
+  'sheepdog':   'BlockdevCreateNotSupported',
+  'ssh':'BlockdevCreateNotSupported',
+  'throttle':   'BlockdevCreateNotSupported',
+  'vdi':'BlockdevCreateNotSupported',
+  'vhdx':   'BlockdevCreateNotSupported',
+  'vmdk':   'BlockdevCreateNotSupported',
+  'vpc':'BlockdevCreateNotSupported',
+  'vvfat':  'BlockdevCreateNotSupported',
+  'vxhs':   'BlockdevCreateNotSupported'
+  } }
+
+##
 # @blockdev-open-tray:
 #
 # Opens a block device's tray. If there is a block driver state tree inserted 
as
-- 
2.13.6

[Qemu-devel] [PATCH v2 02/36] block/qapi: Add qcow2 create options to schema

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 qapi/block-core.json | 45 -
 1 file changed, 44 insertions(+), 1 deletion(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index d256cefc79..74b864d64e 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3359,6 +3359,49 @@
 { 'command': 'blockdev-del', 'data': { 'node-name': 'str' } }
 
 ##
+# @BlockdevQcow2Version:
+#
+# @v2:  The original QCOW2 format as introduced in qemu 0.10 (version 2)
+# @v3:  The extended QCOW2 format as introduced in qemu 1.1 (version 3)
+#
+# Since: 2.12
+##
+{ 'enum': 'BlockdevQcow2Version',
+  'data': [ 'v2', 'v3' ] }
+
+
+##
+# @BlockdevCreateOptionsQcow2:
+#
+# Driver specific image creation options for qcow2.
+#
+# @file Node to create the image format on
+# @size Size of the virtual disk in bytes
+# @version  Compatibility level (default: v3)
+# @backing-file File name of the backing file if a backing file
+#   should be used
+# @backing-fmt  Name of the block driver to use for the backing file
+# @encrypt  Encryption options if the image should be encrypted
+# @cluster-size qcow2 cluster size in bytes (default: 65536)
+# @preallocationPreallocation mode for the new image (default: off)
+# @lazy-refcounts   True if refcounts may be updated lazily (default: off)
+# @refcount-bitsWidth of reference counts in bits (default: 16)
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockdevCreateOptionsQcow2',
+  'data': { 'file': 'BlockdevRef',
+'size': 'size',
+'*version': 'BlockdevQcow2Version',
+'*backing-file':'str',
+'*backing-fmt': 'BlockdevDriver',
+'*encrypt': 'QCryptoBlockCreateOptions',
+'*cluster-size':'size',
+'*preallocation':   'PreallocMode',
+'*lazy-refcounts':  'bool',
+'*refcount-bits':   'int' } }
+
+##
 # @BlockdevCreateNotSupported:
 #
 # This is used for all drivers that don't support creating images.
@@ -3402,7 +3445,7 @@
   'null-co':'BlockdevCreateNotSupported',
   'nvme':   'BlockdevCreateNotSupported',
   'parallels':  'BlockdevCreateNotSupported',
-  'qcow2':  'BlockdevCreateNotSupported',
+  'qcow2':  'BlockdevCreateOptionsQcow2',
   'qcow':   'BlockdevCreateNotSupported',
   'qed':'BlockdevCreateNotSupported',
   'quorum': 'BlockdevCreateNotSupported',
-- 
2.13.6

[Qemu-devel] [PATCH v2 04/36] qcow2: Pass BlockdevCreateOptions to qcow2_create2()

All of the simple options are now passed to qcow2_create2() in a
BlockdevCreateOptions object. Still missing: node-name and the
encryption options.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 190 ++
 1 file changed, 152 insertions(+), 38 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index dc6cdea113..22194180c6 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2665,19 +2665,26 @@ static int64_t qcow2_calc_prealloc_size(int64_t 
total_size,
 return meta_size + aligned_total_size;
 }
 
-static size_t qcow2_opt_get_cluster_size_del(QemuOpts *opts, Error **errp)
+static bool validate_cluster_size(size_t cluster_size, Error **errp)
 {
-size_t cluster_size;
-int cluster_bits;
-
-cluster_size = qemu_opt_get_size_del(opts, BLOCK_OPT_CLUSTER_SIZE,
- DEFAULT_CLUSTER_SIZE);
-cluster_bits = ctz32(cluster_size);
+int cluster_bits = ctz32(cluster_size);
 if (cluster_bits < MIN_CLUSTER_BITS || cluster_bits > MAX_CLUSTER_BITS ||
 (1 << cluster_bits) != cluster_size)
 {
 error_setg(errp, "Cluster size must be a power of two between %d and "
"%dk", 1 << MIN_CLUSTER_BITS, 1 << (MAX_CLUSTER_BITS - 10));
+return false;
+}
+return true;
+}
+
+static size_t qcow2_opt_get_cluster_size_del(QemuOpts *opts, Error **errp)
+{
+size_t cluster_size;
+
+cluster_size = qemu_opt_get_size_del(opts, BLOCK_OPT_CLUSTER_SIZE,
+ DEFAULT_CLUSTER_SIZE);
+if (!validate_cluster_size(cluster_size, errp)) {
 return 0;
 }
 return cluster_size;
@@ -2725,12 +2732,11 @@ static uint64_t 
qcow2_opt_get_refcount_bits_del(QemuOpts *opts, int version,
 return refcount_bits;
 }
 
-static int qcow2_create2(BlockDriverState *bs, int64_t total_size,
- const char *backing_file, const char *backing_format,
- int flags, size_t cluster_size, PreallocMode prealloc,
- QemuOpts *opts, int version, int refcount_order,
- const char *encryptfmt, Error **errp)
+static int qcow2_create2(BlockDriverState *bs,
+ BlockdevCreateOptions *create_options,
+ QemuOpts *opts, const char *encryptfmt, Error **errp)
 {
+BlockdevCreateOptionsQcow2 *qcow2_opts;
 QDict *options;
 
 /*
@@ -2747,10 +2753,92 @@ static int qcow2_create2(BlockDriverState *bs, int64_t 
total_size,
  */
 BlockBackend *blk;
 QCowHeader *header;
+size_t cluster_size;
+int version;
+int refcount_order;
 uint64_t* refcount_table;
 Error *local_err = NULL;
 int ret;
 
+/* Validate options and set default values */
+assert(create_options->driver == BLOCKDEV_DRIVER_QCOW2);
+qcow2_opts = &create_options->u.qcow2;
+
+if (!QEMU_IS_ALIGNED(qcow2_opts->size, BDRV_SECTOR_SIZE)) {
+error_setg(errp, "Image size must be a multiple of 512 bytes");
+ret = -EINVAL;
+goto out;
+}
+
+if (qcow2_opts->has_version) {
+switch (qcow2_opts->version) {
+case BLOCKDEV_QCOW2_VERSION_V2:
+version = 2;
+break;
+case BLOCKDEV_QCOW2_VERSION_V3:
+version = 3;
+break;
+default:
+g_assert_not_reached();
+}
+} else {
+version = 3;
+}
+
+if (qcow2_opts->has_cluster_size) {
+cluster_size = qcow2_opts->cluster_size;
+} else {
+cluster_size = DEFAULT_CLUSTER_SIZE;
+}
+
+if (!validate_cluster_size(cluster_size, errp)) {
+return -EINVAL;
+}
+
+if (!qcow2_opts->has_preallocation) {
+qcow2_opts->preallocation = PREALLOC_MODE_OFF;
+}
+if (qcow2_opts->has_backing_file &&
+qcow2_opts->preallocation != PREALLOC_MODE_OFF)
+{
+error_setg(errp, "Backing file and preallocation cannot be used at "
+   "the same time");
+return -EINVAL;
+}
+if (qcow2_opts->has_backing_fmt && !qcow2_opts->has_backing_file) {
+error_setg(errp, "Backing format cannot be used without backing file");
+return -EINVAL;
+}
+
+if (!qcow2_opts->has_lazy_refcounts) {
+qcow2_opts->lazy_refcounts = false;
+}
+if (version < 3 && qcow2_opts->lazy_refcounts) {
+error_setg(errp, "Lazy refcounts only supported with compatibility "
+   "level 1.1 and above (use compat=1.1 or greater)");
+return -EINVAL;
+}
+
+if (!qcow2_opts->has_refcount_bits) {
+qcow2_opts->refcount_bits = 16;
+}
+if (qcow2_opts->refcount_bits > 64 ||
+!is_power_of_2(qcow2_opts->refcount_bits))
+{
+error_setg(errp, "Refcount width must be a power of two and may not "
+   "exceed 64 bits");
+return -EINVAL;
+}
+if (version < 3 && qcow2_opts->

[Qemu-devel] [PATCH v2 09/36] test-qemu-opts: Test qemu_opts_append()

Basic test for merging two QemuOptsLists.

Signed-off-by: Kevin Wolf 
---
 tests/test-qemu-opts.c | 128 +
 1 file changed, 128 insertions(+)

diff --git a/tests/test-qemu-opts.c b/tests/test-qemu-opts.c
index 5d5a3daa7b..6c3183390b 100644
--- a/tests/test-qemu-opts.c
+++ b/tests/test-qemu-opts.c
@@ -23,6 +23,8 @@ static QemuOptsList opts_list_01 = {
 {
 .name = "str1",
 .type = QEMU_OPT_STRING,
+.help = "Help texts are preserved in qemu_opts_append",
+.def_value_str = "default",
 },{
 .name = "str2",
 .type = QEMU_OPT_STRING,
@@ -32,6 +34,7 @@ static QemuOptsList opts_list_01 = {
 },{
 .name = "number1",
 .type = QEMU_OPT_NUMBER,
+.help = "Having help texts only for some options is okay",
 },{
 .name = "number2",
 .type = QEMU_OPT_NUMBER,
@@ -743,6 +746,129 @@ static void test_opts_parse_size(void)
 qemu_opts_reset(&opts_list_02);
 }
 
+static void append_verify_list_01(QemuOptDesc *desc, bool with_overlapping)
+{
+int i = 0;
+
+if (with_overlapping) {
+g_assert_cmpstr(desc[i].name, ==, "str1");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_STRING);
+g_assert_cmpstr(desc[i].help, ==,
+"Help texts are preserved in qemu_opts_append");
+g_assert_cmpstr(desc[i].def_value_str, ==, "default");
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "str2");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_STRING);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+}
+
+g_assert_cmpstr(desc[i].name, ==, "str3");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_STRING);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "number1");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_NUMBER);
+g_assert_cmpstr(desc[i].help, ==,
+"Having help texts only for some options is okay");
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "number2");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_NUMBER);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, NULL);
+}
+
+static void append_verify_list_02(QemuOptDesc *desc)
+{
+int i = 0;
+
+g_assert_cmpstr(desc[i].name, ==, "str1");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_STRING);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "str2");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_STRING);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "bool1");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_BOOL);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "bool2");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_BOOL);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "size1");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_SIZE);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "size2");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_SIZE);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+i++;
+
+g_assert_cmpstr(desc[i].name, ==, "size3");
+g_assert_cmpint(desc[i].type, ==, QEMU_OPT_SIZE);
+g_assert_cmpstr(desc[i].help, ==, NULL);
+g_assert_cmpstr(desc[i].def_value_str, ==, NULL);
+}
+
+static void test_opts_append_to_null(void)
+{
+QemuOptsList *merged;
+
+merged = qemu_opts_append(NULL, &opts_list_01);
+g_assert(merged != &opts_list_01);
+
+g_assert_cmpstr(merged->name, ==, NULL);
+g_assert_cmpstr(merged->implied_opt_name, ==, NULL);
+g_assert_false(merged->merge_lists);
+
+append_verify_list_01(merged->desc, true);
+
+qemu_opts_free(merged);
+}
+
+static void test_opts_append(void)
+{
+QemuOptsList *first, *merged;
+
+first = qemu_opts_append(NULL, &opts_list_02);
+merged = qemu_opts_append(first, &opts_list_01);
+g_assert(first != &opts_list_02);
+g_assert(merged != &opts_list_01);
+
+g_assert_cmpstr(merged->name, ==, NULL);
+g_assert_cmpstr(merged->implied_opt_name, ==, NULL);
+g_assert_false(merged->merge_lists);
+
+append_verify_list_02(&merged->desc[0]);
+append_verify_list_01(

[Qemu-devel] [PATCH v2 00/36] x-blockdev-create for protocols and qcow2

This series implements a minimal QMP command that allows to create an
image file on the protocol level or an image format on a given block
node.

Eventually, the interface is going to change to some kind of an async
command (possibly a (non-)block job), but that will require more work on
the job infrastructure first, so let's first QAPIfy image creation in
the block drivers. In this series, I'm going for a synchronous command
that is prefixed with x- for now.

This series converts qcow2 and all protocol drivers that allow an actual
image creation. This means that drivers which only check if the already
existing storage is good enough are not converted (e.g. host_device,
iscsi). The old behaviour was useful because 'qemu-img create' wants to
create both protocol and format layer, but with the separation in QMP,
you can just leave out the protocol layer creation when the device
already exists.

Please note that for some of the protocol drivers (gluster, rbd and
sheepdog) I don't have a test setup ready. For those, I only tested
with a fake server address to check that the option are parsed correctly
up to this point and an appropriate error is returned without crashing.

If you are a maintainer of one of these protocols and you are
interested in keeping image creation working for your protocol, you
probably want to test this series on a real setup and give me some
feedback. If you don't, I'll just merge the patches and hope that they
won't break anything.

v2:
- Patch 1 ('block/qapi: Introduce BlockdevCreateOptions'):
  Added nvme as unsupported driver

- Patch 8 ('util: Add qemu_opts_to_qdict_filtered'):
  Fixed use after free with QemuOpts that contained more than one option
  with the same name, documented the behaviour with them

- Patches 9 and 10 (new):
  Added unit tests for qemu_opts_append() and
  qemu_opts_to_qdict_filtered()

- Patch 11 ('qdict: Introduce qdict_rename_keys()'):
  Added unit test, improved documentation

- Patch 12 ('qcow2: Use visitor for options in qcow2_create()'):
  Improved commit message, removed unnecessary movement of declaration

- Patches 13 and 14 ('block: x-blockdev-create QMP command'):
  Move making bdrv_is_whitelisted() public into a separate patch,
  use read-write driver whitelist instead of read-only

- Patch 17 ('gluster: Support .bdrv_co_create'):
  Rebased on top of preallocated truncate

- Patches 18-24 ('rbd: Support .bdrv_co_create'):
  QAPIfied .bdrv_open() implementation so that it can be shared with
  .bdrv_co_create() and specified servers are actually used instead of
  silently ignored

- Patch 25 ('nfs: Use QAPI options in nfs_client_open()'):
  Fixed use of uninitialised variable in the error path

- Patch 27 ('sheepdog: QAPIfy "redundacy" create option'):
  Addressed FIXME to use qemu_strtol()

- Patch 28 ('sheepdog: Support .bdrv_co_create'):
  Renamed 'backing_file' to 'backing-file', rebased on top of
  preallocated truncate


git-backport-diff compared to v1:

Key:
[] : patches are identical
[] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/36:[0001] [FC] 'block/qapi: Introduce BlockdevCreateOptions'
002/36:[] [-C] 'block/qapi: Add qcow2 create options to schema'
003/36:[] [--] 'qcow2: Let qcow2_create() handle protocol layer'
004/36:[] [--] 'qcow2: Pass BlockdevCreateOptions to qcow2_create2()'
005/36:[] [-C] 'qcow2: Use BlockdevRef in qcow2_create2()'
006/36:[] [--] 'qcow2: Use QCryptoBlockCreateOptions in qcow2_create2()'
007/36:[] [--] 'qcow2: Handle full/falloc preallocation in qcow2_create2()'
008/36:[0007] [FC] 'util: Add qemu_opts_to_qdict_filtered()'
009/36:[down] 'test-qemu-opts: Test qemu_opts_append()'
010/36:[down] 'test-qemu-opts: Test qemu_opts_to_qdict_filtered()'
011/36:[0117] [FC] 'qdict: Introduce qdict_rename_keys()'
012/36:[0002] [FC] 'qcow2: Use visitor for options in qcow2_create()'
013/36:[down] 'block: Make bdrv_is_whitelisted() public'
014/36:[0006] [FC] 'block: x-blockdev-create QMP command'
015/36:[] [-C] 'file-posix: Support .bdrv_co_create'
016/36:[] [--] 'file-win32: Support .bdrv_co_create'
017/36:[0130] [FC] 'gluster: Support .bdrv_co_create'
018/36:[down] 'rbd: Fix use after free in qemu_rbd_set_keypairs() error path'
019/36:[down] 'rbd: Factor out qemu_rbd_connect()'
020/36:[down] 'rbd: Remove non-schema options from runtime_opts'
021/36:[down] 'rbd: Pass BlockdevOptionsRbd to qemu_rbd_connect()'
022/36:[0034] [FC] 'rbd: Support .bdrv_co_create'
023/36:[down] 'rbd: Assing s->snap/image_name in qemu_rbd_open()'
024/36:[down] 'rbd: Use qemu_rbd_connect() in qemu_rbd_do_create()'
025/36:[0002] [FC] 'nfs: Use QAPI options in nfs_client_open()'
026/36:[] [-C] 'nfs: Support .bdrv_co_create'
027/36:[0013] [FC] 'sheepdog: QAPIfy "redundacy" create option'
028/36:[0049] [FC] 'sheepdog: Support .bdrv_co_create'
029/36:[] [-C] 'ssh: Use

[Qemu-devel] [PATCH v2 03/36] qcow2: Let qcow2_create() handle protocol layer

Currently, qcow2_create() only parses the QemuOpts and then calls
qcow2_create2() for the actual image creation, which includes both the
creation of the actual file on the file system and writing a valid empty
qcow2 image into that file.

The plan is that qcow2_create2() becomes the function that implements
the functionality for a future 'blockdev-create' QMP command, which only
creates the qcow2 layer on an already opened file node.

This is a first step towards that goal: Let's move out anything that
deals with the protocol layer from qcow2_create2() into qcow2_create().
This means that qcow2_create2() doesn't need a file name any more.

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 64 +++
 1 file changed, 38 insertions(+), 26 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 288b5299d8..dc6cdea113 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2725,7 +2725,7 @@ static uint64_t qcow2_opt_get_refcount_bits_del(QemuOpts 
*opts, int version,
 return refcount_bits;
 }
 
-static int qcow2_create2(const char *filename, int64_t total_size,
+static int qcow2_create2(BlockDriverState *bs, int64_t total_size,
  const char *backing_file, const char *backing_format,
  int flags, size_t cluster_size, PreallocMode prealloc,
  QemuOpts *opts, int version, int refcount_order,
@@ -2751,28 +2751,11 @@ static int qcow2_create2(const char *filename, int64_t 
total_size,
 Error *local_err = NULL;
 int ret;
 
-if (prealloc == PREALLOC_MODE_FULL || prealloc == PREALLOC_MODE_FALLOC) {
-int64_t prealloc_size =
-qcow2_calc_prealloc_size(total_size, cluster_size, refcount_order);
-qemu_opt_set_number(opts, BLOCK_OPT_SIZE, prealloc_size, &error_abort);
-qemu_opt_set(opts, BLOCK_OPT_PREALLOC, PreallocMode_str(prealloc),
- &error_abort);
-}
-
-ret = bdrv_create_file(filename, opts, &local_err);
+blk = blk_new(BLK_PERM_WRITE | BLK_PERM_RESIZE, BLK_PERM_ALL);
+ret = blk_insert_bs(blk, bs, errp);
 if (ret < 0) {
-error_propagate(errp, local_err);
-return ret;
-}
-
-blk = blk_new_open(filename, NULL, NULL,
-   BDRV_O_RDWR | BDRV_O_RESIZE | BDRV_O_PROTOCOL,
-   &local_err);
-if (blk == NULL) {
-error_propagate(errp, local_err);
-return -EIO;
+goto out;
 }
-
 blk_set_allow_write_beyond_eof(blk, true);
 
 /* Write the header */
@@ -2827,7 +2810,8 @@ static int qcow2_create2(const char *filename, int64_t 
total_size,
  */
 options = qdict_new();
 qdict_put_str(options, "driver", "qcow2");
-blk = blk_new_open(filename, NULL, options,
+qdict_put_str(options, "file", bs->node_name);
+blk = blk_new_open(NULL, NULL, options,
BDRV_O_RDWR | BDRV_O_RESIZE | BDRV_O_NO_FLUSH,
&local_err);
 if (blk == NULL) {
@@ -2899,7 +2883,8 @@ static int qcow2_create2(const char *filename, int64_t 
total_size,
  */
 options = qdict_new();
 qdict_put_str(options, "driver", "qcow2");
-blk = blk_new_open(filename, NULL, options,
+qdict_put_str(options, "file", bs->node_name);
+blk = blk_new_open(NULL, NULL, options,
BDRV_O_RDWR | BDRV_O_NO_BACKING | BDRV_O_NO_IO,
&local_err);
 if (blk == NULL) {
@@ -2929,6 +2914,7 @@ static int qcow2_create(const char *filename, QemuOpts 
*opts, Error **errp)
 uint64_t refcount_bits;
 int refcount_order;
 char *encryptfmt = NULL;
+BlockDriverState *bs = NULL;
 Error *local_err = NULL;
 int ret;
 
@@ -2997,12 +2983,38 @@ static int qcow2_create(const char *filename, QemuOpts 
*opts, Error **errp)
 
 refcount_order = ctz32(refcount_bits);
 
-ret = qcow2_create2(filename, size, backing_file, backing_fmt, flags,
+/* Create and open the file (protocol layer) */
+if (prealloc == PREALLOC_MODE_FULL || prealloc == PREALLOC_MODE_FALLOC) {
+int64_t prealloc_size =
+qcow2_calc_prealloc_size(size, cluster_size, refcount_order);
+qemu_opt_set_number(opts, BLOCK_OPT_SIZE, prealloc_size, &error_abort);
+qemu_opt_set(opts, BLOCK_OPT_PREALLOC, PreallocMode_str(prealloc),
+ &error_abort);
+}
+
+ret = bdrv_create_file(filename, opts, errp);
+if (ret < 0) {
+goto finish;
+}
+
+bs = bdrv_open(filename, NULL, NULL,
+   BDRV_O_RDWR | BDRV_O_RESIZE | BDRV_O_PROTOCOL, errp);
+if (bs == NULL) {
+ret = -EIO;
+goto finish;
+}
+
+/* Create the qcow2 image (format layer) */
+ret = qcow2_create2(bs, size, backing_file, backing_fmt, flags,
 cluster_size, prealloc, opts, version, refcount_order,
-encryptfmt, &local_er

[Qemu-devel] [PATCH v2 05/36] qcow2: Use BlockdevRef in qcow2_create2()

Instead of passing a separate BlockDriverState* into qcow2_create2(),
make use of the BlockdevRef that is included in BlockdevCreateOptions.

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 include/block/block.h |  1 +
 block.c   | 47 +++
 block/qcow2.c | 38 --
 3 files changed, 72 insertions(+), 14 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index 947e8876cd..54fe8b7a0e 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -245,6 +245,7 @@ BdrvChild *bdrv_open_child(const char *filename,
BlockDriverState* parent,
const BdrvChildRole *child_role,
bool allow_none, Error **errp);
+BlockDriverState *bdrv_open_blockdev_ref(BlockdevRef *ref, Error **errp);
 void bdrv_set_backing_hd(BlockDriverState *bs, BlockDriverState *backing_hd,
  Error **errp);
 int bdrv_open_backing_file(BlockDriverState *bs, QDict *parent_options,
diff --git a/block.c b/block.c
index 814e5a02da..c0e343d278 100644
--- a/block.c
+++ b/block.c
@@ -35,6 +35,8 @@
 #include "qapi/qmp/qerror.h"
 #include "qapi/qmp/qjson.h"
 #include "qapi/qmp/qstring.h"
+#include "qapi/qobject-output-visitor.h"
+#include "qapi-visit.h"
 #include "sysemu/block-backend.h"
 #include "sysemu/sysemu.h"
 #include "qemu/notify.h"
@@ -2408,6 +2410,51 @@ BdrvChild *bdrv_open_child(const char *filename,
 return c;
 }
 
+/* TODO Future callers may need to specify parent/child_role in order for
+ * option inheritance to work. Existing callers use it for the root node. */
+BlockDriverState *bdrv_open_blockdev_ref(BlockdevRef *ref, Error **errp)
+{
+BlockDriverState *bs = NULL;
+Error *local_err = NULL;
+QObject *obj = NULL;
+QDict *qdict = NULL;
+const char *reference = NULL;
+Visitor *v = NULL;
+
+if (ref->type == QTYPE_QSTRING) {
+reference = ref->u.reference;
+} else {
+BlockdevOptions *options = &ref->u.definition;
+assert(ref->type == QTYPE_QDICT);
+
+v = qobject_output_visitor_new(&obj);
+visit_type_BlockdevOptions(v, NULL, &options, &local_err);
+if (local_err) {
+error_propagate(errp, local_err);
+goto fail;
+}
+visit_complete(v, &obj);
+
+qdict = qobject_to_qdict(obj);
+qdict_flatten(qdict);
+
+/* bdrv_open_inherit() defaults to the values in bdrv_flags (for
+ * compatibility with other callers) rather than what we want as the
+ * real defaults. Apply the defaults here instead. */
+qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off");
+qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off");
+qdict_set_default_str(qdict, BDRV_OPT_READ_ONLY, "off");
+}
+
+bs = bdrv_open_inherit(NULL, reference, qdict, 0, NULL, NULL, errp);
+obj = NULL;
+
+fail:
+qobject_decref(obj);
+visit_free(v);
+return bs;
+}
+
 static BlockDriverState *bdrv_append_temp_snapshot(BlockDriverState *bs,
int flags,
QDict *snapshot_options,
diff --git a/block/qcow2.c b/block/qcow2.c
index 22194180c6..b34924b0f0 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2732,8 +2732,7 @@ static uint64_t qcow2_opt_get_refcount_bits_del(QemuOpts 
*opts, int version,
 return refcount_bits;
 }
 
-static int qcow2_create2(BlockDriverState *bs,
- BlockdevCreateOptions *create_options,
+static int qcow2_create2(BlockdevCreateOptions *create_options,
  QemuOpts *opts, const char *encryptfmt, Error **errp)
 {
 BlockdevCreateOptionsQcow2 *qcow2_opts;
@@ -2751,7 +2750,8 @@ static int qcow2_create2(BlockDriverState *bs,
  * 2 GB for 64k clusters, and we don't want to have a 2 GB initial file
  * size for any qcow2 image.
  */
-BlockBackend *blk;
+BlockBackend *blk = NULL;
+BlockDriverState *bs = NULL;
 QCowHeader *header;
 size_t cluster_size;
 int version;
@@ -2760,10 +2760,15 @@ static int qcow2_create2(BlockDriverState *bs,
 Error *local_err = NULL;
 int ret;
 
-/* Validate options and set default values */
 assert(create_options->driver == BLOCKDEV_DRIVER_QCOW2);
 qcow2_opts = &create_options->u.qcow2;
 
+bs = bdrv_open_blockdev_ref(qcow2_opts->file, errp);
+if (bs == NULL) {
+return -EIO;
+}
+
+/* Validate options and set default values */
 if (!QEMU_IS_ALIGNED(qcow2_opts->size, BDRV_SECTOR_SIZE)) {
 error_setg(errp, "Image size must be a multiple of 512 bytes");
 ret = -EINVAL;
@@ -2792,7 +2797,8 @@ static int qcow2_create2(BlockDriverState *bs,
 }
 
 if (!validate_cluster_size(cluster_size, errp)) {
-return -EINVAL;
+ret = -EINVAL;
+

[Qemu-devel] [PATCH v2 13/36] block: Make bdrv_is_whitelisted() public

We'll use a separate source file for image creation, and we need to
check there whether the requested driver is whitelisted.

Signed-off-by: Kevin Wolf 
---
 include/block/block.h | 1 +
 block.c   | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/include/block/block.h b/include/block/block.h
index 54fe8b7a0e..cfce88cbda 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -225,6 +225,7 @@ char *bdrv_perm_names(uint64_t perm);
 void bdrv_init(void);
 void bdrv_init_with_whitelist(void);
 bool bdrv_uses_whitelist(void);
+int bdrv_is_whitelisted(BlockDriver *drv, bool read_only);
 BlockDriver *bdrv_find_protocol(const char *filename,
 bool allow_protocol_prefix,
 Error **errp);
diff --git a/block.c b/block.c
index c0e343d278..4a7e448226 100644
--- a/block.c
+++ b/block.c
@@ -372,7 +372,7 @@ BlockDriver *bdrv_find_format(const char *format_name)
 return bdrv_do_find_format(format_name);
 }
 
-static int bdrv_is_whitelisted(BlockDriver *drv, bool read_only)
+int bdrv_is_whitelisted(BlockDriver *drv, bool read_only)
 {
 static const char *whitelist_rw[] = {
 CONFIG_BDRV_RW_WHITELIST
-- 
2.13.6

[Qemu-devel] [PATCH v2 06/36] qcow2: Use QCryptoBlockCreateOptions in qcow2_create2()

Instead of passing the encryption format name and the QemuOpts down, use
the QCryptoBlockCreateOptions contained in BlockdevCreateOptions.

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 62 +++
 1 file changed, 45 insertions(+), 17 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index b34924b0f0..9a2028b3cf 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2414,13 +2414,10 @@ static int qcow2_crypt_method_from_format(const char 
*encryptfmt)
 }
 }
 
-static int qcow2_set_up_encryption(BlockDriverState *bs, const char 
*encryptfmt,
-   QemuOpts *opts, Error **errp)
+static QCryptoBlockCreateOptions *
+qcow2_parse_encryption(const char *encryptfmt, QemuOpts *opts, Error **errp)
 {
-BDRVQcow2State *s = bs->opaque;
 QCryptoBlockCreateOptions *cryptoopts = NULL;
-QCryptoBlock *crypto = NULL;
-int ret = -EINVAL;
 QDict *options, *encryptopts;
 int fmt;
 
@@ -2443,10 +2440,31 @@ static int qcow2_set_up_encryption(BlockDriverState 
*bs, const char *encryptfmt,
 error_setg(errp, "Unknown encryption format '%s'", encryptfmt);
 break;
 }
-if (!cryptoopts) {
-ret = -EINVAL;
-goto out;
+
+QDECREF(encryptopts);
+return cryptoopts;
+}
+
+static int qcow2_set_up_encryption(BlockDriverState *bs,
+   QCryptoBlockCreateOptions *cryptoopts,
+   Error **errp)
+{
+BDRVQcow2State *s = bs->opaque;
+QCryptoBlock *crypto = NULL;
+int fmt, ret;
+
+switch (cryptoopts->format) {
+case Q_CRYPTO_BLOCK_FORMAT_LUKS:
+fmt = QCOW_CRYPT_LUKS;
+break;
+case Q_CRYPTO_BLOCK_FORMAT_QCOW:
+fmt = QCOW_CRYPT_AES;
+break;
+default:
+error_setg(errp, "Crypto format not supported in qcow2");
+return -EINVAL;
 }
+
 s->crypt_method_header = fmt;
 
 crypto = qcrypto_block_create(cryptoopts, "encrypt.",
@@ -2454,8 +2472,7 @@ static int qcow2_set_up_encryption(BlockDriverState *bs, 
const char *encryptfmt,
   qcow2_crypto_hdr_write_func,
   bs, errp);
 if (!crypto) {
-ret = -EINVAL;
-goto out;
+return -EINVAL;
 }
 
 ret = qcow2_update_header(bs);
@@ -2464,10 +2481,9 @@ static int qcow2_set_up_encryption(BlockDriverState *bs, 
const char *encryptfmt,
 goto out;
 }
 
+ret = 0;
  out:
-QDECREF(encryptopts);
 qcrypto_block_free(crypto);
-qapi_free_QCryptoBlockCreateOptions(cryptoopts);
 return ret;
 }
 
@@ -2732,8 +2748,7 @@ static uint64_t qcow2_opt_get_refcount_bits_del(QemuOpts 
*opts, int version,
 return refcount_bits;
 }
 
-static int qcow2_create2(BlockdevCreateOptions *create_options,
- QemuOpts *opts, const char *encryptfmt, Error **errp)
+static int qcow2_create2(BlockdevCreateOptions *create_options, Error **errp)
 {
 BlockdevCreateOptionsQcow2 *qcow2_opts;
 QDict *options;
@@ -2963,8 +2978,8 @@ static int qcow2_create2(BlockdevCreateOptions 
*create_options,
 }
 
 /* Want encryption? There you go. */
-if (encryptfmt) {
-ret = qcow2_set_up_encryption(blk_bs(blk), encryptfmt, opts, errp);
+if (qcow2_opts->has_encrypt) {
+ret = qcow2_set_up_encryption(blk_bs(blk), qcow2_opts->encrypt, errp);
 if (ret < 0) {
 goto out;
 }
@@ -3021,6 +3036,7 @@ static int qcow2_create(const char *filename, QemuOpts 
*opts, Error **errp)
 int version;
 uint64_t refcount_bits;
 char *encryptfmt = NULL;
+QCryptoBlockCreateOptions *cryptoopts = NULL;
 BlockDriverState *bs = NULL;
 Error *local_err = NULL;
 int ret;
@@ -3037,6 +3053,7 @@ static int qcow2_create(const char *filename, QemuOpts 
*opts, Error **errp)
 ret = -EINVAL;
 goto finish;
 }
+
 encryptfmt = qemu_opt_get_del(opts, BLOCK_OPT_ENCRYPT_FORMAT);
 if (encryptfmt) {
 if (qemu_opt_get(opts, BLOCK_OPT_ENCRYPT)) {
@@ -3048,6 +3065,14 @@ static int qcow2_create(const char *filename, QemuOpts 
*opts, Error **errp)
 } else if (qemu_opt_get_bool_del(opts, BLOCK_OPT_ENCRYPT, false)) {
 encryptfmt = g_strdup("aes");
 }
+if (encryptfmt) {
+cryptoopts = qcow2_parse_encryption(encryptfmt, opts, errp);
+if (cryptoopts == NULL) {
+ret = -EINVAL;
+goto finish;
+}
+}
+
 cluster_size = qcow2_opt_get_cluster_size_del(opts, &local_err);
 if (local_err) {
 error_propagate(errp, local_err);
@@ -3121,6 +3146,8 @@ static int qcow2_create(const char *filename, QemuOpts 
*opts, Error **errp)
 .backing_file   = backing_file,
 .has_backing_fmt= (backing_fmt != NULL),
 .backing_fmt= backing_drv,
+.has_encrypt= (encryptfmt != NU

[Qemu-devel] [PATCH v2 07/36] qcow2: Handle full/falloc preallocation in qcow2_create2()

Once qcow2_create2() can be called directly on an already existing node,
we must provide the 'full' and 'falloc' preallocation modes outside of
creating the image on the protocol layer. Fortunately, we have
preallocated truncate now which can provide this functionality.

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 28 +++-
 1 file changed, 19 insertions(+), 9 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 9a2028b3cf..64bf2863cd 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2872,6 +2872,25 @@ static int qcow2_create2(BlockdevCreateOptions 
*create_options, Error **errp)
 }
 blk_set_allow_write_beyond_eof(blk, true);
 
+/* Clear the protocol layer and preallocate it if necessary */
+ret = blk_truncate(blk, 0, PREALLOC_MODE_OFF, errp);
+if (ret < 0) {
+goto out;
+}
+
+if (qcow2_opts->preallocation == PREALLOC_MODE_FULL ||
+qcow2_opts->preallocation == PREALLOC_MODE_FALLOC)
+{
+int64_t prealloc_size =
+qcow2_calc_prealloc_size(qcow2_opts->size, cluster_size,
+ refcount_order);
+
+ret = blk_truncate(blk, prealloc_size, qcow2_opts->preallocation, 
errp);
+if (ret < 0) {
+goto out;
+}
+}
+
 /* Write the header */
 QEMU_BUILD_BUG_ON((1 << MIN_CLUSTER_BITS) < sizeof(*header));
 header = g_malloc0(cluster_size);
@@ -3108,15 +3127,6 @@ static int qcow2_create(const char *filename, QemuOpts 
*opts, Error **errp)
 
 
 /* Create and open the file (protocol layer) */
-if (prealloc == PREALLOC_MODE_FULL || prealloc == PREALLOC_MODE_FALLOC) {
-int refcount_order = ctz32(refcount_bits);
-int64_t prealloc_size =
-qcow2_calc_prealloc_size(size, cluster_size, refcount_order);
-qemu_opt_set_number(opts, BLOCK_OPT_SIZE, prealloc_size, &error_abort);
-qemu_opt_set(opts, BLOCK_OPT_PREALLOC, PreallocMode_str(prealloc),
- &error_abort);
-}
-
 ret = bdrv_create_file(filename, opts, errp);
 if (ret < 0) {
 goto finish;
-- 
2.13.6

[Qemu-devel] [PATCH v2 15/36] file-posix: Support .bdrv_co_create

This adds the .bdrv_co_create driver callback to file, which enables
image creation over QMP.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 qapi/block-core.json | 20 +-
 block/file-posix.c   | 77 +---
 2 files changed, 74 insertions(+), 23 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 359195a1a3..0040795603 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3359,6 +3359,24 @@
 { 'command': 'blockdev-del', 'data': { 'node-name': 'str' } }
 
 ##
+# @BlockdevCreateOptionsFile:
+#
+# Driver specific image creation options for file.
+#
+# @filename Filename for the new image file
+# @size Size of the virtual disk in bytes
+# @preallocationPreallocation mode for the new image (default: off)
+# @nocowTurn off copy-on-write (valid only on btrfs; default: off)
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockdevCreateOptionsFile',
+  'data': { 'filename': 'str',
+'size': 'size',
+'*preallocation':   'PreallocMode',
+'*nocow':   'bool' } }
+
+##
 # @BlockdevQcow2Version:
 #
 # @v2:  The original QCOW2 format as introduced in qemu 0.10 (version 2)
@@ -3429,7 +3447,7 @@
   'bochs':  'BlockdevCreateNotSupported',
   'cloop':  'BlockdevCreateNotSupported',
   'dmg':'BlockdevCreateNotSupported',
-  'file':   'BlockdevCreateNotSupported',
+  'file':   'BlockdevCreateOptionsFile',
   'ftp':'BlockdevCreateNotSupported',
   'ftps':   'BlockdevCreateNotSupported',
   'gluster':'BlockdevCreateNotSupported',
diff --git a/block/file-posix.c b/block/file-posix.c
index f1591c3849..ba14ed9459 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -1982,33 +1982,25 @@ static int64_t 
raw_get_allocated_file_size(BlockDriverState *bs)
 return (int64_t)st.st_blocks * 512;
 }
 
-static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
+static int raw_co_create(BlockdevCreateOptions *options, Error **errp)
 {
+BlockdevCreateOptionsFile *file_opts;
 int fd;
 int result = 0;
-int64_t total_size = 0;
-bool nocow = false;
-PreallocMode prealloc;
-char *buf = NULL;
-Error *local_err = NULL;
 
-strstart(filename, "file:", &filename);
+/* Validate options and set default values */
+assert(options->driver == BLOCKDEV_DRIVER_FILE);
+file_opts = &options->u.file;
 
-/* Read out options */
-total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
-  BDRV_SECTOR_SIZE);
-nocow = qemu_opt_get_bool(opts, BLOCK_OPT_NOCOW, false);
-buf = qemu_opt_get_del(opts, BLOCK_OPT_PREALLOC);
-prealloc = qapi_enum_parse(&PreallocMode_lookup, buf,
-   PREALLOC_MODE_OFF, &local_err);
-g_free(buf);
-if (local_err) {
-error_propagate(errp, local_err);
-result = -EINVAL;
-goto out;
+if (!file_opts->has_nocow) {
+file_opts->nocow = false;
+}
+if (!file_opts->has_preallocation) {
+file_opts->preallocation = PREALLOC_MODE_OFF;
 }
 
-fd = qemu_open(filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY,
+/* Create file */
+fd = qemu_open(file_opts->filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY,
0644);
 if (fd < 0) {
 result = -errno;
@@ -2016,7 +2008,7 @@ static int raw_create(const char *filename, QemuOpts 
*opts, Error **errp)
 goto out;
 }
 
-if (nocow) {
+if (file_opts->nocow) {
 #ifdef __linux__
 /* Set NOCOW flag to solve performance issue on fs like btrfs.
  * This is an optimisation. The FS_IOC_SETFLAGS ioctl return value
@@ -2031,7 +2023,8 @@ static int raw_create(const char *filename, QemuOpts 
*opts, Error **errp)
 #endif
 }
 
-result = raw_regular_truncate(fd, total_size, prealloc, errp);
+result = raw_regular_truncate(fd, file_opts->size, 
file_opts->preallocation,
+  errp);
 if (result < 0) {
 goto out_close;
 }
@@ -2045,6 +2038,45 @@ out:
 return result;
 }
 
+static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
+{
+BlockdevCreateOptions options;
+int64_t total_size = 0;
+bool nocow = false;
+PreallocMode prealloc;
+char *buf = NULL;
+Error *local_err = NULL;
+
+/* Skip file: protocol prefix */
+strstart(filename, "file:", &filename);
+
+/* Read out options */
+total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
+  BDRV_SECTOR_SIZE);
+nocow = qemu_opt_get_bool(opts, BLOCK_OPT_NOCOW, false);
+buf = qemu_opt_get_del(opts, BLOCK_OPT_PREALLOC);
+prealloc = qapi_enum_parse(&PreallocMode_lookup, buf,
+   PREALLOC_MODE_OFF, &local_err);
+g_free(buf);
+if (lo

[Qemu-devel] [PATCH v2 11/36] qdict: Introduce qdict_rename_keys()

A few block drivers will need to rename .bdrv_create options for their
QAPIfication, so let's have a helper function for that.

Signed-off-by: Kevin Wolf 
---
 include/qapi/qmp/qdict.h |   6 +++
 qobject/qdict.c  |  34 ++
 tests/check-qdict.c  | 113 +++
 3 files changed, 153 insertions(+)

diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
index ff6f7842c3..7c6d844549 100644
--- a/include/qapi/qmp/qdict.h
+++ b/include/qapi/qmp/qdict.h
@@ -81,4 +81,10 @@ QObject *qdict_crumple(const QDict *src, Error **errp);
 
 void qdict_join(QDict *dest, QDict *src, bool overwrite);
 
+typedef struct QDictRenames {
+const char *from;
+const char *to;
+} QDictRenames;
+bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error 
**errp);
+
 #endif /* QDICT_H */
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 23df84f9cd..229b8c840b 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -1072,3 +1072,37 @@ void qdict_join(QDict *dest, QDict *src, bool overwrite)
 entry = next;
 }
 }
+
+/**
+ * qdict_rename_keys(): Rename keys in qdict according to the replacements
+ * specified in the array renames. The array must be terminated by an entry
+ * with from = NULL.
+ *
+ * The renames are performed individually in the order of the array, so entries
+ * may be renamed multiple times and may or may not conflict depending on the
+ * order of the renames array.
+ *
+ * Returns true for success, false in error cases.
+ */
+bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp)
+{
+QObject *qobj;
+
+while (renames->from) {
+if (qdict_haskey(qdict, renames->from)) {
+if (qdict_haskey(qdict, renames->to)) {
+error_setg(errp, "'%s' and its alias '%s' can't be used at the 
"
+   "same time", renames->to, renames->from);
+return false;
+}
+
+qobj = qdict_get(qdict, renames->from);
+qobject_incref(qobj);
+qdict_put_obj(qdict, renames->to, qobj);
+qdict_del(qdict, renames->from);
+}
+
+renames++;
+}
+return true;
+}
diff --git a/tests/check-qdict.c b/tests/check-qdict.c
index ec628f3453..5f8f3be9ff 100644
--- a/tests/check-qdict.c
+++ b/tests/check-qdict.c
@@ -665,6 +665,117 @@ static void qdict_crumple_test_empty(void)
 QDECREF(dst);
 }
 
+static void qdict_rename_keys_test(void)
+{
+QDict *dict = qdict_new();
+QDict *copy;
+QDictRenames *renames;
+Error *local_err = NULL;
+
+qdict_put_str(dict, "abc", "foo");
+qdict_put_str(dict, "abcdef", "bar");
+qdict_put_int(dict, "number", 42);
+qdict_put_bool(dict, "flag", true);
+qdict_put_null(dict, "nothing");
+
+/* Empty rename list */
+renames = (QDictRenames[]) {
+{ NULL, "this can be anything" }
+};
+copy = qdict_clone_shallow(dict);
+qdict_rename_keys(copy, renames, &error_abort);
+
+g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo");
+g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar");
+g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42);
+g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true);
+g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL);
+
+QDECREF(copy);
+
+/* Simple rename of all entries */
+renames = (QDictRenames[]) {
+{ "abc","str1" },
+{ "abcdef", "str2" },
+{ "number", "int" },
+{ "flag",   "bool" },
+{ "nothing","null" },
+{ NULL , NULL }
+};
+copy = qdict_clone_shallow(dict);
+qdict_rename_keys(copy, renames, &error_abort);
+
+g_assert(!qdict_haskey(copy, "abc"));
+g_assert(!qdict_haskey(copy, "abcdef"));
+g_assert(!qdict_haskey(copy, "number"));
+g_assert(!qdict_haskey(copy, "flag"));
+g_assert(!qdict_haskey(copy, "nothing"));
+
+g_assert_cmpstr(qdict_get_str(copy, "str1"), ==, "foo");
+g_assert_cmpstr(qdict_get_str(copy, "str2"), ==, "bar");
+g_assert_cmpint(qdict_get_int(copy, "int"), ==, 42);
+g_assert_cmpint(qdict_get_bool(copy, "bool"), ==, true);
+g_assert(qobject_type(qdict_get(copy, "null")) == QTYPE_QNULL);
+
+QDECREF(copy);
+
+/* Renames are processed top to bottom */
+renames = (QDictRenames[]) {
+{ "abc","tmp" },
+{ "abcdef", "abc" },
+{ "number", "abcdef" },
+{ "flag",   "number" },
+{ "nothing","flag" },
+{ "tmp","nothing" },
+{ NULL , NULL }
+};
+copy = qdict_clone_shallow(dict);
+qdict_rename_keys(copy, renames, &error_abort);
+
+g_assert_cmpstr(qdict_get_str(copy, "nothing"), ==, "foo");
+g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "bar");
+g_assert_cmpint(qdict_get_int(copy, "abcdef"), ==, 42);
+g_assert_cmpint(qdict_get_bool(copy, "number"), ==, true);
+g_assert(qo

[Qemu-devel] [PATCH v2 08/36] util: Add qemu_opts_to_qdict_filtered()

This allows, given a QemuOpts for a QemuOptsList that was merged from
multiple QemuOptsList, to only consider those options that exist in one
specific list. Block drivers need this to separate format-layer create
options from protocol-level options.

Signed-off-by: Kevin Wolf 
---
 include/qemu/option.h |  2 ++
 util/qemu-option.c| 42 +-
 2 files changed, 39 insertions(+), 5 deletions(-)

diff --git a/include/qemu/option.h b/include/qemu/option.h
index b127fb6db6..306fdb5f7a 100644
--- a/include/qemu/option.h
+++ b/include/qemu/option.h
@@ -124,6 +124,8 @@ void qemu_opts_set_defaults(QemuOptsList *list, const char 
*params,
 int permit_abbrev);
 QemuOpts *qemu_opts_from_qdict(QemuOptsList *list, const QDict *qdict,
Error **errp);
+QDict *qemu_opts_to_qdict_filtered(QemuOpts *opts, QDict *qdict,
+   QemuOptsList *list, bool del);
 QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict);
 void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp);
 
diff --git a/util/qemu-option.c b/util/qemu-option.c
index a401e936da..2b412eff5e 100644
--- a/util/qemu-option.c
+++ b/util/qemu-option.c
@@ -1007,14 +1007,23 @@ void qemu_opts_absorb_qdict(QemuOpts *opts, QDict 
*qdict, Error **errp)
 }
 
 /*
- * Convert from QemuOpts to QDict.
- * The QDict values are of type QString.
+ * Convert from QemuOpts to QDict. The QDict values are of type QString.
+ *
+ * If @list is given, only add those options to the QDict that are contained in
+ * the list. If @del is true, any options added to the QDict are removed from
+ * the QemuOpts, otherwise they remain there.
+ *
+ * If two options in @opts have the same name, they are processed in order
+ * so that the last one wins (consistent with the reverse iteration in
+ * qemu_opt_find()), but all of them are deleted if @del is true.
+ *
  * TODO We'll want to use types appropriate for opt->desc->type, but
  * this is enough for now.
  */
-QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict)
+QDict *qemu_opts_to_qdict_filtered(QemuOpts *opts, QDict *qdict,
+   QemuOptsList *list, bool del)
 {
-QemuOpt *opt;
+QemuOpt *opt, *next;
 
 if (!qdict) {
 qdict = qdict_new();
@@ -1022,12 +1031,35 @@ QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict)
 if (opts->id) {
 qdict_put_str(qdict, "id", opts->id);
 }
-QTAILQ_FOREACH(opt, &opts->head, next) {
+QTAILQ_FOREACH_SAFE(opt, &opts->head, next, next) {
+if (list) {
+QemuOptDesc *desc;
+bool found = false;
+for (desc = list->desc; desc->name; desc++) {
+if (!strcmp(desc->name, opt->name)) {
+found = true;
+break;
+}
+}
+if (!found) {
+continue;
+}
+}
 qdict_put_str(qdict, opt->name, opt->str);
+if (del) {
+qemu_opt_del(opt);
+}
 }
 return qdict;
 }
 
+/* Copy all options in a QemuOpts to the given QDict. See
+ * qemu_opts_to_qdict_filtered() for details. */
+QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict)
+{
+return qemu_opts_to_qdict_filtered(opts, qdict, NULL, false);
+}
+
 /* Validate parsed opts against descriptions where no
  * descriptions were provided in the QemuOptsList.
  */
-- 
2.13.6

[Qemu-devel] [PATCH v2 10/36] test-qemu-opts: Test qemu_opts_to_qdict_filtered()

Signed-off-by: Kevin Wolf 
---
 tests/test-qemu-opts.c | 125 +
 1 file changed, 125 insertions(+)

diff --git a/tests/test-qemu-opts.c b/tests/test-qemu-opts.c
index 6c3183390b..2c422abcd4 100644
--- a/tests/test-qemu-opts.c
+++ b/tests/test-qemu-opts.c
@@ -10,6 +10,7 @@
 #include "qemu/osdep.h"
 #include "qemu/cutils.h"
 #include "qemu/option.h"
+#include "qemu/option_int.h"
 #include "qapi/error.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qstring.h"
@@ -868,6 +869,127 @@ static void test_opts_append(void)
 qemu_opts_free(merged);
 }
 
+static void test_opts_to_qdict_basic(void)
+{
+QemuOpts *opts;
+QDict *dict;
+
+opts = qemu_opts_parse(&opts_list_01, "str1=foo,str2=,str3=bar,number1=42",
+   false, &error_abort);
+g_assert(opts != NULL);
+
+dict = qemu_opts_to_qdict(opts, NULL);
+g_assert(dict != NULL);
+
+g_assert_cmpstr(qdict_get_str(dict, "str1"), ==, "foo");
+g_assert_cmpstr(qdict_get_str(dict, "str2"), ==, "");
+g_assert_cmpstr(qdict_get_str(dict, "str3"), ==, "bar");
+g_assert_cmpstr(qdict_get_str(dict, "number1"), ==, "42");
+g_assert_false(qdict_haskey(dict, "number2"));
+
+QDECREF(dict);
+qemu_opts_del(opts);
+}
+
+static void test_opts_to_qdict_filtered(void)
+{
+QemuOptsList *first, *merged;
+QemuOpts *opts;
+QDict *dict;
+
+first = qemu_opts_append(NULL, &opts_list_02);
+merged = qemu_opts_append(first, &opts_list_01);
+
+opts = qemu_opts_parse(merged,
+   "str1=foo,str2=,str3=bar,bool1=off,number1=42",
+   false, &error_abort);
+g_assert(opts != NULL);
+
+/* Convert to QDict without deleting from opts */
+dict = qemu_opts_to_qdict_filtered(opts, NULL, &opts_list_01, false);
+g_assert(dict != NULL);
+g_assert_cmpstr(qdict_get_str(dict, "str1"), ==, "foo");
+g_assert_cmpstr(qdict_get_str(dict, "str2"), ==, "");
+g_assert_cmpstr(qdict_get_str(dict, "str3"), ==, "bar");
+g_assert_cmpstr(qdict_get_str(dict, "number1"), ==, "42");
+g_assert_false(qdict_haskey(dict, "number2"));
+g_assert_false(qdict_haskey(dict, "bool1"));
+QDECREF(dict);
+
+dict = qemu_opts_to_qdict_filtered(opts, NULL, &opts_list_02, false);
+g_assert(dict != NULL);
+g_assert_cmpstr(qdict_get_str(dict, "str1"), ==, "foo");
+g_assert_cmpstr(qdict_get_str(dict, "str2"), ==, "");
+g_assert_cmpstr(qdict_get_str(dict, "bool1"), ==, "off");
+g_assert_false(qdict_haskey(dict, "str3"));
+g_assert_false(qdict_haskey(dict, "number1"));
+g_assert_false(qdict_haskey(dict, "number2"));
+QDECREF(dict);
+
+/* Now delete converted options from opts */
+dict = qemu_opts_to_qdict_filtered(opts, NULL, &opts_list_01, true);
+g_assert(dict != NULL);
+g_assert_cmpstr(qdict_get_str(dict, "str1"), ==, "foo");
+g_assert_cmpstr(qdict_get_str(dict, "str2"), ==, "");
+g_assert_cmpstr(qdict_get_str(dict, "str3"), ==, "bar");
+g_assert_cmpstr(qdict_get_str(dict, "number1"), ==, "42");
+g_assert_false(qdict_haskey(dict, "number2"));
+g_assert_false(qdict_haskey(dict, "bool1"));
+QDECREF(dict);
+
+dict = qemu_opts_to_qdict_filtered(opts, NULL, &opts_list_02, true);
+g_assert(dict != NULL);
+g_assert_cmpstr(qdict_get_str(dict, "bool1"), ==, "off");
+g_assert_false(qdict_haskey(dict, "str1"));
+g_assert_false(qdict_haskey(dict, "str2"));
+g_assert_false(qdict_haskey(dict, "str3"));
+g_assert_false(qdict_haskey(dict, "number1"));
+g_assert_false(qdict_haskey(dict, "number2"));
+QDECREF(dict);
+
+g_assert_true(QTAILQ_EMPTY(&opts->head));
+
+qemu_opts_del(opts);
+qemu_opts_free(merged);
+}
+
+static void test_opts_to_qdict_duplicates(void)
+{
+QemuOpts *opts;
+QemuOpt *opt;
+QDict *dict;
+
+opts = qemu_opts_parse(&opts_list_03, "foo=a,foo=b", false, &error_abort);
+g_assert(opts != NULL);
+
+/* Verify that opts has two options with the same name */
+opt = QTAILQ_FIRST(&opts->head);
+g_assert_cmpstr(opt->name, ==, "foo");
+g_assert_cmpstr(opt->str , ==, "a");
+
+opt = QTAILQ_NEXT(opt, next);
+g_assert_cmpstr(opt->name, ==, "foo");
+g_assert_cmpstr(opt->str , ==, "b");
+
+opt = QTAILQ_NEXT(opt, next);
+g_assert(opt == NULL);
+
+/* In the conversion to QDict, the last one wins */
+dict = qemu_opts_to_qdict(opts, NULL);
+g_assert(dict != NULL);
+g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "b");
+QDECREF(dict);
+
+/* The last one still wins if entries are deleted, and both are deleted */
+dict = qemu_opts_to_qdict_filtered(opts, NULL, NULL, true);
+g_assert(dict != NULL);
+g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "b");
+QDECREF(dict);
+
+g_assert_true(QTAILQ_EMPTY(&opts->head));
+
+qemu_opts_del(opts);
+}
 
 int main(int argc, char *argv[])
 {
@@ -889,6 +1011,9 @@ int main(int argc, char

[Qemu-devel] [PATCH v2 24/36] rbd: Use qemu_rbd_connect() in qemu_rbd_do_create()

This is almost exactly the same code. The differences are that
qemu_rbd_connect() supports BlockdevOptionsRbd.server and that the cache
mode is set explicitly.

Supporting 'server' is a welcome new feature for image creation.
Caching is disabled by default, so leave it that way.

Signed-off-by: Kevin Wolf 
---
 block/rbd.c | 54 ++
 1 file changed, 10 insertions(+), 44 deletions(-)

diff --git a/block/rbd.c b/block/rbd.c
index a34bf0be46..af8e186106 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -103,6 +103,11 @@ typedef struct BDRVRBDState {
 char *snap;
 } BDRVRBDState;
 
+static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
+BlockdevOptionsRbd *opts, bool cache,
+const char *keypairs, const char *secretid,
+Error **errp);
+
 static char *qemu_rbd_next_tok(char *src, char delim, char **p)
 {
 char *end;
@@ -351,12 +356,6 @@ static int qemu_rbd_do_create(BlockdevCreateOptions 
*options,
 return -EINVAL;
 }
 
-/* TODO Remove the limitation */
-if (opts->location->has_server) {
-error_setg(errp, "Can't specify server for image creation");
-return -EINVAL;
-}
-
 if (opts->has_cluster_size) {
 int64_t objsize = opts->cluster_size;
 if ((objsize - 1) & objsize) {/* not a power of 2? */
@@ -370,54 +369,21 @@ static int qemu_rbd_do_create(BlockdevCreateOptions 
*options,
 obj_order = ctz32(objsize);
 }
 
-ret = rados_create(&cluster, opts->location->user);
+ret = qemu_rbd_connect(&cluster, &io_ctx, opts->location, false, keypairs,
+   NULL, errp);
 if (ret < 0) {
-error_setg_errno(errp, -ret, "error initializing");
 return ret;
 }
 
-/* try default location when conf=NULL, but ignore failure */
-ret = rados_conf_read_file(cluster, opts->location->conf);
-if (opts->location->conf && ret < 0) {
-error_setg_errno(errp, -ret, "error reading conf file %s",
- opts->location->conf);
-ret = -EIO;
-goto shutdown;
-}
-
-ret = qemu_rbd_set_keypairs(cluster, keypairs, errp);
-if (ret < 0) {
-ret = -EIO;
-goto shutdown;
-}
-
-if (qemu_rbd_set_auth(cluster, password_secret, errp) < 0) {
-ret = -EIO;
-goto shutdown;
-}
-
-ret = rados_connect(cluster);
-if (ret < 0) {
-error_setg_errno(errp, -ret, "error connecting");
-goto shutdown;
-}
-
-ret = rados_ioctx_create(cluster, opts->location->pool, &io_ctx);
-if (ret < 0) {
-error_setg_errno(errp, -ret, "error opening pool %s",
- opts->location->pool);
-goto shutdown;
-}
-
 ret = rbd_create(io_ctx, opts->location->image, opts->size, &obj_order);
 if (ret < 0) {
 error_setg_errno(errp, -ret, "error rbd create");
+goto out;
 }
 
-rados_ioctx_destroy(io_ctx);
-
 ret = 0;
-shutdown:
+out:
+rados_ioctx_destroy(io_ctx);
 rados_shutdown(cluster);
 return ret;
 }
-- 
2.13.6

[Qemu-devel] [PATCH v2 17/36] gluster: Support .bdrv_co_create

This adds the .bdrv_co_create driver callback to gluster, which enables
image creation over QMP.

Signed-off-by: Kevin Wolf 
---
 qapi/block-core.json |  18 ++-
 block/gluster.c  | 135 ++-
 2 files changed, 108 insertions(+), 45 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0040795603..74021c51d7 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3377,6 +3377,22 @@
 '*nocow':   'bool' } }
 
 ##
+# @BlockdevCreateOptionsGluster:
+#
+# Driver specific image creation options for gluster.
+#
+# @location Where to store the new image file
+# @size Size of the virtual disk in bytes
+# @preallocationPreallocation mode for the new image (default: off)
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockdevCreateOptionsGluster',
+  'data': { 'location': 'BlockdevOptionsGluster',
+'size': 'size',
+'*preallocation':   'PreallocMode' } }
+
+##
 # @BlockdevQcow2Version:
 #
 # @v2:  The original QCOW2 format as introduced in qemu 0.10 (version 2)
@@ -3450,7 +3466,7 @@
   'file':   'BlockdevCreateOptionsFile',
   'ftp':'BlockdevCreateNotSupported',
   'ftps':   'BlockdevCreateNotSupported',
-  'gluster':'BlockdevCreateNotSupported',
+  'gluster':'BlockdevCreateOptionsGluster',
   'host_cdrom': 'BlockdevCreateNotSupported',
   'host_device':'BlockdevCreateNotSupported',
   'http':   'BlockdevCreateNotSupported',
diff --git a/block/gluster.c b/block/gluster.c
index 1a07d221d1..6e2f0e3185 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -655,9 +655,11 @@ out:
 return -errno;
 }
 
-static struct glfs *qemu_gluster_init(BlockdevOptionsGluster *gconf,
-  const char *filename,
-  QDict *options, Error **errp)
+/* Converts options given in @filename and the @options QDict into the QAPI
+ * object @gconf. */
+static int qemu_gluster_parse(BlockdevOptionsGluster *gconf,
+  const char *filename,
+  QDict *options, Error **errp)
 {
 int ret;
 if (filename) {
@@ -668,8 +670,7 @@ static struct glfs 
*qemu_gluster_init(BlockdevOptionsGluster *gconf,
 "[host[:port]]volume/path[?socket=...]"
 "[,file.debug=N]"
 "[,file.logfile=/path/filename.log]\n");
-errno = -ret;
-return NULL;
+return ret;
 }
 } else {
 ret = qemu_gluster_parse_json(gconf, options, errp);
@@ -685,10 +686,23 @@ static struct glfs 
*qemu_gluster_init(BlockdevOptionsGluster *gconf,
  "file.server.1.transport=unix,"
  "file.server.1.socket=/var/run/glusterd.socket 
..."
  "\n");
-errno = -ret;
-return NULL;
+return ret;
 }
+}
 
+return 0;
+}
+
+static struct glfs *qemu_gluster_init(BlockdevOptionsGluster *gconf,
+  const char *filename,
+  QDict *options, Error **errp)
+{
+int ret;
+
+ret = qemu_gluster_parse(gconf, filename, options, errp);
+if (ret < 0) {
+errno = -ret;
+return NULL;
 }
 
 return qemu_gluster_glfs_init(gconf, errp);
@@ -1021,19 +1035,71 @@ static int qemu_gluster_do_truncate(struct glfs_fd *fd, 
int64_t offset,
 return 0;
 }
 
-static int qemu_gluster_create(const char *filename,
-   QemuOpts *opts, Error **errp)
+static int qemu_gluster_co_create(BlockdevCreateOptions *options,
+  Error **errp)
 {
-BlockdevOptionsGluster *gconf;
+BlockdevCreateOptionsGluster *opts = &options->u.gluster;
 struct glfs *glfs;
 struct glfs_fd *fd = NULL;
 int ret = 0;
-PreallocMode prealloc;
-int64_t total_size = 0;
+
+assert(options->driver == BLOCKDEV_DRIVER_GLUSTER);
+
+glfs = qemu_gluster_glfs_init(opts->location, errp);
+if (!glfs) {
+ret = -errno;
+goto out;
+}
+
+fd = glfs_creat(glfs, opts->location->path,
+O_WRONLY | O_CREAT | O_TRUNC | O_BINARY, S_IRUSR | 
S_IWUSR);
+if (!fd) {
+ret = -errno;
+goto out;
+}
+
+ret = qemu_gluster_do_truncate(fd, opts->size, opts->preallocation, errp);
+
+out:
+if (fd) {
+if (glfs_close(fd) != 0 && ret == 0) {
+ret = -errno;
+}
+}
+glfs_clear_preopened(glfs);
+return ret;
+}
+
+static int qemu_gluster_create(const char *filename,
+   QemuOpts *opts, Error **errp)
+{
+BlockdevCreateOptions *options;
+BlockdevCreateOptionsGluster *gopts;
+BlockdevOptionsGluster *gconf;

[Qemu-devel] [PATCH v2 14/36] block: x-blockdev-create QMP command

This adds a synchronous x-blockdev-create QMP command that can create
qcow2 images on a given node name.

We don't want to block while creating an image, so this is not the final
interface in all aspects, but BlockdevCreateOptionsQcow2 and
.bdrv_co_create() are what they actually might look like in the end. In
any case, this should be good enough to test whether we interpret
BlockdevCreateOptions as we should.

Signed-off-by: Kevin Wolf 
---
 qapi/block-core.json  | 12 
 include/block/block_int.h |  2 ++
 block/create.c| 76 +++
 block/qcow2.c |  3 +-
 block/Makefile.objs   |  2 +-
 5 files changed, 93 insertions(+), 2 deletions(-)
 create mode 100644 block/create.c

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 74b864d64e..359195a1a3 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3464,6 +3464,18 @@
   } }
 
 ##
+# @x-blockdev-create:
+#
+# Create an image format on a given node.
+# TODO Replace with something asynchronous (block job?)
+#
+# Since: 2.12
+##
+{ 'command': 'x-blockdev-create',
+  'data': 'BlockdevCreateOptions',
+  'boxed': true }
+
+##
 # @blockdev-open-tray:
 #
 # Opens a block device's tray. If there is a block driver state tree inserted 
as
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 5ae7738cf8..0b43fae782 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -128,6 +128,8 @@ struct BlockDriver {
 int (*bdrv_file_open)(BlockDriverState *bs, QDict *options, int flags,
   Error **errp);
 void (*bdrv_close)(BlockDriverState *bs);
+int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,
+   Error **errp);
 int (*bdrv_create)(const char *filename, QemuOpts *opts, Error **errp);
 int (*bdrv_make_empty)(BlockDriverState *bs);
 
diff --git a/block/create.c b/block/create.c
new file mode 100644
index 00..dfd31eca37
--- /dev/null
+++ b/block/create.c
@@ -0,0 +1,76 @@
+/*
+ * Block layer code related to image creation
+ *
+ * Copyright (c) 2018 Kevin Wolf 
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to 
deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "block/block_int.h"
+#include "qmp-commands.h"
+#include "qapi/error.h"
+
+typedef struct BlockdevCreateCo {
+BlockDriver *drv;
+BlockdevCreateOptions *opts;
+int ret;
+Error **errp;
+} BlockdevCreateCo;
+
+static void coroutine_fn bdrv_co_create_co_entry(void *opaque)
+{
+BlockdevCreateCo *cco = opaque;
+cco->ret = cco->drv->bdrv_co_create(cco->opts, cco->errp);
+}
+
+void qmp_x_blockdev_create(BlockdevCreateOptions *options, Error **errp)
+{
+const char *fmt = BlockdevDriver_str(options->driver);
+BlockDriver *drv = bdrv_find_format(fmt);
+Coroutine *co;
+BlockdevCreateCo cco;
+
+/* If the driver is in the schema, we know that it exists. But it may not
+ * be whitelisted. */
+assert(drv);
+if (bdrv_uses_whitelist() && !bdrv_is_whitelisted(drv, false)) {
+error_setg(errp, "Driver is not whitelisted");
+return;
+}
+
+/* Call callback if it exists */
+if (!drv->bdrv_co_create) {
+error_setg(errp, "Driver does not support blockdev-create");
+return;
+}
+
+cco = (BlockdevCreateCo) {
+.drv = drv,
+.opts = options,
+.ret = -EINPROGRESS,
+.errp = errp,
+};
+
+co = qemu_coroutine_create(bdrv_co_create_co_entry, &cco);
+qemu_coroutine_enter(co);
+while (cco.ret == -EINPROGRESS) {
+aio_poll(qemu_get_aio_context(), true);
+}
+}
diff --git a/block/qcow2.c b/block/qcow2.c
index 58737d0833..8acb36b0af 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -4463,7 +4463,8 @@ BlockDriver bdrv_qcow2 = {
 .bdrv_reopen_abort= qcow2_reopen_abort,
 .bdrv_join_options= qcow2_join_options,
 .bdrv_child_perm  = b

[Qemu-devel] [PATCH v2 12/36] qcow2: Use visitor for options in qcow2_create()

Instead of manually creating the BlockdevCreateOptions object, use a
visitor to parse the given options into the QAPI object.

This involves translation from the old command line syntax to the syntax
mandated by the QAPI schema. Option names are still checked against
qcow2_create_opts, so only the old option names are allowed on the
command line, even if they are translated in qcow2_create().

In contrast, new option values are optionally recognised besides the old
values: 'compat' accepts 'v2'/'v3' as an alias for '0.10'/'1.1', and
'encrypt.format' accepts 'qcow' as an alias for 'aes' now.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
Reviewed-by: Eric Blake 
---
 block/qcow2.c  | 217 -
 tests/qemu-iotests/049.out |   8 +-
 tests/qemu-iotests/112.out |   4 +-
 3 files changed, 83 insertions(+), 146 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 64bf2863cd..58737d0833 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -38,7 +38,7 @@
 #include "qemu/option_int.h"
 #include "qemu/cutils.h"
 #include "qemu/bswap.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "qapi-visit.h"
 #include "block/crypto.h"
 
@@ -2414,37 +2414,6 @@ static int qcow2_crypt_method_from_format(const char 
*encryptfmt)
 }
 }
 
-static QCryptoBlockCreateOptions *
-qcow2_parse_encryption(const char *encryptfmt, QemuOpts *opts, Error **errp)
-{
-QCryptoBlockCreateOptions *cryptoopts = NULL;
-QDict *options, *encryptopts;
-int fmt;
-
-options = qemu_opts_to_qdict(opts, NULL);
-qdict_extract_subqdict(options, &encryptopts, "encrypt.");
-QDECREF(options);
-
-fmt = qcow2_crypt_method_from_format(encryptfmt);
-
-switch (fmt) {
-case QCOW_CRYPT_LUKS:
-cryptoopts = block_crypto_create_opts_init(
-Q_CRYPTO_BLOCK_FORMAT_LUKS, encryptopts, errp);
-break;
-case QCOW_CRYPT_AES:
-cryptoopts = block_crypto_create_opts_init(
-Q_CRYPTO_BLOCK_FORMAT_QCOW, encryptopts, errp);
-break;
-default:
-error_setg(errp, "Unknown encryption format '%s'", encryptfmt);
-break;
-}
-
-QDECREF(encryptopts);
-return cryptoopts;
-}
-
 static int qcow2_set_up_encryption(BlockDriverState *bs,
QCryptoBlockCreateOptions *cryptoopts,
Error **errp)
@@ -2838,7 +2807,7 @@ static int qcow2_create2(BlockdevCreateOptions 
*create_options, Error **errp)
 }
 if (version < 3 && qcow2_opts->lazy_refcounts) {
 error_setg(errp, "Lazy refcounts only supported with compatibility "
-   "level 1.1 and above (use compat=1.1 or greater)");
+   "level 1.1 and above (use version=v3 or greater)");
 ret = -EINVAL;
 goto out;
 }
@@ -2856,7 +2825,7 @@ static int qcow2_create2(BlockdevCreateOptions 
*create_options, Error **errp)
 }
 if (version < 3 && qcow2_opts->refcount_bits != 16) {
 error_setg(errp, "Different refcount widths than 16 bits require "
-   "compatibility level 1.1 or above (use compat=1.1 or "
+   "compatibility level 1.1 or above (use version=v3 or "
"greater)");
 ret = -EINVAL;
 goto out;
@@ -3043,144 +3012,112 @@ out:
 
 static int qcow2_create(const char *filename, QemuOpts *opts, Error **errp)
 {
-BlockdevCreateOptions create_options;
-char *backing_file = NULL;
-char *backing_fmt = NULL;
-BlockdevDriver backing_drv;
-char *buf = NULL;
-uint64_t size = 0;
-int flags = 0;
-size_t cluster_size = DEFAULT_CLUSTER_SIZE;
-PreallocMode prealloc;
-int version;
-uint64_t refcount_bits;
-char *encryptfmt = NULL;
-QCryptoBlockCreateOptions *cryptoopts = NULL;
+BlockdevCreateOptions *create_options = NULL;
+QDict *qdict = NULL;
+QObject *qobj;
+Visitor *v;
 BlockDriverState *bs = NULL;
 Error *local_err = NULL;
+const char *val;
 int ret;
 
-/* Read out options */
-size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
-BDRV_SECTOR_SIZE);
-backing_file = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FILE);
-backing_fmt = qemu_opt_get_del(opts, BLOCK_OPT_BACKING_FMT);
-backing_drv = qapi_enum_parse(&BlockdevDriver_lookup, backing_fmt,
-  0, &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
+/* Only the keyval visitor supports the dotted syntax needed for
+ * encryption, so go through a QDict before getting a QAPI type. Ignore
+ * options meant for the protocol layer so that the visitor doesn't
+ * complain. */
+qdict = qemu_opts_to_qdict_filtered(opts, NULL, bdrv_qcow2.create_opts,
+true);
+
+/* Handle encryption options */
+val = qdict_get_try_str(qdict, BLOCK_OPT_ENCRYPT);
+i

[Qemu-devel] [PATCH v2 20/36] rbd: Remove non-schema options from runtime_opts

Instead of the QemuOpts in qemu_rbd_connect(), we want to use QAPI
objects. As a preparation, fetch those options directly from the QDict
that .bdrv_open() supports in the rbd driver and that are not in the
schema.

Signed-off-by: Kevin Wolf 
---
 block/rbd.c | 55 ---
 1 file changed, 24 insertions(+), 31 deletions(-)

diff --git a/block/rbd.c b/block/rbd.c
index 4bbcce4eca..2e79c2d1fd 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -326,28 +326,6 @@ static QemuOptsList runtime_opts = {
 /*
  * server.* extracted manually, see qemu_rbd_mon_host()
  */
-{
-.name = "password-secret",
-.type = QEMU_OPT_STRING,
-.help = "ID of secret providing the password",
-},
-
-/*
- * Keys for qemu_rbd_parse_filename(), not in the QAPI schema
- */
-{
-/*
- * HACK: name starts with '=' so that qemu_opts_parse()
- * can't set it
- */
-.name = "=keyvalue-pairs",
-.type = QEMU_OPT_STRING,
-.help = "Legacy rados key/value option parameters",
-},
-{
-.name = "filename",
-.type = QEMU_OPT_STRING,
-},
 { /* end of list */ }
 },
 };
@@ -546,12 +524,13 @@ out:
 
 static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
 char **s_snap, char **s_image_name,
-QDict *options, bool cache, Error **errp)
+QDict *options, bool cache,
+const char *keypairs, const char *secretid,
+Error **errp)
 {
 QemuOpts *opts;
 char *mon_host = NULL;
-const char *pool, *snap, *conf, *user, *image_name, *keypairs;
-const char *secretid;
+const char *pool, *snap, *conf, *user, *image_name;
 Error *local_err = NULL;
 int r;
 
@@ -570,14 +549,11 @@ static int qemu_rbd_connect(rados_t *cluster, 
rados_ioctx_t *io_ctx,
 goto failed_opts;
 }
 
-secretid = qemu_opt_get(opts, "password-secret");
-
 pool   = qemu_opt_get(opts, "pool");
 conf   = qemu_opt_get(opts, "conf");
 snap   = qemu_opt_get(opts, "snapshot");
 user   = qemu_opt_get(opts, "user");
 image_name = qemu_opt_get(opts, "image");
-keypairs   = qemu_opt_get(opts, "=keyvalue-pairs");
 
 if (!pool || !image_name) {
 error_setg(errp, "Parameters 'pool' and 'image' are required");
@@ -662,6 +638,7 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
 BDRVRBDState *s = bs->opaque;
 Error *local_err = NULL;
 const char *filename;
+char *keypairs, *secretid;
 int r;
 
 /* If we are given a filename, parse the filename, with precedence given to
@@ -672,16 +649,28 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
 "This is an unsupported option, and may be deprecated "
 "in the future");
 qemu_rbd_parse_filename(filename, options, &local_err);
+qdict_del(options, "filename");
 if (local_err) {
 error_propagate(errp, local_err);
 return -EINVAL;
 }
 }
 
+keypairs = g_strdup(qdict_get_try_str(options, "=keyvalue-pairs"));
+if (keypairs) {
+qdict_del(options, "=keyvalue-pairs");
+}
+
+secretid = g_strdup(qdict_get_try_str(options, "password-secret"));
+if (secretid) {
+qdict_del(options, "password-secret");
+}
+
 r = qemu_rbd_connect(&s->cluster, &s->io_ctx, &s->snap, &s->image_name,
- options, !(flags & BDRV_O_NOCACHE), errp);
+ options, !(flags & BDRV_O_NOCACHE), keypairs, 
secretid,
+ errp);
 if (r < 0) {
-return r;
+goto out;
 }
 
 /* rbd_open is always r/w */
@@ -708,13 +697,17 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
 }
 }
 
-return 0;
+r = 0;
+goto out;
 
 failed_open:
 rados_ioctx_destroy(s->io_ctx);
 g_free(s->snap);
 g_free(s->image_name);
 rados_shutdown(s->cluster);
+out:
+g_free(keypairs);
+g_free(secretid);
 return r;
 }
 
-- 
2.13.6

[Qemu-devel] [PATCH v2 27/36] sheepdog: QAPIfy "redundacy" create option

The "redundacy" option for Sheepdog image creation is currently a string
that can encode one or two integers depending on its format, which at
the same time implicitly selects a mode.

This patch turns it into a QAPI union and converts the string into such
a QAPI object before interpreting the values.

Signed-off-by: Kevin Wolf 
---
 qapi/block-core.json | 45 +
 block/sheepdog.c | 94 +---
 2 files changed, 112 insertions(+), 27 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 085b791303..2b249c9e3d 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3467,6 +3467,51 @@
 '*cluster-size' :   'size' } }
 
 ##
+# @SheepdogRedundancyType:
+#
+# @full Create a fully replicated vdi with x copies
+# @erasure-codedCreate an erasure coded vdi with x data strips and
+#   y parity strips
+#
+# Since: 2.12
+##
+{ 'enum': 'SheepdogRedundancyType',
+  'data': [ 'full', 'erasure-coded' ] }
+
+##
+# @SheepdogRedundancyFull:
+#
+# @copies   Number of copies to use (between 1 and 31)
+#
+# Since: 2.12
+##
+{ 'struct': 'SheepdogRedundancyFull',
+  'data': { 'copies': 'int' }}
+
+##
+# @SheepdogRedundancyErasureCoded:
+#
+# @data-strips  Number of data strips to use (one of {2,4,8,16})
+# @parity-stripsNumber of parity strips to use (between 1 and 15)
+#
+# Since: 2.12
+##
+{ 'struct': 'SheepdogRedundancyErasureCoded',
+  'data': { 'data-strips': 'int',
+'parity-strips': 'int' }}
+
+##
+# @SheepdogRedundancy:
+#
+# Since: 2.12
+##
+{ 'union': 'SheepdogRedundancy',
+  'base': { 'type': 'SheepdogRedundancyType' },
+  'discriminator': 'type',
+  'data': { 'full': 'SheepdogRedundancyFull',
+'erasure-coded': 'SheepdogRedundancyErasureCoded' } }
+
+##
 # @BlockdevCreateNotSupported:
 #
 # This is used for all drivers that don't support creating images.
diff --git a/block/sheepdog.c b/block/sheepdog.c
index 3c3becf94d..22df2ba9d0 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -1882,6 +1882,48 @@ out_with_err_set:
 return ret;
 }
 
+static int parse_redundancy(BDRVSheepdogState *s, SheepdogRedundancy *opt)
+{
+struct SheepdogInode *inode = &s->inode;
+
+switch (opt->type) {
+case SHEEPDOG_REDUNDANCY_TYPE_FULL:
+if (opt->u.full.copies > SD_MAX_COPIES || opt->u.full.copies < 1) {
+return -EINVAL;
+}
+inode->copy_policy = 0;
+inode->nr_copies = opt->u.full.copies;
+return 0;
+
+case SHEEPDOG_REDUNDANCY_TYPE_ERASURE_CODED:
+{
+int64_t copy = opt->u.erasure_coded.data_strips;
+int64_t parity = opt->u.erasure_coded.parity_strips;
+
+if (copy != 2 && copy != 4 && copy != 8 && copy != 16) {
+return -EINVAL;
+}
+
+if (parity >= SD_EC_MAX_STRIP || parity < 1) {
+return -EINVAL;
+}
+
+/*
+ * 4 bits for parity and 4 bits for data.
+ * We have to compress upper data bits because it can't represent 16
+ */
+inode->copy_policy = ((copy / 2) << 4) + parity;
+inode->nr_copies = copy + parity;
+return 0;
+}
+
+default:
+g_assert_not_reached();
+}
+
+return -EINVAL;
+}
+
 /*
  * Sheepdog support two kinds of redundancy, full replication and erasure
  * coding.
@@ -1892,12 +1934,13 @@ out_with_err_set:
  * # create a erasure coded vdi with x data strips and y parity strips
  * -o redundancy=x:y (x must be one of {2,4,8,16} and 1 <= y < SD_EC_MAX_STRIP)
  */
-static int parse_redundancy(BDRVSheepdogState *s, const char *opt)
+static int parse_redundancy_str(BDRVSheepdogState *s, const char *opt)
 {
-struct SheepdogInode *inode = &s->inode;
+struct SheepdogRedundancy redundancy;
 const char *n1, *n2;
 long copy, parity;
 char p[10];
+int ret;
 
 pstrcpy(p, sizeof(p), opt);
 n1 = strtok(p, ":");
@@ -1907,35 +1950,32 @@ static int parse_redundancy(BDRVSheepdogState *s, const 
char *opt)
 return -EINVAL;
 }
 
-copy = strtol(n1, NULL, 10);
-/* FIXME fix error checking by switching to qemu_strtol() */
-if (copy > SD_MAX_COPIES || copy < 1) {
-return -EINVAL;
-}
-if (!n2) {
-inode->copy_policy = 0;
-inode->nr_copies = copy;
-return 0;
+ret = qemu_strtol(n1, NULL, 10, ©);
+if (ret < 0) {
+return ret;
 }
 
-if (copy != 2 && copy != 4 && copy != 8 && copy != 16) {
-return -EINVAL;
-}
+if (!n2) {
+redundancy = (SheepdogRedundancy) {
+.type   = SHEEPDOG_REDUNDANCY_TYPE_FULL,
+.u.full.copies  = copy,
+};
+} else {
+ret = qemu_strtol(n2, NULL, 10, &parity);
+if (ret < 0) {
+return ret;
+}
 
-parity = strtol(n2, NULL, 10);
-/* FIXME fix error checking by switching to qemu_strtol() */
-if (parity >= SD_

[Qemu-devel] [PATCH v2 26/36] nfs: Support .bdrv_co_create

This adds the .bdrv_co_create driver callback to nfs, which enables
image creation over QMP.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 qapi/block-core.json | 16 +++-
 block/nfs.c  | 74 +---
 2 files changed, 74 insertions(+), 16 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 6c0c16ebe3..085b791303 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3393,6 +3393,20 @@
 '*preallocation':   'PreallocMode' } }
 
 ##
+# @BlockdevCreateOptionsNfs:
+#
+# Driver specific image creation options for NFS.
+#
+# @location Where to store the new image file
+# @size Size of the virtual disk in bytes
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockdevCreateOptionsNfs',
+  'data': { 'location': 'BlockdevOptionsNfs',
+'size': 'size' } }
+
+##
 # @BlockdevQcow2Version:
 #
 # @v2:  The original QCOW2 format as introduced in qemu 0.10 (version 2)
@@ -3491,7 +3505,7 @@
   'iscsi':  'BlockdevCreateNotSupported',
   'luks':   'BlockdevCreateNotSupported',
   'nbd':'BlockdevCreateNotSupported',
-  'nfs':'BlockdevCreateNotSupported',
+  'nfs':'BlockdevCreateOptionsNfs',
   'null-aio':   'BlockdevCreateNotSupported',
   'null-co':'BlockdevCreateNotSupported',
   'nvme':   'BlockdevCreateNotSupported',
diff --git a/block/nfs.c b/block/nfs.c
index 9283bfbaae..c0c153cadb 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -551,33 +551,45 @@ out:
 return ret;
 }
 
-static int64_t nfs_client_open_qdict(NFSClient *client, QDict *options,
- int flags, int open_flags, Error **errp)
+static BlockdevOptionsNfs *nfs_options_qdict_to_qapi(QDict *options,
+ Error **errp)
 {
 BlockdevOptionsNfs *opts = NULL;
 QObject *crumpled = NULL;
 Visitor *v;
 Error *local_err = NULL;
-int ret;
 
 crumpled = qdict_crumple(options, errp);
 if (crumpled == NULL) {
-return -EINVAL;
+return NULL;
 }
 
 v = qobject_input_visitor_new_keyval(crumpled);
 visit_type_BlockdevOptionsNfs(v, NULL, &opts, &local_err);
 visit_free(v);
+qobject_decref(crumpled);
 
 if (local_err) {
-error_propagate(errp, local_err);
+return NULL;
+}
+
+return opts;
+}
+
+static int64_t nfs_client_open_qdict(NFSClient *client, QDict *options,
+ int flags, int open_flags, Error **errp)
+{
+BlockdevOptionsNfs *opts;
+int ret;
+
+opts = nfs_options_qdict_to_qapi(options, errp);
+if (opts == NULL) {
 ret = -EINVAL;
 goto fail;
 }
 
 ret = nfs_client_open(client, opts, flags, open_flags, errp);
 fail:
-qobject_decref(crumpled);
 qapi_free_BlockdevOptionsNfs(opts);
 return ret;
 }
@@ -614,17 +626,42 @@ static QemuOptsList nfs_create_opts = {
 }
 };
 
-static int nfs_file_create(const char *url, QemuOpts *opts, Error **errp)
+static int nfs_file_co_create(BlockdevCreateOptions *options, Error **errp)
 {
-int64_t ret, total_size;
+BlockdevCreateOptionsNfs *opts = &options->u.nfs;
 NFSClient *client = g_new0(NFSClient, 1);
-QDict *options = NULL;
+int ret;
+
+assert(options->driver == BLOCKDEV_DRIVER_NFS);
 
 client->aio_context = qemu_get_aio_context();
 
+ret = nfs_client_open(client, opts->location, O_CREAT, 0, errp);
+if (ret < 0) {
+goto out;
+}
+ret = nfs_ftruncate(client->context, client->fh, opts->size);
+nfs_client_close(client);
+
+out:
+g_free(client);
+return ret;
+}
+
+static int nfs_file_create(const char *url, QemuOpts *opts, Error **errp)
+{
+BlockdevCreateOptions *create_options;
+BlockdevCreateOptionsNfs *nfs_opts;
+QDict *options;
+int ret;
+
+create_options = g_new0(BlockdevCreateOptions, 1);
+create_options->driver = BLOCKDEV_DRIVER_NFS;
+nfs_opts = &create_options->u.nfs;
+
 /* Read out options */
-total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
-  BDRV_SECTOR_SIZE);
+nfs_opts->size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
+  BDRV_SECTOR_SIZE);
 
 options = qdict_new();
 ret = nfs_parse_uri(url, options, errp);
@@ -632,15 +669,21 @@ static int nfs_file_create(const char *url, QemuOpts 
*opts, Error **errp)
 goto out;
 }
 
-ret = nfs_client_open_qdict(client, options, O_CREAT, 0, errp);
+nfs_opts->location = nfs_options_qdict_to_qapi(options, errp);
+if (nfs_opts->location == NULL) {
+ret = -EINVAL;
+goto out;
+}
+
+ret = nfs_file_co_create(create_options, errp);
 if (ret < 0) {
 goto out;
 }
-ret = nfs_ftruncate(client->context, client->fh, total_size);
-nfs_client

Re: [Qemu-devel] [PATCH] hw/acpi-build: build SRAT memory affinity structures for NVDIMM

2018-02-21 Thread Igor Mammedov

On Tue, 20 Feb 2018 17:17:58 -0800
Dan Williams  wrote:

> On Tue, Feb 20, 2018 at 6:10 AM, Igor Mammedov  wrote:
> > On Sat, 17 Feb 2018 14:31:35 +0800
> > Haozhong Zhang  wrote:
> >  
> >> ACPI 6.2A Table 5-129 "SPA Range Structure" requires the proximity
> >> domain of a NVDIMM SPA range must match with corresponding entry in
> >> SRAT table.
> >>
> >> The address ranges of vNVDIMM in QEMU are allocated from the
> >> hot-pluggable address space, which is entirely covered by one SRAT
> >> memory affinity structure. However, users can set the vNVDIMM
> >> proximity domain in NFIT SPA range structure by the 'node' property of
> >> '-device nvdimm' to a value different than the one in the above SRAT
> >> memory affinity structure.
> >>
> >> In order to solve such proximity domain mismatch, this patch build one
> >> SRAT memory affinity structure for each NVDIMM device with the
> >> proximity domain used in NFIT. The remaining hot-pluggable address
> >> space is covered by one or multiple SRAT memory affinity structures
> >> with the proximity domain of the last node as before.
> >>
> >> Signed-off-by: Haozhong Zhang   
> > If we consider hotpluggable system, correctly implemented OS should
> > be able pull proximity from Device::_PXM and override any value from SRAT.
> > Do we really have a problem here (anything that breaks if we would use 
> > _PXM)?
> > Maybe we should add _PXM object to nvdimm device nodes instead of massaging 
> > SRAT?  
> 
> Unfortunately _PXM is an awkward fit. Currently the proximity domain
> is attached to the SPA range structure. The SPA range may be
> associated with multiple DIMM devices and those individual NVDIMMs may
> have conflicting _PXM properties.
There shouldn't be any conflict here as  NVDIMM device's _PXM method,
should override in runtime any proximity specified by parent scope.
(as parent scope I'd also count boot time NFIT/SRAT tables).

To make it more clear we could clear valid proximity domain flag in SPA
like this:

diff --git a/hw/acpi/nvdimm.c b/hw/acpi/nvdimm.c
index 59d6e42..131bca5 100644
--- a/hw/acpi/nvdimm.c
+++ b/hw/acpi/nvdimm.c
@@ -260,9 +260,7 @@ nvdimm_build_structure_spa(GArray *structures, DeviceState 
*dev)
  */
 nfit_spa->flags = cpu_to_le16(1 /* Control region is strictly for
management during hot add/online
-   operation */ |
-  2 /* Data in Proximity Domain field is
-   valid*/);
+   operation */);
 
 /* NUMA node. */
 nfit_spa->proximity_domain = cpu_to_le32(node);

> Even if that was unified across
> DIMMs it is ambiguous whether a DIMM-device _PXM would relate to the
> device's control interface, or the assembled persistent memory SPA
> range.
I'm not sure what you mean under 'device's control interface',
could you clarify where the ambiguity comes from?

I read spec as: _PXM applies to address range covered by NVDIMM
device it belongs to.

As for assembled SPA, I'd assume that it applies to interleaved set
and all NVDIMMs with it should be on the same node. It's somewhat
irrelevant question though as QEMU so far implements only
  1:1:1/SPA:Region Mapping:NVDIMM Device/
mapping.

My main concern with using static configuration tables for proximity
mapping, we'd miss on hotplug side of equation. However if we start
from dynamic side first, we could later complement it with static
tables if there really were need for it.

[Qemu-devel] [PATCH v2 28/36] sheepdog: Support .bdrv_co_create

This adds the .bdrv_co_create driver callback to sheepdog, which enables
image creation over QMP.

Signed-off-by: Kevin Wolf 
---
 qapi/block-core.json |  24 +-
 block/sheepdog.c | 240 +++
 2 files changed, 189 insertions(+), 75 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 2b249c9e3d..f7679fce53 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3512,6 +3512,28 @@
 'erasure-coded': 'SheepdogRedundancyErasureCoded' } }
 
 ##
+# @BlockdevCreateOptionsSheepdog:
+#
+# Driver specific image creation options for Sheepdog.
+#
+# @location Where to store the new image file
+# @size Size of the virtual disk in bytes
+# @backing-file File name of a base image
+# @preallocationPreallocation mode (allowed values: off, full)
+# @redundancy   Redundancy of the image
+# @object-size  Object size of the image
+#
+# Since: 2.12
+##
+{ 'struct': 'BlockdevCreateOptionsSheepdog',
+  'data': { 'location': 'BlockdevOptionsSheepdog',
+'size': 'size',
+'*backing-file':'str',
+'*preallocation':   'PreallocMode',
+'*redundancy':  'SheepdogRedundancy',
+'*object-size': 'size' } }
+
+##
 # @BlockdevCreateNotSupported:
 #
 # This is used for all drivers that don't support creating images.
@@ -3562,7 +3584,7 @@
   'raw':'BlockdevCreateNotSupported',
   'rbd':'BlockdevCreateOptionsRbd',
   'replication':'BlockdevCreateNotSupported',
-  'sheepdog':   'BlockdevCreateNotSupported',
+  'sheepdog':   'BlockdevCreateOptionsSheepdog',
   'ssh':'BlockdevCreateNotSupported',
   'throttle':   'BlockdevCreateNotSupported',
   'vdi':'BlockdevCreateNotSupported',
diff --git a/block/sheepdog.c b/block/sheepdog.c
index 22df2ba9d0..d45cf68ff2 100644
--- a/block/sheepdog.c
+++ b/block/sheepdog.c
@@ -17,6 +17,7 @@
 #include "qapi/error.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/qobject-input-visitor.h"
+#include "qapi/qobject-output-visitor.h"
 #include "qemu/uri.h"
 #include "qemu/error-report.h"
 #include "qemu/option.h"
@@ -533,23 +534,6 @@ static void sd_aio_setup(SheepdogAIOCB *acb, 
BDRVSheepdogState *s,
 qemu_co_mutex_unlock(&s->queue_lock);
 }
 
-static SocketAddress *sd_socket_address(const char *path,
-const char *host, const char *port)
-{
-SocketAddress *addr = g_new0(SocketAddress, 1);
-
-if (path) {
-addr->type = SOCKET_ADDRESS_TYPE_UNIX;
-addr->u.q_unix.path = g_strdup(path);
-} else {
-addr->type = SOCKET_ADDRESS_TYPE_INET;
-addr->u.inet.host = g_strdup(host ?: SD_DEFAULT_ADDR);
-addr->u.inet.port = g_strdup(port ?: stringify(SD_DEFAULT_PORT));
-}
-
-return addr;
-}
-
 static SocketAddress *sd_server_config(QDict *options, Error **errp)
 {
 QDict *server = NULL;
@@ -1882,6 +1866,42 @@ out_with_err_set:
 return ret;
 }
 
+static int sd_create_prealloc(BlockdevOptionsSheepdog *location, int64_t size,
+  Error **errp)
+{
+BlockDriverState *bs;
+Visitor *v;
+QObject *obj = NULL;
+QDict *qdict;
+Error *local_err = NULL;
+int ret;
+
+v = qobject_output_visitor_new(&obj);
+visit_type_BlockdevOptionsSheepdog(v, NULL, &location, &local_err);
+visit_free(v);
+
+if (local_err) {
+error_propagate(errp, local_err);
+qobject_decref(obj);
+return -EINVAL;
+}
+
+qdict = qobject_to_qdict(obj);
+qdict_flatten(qdict);
+
+bs = bdrv_open(NULL, NULL, qdict, BDRV_O_PROTOCOL | BDRV_O_RDWR, errp);
+if (bs == NULL) {
+ret = -EIO;
+goto fail;
+}
+
+ret = sd_prealloc(bs, 0, size, errp);
+fail:
+bdrv_unref(bs);
+QDECREF(qdict);
+return ret;
+}
+
 static int parse_redundancy(BDRVSheepdogState *s, SheepdogRedundancy *opt)
 {
 struct SheepdogInode *inode = &s->inode;
@@ -1934,9 +1954,9 @@ static int parse_redundancy(BDRVSheepdogState *s, 
SheepdogRedundancy *opt)
  * # create a erasure coded vdi with x data strips and y parity strips
  * -o redundancy=x:y (x must be one of {2,4,8,16} and 1 <= y < SD_EC_MAX_STRIP)
  */
-static int parse_redundancy_str(BDRVSheepdogState *s, const char *opt)
+static SheepdogRedundancy *parse_redundancy_str(const char *opt)
 {
-struct SheepdogRedundancy redundancy;
+SheepdogRedundancy *redundancy;
 const char *n1, *n2;
 long copy, parity;
 char p[10];
@@ -1947,26 +1967,27 @@ static int parse_redundancy_str(BDRVSheepdogState *s, 
const char *opt)
 n2 = strtok(NULL, ":");
 
 if (!n1) {
-return -EINVAL;
+return NULL;
 }
 
 ret = qemu_strtol(n1, NULL, 10, ©);
 if (ret < 0) {
-return ret;
+return NULL;
 }
 
+redundancy = g_new0(SheepdogRedundancy, 1);
 if (!n2) {
-

[Qemu-devel] [PATCH v2 19/36] rbd: Factor out qemu_rbd_connect()

The code to establish an RBD connection is duplicated between open and
create. In order to be able to share the code, factor out the code from
qemu_rbd_open() as a first step.

Signed-off-by: Kevin Wolf 
---
 block/rbd.c | 100 
 1 file changed, 60 insertions(+), 40 deletions(-)

diff --git a/block/rbd.c b/block/rbd.c
index 27fa11b473..4bbcce4eca 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -544,32 +544,17 @@ out:
 return rados_str;
 }
 
-static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags,
- Error **errp)
+static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
+char **s_snap, char **s_image_name,
+QDict *options, bool cache, Error **errp)
 {
-BDRVRBDState *s = bs->opaque;
-const char *pool, *snap, *conf, *user, *image_name, *keypairs;
-const char *secretid, *filename;
 QemuOpts *opts;
-Error *local_err = NULL;
 char *mon_host = NULL;
+const char *pool, *snap, *conf, *user, *image_name, *keypairs;
+const char *secretid;
+Error *local_err = NULL;
 int r;
 
-/* If we are given a filename, parse the filename, with precedence given to
- * filename encoded options */
-filename = qdict_get_try_str(options, "filename");
-if (filename) {
-warn_report("'filename' option specified. "
-"This is an unsupported option, and may be deprecated "
-"in the future");
-qemu_rbd_parse_filename(filename, options, &local_err);
-if (local_err) {
-r = -EINVAL;
-error_propagate(errp, local_err);
-goto exit;
-}
-}
-
 opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort);
 qemu_opts_absorb_qdict(opts, options, &local_err);
 if (local_err) {
@@ -600,35 +585,35 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
 goto failed_opts;
 }
 
-r = rados_create(&s->cluster, user);
+r = rados_create(cluster, user);
 if (r < 0) {
 error_setg_errno(errp, -r, "error initializing");
 goto failed_opts;
 }
 
-s->snap = g_strdup(snap);
-s->image_name = g_strdup(image_name);
+*s_snap = g_strdup(snap);
+*s_image_name = g_strdup(image_name);
 
 /* try default location when conf=NULL, but ignore failure */
-r = rados_conf_read_file(s->cluster, conf);
+r = rados_conf_read_file(*cluster, conf);
 if (conf && r < 0) {
 error_setg_errno(errp, -r, "error reading conf file %s", conf);
 goto failed_shutdown;
 }
 
-r = qemu_rbd_set_keypairs(s->cluster, keypairs, errp);
+r = qemu_rbd_set_keypairs(*cluster, keypairs, errp);
 if (r < 0) {
 goto failed_shutdown;
 }
 
 if (mon_host) {
-r = rados_conf_set(s->cluster, "mon_host", mon_host);
+r = rados_conf_set(*cluster, "mon_host", mon_host);
 if (r < 0) {
 goto failed_shutdown;
 }
 }
 
-if (qemu_rbd_set_auth(s->cluster, secretid, errp) < 0) {
+if (qemu_rbd_set_auth(*cluster, secretid, errp) < 0) {
 r = -EIO;
 goto failed_shutdown;
 }
@@ -640,24 +625,65 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
  * librbd defaults to no caching. If write through caching cannot
  * be set up, fall back to no caching.
  */
-if (flags & BDRV_O_NOCACHE) {
-rados_conf_set(s->cluster, "rbd_cache", "false");
+if (cache) {
+rados_conf_set(*cluster, "rbd_cache", "true");
 } else {
-rados_conf_set(s->cluster, "rbd_cache", "true");
+rados_conf_set(*cluster, "rbd_cache", "false");
 }
 
-r = rados_connect(s->cluster);
+r = rados_connect(*cluster);
 if (r < 0) {
 error_setg_errno(errp, -r, "error connecting");
 goto failed_shutdown;
 }
 
-r = rados_ioctx_create(s->cluster, pool, &s->io_ctx);
+r = rados_ioctx_create(*cluster, pool, io_ctx);
 if (r < 0) {
 error_setg_errno(errp, -r, "error opening pool %s", pool);
 goto failed_shutdown;
 }
 
+qemu_opts_del(opts);
+return 0;
+
+failed_shutdown:
+rados_shutdown(*cluster);
+g_free(*s_snap);
+g_free(*s_image_name);
+failed_opts:
+qemu_opts_del(opts);
+g_free(mon_host);
+return r;
+}
+
+static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags,
+ Error **errp)
+{
+BDRVRBDState *s = bs->opaque;
+Error *local_err = NULL;
+const char *filename;
+int r;
+
+/* If we are given a filename, parse the filename, with precedence given to
+ * filename encoded options */
+filename = qdict_get_try_str(options, "filename");
+if (filename) {
+warn_report("'filename' option specified. "
+"This is an unsupported option, and may be deprecated "
+

[Qemu-devel] [PATCH v2 18/36] rbd: Fix use after free in qemu_rbd_set_keypairs() error path

If we want to include the invalid option name in the error message, we
can't free the string earlier than that.

Signed-off-by: Kevin Wolf 
---
 block/rbd.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/block/rbd.c b/block/rbd.c
index 8474b0ba11..27fa11b473 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -268,13 +268,14 @@ static int qemu_rbd_set_keypairs(rados_t cluster, const 
char *keypairs_json,
 key = qstring_get_str(name);
 
 ret = rados_conf_set(cluster, key, qstring_get_str(value));
-QDECREF(name);
 QDECREF(value);
 if (ret < 0) {
 error_setg_errno(errp, -ret, "invalid conf option %s", key);
+QDECREF(name);
 ret = -EINVAL;
 break;
 }
+QDECREF(name);
 }
 
 QDECREF(keypairs);
-- 
2.13.6

[Qemu-devel] [PATCH v2 34/36] block: Fail bdrv_truncate() with negative size

Most callers have their own checks, but something like this should also
be checked centrally. As it happens, x-blockdev-create can pass negative
image sizes to format drivers (because there is no QAPI type that would
reject negative numbers) and triggers the check added by this patch.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/block.c b/block.c
index 4a7e448226..5c874aefa1 100644
--- a/block.c
+++ b/block.c
@@ -3684,6 +3684,11 @@ int bdrv_truncate(BdrvChild *child, int64_t offset, 
PreallocMode prealloc,
 error_setg(errp, "No medium inserted");
 return -ENOMEDIUM;
 }
+if (offset < 0) {
+error_setg(errp, "Image size cannot be negative");
+return -EINVAL;
+}
+
 if (!drv->bdrv_truncate) {
 if (bs->file && drv->is_filter) {
 return bdrv_truncate(bs->file, offset, prealloc, errp);
-- 
2.13.6

[Qemu-devel] [PATCH v2 16/36] file-win32: Support .bdrv_co_create

This adds the .bdrv_co_create driver callback to file-win32, which
enables image creation over QMP.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block/file-win32.c | 45 +
 1 file changed, 37 insertions(+), 8 deletions(-)

diff --git a/block/file-win32.c b/block/file-win32.c
index f24c7bb92c..d572cde357 100644
--- a/block/file-win32.c
+++ b/block/file-win32.c
@@ -553,29 +553,58 @@ static int64_t 
raw_get_allocated_file_size(BlockDriverState *bs)
 return st.st_size;
 }
 
-static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
+static int raw_co_create(BlockdevCreateOptions *options, Error **errp)
 {
+BlockdevCreateOptionsFile *file_opts;
 int fd;
-int64_t total_size = 0;
 
-strstart(filename, "file:", &filename);
+assert(options->driver == BLOCKDEV_DRIVER_FILE);
+file_opts = &options->u.file;
 
-/* Read out options */
-total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
-  BDRV_SECTOR_SIZE);
+if (file_opts->has_preallocation) {
+error_setg(errp, "Preallocation is not supported on Windows");
+return -EINVAL;
+}
+if (file_opts->has_nocow) {
+error_setg(errp, "nocow is not supported on Windows");
+return -EINVAL;
+}
 
-fd = qemu_open(filename, O_WRONLY | O_CREAT | O_TRUNC | O_BINARY,
+fd = qemu_open(file_opts->filename, O_WRONLY | O_CREAT | O_TRUNC | 
O_BINARY,
0644);
 if (fd < 0) {
 error_setg_errno(errp, errno, "Could not create file");
 return -EIO;
 }
 set_sparse(fd);
-ftruncate(fd, total_size);
+ftruncate(fd, file_opts->size);
 qemu_close(fd);
+
 return 0;
 }
 
+static int raw_create(const char *filename, QemuOpts *opts, Error **errp)
+{
+BlockdevCreateOptions options;
+int64_t total_size = 0;
+
+strstart(filename, "file:", &filename);
+
+/* Read out options */
+total_size = ROUND_UP(qemu_opt_get_size_del(opts, BLOCK_OPT_SIZE, 0),
+  BDRV_SECTOR_SIZE);
+
+options = (BlockdevCreateOptions) {
+.driver = BLOCKDEV_DRIVER_FILE,
+.u.file = {
+.filename   = (char *) filename,
+.size   = total_size,
+.has_preallocation  = false,
+.has_nocow  = false,
+},
+};
+return raw_co_create(&options, errp);
+}
 
 static QemuOptsList raw_create_opts = {
 .name = "raw-create-opts",
-- 
2.13.6

[Qemu-devel] [PATCH v2 21/36] rbd: Pass BlockdevOptionsRbd to qemu_rbd_connect()

With the conversion to a QAPI options object, the function is now
prepared to be used in a .bdrv_co_create implementation.

Signed-off-by: Kevin Wolf 
---
 block/rbd.c | 102 +++-
 1 file changed, 52 insertions(+), 50 deletions(-)

diff --git a/block/rbd.c b/block/rbd.c
index 2e79c2d1fd..26641e53e0 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -24,6 +24,8 @@
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qjson.h"
 #include "qapi/qmp/qlist.h"
+#include "qapi/qobject-input-visitor.h"
+#include "qapi-visit.h"
 
 /*
  * When specifying the image filename use:
@@ -482,24 +484,27 @@ static void qemu_rbd_complete_aio(RADOSCB *rcb)
 qemu_aio_unref(acb);
 }
 
-static char *qemu_rbd_mon_host(QDict *options, Error **errp)
+static char *qemu_rbd_mon_host(BlockdevOptionsRbd *opts, Error **errp)
 {
-const char **vals = g_new(const char *, qdict_size(options) + 1);
-char keybuf[32];
+const char **vals;
 const char *host, *port;
 char *rados_str;
-int i;
-
-for (i = 0;; i++) {
-sprintf(keybuf, "server.%d.host", i);
-host = qdict_get_try_str(options, keybuf);
-qdict_del(options, keybuf);
-sprintf(keybuf, "server.%d.port", i);
-port = qdict_get_try_str(options, keybuf);
-qdict_del(options, keybuf);
-if (!host && !port) {
-break;
-}
+InetSocketAddressBaseList *p;
+int i, cnt;
+
+if (!opts->has_server) {
+return NULL;
+}
+
+for (cnt = 0, p = opts->server; p; p = p->next) {
+cnt++;
+}
+
+vals = g_new(const char *, cnt + 1);
+
+for (i = 0, p = opts->server; p; p = p->next, i++) {
+host = p->value->host;
+port = p->value->port;
 if (!host) {
 error_setg(errp, "Parameter server.%d.host is missing", i);
 rados_str = NULL;
@@ -524,56 +529,34 @@ out:
 
 static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
 char **s_snap, char **s_image_name,
-QDict *options, bool cache,
+BlockdevOptionsRbd *opts, bool cache,
 const char *keypairs, const char *secretid,
 Error **errp)
 {
-QemuOpts *opts;
 char *mon_host = NULL;
-const char *pool, *snap, *conf, *user, *image_name;
 Error *local_err = NULL;
 int r;
 
-opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort);
-qemu_opts_absorb_qdict(opts, options, &local_err);
+mon_host = qemu_rbd_mon_host(opts, &local_err);
 if (local_err) {
 error_propagate(errp, local_err);
 r = -EINVAL;
 goto failed_opts;
 }
 
-mon_host = qemu_rbd_mon_host(options, &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
-r = -EINVAL;
-goto failed_opts;
-}
-
-pool   = qemu_opt_get(opts, "pool");
-conf   = qemu_opt_get(opts, "conf");
-snap   = qemu_opt_get(opts, "snapshot");
-user   = qemu_opt_get(opts, "user");
-image_name = qemu_opt_get(opts, "image");
-
-if (!pool || !image_name) {
-error_setg(errp, "Parameters 'pool' and 'image' are required");
-r = -EINVAL;
-goto failed_opts;
-}
-
-r = rados_create(cluster, user);
+r = rados_create(cluster, opts->user);
 if (r < 0) {
 error_setg_errno(errp, -r, "error initializing");
 goto failed_opts;
 }
 
-*s_snap = g_strdup(snap);
-*s_image_name = g_strdup(image_name);
+*s_snap = g_strdup(opts->snapshot);
+*s_image_name = g_strdup(opts->image);
 
 /* try default location when conf=NULL, but ignore failure */
-r = rados_conf_read_file(*cluster, conf);
-if (conf && r < 0) {
-error_setg_errno(errp, -r, "error reading conf file %s", conf);
+r = rados_conf_read_file(*cluster, opts->conf);
+if (opts->has_conf && r < 0) {
+error_setg_errno(errp, -r, "error reading conf file %s", opts->conf);
 goto failed_shutdown;
 }
 
@@ -613,13 +596,12 @@ static int qemu_rbd_connect(rados_t *cluster, 
rados_ioctx_t *io_ctx,
 goto failed_shutdown;
 }
 
-r = rados_ioctx_create(*cluster, pool, io_ctx);
+r = rados_ioctx_create(*cluster, opts->pool, io_ctx);
 if (r < 0) {
-error_setg_errno(errp, -r, "error opening pool %s", pool);
+error_setg_errno(errp, -r, "error opening pool %s", opts->pool);
 goto failed_shutdown;
 }
 
-qemu_opts_del(opts);
 return 0;
 
 failed_shutdown:
@@ -627,7 +609,6 @@ failed_shutdown:
 g_free(*s_snap);
 g_free(*s_image_name);
 failed_opts:
-qemu_opts_del(opts);
 g_free(mon_host);
 return r;
 }
@@ -636,6 +617,9 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict 
*options, int flags,
  Error **errp)
 {
 BDRVRBDState *s = bs->opaque;
+BlockdevOptionsRbd *opts = N

[Qemu-devel] [PATCH v3] specs/qcow2: Fix documentation of the compressed cluster descriptor

This patch fixes several mistakes in the documentation of the
compressed cluster descriptor:

1) the documentation claims that the cluster descriptor contains the
   number of sectors used to store the compressed data, but what it
   actually contains is the number of sectors *minus one* or, in other
   words, the number of additional sectors after the first one.

2) the width of the fields is incorrectly specified. The number of bits
   used by each field is

  x = 62 - (cluster_bits - 8)   for the offset field
  y = (cluster_bits - 8)for the size field

   So the offset field's location is [0, x-1], not [0, x] as stated.

3) the size field does not contain the size of the compressed data,
   but rather the number of sectors where that data is stored. The
   compressed data starts at the exact point specified in the offset
   field and ends when there's enough data to produce a cluster of
   decompressed data. Both points can be in the middle of a sector,
   allowing several compressed clusters to be stored next to one
   another, sharing sectors if necessary.

Signed-off-by: Alberto Garcia 
---

v3: Fix the specification of the width of the fields, and update the
explanation of how the compressed data is stored [Eric].

v2: I realized that the documentation is not completely clear about
the exact location and size of the compressed data, so I updated
the patch to clarify this.

---
 docs/interop/qcow2.txt | 16 +---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt
index d7fdb1fee3..feb711fb6a 100644
--- a/docs/interop/qcow2.txt
+++ b/docs/interop/qcow2.txt
@@ -426,10 +426,20 @@ Standard Cluster Descriptor:
 
 Compressed Clusters Descriptor (x = 62 - (cluster_bits - 8)):
 
-Bit  0 -  x:Host cluster offset. This is usually _not_ aligned to a
-cluster boundary!
+Bit  0 - x-1:   Host cluster offset. This is usually _not_ aligned to a
+cluster or sector boundary!
 
-   x+1 - 61:Compressed size of the images in sectors of 512 bytes
+ x - 61:Number of additional 512-byte sectors used for the
+compressed data, beyond the sector containing the offset
+in the previous field. Some of these sectors may reside
+in the next contiguous host cluster.
+
+Note that the compressed data does not necessarily occupy
+all of the bytes in the final sector; rather, decompression
+stops when it has produced a cluster of data.
+
+Another compressed cluster may map to the tail of the final
+sector used by this compressed cluster.
 
 If a cluster is unallocated, read requests shall read the data from the backing
 file (except if bit 0 in the Standard Cluster Descriptor is set). If there is
-- 
2.11.0

[Qemu-devel] [PATCH v2 35/36] qemu-iotests: Test qcow2 over file image creation with QMP

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 tests/qemu-iotests/206 | 436 +
 tests/qemu-iotests/206.out | 209 ++
 tests/qemu-iotests/group   |   1 +
 3 files changed, 646 insertions(+)
 create mode 100755 tests/qemu-iotests/206
 create mode 100644 tests/qemu-iotests/206.out

diff --git a/tests/qemu-iotests/206 b/tests/qemu-iotests/206
new file mode 100755
index 00..0a18b2b19a
--- /dev/null
+++ b/tests/qemu-iotests/206
@@ -0,0 +1,436 @@
+#!/bin/bash
+#
+# Test qcow2 and file image creation
+#
+# Copyright (C) 2018 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+# creator
+owner=kw...@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+status=1   # failure is the default!
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+
+function do_run_qemu()
+{
+echo Testing: "$@"
+$QEMU -nographic -qmp stdio -serial none "$@"
+echo
+}
+
+function run_qemu()
+{
+do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qmp \
+  | _filter_qemu | _filter_imgfmt \
+  | _filter_actual_image_size
+}
+
+echo
+echo "=== Successful image creation (defaults) ==="
+echo
+
+size=$((128 * 1024 * 1024))
+
+run_qemu <

[Qemu-devel] [PATCH v2 23/36] rbd: Assing s->snap/image_name in qemu_rbd_open()