Re: [Qemu-devel] [PATCH 2/2] hw/pci-host/x86: extend the 64-bit PCI hole relative to the fw-assigned base

[Gerd Hoffmann]

  Hi,

> > Maybe using memdev file backend with manually created sparse file
> > might actually work (with preallocate disabled)
> 
> Thanks, this sounds like a good idea.
> 
> I see shm_open() is used heavily in ivshmem-related tests. I haven't
> looked much at shm_open() before. (I've always known it existed in
> POSIX, but I've never cared.)

How about improving the lovely pci-testdev we have a bit?  Then we can
have huge pci bars without needing backing storage for them ...

HTH,
  Gerd

=== cut here ===
>From 05cfced149b0b5c953391666c3151034bc7fe88b Mon Sep 17 00:00:00 2001
From: Gerd Hoffmann 
Date: Thu, 27 Sep 2018 07:43:10 +0200
Subject: [PATCH] pci-testdev: add optional memory bar

Add memory bar to pci-testdev.  Size is configurable using the membar
property.  Setting the size to zero (default) turns it off.

Signed-off-by: Gerd Hoffmann 
---
 hw/misc/pci-testdev.c | 18 ++
 1 file changed, 18 insertions(+)

diff --git a/hw/misc/pci-testdev.c b/hw/misc/pci-testdev.c
index 32041f535f..af4d678ee4 100644
--- a/hw/misc/pci-testdev.c
+++ b/hw/misc/pci-testdev.c
@@ -85,6 +85,9 @@ typedef struct PCITestDevState {
 MemoryRegion portio;
 IOTest *tests;
 int current;
+
+size_t membar_size;
+MemoryRegion membar;
 } PCITestDevState;
 
 #define TYPE_PCI_TEST_DEV "pci-testdev"
@@ -253,6 +256,15 @@ static void pci_testdev_realize(PCIDevice *pci_dev, Error 
**errp)
 pci_register_bar(pci_dev, 0, PCI_BASE_ADDRESS_SPACE_MEMORY, >mmio);
 pci_register_bar(pci_dev, 1, PCI_BASE_ADDRESS_SPACE_IO, >portio);
 
+if (d->membar_size) {
+memory_region_init(>membar, OBJECT(d), "membar", d->membar_size);
+pci_register_bar(pci_dev, 2,
+ PCI_BASE_ADDRESS_SPACE_MEMORY |
+ PCI_BASE_ADDRESS_MEM_PREFETCH |
+ PCI_BASE_ADDRESS_MEM_TYPE_64,
+ >membar);
+}
+
 d->current = -1;
 d->tests = g_malloc0(IOTEST_MAX * sizeof *d->tests);
 for (i = 0; i < IOTEST_MAX; ++i) {
@@ -305,6 +317,11 @@ static void qdev_pci_testdev_reset(DeviceState *dev)
 pci_testdev_reset(d);
 }
 
+static Property pci_testdev_properties[] = {
+DEFINE_PROP_SIZE("membar", PCITestDevState, membar_size, 0),
+DEFINE_PROP_END_OF_LIST(),
+};
+
 static void pci_testdev_class_init(ObjectClass *klass, void *data)
 {
 DeviceClass *dc = DEVICE_CLASS(klass);
@@ -319,6 +336,7 @@ static void pci_testdev_class_init(ObjectClass *klass, void 
*data)
 dc->desc = "PCI Test Device";
 set_bit(DEVICE_CATEGORY_MISC, dc->categories);
 dc->reset = qdev_pci_testdev_reset;
+dc->props = pci_testdev_properties;
 }
 
 static const TypeInfo pci_testdev_info = {
-- 
2.9.3

Re: [Qemu-devel] [PATCH] qemu/compiler: Wrap __attribute__((flatten)) in a macro

[Thomas Huth]

On 2018-09-26 20:57, Peter Maydell wrote:
> On 26 September 2018 at 16:59, Thomas Huth  wrote:
> 
>> +/*
>> + * Clang 3.4 claims to be compatible with GCC 4.2, but does not have the
>> + * "flatten" attribute, so we've got to handle Clang via __has_attribute 
>> here
>> + */
>> +#if defined(__clang__) && defined(__has_attribute)
>> +# if __has_attribute(flatten)
>> +#  define QEMU_FLATTEN __attribute__((flatten))
>> +# endif
>> +#elif !defined(__clang__) && QEMU_GNUC_PREREQ(4, 1)
>> +# define QEMU_FLATTEN __attribute__((flatten))
>> +#endif
>> +#ifndef QEMU_FLATTEN
>> +# define QEMU_FLATTEN
>> +#endif
> 
> I think it would be cleaner to say: if we have __has_attribute,
> trust it; otherwise fall back to version testing. Also, we
> already require at least GCC 4.1 so a version test against
> that is unnecessary. So something like:
> 
> #ifndef __has_attribute
> #define __has_attribute(x) 0 / compatibility with older GCC */
> #endif
> 
> /*
>  * gcc doesn't provide __has_attribute() until gcc 5,
>  * but we know all the gcc versions we support have flatten.
>  * clang may not have flatten but always has __has_attribute().
>  */
> #if __has_attribute(flatten) || !defined(__clang__)
> # define QEMU_FLATTEN __attribute__((flatten))
> #else
> # define QEMU_FLATTEN
> #endif

Good idea, I'll send a v2 with that.

Speaking of the minimum GCC version that we require: Did we ever
officially define that? Or is it just a result from appendix C in our
qemu-doc? I guess the minimum GCC could be defined to 4.2 since that's
the compiler that was still used in OpenBSD recently, before they
switched to Clang?

Maybe we should add a list in our qemu-doc with some minimum versions
that we require (e.g. also glib), so that it is easier to look this up?

Also when searching for GNUC_PREREQ in the sources, there are at least
some occurances in tests/tcg/arm/fcvt.c and
include/fpu/softfloat-macros.h which we could simplify these days...

 Thomas

Re: [Qemu-devel] [PATCH v4 09/24] memory-device: add and use memory_device_get_region_size()

[David Gibson]

On Wed, Sep 26, 2018 at 11:42:04AM +0200, David Hildenbrand wrote:
> We will factor out get_memory_region() from pc-dimm to memory device code
> soon. Once that is done, get_region_size() can be implemented
> generically and essentially be replaced by
> memory_device_get_region_size (and work only on get_memory_region()).
> 
> We have some users of get_memory_region() (spapr and pc-dimm code) that are
> only interested in the size. So let's rework them to use
> memory_device_get_region_size() first, then we can factor out
> get_memory_region() and eventually remove get_region_size() without
> touching the same code multiple times.
> 
> Signed-off-by: David Hildenbrand 

Reviewed-by: David Gibson 

> ---
>  hw/mem/memory-device.c | 13 ++---
>  hw/mem/pc-dimm.c   | 10 --
>  hw/ppc/spapr.c | 21 +++--
>  include/hw/mem/memory-device.h |  2 ++
>  4 files changed, 23 insertions(+), 23 deletions(-)
> 
> diff --git a/hw/mem/memory-device.c b/hw/mem/memory-device.c
> index e935681438..882364b388 100644
> --- a/hw/mem/memory-device.c
> +++ b/hw/mem/memory-device.c
> @@ -57,10 +57,9 @@ static int memory_device_used_region_size(Object *obj, 
> void *opaque)
>  if (object_dynamic_cast(obj, TYPE_MEMORY_DEVICE)) {
>  const DeviceState *dev = DEVICE(obj);
>  const MemoryDeviceState *md = MEMORY_DEVICE(obj);
> -const MemoryDeviceClass *mdc = MEMORY_DEVICE_GET_CLASS(obj);
>  
>  if (dev->realized) {
> -*size += mdc->get_region_size(md, _abort);
> +*size += memory_device_get_region_size(md, _abort);
>  }
>  }
>  
> @@ -169,7 +168,7 @@ uint64_t memory_device_get_free_addr(MachineState *ms, 
> const uint64_t *hint,
>  uint64_t md_size, md_addr;
>  
>  md_addr = mdc->get_addr(md);
> -md_size = mdc->get_region_size(md, _abort);
> +md_size = memory_device_get_region_size(md, _abort);
>  
>  if (ranges_overlap(md_addr, md_size, new_addr, size)) {
>  if (hint) {
> @@ -268,6 +267,14 @@ void memory_device_unplug_region(MachineState *ms, 
> MemoryRegion *mr)
>  memory_region_del_subregion(>device_memory->mr, mr);
>  }
>  
> +uint64_t memory_device_get_region_size(const MemoryDeviceState *md,
> +   Error **errp)
> +{
> +MemoryDeviceClass *mdc = MEMORY_DEVICE_GET_CLASS(md);
> +
> +return mdc->get_region_size(md, errp);
> +}
> +
>  static const TypeInfo memory_device_info = {
>  .name  = TYPE_MEMORY_DEVICE,
>  .parent= TYPE_INTERFACE,
> diff --git a/hw/mem/pc-dimm.c b/hw/mem/pc-dimm.c
> index c948d57c63..76155c3f5a 100644
> --- a/hw/mem/pc-dimm.c
> +++ b/hw/mem/pc-dimm.c
> @@ -161,16 +161,14 @@ static Property pc_dimm_properties[] = {
>  static void pc_dimm_get_size(Object *obj, Visitor *v, const char *name,
>   void *opaque, Error **errp)
>  {
> +Error *local_err = NULL;
>  uint64_t value;
> -MemoryRegion *mr;
> -PCDIMMDevice *dimm = PC_DIMM(obj);
> -PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(obj);
>  
> -mr = ddc->get_memory_region(dimm, errp);
> -if (!mr) {
> +value = memory_device_get_region_size(MEMORY_DEVICE(obj), _err);
> +if (local_err) {
> +error_propagate(errp, local_err);
>  return;
>  }
> -value = memory_region_size(mr);
>  
>  visit_type_uint64(v, name, , errp);
>  }
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index c078347b66..c08130facb 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -3128,12 +3128,10 @@ static void spapr_memory_plug(HotplugHandler 
> *hotplug_dev, DeviceState *dev,
>  Error *local_err = NULL;
>  sPAPRMachineState *ms = SPAPR_MACHINE(hotplug_dev);
>  PCDIMMDevice *dimm = PC_DIMM(dev);
> -PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm);
> -MemoryRegion *mr = ddc->get_memory_region(dimm, _abort);
>  uint64_t size, addr;
>  uint32_t node;
>  
> -size = memory_region_size(mr);
> +size = memory_device_get_region_size(MEMORY_DEVICE(dev), _abort);
>  
>  pc_dimm_plug(dimm, MACHINE(ms), _err);
>  if (local_err) {
> @@ -3169,9 +3167,7 @@ static void spapr_memory_pre_plug(HotplugHandler 
> *hotplug_dev, DeviceState *dev,
>  const sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(hotplug_dev);
>  sPAPRMachineState *spapr = SPAPR_MACHINE(hotplug_dev);
>  PCDIMMDevice *dimm = PC_DIMM(dev);
> -PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm);
>  Error *local_err = NULL;
> -MemoryRegion *mr;
>  uint64_t size;
>  Object *memdev;
>  hwaddr pagesize;
> @@ -3181,11 +3177,11 @@ static void spapr_memory_pre_plug(HotplugHandler 
> *hotplug_dev, DeviceState *dev,
>  return;
>  }
>  
> -mr = ddc->get_memory_region(dimm, errp);
> -if (!mr) {
> +size = memory_device_get_region_size(MEMORY_DEVICE(dimm), _err);
> +if (local_err) {
> +error_propagate(errp, 

Re: [Qemu-devel] [PATCH v4 03/24] pc-dimm: pass PCDIMMDevice to pc_dimm_.*plug

[David Gibson]

On Wed, Sep 26, 2018 at 11:41:58AM +0200, David Hildenbrand wrote:
> We're plugging/unplugging a PCDIMMDevice, so directly pass this type
> instead of a more generic DeviceState.
> 
> Signed-off-by: David Hildenbrand 

Seems type-safer than the previous version.

Reviewed-by: David Gibson 

and ppc parts:

Acked-by: David Gibson 


> ---
>  hw/i386/pc.c |  6 +++---
>  hw/mem/pc-dimm.c | 16 +++-
>  hw/ppc/spapr.c   |  8 
>  include/hw/mem/pc-dimm.h |  6 +++---
>  4 files changed, 17 insertions(+), 19 deletions(-)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 03148450c8..86c16f9aaf 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -1699,7 +1699,7 @@ static void pc_memory_pre_plug(HotplugHandler 
> *hotplug_dev, DeviceState *dev,
>  return;
>  }
>  
> -pc_dimm_pre_plug(dev, MACHINE(hotplug_dev),
> +pc_dimm_pre_plug(PC_DIMM(dev), MACHINE(hotplug_dev),
>   pcmc->enforce_aligned_dimm ? NULL : _align, 
> errp);
>  }
>  
> @@ -1711,7 +1711,7 @@ static void pc_memory_plug(HotplugHandler *hotplug_dev,
>  PCMachineState *pcms = PC_MACHINE(hotplug_dev);
>  bool is_nvdimm = object_dynamic_cast(OBJECT(dev), TYPE_NVDIMM);
>  
> -pc_dimm_plug(dev, MACHINE(pcms), _err);
> +pc_dimm_plug(PC_DIMM(dev), MACHINE(pcms), _err);
>  if (local_err) {
>  goto out;
>  }
> @@ -1771,7 +1771,7 @@ static void pc_memory_unplug(HotplugHandler 
> *hotplug_dev,
>  goto out;
>  }
>  
> -pc_dimm_unplug(dev, MACHINE(pcms));
> +pc_dimm_unplug(PC_DIMM(dev), MACHINE(pcms));
>  object_unparent(OBJECT(dev));
>  
>   out:
> diff --git a/hw/mem/pc-dimm.c b/hw/mem/pc-dimm.c
> index fb6bcaedc4..2375eb2731 100644
> --- a/hw/mem/pc-dimm.c
> +++ b/hw/mem/pc-dimm.c
> @@ -29,11 +29,11 @@
>  
>  static int pc_dimm_get_free_slot(const int *hint, int max_slots, Error 
> **errp);
>  
> -void pc_dimm_pre_plug(DeviceState *dev, MachineState *machine,
> +void pc_dimm_pre_plug(PCDIMMDevice *dimm, MachineState *machine,
>const uint64_t *legacy_align, Error **errp)
>  {
> -PCDIMMDevice *dimm = PC_DIMM(dev);
>  PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm);
> +DeviceState *dev = DEVICE(dimm);
>  Error *local_err = NULL;
>  MemoryRegion *mr;
>  uint64_t addr, align;
> @@ -69,32 +69,30 @@ out:
>  error_propagate(errp, local_err);
>  }
>  
> -void pc_dimm_plug(DeviceState *dev, MachineState *machine, Error **errp)
> +void pc_dimm_plug(PCDIMMDevice *dimm, MachineState *machine, Error **errp)
>  {
> -PCDIMMDevice *dimm = PC_DIMM(dev);
>  PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm);
>  MemoryRegion *vmstate_mr = ddc->get_vmstate_memory_region(dimm,
>_abort);
>  MemoryRegion *mr = ddc->get_memory_region(dimm, _abort);
>  uint64_t addr;
>  
> -addr = object_property_get_uint(OBJECT(dev), PC_DIMM_ADDR_PROP,
> +addr = object_property_get_uint(OBJECT(dimm), PC_DIMM_ADDR_PROP,
>  _abort);
>  
>  memory_device_plug_region(machine, mr, addr);
> -vmstate_register_ram(vmstate_mr, dev);
> +vmstate_register_ram(vmstate_mr, DEVICE(dimm));
>  }
>  
> -void pc_dimm_unplug(DeviceState *dev, MachineState *machine)
> +void pc_dimm_unplug(PCDIMMDevice *dimm, MachineState *machine)
>  {
> -PCDIMMDevice *dimm = PC_DIMM(dev);
>  PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm);
>  MemoryRegion *vmstate_mr = ddc->get_vmstate_memory_region(dimm,
>_abort);
>  MemoryRegion *mr = ddc->get_memory_region(dimm, _abort);
>  
>  memory_device_unplug_region(machine, mr);
> -vmstate_unregister_ram(vmstate_mr, dev);
> +vmstate_unregister_ram(vmstate_mr, DEVICE(dimm));
>  }
>  
>  static int pc_dimm_slot2bitmap(Object *obj, void *opaque)
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 98868d893a..c078347b66 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -3135,7 +3135,7 @@ static void spapr_memory_plug(HotplugHandler 
> *hotplug_dev, DeviceState *dev,
>  
>  size = memory_region_size(mr);
>  
> -pc_dimm_plug(dev, MACHINE(ms), _err);
> +pc_dimm_plug(dimm, MACHINE(ms), _err);
>  if (local_err) {
>  goto out;
>  }
> @@ -3158,7 +3158,7 @@ static void spapr_memory_plug(HotplugHandler 
> *hotplug_dev, DeviceState *dev,
>  return;
>  
>  out_unplug:
> -pc_dimm_unplug(dev, MACHINE(ms));
> +pc_dimm_unplug(dimm, MACHINE(ms));
>  out:
>  error_propagate(errp, local_err);
>  }
> @@ -3202,7 +3202,7 @@ static void spapr_memory_pre_plug(HotplugHandler 
> *hotplug_dev, DeviceState *dev,
>  return;
>  }
>  
> -pc_dimm_pre_plug(dev, MACHINE(hotplug_dev), NULL, errp);
> +pc_dimm_pre_plug(dimm, MACHINE(hotplug_dev), NULL, errp);
>  }
>  
>  struct sPAPRDIMMState {
> @@ -3314,7 +3314,7 @@ static void 

[Qemu-devel] [PATCH] bitmap: Update count after a merge

[Eric Blake]

We need an accurate count of the number of bits set in a bitmap
after a merge. In particular, since the merge operation short-circuits
a merge from an empty source, if you have bitmaps A, B, and C where
B started empty, then merge C into B, and B into A, an inaccurate
count meant that A did not get the contents of C.

Fixes: be58721db
CC: qemu-sta...@nongnu.org
Signed-off-by: Eric Blake 
---

Probably worth some testsuite coverage, but for a late-night one-liner,
this is as much as I can do today.

 util/hbitmap.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/util/hbitmap.c b/util/hbitmap.c
index bcd304041aa..52e12da4b48 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -753,3 +753,4 @@ bool hbitmap_merge(HBitmap *a, const HBitmap *b)
 a->levels[i][j] |= b->levels[i][j];
 }
 }
+a->count = hb_count_between(a, 0, a->size - 1);

 return true;
 }
-- 
2.17.1

Re: [Qemu-devel] [PATCH 3/3] memory: Fix access_with_adjusted_size(small size) on big-endian memory regions

[Eric Blake]

On 9/26/18 7:24 PM, Philippe Mathieu-Daudé wrote:

Memory regions configured as DEVICE_BIG_ENDIAN (or DEVICE_NATIVE_ENDIAN on
big-endian guest) behave incorrectly when the memory access 'size' is smaller
than the implementation 'access_size'.





Fix this by changing the access_fn() prototype to handle signed shift values,
and modify the memory_region_shift_read|write_access() helpers to correctly
arithmetic shift the opposite direction when the 'shift' value is negative.

Signed-off-by: Philippe Mathieu-Daudé 
---
  memory.c | 34 +++---
  1 file changed, 23 insertions(+), 11 deletions(-)

diff --git a/memory.c b/memory.c
index 48edf7dc23..a9f0fdc440 100644
--- a/memory.c
+++ b/memory.c
@@ -375,18 +375,30 @@ static void adjust_endianness(MemoryRegion *mr, uint64_t 
*data, unsigned size)
  }
  
  static inline void memory_region_shift_read_access(uint64_t *value,

-   unsigned shift,
+   signed shift,


This is more typically spelled s/signed/int/

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PULL 3/3] nbd/server: send more than one extent of base:allocation context

[Eric Blake]

From: Vladimir Sementsov-Ogievskiy 

This is necessary for efficient block-status export, for clients which
support it.  (qemu is not yet such a client, but could become one.)

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Message-Id: <20180704112302.471456-3-vsement...@virtuozzo.com>
[eblake: grammar tweaks]
Signed-off-by: Eric Blake 
---
 nbd/server.c | 87 ++--
 1 file changed, 64 insertions(+), 23 deletions(-)

diff --git a/nbd/server.c b/nbd/server.c
index 12f721482dd..c3dd402b45e 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -1844,37 +1844,68 @@ static int coroutine_fn 
nbd_co_send_sparse_read(NBDClient *client,
 return ret;
 }

-static int blockstatus_to_extent_be(BlockDriverState *bs, uint64_t offset,
-uint64_t bytes, NBDExtent *extent)
+/*
+ * Populate @extents from block status. Update @bytes to be the actual
+ * length encoded (which may be smaller than the original), and update
+ * @nb_extents to the number of extents used.
+ *
+ * Returns zero on success and -errno on bdrv_block_status_above failure.
+ */
+static int blockstatus_to_extents(BlockDriverState *bs, uint64_t offset,
+  uint64_t *bytes, NBDExtent *extents,
+  unsigned int *nb_extents)
 {
-uint64_t remaining_bytes = bytes;
+uint64_t remaining_bytes = *bytes;
+NBDExtent *extent = extents, *extents_end = extents + *nb_extents;
+bool first_extent = true;

+assert(*nb_extents);
 while (remaining_bytes) {
 uint32_t flags;
 int64_t num;
 int ret = bdrv_block_status_above(bs, NULL, offset, remaining_bytes,
   , NULL, NULL);
+
 if (ret < 0) {
 return ret;
 }

 flags = (ret & BDRV_BLOCK_ALLOCATED ? 0 : NBD_STATE_HOLE) |
 (ret & BDRV_BLOCK_ZERO  ? NBD_STATE_ZERO : 0);
-
-if (remaining_bytes == bytes) {
-extent->flags = flags;
-}
-
-if (flags != extent->flags) {
-break;
-}
-
 offset += num;
 remaining_bytes -= num;
+
+if (first_extent) {
+extent->flags = flags;
+extent->length = num;
+first_extent = false;
+continue;
+}
+
+if (flags == extent->flags) {
+/* extend current extent */
+extent->length += num;
+} else {
+if (extent + 1 == extents_end) {
+break;
+}
+
+/* start new extent */
+extent++;
+extent->flags = flags;
+extent->length = num;
+}
+}
+
+extents_end = extent + 1;
+
+for (extent = extents; extent < extents_end; extent++) {
+cpu_to_be32s(>flags);
+cpu_to_be32s(>length);
 }

-cpu_to_be32s(>flags);
-extent->length = cpu_to_be32(bytes - remaining_bytes);
+*bytes -= remaining_bytes;
+*nb_extents = extents_end - extents;

 return 0;
 }
@@ -1910,21 +1941,29 @@ static int nbd_co_send_extents(NBDClient *client, 
uint64_t handle,
 /* Get block status from the exported device and send it to the client */
 static int nbd_co_send_block_status(NBDClient *client, uint64_t handle,
 BlockDriverState *bs, uint64_t offset,
-uint32_t length, bool last,
-uint32_t context_id, Error **errp)
+uint32_t length, bool dont_fragment,
+bool last, uint32_t context_id,
+Error **errp)
 {
 int ret;
-NBDExtent extent;
+unsigned int nb_extents = dont_fragment ? 1 : NBD_MAX_BITMAP_EXTENTS;
+NBDExtent *extents = g_new(NBDExtent, nb_extents);
+uint64_t final_length = length;

-ret = blockstatus_to_extent_be(bs, offset, length, );
+ret = blockstatus_to_extents(bs, offset, _length, extents,
+ _extents);
 if (ret < 0) {
+g_free(extents);
 return nbd_co_send_structured_error(
 client, handle, -ret, "can't get block status", errp);
 }

-return nbd_co_send_extents(client, handle, , 1,
-   be32_to_cpu(extent.length), last,
-   context_id, errp);
+ret = nbd_co_send_extents(client, handle, extents, nb_extents,
+  final_length, last, context_id, errp);
+
+g_free(extents);
+
+return ret;
 }

 /*
@@ -2231,10 +2270,12 @@ static coroutine_fn int nbd_handle_request(NBDClient 
*client,
 (client->export_meta.base_allocation ||
  client->export_meta.bitmap))
 {
+bool dont_fragment = request->flags & NBD_CMD_FLAG_REQ_ONE;
+
 if (client->export_meta.base_allocation) {
 ret = 

[Qemu-devel] [PULL 2/3] qapi: bitmap-merge: document name change

[Eric Blake]

From: John Snow 

We named these using underscores instead of the preferred dash,
document this nearby so we cannot possibly forget to rectify this
when we remove the 'x-' prefixes when the feature becomes stable.

We do not implement the change ahead of time to avoid more work
for libvirt to do in order to figure out how to use the beta version
of the API needlessly.

Reported-by: Eric Blake 
Signed-off-by: John Snow 
Message-Id: <20180919190934.16284-1-js...@redhat.com>
Reviewed-by: Eric Blake 
[eblake: typo fix]
Signed-off-by: Eric Blake 
---
 qapi/block-core.json | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index ac3b48ee54e..58ec9931c7f 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1935,6 +1935,8 @@
 ##
 # @x-block-dirty-bitmap-merge:
 #
+# FIXME: Rename @src_name and @dst_name to src-name and dst-name.
+#
 # Merge @src_name dirty bitmap to @dst_name dirty bitmap. @src_name dirty
 # bitmap is unchanged. On error, @dst_name is unchanged.
 #
-- 
2.17.1

[Qemu-devel] [PULL 1/3] nbd/server: fix bitmap export

[Eric Blake]

From: Vladimir Sementsov-Ogievskiy 

bitmap_to_extents function is broken: it switches dirty variable after
every iteration, however it can process only part of dirty (or zero)
area during one iteration in case when this area is too large for one
extent.

Fortunately, the bug doesn't produce wrong extent flags: it just inserts
a zero-length extent between sequential extents representing large dirty
(or zero) area. However, zero-length extents are forbidden by the NBD
protocol. So, a careful client should consider such a reply as a server
fault, while a less-careful will likely ignore zero-length extents.

The bug can only be triggered by a client that requests block status
for nearly 4G at once (a request of 4G and larger is impossible per
the protocol, and requests smaller than 4G less the bitmap granularity
cause the loop to quit iterating rather than revisit the tail of the
large area); it also cannot trigger if the client used the
NBD_CMD_FLAG_REQ_ONE flag.  Since qemu 3.0 as client (using the
x-dirty-bitmap extension) always passes the flag, it is immune; and
we are not aware of other open-source clients that know how to request
qemu:dirty-bitmap:FOO contexts.  Clients that want to avoid the bug
could cap block status requests to a smaller length, such as 2G or 3G.

Fix this by more careful handling of dirty variable.

Bug was introduced in 3d068aff16
 "nbd/server: implement dirty bitmap export", with the whole function.
and is present in v3.0.0 release.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Message-Id: <20180914165116.23182-1-vsement...@virtuozzo.com>
CC: qemu-sta...@nongnu.org
Reviewed-by: Eric Blake 
[eblake: improved commit message]
Signed-off-by: Eric Blake 
---
 nbd/server.c | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/nbd/server.c b/nbd/server.c
index ea5fe0eb336..12f721482dd 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -1951,6 +1951,8 @@ static unsigned int bitmap_to_extents(BdrvDirtyBitmap 
*bitmap, uint64_t offset,

 assert(begin < overall_end && nb_extents);
 while (begin < overall_end && i < nb_extents) {
+bool next_dirty = !dirty;
+
 if (dirty) {
 end = bdrv_dirty_bitmap_next_zero(bitmap, begin);
 } else {
@@ -1962,6 +1964,7 @@ static unsigned int bitmap_to_extents(BdrvDirtyBitmap 
*bitmap, uint64_t offset,
 end = MIN(bdrv_dirty_bitmap_size(bitmap),
   begin + UINT32_MAX + 1 -
   bdrv_dirty_bitmap_granularity(bitmap));
+next_dirty = dirty;
 }
 if (dont_fragment && end > overall_end) {
 end = overall_end;
@@ -1971,7 +1974,7 @@ static unsigned int bitmap_to_extents(BdrvDirtyBitmap 
*bitmap, uint64_t offset,
 extents[i].flags = cpu_to_be32(dirty ? NBD_STATE_DIRTY : 0);
 i++;
 begin = end;
-dirty = !dirty;
+dirty = next_dirty;
 }

 bdrv_dirty_iter_free(it);
-- 
2.17.1

[Qemu-devel] [PULL 0/3] NBD patches through 2018-09-26

[Eric Blake]

The following changes since commit c5e4e49258e9b89cb34c085a419dd9f862935c48:

  Merge remote-tracking branch 'remotes/xanclic/tags/pull-block-2018-09-25' 
into staging (2018-09-25 16:47:35 +0100)

are available in the Git repository at:

  git://repo.or.cz/qemu/ericb.git tags/pull-nbd-2018-09-26

for you to fetch changes up to fb7afc797e071f2616e1ccc849b39fe43e7033bf:

  nbd/server: send more than one extent of base:allocation context (2018-09-26 
21:37:48 -0500)

Intentionally not included but still probable 3.1 material:
- John: dirty-bitmaps cleanups - while related to NBD fleecing, John's tree is 
better suited for that
- Vladimir: v4 00/10 NBD reconnect - I still haven't had time for a proper 
review
- Eric: NBD fixes for unaligned images - I still have corner cases that require 
a v2
- Vladimir: nbd oldstyle vs. newstyle negotiation - still need a patch written 
to flip the default
- Eric: qemu-img dd improvements - useful in testing nbd, but needs a v2 
written that improves 'qemu-img convert' in general to do subrange work
- Eric?: need a way to cause NBD to give EIO when reading outside dirty bitmap 
area of fleeced image, so that we can then avoid copy-on-write to the fleeced 
image for portions that aren't dirty for less local storage pressure. So far 
just an idea with no patches


nbd patches for 2018-09-26

Fixes for external clients; add reminder to revisit naming of x- command

- Vladimir Sementsov-Ogievskiy: nbd/server: send more than one extent of 
base:allocation context
- John Snow: qapi: bitmap-merge: document name change
- Vladimir Sementsov-Ogievskiy: nbd/server: fix bitmap export


John Snow (1):
  qapi: bitmap-merge: document name change

Vladimir Sementsov-Ogievskiy (2):
  nbd/server: fix bitmap export
  nbd/server: send more than one extent of base:allocation context

 qapi/block-core.json |  2 ++
 nbd/server.c | 92 ++--
 2 files changed, 70 insertions(+), 24 deletions(-)

-- 
2.17.1

Re: [Qemu-devel] [PATCH] dirty-bitmaps: allow merging to disabled bitmaps

[Eric Blake]

On 9/19/18 4:16 PM, John Snow wrote:



On 09/19/2018 05:08 PM, Eric Blake wrote:

On 9/19/18 2:58 PM, John Snow wrote:

We wish to prohibit merging to read-only bitmaps and frozen bitmaps,
but "disabled" bitmaps only preclude their recording of live, new
information. It does not prohibit them from manual writes at the behest
of the user, as is the case for merge operations.

Reported-by: Eric Blake 
Signed-off-by: John Snow 
---
   block/dirty-bitmap.c | 2 +-
   1 file changed, 1 insertion(+), 1 deletion(-)


I can add this to my NBD queue, if desired.



If that makes life easier for you, then please be my guest -- otherwise
there's no conflict in taking it through the bitmaps tree I've got.


At this point, I'm not queuing this through NBD, as I think it is better 
served by just folding this directly into your v3 permissions series (or 
does that need to go up to v4?)




Just let me know either way.



--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH v3 0/5] dirty-bitmaps: fix QMP command permissions

[Eric Blake]

On 9/25/18 6:49 PM, John Snow wrote:

based on: jsnow/bitmaps staging branch

This series builds on a previous standalone patch and adjusts
the permission for all (or most) of the QMP bitmap commands.

John Snow (5):
   block/dirty-bitmaps: add user_modifiable status checker
   block/dirty-bitmaps: fix merge permissions
   block/dirty-bitmaps: allow clear on disabled bitmaps
   block/dirty-bitmaps: prohibit enable/disable on locked/frozen bitmaps
   block/backup: prohibit backup from using in-use bitmaps

  block/dirty-bitmap.c | 13 +---
  blockdev.c   | 75 
  include/block/dirty-bitmap.h |  1 +
  3 files changed, 44 insertions(+), 45 deletions(-)


Should there be any testsuite coverage of these changes?

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH v3 1/5] block/dirty-bitmaps: add user_modifiable status checker

[Eric Blake]

On 9/26/18 7:19 AM, Vladimir Sementsov-Ogievskiy wrote:

26.09.2018 14:53, Vladimir Sementsov-Ogievskiy wrote:

26.09.2018 02:49, John Snow wrote:

Instead of both frozen and qmp_locked checks, wrap it into one check.
frozen implies the bitmap is split in two (for backup), and shouldn't
be modified. qmp_locked implies it's being used by another operation,
like being exported over NBD. In both cases it means we shouldn't allow
the user to modify it in any meaningful way.

Replace any usages where we check both frozen and qmp_locked with the
new check.



to reduce number of '!', we may use opposite check, for ex 
"bdrv_dirty_bitmap_user_locked".


bad reason to rewrite the whole series, so, ignore this comment)


Then again, if you change 'in-use' to 'in use' in the entire series, you 
might as well flip the logic (double-negative logic is slightly harder 
to follow than suitably-named positive logic).


At this point, I have not queued this series on my NBD queue, because 
there may indeed be a reason for v4 to touch up spelling and logic.


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH v3 1/5] block/dirty-bitmaps: add user_modifiable status checker

[Eric Blake]

On 9/26/18 6:53 AM, Vladimir Sementsov-Ogievskiy wrote:

26.09.2018 02:49, John Snow wrote:

Instead of both frozen and qmp_locked checks, wrap it into one check.
frozen implies the bitmap is split in two (for backup), and shouldn't
be modified. qmp_locked implies it's being used by another operation,
like being exported over NBD. In both cases it means we shouldn't allow
the user to modify it in any meaningful way.

Replace any usages where we check both frozen and qmp_locked with the
new check.

Signed-off-by: John Snow 
---
  block/dirty-bitmap.c |  6 ++
  blockdev.c   | 29 -
  include/block/dirty-bitmap.h |  1 +
  3 files changed, 15 insertions(+), 21 deletions(-)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 8ac933cf1c..fc10543ab0 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -176,6 +176,12 @@ bool bdrv_dirty_bitmap_frozen(BdrvDirtyBitmap 
*bitmap)

  return bitmap->successor;
  }
+/* Both conditions disallow user-modification via QMP. */
+bool bdrv_dirty_bitmap_user_modifiable(BdrvDirtyBitmap *bitmap) {
+    return !(bdrv_dirty_bitmap_frozen(bitmap) ||
+ bdrv_dirty_bitmap_qmp_locked(bitmap));
+}


to reduce number of '!', we may use opposite check, for ex 
"bdrv_dirty_bitmap_user_locked".


Meaning make this function return true if locked for one less negation 
in the function body...




anyway,
Reviewed-by: Vladimir Sementsov-Ogievskiy 




+++ b/blockdev.c
@@ -2009,11 +2009,8 @@ static void 
block_dirty_bitmap_clear_prepare(BlkActionState *common,

  return;
  }
-    if (bdrv_dirty_bitmap_frozen(state->bitmap)) {
-    error_setg(errp, "Cannot modify a frozen bitmap");
-    return;
-    } else if (bdrv_dirty_bitmap_qmp_locked(state->bitmap)) {
-    error_setg(errp, "Cannot modify a locked bitmap");
+    if (!bdrv_dirty_bitmap_user_modifiable(state->bitmap)) {
+    error_setg(errp, "Cannot modify a bitmap in-use by another 
operation");

  return;


...and since most callers were negating sense as well?

I'm not sure I'm a fan of "in-use" with the hyphen. It sounds better to 
me to just spell it out as two words. (multiple instances)


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH 2/2] nbd/server: send more than one extent of base:allocation context

[Eric Blake]

On 9/17/18 11:15 AM, Vladimir Sementsov-Ogievskiy wrote:

05.07.2018 19:18, Vladimir Sementsov-Ogievskiy wrote:

05.07.2018 18:59, Eric Blake wrote:

On 07/04/2018 06:23 AM, Vladimir Sementsov-Ogievskiy wrote:

This is necessary for efficient block-status export, for clients which
support it.






This is a feature, so it is indeed 3.1 material.



Reviewed-by: Eric Blake 



Ok, thank you!



hmm, looks like lost patch :(


Not lost, just extremely slow getting queued.  Now on my NBD queue.

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [RFC] [PATCH 1/3] arm: API to check if the register is invariant

[mjaggi]

From: Manish Jaggi 

kvm_arm_is_invariant is added to return true if the register
is invariant. This patch also adds an array invariant_sys_regs
which is lookedup for invaraint register ids.

Currently this patch checks for only MIDR invaraint register.

Signed-off-by: Manish Jaggi 

diff --git a/target/arm/kvm32.c b/target/arm/kvm32.c
index 4e91c11..35a0c38 100644
--- a/target/arm/kvm32.c
+++ b/target/arm/kvm32.c
@@ -220,6 +220,11 @@ int kvm_arch_init_vcpu(CPUState *cs)
 return kvm_arm_init_cpreg_list(cpu);
 }
 
+bool kvm_arm_is_invariant(struct kvm_one_reg *r)
+{
+return false;
+}
+
 typedef struct Reg {
 uint64_t id;
 int offset;
diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c
index e0b8246..edaf2de 100644
--- a/target/arm/kvm64.c
+++ b/target/arm/kvm64.c
@@ -491,6 +491,7 @@ bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
 return true;
 }
 
+#define ARM_CPU_ID_MIDR   3, 0, 0, 0, 0
 #define ARM_CPU_ID_MPIDR   3, 0, 0, 0, 5
 
 int kvm_arch_init_vcpu(CPUState *cs)
@@ -549,6 +550,21 @@ int kvm_arch_init_vcpu(CPUState *cs)
 return kvm_arm_init_cpreg_list(cpu);
 }
 
+static uint64_t invariant_sys_regs[] = {
+   ARM64_SYS_REG(ARM_CPU_ID_MIDR),
+};
+
+bool kvm_arm_is_invariant(struct kvm_one_reg *r)
+{
+int i;
+
+for (i = 0; i < ARRAY_SIZE(invariant_sys_regs); i++)
+if (invariant_sys_regs[i] == r->id)
+return true;
+
+return false;
+}
+
 bool kvm_arm_reg_syncs_via_cpreg_list(uint64_t regidx)
 {
 /* Return true if the regidx is a register we should synchronize
diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
index 863f205..a834f60 100644
--- a/target/arm/kvm_arm.h
+++ b/target/arm/kvm_arm.h
@@ -61,6 +61,13 @@ void kvm_arm_register_device(MemoryRegion *mr, uint64_t 
devid, uint64_t group,
 int kvm_arm_init_cpreg_list(ARMCPU *cpu);
 
 /**
+ * kvm_arm_is_invariant
+ *
+ * Returns: true if r is invariant
+ */ 
+bool kvm_arm_is_invariant(struct kvm_one_reg *r);
+
+/**
  * kvm_arm_reg_syncs_via_cpreg_list
  * regidx: KVM register index
  *
-- 
1.8.3.1

[Qemu-devel] [RFC] [PATCH 3/3] arm: Skip invariant register restore

[mjaggi]

From: Manish Jaggi 

Invariant registers will be skipped from being restored from
guests' context on migrated host.

Signed-off-by: Manish Jaggi 

diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index 65f867d..2d89600 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -451,6 +451,9 @@ bool write_list_to_kvmstate(ARMCPU *cpu, int level)
 default:
 abort();
 }
+if (skip_invariant && kvm_arm_is_invariant()) {
+continue;
+}
 ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, );
 if (ret) {
 /* We might fail for "unknown register" and also for
-- 
1.8.3.1

[Qemu-devel] [RFC] [PATCH 2/3] arm: Introduce skipinvariant command line option

[mjaggi]

From: Manish Jaggi 

This option along with the API kvm_arm_is_invariant will
prevent any invariant registers written to KVM.
This option is handy when doing migration between arm hosts
which use -cpu host option, and are similar in CPU family but
only differ in MIDR value or other invariant registers values.

Signed-off-by: Manish Jaggi 

diff --git a/qemu-options.hx b/qemu-options.hx
index 654ef48..d187f6a 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -4414,3 +4414,14 @@ HXCOMM This is the last statement. Insert new options 
before this line!
 STEXI
 @end table
 ETEXI
+
+DEF("skipinvariant", HAS_ARG, QEMU_OPTION_skipinvariant,
+"-skipinvariant enable[=on|off]\n"
+"migrated guest should use invariant register values of 
host\n"
+"on|off controls migration between arch64 systems using 
-cpu host but with different MIDR values (default:off)\n",
+QEMU_ARCH_ARM)
+STEXI
+@item -skipinvariant enable[=on|off]
+@findex -skipinvariant
+controls migration between arch64 systems using -cpu host but with different 
MIDR values.(default:off)
+ETEXI
diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
index a834f60..8a93b01 100644
--- a/target/arm/kvm_arm.h
+++ b/target/arm/kvm_arm.h
@@ -66,6 +66,7 @@ int kvm_arm_init_cpreg_list(ARMCPU *cpu);
  * Returns: true if r is invariant
  */ 
 bool kvm_arm_is_invariant(struct kvm_one_reg *r);
+extern bool skip_invariant;
 
 /**
  * kvm_arm_reg_syncs_via_cpreg_list
diff --git a/vl.c b/vl.c
index 5ba06ad..f24ca6b 100644
--- a/vl.c
+++ b/vl.c
@@ -144,6 +144,7 @@ const char *mem_path = NULL;
 int mem_prealloc = 0; /* force preallocation of physical target memory */
 bool enable_mlock = false;
 bool enable_cpu_pm = false;
+bool skip_invariant = false;
 int nb_nics;
 NICInfo nd_table[MAX_NICS];
 int autostart;
@@ -420,6 +421,18 @@ static QemuOptsList qemu_msg_opts = {
 },
 };
 
+static QemuOptsList qemu_skipinvariant_opts = {
+.name = "skipinvariant",
+.head = QTAILQ_HEAD_INITIALIZER(qemu_skipinvariant_opts.head),
+.desc = {
+{
+.name = "enable",
+.type = QEMU_OPT_BOOL,
+},
+{ /* end of list */ }
+},
+};
+
 static QemuOptsList qemu_name_opts = {
 .name = "name",
 .implied_opt_name = "guest",
@@ -2994,6 +3007,7 @@ int main(int argc, char **argv, char **envp)
 qemu_add_opts(_icount_opts);
 qemu_add_opts(_semihosting_config_opts);
 qemu_add_opts(_fw_cfg_opts);
+qemu_add_opts(_skipinvariant_opts);
 module_call_init(MODULE_INIT_OPTS);
 
 runstate_init();
@@ -3948,6 +3962,14 @@ int main(int argc, char **argv, char **envp)
 }
 configure_msg(opts);
 break;
+case QEMU_OPTION_skipinvariant:
+opts = 
qemu_opts_parse_noisily(qemu_find_opts("skipinvariant"), optarg,
+   false);
+if (!opts) {
+exit(1);
+}
+skip_invariant = qemu_opt_get_bool(opts, "skipinvariant", 
true);
+break;
 case QEMU_OPTION_dump_vmstate:
 if (vmstate_dump_file) {
 error_report("only one '-dump-vmstate' "
-- 
1.8.3.1

[Qemu-devel] [RFC] [PATCH 0/3] qemu: arm: Migration between machines with different MIDR values

[mjaggi]

From: Manish Jaggi 

QEMU on arm systems use -machine virt -cpu host option for a VM.
Migration thus is limited between machines with same cpu.

This is a limitation if migration is desired between cpus which are of same
family and have only few diferences like bug fixes which have no effect on
VM operation. They just differ in say MIDR values.

This patchset introduces a command line option -skipinvariant. Invariant
registers will be skipped from being restored from guests context on migrated
host.

Mailing list discussion on this topic: 
https://www.mail-archive.com/qemu-devel@nongnu.org/msg560043.html

Manish Jaggi (3):
  arm: API to check if the register is invariant
  arm: Introduce skipinvariant command line option
  arm: Skip invariant register restore

 qemu-options.hx  | 11 +++
 target/arm/kvm.c |  3 +++
 target/arm/kvm32.c   |  5 +
 target/arm/kvm64.c   | 16 
 target/arm/kvm_arm.h |  8 
 vl.c | 22 ++
 6 files changed, 65 insertions(+)

-- 
1.8.3.1

Re: [Qemu-devel] [PATCH v4 01/24] memory-device: fix alignment error message

[David Gibson]

On Wed, Sep 26, 2018 at 11:41:56AM +0200, David Hildenbrand wrote:
> We're missing "x" after the leading 0.
> 
> Signed-off-by: David Hildenbrand 

Reviewed-by: David Gibson 

> ---
>  hw/mem/memory-device.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/mem/memory-device.c b/hw/mem/memory-device.c
> index 6de4f70bb4..0b52fe2c5e 100644
> --- a/hw/mem/memory-device.c
> +++ b/hw/mem/memory-device.c
> @@ -120,7 +120,7 @@ uint64_t memory_device_get_free_addr(MachineState *ms, 
> const uint64_t *hint,
>  
>  /* address_space_start indicates the maximum alignment we expect */
>  if (QEMU_ALIGN_UP(address_space_start, align) != address_space_start) {
> -error_setg(errp, "the alignment (0%" PRIx64 ") is not supported",
> +error_setg(errp, "the alignment (0x%" PRIx64 ") is not supported",
> align);
>  return 0;
>  }

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

[Qemu-devel] [PATCH 2/3] memory: Refactor common shifting code from accessors

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé 
---
 memory.c | 30 +-
 1 file changed, 21 insertions(+), 9 deletions(-)

diff --git a/memory.c b/memory.c
index 385b9d3590..48edf7dc23 100644
--- a/memory.c
+++ b/memory.c
@@ -374,6 +374,21 @@ static void adjust_endianness(MemoryRegion *mr, uint64_t 
*data, unsigned size)
 }
 }
 
+static inline void memory_region_shift_read_access(uint64_t *value,
+   unsigned shift,
+   uint64_t mask,
+   uint64_t tmp)
+{
+*value |= (tmp & mask) << shift;
+}
+
+static inline uint64_t memory_region_shift_write_access(uint64_t *value,
+unsigned shift,
+uint64_t mask)
+{
+return (*value >> shift) & mask;
+}
+
 static hwaddr memory_region_to_absolute_addr(MemoryRegion *mr, hwaddr offset)
 {
 MemoryRegion *root;
@@ -418,7 +433,7 @@ static MemTxResult 
memory_region_oldmmio_read_accessor(MemoryRegion *mr,
 hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
 trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size);
 }
-*value |= (tmp & mask) << shift;
+memory_region_shift_read_access(value, shift, mask, tmp);
 return MEMTX_OK;
 }
 
@@ -444,7 +459,7 @@ static MemTxResult  
memory_region_read_accessor(MemoryRegion *mr,
 hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
 trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size);
 }
-*value |= (tmp & mask) << shift;
+memory_region_shift_read_access(value, shift, mask, tmp);
 return MEMTX_OK;
 }
 
@@ -471,7 +486,7 @@ static MemTxResult 
memory_region_read_with_attrs_accessor(MemoryRegion *mr,
 hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr);
 trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size);
 }
-*value |= (tmp & mask) << shift;
+memory_region_shift_read_access(value, shift, mask, tmp);
 return r;
 }
 
@@ -483,9 +498,8 @@ static MemTxResult 
memory_region_oldmmio_write_accessor(MemoryRegion *mr,
 uint64_t mask,
 MemTxAttrs attrs)
 {
-uint64_t tmp;
+uint64_t tmp = memory_region_shift_write_access(value, shift, mask);
 
-tmp = (*value >> shift) & mask;
 if (mr->subpage) {
 trace_memory_region_subpage_write(get_cpu_index(), mr, addr, tmp, 
size);
 } else if (mr == _mem_notdirty) {
@@ -509,9 +523,8 @@ static MemTxResult 
memory_region_write_accessor(MemoryRegion *mr,
 uint64_t mask,
 MemTxAttrs attrs)
 {
-uint64_t tmp;
+uint64_t tmp = memory_region_shift_write_access(value, shift, mask);
 
-tmp = (*value >> shift) & mask;
 if (mr->subpage) {
 trace_memory_region_subpage_write(get_cpu_index(), mr, addr, tmp, 
size);
 } else if (mr == _mem_notdirty) {
@@ -535,9 +548,8 @@ static MemTxResult 
memory_region_write_with_attrs_accessor(MemoryRegion *mr,
uint64_t mask,
MemTxAttrs attrs)
 {
-uint64_t tmp;
+uint64_t tmp = memory_region_shift_write_access(value, shift, mask);
 
-tmp = (*value >> shift) & mask;
 if (mr->subpage) {
 trace_memory_region_subpage_write(get_cpu_index(), mr, addr, tmp, 
size);
 } else if (mr == _mem_notdirty) {
-- 
2.19.0

[Qemu-devel] [PATCH 0/3] Fix access_with_adjusted_size() on big-endian

[Philippe Mathieu-Daudé]

Hi,

This series fix a bug I'v been hunting for a long time.

With BE regions, if the guest used smaller access than the region
implementation, the shift value is negative, but since access_fn()
uses unsigned type for shift, it result in a huge positive value,
then accessors shift the value which eventually becomes 0.

The fix is simply to use signed type for the shift, and shift to
the opposite direction for negative values.

Regards,

Phil.

Philippe Mathieu-Daudé (3):
  memory: Use MAKE_64BIT_MASK()
  memory: Refactor common shifting code from accessors
  memory: Fix access_with_adjusted_size(small size) on big-endian memory
regions

 memory.c | 58 +++-
 1 file changed, 41 insertions(+), 17 deletions(-)

-- 
2.19.0

[Qemu-devel] [PATCH 1/3] memory: Use MAKE_64BIT_MASK()

[Philippe Mathieu-Daudé]

Suggested-by: Paolo Bonzini 
Signed-off-by: Philippe Mathieu-Daudé 
---
 memory.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/memory.c b/memory.c
index 9b73892768..385b9d3590 100644
--- a/memory.c
+++ b/memory.c
@@ -582,7 +582,7 @@ static MemTxResult access_with_adjusted_size(hwaddr addr,
 
 /* FIXME: support unaligned access? */
 access_size = MAX(MIN(size, access_size_max), access_size_min);
-access_mask = -1ULL >> (64 - access_size * 8);
+access_mask = MAKE_64BIT_MASK(0, access_size * 8);
 if (memory_region_big_endian(mr)) {
 for (i = 0; i < size; i += access_size) {
 r |= access_fn(mr, addr + i, value, access_size,
-- 
2.19.0

[Qemu-devel] [PATCH 3/3] memory: Fix access_with_adjusted_size(small size) on big-endian memory regions

[Philippe Mathieu-Daudé]

Memory regions configured as DEVICE_BIG_ENDIAN (or DEVICE_NATIVE_ENDIAN on
big-endian guest) behave incorrectly when the memory access 'size' is smaller
than the implementation 'access_size'.

In the following code segment from access_with_adjusted_size():

if (memory_region_big_endian(mr)) {
for (i = 0; i < size; i += access_size) {
r |= access_fn(mr, addr + i, value, access_size,
(size - access_size - i) * 8, access_mask, attrs);
}

(size - access_size - i) * 8 is the number of bits that will arithmetic
shift the current value.

Currently we can only 'left' shift a read() access, and 'right' shift a write().

When the access 'size' is smaller than the implementation, we get a negative
number of bits to shift.

For the read() case, a negative 'left' shift is a 'right' shift :)
However since the 'shift' type is unsigned, there is currently no way to
right shift.

Fix this by changing the access_fn() prototype to handle signed shift values,
and modify the memory_region_shift_read|write_access() helpers to correctly
arithmetic shift the opposite direction when the 'shift' value is negative.

Signed-off-by: Philippe Mathieu-Daudé 
---
 memory.c | 34 +++---
 1 file changed, 23 insertions(+), 11 deletions(-)

diff --git a/memory.c b/memory.c
index 48edf7dc23..a9f0fdc440 100644
--- a/memory.c
+++ b/memory.c
@@ -375,18 +375,30 @@ static void adjust_endianness(MemoryRegion *mr, uint64_t 
*data, unsigned size)
 }
 
 static inline void memory_region_shift_read_access(uint64_t *value,
-   unsigned shift,
+   signed shift,
uint64_t mask,
uint64_t tmp)
 {
-*value |= (tmp & mask) << shift;
+if (shift >= 0) {
+*value |= (tmp & mask) << shift;
+} else {
+*value |= (tmp & mask) >> -shift;
+}
 }
 
 static inline uint64_t memory_region_shift_write_access(uint64_t *value,
-unsigned shift,
+signed shift,
 uint64_t mask)
 {
-return (*value >> shift) & mask;
+uint64_t tmp;
+
+if (shift >= 0) {
+tmp = (*value >> shift) & mask;
+} else {
+tmp = (*value << -shift) & mask;
+}
+
+return tmp;
 }
 
 static hwaddr memory_region_to_absolute_addr(MemoryRegion *mr, hwaddr offset)
@@ -415,7 +427,7 @@ static MemTxResult 
memory_region_oldmmio_read_accessor(MemoryRegion *mr,
hwaddr addr,
uint64_t *value,
unsigned size,
-   unsigned shift,
+   signed shift,
uint64_t mask,
MemTxAttrs attrs)
 {
@@ -441,7 +453,7 @@ static MemTxResult  
memory_region_read_accessor(MemoryRegion *mr,
 hwaddr addr,
 uint64_t *value,
 unsigned size,
-unsigned shift,
+signed shift,
 uint64_t mask,
 MemTxAttrs attrs)
 {
@@ -467,7 +479,7 @@ static MemTxResult 
memory_region_read_with_attrs_accessor(MemoryRegion *mr,
   hwaddr addr,
   uint64_t *value,
   unsigned size,
-  unsigned shift,
+  signed shift,
   uint64_t mask,
   MemTxAttrs attrs)
 {
@@ -494,7 +506,7 @@ static MemTxResult 
memory_region_oldmmio_write_accessor(MemoryRegion *mr,
 hwaddr addr,
 uint64_t *value,
 unsigned size,
-unsigned shift,
+signed shift,
 uint64_t mask,
 MemTxAttrs attrs)
 {
@@ -519,7 +531,7 @@ static 

Re: [Qemu-devel] [PATCH] Add "boot_linux" acceptance test

[Philippe Mathieu-Daudé]

On 9/27/18 1:05 AM, Cleber Rosa wrote:
> On 9/26/18 2:27 PM, Philippe Mathieu-Daudé wrote:
>> Hi Cleber,
>>
>> On Thu, Sep 20, 2018 at 6:18 PM Cleber Rosa  wrote:
>>>
>>> This acceptance test, validates that a full blown Linux guest can
>>> successfully boot in QEMU.  In this specific case, the guest
>>> chosen is Fedora version 28.  By passing parameters, the same
>>> test can attempt to boot different distros, arches, etc.
>>>
>>> The method for checking the successfull boot is based on "cloudinit"
>>> and its "phone home" feature.  The guest is given an ISO image
>>> with the location of the phone home server, and the information to
>>> post (the instance ID).  Upon receiving the correct information,
>>> from the guest, the test is considered to have PASSed.
>>>
>>> This test is currently limited to user mode networking only, and
>>> instructs the guest to connect to the "router" address that is hard
>>> coded in QEMU.
>>>
>>> This test requires features present in Avocado version 64.0, and when
>>> running under Python 3, requires a fix to the avocado.utils.vmimage
>>> library (to be included in version 65.0).  To create the cloudinit ISO
>>> image that will be used to configure the guest, the pycdlib library is
>>> also required.  The idea for a effortless execution of this test, is
>>> to set those requirements, that is:
>>>
>>>avocado-framework==65.0
>>>pycdlib==1.6.0
>>>
>>> In the "tests/venv-requirements.txt" file introduced in another patch
>>> series.
>>>
>>> Reference: 
>>> https://lists.gnu.org/archive/html/qemu-devel/2018-09/msg02503.html
>>> Reference: 
>>> https://github.com/avocado-framework/avocado/commit/02c47b1eade667d18fb0adef3293d86a6b5fd2e9
>>> Signed-off-by: Cleber Rosa 
>>> ---
>>>  tests/acceptance/boot_linux.py | 52 ++
>>>  1 file changed, 52 insertions(+)
>>>  create mode 100644 tests/acceptance/boot_linux.py
>>>
>>> diff --git a/tests/acceptance/boot_linux.py b/tests/acceptance/boot_linux.py
>>> new file mode 100644
>>> index 00..658211f15f
>>> --- /dev/null
>>> +++ b/tests/acceptance/boot_linux.py
>>> @@ -0,0 +1,52 @@
>>> +# Functional test that boots a complete Linux system via a cloud image
>>> +#
>>> +# Copyright (c) 2018 Red Hat, Inc.
>>> +#
>>> +# Author:
>>> +#  Cleber Rosa 
>>> +#
>>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>>> +# later.  See the COPYING file in the top-level directory.
>>> +
>>> +import os
>>> +
>>> +from avocado_qemu import Test
>>> +
>>> +from avocado.utils import cloudinit
>>> +from avocado.utils import network
>>> +from avocado.utils import vmimage
>>> +
>>> +
>>> +class BootLinux(Test):
>>> +"""
>>> +Boots a Linux system, checking for a successful initialization
>>> +
>>> +:avocado: enable
>>> +"""
>>> +
>>> +timeout = 600
>>> +
>>> +def test(self):
>>> +self.vm.set_machine(self.params.get('machine', default='pc'))
>>> +self.vm.add_args('-accel', self.params.get('accel', default='kvm'))
>>> +self.vm.add_args('-smp', self.params.get('smp', default='2'))
>>> +self.vm.add_args('-m', self.params.get('memory', default='4096'))
>>> +
>>> +arch = self.params.get('arch', default=os.uname()[4])
>>> +distro = self.params.get('distro', default='fedora')
>>> +version = self.params.get('version', default='28')
>>> +boot = vmimage.get(distro, arch=arch, version=version,
>>> +   cache_dir=self.cache_dirs[0],
>>> +   snapshot_dir=self.workdir)
>>> +self.vm.add_args('-drive', 'file=%s' % boot.path)
>>> +
>>> +cloudinit_iso = os.path.join(self.workdir, 'cloudinit.iso')
>>> +phone_home_port = network.find_free_port()
>>> +cloudinit.iso(cloudinit_iso, self.name,
>>> +  # QEMU's hard coded usermode router address
>>> +  phone_home_host='10.0.2.2',
>>> +  phone_home_port=phone_home_port)
>>> +self.vm.add_args('-drive', 'file=%s' % cloudinit_iso)
>>> +
>>> +self.vm.launch()
>>> +cloudinit.wait_for_phone_home(('0.0.0.0', phone_home_port), 
>>> self.name)
>>> --
>>> 2.17.1
>>>
>>
>> Using:
>>
>> (venv) $ avocado run tests/acceptance
>>
>> I'm getting:
>>
>> DEBUG| DATA (filename=output.expected) => NOT FOUND (data sources:
>> variant, test, file)
>> DEBUG| PARAMS (key=qemu_bin, path=*,
>> default=x86_64-softmmu/qemu-system-x86_64) =>
>> 'x86_64-softmmu/qemu-system-x86_64'
>> DEBUG| PARAMS (key=machine, path=*, default=pc) => 'pc'
>> DEBUG| PARAMS (key=accel, path=*, default=kvm) => 'kvm'
>> DEBUG| PARAMS (key=smp, path=*, default=2) => '2'
>> DEBUG| PARAMS (key=memory, path=*, default=4096) => '4096'
>> DEBUG| PARAMS (key=arch, path=*, default=x86_64) => 'x86_64'
>> DEBUG| PARAMS (key=distro, path=*, default=fedora) => 'fedora'
>> DEBUG| PARAMS (key=version, path=*, default=28) => '28'
>> ERROR|
>> ERROR| Reproduced traceback from:
>> 

[Qemu-devel] [PATCH v9 2/6] s390x/cpumodel: Set up CPU model for AP device support

[Tony Krowiak]

A new CPU model feature and two new CPU model facilities are
introduced to support AP devices for a KVM guest.

CPU model features:

1. The S390_FEAT_AP CPU model feature indicates whether AP
   instructions are available to the guest. This feature will
   be enabled only if the AP instructions are available on the
   linux host as determined by the availability of the
   KVM_S390_VM_CRYPTO_ENABLE_APIE VM attribute which is exposed
   by KVM only if the AP instructions are available on the
   host.

   This feature must be turned on from userspace to execute AP
   instructions on the KVM guest. The QEMU command line to turn
   this feature on looks something like this:

qemu-system-s390x ... -cpu xxx,ap=on ...

   This feature will be supported for zEC12 and newer CPU models.
   The feature will not be supported for older models because
   there are few older systems on which to test and the older
   crypto cards will be going out of service in the relatively
   near future.

CPU model facilities:

1. The S390_FEAT_AP_QUERY_CONFIG_INFO feature indicates whether the
   AP Query Configuration Information (QCI) facility is available
   to the guest as determined by whether the facility is available
   on the host. This feature will be exposed by KVM only if the
   QCI facility is installed on the host.

2. The S390_FEAT_AP_FACILITY_TEST feature indicates whether the AP
   Facility Test (APFT) facility is available to the guest as
   determined by whether the facility is available on the host.
   This feature will be exposed by KVM only if APFT is installed
   on the host.

Signed-off-by: Tony Krowiak 
---
 target/s390x/cpu_features.c | 3 +++
 target/s390x/cpu_features_def.h | 3 +++
 target/s390x/cpu_models.c   | 2 ++
 target/s390x/gen-features.c | 3 +++
 target/s390x/kvm.c  | 5 +
 5 files changed, 16 insertions(+)

diff --git a/target/s390x/cpu_features.c b/target/s390x/cpu_features.c
index 172fb18df718..60cfeba48f4e 100644
--- a/target/s390x/cpu_features.c
+++ b/target/s390x/cpu_features.c
@@ -39,8 +39,10 @@ static const S390FeatDef s390_features[] = {
 FEAT_INIT("srs", S390_FEAT_TYPE_STFL, 9, "Sense-running-status facility"),
 FEAT_INIT("csske", S390_FEAT_TYPE_STFL, 10, "Conditional-SSKE facility"),
 FEAT_INIT("ctop", S390_FEAT_TYPE_STFL, 11, "Configuration-topology 
facility"),
+FEAT_INIT("apqci", S390_FEAT_TYPE_STFL, 12, "Query AP Configuration 
Information facility"),
 FEAT_INIT("ipter", S390_FEAT_TYPE_STFL, 13, "IPTE-range facility"),
 FEAT_INIT("nonqks", S390_FEAT_TYPE_STFL, 14, "Nonquiescing key-setting 
facility"),
+FEAT_INIT("apft", S390_FEAT_TYPE_STFL, 15, "AP Facilities Test facility"),
 FEAT_INIT("etf2", S390_FEAT_TYPE_STFL, 16, "Extended-translation facility 
2"),
 FEAT_INIT("msa-base", S390_FEAT_TYPE_STFL, 17, "Message-security-assist 
facility (excluding subfunctions)"),
 FEAT_INIT("ldisp", S390_FEAT_TYPE_STFL, 18, "Long-displacement facility"),
@@ -129,6 +131,7 @@ static const S390FeatDef s390_features[] = {
 
 FEAT_INIT_MISC("dateh2", "DAT-enhancement facility 2"),
 FEAT_INIT_MISC("cmm", "Collaborative-memory-management facility"),
+FEAT_INIT_MISC("ap", "AP instructions installed"),
 
 FEAT_INIT("plo-cl", S390_FEAT_TYPE_PLO, 0, "PLO Compare and load (32 bit 
in general registers)"),
 FEAT_INIT("plo-clg", S390_FEAT_TYPE_PLO, 1, "PLO Compare and load (64 bit 
in parameter list)"),
diff --git a/target/s390x/cpu_features_def.h b/target/s390x/cpu_features_def.h
index ac2c947f30a8..5fc7e7bf0116 100644
--- a/target/s390x/cpu_features_def.h
+++ b/target/s390x/cpu_features_def.h
@@ -27,8 +27,10 @@ typedef enum {
 S390_FEAT_SENSE_RUNNING_STATUS,
 S390_FEAT_CONDITIONAL_SSKE,
 S390_FEAT_CONFIGURATION_TOPOLOGY,
+S390_FEAT_AP_QUERY_CONFIG_INFO,
 S390_FEAT_IPTE_RANGE,
 S390_FEAT_NONQ_KEY_SETTING,
+S390_FEAT_AP_FACILITIES_TEST,
 S390_FEAT_EXTENDED_TRANSLATION_2,
 S390_FEAT_MSA,
 S390_FEAT_LONG_DISPLACEMENT,
@@ -119,6 +121,7 @@ typedef enum {
 /* Misc */
 S390_FEAT_DAT_ENH_2,
 S390_FEAT_CMM,
+S390_FEAT_AP,
 
 /* PLO */
 S390_FEAT_PLO_CL,
diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
index 265d25c937bb..7c253ff308c5 100644
--- a/target/s390x/cpu_models.c
+++ b/target/s390x/cpu_models.c
@@ -786,6 +786,8 @@ static void check_consistency(const S390CPUModel *model)
 { S390_FEAT_PRNO_TRNG_QRTCR, S390_FEAT_MSA_EXT_5 },
 { S390_FEAT_PRNO_TRNG, S390_FEAT_MSA_EXT_5 },
 { S390_FEAT_SIE_KSS, S390_FEAT_SIE_F2 },
+{ S390_FEAT_AP_QUERY_CONFIG_INFO, S390_FEAT_AP },
+{ S390_FEAT_AP_FACILITIES_TEST, S390_FEAT_AP },
 };
 int i;
 
diff --git a/target/s390x/gen-features.c b/target/s390x/gen-features.c
index 384b61cd67b9..70015eaaf5df 100644
--- a/target/s390x/gen-features.c
+++ b/target/s390x/gen-features.c
@@ -447,6 +447,9 @@ static uint16_t full_GEN12_GA1[] = {
 S390_FEAT_ADAPTER_INT_SUPPRESSION,
 

Re: [Qemu-devel] [PATCH] Add "boot_linux" acceptance test

[Cleber Rosa]

On 9/26/18 2:27 PM, Philippe Mathieu-Daudé wrote:
> Hi Cleber,
> 
> On Thu, Sep 20, 2018 at 6:18 PM Cleber Rosa  wrote:
>>
>> This acceptance test, validates that a full blown Linux guest can
>> successfully boot in QEMU.  In this specific case, the guest
>> chosen is Fedora version 28.  By passing parameters, the same
>> test can attempt to boot different distros, arches, etc.
>>
>> The method for checking the successfull boot is based on "cloudinit"
>> and its "phone home" feature.  The guest is given an ISO image
>> with the location of the phone home server, and the information to
>> post (the instance ID).  Upon receiving the correct information,
>> from the guest, the test is considered to have PASSed.
>>
>> This test is currently limited to user mode networking only, and
>> instructs the guest to connect to the "router" address that is hard
>> coded in QEMU.
>>
>> This test requires features present in Avocado version 64.0, and when
>> running under Python 3, requires a fix to the avocado.utils.vmimage
>> library (to be included in version 65.0).  To create the cloudinit ISO
>> image that will be used to configure the guest, the pycdlib library is
>> also required.  The idea for a effortless execution of this test, is
>> to set those requirements, that is:
>>
>>avocado-framework==65.0
>>pycdlib==1.6.0
>>
>> In the "tests/venv-requirements.txt" file introduced in another patch
>> series.
>>
>> Reference: 
>> https://lists.gnu.org/archive/html/qemu-devel/2018-09/msg02503.html
>> Reference: 
>> https://github.com/avocado-framework/avocado/commit/02c47b1eade667d18fb0adef3293d86a6b5fd2e9
>> Signed-off-by: Cleber Rosa 
>> ---
>>  tests/acceptance/boot_linux.py | 52 ++
>>  1 file changed, 52 insertions(+)
>>  create mode 100644 tests/acceptance/boot_linux.py
>>
>> diff --git a/tests/acceptance/boot_linux.py b/tests/acceptance/boot_linux.py
>> new file mode 100644
>> index 00..658211f15f
>> --- /dev/null
>> +++ b/tests/acceptance/boot_linux.py
>> @@ -0,0 +1,52 @@
>> +# Functional test that boots a complete Linux system via a cloud image
>> +#
>> +# Copyright (c) 2018 Red Hat, Inc.
>> +#
>> +# Author:
>> +#  Cleber Rosa 
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>> +# later.  See the COPYING file in the top-level directory.
>> +
>> +import os
>> +
>> +from avocado_qemu import Test
>> +
>> +from avocado.utils import cloudinit
>> +from avocado.utils import network
>> +from avocado.utils import vmimage
>> +
>> +
>> +class BootLinux(Test):
>> +"""
>> +Boots a Linux system, checking for a successful initialization
>> +
>> +:avocado: enable
>> +"""
>> +
>> +timeout = 600
>> +
>> +def test(self):
>> +self.vm.set_machine(self.params.get('machine', default='pc'))
>> +self.vm.add_args('-accel', self.params.get('accel', default='kvm'))
>> +self.vm.add_args('-smp', self.params.get('smp', default='2'))
>> +self.vm.add_args('-m', self.params.get('memory', default='4096'))
>> +
>> +arch = self.params.get('arch', default=os.uname()[4])
>> +distro = self.params.get('distro', default='fedora')
>> +version = self.params.get('version', default='28')
>> +boot = vmimage.get(distro, arch=arch, version=version,
>> +   cache_dir=self.cache_dirs[0],
>> +   snapshot_dir=self.workdir)
>> +self.vm.add_args('-drive', 'file=%s' % boot.path)
>> +
>> +cloudinit_iso = os.path.join(self.workdir, 'cloudinit.iso')
>> +phone_home_port = network.find_free_port()
>> +cloudinit.iso(cloudinit_iso, self.name,
>> +  # QEMU's hard coded usermode router address
>> +  phone_home_host='10.0.2.2',
>> +  phone_home_port=phone_home_port)
>> +self.vm.add_args('-drive', 'file=%s' % cloudinit_iso)
>> +
>> +self.vm.launch()
>> +cloudinit.wait_for_phone_home(('0.0.0.0', phone_home_port), 
>> self.name)
>> --
>> 2.17.1
>>
> 
> Using:
> 
> (venv) $ avocado run tests/acceptance
> 
> I'm getting:
> 
> DEBUG| DATA (filename=output.expected) => NOT FOUND (data sources:
> variant, test, file)
> DEBUG| PARAMS (key=qemu_bin, path=*,
> default=x86_64-softmmu/qemu-system-x86_64) =>
> 'x86_64-softmmu/qemu-system-x86_64'
> DEBUG| PARAMS (key=machine, path=*, default=pc) => 'pc'
> DEBUG| PARAMS (key=accel, path=*, default=kvm) => 'kvm'
> DEBUG| PARAMS (key=smp, path=*, default=2) => '2'
> DEBUG| PARAMS (key=memory, path=*, default=4096) => '4096'
> DEBUG| PARAMS (key=arch, path=*, default=x86_64) => 'x86_64'
> DEBUG| PARAMS (key=distro, path=*, default=fedora) => 'fedora'
> DEBUG| PARAMS (key=version, path=*, default=28) => '28'
> ERROR|
> ERROR| Reproduced traceback from:
> /build/tests/venv/lib64/python3.6/site-packages/avocado/core/test.py:831
> ERROR| Traceback (most recent call last):
> ERROR|   File 

[Qemu-devel] [PATCH v9 3/6] s390x/kvm: enable AP instruction interpretation for guest

[Tony Krowiak]

From: Tony Krowiak 

Let's use the KVM_SET_DEVICE_ATTR ioctl to enable hardware
interpretation of AP instructions executed on the guest.
If the S390_FEAT_AP feature is switched on for the guest,
AP instructions must be interpreted by default; otherwise,
they will be intercepted.

This attribute setting may be overridden by a device. For example,
a device may want to provide AP instructions to the guest (i.e.,
S390_FEAT_AP turned on), but it may want to emulate them. In this
case, the AP instructions executed on the guest must be
intercepted; so when the device is realized, it must disable
interpretation.

Signed-off-by: Tony Krowiak 
---
 target/s390x/kvm.c | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c
index 5277acd79a2c..d55d24abfd78 100644
--- a/target/s390x/kvm.c
+++ b/target/s390x/kvm.c
@@ -2301,6 +2301,16 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, 
Error **errp)
S390_FEAT_MAX);
 }
 
+static void kvm_s390_configure_apie(bool interpret)
+{
+uint64_t attr = interpret ? KVM_S390_VM_CRYPTO_ENABLE_APIE :
+KVM_S390_VM_CRYPTO_DISABLE_APIE;
+
+if (kvm_vm_check_attr(kvm_state, KVM_S390_VM_CRYPTO, attr)) {
+kvm_s390_set_attr(attr);
+}
+}
+
 void kvm_s390_apply_cpu_model(const S390CPUModel *model, Error **errp)
 {
 struct kvm_s390_vm_cpu_processor prop  = {
@@ -2350,6 +2360,10 @@ void kvm_s390_apply_cpu_model(const S390CPUModel *model, 
Error **errp)
 if (test_bit(S390_FEAT_CMM, model->features)) {
 kvm_s390_enable_cmma();
 }
+
+if (test_bit(S390_FEAT_AP, model->features)) {
+kvm_s390_configure_apie(true);
+}
 }
 
 void kvm_s390_restart_interrupt(S390CPU *cpu)
-- 
2.19.0.221.g150f307

[Qemu-devel] [PATCH v9 6/6] s390: doc: detailed specifications for AP virtualization

[Tony Krowiak]

This patch provides documentation describing the AP architecture and
design concepts behind the virtualization of AP devices. It also
includes an example of how to configure AP devices for exclusive
use of KVM guests.

Signed-off-by: Tony Krowiak 
---
 MAINTAINERS  |   1 +
 docs/vfio-ap.txt | 787 +++
 2 files changed, 788 insertions(+)
 create mode 100644 docs/vfio-ap.txt

diff --git a/MAINTAINERS b/MAINTAINERS
index 29041da69237..b64a12034c2c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1210,6 +1210,7 @@ F: hw/s390x/ap-bridge.c
 F: include/hw/s390x/ap-device.h
 F: include/hw/s390x/ap-bridge.h
 F: hw/vfio/ap.c
+F: docs/vfio-ap.txt
 L: qemu-s3...@nongnu.org
 
 vhost
diff --git a/docs/vfio-ap.txt b/docs/vfio-ap.txt
new file mode 100644
index ..fd17ce48967b
--- /dev/null
+++ b/docs/vfio-ap.txt
@@ -0,0 +1,787 @@
+Adjunct Processor (AP) Device
+=
+
+Contents:
+=
+* Introduction
+* AP Architectural Overview
+* Start Interpretive Execution (SIE) Instruction
+* AP Matrix Configuration on Linux Host
+* Starting a Linux Guest Configured with an AP Matrix
+* Example: Configure AP Matrices for Three Linux Guests
+
+Introduction:
+
+The IBM Adjunct Processor (AP) Cryptographic Facility is comprised
+of three AP instructions and from 1 to 256 PCIe cryptographic adapter cards.
+These AP devices provide cryptographic functions to all CPUs assigned to a
+linux system running in an IBM Z system LPAR.
+
+On s390x, AP adapter cards are exposed via the AP bus. This document
+describes how those cards may be made available to KVM guests using the
+VFIO mediated device framework.
+
+AP Architectural Overview:
+=
+In order understand the terminology used in the rest of this document, let's
+start with some definitions:
+
+* AP adapter
+
+  An AP adapter is an IBM Z adapter card that can perform cryptographic
+  functions. There can be from 0 to 256 adapters assigned to an LPAR; however,
+  the maximum adapter number allowed is determined by machine model. Adapters
+  assigned to the LPAR in which a linux host is running will be available to 
the
+  linux host. Each adapter is identified by a number from 0 to 255. When
+  installed, an AP adapter is accessed by AP instructions executed by any CPU.
+
+* AP domain
+
+  An adapter is partitioned into domains. Each domain can be thought of as
+  a set of hardware registers for processing AP instructions. An adapter can
+  hold up to 256 domains; however, the maximum domain number allowed is
+  determined by machine model. Each domain is identified by a number from 0 to
+  255. Domains can be further classified into two types:
+
+* Usage domains are domains that can be accessed directly to process AP
+  commands
+
+* Control domains are domains that are accessed indirectly by AP
+  commands sent to a usage domain to control or change the domain; for
+  example, to set a secure private key for the domain.
+
+* AP Queue
+
+  An AP queue is the means by which an AP command-request message is sent to an
+  AP usage domain inside a specific AP. An AP queue is identified by a tuple
+  comprised of an AP adapter ID (APID) and an AP queue index (APQI). The
+  APQI corresponds to a given usage domain number within the adapter. This 
tuple
+  forms an AP Queue Number (APQN) uniquely identifying an AP queue. AP
+  instructions include a field containing the APQN to identify the AP queue to
+  which the AP command-request message is to be sent for processing.
+
+* AP Instructions:
+
+  There are three AP instructions:
+
+  * NQAP: to enqueue an AP command-request message to a queue
+  * DQAP: to dequeue an AP command-reply message from a queue
+  * PQAP: to administer the queues
+
+  AP instructions identify the domain that is targeted to process the AP
+  command; this must be one of the usage domains. An AP command may modify a
+  domain that is not one of the usage domains, but the modified domain
+  must be one of the control domains.
+
+Start Interpretive Execution (SIE) Instruction
+==
+A KVM guest is started by executing the Start Interpretive Execution (SIE)
+instruction. The SIE state description is a control block that contains the
+state information for a KVM guest and is supplied as input to the SIE
+instruction. The SIE state description contains a satellite control block 
called
+the Crypto Control Block (CRYCB). The CRYCB contains three fields to identify
+the adapters, usage domains and control domains assigned to the KVM guest:
+
+* The AP Mask (APM) field is a bit mask that identifies the AP adapters 
assigned
+  to the KVM guest. Each bit in the mask, from most significant to least
+  significant bit, corresponds to an APID from 0-255. If a bit is set, the
+  corresponding adapter is valid for use by the KVM guest.
+
+* The AP Queue Mask (AQM) field is a bit mask identifying the AP 

[Qemu-devel] [PATCH v9 4/6] s390x/ap: base Adjunct Processor (AP) object model

[Tony Krowiak]

From: Tony Krowiak 

Introduces the base object model for virtualizing AP devices.

Signed-off-by: Tony Krowiak 
---
 MAINTAINERS  | 12 ++
 hw/s390x/Makefile.objs   |  2 +
 hw/s390x/ap-bridge.c | 81 
 hw/s390x/ap-device.c | 39 +
 hw/s390x/s390-virtio-ccw.c   |  4 ++
 include/hw/s390x/ap-bridge.h | 37 
 include/hw/s390x/ap-device.h | 38 +
 7 files changed, 213 insertions(+)
 create mode 100644 hw/s390x/ap-bridge.c
 create mode 100644 hw/s390x/ap-device.c
 create mode 100644 include/hw/s390x/ap-bridge.h
 create mode 100644 include/hw/s390x/ap-device.h

diff --git a/MAINTAINERS b/MAINTAINERS
index d12518c08f10..97e8ed808bc0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1199,6 +1199,18 @@ F: include/hw/s390x/s390-ccw.h
 T: git git://github.com/cohuck/qemu.git s390-next
 L: qemu-s3...@nongnu.org
 
+vfio-ap
+M: Christian Borntraeger 
+M: Tony Krowiak 
+M: Halil Pasic 
+M: Pierre Morel 
+S: Supported
+F: hw/s390x/ap-device.c
+F: hw/s390x/ap-bridge.c
+F: include/hw/s390x/ap-device.h
+F: include/hw/s390x/ap-bridge.h
+L: qemu-s3...@nongnu.org
+
 vhost
 M: Michael S. Tsirkin 
 S: Supported
diff --git a/hw/s390x/Makefile.objs b/hw/s390x/Makefile.objs
index 93282f7c593c..add89b150d90 100644
--- a/hw/s390x/Makefile.objs
+++ b/hw/s390x/Makefile.objs
@@ -20,3 +20,5 @@ obj-$(CONFIG_TCG) += tod-qemu.o
 obj-$(CONFIG_KVM) += s390-skeys-kvm.o
 obj-$(CONFIG_KVM) += s390-stattrib-kvm.o
 obj-y += s390-ccw.o
+obj-y += ap-device.o
+obj-y += ap-bridge.o
diff --git a/hw/s390x/ap-bridge.c b/hw/s390x/ap-bridge.c
new file mode 100644
index ..8564dfa96ee7
--- /dev/null
+++ b/hw/s390x/ap-bridge.c
@@ -0,0 +1,81 @@
+/*
+ * ap bridge
+ *
+ * Copyright 2018 IBM Corp.
+ * Author(s): Halil Pasic 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/sysbus.h"
+#include "qemu/bitops.h"
+#include "hw/s390x/ap-bridge.h"
+#include "cpu.h"
+
+static char *vfio_ap_bus_get_dev_path(DeviceState *dev)
+{
+/* at most one */
+return g_strdup_printf("/1");
+}
+
+static void vfio_ap_bus_class_init(ObjectClass *klass, void *data)
+{
+BusClass *k = BUS_CLASS(klass);
+
+k->get_dev_path = vfio_ap_bus_get_dev_path;
+/* More than one vfio-ap device does not make sense */
+k->max_dev = 1;
+}
+
+static const TypeInfo vfio_ap_bus_info = {
+.name = TYPE_AP_BUS,
+.parent = TYPE_BUS,
+.instance_size = sizeof(APBus),
+.class_init = vfio_ap_bus_class_init,
+};
+
+void s390_init_ap(void)
+{
+DeviceState *dev;
+
+/* If no AP instructions then no need for AP bridge */
+if (!s390_has_feat(S390_FEAT_AP)) {
+return;
+}
+
+/* Create bridge device */
+dev = qdev_create(NULL, TYPE_AP_BRIDGE);
+object_property_add_child(qdev_get_machine(), TYPE_AP_BRIDGE,
+  OBJECT(dev), NULL);
+qdev_init_nofail(dev);
+
+/* Create bus on bridge device */
+qbus_create(TYPE_AP_BUS, dev, TYPE_AP_BUS);
+ }
+
+
+
+static void ap_bridge_class_init(ObjectClass *klass, void *data)
+{
+DeviceClass *dc = DEVICE_CLASS(klass);
+
+set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
+}
+
+static const TypeInfo ap_bridge_info = {
+.name  = TYPE_AP_BRIDGE,
+.parent= TYPE_SYS_BUS_DEVICE,
+.instance_size = sizeof(APBridge),
+.class_init= ap_bridge_class_init,
+};
+
+static void ap_register(void)
+{
+type_register_static(_bridge_info);
+type_register_static(_ap_bus_info);
+}
+
+type_init(ap_register)
diff --git a/hw/s390x/ap-device.c b/hw/s390x/ap-device.c
new file mode 100644
index ..3cd4bae52591
--- /dev/null
+++ b/hw/s390x/ap-device.c
@@ -0,0 +1,39 @@
+/*
+ * Adjunct Processor (AP) matrix device
+ *
+ * Copyright 2018 IBM Corp.
+ * Author(s): Tony Krowiak 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#include "qemu/osdep.h"
+#include "qemu/module.h"
+#include "qapi/error.h"
+#include "hw/qdev.h"
+#include "hw/s390x/ap-device.h"
+
+static void ap_class_init(ObjectClass *klass, void *data)
+{
+DeviceClass *dc = DEVICE_CLASS(klass);
+
+dc->desc = "AP device class";
+dc->hotpluggable = false;
+}
+
+static const TypeInfo ap_device_info = {
+.name = AP_DEVICE_TYPE,
+.parent = TYPE_DEVICE,
+.instance_size = sizeof(APDevice),
+.class_size = sizeof(APDeviceClass),
+.class_init = ap_class_init,
+.abstract = true,
+};
+
+static void ap_device_register(void)
+{
+type_register_static(_device_info);
+}
+
+type_init(ap_device_register)
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index f0f7fdcaddf2..3c100c24f3e8 100644
--- 

[Qemu-devel] [PATCH v9 1/6] linux-headers: linux header updates for AP support

[Tony Krowiak]

Updates the linux header files in preparation for introduction
of the VFIO AP device:

* Added a feature ID to indicate AP facilities are installed

* Added device attributes to the KVM_S390_VM_CRYPTO group
  to indicate whether AP instructions are to be interpreted

* Added VFIO device information for AP devices

Signed-off-by: Tony Krowiak 
---
 linux-headers/asm-s390/kvm.h | 3 +++
 linux-headers/linux/vfio.h   | 2 ++
 2 files changed, 5 insertions(+)

diff --git a/linux-headers/asm-s390/kvm.h b/linux-headers/asm-s390/kvm.h
index 1ab9901911bf..b4948706468e 100644
--- a/linux-headers/asm-s390/kvm.h
+++ b/linux-headers/asm-s390/kvm.h
@@ -130,6 +130,7 @@ struct kvm_s390_vm_cpu_machine {
 #define KVM_S390_VM_CPU_FEAT_PFMFI 11
 #define KVM_S390_VM_CPU_FEAT_SIGPIF12
 #define KVM_S390_VM_CPU_FEAT_KSS   13
+#define KVM_S390_VM_CPU_FEAT_AP14
 struct kvm_s390_vm_cpu_feat {
__u64 feat[16];
 };
@@ -160,6 +161,8 @@ struct kvm_s390_vm_cpu_subfunc {
 #define KVM_S390_VM_CRYPTO_ENABLE_DEA_KW   1
 #define KVM_S390_VM_CRYPTO_DISABLE_AES_KW  2
 #define KVM_S390_VM_CRYPTO_DISABLE_DEA_KW  3
+#define KVM_S390_VM_CRYPTO_ENABLE_APIE 4
+#define KVM_S390_VM_CRYPTO_DISABLE_APIE5
 
 /* kvm attributes for migration mode */
 #define KVM_S390_VM_MIGRATION_STOP 0
diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index 3615a269d378..838919a4c03a 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -200,6 +200,7 @@ struct vfio_device_info {
 #define VFIO_DEVICE_FLAGS_PLATFORM (1 << 2)/* vfio-platform device */
 #define VFIO_DEVICE_FLAGS_AMBA  (1 << 3)   /* vfio-amba device */
 #define VFIO_DEVICE_FLAGS_CCW  (1 << 4)/* vfio-ccw device */
+#define VFIO_DEVICE_FLAGS_AP (1 << 5)  /* vfio-ap device */
__u32   num_regions;/* Max region index + 1 */
__u32   num_irqs;   /* Max IRQ index + 1 */
 };
@@ -215,6 +216,7 @@ struct vfio_device_info {
 #define VFIO_DEVICE_API_PLATFORM_STRING"vfio-platform"
 #define VFIO_DEVICE_API_AMBA_STRING"vfio-amba"
 #define VFIO_DEVICE_API_CCW_STRING "vfio-ccw"
+#define VFIO_DEVICE_API_AP_STRING  "vfio-ap"
 
 /**
  * VFIO_DEVICE_GET_REGION_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 8,
-- 
2.19.0.221.g150f307

Re: [Qemu-devel] [RFC PATCH] ssi-sd: Make devices picking up backends unavailable with -device

[Philippe Mathieu-Daudé]

On 9/26/18 1:46 PM, Markus Armbruster wrote:
> Philippe Mathieu-Daudé  writes:
> 
>> Hi Markus,
>>
>> On 9/26/18 11:00 AM, Markus Armbruster wrote:
>>> Device models aren't supposed to go on fishing expeditions for
>>> backends.  They should expose suitable properties for the user to set.
>>> For onboard devices, board code sets them.
>>>
>>> Device ssi-sd picks up its block backend in its init() method with
>>> drive_get_next() instead.  This mistake is already marked FIXME since
>>> commit af9e40a.
>>>
>>> Unset user_creatable to remove the mistake from our external
>>> interface.  Since the SSI bus doesn't support hotplug, only -device
>>> can be affected.  Only certain ARM machines provide an SSI bus.
>>
>> There is also a MicroBlaze machine.
> 
> ssi-sd isn't linked into that one:
> qemu-system-microblaze: -device ssi-sd: 'ssi-sd' is not a valid device model 
> name
> 
>> Note that out of those target, we could model SSI buses on
>> southbridges/LPC devices but there is no interest.
>>
>> I'm also spending some hobby time on a MIPS SoC which exposes a SSI bus.
>>
>>> Signed-off-by: Markus Armbruster 
>>> ---
>>> Are there valid uses of -device ssi-sd?  If no, this patch is fine.
>>> If yes, this patch breaks them.  But fixing the FIXME will also break
>>> them.  What should we do?
>>
>> This device was probably orphan (not used) for a long time, and seems
>> unusable in it current state to me, see:
>> http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg01757.html
> 
> The more that's true, the less we have to worry about preserving
> compatibility ;)
> 
>> I have to address Peter's comment to my patch noted in my TODO (low
>> priority), so I'm OK to get ride of the FIXME.
>>
>> I suppose it is easier for you to get your patch in, then let me clean
>> that later.
> 
> Works for me.

So using 'user_creatable = false':
Acked-by: Philippe Mathieu-Daudé 

[Qemu-devel] [PATCH v9 0/6] s390x: vfio-ap: guest dedicated crypto adapters

[Tony Krowiak]

From: Tony Krowiak 

This patch series is the QEMU counterpart to the KVM/kernel support for 
guest dedicated crypto adapters. The KVM/kernel model is built on the 
VFIO mediated device framework and provides the infrastructure for 
granting exclusive guest access to crypto devices installed on the linux 
host. This patch series introduces a new QEMU command line option, QEMU 
object model and CPU model features to exploit the KVM/kernel model.

See the detailed specifications for AP virtualization provided by this 
patch set in docs/vfio-ap.txt for a more complete discussion of the 
design introduced by this patch series.

v8 => v9 Change log:
===
* Removed all references to VFIO in AP bridge and bus
* Expose AP feature only if the KVM_S390_VM_CRYPTO_ENABLE_APIE VM attribute
  is exposed by KVM - i.e., if AP instructions are available on the linux
  host.
* Enable AP interpretation only if AP feature is switched on; no need to
  disable because it is disabled by default.

v7 => v8 Change log:
===
* Enable SIE interpretation AP instructions if the CPU model feature for
  AP instructions is turned on for the guest.

v6 => v7 Change log;
===
* Changed email address for Signed-off-by

v5 => v6 Change log:
===
* Added reset handling fo vfio-ap device
* Added a bridge/bus to AP device object model - thanks to Halil Pasic

v4 => v5 Change log:
===
* Added MAINTAINERS entries for VFIO AP
* Added explanation for why we are only supporting zEC12 and newer CPU 
  models.
* Changed CPU model feature qci=on|off to apqci=on|off
* Misc. minor changes

v3 => v4 Change log:
===
* Made vfio-ap device unpluggable for now
* Renamed command line CPU model feature for QCI: qci=on -> apqci=on
* Removed call to KVM_S390_VM_CRYPTO_INTERPRET_AP ioctl - ioctl was 
  removed from kernel and AP instruction interpretation is set from the
  VFIO device driver
* Added check to ensure only one vfio-ap device can be configured per 
  guest
* Removed AP instruction interception handlers: AP instructions will be 
  interpreted by default if AP facilities are installed to handle the case
  where feature ap=on and no vfio-ap device is configured for the guest.

Tony Krowiak (6):
  linux-headers: linux header updates for AP support
  s390x/cpumodel: Set up CPU model for AP device support
  s390x/kvm: enable AP instruction interpretation for guest
  s390x/ap: base Adjunct Processor (AP) object model
  s390x/vfio: ap: Introduce VFIO AP device
  s390: doc: detailed specifications for AP virtualization

 MAINTAINERS   |  14 +
 default-configs/s390x-softmmu.mak |   1 +
 docs/vfio-ap.txt  | 787 ++
 hw/s390x/Makefile.objs|   2 +
 hw/s390x/ap-bridge.c  |  81 +++
 hw/s390x/ap-device.c  |  39 ++
 hw/s390x/s390-virtio-ccw.c|   4 +
 hw/vfio/Makefile.objs |   1 +
 hw/vfio/ap.c  | 181 +++
 include/hw/s390x/ap-bridge.h  |  37 ++
 include/hw/s390x/ap-device.h  |  38 ++
 include/hw/vfio/vfio-common.h |   1 +
 linux-headers/asm-s390/kvm.h  |   3 +
 linux-headers/linux/vfio.h|   2 +
 target/s390x/cpu_features.c   |   3 +
 target/s390x/cpu_features_def.h   |   3 +
 target/s390x/cpu_models.c |   2 +
 target/s390x/gen-features.c   |   3 +
 target/s390x/kvm.c|  19 +
 19 files changed, 1221 insertions(+)
 create mode 100644 docs/vfio-ap.txt
 create mode 100644 hw/s390x/ap-bridge.c
 create mode 100644 hw/s390x/ap-device.c
 create mode 100644 hw/vfio/ap.c
 create mode 100644 include/hw/s390x/ap-bridge.h
 create mode 100644 include/hw/s390x/ap-device.h

-- 
2.19.0.221.g150f307

[Qemu-devel] [PATCH v9 5/6] s390x/vfio: ap: Introduce VFIO AP device

[Tony Krowiak]

Introduces a VFIO based AP device. The device is defined via
the QEMU command line by specifying:

-device vfio-ap,sysfsdev=

There may be only one vfio-ap device configured for a guest.

The mediated matrix device is created by the VFIO AP device
driver by writing a UUID to a sysfs attribute file (see
docs/vfio-ap.txt). The mediated matrix device will be named
after the UUID. Symbolic links to the $uuid are created in
many places, so the path to the mediated matrix device $uuid
can be specified in any of the following ways:

/sys/devices/vfio_ap/matrix/$uuid
/sys/devices/vfio_ap/matrix/mdev_supported_types/vfio_ap-passthrough/devices/$uuid
/sys/bus/mdev/devices/$uuid
/sys/bus/mdev/drivers/vfio_mdev/$uuid

When the vfio-ap device is realized, it acquires and opens the
VFIO iommu group to which the mediated matrix device is
bound. This causes a VFIO group notification event to be
signaled. The vfio_ap device driver's group notification
handler will get called at which time the device driver
will configure the the AP devices to which the guest will
be granted access.

Signed-off-by: Tony Krowiak 
---
 MAINTAINERS   |   1 +
 default-configs/s390x-softmmu.mak |   1 +
 hw/vfio/Makefile.objs |   1 +
 hw/vfio/ap.c  | 181 ++
 include/hw/vfio/vfio-common.h |   1 +
 5 files changed, 185 insertions(+)
 create mode 100644 hw/vfio/ap.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 97e8ed808bc0..29041da69237 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1209,6 +1209,7 @@ F: hw/s390x/ap-device.c
 F: hw/s390x/ap-bridge.c
 F: include/hw/s390x/ap-device.h
 F: include/hw/s390x/ap-bridge.h
+F: hw/vfio/ap.c
 L: qemu-s3...@nongnu.org
 
 vhost
diff --git a/default-configs/s390x-softmmu.mak 
b/default-configs/s390x-softmmu.mak
index d6b67d50f0e4..5eef37592451 100644
--- a/default-configs/s390x-softmmu.mak
+++ b/default-configs/s390x-softmmu.mak
@@ -7,3 +7,4 @@ CONFIG_S390_FLIC=y
 CONFIG_S390_FLIC_KVM=$(CONFIG_KVM)
 CONFIG_VFIO_CCW=$(CONFIG_LINUX)
 CONFIG_WDT_DIAG288=y
+CONFIG_VFIO_AP=$(CONFIG_LINUX)
diff --git a/hw/vfio/Makefile.objs b/hw/vfio/Makefile.objs
index a2e7a0a7cf02..8b3f664d85f7 100644
--- a/hw/vfio/Makefile.objs
+++ b/hw/vfio/Makefile.objs
@@ -6,4 +6,5 @@ obj-$(CONFIG_SOFTMMU) += platform.o
 obj-$(CONFIG_VFIO_XGMAC) += calxeda-xgmac.o
 obj-$(CONFIG_VFIO_AMD_XGBE) += amd-xgbe.o
 obj-$(CONFIG_SOFTMMU) += spapr.o
+obj-$(CONFIG_VFIO_AP) += ap.o
 endif
diff --git a/hw/vfio/ap.c b/hw/vfio/ap.c
new file mode 100644
index ..429988f23f98
--- /dev/null
+++ b/hw/vfio/ap.c
@@ -0,0 +1,181 @@
+/*
+ * VFIO based AP matrix device assignment
+ *
+ * Copyright 2018 IBM Corp.
+ * Author(s): Tony Krowiak 
+ *Halil Pasic 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#include 
+#include 
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/sysbus.h"
+#include "hw/vfio/vfio.h"
+#include "hw/vfio/vfio-common.h"
+#include "hw/s390x/ap-device.h"
+#include "qemu/error-report.h"
+#include "qemu/queue.h"
+#include "qemu/option.h"
+#include "qemu/config-file.h"
+#include "cpu.h"
+#include "kvm_s390x.h"
+#include "sysemu/sysemu.h"
+#include "hw/s390x/ap-bridge.h"
+#include "exec/address-spaces.h"
+
+#define VFIO_AP_DEVICE_TYPE  "vfio-ap"
+
+typedef struct VFIOAPDevice {
+APDevice apdev;
+VFIODevice vdev;
+} VFIOAPDevice;
+
+static void vfio_ap_compute_needs_reset(VFIODevice *vdev)
+{
+vdev->needs_reset = false;
+}
+
+/*
+ * We don't need vfio_hot_reset_multi and vfio_eoi operations for
+ * vfio-ap device now.
+ */
+struct VFIODeviceOps vfio_ap_ops = {
+.vfio_compute_needs_reset = vfio_ap_compute_needs_reset,
+};
+
+static void vfio_ap_put_device(VFIOAPDevice *vapdev)
+{
+g_free(vapdev->vdev.name);
+vfio_put_base_device(>vdev);
+}
+
+static VFIOGroup *vfio_ap_get_group(VFIOAPDevice *vapdev, Error **errp)
+{
+char *tmp, group_path[PATH_MAX];
+ssize_t len;
+int groupid;
+
+tmp = g_strdup_printf("%s/iommu_group", vapdev->vdev.sysfsdev);
+len = readlink(tmp, group_path, sizeof(group_path));
+g_free(tmp);
+
+if (len <= 0 || len >= sizeof(group_path)) {
+error_setg(errp, "%s: no iommu_group found for %s",
+   VFIO_AP_DEVICE_TYPE, vapdev->vdev.sysfsdev);
+return NULL;
+}
+
+group_path[len] = 0;
+
+if (sscanf(basename(group_path), "%d", ) != 1) {
+error_setg(errp, "vfio: failed to read %s", group_path);
+return NULL;
+}
+
+return vfio_get_group(groupid, _space_memory, errp);
+}
+
+static void vfio_ap_realize(DeviceState *dev, Error **errp)
+{
+int ret;
+char *mdevid;
+Error *local_err = NULL;
+VFIOGroup *vfio_group;
+APDevice *apdev = DO_UPCAST(APDevice, parent_obj, dev);
+VFIOAPDevice *vapdev = DO_UPCAST(VFIOAPDevice, apdev, apdev);
+
+vapdev = 

Re: [Qemu-devel] [Qemu-arm] [PATCH] virt: Suppress external aborts on virt-2.10 and earlier

[Philippe Mathieu-Daudé]

On 9/26/18 2:29 PM, Peter Maydell wrote:
> On 26 September 2018 at 11:35, Philippe Mathieu-Daudé  wrote:
>> Hi Peter,
>>
>> On 9/25/18 4:41 PM, Peter Maydell wrote:
>>> In commit c79c0a314c43b78 we enabled emulation of external aborts
>>> when the guest attempts to access a physical address with no
>>> mapped device. In commit 4672cbd7bed88dc6 we suppress this for
>>> most legacy boards to prevent breakage of previously working
>>> guests, but we didn't suppress it in the 'virt' board, with
>>> the rationale "we know that guests won't try to prod devices
>>> that we don't describe in the device tree or ACPI tables". This
>>> is mostly true, but we've had a report of a Linux guest image
>>> that this did break. The problem seems to be that the guest
>>> is (incorrectly) configured with a DEBUG_UART_PHYS value that
>>> tells it there is a uart at 0x10009000 (which is true for
>>> vexpress but not for virt), so in early bootup the kernel
>>> probes this bogus address.
>>>
>>> This is a misconfigured guest, so we don't need to worry
>>> about it too much, but we can arrange that guests that ran
>>> on QEMU v2.10 (before c79c0a314c43b78) will still run on
>>> the "virt-2.10" board model, by suppressing external aborts
>>> only for that version and earlier. This seems a reasonable
>>> compromise.
>>
>> I tried another approach to keep MEMTX_DECODE_ERROR delivered (out of
>> 0x10009000) for 2.10, using the UNIMP device, but the code looks uglier
>> (create too much generic code for a single issue):
> 
> I definitely don't want to do that...
> 
>> I prefer your compromise.
>>
>>>
>>> Cc: qemu-sta...@nongnu.org
>>> Signed-off-by: Peter Maydell 
>>> ---
>>>  hw/arm/virt.c | 1 +
>>>  1 file changed, 1 insertion(+)
>>>
>>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>>> index 0b57f87abcb..3ba310a37b6 100644
>>> --- a/hw/arm/virt.c
>>> +++ b/hw/arm/virt.c
>>> @@ -1926,6 +1926,7 @@ static void virt_machine_2_10_options(MachineClass 
>>> *mc)
>>>  {
>>>  virt_machine_2_11_options(mc);
>>>  SET_MACHINE_COMPAT(mc, VIRT_COMPAT_2_10);
>>
>> Can you add a short comment here?
>>
>>/* See https://bugs.launchpad.net/qemu/+bug/???:
>> * Some misconfigured Linux guest poke for vexpress uart
>> * at 0x10009000 */
> 
> I view it as more "virt-2.10 is supposed to behave like
> the 2.10 release virt" rather than a specific fix for
> this guest, I think.

You right, fine. Very useful short explanation btw.

Re: [Qemu-devel] [PATCH v5 2/8] target/mips: Support R5900 specific three-operand MULT and MULTU

[Maciej W. Rozycki]

On Thu, 27 Sep 2018, Philippe Mathieu-Daudé wrote:

> > +static void gen_mul_r5900(DisasContext *ctx, uint32_t opc,
> > +  int acc, int rd, int rs, int rt)
> 
> Since we have acc = 0 we can directly use cpu_LO[0] and cpu_HI[0],
> removing needs for an 'acc' argument.

 Corresponding MULT1 and MULTU1 instructions use `acc = 1' though, as do 
similar variations of all the other MDU instructions.  Maybe the addition 
of the required `acc' argument could be staged for a separate patch with 
all the pipeline 1 MDU instructions though.

  Maciej

Re: [Qemu-devel] [PATCH 2/2] hw/pci-host/x86: extend the 64-bit PCI hole relative to the fw-assigned base

[Laszlo Ersek]

On 09/26/18 18:26, Alex Williamson wrote:
> On Wed, 26 Sep 2018 13:12:47 +0200
> Laszlo Ersek  wrote:
> 
>> On 09/25/18 22:36, Alex Williamson wrote:
>>> On Tue, 25 Sep 2018 00:13:46 +0200
>>> Laszlo Ersek  wrote:
>>>   
 In commit 9fa99d2519cb ("hw/pci-host: Fix x86 Host Bridges 64bit PCI
 hole", 2017-11-16), we meant to expose such a 64-bit PCI MMIO aperture in
 the ACPI DSDT that would be at least as large as the new "pci-hole64-size"
 property (2GB on i440fx, 32GB on q35). The goal was to offer "enough"
 64-bit MMIO aperture to the guest OS for hotplug purposes.

 In that commit, we added or modified five functions:

 - pc_pci_hole64_start(): shared between i440fx and q35. Provides a default
   64-bit base, which starts beyond the cold-plugged 64-bit RAM, and skips
   the DIMM hotplug area too (if any).

 - i440fx_pcihost_get_pci_hole64_start(), q35_host_get_pci_hole64_start():
   board-specific 64-bit base property getters called abstractly by the
   ACPI generator. Both of these fall back to pc_pci_hole64_start() if the
   firmware didn't program any 64-bit hole (i.e. if the firmware didn't
   assign a 64-bit GPA to any MMIO BAR on any device). Otherwise, they
   honor the firmware's BAR assignments (i.e., they treat the lowest 64-bit
   GPA programmed by the firmware as the base address for the aperture).

 - i440fx_pcihost_get_pci_hole64_end(), q35_host_get_pci_hole64_end():
   these intended to extend the aperture to our size recommendation,
   calculated relative to the base of the aperture.

 Despite the original intent, i440fx_pcihost_get_pci_hole64_end() and
 q35_host_get_pci_hole64_end() currently only extend the aperture relative
 to the default base (pc_pci_hole64_start()), ignoring any programming done
 by the firmware. This means that our size recommendation may not be met.
 Fix it by honoring the firmware's address assignments.

 The strange extension sizes were spotted by Alex, in the log of a guest
 kernel running on top of OVMF (which does assign 64-bit GPAs to BARs).

 This change only affects DSDT generation, therefore no new compat property
 is being introduced. Also, because SeaBIOS never assigns 64-bit GPAs to
 64-bit BARs, the patch makes no difference to SeaBIOS guests.  
>>>
>>> This is not exactly true, SeaBIOS will make use of 64-bit MMIO, but
>>> only if it cannot satisfy all the BARs from 32-bit MMMIO, see
>>> src/fw/pciinit.c:pci_bios_map_devices.  Create a VM with several
>>> assigned GPUs and you'll eventually cross that threshold and all 64-bit
>>> BARs will be moved above 4G.  I'm sure a few sufficiently sized ivshmem
>>> devices could do the same.  Thanks,  
>>
>> The effect of this patch is not hard to demonstrate with SeaBIOS+Q35,
>> when using e.g. 5GB of guest RAM and a 4GB ivshmem-plain device.
>>
>> However, using SeaBIOS+i440fx, I can't show the difference. I've been
>> experimenting with various ivshmem devices (even multiple at the same
>> time, with different sizes). The "all or nothing" nature of SeaBIOS's
>> high allocation of the 64-bit BARs, combined with hugepage alignment
>> inside SeaBIOS, combined with the small (2GB) rounding size used in QEMU
>> for i440fx, seem to make it surprisingly difficult to trigger the issue.
>>
>> I figure I should:
>>
>> (1) remove the sentence "the patch makes no difference to SeaBIOS
>> guests" from the commit message,
>>
>> (2) include the DSDT diff on SeaBIOS/q35 in the commit message,
>>
>> (3) remain silent on SeaBIOS/i440fx, in the commit message,
>>
>> (4) append a new patch, for "bios-tables-test", so that the ACPI gen
>> change is validated as part of the test suite, on SeaBIOS/q35.
>>
>> Regarding (4):
>>
>> - is it OK if I add the test only for Q35?
>>
>> - what guest RAM size am I allowed to use in the test suite? In my own
>> SeaBIOS/Q35 reproducer I currently use 5GB, but I'm not sure if it's
>> acceptable for the test suite.
> 
> Seems like you've done due diligence, the plan looks ok to me.
> Regarding the test memory allocation, is it possible and reasonable to
> perhaps create a 256MB shared memory area and re-use it for multiple
> ivshmem devices?  ie. rather than 1, 4GB ivshmem device, use 16, 256MB
> devices, all with the same backing.  Thanks,

In order to show that the patch makes a difference, on SeaBIOS+Q35, I
must have SeaBIOS place the lowest 64-bit BAR strictly above end-of-RAM
/ end-of-DIMM-hotplug-area (i.e., strictly past the return value of
pc_pci_hole64_start()). That's not easy, assuming test suite constraints:

- With a small guest RAM size (and no DIMM hotplug area),
pc_pci_hole64_start() returns 4GB. Unfortunately, 4GB is well aligned
for 256MB BARs, so that's where SeaBIOS will place the lowest such BAR
too. Therefore the patch makes no difference.

- If I try to mis-align pc_pci_hole64_start() for the 256MB BAR size, by
adding a DIMM 

Re: [Qemu-devel] [PATCH v5 2/8] target/mips: Support R5900 specific three-operand MULT and MULTU

[Philippe Mathieu-Daudé]

On 9/15/18 11:25 AM, Fredrik Noring wrote:

Can you copy/paste some info regarding those instructions from the ISA
here, to note how they differ? ...

> Signed-off-by: Fredrik Noring 
> ---
>  target/mips/translate.c | 53 
> +
>  1 file changed, 53 insertions(+)
> 
> diff --git a/target/mips/translate.c b/target/mips/translate.c
> index ab16cdb911..fb571e278e 100644
> --- a/target/mips/translate.c
> +++ b/target/mips/translate.c
> @@ -3768,6 +3768,57 @@ static void gen_muldiv(DisasContext *ctx, uint32_t opc,
>  tcg_temp_free(t1);
>  }
>  

... Another option is to describe in a big comment here.

> +static void gen_mul_r5900(DisasContext *ctx, uint32_t opc,
> +  int acc, int rd, int rs, int rt)

Since we have acc = 0 we can directly use cpu_LO[0] and cpu_HI[0],
removing needs for an 'acc' argument.

> +{
> +TCGv t0 = tcg_temp_new();
> +TCGv t1 = tcg_temp_new();
> +
> +gen_load_gpr(t0, rs);
> +gen_load_gpr(t1, rt);
> +
> +switch (opc) {
> +case OPC_MULT:
> +{
> +TCGv_i32 t2 = tcg_temp_new_i32();
> +TCGv_i32 t3 = tcg_temp_new_i32();
> +tcg_gen_trunc_tl_i32(t2, t0);
> +tcg_gen_trunc_tl_i32(t3, t1);
> +tcg_gen_muls2_i32(t2, t3, t2, t3);
> +if (rd)

Check QEMU CODING_STYLE "Block structure":

  Every indented statement is braced; even if the block contains
  just one statement.  The opening brace is on the line that
  contains the control flow statement that introduces the new block;
  the closing brace is on the same line as the else keyword, or on
  a line by itself if there is no else keyword.

QEMU scripts/checkpatch.pl reports such mistakes.

> +tcg_gen_ext_i32_tl(cpu_gpr[rd], t2);

I'd use:

   gen_move_low32(cpu_gpr[rd], t2);

> +tcg_gen_ext_i32_tl(cpu_LO[acc], t2);

So:

   tcg_gen_ext_i32_tl(cpu_LO[0], t2);

> +tcg_gen_ext_i32_tl(cpu_HI[acc], t3);

And:

   tcg_gen_ext_i32_tl(cpu_HI[0], t3);

> +tcg_temp_free_i32(t2);
> +tcg_temp_free_i32(t3);
> +}
> +break;
> +case OPC_MULTU:
> +{
> +TCGv_i32 t2 = tcg_temp_new_i32();
> +TCGv_i32 t3 = tcg_temp_new_i32();
> +tcg_gen_trunc_tl_i32(t2, t0);
> +tcg_gen_trunc_tl_i32(t3, t1);
> +tcg_gen_mulu2_i32(t2, t3, t2, t3);
> +if (rd)
> +tcg_gen_ext_i32_tl(cpu_gpr[rd], t2);
> +tcg_gen_ext_i32_tl(cpu_LO[acc], t2);
> +tcg_gen_ext_i32_tl(cpu_HI[acc], t3);

Same comments from MULT apply here.

> +tcg_temp_free_i32(t2);
> +tcg_temp_free_i32(t3);
> +}
> +break;
> +default:
> +MIPS_INVAL("mul R5900");
> +generate_exception_end(ctx, EXCP_RI);
> +goto out;
> +}
> +
> + out:
> +tcg_temp_free(t0);
> +tcg_temp_free(t1);
> +}
> +
>  static void gen_mul_vr54xx (DisasContext *ctx, uint32_t opc,
>  int rd, int rs, int rt)
>  {
> @@ -22378,6 +22429,8 @@ static void decode_opc_special_legacy(CPUMIPSState 
> *env, DisasContext *ctx)
>  check_insn(ctx, INSN_VR54XX);
>  op1 = MASK_MUL_VR54XX(ctx->opcode);
>  gen_mul_vr54xx(ctx, op1, rd, rs, rt);
> +} else if (ctx->insn_flags & INSN_R5900) {
> +gen_mul_r5900(ctx, op1, 0, rd, rs, rt);

Removing 'acc' arg:

   gen_mul_r5900(ctx, op1, rd, rs, rt);

Note, these instructions are also valid on the R3900 (which also has
MADD/MADDU).

Would gen_mul_toshiba() be a better common name? I don't like it but
can't think of another.

>  } else {
>  gen_muldiv(ctx, op1, rd & 3, rs, rt);
>  }
> 
Regards,

Phil.

[Qemu-devel] [Bug 1793904] Re: files are randomly overwritten by Zero Bytes

[Hans]

Hi John,


As for your second question: 
ii  glusterfs-comm 4.1.3-1  amd64GlusterFS common libraries and tr


As for the other parts:  I was playing with this one ealier, therefore this is 
not completely relieable: 

qemu-img check images/conesphere_internet_meeting1.qcow2 
No errors were found on the image.
143086/491520 = 29.11% allocated, 7.72% fragmented, 0.00% compressed clusters
Image end offset: 9379708928


I will try to reproduce  your Ideas as soon as I can. :)

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1793904

Title:
  files are randomly overwritten by Zero Bytes

Status in QEMU:
  New

Bug description:
  Hello together,

  I am currently tracking down a "Hard to reproduce" bug on my systems
  that I first discovered during gitlab installation:

  
  Here is the Text from the Gitlab Bug 
https://gitlab.com/gitlab-org/gitlab-ce/issues/51023
  
--

  Steps to reproduce

  I still do not have all the steps together to reproduce, so far it is:
  apt install gitlab-ce and
  gitlab-rake backup:recovery
  Then it works for some time before it fails.

  What is the current bug behavior?

  I have a 12 hour old Installation of gitlab ce 11.2.3-ce.0 for debian
  stretch on a fresh debian stretch system together with our imported
  data. However it turns out that some gitlab related files contain Zero
  bytes instead of actual data.

  root@gitlab:~# xxd -l 16 /opt/gitlab/bin/gitlab-ctl
  :          

  This behaviour is somewhat strange because it was working for a few
  minutes/hours. I did write a shell script to find out which files are
  affected of this memory loss. It turns out that only files located
  under /opt/gitlab are affected, if I rule out files like
  /var/log/faillog and some postgresql table files.

  What I find even stranger is that it does not seem to affect
  Logfiles/databases/git_repositorys but application files, like .rb
  scripts. and not all of them. No non gitlab package is affected.

  What is the expected correct behavior?
  Binarys and .rb files should stay as they are.

  Possible fixes

  I am still investigating, I hope that it is not an infrastructure problem 
(libvirt/qemu/glusterfs) it can still be one but the point that files of 
/opt/gitlab are affected and not any logfile and that we to not have similar 
problems with any other system leads me to the application for now.
  If I would have used docker the same problem might have caused a reboot of 
the container.
  But for the Debian package it is a bit of work to recover. That is all a 
workaround, however.
  
-

  I do have found 2 more systems having the same problem with different
  software:

  root@erp:~# xxd -l 16 /usr/share/perl/5.26.2/constant.pm
  :          

  The Filesize itself is, compared with another machine 1660 Bytes
  for both the corrupted and the intact file. It looks to me from the
  outside that if some data in the qcow2 file is written too many bytes
  get written so it sometimes overwites data of existing files located
  right after the position in memory where the write goes to.

  I would like to rule out Linux+Ext4 filesystems because I find it
  highly unlikely that such an error keeps undiscovered in that part of
  the environment for long. I think the same might go for qemu.

  Which leaves qemu, gemu+gluster:// mount, qcow2 volumes, glusterfs,
  network. So I am now going to check if I can find any system which
  gets its volumes via fusermount instead of gluster:// path if the
  error is gone there. This may take a while.

  
  - some software versions---

  QEMU emulator version 2.12.0 (Debian 1:2.12+dfsg-3)
  Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

  libvirt-daemon-driver-storage-gluster/testing,unstable,now 4.6.0-2
  amd64 [installed]

  ii  glusterfs-client   4.1.3-1amd64

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1793904/+subscriptions

Re: [Qemu-devel] [PATCH v5 2/8] target/mips: Support R5900 specific three-operand MULT and MULTU

[Philippe Mathieu-Daudé]

On 9/26/18 11:59 PM, Philippe Mathieu-Daudé wrote:
> On 9/25/18 2:20 PM, Philippe Mathieu-Daudé wrote:
>> Hi Fredrik,
>>
>> On 9/15/18 11:25 AM, Fredrik Noring wrote:
>>> Signed-off-by: Fredrik Noring 
>>> ---
>>>  target/mips/translate.c | 53 
>>> +
>>>  1 file changed, 53 insertions(+)
>>>
>>> diff --git a/target/mips/translate.c b/target/mips/translate.c
>>> index ab16cdb911..fb571e278e 100644
>>> --- a/target/mips/translate.c
>>> +++ b/target/mips/translate.c
>>> @@ -3768,6 +3768,57 @@ static void gen_muldiv(DisasContext *ctx, uint32_t 
>>> opc,
>>>  tcg_temp_free(t1);
>>>  }
>>>  
>>> +static void gen_mul_r5900(DisasContext *ctx, uint32_t opc,
>>> +  int acc, int rd, int rs, int rt)
>>> +{
>>> +TCGv t0 = tcg_temp_new();
>>> +TCGv t1 = tcg_temp_new();
>>> +
>>> +gen_load_gpr(t0, rs);
>>> +gen_load_gpr(t1, rt);
>>> +
>>> +switch (opc) {
>>> +case OPC_MULT:
>>> +{
>>> +TCGv_i32 t2 = tcg_temp_new_i32();
>>> +TCGv_i32 t3 = tcg_temp_new_i32();
>>> +tcg_gen_trunc_tl_i32(t2, t0);
>>> +tcg_gen_trunc_tl_i32(t3, t1);
>>> +tcg_gen_muls2_i32(t2, t3, t2, t3);
>>> +if (rd)
>>> +tcg_gen_ext_i32_tl(cpu_gpr[rd], t2);
>>> +tcg_gen_ext_i32_tl(cpu_LO[acc], t2);
>>> +tcg_gen_ext_i32_tl(cpu_HI[acc], t3);
>>> +tcg_temp_free_i32(t2);
>>> +tcg_temp_free_i32(t3);
>>> +}
>>> +break;
>>> +case OPC_MULTU:
>>> +{
>>> +TCGv_i32 t2 = tcg_temp_new_i32();
>>> +TCGv_i32 t3 = tcg_temp_new_i32();
>>> +tcg_gen_trunc_tl_i32(t2, t0);
>>> +tcg_gen_trunc_tl_i32(t3, t1);
>>> +tcg_gen_mulu2_i32(t2, t3, t2, t3);
>>> +if (rd)
>>> +tcg_gen_ext_i32_tl(cpu_gpr[rd], t2);
>>> +tcg_gen_ext_i32_tl(cpu_LO[acc], t2);
>>> +tcg_gen_ext_i32_tl(cpu_HI[acc], t3);
>>> +tcg_temp_free_i32(t2);
>>> +tcg_temp_free_i32(t3);
>>> +}
>>> +break;
>>> +default:
>>> +MIPS_INVAL("mul R5900");
>>> +generate_exception_end(ctx, EXCP_RI);
>>> +goto out;
>>> +}
>>> +
>>> + out:
>>> +tcg_temp_free(t0);
>>> +tcg_temp_free(t1);
>>> +}
>>
>> Why not simply modify gen_muldiv?
> 
> Testing my own patch eh...
> 
> I missed the check_dsp() call:
> 
> if (acc != 0 && unlikely(!(ctx->insn_flags & INSN_R5900))) {
> check_dsp(ctx);
> }
> 
> Anyway if we plan to also add MADD/MADDU, using another function might
> be cleaner.
> 
>>
>> -- >8 --
>> @@ -3630,29 +3630,35 @@ static void gen_muldiv(DisasContext *ctx,
>> uint32_t opc,
>>  break;
>>  case OPC_MULT:
>>  {
>>  TCGv_i32 t2 = tcg_temp_new_i32();
>>  TCGv_i32 t3 = tcg_temp_new_i32();
>>  tcg_gen_trunc_tl_i32(t2, t0);
>>  tcg_gen_trunc_tl_i32(t3, t1);
>>  tcg_gen_muls2_i32(t2, t3, t2, t3);
>> +if (ctx->insn_flags & INSN_R5900 && rd) {
>> +tcg_gen_ext_i32_tl(cpu_gpr[rd], t2);
>> +}
>>  tcg_gen_ext_i32_tl(cpu_LO[acc], t2);
>>  tcg_gen_ext_i32_tl(cpu_HI[acc], t3);
>>  tcg_temp_free_i32(t2);
>>  tcg_temp_free_i32(t3);
>>  }
>>  break;
>>  case OPC_MULTU:
>>  {
>>  TCGv_i32 t2 = tcg_temp_new_i32();
>>  TCGv_i32 t3 = tcg_temp_new_i32();
>>  tcg_gen_trunc_tl_i32(t2, t0);
>>  tcg_gen_trunc_tl_i32(t3, t1);
>>  tcg_gen_mulu2_i32(t2, t3, t2, t3);
>> +if (ctx->insn_flags & INSN_R5900 && rd) {
>> +tcg_gen_ext_i32_tl(cpu_LO[acc], t2);
>> +}
>>  tcg_gen_ext_i32_tl(cpu_LO[acc], t2);
>>  tcg_gen_ext_i32_tl(cpu_HI[acc], t3);
>>  tcg_temp_free_i32(t2);
>>  tcg_temp_free_i32(t3);
>>  }
>>  break;
>> ---
>>
>>> +
>>>  static void gen_mul_vr54xx (DisasContext *ctx, uint32_t opc,
>>>  int rd, int rs, int rt)
>>>  {
>>> @@ -22378,6 +22429,8 @@ static void decode_opc_special_legacy(CPUMIPSState 
>>> *env, DisasContext *ctx)
>>>  check_insn(ctx, INSN_VR54XX);
>>>  op1 = MASK_MUL_VR54XX(ctx->opcode);
>>>  gen_mul_vr54xx(ctx, op1, rd, rs, rt);
>>> +} else if (ctx->insn_flags & INSN_R5900) {
>>> +gen_mul_r5900(ctx, op1, 0, rd, rs, rt);

And I now notice you take all the $rd bits, so my patch proposal should
also require:

} else if (ctx->insn_flags & INSN_R5900) {
gen_muldiv(ctx, op1, 0, rd, rs, rt);

Which a this point makes it pointless.

I'll restart reviewing your patch :)

>>>  } else {
>>>  gen_muldiv(ctx, op1, rd & 3, rs, rt);
>>>  }
>>>

Re: [Qemu-devel] [PATCH v5 2/8] target/mips: Support R5900 specific three-operand MULT and MULTU

[Philippe Mathieu-Daudé]

On 9/25/18 2:20 PM, Philippe Mathieu-Daudé wrote:
> Hi Fredrik,
> 
> On 9/15/18 11:25 AM, Fredrik Noring wrote:
>> Signed-off-by: Fredrik Noring 
>> ---
>>  target/mips/translate.c | 53 
>> +
>>  1 file changed, 53 insertions(+)
>>
>> diff --git a/target/mips/translate.c b/target/mips/translate.c
>> index ab16cdb911..fb571e278e 100644
>> --- a/target/mips/translate.c
>> +++ b/target/mips/translate.c
>> @@ -3768,6 +3768,57 @@ static void gen_muldiv(DisasContext *ctx, uint32_t 
>> opc,
>>  tcg_temp_free(t1);
>>  }
>>  
>> +static void gen_mul_r5900(DisasContext *ctx, uint32_t opc,
>> +  int acc, int rd, int rs, int rt)
>> +{
>> +TCGv t0 = tcg_temp_new();
>> +TCGv t1 = tcg_temp_new();
>> +
>> +gen_load_gpr(t0, rs);
>> +gen_load_gpr(t1, rt);
>> +
>> +switch (opc) {
>> +case OPC_MULT:
>> +{
>> +TCGv_i32 t2 = tcg_temp_new_i32();
>> +TCGv_i32 t3 = tcg_temp_new_i32();
>> +tcg_gen_trunc_tl_i32(t2, t0);
>> +tcg_gen_trunc_tl_i32(t3, t1);
>> +tcg_gen_muls2_i32(t2, t3, t2, t3);
>> +if (rd)
>> +tcg_gen_ext_i32_tl(cpu_gpr[rd], t2);
>> +tcg_gen_ext_i32_tl(cpu_LO[acc], t2);
>> +tcg_gen_ext_i32_tl(cpu_HI[acc], t3);
>> +tcg_temp_free_i32(t2);
>> +tcg_temp_free_i32(t3);
>> +}
>> +break;
>> +case OPC_MULTU:
>> +{
>> +TCGv_i32 t2 = tcg_temp_new_i32();
>> +TCGv_i32 t3 = tcg_temp_new_i32();
>> +tcg_gen_trunc_tl_i32(t2, t0);
>> +tcg_gen_trunc_tl_i32(t3, t1);
>> +tcg_gen_mulu2_i32(t2, t3, t2, t3);
>> +if (rd)
>> +tcg_gen_ext_i32_tl(cpu_gpr[rd], t2);
>> +tcg_gen_ext_i32_tl(cpu_LO[acc], t2);
>> +tcg_gen_ext_i32_tl(cpu_HI[acc], t3);
>> +tcg_temp_free_i32(t2);
>> +tcg_temp_free_i32(t3);
>> +}
>> +break;
>> +default:
>> +MIPS_INVAL("mul R5900");
>> +generate_exception_end(ctx, EXCP_RI);
>> +goto out;
>> +}
>> +
>> + out:
>> +tcg_temp_free(t0);
>> +tcg_temp_free(t1);
>> +}
> 
> Why not simply modify gen_muldiv?

Testing my own patch eh...

I missed the check_dsp() call:

if (acc != 0 && unlikely(!(ctx->insn_flags & INSN_R5900))) {
check_dsp(ctx);
}

Anyway if we plan to also add MADD/MADDU, using another function might
be cleaner.

> 
> -- >8 --
> @@ -3630,29 +3630,35 @@ static void gen_muldiv(DisasContext *ctx,
> uint32_t opc,
>  break;
>  case OPC_MULT:
>  {
>  TCGv_i32 t2 = tcg_temp_new_i32();
>  TCGv_i32 t3 = tcg_temp_new_i32();
>  tcg_gen_trunc_tl_i32(t2, t0);
>  tcg_gen_trunc_tl_i32(t3, t1);
>  tcg_gen_muls2_i32(t2, t3, t2, t3);
> +if (ctx->insn_flags & INSN_R5900 && rd) {
> +tcg_gen_ext_i32_tl(cpu_gpr[rd], t2);
> +}
>  tcg_gen_ext_i32_tl(cpu_LO[acc], t2);
>  tcg_gen_ext_i32_tl(cpu_HI[acc], t3);
>  tcg_temp_free_i32(t2);
>  tcg_temp_free_i32(t3);
>  }
>  break;
>  case OPC_MULTU:
>  {
>  TCGv_i32 t2 = tcg_temp_new_i32();
>  TCGv_i32 t3 = tcg_temp_new_i32();
>  tcg_gen_trunc_tl_i32(t2, t0);
>  tcg_gen_trunc_tl_i32(t3, t1);
>  tcg_gen_mulu2_i32(t2, t3, t2, t3);
> +if (ctx->insn_flags & INSN_R5900 && rd) {
> +tcg_gen_ext_i32_tl(cpu_LO[acc], t2);
> +}
>  tcg_gen_ext_i32_tl(cpu_LO[acc], t2);
>  tcg_gen_ext_i32_tl(cpu_HI[acc], t3);
>  tcg_temp_free_i32(t2);
>  tcg_temp_free_i32(t3);
>  }
>  break;
> ---
> 
>> +
>>  static void gen_mul_vr54xx (DisasContext *ctx, uint32_t opc,
>>  int rd, int rs, int rt)
>>  {
>> @@ -22378,6 +22429,8 @@ static void decode_opc_special_legacy(CPUMIPSState 
>> *env, DisasContext *ctx)
>>  check_insn(ctx, INSN_VR54XX);
>>  op1 = MASK_MUL_VR54XX(ctx->opcode);
>>  gen_mul_vr54xx(ctx, op1, rd, rs, rt);
>> +} else if (ctx->insn_flags & INSN_R5900) {
>> +gen_mul_r5900(ctx, op1, 0, rd, rs, rt);
>>  } else {
>>  gen_muldiv(ctx, op1, rd & 3, rs, rt);
>>  }
>>

Re: [Qemu-devel] [PATCH] virtio: do not take address of packed members

[Philippe Mathieu-Daudé]

On 9/26/18 11:20 PM, Paolo Bonzini wrote:
> The address of a packed member is not packed, which may cause accesses
> to unaligned pointers.  Avoid this by reading the packed value before
> passing it to another function.
> 
> Cc: Jason Wang 
> Cc: Peter Maydell 
> Reviewed-by: Eric Blake 
> Cc: qemu-sta...@nongnu.org
> Signed-off-by: Paolo Bonzini 
> ---
> Resubmission of the patch at 
> https://patchwork.kernel.org/patch/10160229/
> The vhost part has since been fixed independently (commit
> 69aff03064, "vhost-user: fix misaligned access to payload",
> 2018-01-18).
> 
>  hw/char/virtio-serial-bus.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/char/virtio-serial-bus.c b/hw/char/virtio-serial-bus.c
> index d2dd8ab502..04e3ebe352 100644
> --- a/hw/char/virtio-serial-bus.c
> +++ b/hw/char/virtio-serial-bus.c
> @@ -667,9 +667,9 @@ static void virtio_serial_save_device(VirtIODevice *vdev, 
> QEMUFile *f)
>  
>  /* The config space (ignored on the far end in current versions) */
>  get_config(vdev, (uint8_t *));
> -qemu_put_be16s(f, );
> -qemu_put_be16s(f, );
> -qemu_put_be32s(f, _nr_ports);
> +qemu_put_be16(f, config.cols);
> +qemu_put_be16(f, config.rows);
> +qemu_put_be32(f, config.max_nr_ports);

Clever.

Reviewed-by: Philippe Mathieu-Daudé 

>  
>  /* The ports map */
>  max_nr_ports = s->serial.max_virtserial_ports;
> 

[Qemu-devel] [PATCH] util/hbitmaps: recalculate count on merge

[John Snow]

We have been neglecting to do so, which results in wrong counts
after merge. In the worst case, we may think the bitmap is empty
when it has had new writes merged into it.

Reported-by: Eric Blake 
Signed-off-by: John Snow 
---
 util/hbitmap.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/util/hbitmap.c b/util/hbitmap.c
index d5aca5159f..28e9c523ab 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -759,6 +759,9 @@ bool hbitmap_merge(const HBitmap *a, const HBitmap *b, 
HBitmap *result)
 }
 }
 
+/* Recompute the dirty count */
+a->count = hb_count_between(a, 0, a->size - 1);
+
 return true;
 }
 
-- 
2.14.4

[Qemu-devel] [PATCH] virtio: do not take address of packed members

[Paolo Bonzini]

The address of a packed member is not packed, which may cause accesses
to unaligned pointers.  Avoid this by reading the packed value before
passing it to another function.

Cc: Jason Wang 
Cc: Peter Maydell 
Reviewed-by: Eric Blake 
Cc: qemu-sta...@nongnu.org
Signed-off-by: Paolo Bonzini 
---
Resubmission of the patch at 
https://patchwork.kernel.org/patch/10160229/
The vhost part has since been fixed independently (commit
69aff03064, "vhost-user: fix misaligned access to payload",
2018-01-18).

 hw/char/virtio-serial-bus.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/hw/char/virtio-serial-bus.c b/hw/char/virtio-serial-bus.c
index d2dd8ab502..04e3ebe352 100644
--- a/hw/char/virtio-serial-bus.c
+++ b/hw/char/virtio-serial-bus.c
@@ -667,9 +667,9 @@ static void virtio_serial_save_device(VirtIODevice *vdev, 
QEMUFile *f)
 
 /* The config space (ignored on the far end in current versions) */
 get_config(vdev, (uint8_t *));
-qemu_put_be16s(f, );
-qemu_put_be16s(f, );
-qemu_put_be32s(f, _nr_ports);
+qemu_put_be16(f, config.cols);
+qemu_put_be16(f, config.rows);
+qemu_put_be32(f, config.max_nr_ports);
 
 /* The ports map */
 max_nr_ports = s->serial.max_virtserial_ports;
-- 
2.17.1

[Qemu-devel] [Bug 1793904] Re: files are randomly overwritten by Zero Bytes

[John Snow]

Looks like you've narrowed it down to QEMU's usage of gluster, but some
additional details might still be nice just to be sure:

Can you produce the command-line for QEMU so we can see the
configuration you're running?

I believe on Debian that QEMU makes use of the glusterfs-common package,
can you post that version too?

Once you are witnessing files coming back as zeroes, if you shut the VM
down (cleanly if at all possible), are you able to run qemu-img check on
the qcow2 file? Do you see any errors?

And, as a debugging step, are you able to test with a raw file over
gluster:// to see if you can reproduce that way? I'm wondering if
there's an interaction with qcow2 and gluster that can maybe be
eliminated or highlighted here. (qcow2 might drive gluster in ways
differently than raw does, which might help narrow down the problem one
way or the other.)

--js

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1793904

Title:
  files are randomly overwritten by Zero Bytes

Status in QEMU:
  New

Bug description:
  Hello together,

  I am currently tracking down a "Hard to reproduce" bug on my systems
  that I first discovered during gitlab installation:

  
  Here is the Text from the Gitlab Bug 
https://gitlab.com/gitlab-org/gitlab-ce/issues/51023
  
--

  Steps to reproduce

  I still do not have all the steps together to reproduce, so far it is:
  apt install gitlab-ce and
  gitlab-rake backup:recovery
  Then it works for some time before it fails.

  What is the current bug behavior?

  I have a 12 hour old Installation of gitlab ce 11.2.3-ce.0 for debian
  stretch on a fresh debian stretch system together with our imported
  data. However it turns out that some gitlab related files contain Zero
  bytes instead of actual data.

  root@gitlab:~# xxd -l 16 /opt/gitlab/bin/gitlab-ctl
  :          

  This behaviour is somewhat strange because it was working for a few
  minutes/hours. I did write a shell script to find out which files are
  affected of this memory loss. It turns out that only files located
  under /opt/gitlab are affected, if I rule out files like
  /var/log/faillog and some postgresql table files.

  What I find even stranger is that it does not seem to affect
  Logfiles/databases/git_repositorys but application files, like .rb
  scripts. and not all of them. No non gitlab package is affected.

  What is the expected correct behavior?
  Binarys and .rb files should stay as they are.

  Possible fixes

  I am still investigating, I hope that it is not an infrastructure problem 
(libvirt/qemu/glusterfs) it can still be one but the point that files of 
/opt/gitlab are affected and not any logfile and that we to not have similar 
problems with any other system leads me to the application for now.
  If I would have used docker the same problem might have caused a reboot of 
the container.
  But for the Debian package it is a bit of work to recover. That is all a 
workaround, however.
  
-

  I do have found 2 more systems having the same problem with different
  software:

  root@erp:~# xxd -l 16 /usr/share/perl/5.26.2/constant.pm
  :          

  The Filesize itself is, compared with another machine 1660 Bytes
  for both the corrupted and the intact file. It looks to me from the
  outside that if some data in the qcow2 file is written too many bytes
  get written so it sometimes overwites data of existing files located
  right after the position in memory where the write goes to.

  I would like to rule out Linux+Ext4 filesystems because I find it
  highly unlikely that such an error keeps undiscovered in that part of
  the environment for long. I think the same might go for qemu.

  Which leaves qemu, gemu+gluster:// mount, qcow2 volumes, glusterfs,
  network. So I am now going to check if I can find any system which
  gets its volumes via fusermount instead of gluster:// path if the
  error is gone there. This may take a while.

  
  - some software versions---

  QEMU emulator version 2.12.0 (Debian 1:2.12+dfsg-3)
  Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

  libvirt-daemon-driver-storage-gluster/testing,unstable,now 4.6.0-2
  amd64 [installed]

  ii  glusterfs-client   4.1.3-1amd64

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1793904/+subscriptions

Re: [Qemu-devel] [PATCH 2/2] hw/pci-host/x86: extend the 64-bit PCI hole relative to the fw-assigned base

[Laszlo Ersek]

(+Eric)

On 09/26/18 18:26, Alex Williamson wrote:
> On Wed, 26 Sep 2018 13:12:47 +0200
> Laszlo Ersek  wrote:
> 
>> On 09/25/18 22:36, Alex Williamson wrote:
>>> On Tue, 25 Sep 2018 00:13:46 +0200
>>> Laszlo Ersek  wrote:
>>>   
 In commit 9fa99d2519cb ("hw/pci-host: Fix x86 Host Bridges 64bit PCI
 hole", 2017-11-16), we meant to expose such a 64-bit PCI MMIO aperture in
 the ACPI DSDT that would be at least as large as the new "pci-hole64-size"
 property (2GB on i440fx, 32GB on q35). The goal was to offer "enough"
 64-bit MMIO aperture to the guest OS for hotplug purposes.

 In that commit, we added or modified five functions:

 - pc_pci_hole64_start(): shared between i440fx and q35. Provides a default
   64-bit base, which starts beyond the cold-plugged 64-bit RAM, and skips
   the DIMM hotplug area too (if any).

 - i440fx_pcihost_get_pci_hole64_start(), q35_host_get_pci_hole64_start():
   board-specific 64-bit base property getters called abstractly by the
   ACPI generator. Both of these fall back to pc_pci_hole64_start() if the
   firmware didn't program any 64-bit hole (i.e. if the firmware didn't
   assign a 64-bit GPA to any MMIO BAR on any device). Otherwise, they
   honor the firmware's BAR assignments (i.e., they treat the lowest 64-bit
   GPA programmed by the firmware as the base address for the aperture).

 - i440fx_pcihost_get_pci_hole64_end(), q35_host_get_pci_hole64_end():
   these intended to extend the aperture to our size recommendation,
   calculated relative to the base of the aperture.

 Despite the original intent, i440fx_pcihost_get_pci_hole64_end() and
 q35_host_get_pci_hole64_end() currently only extend the aperture relative
 to the default base (pc_pci_hole64_start()), ignoring any programming done
 by the firmware. This means that our size recommendation may not be met.
 Fix it by honoring the firmware's address assignments.

 The strange extension sizes were spotted by Alex, in the log of a guest
 kernel running on top of OVMF (which does assign 64-bit GPAs to BARs).

 This change only affects DSDT generation, therefore no new compat property
 is being introduced. Also, because SeaBIOS never assigns 64-bit GPAs to
 64-bit BARs, the patch makes no difference to SeaBIOS guests.  
>>>
>>> This is not exactly true, SeaBIOS will make use of 64-bit MMIO, but
>>> only if it cannot satisfy all the BARs from 32-bit MMMIO, see
>>> src/fw/pciinit.c:pci_bios_map_devices.  Create a VM with several
>>> assigned GPUs and you'll eventually cross that threshold and all 64-bit
>>> BARs will be moved above 4G.  I'm sure a few sufficiently sized ivshmem
>>> devices could do the same.  Thanks,  
>>
>> The effect of this patch is not hard to demonstrate with SeaBIOS+Q35,
>> when using e.g. 5GB of guest RAM and a 4GB ivshmem-plain device.
>>
>> However, using SeaBIOS+i440fx, I can't show the difference. I've been
>> experimenting with various ivshmem devices (even multiple at the same
>> time, with different sizes). The "all or nothing" nature of SeaBIOS's
>> high allocation of the 64-bit BARs, combined with hugepage alignment
>> inside SeaBIOS, combined with the small (2GB) rounding size used in QEMU
>> for i440fx, seem to make it surprisingly difficult to trigger the issue.
>>
>> I figure I should:
>>
>> (1) remove the sentence "the patch makes no difference to SeaBIOS
>> guests" from the commit message,
>>
>> (2) include the DSDT diff on SeaBIOS/q35 in the commit message,
>>
>> (3) remain silent on SeaBIOS/i440fx, in the commit message,
>>
>> (4) append a new patch, for "bios-tables-test", so that the ACPI gen
>> change is validated as part of the test suite, on SeaBIOS/q35.
>>
>> Regarding (4):
>>
>> - is it OK if I add the test only for Q35?
>>
>> - what guest RAM size am I allowed to use in the test suite? In my own
>> SeaBIOS/Q35 reproducer I currently use 5GB, but I'm not sure if it's
>> acceptable for the test suite.
> 
> Seems like you've done due diligence, the plan looks ok to me.
> Regarding the test memory allocation, is it possible and reasonable to
> perhaps create a 256MB shared memory area and re-use it for multiple
> ivshmem devices?  ie. rather than 1, 4GB ivshmem device, use 16, 256MB
> devices, all with the same backing.  Thanks,

This too sounds useful. AIUI, ftruncate() is neither forbidden, nor
required, to allocate filesystem extents when increasing the size of a
file. Using one smaller regular temporary file as the common foundation
for multiple "memory-backend-file" objects will save space on the fs if
ftruncate() happens to allocate extents.

(I've also thought of passing the same "memory-backend-file" object to
multiple ivshmem-plain devices, but ivshmem_plain_realize()
[hw/misc/ivshmem.c] checks whether the HostMemoryBackend is already mapped.)

Thanks!
Laszlo

Re: [Qemu-devel] [PATCH 2/2] hw/pci-host/x86: extend the 64-bit PCI hole relative to the fw-assigned base

[Laszlo Ersek]

(+Eric)

On 09/26/18 14:10, Igor Mammedov wrote:
> On Wed, 26 Sep 2018 13:35:14 +0200
> Laszlo Ersek  wrote:
> 
>> On 09/26/18 13:12, Laszlo Ersek wrote:
>>
>>> (4) append a new patch, for "bios-tables-test", so that the ACPI gen
>>> change is validated as part of the test suite, on SeaBIOS/q35.
>>>
>>> Regarding (4):
>>>
>>> - is it OK if I add the test only for Q35?
>>>
>>> - what guest RAM size am I allowed to use in the test suite? In my own
>>> SeaBIOS/Q35 reproducer I currently use 5GB, but I'm not sure if it's
>>> acceptable for the test suite.  
>>
>> And, even if the patch's effect can be shown with little guest DRAM, the
>> test case still requires a multi-gig ivshmem-plain device. In
>> "tests/ivshmem-test.c", I see how it is set up -- the backend is set up
>> with shm_open(). The file created under /dev/shm (on Linux) might
>> require host RAM just the same as normal guest DRAM (especially with
>> memory overcommit disabled on the host), correct?
> with over commit disable or cgroups limits enforced (I'd expect that
> in automated testing env i.e. travis or something else)
> allocating such amount of RAM probably would fail like crazy.
> 
> Maybe using memdev file backend with manually created sparse file
> might actually work (with preallocate disabled)

Thanks, this sounds like a good idea.

I see shm_open() is used heavily in ivshmem-related tests. I haven't
looked much at shm_open() before. (I've always known it existed in
POSIX, but I've never cared.)

So now I first checked what shm_open() would give me over a regular
temporary file created with open(); after all, the file descriptor
returned by either would have to be mmap()'d. From the rationale in POSIX:

,

it seems like the {shm_open(), mmap()} combo has two significant
guarantees over {open(), mmap()}:

- the namespace may be distinct (there need not be a writeable
  filesystem at all),

- the shared object will *always* be locked in RAM ("Shared memory is
  not just simply providing common access to data, it is providing the
  fastest possible communication between the processes").

The rationale seems to permit, on purpose, an shm_open() implementation
that is actually based on open(), using a special file system -- and
AIUI, /dev/shm is just that, on Linux.

Eric, does the above sound more or less correct?

If it is correct, then I think shm_open() is exactly what I *don't* want
for this use case. Because, while I do need a pathname for an
mmap()-able object (regular file, or otherwise), just so I can do:

  -object memory-backend-file,id=mem-obj,...,mem-path=... \
  -device ivshmem-plain,memdev=mem-obj,...

, I want the underlying object to put as little pressure on the system
that runs the test suite as possible.

This means I should specifically ask for a regular file, to be mmap()'d
(with MAP_SHARED). Then the kernel knows in advance that it can always
page out the dirty stuff, and the mapping shouldn't clash with cgroups,
or disabled memory overcommit.

Now, in order to make that actually safe, I should in theory ask for
preallocation on the filesystem (otherwise, if the filesystem runs out
of space, while the kernel is allocating fs extents in order to flush
the dirty pages to them, the process gets a SIGBUS, IIRC). However,
because I know that nothing will be in fact dirtied, I can minimize the
footprint on the filesystem as well, and forego preallocation too.

This suggests that, in my test case,
- I call g_file_open_tmp() for creating the temporary file,
- pass the returned fd to ftruncate() for resizing the temporary file,
- pass the returned pathname to the "memory-backend-file" objects, in
  the "mem-path" property,
- set "share=on",
- set "prealloc=off",
- "discard-data" is irrelevant (there won't be any dirty pages).

Thanks
Laszlo

Re: [Qemu-devel] [resend PATCH v2] qga-win: add support for qmp_guest_fsfreeze_freeze_list

[Michael Roth]

Quoting Chen Hanxiao (2018-09-18 21:52:07)
> From: Chen Hanxiao 
> 
> This patch add support for freeze specified fs.
> 
> The valid mountpoints list member are [1]:
> 
>   The path of a mounted folder, for example, Y:\MountX\
>   A drive letter, for example, D:\
>   A volume GUID path of the form \\?\Volume{GUID}\,
>   where GUID identifies the volume
>   A UNC path that specifies a remote file share,
>   for example, \\Clusterx\Share1\
> 
> [1] 
> https://docs.microsoft.com/en-us/windows/desktop/api/vsbackup/nf-vsbackup-ivssbackupcomponents-addtosnapshotset
> 
> Cc: Michael Roth 
> Signed-off-by: Chen Hanxiao 

Already applied previous submission:

  https://lists.gnu.org/archive/html/qemu-devel/2018-09/msg00395.html

Will send a pull soon.

> ---
> v2:
>   optimize internal logic blocks
> 
>  qga/commands-win32.c| 21 -
>  qga/main.c  |  2 +-
>  qga/vss-win32.c |  5 +-
>  qga/vss-win32.h |  3 +-
>  qga/vss-win32/requester.cpp | 92 ++---
>  qga/vss-win32/requester.h   | 13 --
>  6 files changed, 91 insertions(+), 45 deletions(-)
> 
> diff --git a/qga/commands-win32.c b/qga/commands-win32.c
> index 98d9735389..1d627f73c1 100644
> --- a/qga/commands-win32.c
> +++ b/qga/commands-win32.c
> @@ -776,6 +776,13 @@ GuestFsfreezeStatus qmp_guest_fsfreeze_status(Error 
> **errp)
>   * The frozen state is limited for up to 10 seconds by VSS.
>   */
>  int64_t qmp_guest_fsfreeze_freeze(Error **errp)
> +{
> +return qmp_guest_fsfreeze_freeze_list(false, NULL, errp);
> +}
> +
> +int64_t qmp_guest_fsfreeze_freeze_list(bool has_mountpoints,
> +   strList *mountpoints,
> +   Error **errp)
>  {
>  int i;
>  Error *local_err = NULL;
> @@ -790,7 +797,7 @@ int64_t qmp_guest_fsfreeze_freeze(Error **errp)
>  /* cannot risk guest agent blocking itself on a write in this state */
>  ga_set_frozen(ga_state);
> 
> -qga_vss_fsfreeze(, true, _err);
> +qga_vss_fsfreeze(, true, mountpoints, _err);
>  if (local_err) {
>  error_propagate(errp, local_err);
>  goto error;
> @@ -808,15 +815,6 @@ error:
>  return 0;
>  }
> 
> -int64_t qmp_guest_fsfreeze_freeze_list(bool has_mountpoints,
> -   strList *mountpoints,
> -   Error **errp)
> -{
> -error_setg(errp, QERR_UNSUPPORTED);
> -
> -return 0;
> -}
> -
>  /*
>   * Thaw local file systems using Volume Shadow-copy Service.
>   */
> @@ -829,7 +827,7 @@ int64_t qmp_guest_fsfreeze_thaw(Error **errp)
>  return 0;
>  }
> 
> -qga_vss_fsfreeze(, false, errp);
> +qga_vss_fsfreeze(, false, NULL, errp);
> 
>  ga_unset_frozen(ga_state);
>  return i;
> @@ -1646,7 +1644,6 @@ GList *ga_command_blacklist_init(GList *blacklist)
>  "guest-set-vcpus",
>  "guest-get-memory-blocks", "guest-set-memory-blocks",
>  "guest-get-memory-block-size",
> -"guest-fsfreeze-freeze-list",
>  NULL};
>  char **p = (char **)list_unsupported;
> 
> diff --git a/qga/main.c b/qga/main.c
> index 6d70242d05..13bfff5f0a 100644
> --- a/qga/main.c
> +++ b/qga/main.c
> @@ -151,7 +151,7 @@ static void quit_handler(int sig)
>  WaitForSingleObject(hEventTimeout, 0);
>  CloseHandle(hEventTimeout);
>  }
> -qga_vss_fsfreeze(, false, );
> +qga_vss_fsfreeze(, false, NULL, );
>  if (err) {
>  g_debug("Error unfreezing filesystems prior to exiting: %s",
>  error_get_pretty(err));
> diff --git a/qga/vss-win32.c b/qga/vss-win32.c
> index a541f3ae01..f444a25a70 100644
> --- a/qga/vss-win32.c
> +++ b/qga/vss-win32.c
> @@ -147,7 +147,8 @@ void ga_uninstall_vss_provider(void)
>  }
> 
>  /* Call VSS requester and freeze/thaw filesystems and applications */
> -void qga_vss_fsfreeze(int *nr_volume, bool freeze, Error **errp)
> +void qga_vss_fsfreeze(int *nr_volume, bool freeze,
> +  strList *mountpoints, Error **errp)
>  {
>  const char *func_name = freeze ? "requester_freeze" : "requester_thaw";
>  QGAVSSRequesterFunc func;
> @@ -164,5 +165,5 @@ void qga_vss_fsfreeze(int *nr_volume, bool freeze, Error 
> **errp)
>  return;
>  }
> 
> -func(nr_volume, );
> +func(nr_volume, mountpoints, );
>  }
> diff --git a/qga/vss-win32.h b/qga/vss-win32.h
> index 4f8e39aa5c..ce2abe5a72 100644
> --- a/qga/vss-win32.h
> +++ b/qga/vss-win32.h
> @@ -22,6 +22,7 @@ bool vss_initialized(void);
>  int ga_install_vss_provider(void);
>  void ga_uninstall_vss_provider(void);
> 
> -void qga_vss_fsfreeze(int *nr_volume, bool freeze, Error **errp);
> +void qga_vss_fsfreeze(int *nr_volume, bool freeze,
> +  strList *mountpints, Error **errp);
> 
>  #endif
> diff --git a/qga/vss-win32/requester.cpp b/qga/vss-win32/requester.cpp
> index 3d9c9716c0..5378c55d23 100644
> --- 

Re: [Qemu-devel] [PATCH v3 0/5] dirty-bitmaps: fix QMP command permissions

[John Snow]

On 09/26/2018 08:19 AM, Vladimir Sementsov-Ogievskiy wrote:
> 26.09.2018 02:49, John Snow wrote:
>> based on: jsnow/bitmaps staging branch
>>
>> This series builds on a previous standalone patch and adjusts
>> the permission for all (or most) of the QMP bitmap commands.
>>
>> John Snow (5):
>>    block/dirty-bitmaps: add user_modifiable status checker
>>    block/dirty-bitmaps: fix merge permissions
>>    block/dirty-bitmaps: allow clear on disabled bitmaps
>>    block/dirty-bitmaps: prohibit enable/disable on locked/frozen bitmaps
>>    block/backup: prohibit backup from using in-use bitmaps
>>
>>   block/dirty-bitmap.c | 13 +---
>>   blockdev.c   | 75
>> 
>>   include/block/dirty-bitmap.h |  1 +
>>   3 files changed, 44 insertions(+), 45 deletions(-)
>>
> 
> Great! Thank you for clearing this. I contributed a lot to this mess
> with my qmp-locked(
> 

It happens. I should have caught it too, but I think the ramifications
here aren't so bad.

> PS: I have a draft patch in my current developments to allow set/reset
> bits in disabled bitmaps, which is needed to use BdrvDirtyBitmap as a
> shared named copy-bitmap between fleecing-hook filter and backup job. So
> "disabled" is actually only for use in bdrv_set_dirty(), to disable
> automatic bitmap updates on guest writes.
> 

OK, I'll keep an eye out for the series when it comes.

Thanks for your reviews, touched up commit message on #2 to reflect that
you had already fixed the problem, and staged to my bitmaps branch:

Thanks, applied to my bitmaps tree:

https://github.com/jnsnow/qemu/commits/bitmaps
https://github.com/jnsnow/qemu.git

--js

Re: [Qemu-devel] QEMU and Kconfig

[Peter Maydell]

On 26 September 2018 at 14:36, Paolo Bonzini  wrote:
> Here is a minimal example:
>
> # hw/scsi/Kconfig
> config SCSI
>
> config ESP
> select SCSI
>
> config ESP_PCI
> default y
> select ESP
> depends on PCI
>
> # hw/pci/Kconfig
> config PCI
>
> # hw/pci-host/Kconfig
> config PCI_GENERIC
> select PCI
>
> # hw/arm/Kconfig
> config ARM_VIRT
> select PCI_GENERIC
> default y
>
> # hw/sparc/Kconfig
> config SUN4M
> select ESP
> default y

What is the syntactic thing in this example which distinguishes
"user can toggle this" (ESP_PCI, ARM_VIRT, SUN4M) from "user
can't toggle this, it's just an internal thing selected by
other nodes" (the rest) ? I'm assuming we'd have some sort
of UI thingy that presents the user only with the user-settable
options.

thanks
-- PMM

[Qemu-devel] [PATCH v2 14/15] target/arm: Rewrite vector gather first-fault loads

[Richard Henderson]

This implements the feature for softmmu, and moves the
main loop out of a macro and into a function.

Reviewed-by: Peter Maydell 
Tested-by: Laurent Desnogues 
Signed-off-by: Richard Henderson 
---
 target/arm/helper-sve.h|  84 ---
 target/arm/sve_helper.c| 290 +++--
 target/arm/translate-sve.c |  84 +--
 3 files changed, 321 insertions(+), 137 deletions(-)

diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h
index 6b9b93af45..9e79182ab4 100644
--- a/target/arm/helper-sve.h
+++ b/target/arm/helper-sve.h
@@ -1401,69 +1401,111 @@ DEF_HELPER_FLAGS_6(sve_ldsds_be_zd, TCG_CALL_NO_WG,
 
 DEF_HELPER_FLAGS_6(sve_ldffbsu_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldffhsu_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldffhsu_le_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldffssu_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldffhsu_be_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldffss_le_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldffss_be_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 DEF_HELPER_FLAGS_6(sve_ldffbss_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldffhss_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldffhss_le_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldffhss_be_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 
 DEF_HELPER_FLAGS_6(sve_ldffbsu_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldffhsu_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldffhsu_le_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldffssu_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldffhsu_be_zss, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldffss_le_zss, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldffss_be_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 DEF_HELPER_FLAGS_6(sve_ldffbss_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldffhss_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldffhss_le_zss, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldffhss_be_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 
 DEF_HELPER_FLAGS_6(sve_ldffbdu_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldffhdu_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldffhdu_le_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldffsdu_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldffhdu_be_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldffddu_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldffsdu_le_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldffsdu_be_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldffdd_le_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldffdd_be_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 DEF_HELPER_FLAGS_6(sve_ldffbds_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldffhds_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldffhds_le_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldffsds_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldffhds_be_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldffsds_le_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldffsds_be_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 
 DEF_HELPER_FLAGS_6(sve_ldffbdu_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldffhdu_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldffhdu_le_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldffsdu_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldffhdu_be_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldffddu_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldffsdu_le_zss, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldffsdu_be_zss, TCG_CALL_NO_WG,
+  

Re: [Qemu-devel] [PATCH v3 2/5] block/dirty-bitmaps: fix merge permissions

[John Snow]

On 09/26/2018 08:08 AM, Vladimir Sementsov-Ogievskiy wrote:
> 26.09.2018 14:55, Vladimir Sementsov-Ogievskiy wrote:
>> 26.09.2018 02:49, John Snow wrote:
>>> We wish to prohibit merging to read-only bitmaps and frozen bitmaps,
>>> but "disabled" bitmaps only preclude their recording of live, new
>>> information. It does not prohibit them from manual writes at the behest
>>> of the user, as is the case for merge operations.
>>>
>>> Allow the merge to "disabled" bitmaps,
>>> and prohibit merging to "locked" ones.
>>
>> only the second part is here..
> 
> Hm, the first one is in first separate patch? With commit message fixed
> to only second part, of course:
> 
> 
> Reviewed-by: Vladimir Sementsov-Ogievskiy 
> 

Ah, you actually changed it yourself in
b3024909c4b62a989261c288c797f86b150c43fb, for the merge transaction
series -- it just goes unmentioned.

When I rebased on top of the staging branch I lost that change as it was
already made.

I'll update the commit message.

--js

>>
>>>
>>> Reported-by: Eric Blake 
>>> Signed-off-by: John Snow 
>>> ---
>>>   block/dirty-bitmap.c | 6 +++---
>>>   1 file changed, 3 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
>>> index fc10543ab0..53b7d282c4 100644
>>> --- a/block/dirty-bitmap.c
>>> +++ b/block/dirty-bitmap.c
>>> @@ -806,9 +806,9 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap
>>> *dest, const BdrvDirtyBitmap *src,
>>>     qemu_mutex_lock(dest->mutex);
>>>   -    if (bdrv_dirty_bitmap_frozen(dest)) {
>>> -    error_setg(errp, "Bitmap '%s' is frozen and cannot be
>>> modified",
>>> -   dest->name);
>>> +    if (!bdrv_dirty_bitmap_user_modifiable(dest)) {
>>> +    error_setg(errp, "Bitmap '%s' is currently in-use by another"
>>> +    " operation and cannot be modified", dest->name);
>>>   goto out;
>>>   }
>>
>>
> 
> 

[Qemu-devel] [PATCH v2 15/15] target/arm: Pass TCGMemOpIdx to sve memory helpers

[Richard Henderson]

There is quite a lot of code required to compute cpu_mem_index,
or even put together the full TCGMemOpIdx.  This can easily be
done at translation time.

Reviewed-by: Peter Maydell 
Tested-by: Laurent Desnogues 
Signed-off-by: Richard Henderson 
---
 target/arm/internals.h |   5 ++
 target/arm/sve_helper.c| 138 +++--
 target/arm/translate-sve.c |  67 +++---
 3 files changed, 121 insertions(+), 89 deletions(-)

diff --git a/target/arm/internals.h b/target/arm/internals.h
index dc9357766c..24c0444c8d 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -796,4 +796,9 @@ static inline uint32_t arm_debug_exception_fsr(CPUARMState 
*env)
 }
 }
 
+/* Note make_memop_idx reserves 4 bits for mmu_idx, and MO_BSWAP is bit 3.
+ * Thus a TCGMemOpIdx, without any MO_ALIGN bits, fits in 8 bits.
+ */
+#define MEMOPIDX_SHIFT  8
+
 #endif
diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c
index 7756c0b098..8cbc6516ab 100644
--- a/target/arm/sve_helper.c
+++ b/target/arm/sve_helper.c
@@ -19,6 +19,7 @@
 
 #include "qemu/osdep.h"
 #include "cpu.h"
+#include "internals.h"
 #include "exec/exec-all.h"
 #include "exec/cpu_ldst.h"
 #include "exec/helper-proto.h"
@@ -3990,7 +3991,7 @@ typedef intptr_t sve_ld1_host_fn(void *vd, void *vg, void 
*host,
  * The controlling predicate is known to be true.
  */
 typedef void sve_ld1_tlb_fn(CPUARMState *env, void *vd, intptr_t reg_off,
-target_ulong vaddr, int mmu_idx, uintptr_t ra);
+target_ulong vaddr, TCGMemOpIdx oi, uintptr_t ra);
 typedef sve_ld1_tlb_fn sve_st1_tlb_fn;
 
 /*
@@ -4017,16 +4018,15 @@ static intptr_t sve_##NAME##_host(void *vd, void *vg, 
void *host,   \
 #ifdef CONFIG_SOFTMMU
 #define DO_LD_TLB(NAME, H, TYPEE, TYPEM, HOST, MOEND, TLB) \
 static void sve_##NAME##_tlb(CPUARMState *env, void *vd, intptr_t reg_off,  \
- target_ulong addr, int mmu_idx, uintptr_t ra)  \
+ target_ulong addr, TCGMemOpIdx oi, uintptr_t ra)  
\
 {   \
-TCGMemOpIdx oi = make_memop_idx(ctz32(sizeof(TYPEM)) | MOEND, mmu_idx); \
 TYPEM val = TLB(env, addr, oi, ra); \
 *(TYPEE *)(vd + H(reg_off)) = val;  \
 }
 #else
 #define DO_LD_TLB(NAME, H, TYPEE, TYPEM, HOST, MOEND, TLB)  \
 static void sve_##NAME##_tlb(CPUARMState *env, void *vd, intptr_t reg_off,  \
- target_ulong addr, int mmu_idx, uintptr_t ra)  \
+ target_ulong addr, TCGMemOpIdx oi, uintptr_t ra)  
\
 {   \
 TYPEM val = HOST(g2h(addr));\
 *(TYPEE *)(vd + H(reg_off)) = val;  \
@@ -4154,11 +4154,13 @@ static void sve_ld1_r(CPUARMState *env, void *vg, const 
target_ulong addr,
   sve_ld1_host_fn *host_fn,
   sve_ld1_tlb_fn *tlb_fn)
 {
-void *vd = >vfp.zregs[simd_data(desc)];
+const TCGMemOpIdx oi = extract32(desc, SIMD_DATA_SHIFT, MEMOPIDX_SHIFT);
+const int mmu_idx = get_mmuidx(oi);
+const unsigned rd = extract32(desc, SIMD_DATA_SHIFT + MEMOPIDX_SHIFT, 5);
+void *vd = >vfp.zregs[rd];
 const int diffsz = esz - msz;
 const intptr_t reg_max = simd_oprsz(desc);
 const intptr_t mem_max = reg_max >> diffsz;
-const int mmu_idx = cpu_mmu_index(env, false);
 ARMVectorReg scratch;
 void *host;
 intptr_t split, reg_off, mem_off;
@@ -4232,7 +4234,7 @@ static void sve_ld1_r(CPUARMState *env, void *vg, const 
target_ulong addr,
  * on I/O memory, it may succeed but not bring in the TLB entry.
  * But even then we have still made forward progress.
  */
-tlb_fn(env, , reg_off, addr + mem_off, mmu_idx, retaddr);
+tlb_fn(env, , reg_off, addr + mem_off, oi, retaddr);
 reg_off += 1 << esz;
 }
 #endif
@@ -4293,9 +4295,9 @@ static void sve_ld2_r(CPUARMState *env, void *vg, 
target_ulong addr,
   uint32_t desc, int size, uintptr_t ra,
   sve_ld1_tlb_fn *tlb_fn)
 {
-const int mmu_idx = cpu_mmu_index(env, false);
+const TCGMemOpIdx oi = extract32(desc, SIMD_DATA_SHIFT, MEMOPIDX_SHIFT);
+const unsigned rd = extract32(desc, SIMD_DATA_SHIFT + MEMOPIDX_SHIFT, 5);
 intptr_t i, oprsz = simd_oprsz(desc);
-unsigned rd = simd_data(desc);
 ARMVectorReg scratch[2] = { };
 
 set_helper_retaddr(ra);
@@ -4303,8 +4305,8 @@ static void sve_ld2_r(CPUARMState *env, void *vg, 
target_ulong addr,
 uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3));
 do {
 if (pg & 1) {
-tlb_fn(env, [0], i, addr, mmu_idx, ra);
-tlb_fn(env, [1], i, addr + size, 

[Qemu-devel] [PATCH v2 07/15] target/arm: Rewrite helper_sve_ld1*_r using pages

[Richard Henderson]

Uses tlb_vaddr_to_host for correct operation with softmmu.
Optimize for accesses within a single page or pair of pages.

Signed-off-by: Richard Henderson 
---
 target/arm/sve_helper.c | 731 +++-
 1 file changed, 569 insertions(+), 162 deletions(-)

diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c
index 0f98097253..d628978431 100644
--- a/target/arm/sve_helper.c
+++ b/target/arm/sve_helper.c
@@ -1688,6 +1688,47 @@ static void swap_memmove(void *vd, void *vs, size_t n)
 }
 }
 
+/* Similarly for memset of 0.  */
+static void swap_memzero(void *vd, size_t n)
+{
+uintptr_t d = (uintptr_t)vd;
+uintptr_t o = (d | n) & 7;
+size_t i;
+
+/* Usually, the first bit of a predicate is set, so N is 0.  */
+if (likely(n == 0)) {
+return;
+}
+
+#ifndef HOST_WORDS_BIGENDIAN
+o = 0;
+#endif
+switch (o) {
+case 0:
+memset(vd, 0, n);
+break;
+
+case 4:
+for (i = 0; i < n; i += 4) {
+*(uint32_t *)H1_4(d + i) = 0;
+}
+break;
+
+case 2:
+case 6:
+for (i = 0; i < n; i += 2) {
+*(uint16_t *)H1_2(d + i) = 0;
+}
+break;
+
+default:
+for (i = 0; i < n; i++) {
+*(uint8_t *)H1(d + i) = 0;
+}
+break;
+}
+}
+
 void HELPER(sve_ext)(void *vd, void *vn, void *vm, uint32_t desc)
 {
 intptr_t opr_sz = simd_oprsz(desc);
@@ -3927,32 +3968,323 @@ void HELPER(sve_fcmla_zpzzz_d)(CPUARMState *env, void 
*vg, uint32_t desc)
 /*
  * Load contiguous data, protected by a governing predicate.
  */
-#define DO_LD1(NAME, FN, TYPEE, TYPEM, H)  \
-static void do_##NAME(CPUARMState *env, void *vd, void *vg, \
-  target_ulong addr, intptr_t oprsz,   \
-  uintptr_t ra)\
-{  \
-intptr_t i = 0;\
-do {   \
-uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3));\
-do {   \
-TYPEM m = 0;   \
-if (pg & 1) {  \
-m = FN(env, addr, ra); \
-}  \
-*(TYPEE *)(vd + H(i)) = m; \
-i += sizeof(TYPEE), pg >>= sizeof(TYPEE);  \
-addr += sizeof(TYPEM); \
-} while (i & 15);  \
-} while (i < oprsz);   \
-}  \
-void HELPER(NAME)(CPUARMState *env, void *vg,  \
-  target_ulong addr, uint32_t desc)\
-{  \
-do_##NAME(env, >vfp.zregs[simd_data(desc)], vg,   \
-  addr, simd_oprsz(desc), GETPC());\
+
+/*
+ * Load elements into @vd, controlled by @vg, from @host + @mem_ofs.
+ * Memory is valid through @host + @mem_max.  The register element
+ * indicies are inferred from @mem_ofs, as modified by the types for
+ * which the helper is built.  Return the @mem_ofs of the first element
+ * not loaded (which is @mem_max if they are all loaded).
+ *
+ * For softmmu, we have fully validated the guest page.  For user-only,
+ * we cannot fully validate without taking the mmap lock, but since we
+ * know the access is within one host page, if any access is valid they
+ * all must be valid.  However, when @vg is all false, it may be that
+ * no access is valid.
+ */
+typedef intptr_t sve_ld1_host_fn(void *vd, void *vg, void *host,
+ intptr_t mem_ofs, intptr_t mem_max);
+
+/*
+ * Load one element into @vd + @reg_off from (@env, @vaddr, @ra).
+ * The controlling predicate is known to be true.
+ */
+typedef void sve_ld1_tlb_fn(CPUARMState *env, void *vd, intptr_t reg_off,
+target_ulong vaddr, int mmu_idx, uintptr_t ra);
+
+/*
+ * Generate the above primitives.
+ */
+
+#define DO_LD_HOST(NAME, H, TYPEE, TYPEM, HOST) \
+static intptr_t sve_##NAME##_host(void *vd, void *vg, void *host,   \
+  intptr_t mem_off, const intptr_t mem_max) \
+{   \
+intptr_t reg_off = mem_off * (sizeof(TYPEE) / sizeof(TYPEM));   \
+uint64_t *pg = vg;  \
+while (mem_off + sizeof(TYPEM) <= mem_max) {\
+TYPEM val = 0;  \
+if (likely((pg[reg_off >> 6] >> (reg_off & 63)) & 1)) { \
+val = HOST(host + mem_off); \

[Qemu-devel] [PATCH v2 09/15] target/arm: Rewrite helper_sve_st[1234]*_r

[Richard Henderson]

This fixes the endianness problem for softmmu, and moves the
main loop out of a macro and into an inlined function.

Reviewed-by: Peter Maydell 
Tested-by: Laurent Desnogues 
Signed-off-by: Richard Henderson 
---
 target/arm/sve_helper.c | 351 
 1 file changed, 172 insertions(+), 179 deletions(-)

diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c
index f712b382f8..0b1e06823b 100644
--- a/target/arm/sve_helper.c
+++ b/target/arm/sve_helper.c
@@ -3991,6 +3991,7 @@ typedef intptr_t sve_ld1_host_fn(void *vd, void *vg, void 
*host,
  */
 typedef void sve_ld1_tlb_fn(CPUARMState *env, void *vd, intptr_t reg_off,
 target_ulong vaddr, int mmu_idx, uintptr_t ra);
+typedef sve_ld1_tlb_fn sve_st1_tlb_fn;
 
 /*
  * Generate the above primitives.
@@ -4668,214 +4669,206 @@ DO_LDFF1_LDNF1_2(dd,  3, 3)
 /*
  * Store contiguous data, protected by a governing predicate.
  */
-#define DO_ST1(NAME, FN, TYPEE, TYPEM, H)  \
-void HELPER(NAME)(CPUARMState *env, void *vg,  \
-  target_ulong addr, uint32_t desc)\
-{  \
-intptr_t i, oprsz = simd_oprsz(desc);  \
-intptr_t ra = GETPC(); \
-unsigned rd = simd_data(desc); \
-void *vd = >vfp.zregs[rd];\
-for (i = 0; i < oprsz; ) { \
-uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3));\
-do {   \
-if (pg & 1) {  \
-TYPEM m = *(TYPEE *)(vd + H(i));   \
-FN(env, addr, m, ra);  \
-}  \
-i += sizeof(TYPEE), pg >>= sizeof(TYPEE);  \
-addr += sizeof(TYPEM); \
-} while (i & 15);  \
-}  \
+
+#ifdef CONFIG_SOFTMMU
+#define DO_ST_TLB(NAME, H, TYPEM, HOST, MOEND, TLB) \
+static void sve_##NAME##_tlb(CPUARMState *env, void *vd, intptr_t reg_off,  \
+ target_ulong addr, int mmu_idx, uintptr_t ra)  \
+{   \
+TCGMemOpIdx oi = make_memop_idx(ctz32(sizeof(TYPEM)) | MOEND, mmu_idx); \
+TLB(env, addr, *(TYPEM *)(vd + H(reg_off)), oi, ra);\
 }
-
-#define DO_ST1_D(NAME, FN, TYPEM)  \
-void HELPER(NAME)(CPUARMState *env, void *vg,  \
-  target_ulong addr, uint32_t desc)\
-{  \
-intptr_t i, oprsz = simd_oprsz(desc) / 8;  \
-intptr_t ra = GETPC(); \
-unsigned rd = simd_data(desc); \
-uint64_t *d = >vfp.zregs[rd].d[0];\
-uint8_t *pg = vg;  \
-for (i = 0; i < oprsz; i += 1) {   \
-if (pg[H1(i)] & 1) {   \
-FN(env, addr, d[i], ra);   \
-}  \
-addr += sizeof(TYPEM); \
-}  \
+#else
+#define DO_ST_TLB(NAME, H, TYPEM, HOST, MOEND, TLB) \
+static void sve_##NAME##_tlb(CPUARMState *env, void *vd, intptr_t reg_off,  \
+ target_ulong addr, int mmu_idx, uintptr_t ra)  \
+{   \
+HOST(g2h(addr), *(TYPEM *)(vd + H(reg_off)));   \
 }
+#endif
 
-#define DO_ST2(NAME, FN, TYPEE, TYPEM, H)  \
-void HELPER(NAME)(CPUARMState *env, void *vg,  \
-  target_ulong addr, uint32_t desc)\
-{  \
-intptr_t i, oprsz = simd_oprsz(desc);  \
-intptr_t ra = GETPC(); \
-unsigned rd = simd_data(desc); \
-void *d1 = >vfp.zregs[rd];\
-void *d2 = >vfp.zregs[(rd + 1) & 31]; \
-for (i = 0; i < oprsz; ) { \
-uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3));\
-do {   \
-if (pg & 1) {  \
-TYPEM m1 = *(TYPEE *)(d1 + H(i));  \
-TYPEM m2 = *(TYPEE *)(d2 + H(i));  \
-FN(env, addr, m1, ra); \
-FN(env, addr + sizeof(TYPEM), m2, ra); \
-}  

[Qemu-devel] [PATCH v2 12/15] target/arm: Rewrite vector gather loads

[Richard Henderson]

This fixes the endianness problem for softmmu, and moves
the main loop out of a macro and into an inlined function.

Reviewed-by: Peter Maydell 
Tested-by: Laurent Desnogues 
Signed-off-by: Richard Henderson 
---
 target/arm/helper-sve.h|  84 +
 target/arm/sve_helper.c| 225 --
 target/arm/translate-sve.c | 244 +
 3 files changed, 386 insertions(+), 167 deletions(-)

diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h
index 1ad043101a..49d1c09e30 100644
--- a/target/arm/helper-sve.h
+++ b/target/arm/helper-sve.h
@@ -1292,69 +1292,111 @@ DEF_HELPER_FLAGS_4(sve_st1sd_be_r, TCG_CALL_NO_WG, 
void, env, ptr, tl, i32)
 
 DEF_HELPER_FLAGS_6(sve_ldbsu_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldhsu_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldhsu_le_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldssu_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldhsu_be_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldss_le_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldss_be_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 DEF_HELPER_FLAGS_6(sve_ldbss_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldhss_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldhss_le_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldhss_be_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 
 DEF_HELPER_FLAGS_6(sve_ldbsu_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldhsu_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldhsu_le_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldssu_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldhsu_be_zss, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldss_le_zss, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldss_be_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 DEF_HELPER_FLAGS_6(sve_ldbss_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldhss_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldhss_le_zss, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldhss_be_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 
 DEF_HELPER_FLAGS_6(sve_ldbdu_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldhdu_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldhdu_le_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldsdu_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldhdu_be_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldddu_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldsdu_le_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldsdu_be_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_lddd_le_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_lddd_be_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 DEF_HELPER_FLAGS_6(sve_ldbds_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldhds_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldhds_le_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldsds_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldhds_be_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldsds_le_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldsds_be_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 
 DEF_HELPER_FLAGS_6(sve_ldbdu_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldhdu_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldhdu_le_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldsdu_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldhdu_be_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_ldddu_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_ldsdu_le_zss, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_ldsdu_be_zss, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, 

[Qemu-devel] [PATCH v2 08/15] target/arm: Rewrite helper_sve_ld[234]*_r

[Richard Henderson]

Use the same *_tlb primitives as we use for ld1.

For linux-user, this hoists the set of helper_retaddr.  For softmmu,
hoists the computation of the current mmu_idx outside the loop,
fixes the endianness problem, and moves the main loop out of a
macro and into an inlined function.

Reviewed-by: Peter Maydell 
Tested-by: Laurent Desnogues 
Signed-off-by: Richard Henderson 
---
 target/arm/sve_helper.c | 210 ++--
 1 file changed, 117 insertions(+), 93 deletions(-)

diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c
index d628978431..f712b382f8 100644
--- a/target/arm/sve_helper.c
+++ b/target/arm/sve_helper.c
@@ -4285,109 +4285,133 @@ DO_LD1_2(ld1dd,  3, 3)
 #undef DO_LD1_1
 #undef DO_LD1_2
 
-#define DO_LD2(NAME, FN, TYPEE, TYPEM, H)  \
-void HELPER(NAME)(CPUARMState *env, void *vg,  \
-  target_ulong addr, uint32_t desc)\
-{  \
-intptr_t i, oprsz = simd_oprsz(desc);  \
-intptr_t ra = GETPC(); \
-unsigned rd = simd_data(desc); \
-void *d1 = >vfp.zregs[rd];\
-void *d2 = >vfp.zregs[(rd + 1) & 31]; \
-for (i = 0; i < oprsz; ) { \
-uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3));\
-do {   \
-TYPEM m1 = 0, m2 = 0;  \
-if (pg & 1) {  \
-m1 = FN(env, addr, ra);\
-m2 = FN(env, addr + sizeof(TYPEM), ra);\
-}  \
-*(TYPEE *)(d1 + H(i)) = m1;\
-*(TYPEE *)(d2 + H(i)) = m2;\
-i += sizeof(TYPEE), pg >>= sizeof(TYPEE);  \
-addr += 2 * sizeof(TYPEM); \
-} while (i & 15);  \
-}  \
+/*
+ * Common helpers for all contiguous 2,3,4-register predicated loads.
+ */
+static void sve_ld2_r(CPUARMState *env, void *vg, target_ulong addr,
+  uint32_t desc, int size, uintptr_t ra,
+  sve_ld1_tlb_fn *tlb_fn)
+{
+const int mmu_idx = cpu_mmu_index(env, false);
+intptr_t i, oprsz = simd_oprsz(desc);
+unsigned rd = simd_data(desc);
+ARMVectorReg scratch[2] = { };
+
+set_helper_retaddr(ra);
+for (i = 0; i < oprsz; ) {
+uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3));
+do {
+if (pg & 1) {
+tlb_fn(env, [0], i, addr, mmu_idx, ra);
+tlb_fn(env, [1], i, addr + size, mmu_idx, ra);
+}
+i += size, pg >>= size;
+addr += 2 * size;
+} while (i & 15);
+}
+set_helper_retaddr(0);
+
+/* Wait until all exceptions have been raised to write back.  */
+memcpy(>vfp.zregs[rd], [0], oprsz);
+memcpy(>vfp.zregs[(rd + 1) & 31], [1], oprsz);
 }
 
-#define DO_LD3(NAME, FN, TYPEE, TYPEM, H)  \
-void HELPER(NAME)(CPUARMState *env, void *vg,  \
-  target_ulong addr, uint32_t desc)\
-{  \
-intptr_t i, oprsz = simd_oprsz(desc);  \
-intptr_t ra = GETPC(); \
-unsigned rd = simd_data(desc); \
-void *d1 = >vfp.zregs[rd];\
-void *d2 = >vfp.zregs[(rd + 1) & 31]; \
-void *d3 = >vfp.zregs[(rd + 2) & 31]; \
-for (i = 0; i < oprsz; ) { \
-uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3));\
-do {   \
-TYPEM m1 = 0, m2 = 0, m3 = 0;  \
-if (pg & 1) {  \
-m1 = FN(env, addr, ra);\
-m2 = FN(env, addr + sizeof(TYPEM), ra);\
-m3 = FN(env, addr + 2 * sizeof(TYPEM), ra); \
-}  \
-*(TYPEE *)(d1 + H(i)) = m1;\
-*(TYPEE *)(d2 + H(i)) = m2;\
-*(TYPEE *)(d3 + H(i)) = m3;\
-i += sizeof(TYPEE), pg >>= sizeof(TYPEE);  \
-addr += 3 * sizeof(TYPEM); \
-} while (i & 15);  \
-}  \
+static void sve_ld3_r(CPUARMState *env, void *vg, target_ulong addr,
+  uint32_t desc, int size, uintptr_t ra,
+  sve_ld1_tlb_fn *tlb_fn)
+{
+const int mmu_idx = 

[Qemu-devel] [PATCH v2 13/15] target/arm: Rewrite vector gather stores

[Richard Henderson]

This fixes the endianness problem for softmmu, and moves
the main loop out of a macro and into an inlined function.

Reviewed-by: Peter Maydell 
Tested-by: Laurent Desnogues 
Signed-off-by: Richard Henderson 
---
 target/arm/helper-sve.h|  52 ++
 target/arm/sve_helper.c| 139 -
 target/arm/translate-sve.c |  74 +---
 3 files changed, 177 insertions(+), 88 deletions(-)

diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h
index 49d1c09e30..6b9b93af45 100644
--- a/target/arm/helper-sve.h
+++ b/target/arm/helper-sve.h
@@ -1468,41 +1468,67 @@ DEF_HELPER_FLAGS_6(sve_ldffsds_zd, TCG_CALL_NO_WG,
 
 DEF_HELPER_FLAGS_6(sve_stbs_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_sths_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_sths_le_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_stss_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_sths_be_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_stss_le_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_stss_be_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 
 DEF_HELPER_FLAGS_6(sve_stbs_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_sths_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_sths_le_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_stss_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_sths_be_zss, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_stss_le_zss, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_stss_be_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 
 DEF_HELPER_FLAGS_6(sve_stbd_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_sthd_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_sthd_le_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_stsd_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_sthd_be_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_stdd_zsu, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_stsd_le_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_stsd_be_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_stdd_le_zsu, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_stdd_be_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 
 DEF_HELPER_FLAGS_6(sve_stbd_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_sthd_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_sthd_le_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_stsd_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_sthd_be_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_stdd_zss, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_stsd_le_zss, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_stsd_be_zss, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_stdd_le_zss, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_stdd_be_zss, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
 
 DEF_HELPER_FLAGS_6(sve_stbd_zd, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_sthd_zd, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_sthd_le_zd, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_stsd_zd, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_sthd_be_zd, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
-DEF_HELPER_FLAGS_6(sve_stdd_zd, TCG_CALL_NO_WG,
+DEF_HELPER_FLAGS_6(sve_stsd_le_zd, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_stsd_be_zd, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_stdd_le_zd, TCG_CALL_NO_WG,
+   void, env, ptr, ptr, ptr, tl, i32)
+DEF_HELPER_FLAGS_6(sve_stdd_be_zd, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c
index c225cd0488..a95e445b22 100644
--- a/target/arm/sve_helper.c
+++ b/target/arm/sve_helper.c
@@ -5136,61 +5136,100 @@ DO_LDFF1_ZPZ_D(sve_ldffsds_zd, uint64_t, int32_t,  
cpu_ldl_data_ra)
 
 /* Stores with a vector index.  */
 
-#define DO_ST1_ZPZ_S(NAME, TYPEI, FN)

[Qemu-devel] [PATCH v2 06/15] target/arm: Clear unused predicate bits for LD1RQ

[Richard Henderson]

The 16-byte load only uses 16 predicate bits.  But while
reusing the other load infrastructure, we find other bits
that are set and trigger an assert.  To avoid this and
retain the assert, zero-extend the predicate that we pass
to the LD1 helper.

Tested-by: Laurent Desnogues 
Reported-by: Laurent Desnogues 
Signed-off-by: Richard Henderson 
---
 target/arm/translate-sve.c | 25 +++--
 1 file changed, 23 insertions(+), 2 deletions(-)

diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c
index 667879564f..4ee3bbca29 100644
--- a/target/arm/translate-sve.c
+++ b/target/arm/translate-sve.c
@@ -4765,12 +4765,33 @@ static void do_ldrq(DisasContext *s, int zt, int pg, 
TCGv_i64 addr, int msz)
 unsigned vsz = vec_full_reg_size(s);
 TCGv_ptr t_pg;
 TCGv_i32 desc;
+int poff;
 
 /* Load the first quadword using the normal predicated load helpers.  */
 desc = tcg_const_i32(simd_desc(16, 16, zt));
-t_pg = tcg_temp_new_ptr();
 
-tcg_gen_addi_ptr(t_pg, cpu_env, pred_full_reg_offset(s, pg));
+poff = pred_full_reg_offset(s, pg);
+if (vsz > 16) {
+/*
+ * Zero-extend the first 16 bits of the predicate into a temporary.
+ * This avoids triggering an assert making sure we don't have bits
+ * set within a predicate beyond VQ, but we have lowered VQ to 1
+ * for this load operation.
+ */
+TCGv_i64 tmp = tcg_temp_new_i64();
+#ifdef HOST_WORDS_BIGENDIAN
+poff += 6;
+#endif
+tcg_gen_ld16u_i64(tmp, cpu_env, poff);
+
+poff = offsetof(CPUARMState, vfp.preg_tmp);
+tcg_gen_st_i64(tmp, cpu_env, poff);
+tcg_temp_free_i64(tmp);
+}
+
+t_pg = tcg_temp_new_ptr();
+tcg_gen_addi_ptr(t_pg, cpu_env, poff);
+
 fns[msz](cpu_env, t_pg, addr, desc);
 
 tcg_temp_free_ptr(t_pg);
-- 
2.17.1

[Qemu-devel] [PATCH v2 10/15] target/arm: Split contiguous loads for endianness

[Richard Henderson]

We can choose the endianness at translation time, rather than
re-computing it at execution time.

Tested-by: Laurent Desnogues 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 target/arm/helper-sve.h| 117 +++---
 target/arm/sve_helper.c|  70 ++---
 target/arm/translate-sve.c | 196 +
 3 files changed, 252 insertions(+), 131 deletions(-)

diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h
index 023952a9a4..526caec8da 100644
--- a/target/arm/helper-sve.h
+++ b/target/arm/helper-sve.h
@@ -1128,20 +1128,35 @@ DEF_HELPER_FLAGS_4(sve_ld2bb_r, TCG_CALL_NO_WG, void, 
env, ptr, tl, i32)
 DEF_HELPER_FLAGS_4(sve_ld3bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 DEF_HELPER_FLAGS_4(sve_ld4bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 
-DEF_HELPER_FLAGS_4(sve_ld1hh_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_ld2hh_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_ld3hh_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_ld4hh_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld1hh_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld2hh_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld3hh_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld4hh_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 
-DEF_HELPER_FLAGS_4(sve_ld1ss_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_ld2ss_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_ld3ss_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_ld4ss_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld1hh_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld2hh_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld3hh_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld4hh_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 
-DEF_HELPER_FLAGS_4(sve_ld1dd_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_ld2dd_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_ld3dd_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_ld4dd_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld1ss_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld2ss_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld3ss_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld4ss_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+
+DEF_HELPER_FLAGS_4(sve_ld1ss_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld2ss_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld3ss_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld4ss_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+
+DEF_HELPER_FLAGS_4(sve_ld1dd_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld2dd_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld3dd_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld4dd_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+
+DEF_HELPER_FLAGS_4(sve_ld1dd_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld2dd_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld3dd_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld4dd_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 
 DEF_HELPER_FLAGS_4(sve_ld1bhu_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 DEF_HELPER_FLAGS_4(sve_ld1bsu_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
@@ -1150,13 +1165,21 @@ DEF_HELPER_FLAGS_4(sve_ld1bhs_r, TCG_CALL_NO_WG, void, 
env, ptr, tl, i32)
 DEF_HELPER_FLAGS_4(sve_ld1bss_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 DEF_HELPER_FLAGS_4(sve_ld1bds_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 
-DEF_HELPER_FLAGS_4(sve_ld1hsu_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_ld1hdu_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_ld1hss_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_ld1hds_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld1hsu_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld1hdu_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld1hss_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld1hds_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 
-DEF_HELPER_FLAGS_4(sve_ld1sdu_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_ld1sds_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld1hsu_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld1hdu_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_ld1hss_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)

[Qemu-devel] [PATCH v2 04/15] target/arm: Handle SVE vector length changes in system mode

[Richard Henderson]

SVE vector length can change when changing EL, or when writing
to one of the ZCR_ELn registers.

For correctness, our implementation requires that predicate bits
that are inaccessible are never set.  Which means noticing length
changes and zeroing the appropriate register bits.

Tested-by: Laurent Desnogues 
Signed-off-by: Richard Henderson 
---
 target/arm/cpu.h   |   4 ++
 target/arm/cpu64.c |  42 --
 target/arm/helper.c| 127 -
 target/arm/op_helper.c |   1 +
 4 files changed, 119 insertions(+), 55 deletions(-)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 65c0fa0a65..a4ee83dc77 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -910,6 +910,10 @@ int arm_cpu_write_elf32_note(WriteCoreDumpFunction f, 
CPUState *cs,
 int aarch64_cpu_gdb_read_register(CPUState *cpu, uint8_t *buf, int reg);
 int aarch64_cpu_gdb_write_register(CPUState *cpu, uint8_t *buf, int reg);
 void aarch64_sve_narrow_vq(CPUARMState *env, unsigned vq);
+void aarch64_sve_change_el(CPUARMState *env, int old_el, int new_el);
+#else
+static inline void aarch64_sve_narrow_vq(CPUARMState *env, unsigned vq) { }
+static inline void aarch64_sve_change_el(CPUARMState *env, int o, int n) { }
 #endif
 
 target_ulong do_arm_semihosting(CPUARMState *env);
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 800bff780e..db71504cb5 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -410,45 +410,3 @@ static void aarch64_cpu_register_types(void)
 }
 
 type_init(aarch64_cpu_register_types)
-
-/* The manual says that when SVE is enabled and VQ is widened the
- * implementation is allowed to zero the previously inaccessible
- * portion of the registers.  The corollary to that is that when
- * SVE is enabled and VQ is narrowed we are also allowed to zero
- * the now inaccessible portion of the registers.
- *
- * The intent of this is that no predicate bit beyond VQ is ever set.
- * Which means that some operations on predicate registers themselves
- * may operate on full uint64_t or even unrolled across the maximum
- * uint64_t[4].  Performing 4 bits of host arithmetic unconditionally
- * may well be cheaper than conditionals to restrict the operation
- * to the relevant portion of a uint16_t[16].
- *
- * TODO: Need to call this for changes to the real system registers
- * and EL state changes.
- */
-void aarch64_sve_narrow_vq(CPUARMState *env, unsigned vq)
-{
-int i, j;
-uint64_t pmask;
-
-assert(vq >= 1 && vq <= ARM_MAX_VQ);
-assert(vq <= arm_env_get_cpu(env)->sve_max_vq);
-
-/* Zap the high bits of the zregs.  */
-for (i = 0; i < 32; i++) {
-memset(>vfp.zregs[i].d[2 * vq], 0, 16 * (ARM_MAX_VQ - vq));
-}
-
-/* Zap the high bits of the pregs and ffr.  */
-pmask = 0;
-if (vq & 3) {
-pmask = ~(-1ULL << (16 * (vq & 3)));
-}
-for (j = vq / 4; j < ARM_MAX_VQ / 4; j++) {
-for (i = 0; i < 17; ++i) {
-env->vfp.pregs[i].p[j] &= pmask;
-}
-pmask = 0;
-}
-}
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 52fc9d1d4c..9b1f868efa 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -4461,11 +4461,44 @@ static int sve_exception_el(CPUARMState *env, int el)
 return 0;
 }
 
+/*
+ * Given that SVE is enabled, return the vector length for EL.
+ */
+static uint32_t sve_zcr_len_for_el(CPUARMState *env, int el)
+{
+ARMCPU *cpu = arm_env_get_cpu(env);
+uint32_t zcr_len = cpu->sve_max_vq - 1;
+
+if (el <= 1) {
+zcr_len = MIN(zcr_len, 0xf & (uint32_t)env->vfp.zcr_el[1]);
+}
+if (el < 2 && arm_feature(env, ARM_FEATURE_EL2)) {
+zcr_len = MIN(zcr_len, 0xf & (uint32_t)env->vfp.zcr_el[2]);
+}
+if (el < 3 && arm_feature(env, ARM_FEATURE_EL3)) {
+zcr_len = MIN(zcr_len, 0xf & (uint32_t)env->vfp.zcr_el[3]);
+}
+return zcr_len;
+}
+
 static void zcr_write(CPUARMState *env, const ARMCPRegInfo *ri,
   uint64_t value)
 {
+int cur_el = arm_current_el(env);
+int old_len = sve_zcr_len_for_el(env, cur_el);
+int new_len;
+
 /* Bits other than [3:0] are RAZ/WI.  */
 raw_write(env, ri, value & 0xf);
+
+/*
+ * Because we arrived here, we know both FP and SVE are enabled;
+ * otherwise we would have trapped access to the ZCR_ELn register.
+ */
+new_len = sve_zcr_len_for_el(env, cur_el);
+if (new_len < old_len) {
+aarch64_sve_narrow_vq(env, new_len + 1);
+}
 }
 
 static const ARMCPRegInfo zcr_el1_reginfo = {
@@ -8305,8 +8338,11 @@ static void arm_cpu_do_interrupt_aarch64(CPUState *cs)
 unsigned int new_el = env->exception.target_el;
 target_ulong addr = env->cp15.vbar_el[new_el];
 unsigned int new_mode = aarch64_pstate_mode(new_el, true);
+unsigned int cur_el = arm_current_el(env);
 
-if (arm_current_el(env) < new_el) {
+aarch64_sve_change_el(env, cur_el, new_el);
+
+if (cur_el < new_el) {
 /* Entry vector offset 

[Qemu-devel] [PATCH v2 00/15] target/arm: sve system mode patches

[Richard Henderson]

For v2, I've dropped a few patches and adjusted for some review on v1.

In particular, the patches that adjust the ID system registers are
dropped, so there's not actually a way to run any SVE code in system
mode with just this.  But there's no point keeping these out of tree
while work continues on the system register issue.

Patches without review: 3, 4, 5, 6, 7.


r~


Richard Henderson (15):
  target/arm: Define ID_AA64ZFR0_EL1
  target/arm: Adjust sve_exception_el
  target/arm: Pass in current_el to fp and sve_exception_el
  target/arm: Handle SVE vector length changes in system mode
  target/arm: Adjust aarch64_cpu_dump_state for system mode SVE
  target/arm: Clear unused predicate bits for LD1RQ
  target/arm: Rewrite helper_sve_ld1*_r using pages
  target/arm: Rewrite helper_sve_ld[234]*_r
  target/arm: Rewrite helper_sve_st[1234]*_r
  target/arm: Split contiguous loads for endianness
  target/arm: Split contiguous stores for endianness
  target/arm: Rewrite vector gather loads
  target/arm: Rewrite vector gather stores
  target/arm: Rewrite vector gather first-fault loads
  target/arm: Pass TCGMemOpIdx to sve memory helpers

 target/arm/cpu.h   |8 +
 target/arm/helper-sve.h|  385 +--
 target/arm/internals.h |5 +
 target/arm/cpu64.c |   42 -
 target/arm/helper.c|  237 +++--
 target/arm/op_helper.c |1 +
 target/arm/sve_helper.c| 1961 
 target/arm/translate-a64.c |8 +-
 target/arm/translate-sve.c |  670 
 9 files changed, 2267 insertions(+), 1050 deletions(-)

-- 
2.17.1

[Qemu-devel] [PATCH v2 05/15] target/arm: Adjust aarch64_cpu_dump_state for system mode SVE

[Richard Henderson]

Use the existing helpers to determine if (1) the fpu is enabled,
(2) sve state is enabled, and (3) the current sve vector length.

Tested-by: Laurent Desnogues 
Signed-off-by: Richard Henderson 
---
 target/arm/cpu.h   | 4 
 target/arm/helper.c| 6 +++---
 target/arm/translate-a64.c | 8 ++--
 3 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index a4ee83dc77..da4d3888ea 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -920,6 +920,10 @@ target_ulong do_arm_semihosting(CPUARMState *env);
 void aarch64_sync_32_to_64(CPUARMState *env);
 void aarch64_sync_64_to_32(CPUARMState *env);
 
+int fp_exception_el(CPUARMState *env, int cur_el);
+int sve_exception_el(CPUARMState *env, int cur_el);
+uint32_t sve_zcr_len_for_el(CPUARMState *env, int el);
+
 static inline bool is_a64(CPUARMState *env)
 {
 return env->aarch64;
diff --git a/target/arm/helper.c b/target/arm/helper.c
index 9b1f868efa..05329accd5 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -4406,7 +4406,7 @@ static const ARMCPRegInfo debug_lpae_cp_reginfo[] = {
  * take care of raising that exception.
  * C.f. the ARM pseudocode function CheckSVEEnabled.
  */
-static int sve_exception_el(CPUARMState *env, int el)
+int sve_exception_el(CPUARMState *env, int el)
 {
 #ifndef CONFIG_USER_ONLY
 if (el <= 1) {
@@ -4464,7 +4464,7 @@ static int sve_exception_el(CPUARMState *env, int el)
 /*
  * Given that SVE is enabled, return the vector length for EL.
  */
-static uint32_t sve_zcr_len_for_el(CPUARMState *env, int el)
+uint32_t sve_zcr_len_for_el(CPUARMState *env, int el)
 {
 ARMCPU *cpu = arm_env_get_cpu(env);
 uint32_t zcr_len = cpu->sve_max_vq - 1;
@@ -12547,7 +12547,7 @@ uint32_t HELPER(crc32c)(uint32_t acc, uint32_t val, 
uint32_t bytes)
 /* Return the exception level to which FP-disabled exceptions should
  * be taken, or 0 if FP is enabled.
  */
-static int fp_exception_el(CPUARMState *env, int cur_el)
+int fp_exception_el(CPUARMState *env, int cur_el)
 {
 #ifndef CONFIG_USER_ONLY
 int fpen;
diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 8ca3876707..8a24278d79 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -166,11 +166,15 @@ void aarch64_cpu_dump_state(CPUState *cs, FILE *f,
 cpu_fprintf(f, "\n");
 return;
 }
+if (fp_exception_el(env, el) != 0) {
+cpu_fprintf(f, "FPU disabled\n");
+return;
+}
 cpu_fprintf(f, " FPCR=%08x FPSR=%08x\n",
 vfp_get_fpcr(env), vfp_get_fpsr(env));
 
-if (arm_feature(env, ARM_FEATURE_SVE)) {
-int j, zcr_len = env->vfp.zcr_el[1] & 0xf; /* fix for system mode */
+if (arm_feature(env, ARM_FEATURE_SVE) && sve_exception_el(env, el) == 0) {
+int j, zcr_len = sve_zcr_len_for_el(env, el);
 
 for (i = 0; i <= FFR_PRED_NUM; i++) {
 bool eol;
-- 
2.17.1

[Qemu-devel] [PATCH v2 11/15] target/arm: Split contiguous stores for endianness

[Richard Henderson]

We can choose the endianness at translation time, rather than
re-computing it at execution time.

Tested-by: Laurent Desnogues 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 target/arm/helper-sve.h| 48 +
 target/arm/sve_helper.c| 11 --
 target/arm/translate-sve.c | 72 +-
 3 files changed, 96 insertions(+), 35 deletions(-)

diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h
index 526caec8da..1ad043101a 100644
--- a/target/arm/helper-sve.h
+++ b/target/arm/helper-sve.h
@@ -1248,29 +1248,47 @@ DEF_HELPER_FLAGS_4(sve_st2bb_r, TCG_CALL_NO_WG, void, 
env, ptr, tl, i32)
 DEF_HELPER_FLAGS_4(sve_st3bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 DEF_HELPER_FLAGS_4(sve_st4bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 
-DEF_HELPER_FLAGS_4(sve_st1hh_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_st2hh_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_st3hh_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_st4hh_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st1hh_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st2hh_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st3hh_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st4hh_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 
-DEF_HELPER_FLAGS_4(sve_st1ss_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_st2ss_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_st3ss_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_st4ss_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st1hh_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st2hh_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st3hh_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st4hh_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 
-DEF_HELPER_FLAGS_4(sve_st1dd_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_st2dd_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_st3dd_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_st4dd_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st1ss_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st2ss_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st3ss_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st4ss_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+
+DEF_HELPER_FLAGS_4(sve_st1ss_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st2ss_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st3ss_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st4ss_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+
+DEF_HELPER_FLAGS_4(sve_st1dd_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st2dd_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st3dd_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st4dd_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+
+DEF_HELPER_FLAGS_4(sve_st1dd_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st2dd_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st3dd_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st4dd_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 
 DEF_HELPER_FLAGS_4(sve_st1bh_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 DEF_HELPER_FLAGS_4(sve_st1bs_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 DEF_HELPER_FLAGS_4(sve_st1bd_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 
-DEF_HELPER_FLAGS_4(sve_st1hs_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
-DEF_HELPER_FLAGS_4(sve_st1hd_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st1hs_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st1hd_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st1hs_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st1hd_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 
-DEF_HELPER_FLAGS_4(sve_st1sd_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st1sd_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
+DEF_HELPER_FLAGS_4(sve_st1sd_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32)
 
 DEF_HELPER_FLAGS_6(sve_ldbsu_zsu, TCG_CALL_NO_WG,
void, env, ptr, ptr, ptr, tl, i32)
diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c
index d31988b46a..426353984e 100644
--- a/target/arm/sve_helper.c
+++ b/target/arm/sve_helper.c
@@ -4836,12 +4836,17 @@ void __attribute__((flatten)) 
HELPER(sve_st##N##NAME##_r)   \
 }
 
 #define DO_STN_2(N, NAME, ESIZE, MSIZE) \
-void __attribute__((flatten)) HELPER(sve_st##N##NAME##_r) \
+void 

[Qemu-devel] [PATCH v2 01/15] target/arm: Define ID_AA64ZFR0_EL1

[Richard Henderson]

Given that the only field defined for this new register may only
be 0, we don't actually need to change anything except the name.

Reviewed-by: Peter Maydell 
Tested-by: Laurent Desnogues 
Signed-off-by: Richard Henderson 
---
 target/arm/helper.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 64b1564594..ef85ef230a 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -5018,9 +5018,10 @@ void register_cp_regs_for_features(ARMCPU *cpu)
   .opc0 = 3, .opc1 = 0, .crn = 0, .crm = 4, .opc2 = 3,
   .access = PL1_R, .type = ARM_CP_CONST,
   .resetvalue = 0 },
-{ .name = "ID_AA64PFR4_EL1_RESERVED", .state = ARM_CP_STATE_AA64,
+{ .name = "ID_AA64ZFR0_EL1", .state = ARM_CP_STATE_AA64,
   .opc0 = 3, .opc1 = 0, .crn = 0, .crm = 4, .opc2 = 4,
   .access = PL1_R, .type = ARM_CP_CONST,
+  /* At present, only SVEver == 0 is defined anyway.  */
   .resetvalue = 0 },
 { .name = "ID_AA64PFR5_EL1_RESERVED", .state = ARM_CP_STATE_AA64,
   .opc0 = 3, .opc1 = 0, .crn = 0, .crm = 4, .opc2 = 5,
-- 
2.17.1

[Qemu-devel] [PATCH v2 03/15] target/arm: Pass in current_el to fp and sve_exception_el

[Richard Henderson]

We are going to want to determine whether sve is enabled
for EL other than current.

Tested-by: Laurent Desnogues 
Signed-off-by: Richard Henderson 
---
 target/arm/helper.c | 21 +
 1 file changed, 9 insertions(+), 12 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index 38a9d32dc4..52fc9d1d4c 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -4406,12 +4406,10 @@ static const ARMCPRegInfo debug_lpae_cp_reginfo[] = {
  * take care of raising that exception.
  * C.f. the ARM pseudocode function CheckSVEEnabled.
  */
-static int sve_exception_el(CPUARMState *env)
+static int sve_exception_el(CPUARMState *env, int el)
 {
 #ifndef CONFIG_USER_ONLY
-unsigned current_el = arm_current_el(env);
-
-if (current_el <= 1) {
+if (el <= 1) {
 bool disabled = false;
 
 /* The CPACR.ZEN controls traps to EL1:
@@ -4422,7 +4420,7 @@ static int sve_exception_el(CPUARMState *env)
 if (!extract32(env->cp15.cpacr_el1, 16, 1)) {
 disabled = true;
 } else if (!extract32(env->cp15.cpacr_el1, 17, 1)) {
-disabled = current_el == 0;
+disabled = el == 0;
 }
 if (disabled) {
 /* route_to_el2 */
@@ -4435,7 +4433,7 @@ static int sve_exception_el(CPUARMState *env)
 if (!extract32(env->cp15.cpacr_el1, 20, 1)) {
 disabled = true;
 } else if (!extract32(env->cp15.cpacr_el1, 21, 1)) {
-disabled = current_el == 0;
+disabled = el == 0;
 }
 if (disabled) {
 return 0;
@@ -4445,7 +4443,7 @@ static int sve_exception_el(CPUARMState *env)
 /* CPTR_EL2.  Since TZ and TFP are positive,
  * they will be zero when EL2 is not present.
  */
-if (current_el <= 2 && !arm_is_secure_below_el3(env)) {
+if (el <= 2 && !arm_is_secure_below_el3(env)) {
 if (env->cp15.cptr_el[2] & CPTR_TZ) {
 return 2;
 }
@@ -12513,11 +12511,10 @@ uint32_t HELPER(crc32c)(uint32_t acc, uint32_t val, 
uint32_t bytes)
 /* Return the exception level to which FP-disabled exceptions should
  * be taken, or 0 if FP is enabled.
  */
-static inline int fp_exception_el(CPUARMState *env)
+static int fp_exception_el(CPUARMState *env, int cur_el)
 {
 #ifndef CONFIG_USER_ONLY
 int fpen;
-int cur_el = arm_current_el(env);
 
 /* CPACR and the CPTR registers don't exist before v6, so FP is
  * always accessible
@@ -12580,7 +12577,8 @@ void cpu_get_tb_cpu_state(CPUARMState *env, 
target_ulong *pc,
   target_ulong *cs_base, uint32_t *pflags)
 {
 ARMMMUIdx mmu_idx = core_to_arm_mmu_idx(env, cpu_mmu_index(env, false));
-int fp_el = fp_exception_el(env);
+int current_el = arm_current_el(env);
+int fp_el = fp_exception_el(env, current_el);
 uint32_t flags;
 
 if (is_a64(env)) {
@@ -12591,7 +12589,7 @@ void cpu_get_tb_cpu_state(CPUARMState *env, 
target_ulong *pc,
 flags |= (arm_regime_tbi1(env, mmu_idx) << ARM_TBFLAG_TBI1_SHIFT);
 
 if (arm_feature(env, ARM_FEATURE_SVE)) {
-int sve_el = sve_exception_el(env);
+int sve_el = sve_exception_el(env, current_el);
 uint32_t zcr_len;
 
 /* If SVE is disabled, but FP is enabled,
@@ -12600,7 +12598,6 @@ void cpu_get_tb_cpu_state(CPUARMState *env, 
target_ulong *pc,
 if (sve_el != 0 && fp_el == 0) {
 zcr_len = 0;
 } else {
-int current_el = arm_current_el(env);
 ARMCPU *cpu = arm_env_get_cpu(env);
 
 zcr_len = cpu->sve_max_vq - 1;
-- 
2.17.1

[Qemu-devel] [PATCH v2 02/15] target/arm: Adjust sve_exception_el

[Richard Henderson]

Check for EL3 before testing CPTR_EL3.EZ.  Return 0 when the exception
should be routed via AdvSIMDFPAccessTrap.  Mirror the structure of
CheckSVEEnabled more closely.

Fixes: 5be5e8eda78
Reviewed-by: Peter Maydell 
Tested-by: Laurent Desnogues 
Signed-off-by: Richard Henderson 
---
 target/arm/helper.c | 96 ++---
 1 file changed, 46 insertions(+), 50 deletions(-)

diff --git a/target/arm/helper.c b/target/arm/helper.c
index ef85ef230a..38a9d32dc4 100644
--- a/target/arm/helper.c
+++ b/target/arm/helper.c
@@ -4400,67 +4400,63 @@ static const ARMCPRegInfo debug_lpae_cp_reginfo[] = {
 REGINFO_SENTINEL
 };
 
-/* Return the exception level to which SVE-disabled exceptions should
- * be taken, or 0 if SVE is enabled.
+/* Return the exception level to which exceptions should be taken
+ * via SVEAccessTrap.  If an exception should be routed through
+ * AArch64.AdvSIMDFPAccessTrap, return 0; fp_exception_el should
+ * take care of raising that exception.
+ * C.f. the ARM pseudocode function CheckSVEEnabled.
  */
 static int sve_exception_el(CPUARMState *env)
 {
 #ifndef CONFIG_USER_ONLY
 unsigned current_el = arm_current_el(env);
 
-/* The CPACR.ZEN controls traps to EL1:
- * 0, 2 : trap EL0 and EL1 accesses
- * 1: trap only EL0 accesses
- * 3: trap no accesses
+if (current_el <= 1) {
+bool disabled = false;
+
+/* The CPACR.ZEN controls traps to EL1:
+ * 0, 2 : trap EL0 and EL1 accesses
+ * 1: trap only EL0 accesses
+ * 3: trap no accesses
+ */
+if (!extract32(env->cp15.cpacr_el1, 16, 1)) {
+disabled = true;
+} else if (!extract32(env->cp15.cpacr_el1, 17, 1)) {
+disabled = current_el == 0;
+}
+if (disabled) {
+/* route_to_el2 */
+return (arm_feature(env, ARM_FEATURE_EL2)
+&& !arm_is_secure(env)
+&& (env->cp15.hcr_el2 & HCR_TGE) ? 2 : 1);
+}
+
+/* Check CPACR.FPEN.  */
+if (!extract32(env->cp15.cpacr_el1, 20, 1)) {
+disabled = true;
+} else if (!extract32(env->cp15.cpacr_el1, 21, 1)) {
+disabled = current_el == 0;
+}
+if (disabled) {
+return 0;
+}
+}
+
+/* CPTR_EL2.  Since TZ and TFP are positive,
+ * they will be zero when EL2 is not present.
  */
-switch (extract32(env->cp15.cpacr_el1, 16, 2)) {
-default:
-if (current_el <= 1) {
-/* Trap to PL1, which might be EL1 or EL3 */
-if (arm_is_secure(env) && !arm_el_is_aa64(env, 3)) {
-return 3;
-}
-return 1;
+if (current_el <= 2 && !arm_is_secure_below_el3(env)) {
+if (env->cp15.cptr_el[2] & CPTR_TZ) {
+return 2;
 }
-break;
-case 1:
-if (current_el == 0) {
-return 1;
+if (env->cp15.cptr_el[2] & CPTR_TFP) {
+return 0;
 }
-break;
-case 3:
-break;
 }
 
-/* Similarly for CPACR.FPEN, after having checked ZEN.  */
-switch (extract32(env->cp15.cpacr_el1, 20, 2)) {
-default:
-if (current_el <= 1) {
-if (arm_is_secure(env) && !arm_el_is_aa64(env, 3)) {
-return 3;
-}
-return 1;
-}
-break;
-case 1:
-if (current_el == 0) {
-return 1;
-}
-break;
-case 3:
-break;
-}
-
-/* CPTR_EL2.  Check both TZ and TFP.  */
-if (current_el <= 2
-&& (env->cp15.cptr_el[2] & (CPTR_TFP | CPTR_TZ))
-&& !arm_is_secure_below_el3(env)) {
-return 2;
-}
-
-/* CPTR_EL3.  Check both EZ and TFP.  */
-if (!(env->cp15.cptr_el[3] & CPTR_EZ)
-|| (env->cp15.cptr_el[3] & CPTR_TFP)) {
+/* CPTR_EL3.  Since EZ is negative we must check for EL3.  */
+if (arm_feature(env, ARM_FEATURE_EL3)
+&& !(env->cp15.cptr_el[3] & CPTR_EZ)) {
 return 3;
 }
 #endif
-- 
2.17.1

Re: [Qemu-devel] [PATCH v3 2/5] block/dirty-bitmaps: fix merge permissions

[John Snow]

On 09/26/2018 08:08 AM, Vladimir Sementsov-Ogievskiy wrote:
> 26.09.2018 14:55, Vladimir Sementsov-Ogievskiy wrote:
>> 26.09.2018 02:49, John Snow wrote:
>>> We wish to prohibit merging to read-only bitmaps and frozen bitmaps,
>>> but "disabled" bitmaps only preclude their recording of live, new
>>> information. It does not prohibit them from manual writes at the behest
>>> of the user, as is the case for merge operations.
>>>
>>> Allow the merge to "disabled" bitmaps,
>>> and prohibit merging to "locked" ones.
>>
>> only the second part is here..
> 
> Hm, the first one is in first separate patch? With commit message fixed
> to only second part, of course:
> 

Ah, yeah, I'll make that clearer. Got lost in the patch reordering. Thanks!

> 
> Reviewed-by: Vladimir Sementsov-Ogievskiy 
> 
>>
>>>
>>> Reported-by: Eric Blake 
>>> Signed-off-by: John Snow 
>>> ---
>>>   block/dirty-bitmap.c | 6 +++---
>>>   1 file changed, 3 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
>>> index fc10543ab0..53b7d282c4 100644
>>> --- a/block/dirty-bitmap.c
>>> +++ b/block/dirty-bitmap.c
>>> @@ -806,9 +806,9 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap
>>> *dest, const BdrvDirtyBitmap *src,
>>>     qemu_mutex_lock(dest->mutex);
>>>   -    if (bdrv_dirty_bitmap_frozen(dest)) {
>>> -    error_setg(errp, "Bitmap '%s' is frozen and cannot be
>>> modified",
>>> -   dest->name);
>>> +    if (!bdrv_dirty_bitmap_user_modifiable(dest)) {
>>> +    error_setg(errp, "Bitmap '%s' is currently in-use by another"
>>> +    " operation and cannot be modified", dest->name);
>>>   goto out;
>>>   }
>>
>>
> 
> 

Re: [Qemu-devel] [PATCH] qemu/compiler: Wrap __attribute__((flatten)) in a macro

[Peter Maydell]

On 26 September 2018 at 16:59, Thomas Huth  wrote:

> +/*
> + * Clang 3.4 claims to be compatible with GCC 4.2, but does not have the
> + * "flatten" attribute, so we've got to handle Clang via __has_attribute here
> + */
> +#if defined(__clang__) && defined(__has_attribute)
> +# if __has_attribute(flatten)
> +#  define QEMU_FLATTEN __attribute__((flatten))
> +# endif
> +#elif !defined(__clang__) && QEMU_GNUC_PREREQ(4, 1)
> +# define QEMU_FLATTEN __attribute__((flatten))
> +#endif
> +#ifndef QEMU_FLATTEN
> +# define QEMU_FLATTEN
> +#endif

I think it would be cleaner to say: if we have __has_attribute,
trust it; otherwise fall back to version testing. Also, we
already require at least GCC 4.1 so a version test against
that is unnecessary. So something like:


#ifndef __has_attribute
#define __has_attribute(x) 0 / compatibility with older GCC */
#endif

/*
 * gcc doesn't provide __has_attribute() until gcc 5,
 * but we know all the gcc versions we support have flatten.
 * clang may not have flatten but always has __has_attribute().
 */
#if __has_attribute(flatten) || !defined(__clang__)
# define QEMU_FLATTEN __attribute__((flatten))
#else
# define QEMU_FLATTEN
#endif

> +
>  #ifndef __has_feature
>  #define __has_feature(x) 0 /* compatibility with non-clang compilers */
>  #endif
> --
> 1.8.3.1

thanks
-- PMM

[Qemu-devel] [PULL 09/13] qht: constify qht_lookup

[Richard Henderson]

From: "Emilio G. Cota" 

seqlock_read_begin takes a const param since c04649eeea
("seqlock: constify seqlock_read_begin", 2018-08-23), so
we can constify the entire lookup.

Signed-off-by: Emilio G. Cota 
Signed-off-by: Richard Henderson 
---
 include/qemu/qht.h |  4 ++--
 util/qht.c | 16 
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/include/qemu/qht.h b/include/qemu/qht.h
index 6484837487..2e2d6bca93 100644
--- a/include/qemu/qht.h
+++ b/include/qemu/qht.h
@@ -104,7 +104,7 @@ bool qht_insert(struct qht *ht, void *p, uint32_t hash, 
void **existing);
  * Returns the corresponding pointer when a match is found.
  * Returns NULL otherwise.
  */
-void *qht_lookup_custom(struct qht *ht, const void *userp, uint32_t hash,
+void *qht_lookup_custom(const struct qht *ht, const void *userp, uint32_t hash,
 qht_lookup_func_t func);
 
 /**
@@ -115,7 +115,7 @@ void *qht_lookup_custom(struct qht *ht, const void *userp, 
uint32_t hash,
  *
  * Calls qht_lookup_custom() using @ht's default comparison function.
  */
-void *qht_lookup(struct qht *ht, const void *userp, uint32_t hash);
+void *qht_lookup(const struct qht *ht, const void *userp, uint32_t hash);
 
 /**
  * qht_remove - remove a pointer from the hash table
diff --git a/util/qht.c b/util/qht.c
index 3564a7e20f..020dfe6912 100644
--- a/util/qht.c
+++ b/util/qht.c
@@ -236,7 +236,7 @@ static inline void qht_head_init(struct qht_bucket *b)
 }
 
 static inline
-struct qht_bucket *qht_map_to_bucket(struct qht_map *map, uint32_t hash)
+struct qht_bucket *qht_map_to_bucket(const struct qht_map *map, uint32_t hash)
 {
 return >buckets[hash & (map->n_buckets - 1)];
 }
@@ -482,10 +482,10 @@ bool qht_reset_size(struct qht *ht, size_t n_elems)
 }
 
 static inline
-void *qht_do_lookup(struct qht_bucket *head, qht_lookup_func_t func,
+void *qht_do_lookup(const struct qht_bucket *head, qht_lookup_func_t func,
 const void *userp, uint32_t hash)
 {
-struct qht_bucket *b = head;
+const struct qht_bucket *b = head;
 int i;
 
 do {
@@ -509,7 +509,7 @@ void *qht_do_lookup(struct qht_bucket *head, 
qht_lookup_func_t func,
 }
 
 static __attribute__((noinline))
-void *qht_lookup__slowpath(struct qht_bucket *b, qht_lookup_func_t func,
+void *qht_lookup__slowpath(const struct qht_bucket *b, qht_lookup_func_t func,
const void *userp, uint32_t hash)
 {
 unsigned int version;
@@ -522,11 +522,11 @@ void *qht_lookup__slowpath(struct qht_bucket *b, 
qht_lookup_func_t func,
 return ret;
 }
 
-void *qht_lookup_custom(struct qht *ht, const void *userp, uint32_t hash,
+void *qht_lookup_custom(const struct qht *ht, const void *userp, uint32_t hash,
 qht_lookup_func_t func)
 {
-struct qht_bucket *b;
-struct qht_map *map;
+const struct qht_bucket *b;
+const struct qht_map *map;
 unsigned int version;
 void *ret;
 
@@ -545,7 +545,7 @@ void *qht_lookup_custom(struct qht *ht, const void *userp, 
uint32_t hash,
 return qht_lookup__slowpath(b, func, userp, hash);
 }
 
-void *qht_lookup(struct qht *ht, const void *userp, uint32_t hash)
+void *qht_lookup(const struct qht *ht, const void *userp, uint32_t hash)
 {
 return qht_lookup_custom(ht, userp, hash, ht->cmp);
 }
-- 
2.17.1

[Qemu-devel] [PULL 11/13] qht: constify arguments to some internal functions

[Richard Henderson]

From: "Emilio G. Cota" 

These functions do not modify their @ht or @bucket arguments.
Constify those arguments.

Signed-off-by: Emilio G. Cota 
Signed-off-by: Richard Henderson 
---
 util/qht.c | 16 ++--
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/util/qht.c b/util/qht.c
index 4378775d68..aa51be3c52 100644
--- a/util/qht.c
+++ b/util/qht.c
@@ -268,7 +268,8 @@ static void qht_map_unlock_buckets(struct qht_map *map)
  * Call with at least a bucket lock held.
  * @map should be the value read before acquiring the lock (or locks).
  */
-static inline bool qht_map_is_stale__locked(struct qht *ht, struct qht_map 
*map)
+static inline bool qht_map_is_stale__locked(const struct qht *ht,
+const struct qht_map *map)
 {
 return map != ht->map;
 }
@@ -337,12 +338,12 @@ struct qht_bucket *qht_bucket_lock__no_stale(struct qht 
*ht, uint32_t hash,
 return b;
 }
 
-static inline bool qht_map_needs_resize(struct qht_map *map)
+static inline bool qht_map_needs_resize(const struct qht_map *map)
 {
 return atomic_read(>n_added_buckets) > map->n_added_buckets_threshold;
 }
 
-static inline void qht_chain_destroy(struct qht_bucket *head)
+static inline void qht_chain_destroy(const struct qht_bucket *head)
 {
 struct qht_bucket *curr = head->next;
 struct qht_bucket *prev;
@@ -550,8 +551,11 @@ void *qht_lookup(const struct qht *ht, const void *userp, 
uint32_t hash)
 return qht_lookup_custom(ht, userp, hash, ht->cmp);
 }
 
-/* call with head->lock held */
-static void *qht_insert__locked(struct qht *ht, struct qht_map *map,
+/*
+ * call with head->lock held
+ * @ht is const since it is only used for ht->cmp()
+ */
+static void *qht_insert__locked(const struct qht *ht, struct qht_map *map,
 struct qht_bucket *head, void *p, uint32_t 
hash,
 bool *needs_resize)
 {
@@ -645,7 +649,7 @@ bool qht_insert(struct qht *ht, void *p, uint32_t hash, 
void **existing)
 return false;
 }
 
-static inline bool qht_entry_is_last(struct qht_bucket *b, int pos)
+static inline bool qht_entry_is_last(const struct qht_bucket *b, int pos)
 {
 if (pos == QHT_BUCKET_ENTRIES - 1) {
 if (b->next == NULL) {
-- 
2.17.1

[Qemu-devel] [PULL 13/13] tcg/i386: fix vector operations on 32-bit hosts

[Richard Henderson]

From: Roman Kapl 

The TCG backend uses LOWREGMASK to get the low 3 bits of register numbers.
This was defined as no-op for 32-bit x86, with the assumption that we have
eight registers anyway. This assumption is not true once we have xmm regs.

Since LOWREGMASK was a no-op, xmm register indidices were wrong in opcodes
and have overflown into other opcode fields, wreaking havoc.

To trigger these problems, you can try running the "movi d8, #0x0" AArch64
instruction on 32-bit x86. "vpxor %xmm0, %xmm0, %xmm0" should be generated,
but instead TCG generated "vpxor %xmm0, %xmm0, %xmm2".

Fixes: 770c2fc7bb ("Add vector operations")
Signed-off-by: Roman Kapl 
Message-Id: <20180824131734.18557-1-...@sysgo.com>
Signed-off-by: Richard Henderson 
---
 tcg/i386/tcg-target.inc.c | 4 
 1 file changed, 4 deletions(-)

diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c
index a91e4f1313..436195894b 100644
--- a/tcg/i386/tcg-target.inc.c
+++ b/tcg/i386/tcg-target.inc.c
@@ -302,11 +302,7 @@ static inline int tcg_target_const_match(tcg_target_long 
val, TCGType type,
 return 0;
 }
 
-#if TCG_TARGET_REG_BITS == 64
 # define LOWREGMASK(x) ((x) & 7)
-#else
-# define LOWREGMASK(x) (x)
-#endif
 
 #define P_EXT  0x100   /* 0x0f opcode prefix */
 #define P_EXT38 0x200   /* 0x0f 0x38 opcode prefix */
-- 
2.17.1

[Qemu-devel] [PULL 07/13] qht: drop ht argument from qht iterators

[Richard Henderson]

From: "Emilio G. Cota" 

Accessing the HT from an iterator results almost always
in a deadlock. Given that only one qht-internal function
uses this argument, drop it from the interface.

Suggested-by: Alex Bennée 
Signed-off-by: Emilio G. Cota 
Signed-off-by: Richard Henderson 
---
 include/qemu/qht.h|  5 ++---
 accel/tcg/translate-all.c |  6 ++
 tests/test-qht.c  |  6 +++---
 util/qht.c| 29 +++--
 util/qsp.c| 11 +--
 5 files changed, 31 insertions(+), 26 deletions(-)

diff --git a/include/qemu/qht.h b/include/qemu/qht.h
index 3a9618db69..6484837487 100644
--- a/include/qemu/qht.h
+++ b/include/qemu/qht.h
@@ -43,9 +43,8 @@ struct qht_stats {
 };
 
 typedef bool (*qht_lookup_func_t)(const void *obj, const void *userp);
-typedef void (*qht_iter_func_t)(struct qht *ht, void *p, uint32_t h, void *up);
-typedef bool (*qht_iter_bool_func_t)(struct qht *ht, void *p, uint32_t h,
- void *up);
+typedef void (*qht_iter_func_t)(void *p, uint32_t h, void *up);
+typedef bool (*qht_iter_bool_func_t)(void *p, uint32_t h, void *up);
 
 #define QHT_MODE_AUTO_RESIZE 0x1 /* auto-resize when heavily loaded */
 #define QHT_MODE_RAW_MUTEXES 0x2 /* bypass the profiler (QSP) */
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 898c3bb3d1..9ffbbc2fbd 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1282,8 +1282,7 @@ void tb_flush(CPUState *cpu)
  */
 #ifdef CONFIG_USER_ONLY
 
-static void
-do_tb_invalidate_check(struct qht *ht, void *p, uint32_t hash, void *userp)
+static void do_tb_invalidate_check(void *p, uint32_t hash, void *userp)
 {
 TranslationBlock *tb = p;
 target_ulong addr = *(target_ulong *)userp;
@@ -1304,8 +1303,7 @@ static void tb_invalidate_check(target_ulong address)
 qht_iter(_ctx.htable, do_tb_invalidate_check, );
 }
 
-static void
-do_tb_page_check(struct qht *ht, void *p, uint32_t hash, void *userp)
+static void do_tb_page_check(void *p, uint32_t hash, void *userp)
 {
 TranslationBlock *tb = p;
 int flags1, flags2;
diff --git a/tests/test-qht.c b/tests/test-qht.c
index 1ec039d636..4d23cefab6 100644
--- a/tests/test-qht.c
+++ b/tests/test-qht.c
@@ -98,7 +98,7 @@ static void check(int a, int b, bool expected)
 qht_statistics_destroy();
 }
 
-static void count_func(struct qht *ht, void *p, uint32_t hash, void *userp)
+static void count_func(void *p, uint32_t hash, void *userp)
 {
 unsigned int *curr = userp;
 
@@ -122,7 +122,7 @@ static void iter_check(unsigned int count)
 g_assert_cmpuint(curr, ==, count);
 }
 
-static void sum_func(struct qht *ht, void *p, uint32_t hash, void *userp)
+static void sum_func(void *p, uint32_t hash, void *userp)
 {
 uint32_t *sum = userp;
 uint32_t a = *(uint32_t *)p;
@@ -138,7 +138,7 @@ static void iter_sum_check(unsigned int expected)
 g_assert_cmpuint(sum, ==, expected);
 }
 
-static bool rm_mod_func(struct qht *ht, void *p, uint32_t hash, void *userp)
+static bool rm_mod_func(void *p, uint32_t hash, void *userp)
 {
 uint32_t a = *(uint32_t *)p;
 unsigned int mod = *(unsigned int *)userp;
diff --git a/util/qht.c b/util/qht.c
index c190e89f5b..50ed7a2102 100644
--- a/util/qht.c
+++ b/util/qht.c
@@ -746,7 +746,7 @@ bool qht_remove(struct qht *ht, const void *p, uint32_t 
hash)
 return ret;
 }
 
-static inline void qht_bucket_iter(struct qht *ht, struct qht_bucket *head,
+static inline void qht_bucket_iter(struct qht_bucket *head,
const struct qht_iter *iter, void *userp)
 {
 struct qht_bucket *b = head;
@@ -759,10 +759,10 @@ static inline void qht_bucket_iter(struct qht *ht, struct 
qht_bucket *head,
 }
 switch (iter->type) {
 case QHT_ITER_VOID:
-iter->f.retvoid(ht, b->pointers[i], b->hashes[i], userp);
+iter->f.retvoid(b->pointers[i], b->hashes[i], userp);
 break;
 case QHT_ITER_RM:
-if (iter->f.retbool(ht, b->pointers[i], b->hashes[i], userp)) {
+if (iter->f.retbool(b->pointers[i], b->hashes[i], userp)) {
 /* replace i with the last valid element in the bucket */
 seqlock_write_begin(>sequence);
 qht_bucket_remove_entry(b, i);
@@ -782,14 +782,14 @@ static inline void qht_bucket_iter(struct qht *ht, struct 
qht_bucket *head,
 }
 
 /* call with all of the map's locks held */
-static inline void qht_map_iter__all_locked(struct qht *ht, struct qht_map 
*map,
+static inline void qht_map_iter__all_locked(struct qht_map *map,
 const struct qht_iter *iter,
 void *userp)
 {
 size_t i;
 
 for (i = 0; i < map->n_buckets; i++) {
-qht_bucket_iter(ht, >buckets[i], iter, userp);
+qht_bucket_iter(>buckets[i], iter, userp);
 }
 }
 
@@ 

[Qemu-devel] [PULL 05/13] test-qht: test deletion of the last entry in a bucket

[Richard Henderson]

From: "Emilio G. Cota" 

This improves coverage by one (!) LoC in qht.c, bringing the
coverage rate up from 90.00% to 90.28%.

Reviewed-by: Alex Bennée 
Signed-off-by: Emilio G. Cota 
Signed-off-by: Richard Henderson 
---
 tests/test-qht.c | 13 -
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/test-qht.c b/tests/test-qht.c
index 05b1d6807a..77666e8c5f 100644
--- a/tests/test-qht.c
+++ b/tests/test-qht.c
@@ -172,9 +172,20 @@ static void qht_do_test(unsigned int mode, size_t 
init_entries)
 
 qht_init(, is_equal, 0, mode);
 rm_nonexist(0, 4);
+/*
+ * Test that we successfully delete the last element in a bucket.
+ * This is a hard-to-reach code path when resizing is on, but without
+ * resizing we can easily hit it if init_entries <= 1.
+ * Given that the number of elements per bucket can be 4 or 6 depending on
+ * the host's pointer size, test the removal of the 4th and 6th elements.
+ */
 insert(0, 4);
 rm_nonexist(5, 6);
-insert(4, 6);
+rm(3, 4);
+check_n(3);
+insert(3, 6);
+rm(5, 6);
+check_n(5);
 rm_nonexist(7, 8);
 iter_rm_mod(1);
 
-- 
2.17.1

[Qemu-devel] [PULL 10/13] qht: constify qht_statistics_init

[Richard Henderson]

From: "Emilio G. Cota" 

Signed-off-by: Emilio G. Cota 
Signed-off-by: Richard Henderson 
---
 include/qemu/qht.h | 2 +-
 util/qht.c | 8 
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/include/qemu/qht.h b/include/qemu/qht.h
index 2e2d6bca93..758c7ac6c8 100644
--- a/include/qemu/qht.h
+++ b/include/qemu/qht.h
@@ -211,7 +211,7 @@ void qht_iter_remove(struct qht *ht, qht_iter_bool_func_t 
func, void *userp);
  * When done with @stats, pass the struct to qht_statistics_destroy().
  * Failing to do this will leak memory.
  */
-void qht_statistics_init(struct qht *ht, struct qht_stats *stats);
+void qht_statistics_init(const struct qht *ht, struct qht_stats *stats);
 
 /**
  * qht_statistics_destroy - Destroy a  qht_stats
diff --git a/util/qht.c b/util/qht.c
index 020dfe6912..4378775d68 100644
--- a/util/qht.c
+++ b/util/qht.c
@@ -895,9 +895,9 @@ bool qht_resize(struct qht *ht, size_t n_elems)
 }
 
 /* pass @stats to qht_statistics_destroy() when done */
-void qht_statistics_init(struct qht *ht, struct qht_stats *stats)
+void qht_statistics_init(const struct qht *ht, struct qht_stats *stats)
 {
-struct qht_map *map;
+const struct qht_map *map;
 int i;
 
 map = atomic_rcu_read(>map);
@@ -914,8 +914,8 @@ void qht_statistics_init(struct qht *ht, struct qht_stats 
*stats)
 stats->head_buckets = map->n_buckets;
 
 for (i = 0; i < map->n_buckets; i++) {
-struct qht_bucket *head = >buckets[i];
-struct qht_bucket *b;
+const struct qht_bucket *head = >buckets[i];
+const struct qht_bucket *b;
 unsigned int version;
 size_t buckets;
 size_t entries;
-- 
2.17.1

[Qemu-devel] [PULL 03/13] test-qht: test qht_iter_remove

[Richard Henderson]

From: "Emilio G. Cota" 

Reviewed-by: Alex Bennée 
Signed-off-by: Emilio G. Cota 
Signed-off-by: Richard Henderson 
---
 tests/test-qht.c | 50 ++--
 1 file changed, 48 insertions(+), 2 deletions(-)

diff --git a/tests/test-qht.c b/tests/test-qht.c
index dda6a067be..283fb3db39 100644
--- a/tests/test-qht.c
+++ b/tests/test-qht.c
@@ -108,6 +108,49 @@ static void iter_check(unsigned int count)
 g_assert_cmpuint(curr, ==, count);
 }
 
+static void sum_func(struct qht *ht, void *p, uint32_t hash, void *userp)
+{
+uint32_t *sum = userp;
+uint32_t a = *(uint32_t *)p;
+
+*sum += a;
+}
+
+static void iter_sum_check(unsigned int expected)
+{
+unsigned int sum = 0;
+
+qht_iter(, sum_func, );
+g_assert_cmpuint(sum, ==, expected);
+}
+
+static bool rm_mod_func(struct qht *ht, void *p, uint32_t hash, void *userp)
+{
+uint32_t a = *(uint32_t *)p;
+unsigned int mod = *(unsigned int *)userp;
+
+return a % mod == 0;
+}
+
+static void iter_rm_mod(unsigned int mod)
+{
+qht_iter_remove(, rm_mod_func, );
+}
+
+static void iter_rm_mod_check(unsigned int mod)
+{
+unsigned int expected = 0;
+unsigned int i;
+
+for (i = 0; i < N; i++) {
+if (i % mod == 0) {
+continue;
+}
+expected += i;
+}
+iter_sum_check(expected);
+}
+
 static void qht_do_test(unsigned int mode, size_t init_entries)
 {
 /* under KVM we might fetch stats from an uninitialized qht */
@@ -138,8 +181,11 @@ static void qht_do_test(unsigned int mode, size_t 
init_entries)
 insert(10, 150);
 check_n(N);
 
-rm(1, 2);
-check_n(N - 1);
+qht_reset();
+insert(0, N);
+iter_rm_mod(10);
+iter_rm_mod_check(10);
+check_n(N * 9 / 10);
 qht_reset_size(, 0);
 check_n(0);
 check(0, N, false);
-- 
2.17.1

[Qemu-devel] [PULL 08/13] qht: fix comment in qht_bucket_remove_entry

[Richard Henderson]

From: "Emilio G. Cota" 

Signed-off-by: Emilio G. Cota 
Signed-off-by: Richard Henderson 
---
 util/qht.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/util/qht.c b/util/qht.c
index 50ed7a2102..3564a7e20f 100644
--- a/util/qht.c
+++ b/util/qht.c
@@ -671,7 +671,7 @@ qht_entry_move(struct qht_bucket *to, int i, struct 
qht_bucket *from, int j)
 }
 
 /*
- * Find the last valid entry in @head, and swap it with @orig[pos], which has
+ * Find the last valid entry in @orig, and swap it with @orig[pos], which has
  * just been invalidated.
  */
 static inline void qht_bucket_remove_entry(struct qht_bucket *orig, int pos)
-- 
2.17.1

[Qemu-devel] [PULL 06/13] test-qht: speed up + test qht_resize

[Richard Henderson]

From: "Emilio G. Cota" 

Perform first the tests that exercise code paths that are
easier to hit at small table sizes, and then resize the table
to speed up subsequent tests. If this resize is not too large,
we can make the test faster with no code coverage loss.

- With gcov enabled:

Before: 20.568s, 90.28% qht.c coverage
After:   5.168s, 93.06% qht.c coverage

The coverage increase is entirely due to calling qht_resize,
which we weren't calling before. Note that the code paths
that remain to be tested are either error handling or
can only occur when several threads are accessing the
hash table concurrently (e.g. seqlock retry, trylock fail).

- Without gcov:

Before: 1.987s
After:  0.528s

The speedup is almost the same as with gcov, although the
"before" run is a lot faster.

Reviewed-by: Alex Bennée 
Signed-off-by: Emilio G. Cota 
Signed-off-by: Richard Henderson 
---
 tests/test-qht.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/tests/test-qht.c b/tests/test-qht.c
index 77666e8c5f..1ec039d636 100644
--- a/tests/test-qht.c
+++ b/tests/test-qht.c
@@ -189,6 +189,10 @@ static void qht_do_test(unsigned int mode, size_t 
init_entries)
 rm_nonexist(7, 8);
 iter_rm_mod(1);
 
+if (!(mode & QHT_MODE_AUTO_RESIZE)) {
+qht_resize(, init_entries * 4 + 4);
+}
+
 check_n(0);
 rm_nonexist(0, 10);
 insert(0, N);
-- 
2.17.1

[Qemu-devel] [PULL 04/13] test-qht: test removal of non-existent entries

[Richard Henderson]

From: "Emilio G. Cota" 

This improves qht.c code coverage from 89.44% to 90.00%.

Reviewed-by: Alex Bennée 
Signed-off-by: Emilio G. Cota 
Signed-off-by: Richard Henderson 
---
 tests/test-qht.c | 26 --
 1 file changed, 24 insertions(+), 2 deletions(-)

diff --git a/tests/test-qht.c b/tests/test-qht.c
index 283fb3db39..05b1d6807a 100644
--- a/tests/test-qht.c
+++ b/tests/test-qht.c
@@ -41,7 +41,7 @@ static void insert(int a, int b)
 }
 }
 
-static void rm(int init, int end)
+static void do_rm(int init, int end, bool exist)
 {
 int i;
 
@@ -49,10 +49,24 @@ static void rm(int init, int end)
 uint32_t hash;
 
 hash = arr[i];
-g_assert_true(qht_remove(, [i], hash));
+if (exist) {
+g_assert_true(qht_remove(, [i], hash));
+} else {
+g_assert_false(qht_remove(, [i], hash));
+}
 }
 }
 
+static void rm(int init, int end)
+{
+do_rm(init, end, true);
+}
+
+static void rm_nonexist(int init, int end)
+{
+do_rm(init, end, false);
+}
+
 static void check(int a, int b, bool expected)
 {
 struct qht_stats stats;
@@ -157,8 +171,15 @@ static void qht_do_test(unsigned int mode, size_t 
init_entries)
 check_n(0);
 
 qht_init(, is_equal, 0, mode);
+rm_nonexist(0, 4);
+insert(0, 4);
+rm_nonexist(5, 6);
+insert(4, 6);
+rm_nonexist(7, 8);
+iter_rm_mod(1);
 
 check_n(0);
+rm_nonexist(0, 10);
 insert(0, N);
 check(0, N, true);
 check_n(N);
@@ -183,6 +204,7 @@ static void qht_do_test(unsigned int mode, size_t 
init_entries)
 
 qht_reset();
 insert(0, N);
+rm_nonexist(N, N + 32);
 iter_rm_mod(10);
 iter_rm_mod_check(10);
 check_n(N * 9 / 10);
-- 
2.17.1

[Qemu-devel] [PULL 00/13] tcg-next patch queue

[Richard Henderson]

The following changes since commit c5e4e49258e9b89cb34c085a419dd9f862935c48:

  Merge remote-tracking branch 'remotes/xanclic/tags/pull-block-2018-09-25' 
into staging (2018-09-25 16:47:35 +0100)

are available in the Git repository at:

  https://github.com/rth7680/qemu.git tags/pull-tcg-20180926

for you to fetch changes up to 93bf9a42733321fb632bcb9eafd049ef0e3d9417:

  tcg/i386: fix vector operations on 32-bit hosts (2018-09-26 09:02:51 -0700)


Queued tcg patches


Emilio G. Cota (12):
  qht: remove unused map param from qht_remove__locked
  qht: add qht_iter_remove
  test-qht: test qht_iter_remove
  test-qht: test removal of non-existent entries
  test-qht: test deletion of the last entry in a bucket
  test-qht: speed up + test qht_resize
  qht: drop ht argument from qht iterators
  qht: fix comment in qht_bucket_remove_entry
  qht: constify qht_lookup
  qht: constify qht_statistics_init
  qht: constify arguments to some internal functions
  qht-bench: add -p flag to precompute hash values

Roman Kapl (1):
  tcg/i386: fix vector operations on 32-bit hosts

 include/qemu/qht.h|  26 +++--
 accel/tcg/translate-all.c |   6 +-
 tcg/i386/tcg-target.inc.c |   4 --
 tests/qht-bench.c |  26 +++--
 tests/test-qht.c  |  93 +--
 util/qht.c| 138 ++
 util/qsp.c|  11 ++--
 7 files changed, 241 insertions(+), 63 deletions(-)

[Qemu-devel] [PULL 12/13] qht-bench: add -p flag to precompute hash values

[Richard Henderson]

From: "Emilio G. Cota" 

Precomputing the hash values allows us to perform more frequent
accesses to the hash table, thereby reaching higher throughputs.

We keep the old behaviour by default, since (1) we might confuse
users if they measured a speedup without changing anything in
the QHT implementation, and (2) benchmarking the hash function
"on line" is also valuable.

Before:
$ taskset -c 0 tests/qht-bench -n 1
 Throughput:38.18 MT/s

After:
$ taskset -c 0 tests/qht-bench -n 1
 Throughput:38.16 MT/s

After (with precomputing):
$ taskset -c 0 tests/qht-bench -n 1 -p
 Throughput:50.87 MT/s

Signed-off-by: Emilio G. Cota 
Signed-off-by: Richard Henderson 
---
 tests/qht-bench.c | 26 --
 1 file changed, 20 insertions(+), 6 deletions(-)

diff --git a/tests/qht-bench.c b/tests/qht-bench.c
index f492b3a20a..2089e2bed1 100644
--- a/tests/qht-bench.c
+++ b/tests/qht-bench.c
@@ -53,6 +53,7 @@ static unsigned long resize_delay = 1000;
 static double resize_rate; /* 0.0 to 1.0 */
 static unsigned int n_rz_threads = 1;
 static QemuThread *rz_threads;
+static bool precompute_hash;
 
 static double update_rate; /* 0.0 to 1.0 */
 static uint64_t update_threshold;
@@ -101,11 +102,18 @@ static bool is_equal(const void *ap, const void *bp)
 return *a == *b;
 }
 
-static inline uint32_t h(unsigned long v)
+static uint32_t h(unsigned long v)
 {
 return tb_hash_func7(v, 0, 0, 0, 0);
 }
 
+static uint32_t hval(unsigned long v)
+{
+return v;
+}
+
+static uint32_t (*hfunc)(unsigned long v) = h;
+
 /*
  * From: https://en.wikipedia.org/wiki/Xorshift
  * This is faster than rand_r(), and gives us a wider range (RAND_MAX is only
@@ -149,7 +157,7 @@ static void do_rw(struct thread_info *info)
 bool read;
 
 p = [info->r & (lookup_range - 1)];
-hash = h(*p);
+hash = hfunc(*p);
 read = qht_lookup(, p, hash);
 if (read) {
 stats->rd++;
@@ -158,7 +166,7 @@ static void do_rw(struct thread_info *info)
 }
 } else {
 p = [info->r & (update_range - 1)];
-hash = h(*p);
+hash = hfunc(*p);
 if (info->write_op) {
 bool written = false;
 
@@ -289,7 +297,9 @@ static void htable_init(void)
 /* avoid allocating memory later by allocating all the keys now */
 keys = g_malloc(sizeof(*keys) * n);
 for (i = 0; i < n; i++) {
-keys[i] = populate_offset + i;
+long val = populate_offset + i;
+
+keys[i] = precompute_hash ? h(val) : hval(val);
 }
 
 /* some sanity checks */
@@ -321,7 +331,7 @@ static void htable_init(void)
 
 r = xorshift64star(r);
 p = [r & (init_range - 1)];
-hash = h(*p);
+hash = hfunc(*p);
 if (qht_insert(, p, hash, NULL)) {
 break;
 }
@@ -412,7 +422,7 @@ static void parse_args(int argc, char *argv[])
 int c;
 
 for (;;) {
-c = getopt(argc, argv, "d:D:g:k:K:l:hn:N:o:r:Rs:S:u:");
+c = getopt(argc, argv, "d:D:g:k:K:l:hn:N:o:pr:Rs:S:u:");
 if (c < 0) {
 break;
 }
@@ -451,6 +461,10 @@ static void parse_args(int argc, char *argv[])
 case 'o':
 populate_offset = atol(optarg);
 break;
+case 'p':
+precompute_hash = true;
+hfunc = hval;
+break;
 case 'r':
 update_range = pow2ceil(atol(optarg));
 break;
-- 
2.17.1

[Qemu-devel] [PULL 01/13] qht: remove unused map param from qht_remove__locked

[Richard Henderson]

From: "Emilio G. Cota" 

Reviewed-by: Alex Bennée 
Signed-off-by: Emilio G. Cota 
Signed-off-by: Richard Henderson 
---
 util/qht.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/util/qht.c b/util/qht.c
index 1e3a072e25..28d9273371 100644
--- a/util/qht.c
+++ b/util/qht.c
@@ -692,8 +692,7 @@ static inline void qht_bucket_remove_entry(struct 
qht_bucket *orig, int pos)
 
 /* call with b->lock held */
 static inline
-bool qht_remove__locked(struct qht_map *map, struct qht_bucket *head,
-const void *p, uint32_t hash)
+bool qht_remove__locked(struct qht_bucket *head, const void *p, uint32_t hash)
 {
 struct qht_bucket *b = head;
 int i;
@@ -728,7 +727,7 @@ bool qht_remove(struct qht *ht, const void *p, uint32_t 
hash)
 qht_debug_assert(p);
 
 b = qht_bucket_lock__no_stale(ht, hash, );
-ret = qht_remove__locked(map, b, p, hash);
+ret = qht_remove__locked(b, p, hash);
 qht_bucket_debug__locked(b);
 qemu_spin_unlock(>lock);
 return ret;
-- 
2.17.1

[Qemu-devel] [PULL 02/13] qht: add qht_iter_remove

[Richard Henderson]

From: "Emilio G. Cota" 

This currently has no users, but the use case is so common that I
think we must support it.

Note that without the appended we cannot safely remove a set of
elements; a 2-step approach (i.e. qht_iter first, keep track of
the to-be-deleted elements, and then a bunch of qht_remove calls)
would be racy, since between the iteration and the removals other
threads might insert additional elements.

Reviewed-by: Alex Bennée 
Signed-off-by: Emilio G. Cota 
Signed-off-by: Richard Henderson 
---
 include/qemu/qht.h | 19 
 util/qht.c | 74 +-
 2 files changed, 85 insertions(+), 8 deletions(-)

diff --git a/include/qemu/qht.h b/include/qemu/qht.h
index c9a11cc29a..3a9618db69 100644
--- a/include/qemu/qht.h
+++ b/include/qemu/qht.h
@@ -44,6 +44,8 @@ struct qht_stats {
 
 typedef bool (*qht_lookup_func_t)(const void *obj, const void *userp);
 typedef void (*qht_iter_func_t)(struct qht *ht, void *p, uint32_t h, void *up);
+typedef bool (*qht_iter_bool_func_t)(struct qht *ht, void *p, uint32_t h,
+ void *up);
 
 #define QHT_MODE_AUTO_RESIZE 0x1 /* auto-resize when heavily loaded */
 #define QHT_MODE_RAW_MUTEXES 0x2 /* bypass the profiler (QSP) */
@@ -179,9 +181,26 @@ bool qht_resize(struct qht *ht, size_t n_elems);
  *
  * Each time it is called, user-provided @func is passed a pointer-hash pair,
  * plus @userp.
+ *
+ * Note: @ht cannot be accessed from @func
+ * See also: qht_iter_remove()
  */
 void qht_iter(struct qht *ht, qht_iter_func_t func, void *userp);
 
+/**
+ * qht_iter_remove - Iterate over a QHT, optionally removing entries
+ * @ht: QHT to be iterated over
+ * @func: function to be called for each entry in QHT
+ * @userp: additional pointer to be passed to @func
+ *
+ * Each time it is called, user-provided @func is passed a pointer-hash pair,
+ * plus @userp. If @func returns true, the pointer-hash pair is removed.
+ *
+ * Note: @ht cannot be accessed from @func
+ * See also: qht_iter()
+ */
+void qht_iter_remove(struct qht *ht, qht_iter_bool_func_t func, void *userp);
+
 /**
  * qht_statistics_init - Gather statistics from a QHT
  * @ht: QHT to gather statistics from
diff --git a/util/qht.c b/util/qht.c
index 28d9273371..c190e89f5b 100644
--- a/util/qht.c
+++ b/util/qht.c
@@ -89,6 +89,19 @@
 #define QHT_BUCKET_ENTRIES 4
 #endif
 
+enum qht_iter_type {
+QHT_ITER_VOID,/* do nothing; use retvoid */
+QHT_ITER_RM,  /* remove element if retbool returns true */
+};
+
+struct qht_iter {
+union {
+qht_iter_func_t retvoid;
+qht_iter_bool_func_t retbool;
+} f;
+enum qht_iter_type type;
+};
+
 /*
  * Do _not_ use qemu_mutex_[try]lock directly! Use these macros, otherwise
  * the profiler (QSP) will deadlock.
@@ -733,9 +746,10 @@ bool qht_remove(struct qht *ht, const void *p, uint32_t 
hash)
 return ret;
 }
 
-static inline void qht_bucket_iter(struct qht *ht, struct qht_bucket *b,
-   qht_iter_func_t func, void *userp)
+static inline void qht_bucket_iter(struct qht *ht, struct qht_bucket *head,
+   const struct qht_iter *iter, void *userp)
 {
+struct qht_bucket *b = head;
 int i;
 
 do {
@@ -743,7 +757,25 @@ static inline void qht_bucket_iter(struct qht *ht, struct 
qht_bucket *b,
 if (b->pointers[i] == NULL) {
 return;
 }
-func(ht, b->pointers[i], b->hashes[i], userp);
+switch (iter->type) {
+case QHT_ITER_VOID:
+iter->f.retvoid(ht, b->pointers[i], b->hashes[i], userp);
+break;
+case QHT_ITER_RM:
+if (iter->f.retbool(ht, b->pointers[i], b->hashes[i], userp)) {
+/* replace i with the last valid element in the bucket */
+seqlock_write_begin(>sequence);
+qht_bucket_remove_entry(b, i);
+seqlock_write_end(>sequence);
+qht_bucket_debug__locked(b);
+/* reevaluate i, since it just got replaced */
+i--;
+continue;
+}
+break;
+default:
+g_assert_not_reached();
+}
 }
 b = b->next;
 } while (b);
@@ -751,26 +783,48 @@ static inline void qht_bucket_iter(struct qht *ht, struct 
qht_bucket *b,
 
 /* call with all of the map's locks held */
 static inline void qht_map_iter__all_locked(struct qht *ht, struct qht_map 
*map,
-qht_iter_func_t func, void *userp)
+const struct qht_iter *iter,
+void *userp)
 {
 size_t i;
 
 for (i = 0; i < map->n_buckets; i++) {
-qht_bucket_iter(ht, >buckets[i], func, userp);
+qht_bucket_iter(ht, >buckets[i], iter, userp);
 }

Re: [Qemu-devel] [PATCH] Add "boot_linux" acceptance test

[Philippe Mathieu-Daudé]

Hi Cleber,

On Thu, Sep 20, 2018 at 6:18 PM Cleber Rosa  wrote:
>
> This acceptance test, validates that a full blown Linux guest can
> successfully boot in QEMU.  In this specific case, the guest
> chosen is Fedora version 28.  By passing parameters, the same
> test can attempt to boot different distros, arches, etc.
>
> The method for checking the successfull boot is based on "cloudinit"
> and its "phone home" feature.  The guest is given an ISO image
> with the location of the phone home server, and the information to
> post (the instance ID).  Upon receiving the correct information,
> from the guest, the test is considered to have PASSed.
>
> This test is currently limited to user mode networking only, and
> instructs the guest to connect to the "router" address that is hard
> coded in QEMU.
>
> This test requires features present in Avocado version 64.0, and when
> running under Python 3, requires a fix to the avocado.utils.vmimage
> library (to be included in version 65.0).  To create the cloudinit ISO
> image that will be used to configure the guest, the pycdlib library is
> also required.  The idea for a effortless execution of this test, is
> to set those requirements, that is:
>
>avocado-framework==65.0
>pycdlib==1.6.0
>
> In the "tests/venv-requirements.txt" file introduced in another patch
> series.
>
> Reference: https://lists.gnu.org/archive/html/qemu-devel/2018-09/msg02503.html
> Reference: 
> https://github.com/avocado-framework/avocado/commit/02c47b1eade667d18fb0adef3293d86a6b5fd2e9
> Signed-off-by: Cleber Rosa 
> ---
>  tests/acceptance/boot_linux.py | 52 ++
>  1 file changed, 52 insertions(+)
>  create mode 100644 tests/acceptance/boot_linux.py
>
> diff --git a/tests/acceptance/boot_linux.py b/tests/acceptance/boot_linux.py
> new file mode 100644
> index 00..658211f15f
> --- /dev/null
> +++ b/tests/acceptance/boot_linux.py
> @@ -0,0 +1,52 @@
> +# Functional test that boots a complete Linux system via a cloud image
> +#
> +# Copyright (c) 2018 Red Hat, Inc.
> +#
> +# Author:
> +#  Cleber Rosa 
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later.  See the COPYING file in the top-level directory.
> +
> +import os
> +
> +from avocado_qemu import Test
> +
> +from avocado.utils import cloudinit
> +from avocado.utils import network
> +from avocado.utils import vmimage
> +
> +
> +class BootLinux(Test):
> +"""
> +Boots a Linux system, checking for a successful initialization
> +
> +:avocado: enable
> +"""
> +
> +timeout = 600
> +
> +def test(self):
> +self.vm.set_machine(self.params.get('machine', default='pc'))
> +self.vm.add_args('-accel', self.params.get('accel', default='kvm'))
> +self.vm.add_args('-smp', self.params.get('smp', default='2'))
> +self.vm.add_args('-m', self.params.get('memory', default='4096'))
> +
> +arch = self.params.get('arch', default=os.uname()[4])
> +distro = self.params.get('distro', default='fedora')
> +version = self.params.get('version', default='28')
> +boot = vmimage.get(distro, arch=arch, version=version,
> +   cache_dir=self.cache_dirs[0],
> +   snapshot_dir=self.workdir)
> +self.vm.add_args('-drive', 'file=%s' % boot.path)
> +
> +cloudinit_iso = os.path.join(self.workdir, 'cloudinit.iso')
> +phone_home_port = network.find_free_port()
> +cloudinit.iso(cloudinit_iso, self.name,
> +  # QEMU's hard coded usermode router address
> +  phone_home_host='10.0.2.2',
> +  phone_home_port=phone_home_port)
> +self.vm.add_args('-drive', 'file=%s' % cloudinit_iso)
> +
> +self.vm.launch()
> +cloudinit.wait_for_phone_home(('0.0.0.0', phone_home_port), 
> self.name)
> --
> 2.17.1
>

Using:

(venv) $ avocado run tests/acceptance

I'm getting:

DEBUG| DATA (filename=output.expected) => NOT FOUND (data sources:
variant, test, file)
DEBUG| PARAMS (key=qemu_bin, path=*,
default=x86_64-softmmu/qemu-system-x86_64) =>
'x86_64-softmmu/qemu-system-x86_64'
DEBUG| PARAMS (key=machine, path=*, default=pc) => 'pc'
DEBUG| PARAMS (key=accel, path=*, default=kvm) => 'kvm'
DEBUG| PARAMS (key=smp, path=*, default=2) => '2'
DEBUG| PARAMS (key=memory, path=*, default=4096) => '4096'
DEBUG| PARAMS (key=arch, path=*, default=x86_64) => 'x86_64'
DEBUG| PARAMS (key=distro, path=*, default=fedora) => 'fedora'
DEBUG| PARAMS (key=version, path=*, default=28) => '28'
ERROR|
ERROR| Reproduced traceback from:
/build/tests/venv/lib64/python3.6/site-packages/avocado/core/test.py:831
ERROR| Traceback (most recent call last):
ERROR|   File "/home/philmd/source/qemu/tests/acceptance/boot_linux.py",
line 40, in test
ERROR| snapshot_dir=self.workdir)
ERROR|   File 
"/build/tests/venv/lib64/python3.6/site-packages/avocado/utils/vmimage.py",
line 371, in get
ERROR| 

[Qemu-devel] [PATCH v2 0/2] Maintainership changes

[Jeff Cody]

Changes from v1:

- cc'ed Peter (meant to for v1, forgot)
- added John's 'jobs' branch to his git url
- kept myself as maintainer for vhdx
- moved sheepdog to 'Odd Fixes'

I'm not going to be involved with day-to-day qemu development, so
this necessitates some changes.

Jeff Cody (2):
  MAINTAINERS: Replace myself with John Snow for block jobs
  MAINTAINERS: Remove myself as block maintainer

 MAINTAINERS | 21 -
 1 file changed, 4 insertions(+), 17 deletions(-)

-- 
2.17.1

[Qemu-devel] [PATCH v2 2/2] MAINTAINERS: Remove myself as block maintainer

[Jeff Cody]

I'll not be involved in day-to-day qemu development.  Remove myself as
maintainer from the remainder of the network block drivers, and revert
them to the general block layer maintainership.

Move 'sheepdog' to the 'Odd Fixes' support level.

For VHDX, added my personal email address as a maintainer, as I can
answer questions or send the occassional bug fix.  Leaving it as
'Supported', instead of 'Odd Fixes', because I think the rest of the
block layer maintainers and developers will upkeep it as well, if
needed.

Signed-off-by: Jeff Cody 
---
 MAINTAINERS | 17 ++---
 1 file changed, 2 insertions(+), 15 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 0e22b795e6..7147c44bbc 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1988,28 +1988,23 @@ F: block/vmdk.c
 
 RBD
 M: Josh Durgin 
-M: Jeff Cody 
 L: qemu-bl...@nongnu.org
 S: Supported
 F: block/rbd.c
-T: git git://github.com/codyprime/qemu-kvm-jtc.git block
 
 Sheepdog
 M: Hitoshi Mitake 
 M: Liu Yuan 
-M: Jeff Cody 
 L: qemu-bl...@nongnu.org
 L: sheep...@lists.wpkg.org
-S: Supported
+S: Odd Fixes
 F: block/sheepdog.c
-T: git git://github.com/codyprime/qemu-kvm-jtc.git block
 
 VHDX
-M: Jeff Cody 
+M: Jeff Cody 
 L: qemu-bl...@nongnu.org
 S: Supported
 F: block/vhdx*
-T: git git://github.com/codyprime/qemu-kvm-jtc.git block
 
 VDI
 M: Stefan Weil 
@@ -2040,34 +2035,26 @@ F: docs/interop/nbd.txt
 T: git git://repo.or.cz/qemu/ericb.git nbd
 
 NFS
-M: Jeff Cody 
 M: Peter Lieven 
 L: qemu-bl...@nongnu.org
 S: Maintained
 F: block/nfs.c
-T: git git://github.com/codyprime/qemu-kvm-jtc.git block
 
 SSH
 M: Richard W.M. Jones 
-M: Jeff Cody 
 L: qemu-bl...@nongnu.org
 S: Supported
 F: block/ssh.c
-T: git git://github.com/codyprime/qemu-kvm-jtc.git block
 
 CURL
-M: Jeff Cody 
 L: qemu-bl...@nongnu.org
 S: Supported
 F: block/curl.c
-T: git git://github.com/codyprime/qemu-kvm-jtc.git block
 
 GLUSTER
-M: Jeff Cody 
 L: qemu-bl...@nongnu.org
 S: Supported
 F: block/gluster.c
-T: git git://github.com/codyprime/qemu-kvm-jtc.git block
 
 Null Block Driver
 M: Fam Zheng 
-- 
2.17.1

[Qemu-devel] [PATCH v2 1/2] MAINTAINERS: Replace myself with John Snow for block jobs

[Jeff Cody]

I'll not be involved with day-to-day qemu development, and John
Snow is a block jobs wizard.  Have him take over block job
maintainership duties.

Signed-off-by: Jeff Cody 
---
 MAINTAINERS | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index ce7c351afa..0e22b795e6 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1455,7 +1455,7 @@ F: include/scsi/*
 F: scsi/*
 
 Block Jobs
-M: Jeff Cody 
+M: John Snow 
 L: qemu-bl...@nongnu.org
 S: Supported
 F: blockjob.c
@@ -1468,7 +1468,7 @@ F: block/commit.c
 F: block/stream.c
 F: block/mirror.c
 F: qapi/job.json
-T: git git://github.com/codyprime/qemu-kvm-jtc.git block
+T: git git://github.com/jnsnow/qemu.git jobs
 
 Block QAPI, monitor, command line
 M: Markus Armbruster 
-- 
2.17.1

Re: [Qemu-devel] Converting PCIDevice to VirtIODevice

[Laszlo Ersek]

On 09/26/18 17:33, Sameeh Jubran wrote:
> Hi All,
> 
> I have used the function "pci_qdev_find_device" to find a device using
> it's id. This is a virtio device and I'm trying to convert it to
> VirtIODevice.
> 
> What's the best way to do this? Simply converting it to DeviceState
> doesn't work and I think I should access the underlying virtio pci bus
> and through it access the virtio-device, but couldn't find any elegant
> way of doing so.

pci_qdev_find_device() produces a pointer-to-PCIDevice.

In virtio_pci_realize() [hw/virtio/virtio-pci.c] we see that the
following works:

VirtIOPCIProxy *proxy = VIRTIO_PCI(pci_dev);

And in virtio_pci_pre_plugged() [hw/virtio/virtio-pci.c], we find:

VirtIODevice *vdev = virtio_bus_get_device(>bus);

Hope this helps (in fact, I hope it *works*! :) )
Laszlo

Re: [Qemu-devel] [PATCH v5 6/8] target/mips: Define the R5900 CPU

[Fredrik Noring]

Hi Jürgen, Maciej,

> The original website is down, but there is a backup:
> http://ps2linux.no-ip.info/playstation2-linux.com/faq.html#Availability__When_Where_and_how_much
> 
> There was also the question to release the documents for free.
> http://ps2linux.no-ip.info/playstation2-linux.com/forum/message23e0.html?msg_id=51027

Thanks, Jürgen!

> > > > +.CP0_PRid = 0x3800,
> > > 
> > > "The implementation number of the C790 processor is 0x38".
> > 
> >  I'll leave the answer to what the values of PRId are for the R5900 vs the 
> > C790 to Fredrik or Jürgen.  I suspect that the difference is in the Rev 
> > field only.
> 
> According to the documentation tx79architecture.pdf the C790 has 0x3800.
> According to the documentation coreum_e.pdf the R5900 has 0x2Exx.
> xx depends on the revision. I thought that I have seen several revisions,
> but in my log files I found only 0x2e31.

I have verified with hardware that model SCPH-77004 is 0x2e42 whereas an
older model such as SCPH-50004 is 0x2e31. Someone has made a substantial
table of models at

http://www.ps2-home.com/forum/app.php/page/ps2ident-database

where the columns "EE implementation" and "EE revision" correspond to PRId.

Fredrik

Re: [Qemu-devel] [PATCH 2/2] MAINTAINERS: Remove myself as block maintainer

[Jeff Cody]

On Tue, Sep 25, 2018 at 03:49:36PM +0800, Fam Zheng wrote:
> On Tue, 09/25 09:37, Markus Armbruster wrote:
> > Do we want to have a dedicated VHDX driver submaintainer again?  Fam,
> > you're maintaining VMDK, could you cover VHDX as well?
> 
> I don't know a lot VHDX internals. Considering my capacity at the moment I'd
> rather not take this one.
> 
> Fam

Anyone can feel free to email me at codypr...@gmail.com with any VHDX
questions (I am subbed to the qemu mailing list there as well).  I'll at
least do my best to answer :)

Re: [Qemu-devel] [PATCH v2 08/12] cpus: always call seqlock_write in cpu_update_icount

[Emilio G. Cota]

On Wed, Sep 26, 2018 at 10:23:25 +0200, Paolo Bonzini wrote:
> On 24/09/2018 20:46, Emilio G. Cota wrote:
> > Applying this on my local tree is deadlocking icount, since
> > cpu_update_icount is called from cpu_get_icount_raw_locked:
> > 
> > #6  cpu_update_icount (cpu=) at /data/src/qemu/cpus.c:257
> > #7  0x55a6fbc7ae5c in cpu_get_icount_raw_locked () at 
> > /data/src/qemu/cpus.c:271
> > #8  0x55a6fbc7ae99 in cpu_get_icount_locked () at 
> > /data/src/qemu/cpus.c:279
> > #9  0x55a6fbc7b3ac in cpu_get_icount () at /data/src/qemu/cpus.c:302
> > #10 0x55a6fc0f3a05 in qemu_clock_get_ns 
> > (type=type@entry=QEMU_CLOCK_VIRTUAL) at /data/src/qemu/util/qemu-timer.c:601
> > 
> > I am however not sure what Paolo's queued tree looks like, so I
> > might be missing something.
> 
> No, you're not missing anything.
> 
> Looking at other callers of cpu_update_icount, this should be the fix:
> 
> diff --git a/cpus.c b/cpus.c
(snip)

This does indeed fix the deadlock. Feel free to add my

Tested-by: Emilio G. Cota 

in the eventual patch.

Thanks,

Emilio

[Qemu-devel] [PULL 16/16] migration/ram.c: Avoid taking address of fields in packed MultiFDInit_t struct

[Dr. David Alan Gilbert (git)]

From: Peter Maydell 

Taking the address of a field in a packed struct is a bad idea, because
it might not be actually aligned enough for that pointer type (and
thus cause a crash on dereference on some host architectures). Newer
versions of clang warn about this:

migration/ram.c:651:19: warning: taking address of packed member 'magic' of 
class or structure 'MultiFDInit_t' may result in an unaligned pointer value 
[-Waddress-of-packed-member]
migration/ram.c:652:19: warning: taking address of packed member 'version' of 
class or structure 'MultiFDInit_t' may result in an unaligned pointer value 
[-Waddress-of-packed-member]
migration/ram.c:737:19: warning: taking address of packed member 'magic' of 
class or structure 'MultiFDPacket_t' may result in an unaligned pointer value 
[-Waddress-of-packed-member]
migration/ram.c:745:19: warning: taking address of packed member 'version' of 
class or structure 'MultiFDPacket_t' may result in an unaligned pointer value 
[-Waddress-of-packed-member]
migration/ram.c:755:19: warning: taking address of packed member 'size' of 
class or structure 'MultiFDPacket_t' may result in an unaligned pointer value 
[-Waddress-of-packed-member]

Avoid the bug by not using the "modify in place" byteswapping
functions.

Signed-off-by: Peter Maydell 
Message-Id: <20180925161924.7832-1-peter.mayd...@linaro.org>
Reviewed-by: Marc-André Lureau 
Signed-off-by: Dr. David Alan Gilbert 
---
 migration/ram.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 408d237700..bc38d98cc3 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -651,8 +651,8 @@ static int multifd_recv_initial_packet(QIOChannel *c, Error 
**errp)
 return -1;
 }
 
-be32_to_cpus();
-be32_to_cpus();
+msg.magic = be32_to_cpu(msg.magic);
+msg.version = be32_to_cpu(msg.version);
 
 if (msg.magic != MULTIFD_MAGIC) {
 error_setg(errp, "multifd: received packet magic %x "
@@ -737,7 +737,7 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, 
Error **errp)
 RAMBlock *block;
 int i;
 
-be32_to_cpus(>magic);
+packet->magic = be32_to_cpu(packet->magic);
 if (packet->magic != MULTIFD_MAGIC) {
 error_setg(errp, "multifd: received packet "
"magic %x and expected magic %x",
@@ -745,7 +745,7 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, 
Error **errp)
 return -1;
 }
 
-be32_to_cpus(>version);
+packet->version = be32_to_cpu(packet->version);
 if (packet->version != MULTIFD_VERSION) {
 error_setg(errp, "multifd: received packet "
"version %d and expected version %d",
@@ -755,7 +755,7 @@ static int multifd_recv_unfill_packet(MultiFDRecvParams *p, 
Error **errp)
 
 p->flags = be32_to_cpu(packet->flags);
 
-be32_to_cpus(>size);
+packet->size = be32_to_cpu(packet->size);
 if (packet->size > migrate_multifd_page_count()) {
 error_setg(errp, "multifd: received packet "
"with size %d and expected maximum size %d",
-- 
2.17.1

[Qemu-devel] [PULL 11/16] migration/postcopy: Clear have_listen_thread

[Dr. David Alan Gilbert (git)]

From: "Dr. David Alan Gilbert" 

Clear have_listen_thread when we exit the thread.
The fallout from this was that various things thought there was
an ongoing postcopy after the postcopy had finished.

The case that failed was postcopy->savevm->loadvm.

This corresponds to RH bug https://bugzilla.redhat.com/show_bug.cgi?id=1608765

Signed-off-by: Dr. David Alan Gilbert 
Message-Id: <20180914170430.54271-2-dgilb...@redhat.com>
Reviewed-by: Peter Xu 
Signed-off-by: Dr. David Alan Gilbert 
---
 migration/savevm.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/migration/savevm.c b/migration/savevm.c
index 9692577318..d35e87b88c 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -1679,6 +1679,7 @@ static void *postcopy_ram_listen_thread(void *opaque)
 qemu_loadvm_state_cleanup();
 
 rcu_unregister_thread();
+mis->have_listen_thread = false;
 return NULL;
 }
 
-- 
2.17.1

[Qemu-devel] [PULL 15/16] migration: fix the compression code

[Dr. David Alan Gilbert (git)]

From: Fei Li 

Add judgement in compress_threads_save_cleanup() to check whether the
static CompressParam *comp_param has been allocated. If not, just
return; or else segmentation fault will occur when using the NULL
comp_param's parameters.  One test case can reproduce this is: set
the compression on and migrate to a wrong nonexistent host IP address.

Our current code does not judge before handling comp_param[idx]'s quit
and cond that whether they have been initialized. If not initialized,
"qemu_mutex_lock_impl: Assertion `mutex->initialized' failed." will
occur. Fix this by squashing the terminate_compression_threads() into
compress_threads_save_cleanup() and employing the existing judgement
condition.  One test case can reproduce this error is: set the
compression on and fail to fully setup the default eight compression
thread in compress_threads_save_setup().

Signed-off-by: Fei Li 
Message-Id: <20180925091440.18910-1-...@suse.com>
Reviewed-by: Peter Xu 
Signed-off-by: Dr. David Alan Gilbert 
---
 migration/ram.c | 24 
 1 file changed, 8 insertions(+), 16 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 0fdaa8efa3..408d237700 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -431,28 +431,14 @@ static void *do_data_compress(void *opaque)
 return NULL;
 }
 
-static inline void terminate_compression_threads(void)
-{
-int idx, thread_count;
-
-thread_count = migrate_compress_threads();
-
-for (idx = 0; idx < thread_count; idx++) {
-qemu_mutex_lock(_param[idx].mutex);
-comp_param[idx].quit = true;
-qemu_cond_signal(_param[idx].cond);
-qemu_mutex_unlock(_param[idx].mutex);
-}
-}
-
 static void compress_threads_save_cleanup(void)
 {
 int i, thread_count;
 
-if (!migrate_use_compression()) {
+if (!migrate_use_compression() || !comp_param) {
 return;
 }
-terminate_compression_threads();
+
 thread_count = migrate_compress_threads();
 for (i = 0; i < thread_count; i++) {
 /*
@@ -462,6 +448,12 @@ static void compress_threads_save_cleanup(void)
 if (!comp_param[i].file) {
 break;
 }
+
+qemu_mutex_lock(_param[i].mutex);
+comp_param[i].quit = true;
+qemu_cond_signal(_param[i].cond);
+qemu_mutex_unlock(_param[i].mutex);
+
 qemu_thread_join(compress_threads + i);
 qemu_mutex_destroy(_param[i].mutex);
 qemu_cond_destroy(_param[i].cond);
-- 
2.17.1

Re: [Qemu-devel] [PATCH v2] qga: ignore non present cpus when handling qmp_guest_get_vcpus()

[Michael Roth]

Quoting Igor Mammedov (2018-09-06 07:51:54)
> If VM has VCPUs plugged sparselly (for example a VM started with
> 3 VCPUs (cpu0, cpu1 and cpu2) and then cpu1 was hotunplugged so
> only cpu0 and cpu2 are present), QGA will rise a error
>   error: internal error: unable to execute QEMU agent command 
> 'guest-get-vcpus':
>   open("/sys/devices/system/cpu/cpu1/"): No such file or directory
> when
>   virsh vcpucount FOO --guest
> is executed.
> Fix it by ignoring non present CPUs when fetching CPUs status from sysfs.
> 
> Signed-off-by: Igor Mammedov 

Thanks, applied to qga tree:
  https://github.com/mdroth/qemu/commits/qga

> ---
> v2:
>   do not create CPU entry if cpu isn't present
>   (Laszlo Ersek )
> ---
>  qga/commands-posix.c | 115 
> ++-
>  1 file changed, 59 insertions(+), 56 deletions(-)
> 
> diff --git a/qga/commands-posix.c b/qga/commands-posix.c
> index 37e8a2d..42d30f0 100644
> --- a/qga/commands-posix.c
> +++ b/qga/commands-posix.c
> @@ -2035,61 +2035,56 @@ static long sysconf_exact(int name, const char 
> *name_str, Error **errp)
>   * Written members remain unmodified on error.
>   */
>  static void transfer_vcpu(GuestLogicalProcessor *vcpu, bool sys2vcpu,
> -  Error **errp)
> +  char *dirpath, Error **errp)
>  {
> -char *dirpath;
> +int fd;
> +int res;
>  int dirfd;
> +static const char fn[] = "online";
> 
> -dirpath = g_strdup_printf("/sys/devices/system/cpu/cpu%" PRId64 "/",
> -  vcpu->logical_id);
>  dirfd = open(dirpath, O_RDONLY | O_DIRECTORY);
>  if (dirfd == -1) {
>  error_setg_errno(errp, errno, "open(\"%s\")", dirpath);
> -} else {
> -static const char fn[] = "online";
> -int fd;
> -int res;
> -
> -fd = openat(dirfd, fn, sys2vcpu ? O_RDONLY : O_RDWR);
> -if (fd == -1) {
> -if (errno != ENOENT) {
> -error_setg_errno(errp, errno, "open(\"%s/%s\")", dirpath, 
> fn);
> -} else if (sys2vcpu) {
> -vcpu->online = true;
> -vcpu->can_offline = false;
> -} else if (!vcpu->online) {
> -error_setg(errp, "logical processor #%" PRId64 " can't be "
> -   "offlined", vcpu->logical_id);
> -} /* otherwise pretend successful re-onlining */
> -} else {
> -unsigned char status;
> -
> -res = pread(fd, , 1, 0);
> -if (res == -1) {
> -error_setg_errno(errp, errno, "pread(\"%s/%s\")", dirpath, 
> fn);
> -} else if (res == 0) {
> -error_setg(errp, "pread(\"%s/%s\"): unexpected EOF", dirpath,
> -   fn);
> -} else if (sys2vcpu) {
> -vcpu->online = (status != '0');
> -vcpu->can_offline = true;
> -} else if (vcpu->online != (status != '0')) {
> -status = '0' + vcpu->online;
> -if (pwrite(fd, , 1, 0) == -1) {
> -error_setg_errno(errp, errno, "pwrite(\"%s/%s\")", 
> dirpath,
> - fn);
> -}
> -} /* otherwise pretend successful re-(on|off)-lining */
> +return;
> +}
> 
> -res = close(fd);
> -g_assert(res == 0);
> -}
> +fd = openat(dirfd, fn, sys2vcpu ? O_RDONLY : O_RDWR);
> +if (fd == -1) {
> +if (errno != ENOENT) {
> +error_setg_errno(errp, errno, "open(\"%s/%s\")", dirpath, fn);
> +} else if (sys2vcpu) {
> +vcpu->online = true;
> +vcpu->can_offline = false;
> +} else if (!vcpu->online) {
> +error_setg(errp, "logical processor #%" PRId64 " can't be "
> +   "offlined", vcpu->logical_id);
> +} /* otherwise pretend successful re-onlining */
> +} else {
> +unsigned char status;
> +
> +res = pread(fd, , 1, 0);
> +if (res == -1) {
> +error_setg_errno(errp, errno, "pread(\"%s/%s\")", dirpath, fn);
> +} else if (res == 0) {
> +error_setg(errp, "pread(\"%s/%s\"): unexpected EOF", dirpath,
> +   fn);
> +} else if (sys2vcpu) {
> +vcpu->online = (status != '0');
> +vcpu->can_offline = true;
> +} else if (vcpu->online != (status != '0')) {
> +status = '0' + vcpu->online;
> +if (pwrite(fd, , 1, 0) == -1) {
> +error_setg_errno(errp, errno, "pwrite(\"%s/%s\")", dirpath,
> + fn);
> +}
> +} /* otherwise pretend successful re-(on|off)-lining */
> 
> -res = close(dirfd);
> +res = close(fd);
>  g_assert(res == 0);
>  }
> 
> -g_free(dirpath);
> +res = close(dirfd);
> +g_assert(res == 0);
>  }
> 
>  GuestLogicalProcessorList 

[Qemu-devel] [PULL 12/16] migration: cleanup in error paths in loadvm

[Dr. David Alan Gilbert (git)]

From: "Dr. David Alan Gilbert" 

There's a couple of error paths in qemu_loadvm_state
which happen early on but after we've initialised the
load state; that needs to be cleaned up otherwise
we can hit asserts if the state gets reinitialised later.

Signed-off-by: Dr. David Alan Gilbert 
Message-Id: <20180914170430.54271-3-dgilb...@redhat.com>
Reviewed-by: Peter Xu 
Signed-off-by: Dr. David Alan Gilbert 
---
 migration/savevm.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/migration/savevm.c b/migration/savevm.c
index d35e87b88c..2d10e45582 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -2333,11 +2333,13 @@ int qemu_loadvm_state(QEMUFile *f)
 if (migrate_get_current()->send_configuration) {
 if (qemu_get_byte(f) != QEMU_VM_CONFIGURATION) {
 error_report("Configuration section missing");
+qemu_loadvm_state_cleanup();
 return -EINVAL;
 }
 ret = vmstate_load_state(f, _configuration, _state, 0);
 
 if (ret) {
+qemu_loadvm_state_cleanup();
 return ret;
 }
 }
-- 
2.17.1

[Qemu-devel] [PULL 10/16] tests/migration: Add migration-test header file

[Dr. David Alan Gilbert (git)]

From: Wei Huang 

This patch moves the settings related migration-test from the
migration-test.c file to a new header file.

Reviewed-by: Juan Quintela 
Reviewed-by: Andrew Jones 
Signed-off-by: Wei Huang 
Message-Id: <1536174934-26022-4-git-send-email-...@redhat.com>
Signed-off-by: Juan Quintela 
Signed-off-by: Dr. David Alan Gilbert 
---
 tests/migration-test.c   | 28 ++--
 tests/migration/migration-test.h | 21 +
 2 files changed, 39 insertions(+), 10 deletions(-)
 create mode 100644 tests/migration/migration-test.h

diff --git a/tests/migration-test.c b/tests/migration-test.c
index fe6b41a38d..17c689601f 100644
--- a/tests/migration-test.c
+++ b/tests/migration-test.c
@@ -21,11 +21,13 @@
 #include "chardev/char.h"
 #include "sysemu/sysemu.h"
 
+#include "migration/migration-test.h"
+
 /* TODO actually test the results and get rid of this */
 #define qtest_qmp_discard_response(...) qobject_unref(qtest_qmp(__VA_ARGS__))
 
-const unsigned start_address = 1024 * 1024;
-const unsigned end_address = 100 * 1024 * 1024;
+unsigned start_address;
+unsigned end_address;
 bool got_stop;
 static bool uffd_feature_thread_id;
 
@@ -80,8 +82,8 @@ static bool ufd_version_check(void)
 
 static const char *tmpfs;
 
-/* A simple PC boot sector that modifies memory (1-100MB) quickly
- * outputting a 'B' every so often if it's still running.
+/* The boot file modifies memory area in [start_address, end_address)
+ * repeatedly. It outputs a 'B' at a fixed rate while it's still running.
  */
 #include "tests/migration/i386/a-b-bootblock.h"
 
@@ -270,11 +272,11 @@ static void wait_for_migration_pass(QTestState *who)
 static void check_guests_ram(QTestState *who)
 {
 /* Our ASM test will have been incrementing one byte from each page from
- * 1MB to <100MB in order.
- * This gives us a constraint that any page's byte should be equal or less
- * than the previous pages byte (mod 256); and they should all be equal
- * except for one transition at the point where we meet the incrementer.
- * (We're running this with the guest stopped).
+ * start_address to < end_address in order. This gives us a constraint
+ * that any page's byte should be equal or less than the previous pages
+ * byte (mod 256); and they should all be equal except for one transition
+ * at the point where we meet the incrementer. (We're running this with
+ * the guest stopped).
  */
 unsigned address;
 uint8_t first_byte;
@@ -285,7 +287,8 @@ static void check_guests_ram(QTestState *who)
 qtest_memread(who, start_address, _byte, 1);
 last_byte = first_byte;
 
-for (address = start_address + 4096; address < end_address; address += 
4096)
+for (address = start_address + TEST_MEM_PAGE_SIZE; address < end_address;
+ address += TEST_MEM_PAGE_SIZE)
 {
 uint8_t b;
 qtest_memread(who, address, , 1);
@@ -437,6 +440,8 @@ static int test_migrate_start(QTestState **from, QTestState 
**to,
   " -drive file=%s,format=raw"
   " -incoming %s",
   accel, tmpfs, bootpath, uri);
+start_address = X86_TEST_MEM_START;
+end_address = X86_TEST_MEM_END;
 } else if (strcmp(arch, "ppc64") == 0) {
 cmd_src = g_strdup_printf("-machine accel=%s -m 256M"
   " -name source,debug-threads=on"
@@ -451,6 +456,9 @@ static int test_migrate_start(QTestState **from, QTestState 
**to,
   " -serial file:%s/dest_serial"
   " -incoming %s",
   accel, tmpfs, uri);
+
+start_address = PPC_TEST_MEM_START;
+end_address = PPC_TEST_MEM_END;
 } else {
 g_assert_not_reached();
 }
diff --git a/tests/migration/migration-test.h b/tests/migration/migration-test.h
new file mode 100644
index 00..c4c0c526b6
--- /dev/null
+++ b/tests/migration/migration-test.h
@@ -0,0 +1,21 @@
+/*
+ * Copyright (c) 2018 Red Hat, Inc. and/or its affiliates
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#ifndef _TEST_MIGRATION_H_
+#define _TEST_MIGRATION_H_
+
+/* Common */
+#define TEST_MEM_PAGE_SIZE 4096
+
+/* x86 */
+#define X86_TEST_MEM_START (1 * 1024 * 1024)
+#define X86_TEST_MEM_END   (100 * 1024 * 1024)
+
+/* PPC */
+#define PPC_TEST_MEM_START (1 * 1024 * 1024)
+#define PPC_TEST_MEM_END   (100 * 1024 * 1024)
+
+#endif /* _TEST_MIGRATION_H_ */
-- 
2.17.1

[Qemu-devel] [PULL 13/16] tests/migration: Speed up the test on ppc64

[Dr. David Alan Gilbert (git)]

From: Thomas Huth 

The SLOF boot process is always quite slow ... but we can speed it up
a little bit by specifying "-nodefaults" and by using the "nvramrc"
variable instead of "boot-command" (since "nvramrc" is evaluated earlier
in the SLOF boot process than "boot-command").

Signed-off-by: Thomas Huth 
Message-Id: <1537204330-16076-1-git-send-email-th...@redhat.com>
Reviewed-by: Dr. David Alan Gilbert 
Reviewed-by: Laurent Vivier 
Signed-off-by: Dr. David Alan Gilbert 
---
 tests/migration-test.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tests/migration-test.c b/tests/migration-test.c
index 17c689601f..20f38f1930 100644
--- a/tests/migration-test.c
+++ b/tests/migration-test.c
@@ -443,11 +443,11 @@ static int test_migrate_start(QTestState **from, 
QTestState **to,
 start_address = X86_TEST_MEM_START;
 end_address = X86_TEST_MEM_END;
 } else if (strcmp(arch, "ppc64") == 0) {
-cmd_src = g_strdup_printf("-machine accel=%s -m 256M"
+cmd_src = g_strdup_printf("-machine accel=%s -m 256M -nodefaults"
   " -name source,debug-threads=on"
   " -serial file:%s/src_serial"
-  " -prom-env '"
-  "boot-command=hex .\" _\" begin %x %x "
+  " -prom-env 'use-nvramrc?=true' -prom-env "
+  "'nvramrc=hex .\" _\" begin %x %x "
   "do i c@ 1 + i c! 1000 +loop .\" B\" 0 "
   "until'",  accel, tmpfs, end_address,
   start_address);
-- 
2.17.1

[Qemu-devel] [PULL 06/16] migration: show the statistics of compression

[Dr. David Alan Gilbert (git)]

From: Xiao Guangrong 

Currently, it includes:
pages: amount of pages compressed and transferred to the target VM
busy: amount of count that no free thread to compress data
busy-rate: rate of thread busy
compressed-size: amount of bytes after compression
compression-rate: rate of compressed size

Reviewed-by: Juan Quintela 
Reviewed-by: Peter Xu 
Signed-off-by: Xiao Guangrong 
Message-Id: <20180906070101.27280-3-xiaoguangr...@tencent.com>
Signed-off-by: Juan Quintela 
Signed-off-by: Dr. David Alan Gilbert 
---
 hmp.c | 13 +
 migration/migration.c | 12 
 migration/ram.c   | 41 -
 migration/ram.h   |  1 +
 qapi/migration.json   | 26 +-
 5 files changed, 91 insertions(+), 2 deletions(-)

diff --git a/hmp.c b/hmp.c
index 3a9f797677..61ef120423 100644
--- a/hmp.c
+++ b/hmp.c
@@ -271,6 +271,19 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict)
info->xbzrle_cache->overflow);
 }
 
+if (info->has_compression) {
+monitor_printf(mon, "compression pages: %" PRIu64 " pages\n",
+   info->compression->pages);
+monitor_printf(mon, "compression busy: %" PRIu64 "\n",
+   info->compression->busy);
+monitor_printf(mon, "compression busy rate: %0.2f\n",
+   info->compression->busy_rate);
+monitor_printf(mon, "compressed size: %" PRIu64 "\n",
+   info->compression->compressed_size);
+monitor_printf(mon, "compression rate: %0.2f\n",
+   info->compression->compression_rate);
+}
+
 if (info->has_cpu_throttle_percentage) {
 monitor_printf(mon, "cpu throttle percentage: %" PRIu64 "\n",
info->cpu_throttle_percentage);
diff --git a/migration/migration.c b/migration/migration.c
index 05d0a7296a..6a7731de50 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -758,6 +758,18 @@ static void populate_ram_info(MigrationInfo *info, 
MigrationState *s)
 info->xbzrle_cache->overflow = xbzrle_counters.overflow;
 }
 
+if (migrate_use_compression()) {
+info->has_compression = true;
+info->compression = g_malloc0(sizeof(*info->compression));
+info->compression->pages = compression_counters.pages;
+info->compression->busy = compression_counters.busy;
+info->compression->busy_rate = compression_counters.busy_rate;
+info->compression->compressed_size =
+compression_counters.compressed_size;
+info->compression->compression_rate =
+compression_counters.compression_rate;
+}
+
 if (cpu_throttle_active()) {
 info->has_cpu_throttle_percentage = true;
 info->cpu_throttle_percentage = cpu_throttle_get_percentage();
diff --git a/migration/ram.c b/migration/ram.c
index 2c039892d3..7c12f2792c 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -301,6 +301,15 @@ struct RAMState {
 uint64_t num_dirty_pages_period;
 /* xbzrle misses since the beginning of the period */
 uint64_t xbzrle_cache_miss_prev;
+
+/* compression statistics since the beginning of the period */
+/* amount of count that no free thread to compress data */
+uint64_t compress_thread_busy_prev;
+/* amount bytes after compression */
+uint64_t compressed_size_prev;
+/* amount of compressed pages */
+uint64_t compress_pages_prev;
+
 /* total handled target pages at the beginning of period */
 uint64_t target_page_count_prev;
 /* total handled target pages since start */
@@ -338,6 +347,8 @@ struct PageSearchStatus {
 };
 typedef struct PageSearchStatus PageSearchStatus;
 
+CompressionStats compression_counters;
+
 struct CompressParam {
 bool done;
 bool quit;
@@ -1593,6 +1604,7 @@ uint64_t ram_pagesize_summary(void)
 static void migration_update_rates(RAMState *rs, int64_t end_time)
 {
 uint64_t page_count = rs->target_page_count - rs->target_page_count_prev;
+double compressed_size;
 
 /* calculate period counters */
 ram_counters.dirty_pages_rate = rs->num_dirty_pages_period * 1000
@@ -1607,6 +1619,26 @@ static void migration_update_rates(RAMState *rs, int64_t 
end_time)
 rs->xbzrle_cache_miss_prev) / page_count;
 rs->xbzrle_cache_miss_prev = xbzrle_counters.cache_miss;
 }
+
+if (migrate_use_compression()) {
+compression_counters.busy_rate = (double)(compression_counters.busy -
+rs->compress_thread_busy_prev) / page_count;
+rs->compress_thread_busy_prev = compression_counters.busy;
+
+compressed_size = compression_counters.compressed_size -
+  rs->compressed_size_prev;
+if (compressed_size) {
+double uncompressed_size = (compression_counters.pages -
+rs->compress_pages_prev) 

Re: [Qemu-devel] [PATCH v3 1/5] qga: win32: fix crashes when PCI info cannot be retrived

[Michael Roth]

Quoting Tomáš Golembiovský (2018-09-07 06:42:09)
> The guest-get-fsinfo command collects also information about PCI
> controller where the disk is attached. When this fails for some reasons
> it tries to return just the partial information. However in certain
> cases the pointer to the structure was not initialized and was set to
> NULL. This breaks the serializer and leads to a crash of the guest agent.
> 
> Signed-off-by: Tomáš Golembiovský 

For a win10 guest started with:

qemu-system-x86_64 -m 2G -smp 2,cores=2,sockets=1 -drive 
file=/home/mdroth/vm/win10_pro_n_snap0.qcow2,if=virtio -drive 
file=/home/mdroth/vm/virtio-win-0.1.102.iso,if=ide,media=cdrom -rtc 
base=localtime,driftfix=slew -vga std -boot d -name vm4 -netdev 
tap,script=/etc/qemu-ifup,vhost=on,id=vnet0 -device 
virtio-net-pci,mac=52:54:00:12:34:04,id=vnic0,netdev=vnet0,disable-modern=true 
-vnc :4 -device virtio-serial -balloon virtio -mon chardev=hmp0 -chardev 
socket,path=/tmp/vm4-hmp0.sock,server,nowait,id=hmp0 -mon 
chardev=qmp0,mode=control -chardev 
socket,path=/tmp/vm4-qmp0.sock,server,nowait,id=qmp0 -device 
virtserialport,chardev=vs0,name=vs0,id=vs_vs0 -chardev 
socket,path=/tmp/vm4-vs0.sock,server,nowait,id=vs0 -device 
virtserialport,chardev=vs1,name=vs1,id=vs_vs1 -chardev 
socket,path=/tmp/vm4-vs1.sock,server,nowait,id=vs1 -device 
virtserialport,chardev=qga,name=org.qemu.guest_agent.0,id=vs_qga -chardev 
socket,path=/tmp/vm4-qga.sock,server,nowait,id=qga -device 
isa-serial,chardev=serial0,id=serial_serial0 -chardev 
socket,path=/tmp/vm4-serial0.sock,server,nowait,id=serial0 -L 
/home/mdroth/w/build/qemu-2.11.2-build/pc-bios --enable-kvm

this yields the following:

{'execute':'guest-get-fsinfo'}
{"return": [{"name": "?\\Volume{83835b2d-0032-11e6-a84f-806e6f6e6963}\\", 
"total-bytes": 160755712, "mountpoint": "D:\\", "disk": [{"bus-type": "ide", 
"bus": 0, "unit": 0, "pci-controller": {"bus": 0, "slot": 0, "domain": 0, 
"function": 0}, "target": 0}], "used-bytes": 160755712, "type": "CDFS"}, 
{"name": "?\\Volume{2ea839c6----80620c00}\\", "mountpoint": 
"System Reserved", "disk": [{"bus-type": "scsi", "bus": 0, "unit": 0, 
"pci-controller": {"bus": 0, "slot": 0, "domain": 0, "function": 0}, "target": 
0}], "type": "NTFS"}, {"name": 
"?\\Volume{2ea839c6----501f}\\", "total-bytes": 
52665839616, "mountpoint": "C:\\", "disk": [{"bus-type": "scsi", "bus": 0, 
"unit": 0, "pci-controller": {"bus": 0, "slot": 0, "domain": 0, "function": 0}, 
"target": 0}], "used-bytes": 25265487872, "type": "NTFS"}, {"name": 
"?\\Volume{2ea839c6----1000}\\", "mountpoint": "System 
Reserved", "disk": [{"bus-type": "scsi", "bus": 0, "unit": 0, "pci-controller": 
{"bus": 0, "slot": 0, "domain": 0, "function": 0}, "target": 0}], "type": 
"NTFS"}]}

domain/bus/slot/function=0 are valid PCI addresses so initializing to 0 is
wrong. Sameeh had a previous series that initializes to -1 that I think
is more appropriate (it hasn't gone in yet since we opted not to enable
CONFIG_QGA_NTDDSCSI for 3.0 since the PCI stuff seems be generally
broken for Windows, also because the 2nd patch needs some fixups:

  https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg07500.html

With that series (and some fixups I have on top at
https://github.com/mdroth/qemu/commits/qga-test), we get the following
output:

{'execute':'guest-get-fsinfo'}
{"return": [{"name": "?\\Volume{83835b2d-0032-11e6-a84f-806e6f6e6963}\\", 
"total-bytes": 160755712, "mountpoint": "D:\\", "disk": [{"bus-type": "ide", 
"bus": 0, "unit": 0, "pci-controller": {"bus": -1, "slot": -1, "domain": -1, 
"function": -1}, "target": 0}], "used-bytes": 160755712, "type": "CDFS"}, 
{"name": "?\\Volume{2ea839c6----80620c00}\\", "mountpoint": 
"System Reserved", "disk": [{"bus-type": "scsi", "bus": 0, "unit": 0, 
"pci-controller": {"bus": -1, "slot": -1, "domain": 0, "function": 3}, 
"target": 0}], "type": "NTFS"}, {"name": 
"?\\Volume{2ea839c6----501f}\\", "total-bytes": 
52665839616, "mountpoint": "C:\\", "disk": [{"bus-type": "scsi", "bus": 0, 
"unit": 0, "pci-controller": {"bus": -1, "slot": -1, "domain": 0, "function": 
2}, "target": 0}], "used-bytes": 25267560448, "type": "NTFS"}, {"name": 
"?\\Volume{2ea839c6----1000}\\", "mountpoint": "System 
Reserved", "disk": [{"bus-type": "scsi", "bus": 0, "unit": 0, "pci-controller": 
{"bus": -1, "slot": -1, "domain": 0, "function": 1}, "target": 0}], "type": 
"NTFS"}]}

Here we see the non-sensical PCI topology info I mentioned previously.
There are values like '"function": 3' even though there are no
multifunction devices present. This will be exposed to users if we
enable CONFIG_QGA_NTDDSCSI with things as they stand. Currently we
just get an empty array for "disk" field of GuestFilesystemInfo
for w32, which fortunately aligns with the current QAPI schema (it's
an array since the volume can span multiple disks). I'm not