Re: [PATCH v4 10/33] i386: Add get/set/migrate support for SGX_LEPUBKEYHASH MSRs

[Philippe Mathieu-Daudé]

On 9/16/21 8:08 AM, Yang Zhong wrote:
> On Tue, Sep 14, 2021 at 08:38:59AM +0200, Philippe Mathieu-Daudé wrote:
>> On 7/19/21 1:21 PM, Yang Zhong wrote:
>>> From: Sean Christopherson 
>>>
>>> On real hardware, on systems that supports SGX Launch Control, those
>>> MSRs are initialized to digest of Intel's signing key; on systems that
>>> don't support SGX Launch Control, those MSRs are not available but
>>> hardware always uses digest of Intel's signing key in EINIT.
>>>
>>> KVM advertises SGX LC via CPUID if and only if the MSRs are writable.
>>> Unconditionally initialize those MSRs to digest of Intel's signing key
>>> when CPU is realized and reset to reflect the fact. This avoids
>>> potential bug in case kvm_arch_put_registers() is called before
>>> kvm_arch_get_registers() is called, in which case guest's virtual
>>> SGX_LEPUBKEYHASH MSRs will be set to 0, although KVM initializes those
>>> to digest of Intel's signing key by default, since KVM allows those MSRs
>>> to be updated by Qemu to support live migration.
>>>
>>> Save/restore the SGX Launch Enclave Public Key Hash MSRs if SGX Launch
>>> Control (LC) is exposed to the guest. Likewise, migrate the MSRs if they
>>> are writable by the guest.
>>>
>>> Signed-off-by: Sean Christopherson 
>>> Signed-off-by: Kai Huang 
>>> Signed-off-by: Yang Zhong 
>>> ---
>>>  target/i386/cpu.c | 17 -
>>>  target/i386/cpu.h |  1 +
>>>  target/i386/kvm/kvm.c | 22 ++
>>>  target/i386/machine.c | 20 
>>>  4 files changed, 59 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
>>> index 840f825431..cea4307930 100644
>>> --- a/target/i386/cpu.c
>>> +++ b/target/i386/cpu.c
>>> @@ -5673,6 +5673,16 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, 
>>> uint32_t count,
>>>  }
>>>  }
>>>  
>>> +#ifndef CONFIG_USER_ONLY
>>> +static void x86_cpu_set_sgxlepubkeyhash(CPUX86State *env)
>>> +{
>>> +env->msr_ia32_sgxlepubkeyhash[0] = 0xa6053e051270b7acULL;
>>> +env->msr_ia32_sgxlepubkeyhash[1] = 0x6cfbe8ba8b3b413dULL;
>>> +env->msr_ia32_sgxlepubkeyhash[2] = 0xc4916d99f2b3735dULL;
>>> +env->msr_ia32_sgxlepubkeyhash[3] = 0xd4f8c05909f9bb3bULL;
>>> +}
>>> +#endif
>>
>> Maybe easier to move the #ifdef'ry inside the function.
>>
> 
>   Thanks for comments, since this function is pure void function, we can move 
> this #ifdef
>   into function.
>  
>> Where these values come from btw?
> 
>   Those MSR values are intel default values, which were defined in Skylake 
> platform.

Could you add a comment (and reference if possible) about them please?

Thanks,

Phil.

Re: [PATCH v2 1/3] monitor: Add HMP and QMP interfaces

[Yang Zhong]

On Mon, Sep 13, 2021 at 02:24:56PM +0100, Daniel P. Berrangé wrote:
> On Mon, Sep 13, 2021 at 08:52:28PM +0800, Yang Zhong wrote:
> > On Mon, Sep 13, 2021 at 01:56:13PM +0100, Daniel P. Berrangé wrote:
> > > On Mon, Sep 13, 2021 at 02:48:37PM +0200, Paolo Bonzini wrote:
> > > > On 13/09/21 11:35, Daniel P. Berrangé wrote:
> > > > > >g_autoptr(Error) err = NULL
> > > > > I was mistaken here - Error shouldn't use g_autoptr, just
> > > > > 
> > > > > Error err = NULL;
> > > > > 
> > > > > > > +SGXInfo *info = qmp_query_sgx(NULL);
> > > > > > Pass in &err not NULL;
> > > > > > 
> > > > > > Also  declare it with  'g_autoptr(SGXInfo) info = ...'
> > > > > > 
> > > > > > And then
> > > > > > 
> > > > > > if (err) {
> > > > > >monitor_printf(mon, "%s\n", error_get_pretty(err);
> > > > > Then use the simpler:
> > > > > 
> > > > >  error_report_err(err);
> > > > 
> > > > Indeed.
> > > > 
> > > > That said, more long term (but this is something Coccinelle could help 
> > > > with)
> > > > perhaps error_report_err should not free the error, and instead we 
> > > > should
> > > > use g_autoptr(Error) in the callers.  I don't like functions that do not
> > > > have free in their name and yet free a pointer...
> > > 
> > > Yes, this error_report_err surprises me every 6 months when I
> > > come to deal with it. So I think using g_autoptr would be a
> > > nice replacement, with no additional burden in terms of lines
> > > of code in callers too.
> > >
> > 
> >   Do we need call qapi_free_SGXInfo(info) here?
> > 
> >   In previous code design, the code like below:
> > 
> >   SGXInfo *info = qmp_query_sgx(&err);
> >   ..
> >   qapi_free_SGXInfo(info);
> 
> I suggested "g_autoptr(SGXInfo) info" for the declaration to avoid
> the need for qapi_free_SGXInfo calls
>

  Daniel, thanks!

  Paolo, i checked the sgx branch of your gitlab, we need add this definition
  "g_autoptr(SGXInfo) info" into hmp_info_sgx() function. thanks!

  Yang
   
> 
> Regards,
> Daniel
> -- 
> |: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o-https://fstop138.berrange.com :|
> |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [RFC v6] virtio/vsock: add two more queues for datagram types

[Stefano Garzarella]

On Wed, Sep 15, 2021 at 08:59:17PM -0700, Jiang Wang . wrote:

On Tue, Sep 14, 2021 at 5:46 AM Stefan Hajnoczi  wrote:


On Mon, Sep 13, 2021 at 10:18:43PM +, Jiang Wang wrote:
> Datagram sockets are connectionless and unreliable.
> The sender does not know the capacity of the receiver
> and may send more packets than the receiver can handle.
>
> Add two more dedicate virtqueues for datagram sockets,
> so that it will not unfairly steal resources from
> stream and future connection-oriented sockets.
>
> Signed-off-by: Jiang Wang 
> ---
> v1 -> v2: use qemu cmd option to control number of queues,
> removed configuration settings for dgram.
> v2 -> v3: use ioctl to get features and decide number of
> virt queues, instead of qemu cmd option.
> v3 -> v4: change DGRAM feature bit value to 2. Add an argument
> in vhost_vsock_common_realize to indicate dgram is supported or not.
> v4 -> v5: don't open dev to get vhostfd. Removed leftover definition of
> enable_dgram
> v5 -> v6: fix style errors. Imporve error handling of
> vhost_vsock_dgram_supported. Rename MAX_VQS_WITH_DGRAM and another 
one.
>
>  hw/virtio/vhost-user-vsock.c  |  2 +-
>  hw/virtio/vhost-vsock-common.c| 25 --
>  hw/virtio/vhost-vsock.c   | 34 ++-
>  include/hw/virtio/vhost-vsock-common.h|  6 ++--
>  include/hw/virtio/vhost-vsock.h   |  3 ++
>  include/standard-headers/linux/virtio_vsock.h |  1 +
>  6 files changed, 64 insertions(+), 7 deletions(-)
>
> diff --git a/hw/virtio/vhost-user-vsock.c b/hw/virtio/vhost-user-vsock.c
> index 6095ed7349..e9ec0e1c00 100644
> --- a/hw/virtio/vhost-user-vsock.c
> +++ b/hw/virtio/vhost-user-vsock.c
> @@ -105,7 +105,7 @@ static void vuv_device_realize(DeviceState *dev, Error 
**errp)
>  return;
>  }
>
> -vhost_vsock_common_realize(vdev, "vhost-user-vsock");
> +vhost_vsock_common_realize(vdev, "vhost-user-vsock", false);

VIRTIO_VSOCK_F_DGRAM support should work equally well for
vhost-user-vsock. I don't think there is a need to disable it here.


Stefano mentioned in previous email ( V3 ) that I can disable dgram
for vhost-user-vsock for now. I think we need some extra changes to
fully support vhost-vsock-user, like feature discovery?


I think Stefan is suggesting something similar of what we discussed 
here:

https://lists.nongnu.org/archive/html/qemu-devel/2021-06/msg06689.html

We can allocate all the queues, and choose at runtime which queue to use 
for events, that is the only queue used by QEMU.


We can check that in vhost_vsock_common_start(), just before starting 
the device, where we know the features acked by the guest 
(vdev->guest_features).


And I agree that would be the best approach, since we don't need 
discovery anymore, and the same code works also for vhost-user-vsock as 
is.





>
>  vhost_dev_set_config_notifier(&vvc->vhost_dev, &vsock_ops);
>
> diff --git a/hw/virtio/vhost-vsock-common.c 
> b/hw/virtio/vhost-vsock-common.c

> index 4ad6e234ad..d94636e04e 100644
> --- a/hw/virtio/vhost-vsock-common.c
> +++ b/hw/virtio/vhost-vsock-common.c
> @@ -17,6 +17,8 @@
>  #include "hw/virtio/vhost-vsock.h"
>  #include "qemu/iov.h"
>  #include "monitor/monitor.h"
> +#include 
> +#include 
>
>  int vhost_vsock_common_start(VirtIODevice *vdev)
>  {
> @@ -196,9 +198,11 @@ int vhost_vsock_common_post_load(void *opaque, 
> int version_id)

>  return 0;
>  }
>
> -void vhost_vsock_common_realize(VirtIODevice *vdev, const char *name)
> +void vhost_vsock_common_realize(VirtIODevice *vdev, const char *name,
> +   bool enable_dgram)
>  {
>  VHostVSockCommon *vvc = VHOST_VSOCK_COMMON(vdev);
> +int nvqs = VHOST_VSOCK_NVQS;
>
>  virtio_init(vdev, name, VIRTIO_ID_VSOCK,
>  sizeof(struct virtio_vsock_config));
> @@ -209,12 +213,20 @@ void vhost_vsock_common_realize(VirtIODevice *vdev, 
const char *name)
>  vvc->trans_vq = virtio_add_queue(vdev, VHOST_VSOCK_QUEUE_SIZE,
> vhost_vsock_common_handle_output);
>
> +if (enable_dgram) {
> +nvqs = VHOST_VSOCK_NVQS_DGRAM;
> +vvc->dgram_recv_vq = virtio_add_queue(vdev, 
> VHOST_VSOCK_QUEUE_SIZE,

> +  
vhost_vsock_common_handle_output);
> +vvc->dgram_trans_vq = virtio_add_queue(vdev, VHOST_VSOCK_QUEUE_SIZE,
> +  
vhost_vsock_common_handle_output);
> +}

I think the virtqueues can be added unconditionally.


OK.

> +
>  /* The event queue belongs to QEMU */
>  vvc->event_vq = virtio_add_queue(vdev, VHOST_VSOCK_QUEUE_SIZE,
> vhost_vsock_common_handle_output);
>
> -vvc->vhost_dev.nvqs = ARRAY_SIZE(vvc->vhost_vqs);
> -vvc->vhost_dev.vqs = vvc->vhost_vqs;
> +vvc->vhost_dev.nvqs = nvqs;
> +vvc->vhost_dev.vqs = g_new0(struct vhost_vir

Re: [PATCH v4 10/33] i386: Add get/set/migrate support for SGX_LEPUBKEYHASH MSRs

[Yang Zhong]

On Tue, Sep 14, 2021 at 08:38:59AM +0200, Philippe Mathieu-Daudé wrote:
> On 7/19/21 1:21 PM, Yang Zhong wrote:
> > From: Sean Christopherson 
> > 
> > On real hardware, on systems that supports SGX Launch Control, those
> > MSRs are initialized to digest of Intel's signing key; on systems that
> > don't support SGX Launch Control, those MSRs are not available but
> > hardware always uses digest of Intel's signing key in EINIT.
> > 
> > KVM advertises SGX LC via CPUID if and only if the MSRs are writable.
> > Unconditionally initialize those MSRs to digest of Intel's signing key
> > when CPU is realized and reset to reflect the fact. This avoids
> > potential bug in case kvm_arch_put_registers() is called before
> > kvm_arch_get_registers() is called, in which case guest's virtual
> > SGX_LEPUBKEYHASH MSRs will be set to 0, although KVM initializes those
> > to digest of Intel's signing key by default, since KVM allows those MSRs
> > to be updated by Qemu to support live migration.
> > 
> > Save/restore the SGX Launch Enclave Public Key Hash MSRs if SGX Launch
> > Control (LC) is exposed to the guest. Likewise, migrate the MSRs if they
> > are writable by the guest.
> > 
> > Signed-off-by: Sean Christopherson 
> > Signed-off-by: Kai Huang 
> > Signed-off-by: Yang Zhong 
> > ---
> >  target/i386/cpu.c | 17 -
> >  target/i386/cpu.h |  1 +
> >  target/i386/kvm/kvm.c | 22 ++
> >  target/i386/machine.c | 20 
> >  4 files changed, 59 insertions(+), 1 deletion(-)
> > 
> > diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> > index 840f825431..cea4307930 100644
> > --- a/target/i386/cpu.c
> > +++ b/target/i386/cpu.c
> > @@ -5673,6 +5673,16 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, 
> > uint32_t count,
> >  }
> >  }
> >  
> > +#ifndef CONFIG_USER_ONLY
> > +static void x86_cpu_set_sgxlepubkeyhash(CPUX86State *env)
> > +{
> > +env->msr_ia32_sgxlepubkeyhash[0] = 0xa6053e051270b7acULL;
> > +env->msr_ia32_sgxlepubkeyhash[1] = 0x6cfbe8ba8b3b413dULL;
> > +env->msr_ia32_sgxlepubkeyhash[2] = 0xc4916d99f2b3735dULL;
> > +env->msr_ia32_sgxlepubkeyhash[3] = 0xd4f8c05909f9bb3bULL;
> > +}
> > +#endif
> 
> Maybe easier to move the #ifdef'ry inside the function.
>

  Thanks for comments, since this function is pure void function, we can move 
this #ifdef
  into function.
 
> Where these values come from btw?

  Those MSR values are intel default values, which were defined in Skylake 
platform.

  Yang

> 
> > @@ -6186,6 +6198,10 @@ static void x86_cpu_realizefn(DeviceState *dev, 
> > Error **errp)
> > & CPUID_EXT2_AMD_ALIASES);
> >  }
> >  
> > +#ifndef CONFIG_USER_ONLY
> > +x86_cpu_set_sgxlepubkeyhash(env);
> > +#endif
> > +

[PULL 5/6] ui/gtk-egl: Wait for the draw signal for dmabuf blobs

[Gerd Hoffmann]

From: Vivek Kasireddy 

Instead of immediately drawing and submitting, queue and wait
for the draw signal if the dmabuf submitted is a blob.

Cc: Gerd Hoffmann 
Reviewed-by: Gerd Hoffmann 
Signed-off-by: Vivek Kasireddy 
Message-Id: <20210914211837.3229977-5-vivek.kasire...@intel.com>
Signed-off-by: Gerd Hoffmann 
---
 include/ui/gtk.h |  2 ++
 ui/gtk-egl.c | 15 +++
 ui/gtk.c |  2 +-
 3 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/include/ui/gtk.h b/include/ui/gtk.h
index 43854f350907..7d22affd381a 100644
--- a/include/ui/gtk.h
+++ b/include/ui/gtk.h
@@ -182,6 +182,8 @@ void gd_egl_cursor_dmabuf(DisplayChangeListener *dcl,
   uint32_t hot_x, uint32_t hot_y);
 void gd_egl_cursor_position(DisplayChangeListener *dcl,
 uint32_t pos_x, uint32_t pos_y);
+void gd_egl_flush(DisplayChangeListener *dcl,
+  uint32_t x, uint32_t y, uint32_t w, uint32_t h);
 void gd_egl_scanout_flush(DisplayChangeListener *dcl,
   uint32_t x, uint32_t y, uint32_t w, uint32_t h);
 void gtk_egl_init(DisplayGLMode mode);
diff --git a/ui/gtk-egl.c b/ui/gtk-egl.c
index 3a90aeb2b9e1..72ce5e1f8f41 100644
--- a/ui/gtk-egl.c
+++ b/ui/gtk-egl.c
@@ -309,6 +309,21 @@ void gd_egl_scanout_flush(DisplayChangeListener *dcl,
 eglSwapBuffers(qemu_egl_display, vc->gfx.esurface);
 }
 
+void gd_egl_flush(DisplayChangeListener *dcl,
+  uint32_t x, uint32_t y, uint32_t w, uint32_t h)
+{
+VirtualConsole *vc = container_of(dcl, VirtualConsole, gfx.dcl);
+GtkWidget *area = vc->gfx.drawing_area;
+
+if (vc->gfx.guest_fb.dmabuf) {
+graphic_hw_gl_block(vc->gfx.dcl.con, true);
+gtk_widget_queue_draw_area(area, x, y, w, h);
+return;
+}
+
+gd_egl_scanout_flush(&vc->gfx.dcl, x, y, w, h);
+}
+
 void gtk_egl_init(DisplayGLMode mode)
 {
 GdkDisplay *gdk_display = gdk_display_get_default();
diff --git a/ui/gtk.c b/ui/gtk.c
index 5105c0a33ff1..b0564d80c191 100644
--- a/ui/gtk.c
+++ b/ui/gtk.c
@@ -637,7 +637,7 @@ static const DisplayChangeListenerOps dcl_egl_ops = {
 .dpy_gl_scanout_dmabuf   = gd_egl_scanout_dmabuf,
 .dpy_gl_cursor_dmabuf= gd_egl_cursor_dmabuf,
 .dpy_gl_cursor_position  = gd_egl_cursor_position,
-.dpy_gl_update   = gd_egl_scanout_flush,
+.dpy_gl_update   = gd_egl_flush,
 .dpy_gl_release_dmabuf   = gd_gl_release_dmabuf,
 .dpy_has_dmabuf  = gd_has_dmabuf,
 };
-- 
2.31.1

[PULL 4/6] ui: Create sync objects and fences only for blobs

[Gerd Hoffmann]

From: Vivek Kasireddy 

Create sync objects and fences only for dmabufs that are blobs. Once a
fence is created (after glFlush) and is signalled,
graphic_hw_gl_flushed() will be called and virtio-gpu cmd processing
will be resumed.

Cc: Gerd Hoffmann 
Signed-off-by: Vivek Kasireddy 
Message-Id: <20210914211837.3229977-4-vivek.kasire...@intel.com>
Signed-off-by: Gerd Hoffmann 
---
 include/ui/console.h|  1 +
 include/ui/egl-helpers.h|  1 +
 include/ui/gtk.h|  1 +
 hw/display/virtio-gpu-udmabuf.c |  1 +
 ui/gtk-egl.c| 25 +
 ui/gtk-gl-area.c| 26 ++
 ui/gtk.c| 13 +
 7 files changed, 68 insertions(+)

diff --git a/include/ui/console.h b/include/ui/console.h
index 45ec1291743b..244664d727a4 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -170,6 +170,7 @@ typedef struct QemuDmaBuf {
 bool  y0_top;
 void  *sync;
 int   fence_fd;
+bool  allow_fences;
 } QemuDmaBuf;
 
 typedef struct DisplayState DisplayState;
diff --git a/include/ui/egl-helpers.h b/include/ui/egl-helpers.h
index 2c3ba92b53e1..2fb6e0dd6b87 100644
--- a/include/ui/egl-helpers.h
+++ b/include/ui/egl-helpers.h
@@ -19,6 +19,7 @@ typedef struct egl_fb {
 GLuint texture;
 GLuint framebuffer;
 bool delete_texture;
+QemuDmaBuf *dmabuf;
 } egl_fb;
 
 void egl_fb_destroy(egl_fb *fb);
diff --git a/include/ui/gtk.h b/include/ui/gtk.h
index 8e98a79ac813..43854f350907 100644
--- a/include/ui/gtk.h
+++ b/include/ui/gtk.h
@@ -155,6 +155,7 @@ extern bool gtk_use_gl_area;
 /* ui/gtk.c */
 void gd_update_windowsize(VirtualConsole *vc);
 int gd_monitor_update_interval(GtkWidget *widget);
+void gd_hw_gl_flushed(void *vc);
 
 /* ui/gtk-egl.c */
 void gd_egl_init(VirtualConsole *vc);
diff --git a/hw/display/virtio-gpu-udmabuf.c b/hw/display/virtio-gpu-udmabuf.c
index 3c01a415e71b..c6f7f587847f 100644
--- a/hw/display/virtio-gpu-udmabuf.c
+++ b/hw/display/virtio-gpu-udmabuf.c
@@ -185,6 +185,7 @@ static VGPUDMABuf
 dmabuf->buf.stride = fb->stride;
 dmabuf->buf.fourcc = qemu_pixman_to_drm_format(fb->format);
 dmabuf->buf.fd = res->dmabuf_fd;
+dmabuf->buf.allow_fences = true;
 
 dmabuf->scanout_id = scanout_id;
 QTAILQ_INSERT_HEAD(&g->dmabuf.bufs, dmabuf, next);
diff --git a/ui/gtk-egl.c b/ui/gtk-egl.c
index b671181272d5..3a90aeb2b9e1 100644
--- a/ui/gtk-egl.c
+++ b/ui/gtk-egl.c
@@ -12,6 +12,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/main-loop.h"
 
 #include "trace.h"
 
@@ -94,6 +95,18 @@ void gd_egl_draw(VirtualConsole *vc)
 }
 
 glFlush();
+#ifdef CONFIG_GBM
+if (vc->gfx.guest_fb.dmabuf) {
+QemuDmaBuf *dmabuf = vc->gfx.guest_fb.dmabuf;
+
+egl_dmabuf_create_fence(dmabuf);
+if (dmabuf->fence_fd > 0) {
+qemu_set_fd_handler(dmabuf->fence_fd, gd_hw_gl_flushed, NULL, vc);
+return;
+}
+graphic_hw_gl_block(vc->gfx.dcl.con, false);
+}
+#endif
 graphic_hw_gl_flushed(vc->gfx.dcl.con);
 }
 
@@ -209,6 +222,8 @@ void gd_egl_scanout_dmabuf(DisplayChangeListener *dcl,
QemuDmaBuf *dmabuf)
 {
 #ifdef CONFIG_GBM
+VirtualConsole *vc = container_of(dcl, VirtualConsole, gfx.dcl);
+
 egl_dmabuf_import_texture(dmabuf);
 if (!dmabuf->texture) {
 return;
@@ -217,6 +232,10 @@ void gd_egl_scanout_dmabuf(DisplayChangeListener *dcl,
 gd_egl_scanout_texture(dcl, dmabuf->texture,
false, dmabuf->width, dmabuf->height,
0, 0, dmabuf->width, dmabuf->height);
+
+if (dmabuf->allow_fences) {
+vc->gfx.guest_fb.dmabuf = dmabuf;
+}
 #endif
 }
 
@@ -281,6 +300,12 @@ void gd_egl_scanout_flush(DisplayChangeListener *dcl,
 egl_fb_blit(&vc->gfx.win_fb, &vc->gfx.guest_fb, !vc->gfx.y0_top);
 }
 
+#ifdef CONFIG_GBM
+if (vc->gfx.guest_fb.dmabuf) {
+egl_dmabuf_create_sync(vc->gfx.guest_fb.dmabuf);
+}
+#endif
+
 eglSwapBuffers(qemu_egl_display, vc->gfx.esurface);
 }
 
diff --git a/ui/gtk-gl-area.c b/ui/gtk-gl-area.c
index dd5783fec784..b23523748e7f 100644
--- a/ui/gtk-gl-area.c
+++ b/ui/gtk-gl-area.c
@@ -8,6 +8,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/main-loop.h"
 
 #include "trace.h"
 
@@ -71,7 +72,25 @@ void gd_gl_area_draw(VirtualConsole *vc)
 surface_gl_render_texture(vc->gfx.gls, vc->gfx.ds);
 }
 
+#ifdef CONFIG_GBM
+if (vc->gfx.guest_fb.dmabuf) {
+egl_dmabuf_create_sync(vc->gfx.guest_fb.dmabuf);
+}
+#endif
+
 glFlush();
+#ifdef CONFIG_GBM
+if (vc->gfx.guest_fb.dmabuf) {
+QemuDmaBuf *dmabuf = vc->gfx.guest_fb.dmabuf;
+
+egl_dmabuf_create_fence(dmabuf);
+if (dmabuf->fence_fd > 0) {
+qemu_set_fd_handler(dmabuf->fence_fd, gd_hw_gl_flushed, NULL, vc);
+return;
+}
+graphic_hw_gl_block(vc->gfx.dcl.con, false);
+}
+#endif
 g

[PULL 6/6] virtio-gpu: Add gl_flushed callback

[Gerd Hoffmann]

From: Vivek Kasireddy 

Adding this callback provides a way to resume the processing of
cmds in fenceq and cmdq that were not processed because the UI
was waiting on a fence and blocked cmd processing.

Cc: Gerd Hoffmann 
Reviewed-by: Gerd Hoffmann 
Signed-off-by: Vivek Kasireddy 
Message-Id: <20210914211837.3229977-6-vivek.kasire...@intel.com>
Signed-off-by: Gerd Hoffmann 
---
 hw/display/virtio-gpu.c | 32 ++--
 1 file changed, 30 insertions(+), 2 deletions(-)

diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
index 72da5bf5002c..182e0868b09d 100644
--- a/hw/display/virtio-gpu.c
+++ b/hw/display/virtio-gpu.c
@@ -985,8 +985,10 @@ void virtio_gpu_simple_process_cmd(VirtIOGPU *g,
 break;
 }
 if (!cmd->finished) {
-virtio_gpu_ctrl_response_nodata(g, cmd, cmd->error ? cmd->error :
-VIRTIO_GPU_RESP_OK_NODATA);
+if (!g->parent_obj.renderer_blocked) {
+virtio_gpu_ctrl_response_nodata(g, cmd, cmd->error ? cmd->error :
+VIRTIO_GPU_RESP_OK_NODATA);
+}
 }
 }
 
@@ -1042,6 +1044,30 @@ void virtio_gpu_process_cmdq(VirtIOGPU *g)
 g->processing_cmdq = false;
 }
 
+static void virtio_gpu_process_fenceq(VirtIOGPU *g)
+{
+struct virtio_gpu_ctrl_command *cmd, *tmp;
+
+QTAILQ_FOREACH_SAFE(cmd, &g->fenceq, next, tmp) {
+trace_virtio_gpu_fence_resp(cmd->cmd_hdr.fence_id);
+virtio_gpu_ctrl_response_nodata(g, cmd, VIRTIO_GPU_RESP_OK_NODATA);
+QTAILQ_REMOVE(&g->fenceq, cmd, next);
+g_free(cmd);
+g->inflight--;
+if (virtio_gpu_stats_enabled(g->parent_obj.conf)) {
+fprintf(stderr, "inflight: %3d (-)\r", g->inflight);
+}
+}
+}
+
+static void virtio_gpu_handle_gl_flushed(VirtIOGPUBase *b)
+{
+VirtIOGPU *g = container_of(b, VirtIOGPU, parent_obj);
+
+virtio_gpu_process_fenceq(g);
+virtio_gpu_process_cmdq(g);
+}
+
 static void virtio_gpu_handle_ctrl(VirtIODevice *vdev, VirtQueue *vq)
 {
 VirtIOGPU *g = VIRTIO_GPU(vdev);
@@ -1400,10 +1426,12 @@ static void virtio_gpu_class_init(ObjectClass *klass, 
void *data)
 DeviceClass *dc = DEVICE_CLASS(klass);
 VirtioDeviceClass *vdc = VIRTIO_DEVICE_CLASS(klass);
 VirtIOGPUClass *vgc = VIRTIO_GPU_CLASS(klass);
+VirtIOGPUBaseClass *vgbc = &vgc->parent;
 
 vgc->handle_ctrl = virtio_gpu_handle_ctrl;
 vgc->process_cmd = virtio_gpu_simple_process_cmd;
 vgc->update_cursor_data = virtio_gpu_update_cursor_data;
+vgbc->gl_flushed = virtio_gpu_handle_gl_flushed;
 
 vdc->realize = virtio_gpu_device_realize;
 vdc->reset = virtio_gpu_reset;
-- 
2.31.1

[PULL 3/6] ui/egl: Add egl helpers to help with synchronization

[Gerd Hoffmann]

From: Vivek Kasireddy 

These egl helpers would be used for creating and waiting on
a sync object.

Cc: Gerd Hoffmann 
Reviewed-by: Gerd Hoffmann 
Signed-off-by: Vivek Kasireddy 
Message-Id: <20210914211837.3229977-3-vivek.kasire...@intel.com>
Signed-off-by: Gerd Hoffmann 
---
 include/ui/console.h |  2 ++
 include/ui/egl-helpers.h |  2 ++
 ui/egl-helpers.c | 26 ++
 3 files changed, 30 insertions(+)

diff --git a/include/ui/console.h b/include/ui/console.h
index 3be21497a2e8..45ec1291743b 100644
--- a/include/ui/console.h
+++ b/include/ui/console.h
@@ -168,6 +168,8 @@ typedef struct QemuDmaBuf {
 uint64_t  modifier;
 uint32_t  texture;
 bool  y0_top;
+void  *sync;
+int   fence_fd;
 } QemuDmaBuf;
 
 typedef struct DisplayState DisplayState;
diff --git a/include/ui/egl-helpers.h b/include/ui/egl-helpers.h
index f1bf8f97fc33..2c3ba92b53e1 100644
--- a/include/ui/egl-helpers.h
+++ b/include/ui/egl-helpers.h
@@ -45,6 +45,8 @@ int egl_get_fd_for_texture(uint32_t tex_id, EGLint *stride, 
EGLint *fourcc,
 
 void egl_dmabuf_import_texture(QemuDmaBuf *dmabuf);
 void egl_dmabuf_release_texture(QemuDmaBuf *dmabuf);
+void egl_dmabuf_create_sync(QemuDmaBuf *dmabuf);
+void egl_dmabuf_create_fence(QemuDmaBuf *dmabuf);
 
 #endif
 
diff --git a/ui/egl-helpers.c b/ui/egl-helpers.c
index 6d0cb2b5cb93..385a3fa75223 100644
--- a/ui/egl-helpers.c
+++ b/ui/egl-helpers.c
@@ -287,6 +287,32 @@ void egl_dmabuf_release_texture(QemuDmaBuf *dmabuf)
 dmabuf->texture = 0;
 }
 
+void egl_dmabuf_create_sync(QemuDmaBuf *dmabuf)
+{
+EGLSyncKHR sync;
+
+if (epoxy_has_egl_extension(qemu_egl_display,
+"EGL_KHR_fence_sync") &&
+epoxy_has_egl_extension(qemu_egl_display,
+"EGL_ANDROID_native_fence_sync")) {
+sync = eglCreateSyncKHR(qemu_egl_display,
+EGL_SYNC_NATIVE_FENCE_ANDROID, NULL);
+if (sync != EGL_NO_SYNC_KHR) {
+dmabuf->sync = sync;
+}
+}
+}
+
+void egl_dmabuf_create_fence(QemuDmaBuf *dmabuf)
+{
+if (dmabuf->sync) {
+dmabuf->fence_fd = eglDupNativeFenceFDANDROID(qemu_egl_display,
+  dmabuf->sync);
+eglDestroySyncKHR(qemu_egl_display, dmabuf->sync);
+dmabuf->sync = NULL;
+}
+}
+
 #endif /* CONFIG_GBM */
 
 /* -- */
-- 
2.31.1

[PULL 2/6] ui/gtk: Create a common release_dmabuf helper

[Gerd Hoffmann]

From: Vivek Kasireddy 

Since the texture release mechanism is same for both gtk-egl
and gtk-glarea, move the helper from gtk-egl to common gtk
code so that it can be shared by both gtk backends.

Cc: Gerd Hoffmann 
Reviewed-by: Gerd Hoffmann 
Signed-off-by: Vivek Kasireddy 
Message-Id: <20210914211837.3229977-2-vivek.kasire...@intel.com>
Signed-off-by: Gerd Hoffmann 
---
 include/ui/gtk.h |  2 --
 ui/gtk-egl.c |  8 
 ui/gtk.c | 11 ++-
 3 files changed, 10 insertions(+), 11 deletions(-)

diff --git a/include/ui/gtk.h b/include/ui/gtk.h
index 7835ef1a7108..8e98a79ac813 100644
--- a/include/ui/gtk.h
+++ b/include/ui/gtk.h
@@ -181,8 +181,6 @@ void gd_egl_cursor_dmabuf(DisplayChangeListener *dcl,
   uint32_t hot_x, uint32_t hot_y);
 void gd_egl_cursor_position(DisplayChangeListener *dcl,
 uint32_t pos_x, uint32_t pos_y);
-void gd_egl_release_dmabuf(DisplayChangeListener *dcl,
-   QemuDmaBuf *dmabuf);
 void gd_egl_scanout_flush(DisplayChangeListener *dcl,
   uint32_t x, uint32_t y, uint32_t w, uint32_t h);
 void gtk_egl_init(DisplayGLMode mode);
diff --git a/ui/gtk-egl.c b/ui/gtk-egl.c
index 2a2e6d3a17d4..b671181272d5 100644
--- a/ui/gtk-egl.c
+++ b/ui/gtk-egl.c
@@ -249,14 +249,6 @@ void gd_egl_cursor_position(DisplayChangeListener *dcl,
 vc->gfx.cursor_y = pos_y * vc->gfx.scale_y;
 }
 
-void gd_egl_release_dmabuf(DisplayChangeListener *dcl,
-   QemuDmaBuf *dmabuf)
-{
-#ifdef CONFIG_GBM
-egl_dmabuf_release_texture(dmabuf);
-#endif
-}
-
 void gd_egl_scanout_flush(DisplayChangeListener *dcl,
   uint32_t x, uint32_t y, uint32_t w, uint32_t h)
 {
diff --git a/ui/gtk.c b/ui/gtk.c
index cfb0728d1fb4..784a2f6c749c 100644
--- a/ui/gtk.c
+++ b/ui/gtk.c
@@ -575,6 +575,14 @@ static bool gd_has_dmabuf(DisplayChangeListener *dcl)
 return vc->gfx.has_dmabuf;
 }
 
+static void gd_gl_release_dmabuf(DisplayChangeListener *dcl,
+ QemuDmaBuf *dmabuf)
+{
+#ifdef CONFIG_GBM
+egl_dmabuf_release_texture(dmabuf);
+#endif
+}
+
 /** DisplayState Callbacks (opengl version) **/
 
 static const DisplayChangeListenerOps dcl_gl_area_ops = {
@@ -593,6 +601,7 @@ static const DisplayChangeListenerOps dcl_gl_area_ops = {
 .dpy_gl_scanout_disable  = gd_gl_area_scanout_disable,
 .dpy_gl_update   = gd_gl_area_scanout_flush,
 .dpy_gl_scanout_dmabuf   = gd_gl_area_scanout_dmabuf,
+.dpy_gl_release_dmabuf   = gd_gl_release_dmabuf,
 .dpy_has_dmabuf  = gd_has_dmabuf,
 };
 
@@ -615,8 +624,8 @@ static const DisplayChangeListenerOps dcl_egl_ops = {
 .dpy_gl_scanout_dmabuf   = gd_egl_scanout_dmabuf,
 .dpy_gl_cursor_dmabuf= gd_egl_cursor_dmabuf,
 .dpy_gl_cursor_position  = gd_egl_cursor_position,
-.dpy_gl_release_dmabuf   = gd_egl_release_dmabuf,
 .dpy_gl_update   = gd_egl_scanout_flush,
+.dpy_gl_release_dmabuf   = gd_gl_release_dmabuf,
 .dpy_has_dmabuf  = gd_has_dmabuf,
 };
 
-- 
2.31.1

[PULL 1/6] qxl: fix pre-save logic

[Gerd Hoffmann]

Oops.  Logic is backwards.

Fixes: 39b8a183e2f3 ("qxl: remove assert in qxl_pre_save.")
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/610
Resolves: https://bugzilla.redhat.com//show_bug.cgi?id=2002907
Signed-off-by: Gerd Hoffmann 
Reviewed-by: Daniel P. Berrangé 
Reviewed-by: Marc-André Lureau 
Message-Id: <20210910094203.3582378-1-kra...@redhat.com>
---
 hw/display/qxl.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/display/qxl.c b/hw/display/qxl.c
index 43482d4364ba..29c80b4289b7 100644
--- a/hw/display/qxl.c
+++ b/hw/display/qxl.c
@@ -2252,7 +2252,7 @@ static int qxl_pre_save(void *opaque)
 } else {
 d->last_release_offset = (uint8_t *)d->last_release - ram_start;
 }
-if (d->last_release_offset < d->vga.vram_size) {
+if (d->last_release_offset >= d->vga.vram_size) {
 return 1;
 }
 
-- 
2.31.1

[PULL 0/6] Vga 20210916 patches

[Gerd Hoffmann]

The following changes since commit 831aaf24967a49d7750090b9dcfd6bf356f16529:

  Merge remote-tracking branch 'remotes/marcandre/tags/misc-pull-request' into 
staging (2021-09-14 18:14:56 +0100)

are available in the Git repository at:

  git://git.kraxel.org/qemu tags/vga-20210916-pull-request

for you to fetch changes up to b3a5dfdea99da55fdc70538eeeb2227ebe6d6a5f:

  virtio-gpu: Add gl_flushed callback (2021-09-15 08:42:00 +0200)


virtio-gpu + ui: fence syncronization.
qxl: unbreak live migration.



Gerd Hoffmann (1):
  qxl: fix pre-save logic

Vivek Kasireddy (5):
  ui/gtk: Create a common release_dmabuf helper
  ui/egl: Add egl helpers to help with synchronization
  ui: Create sync objects and fences only for blobs
  ui/gtk-egl: Wait for the draw signal for dmabuf blobs
  virtio-gpu: Add gl_flushed callback

 include/ui/console.h|  3 +++
 include/ui/egl-helpers.h|  3 +++
 include/ui/gtk.h|  5 ++--
 hw/display/qxl.c|  2 +-
 hw/display/virtio-gpu-udmabuf.c |  1 +
 hw/display/virtio-gpu.c | 32 --
 ui/egl-helpers.c| 26 ++
 ui/gtk-egl.c| 48 +++--
 ui/gtk-gl-area.c| 26 ++
 ui/gtk.c| 26 --
 10 files changed, 157 insertions(+), 15 deletions(-)

-- 
2.31.1

[PATCH] MAINTAINERS: add myself as a reviewer for KVM guest cpu related changes

[Ani Sinha]

I have looked into cpu features for KVM guests as a part of a different
project. Would be interested to follow and review patches in this space.

Signed-off-by: Ani Sinha 
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 6c20634d63..3a3167c499 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -364,6 +364,7 @@ Guest CPU Cores (KVM)
 -
 Overall KVM CPUs
 M: Paolo Bonzini 
+R: Ani Sinha 
 L: k...@vger.kernel.org
 S: Supported
 F: */*/kvm*
-- 
2.25.1

Re: [PATCH v5 02/16] tcg/s390x: Rename from tcg/s390

[Philippe Mathieu-Daudé]

On 9/15/21 11:31 PM, Richard Henderson wrote:
> This emphasizes that we don't support s390, only 64-bit s390x hosts.
> 
> Reviewed-by: David Hildenbrand 
> Signed-off-by: Richard Henderson 
> ---
>  meson.build  | 2 --
>  tcg/{s390 => s390x}/tcg-target-con-set.h | 0
>  tcg/{s390 => s390x}/tcg-target-con-str.h | 0
>  tcg/{s390 => s390x}/tcg-target.h | 0
>  tcg/{s390 => s390x}/tcg-target.c.inc | 0
>  5 files changed, 2 deletions(-)
>  rename tcg/{s390 => s390x}/tcg-target-con-set.h (100%)
>  rename tcg/{s390 => s390x}/tcg-target-con-str.h (100%)
>  rename tcg/{s390 => s390x}/tcg-target.h (100%)
>  rename tcg/{s390 => s390x}/tcg-target.c.inc (100%)

Reviewed-by: Philippe Mathieu-Daudé 

Re: [PATCH v5 02/16] tcg/s390x: Rename from tcg/s390

[Thomas Huth]

On 15/09/2021 23.31, Richard Henderson wrote:

This emphasizes that we don't support s390, only 64-bit s390x hosts.

Reviewed-by: David Hildenbrand 
Signed-off-by: Richard Henderson 
---
  meson.build  | 2 --
  tcg/{s390 => s390x}/tcg-target-con-set.h | 0
  tcg/{s390 => s390x}/tcg-target-con-str.h | 0
  tcg/{s390 => s390x}/tcg-target.h | 0
  tcg/{s390 => s390x}/tcg-target.c.inc | 0
  5 files changed, 2 deletions(-)
  rename tcg/{s390 => s390x}/tcg-target-con-set.h (100%)
  rename tcg/{s390 => s390x}/tcg-target-con-str.h (100%)
  rename tcg/{s390 => s390x}/tcg-target.h (100%)
  rename tcg/{s390 => s390x}/tcg-target.c.inc (100%)

diff --git a/meson.build b/meson.build
index 2711cbb789..1cf370ab56 100644
--- a/meson.build
+++ b/meson.build
@@ -265,8 +265,6 @@ if not get_option('tcg').disabled()
  tcg_arch = 'tci'
elif config_host['ARCH'] == 'sparc64'
  tcg_arch = 'sparc'
-  elif config_host['ARCH'] == 's390x'
-tcg_arch = 's390'
elif config_host['ARCH'] in ['x86_64', 'x32']
  tcg_arch = 'i386'
elif config_host['ARCH'] == 'ppc64'
diff --git a/tcg/s390/tcg-target-con-set.h b/tcg/s390x/tcg-target-con-set.h
similarity index 100%
rename from tcg/s390/tcg-target-con-set.h
rename to tcg/s390x/tcg-target-con-set.h
diff --git a/tcg/s390/tcg-target-con-str.h b/tcg/s390x/tcg-target-con-str.h
similarity index 100%
rename from tcg/s390/tcg-target-con-str.h
rename to tcg/s390x/tcg-target-con-str.h
diff --git a/tcg/s390/tcg-target.h b/tcg/s390x/tcg-target.h
similarity index 100%
rename from tcg/s390/tcg-target.h
rename to tcg/s390x/tcg-target.h
diff --git a/tcg/s390/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
similarity index 100%
rename from tcg/s390/tcg-target.c.inc
rename to tcg/s390x/tcg-target.c.inc



Good idea!

Reviewed-by: Thomas Huth 

Re: [PATCH v1 1/1] hw/riscv: opentitan: Correct the USB Dev address

[Bin Meng]

On Thu, Sep 16, 2021 at 12:37 PM Alistair Francis
 wrote:
>
> From: Alistair Francis 
>
> Signed-off-by: Alistair Francis 
> ---
>  hw/riscv/opentitan.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>

Reviewed-by: Bin Meng 

Re: [qemu-web PATCH] Gemfile: Add webrick bundle dependency

[Philippe Mathieu-Daudé]

On 9/16/21 6:55 AM, Thomas Huth wrote:
> On 16/09/2021 06.24, Philippe Mathieu-Daudé wrote:
>> On 9/16/21 12:27 AM, Paolo Bonzini wrote:
>>> On 14/09/21 10:21, Daniel P. Berrangé wrote:
 Yes, this is needed to fix Ruby 3.

 I didn't propose it myself as I'm not sure if it in turn breaks people
 with Ruby 2.x ?
>>>
>>> Does it pass the GitLab CI (which uses 2.5)?
>>
>> Yes:
>> https://gitlab.com/philmd/qemu-web/-/jobs/1594757246#L466
>> Installing webrick 1.7.0
>> Bundle complete! 5 Gemfile dependencies, 32 gems now installed.
> 
> The patch is also not breaking anything for me here (with Ruby 2.5.9).
> Pushed now.

That was fast! Thanks :)

Re: [qemu-web PATCH] Gemfile: Add webrick bundle dependency

[Thomas Huth]

On 16/09/2021 06.24, Philippe Mathieu-Daudé wrote:

On 9/16/21 12:27 AM, Paolo Bonzini wrote:

On 14/09/21 10:21, Daniel P. Berrangé wrote:

Yes, this is needed to fix Ruby 3.

I didn't propose it myself as I'm not sure if it in turn breaks people
with Ruby 2.x ?


Does it pass the GitLab CI (which uses 2.5)?


Yes:
https://gitlab.com/philmd/qemu-web/-/jobs/1594757246#L466
Installing webrick 1.7.0
Bundle complete! 5 Gemfile dependencies, 32 gems now installed.


The patch is also not breaking anything for me here (with Ruby 2.5.9). 
Pushed now.


 Thanks,
  Thomas

Re: [PATCH v3 13/16] iotests/linters: Add entry point for Python CI linters

[Philippe Mathieu-Daudé]

On 9/16/21 6:09 AM, John Snow wrote:
> Add a main() function to linters.py so that the Python CI infrastructure
> has something it can run.
> 
> Now, linters.py represents an invocation of the linting scripts that
> more resembles a "normal" execution of pylint/mypy, like you'd expect to
> use if 'qemu' was a bona-fide package you obtained from PyPI.
> 
> 297, by contrast, now represents the iotests-specific configuration bits
> you need to get it to function correctly as a part of iotests, and with
> 'qemu' as a namespace package that isn't "installed" to the current
> environment, but just lives elsewhere in our source tree.
> 
> By doing this, we will able to run the same linting configuration from
> the Python CI tests without calling iotest logging functions or messing
> around with PYTHONPATH / MYPYPATH.
> 
> iotest 297 continues to operate in a standalone fashion for now --
> presumably, it's convenient for block maintainers and contributors to
> run in this manner. We can either remove this functionality at a later
> date if everyone is happy with the Python CI, or we can opt to continue
> to maintain it. Please let me know how you feel.
> 
> See the following commit for how this is used from the Python packaging side.
> 
> Signed-off-by: John Snow 
> Reviewed-by: Vladimir Sementsov-Ogievskiy 
> ---
>  tests/qemu-iotests/linters.py | 13 +
>  1 file changed, 13 insertions(+)

Reviewed-by: Philippe Mathieu-Daudé 

Re: [PATCH v11 05/10] arm/hvf: Add a WFI handler

[Philippe Mathieu-Daudé]

On 9/15/21 8:10 PM, Alexander Graf wrote:
> From: Peter Collingbourne 
> 
> Sleep on WFI until the VTIMER is due but allow ourselves to be woken
> up on IPI.
> 
> In this implementation IPI is blocked on the CPU thread at startup and
> pselect() is used to atomically unblock the signal and begin sleeping.
> The signal is sent unconditionally so there's no need to worry about
> races between actually sleeping and the "we think we're sleeping"
> state. It may lead to an extra wakeup but that's better than missing
> it entirely.
> 
> Signed-off-by: Peter Collingbourne 
> [agraf: Remove unused 'set' variable, always advance PC on WFX trap,
> support vm stop / continue operations and cntv offsets]
> Signed-off-by: Alexander Graf 
> Acked-by: Roman Bolshakov 
> Reviewed-by: Sergio Lopez 
> 
> ---

> diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
> index 8fe008dab5..49f265cc08 100644
> --- a/target/arm/hvf/hvf.c
> +++ b/target/arm/hvf/hvf.c
> @@ -2,6 +2,7 @@
>   * QEMU Hypervisor.framework support for Apple Silicon
>  
>   * Copyright 2020 Alexander Graf 
> + * Copyright 2020 Google LLC
>   *
>   * This work is licensed under the terms of the GNU GPL, version 2 or later.
>   * See the COPYING file in the top-level directory.
> @@ -490,6 +491,7 @@ int hvf_arch_init_vcpu(CPUState *cpu)
>  
>  void hvf_kick_vcpu_thread(CPUState *cpu)
>  {
> +cpus_kick_thread(cpu);

Doesn't this belong to the previous patch?

>  hv_vcpus_exit(&cpu->hvf->fd, 1);
>  }

> +static void hvf_wfi(CPUState *cpu)
> +{
> +ARMCPU *arm_cpu = ARM_CPU(cpu);
> +hv_return_t r;
> +uint64_t ctl;
> +uint64_t cval;
> +int64_t ticks_to_sleep;
> +uint64_t seconds;
> +uint64_t nanos;
> +
> +if (cpu->interrupt_request & (CPU_INTERRUPT_HARD | CPU_INTERRUPT_FIQ)) {
> +/* Interrupt pending, no need to wait */
> +return;
> +}
> +
> +r = hv_vcpu_get_sys_reg(cpu->hvf->fd, HV_SYS_REG_CNTV_CTL_EL0, &ctl);
> +assert_hvf_ok(r);
> +
> +if (!(ctl & 1) || (ctl & 2)) {
> +/* Timer disabled or masked, just wait for an IPI. */
> +hvf_wait_for_ipi(cpu, NULL);
> +return;
> +}
> +
> +r = hv_vcpu_get_sys_reg(cpu->hvf->fd, HV_SYS_REG_CNTV_CVAL_EL0, &cval);
> +assert_hvf_ok(r);
> +
> +ticks_to_sleep = cval - hvf_vtimer_val();
> +if (ticks_to_sleep < 0) {
> +return;
> +}
> +
> +nanos = ticks_to_sleep * gt_cntfrq_period_ns(arm_cpu);
> +seconds = nanos / NANOSECONDS_PER_SECOND;

muldiv64()?

> +nanos -= (seconds * NANOSECONDS_PER_SECOND);
> +
> +/*
> + * Don't sleep for less than the time a context switch would take,
> + * so that we can satisfy fast timer requests on the same CPU.
> + * Measurements on M1 show the sweet spot to be ~2ms.
> + */
> +if (!seconds && nanos < (2 * SCALE_MS)) {
> +return;
> +}
> +
> +struct timespec ts = { seconds, nanos };

QEMU style still declares variables at top of function/block.

> +hvf_wait_for_ipi(cpu, &ts);
> +}

Re: [ RFC v2 2/9] target/riscv: pmu: Rename the counters extension to pmu

[Alistair Francis]

On Fri, Sep 10, 2021 at 6:28 AM Atish Patra  wrote:
>
> The PMU counters are supported via cpu config "Counters" which doesn't
> indicate the correct purpose of those counters.
>
> Rename the config property to pmu to indicate that these counters
> are performance monitoring counters. This aligns with cpu options for
> ARM architecture as well.
>
> Signed-off-by: Atish Patra 

Reviewed-by: Alistair Francis 

Alistair

> ---
>  target/riscv/cpu.c | 2 +-
>  target/riscv/cpu.h | 2 +-
>  target/riscv/csr.c | 2 +-
>  3 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
> index 991a6bb7604f..7a486450ebc6 100644
> --- a/target/riscv/cpu.c
> +++ b/target/riscv/cpu.c
> @@ -587,7 +587,7 @@ static Property riscv_cpu_properties[] = {
>  DEFINE_PROP_BOOL("x-b", RISCVCPU, cfg.ext_b, false),
>  DEFINE_PROP_BOOL("x-h", RISCVCPU, cfg.ext_h, false),
>  DEFINE_PROP_BOOL("x-v", RISCVCPU, cfg.ext_v, false),
> -DEFINE_PROP_BOOL("Counters", RISCVCPU, cfg.ext_counters, true),
> +DEFINE_PROP_BOOL("pmu", RISCVCPU, cfg.ext_pmu, true),
>  DEFINE_PROP_BOOL("Zifencei", RISCVCPU, cfg.ext_ifencei, true),
>  DEFINE_PROP_BOOL("Zicsr", RISCVCPU, cfg.ext_icsr, true),
>  DEFINE_PROP_STRING("priv_spec", RISCVCPU, cfg.priv_spec),
> diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
> index bf1c899c00b8..5e67003e58a3 100644
> --- a/target/riscv/cpu.h
> +++ b/target/riscv/cpu.h
> @@ -293,7 +293,7 @@ struct RISCVCPU {
>  bool ext_u;
>  bool ext_h;
>  bool ext_v;
> -bool ext_counters;
> +bool ext_pmu;
>  bool ext_ifencei;
>  bool ext_icsr;
>
> diff --git a/target/riscv/csr.c b/target/riscv/csr.c
> index 0515d851b948..c3ce7d83a6b2 100644
> --- a/target/riscv/csr.c
> +++ b/target/riscv/csr.c
> @@ -64,7 +64,7 @@ static RISCVException ctr(CPURISCVState *env, int csrno)
>  RISCVCPU *cpu = RISCV_CPU(cs);
>  int ctr_index;
>
> -if (!cpu->cfg.ext_counters) {
> +if (!cpu->cfg.ext_pmu) {
>  /* The Counters extensions is not enabled */
>  return RISCV_EXCP_ILLEGAL_INST;
>  }
> --
> 2.31.1
>
>

Re: [PATCH] target/riscv: csr: Rename HCOUNTEREN_CY and friends

[Alistair Francis]

On Wed, Sep 15, 2021 at 6:47 PM Bin Meng  wrote:
>
> The macro name HCOUNTEREN_CY suggests it is for CSR HCOUNTEREN, but
> in fact it applies to M-mode and S-mode CSR too. Rename these macros
> to have the COUNTEREN_ prefix.
>
> Signed-off-by: Bin Meng 

Reviewed-by: Alistair Francis 

Alistair

> ---
>
>  target/riscv/cpu_bits.h |  8 
>  target/riscv/csr.c  | 24 
>  2 files changed, 16 insertions(+), 16 deletions(-)
>
> diff --git a/target/riscv/cpu_bits.h b/target/riscv/cpu_bits.h
> index 7330ff5a19..34564d367c 100644
> --- a/target/riscv/cpu_bits.h
> +++ b/target/riscv/cpu_bits.h
> @@ -397,10 +397,10 @@
>  #define HSTATUS32_WPRI   0xFF8FF87E
>  #define HSTATUS64_WPRI   0xFF8FF87EULL
>
> -#define HCOUNTEREN_CY(1 << 0)
> -#define HCOUNTEREN_TM(1 << 1)
> -#define HCOUNTEREN_IR(1 << 2)
> -#define HCOUNTEREN_HPM3  (1 << 3)
> +#define COUNTEREN_CY (1 << 0)
> +#define COUNTEREN_TM (1 << 1)
> +#define COUNTEREN_IR (1 << 2)
> +#define COUNTEREN_HPM3   (1 << 3)
>
>  /* Privilege modes */
>  #define PRV_U 0
> diff --git a/target/riscv/csr.c b/target/riscv/csr.c
> index 50a2c3a3b4..1cd34a6453 100644
> --- a/target/riscv/csr.c
> +++ b/target/riscv/csr.c
> @@ -71,20 +71,20 @@ static RISCVException ctr(CPURISCVState *env, int csrno)
>  if (riscv_cpu_virt_enabled(env)) {
>  switch (csrno) {
>  case CSR_CYCLE:
> -if (!get_field(env->hcounteren, HCOUNTEREN_CY) &&
> -get_field(env->mcounteren, HCOUNTEREN_CY)) {
> +if (!get_field(env->hcounteren, COUNTEREN_CY) &&
> +get_field(env->mcounteren, COUNTEREN_CY)) {
>  return RISCV_EXCP_VIRT_INSTRUCTION_FAULT;
>  }
>  break;
>  case CSR_TIME:
> -if (!get_field(env->hcounteren, HCOUNTEREN_TM) &&
> -get_field(env->mcounteren, HCOUNTEREN_TM)) {
> +if (!get_field(env->hcounteren, COUNTEREN_TM) &&
> +get_field(env->mcounteren, COUNTEREN_TM)) {
>  return RISCV_EXCP_VIRT_INSTRUCTION_FAULT;
>  }
>  break;
>  case CSR_INSTRET:
> -if (!get_field(env->hcounteren, HCOUNTEREN_IR) &&
> -get_field(env->mcounteren, HCOUNTEREN_IR)) {
> +if (!get_field(env->hcounteren, COUNTEREN_IR) &&
> +get_field(env->mcounteren, COUNTEREN_IR)) {
>  return RISCV_EXCP_VIRT_INSTRUCTION_FAULT;
>  }
>  break;
> @@ -98,20 +98,20 @@ static RISCVException ctr(CPURISCVState *env, int csrno)
>  if (riscv_cpu_is_32bit(env)) {
>  switch (csrno) {
>  case CSR_CYCLEH:
> -if (!get_field(env->hcounteren, HCOUNTEREN_CY) &&
> -get_field(env->mcounteren, HCOUNTEREN_CY)) {
> +if (!get_field(env->hcounteren, COUNTEREN_CY) &&
> +get_field(env->mcounteren, COUNTEREN_CY)) {
>  return RISCV_EXCP_VIRT_INSTRUCTION_FAULT;
>  }
>  break;
>  case CSR_TIMEH:
> -if (!get_field(env->hcounteren, HCOUNTEREN_TM) &&
> -get_field(env->mcounteren, HCOUNTEREN_TM)) {
> +if (!get_field(env->hcounteren, COUNTEREN_TM) &&
> +get_field(env->mcounteren, COUNTEREN_TM)) {
>  return RISCV_EXCP_VIRT_INSTRUCTION_FAULT;
>  }
>  break;
>  case CSR_INSTRETH:
> -if (!get_field(env->hcounteren, HCOUNTEREN_IR) &&
> -get_field(env->mcounteren, HCOUNTEREN_IR)) {
> +if (!get_field(env->hcounteren, COUNTEREN_IR) &&
> +get_field(env->mcounteren, COUNTEREN_IR)) {
>  return RISCV_EXCP_VIRT_INSTRUCTION_FAULT;
>  }
>  break;
> --
> 2.25.1
>
>

Re: [PATCH] hw/intc: GIC maintenance interrupt not triggered

[Philippe Mathieu-Daudé]

On 9/15/21 10:58 PM, Shashi Mallela wrote:
> During sbsa acs level 3 testing,it is seen that the GIC
> maintenance interrupts are not triggered and the related test
> cases failed.On debugging the cause,found that the value of
> MISR register (from maintenance_interrupt_state()) was being
> passed to qemu_set_irq() as level.Updated logic to set level
> to 1 if any of the maintenance interrupt attributes are set.
> Confirmed that the GIC maintanence interrupts are triggered and
> sbsa acs test cases passed with this change.
> 

Fixes: c5fc89b36c0 ("hw/intc/arm_gicv3: Implement
gicv3_cpuif_virt_update()")

> Signed-off-by: Shashi Mallela 
> ---
>  hw/intc/arm_gicv3_cpuif.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c
> index 462a35f66e..34691d4fe2 100644
> --- a/hw/intc/arm_gicv3_cpuif.c
> +++ b/hw/intc/arm_gicv3_cpuif.c
> @@ -418,7 +418,9 @@ static void gicv3_cpuif_virt_update(GICv3CPUState *cs)
>  }
>  
>  if (cs->ich_hcr_el2 & ICH_HCR_EL2_EN) {
> -maintlevel = maintenance_interrupt_state(cs);
> +if (maintenance_interrupt_state(cs)) {
> +maintlevel = 1;
> +}

Or:
   maintlevel = !!maintenance_interrupt_state(cs);

But your style is more explicit.

Reviewed-by: Philippe Mathieu-Daudé 

[PATCH v1 1/1] hw/riscv: opentitan: Correct the USB Dev address

[Alistair Francis]

From: Alistair Francis 

Signed-off-by: Alistair Francis 
---
 hw/riscv/opentitan.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/riscv/opentitan.c b/hw/riscv/opentitan.c
index 36a41c8b5b..d38c43abc1 100644
--- a/hw/riscv/opentitan.c
+++ b/hw/riscv/opentitan.c
@@ -39,12 +39,12 @@ static const MemMapEntry ibex_memmap[] = {
 [IBEX_DEV_TIMER] =  {  0x4010,  0x1000  },
 [IBEX_DEV_SENSOR_CTRL] ={  0x4011,  0x1000  },
 [IBEX_DEV_OTP_CTRL] =   {  0x4013,  0x4000  },
+[IBEX_DEV_USBDEV] = {  0x4015,  0x1000  },
 [IBEX_DEV_PWRMGR] = {  0x4040,  0x1000  },
 [IBEX_DEV_RSTMGR] = {  0x4041,  0x1000  },
 [IBEX_DEV_CLKMGR] = {  0x4042,  0x1000  },
 [IBEX_DEV_PINMUX] = {  0x4046,  0x1000  },
 [IBEX_DEV_PADCTRL] ={  0x4047,  0x1000  },
-[IBEX_DEV_USBDEV] = {  0x4050,  0x1000  },
 [IBEX_DEV_FLASH_CTRL] = {  0x4100,  0x1000  },
 [IBEX_DEV_PLIC] =   {  0x4101,  0x1000  },
 [IBEX_DEV_AES] ={  0x4110,  0x1000  },
-- 
2.31.1

Re: [PATCH v3 04/16] iotests/migrate-bitmaps-test: delint

[Philippe Mathieu-Daudé]

On 9/16/21 6:09 AM, John Snow wrote:
> Mostly uninteresting stuff. Move the test injections under a function
> named main() so that the variables used during that process aren't in
> the global scope.
> 
> Signed-off-by: John Snow 
> ---
>  tests/qemu-iotests/tests/migrate-bitmaps-test | 50 +++
>  1 file changed, 28 insertions(+), 22 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé 

Re: [PATCH v3 02/16] iotests/mirror-top-perms: Adjust imports

[Philippe Mathieu-Daudé]

On 9/16/21 6:09 AM, John Snow wrote:
> We need to import things from the qemu namespace; importing the
> namespace alone doesn't bring the submodules with it -- unless someone
> else (like iotests.py) imports them too.
> 
> Adjust the imports.
> 
> Signed-off-by: John Snow 
> ---
>  tests/qemu-iotests/tests/mirror-top-perms | 7 ---
>  1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/tests/qemu-iotests/tests/mirror-top-perms 
> b/tests/qemu-iotests/tests/mirror-top-perms
> index 2fc8dd66e0..de18182590 100755
> --- a/tests/qemu-iotests/tests/mirror-top-perms
> +++ b/tests/qemu-iotests/tests/mirror-top-perms
> @@ -25,7 +25,8 @@ from iotests import qemu_img
>  
>  # Import qemu after iotests.py has amended sys.path
>  # pylint: disable=wrong-import-order
> -import qemu
> +from qemu import qmp
> +from qemu.machine import machine

Not straight-forward import name...

Reviewed-by: Philippe Mathieu-Daudé 

Re: [PATCH v3 08/16] iotests/297: Create main() function

[Philippe Mathieu-Daudé]

On 9/16/21 6:09 AM, John Snow wrote:
> Instead of running "run_linters" directly, create a main() function that
> will be responsible for environment setup, leaving run_linters()
> responsible only for execution of the linters.
> 
> (That environment setup will be moved over in forthcoming commits.)
> 
> Signed-off-by: John Snow 
> Reviewed-by: Vladimir Sementsov-Ogievskiy 
> ---
>  tests/qemu-iotests/297 | 12 
>  1 file changed, 8 insertions(+), 4 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé 

[PATCH v3 16/16] iotests/linters: check mypy files all at once

[John Snow]

We can circumvent the '__main__' redefinition problem by passing
--scripts-are-modules. Take mypy out of the loop per-filename and check
everything in one go: it's quite a bit faster.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/linters.py | 62 ---
 1 file changed, 29 insertions(+), 33 deletions(-)

diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py
index 9c97324e87..ac9f77c5ac 100755
--- a/tests/qemu-iotests/linters.py
+++ b/tests/qemu-iotests/linters.py
@@ -82,40 +82,36 @@ def run_linters(
 print('=== mypy ===')
 sys.stdout.flush()
 
-# We have to call mypy separately for each file.  Otherwise, it
-# will interpret all given files as belonging together (i.e., they
-# may not both define the same classes, etc.; most notably, they
-# must not both define the __main__ module).
-for filename in files:
-p = subprocess.run(
-(
-'python3', '-m', 'mypy',
-'--warn-unused-configs',
-'--disallow-subclassing-any',
-'--disallow-any-generics',
-'--disallow-incomplete-defs',
-'--disallow-untyped-decorators',
-'--no-implicit-optional',
-'--warn-redundant-casts',
-'--warn-unused-ignores',
-'--no-implicit-reexport',
-'--namespace-packages',
-# Until we can use mypy >= 0.920, see
-# https://github.com/python/mypy/issues/9852
-'--no-incremental',
-filename,
-),
-cwd=directory,
-env=env,
-check=False,
-stdout=subprocess.PIPE,
-stderr=subprocess.STDOUT,
-universal_newlines=True
-)
+p = subprocess.run(
+(
+'python3', '-m', 'mypy',
+'--warn-unused-configs',
+'--disallow-subclassing-any',
+'--disallow-any-generics',
+'--disallow-incomplete-defs',
+'--disallow-untyped-decorators',
+'--no-implicit-optional',
+'--warn-redundant-casts',
+'--warn-unused-ignores',
+'--no-implicit-reexport',
+'--namespace-packages',
+# Until we can use mypy >= 0.920, see
+# https://github.com/python/mypy/issues/9852
+'--no-incremental',
+'--scripts-are-modules',
+*files,
+),
+cwd=directory,
+env=env,
+check=False,
+stdout=subprocess.PIPE,
+stderr=subprocess.STDOUT,
+universal_newlines=True
+)
 
-ret += p.returncode
-if p.returncode != 0:
-print(p.stdout)
+ret += p.returncode
+if p.returncode != 0:
+print(p.stdout)
 
 return ret
 
-- 
2.31.1

[PATCH v3 14/16] iotests/linters: Add workaround for mypy bug #9852

[John Snow]

This one is insidious: if you use the invocation
"from {namespace} import {subpackage}" as mirror-top-perms does,
mypy will fail on every-other invocation *if* the package being
imported is a package.

Now, I could just edit mirror-top-perms to avoid this invocation, but
since I tripped on a landmine, I might as well head it off at the pass
and make sure nobody else trips on the same landmine.

It seems to have something to do with the order in which files are
checked as well, meaning the random order in which set(os.listdir())
produces the list of files to test will cause problems intermittently.

mypy >= 0.920 isn't released yet, so add this workaround for now.

See also:
 https://github.com/python/mypy/issues/11010
 https://github.com/python/mypy/issues/9852

Signed-off-by: John Snow 
---
 tests/qemu-iotests/linters.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py
index 4df062a973..9c97324e87 100755
--- a/tests/qemu-iotests/linters.py
+++ b/tests/qemu-iotests/linters.py
@@ -100,6 +100,9 @@ def run_linters(
 '--warn-unused-ignores',
 '--no-implicit-reexport',
 '--namespace-packages',
+# Until we can use mypy >= 0.920, see
+# https://github.com/python/mypy/issues/9852
+'--no-incremental',
 filename,
 ),
 cwd=directory,
-- 
2.31.1

Re: [qemu-web PATCH] Gemfile: Add webrick bundle dependency

[Philippe Mathieu-Daudé]

On 9/16/21 12:27 AM, Paolo Bonzini wrote:
> On 14/09/21 10:21, Daniel P. Berrangé wrote:
>> Yes, this is needed to fix Ruby 3.
>>
>> I didn't propose it myself as I'm not sure if it in turn breaks people
>> with Ruby 2.x ?
> 
> Does it pass the GitLab CI (which uses 2.5)?

Yes:
https://gitlab.com/philmd/qemu-web/-/jobs/1594757246#L466
Installing webrick 1.7.0
Bundle complete! 5 Gemfile dependencies, 32 gems now installed.

[PATCH v3 09/16] iotests/297: Separate environment setup from test execution

[John Snow]

Move environment setup into main(), leaving pure test execution behind
in run_linters().

Signed-off-by: John Snow 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/297 | 36 +---
 1 file changed, 21 insertions(+), 15 deletions(-)

diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
index 6df952808d..08d2b87108 100755
--- a/tests/qemu-iotests/297
+++ b/tests/qemu-iotests/297
@@ -21,7 +21,7 @@ import re
 import shutil
 import subprocess
 import sys
-from typing import List
+from typing import List, Mapping, Optional
 
 import iotests
 
@@ -64,24 +64,16 @@ def get_test_files(directory: str = '.') -> List[str]:
 return list(filter(lambda f: is_python_file(f, directory), check_tests))
 
 
-def run_linters():
-files = get_test_files()
-
-iotests.logger.debug('Files to be checked:')
-iotests.logger.debug(', '.join(sorted(files)))
+def run_linters(
+files: List[str],
+env: Optional[Mapping[str, str]] = None,
+) -> None:
 
 print('=== pylint ===')
 sys.stdout.flush()
 
 # Todo notes are fine, but fixme's or xxx's should probably just be
 # fixed (in tests, at least)
-env = os.environ.copy()
-qemu_module_path = os.path.join(os.path.dirname(__file__),
-'..', '..', 'python')
-try:
-env['PYTHONPATH'] += os.pathsep + qemu_module_path
-except KeyError:
-env['PYTHONPATH'] = qemu_module_path
 subprocess.run(
 ('python3', '-m', 'pylint', '--score=n', '--notes=FIXME,XXX', *files),
 env=env,
@@ -95,7 +87,6 @@ def run_linters():
 # will interpret all given files as belonging together (i.e., they
 # may not both define the same classes, etc.; most notably, they
 # must not both define the __main__ module).
-env['MYPYPATH'] = env['PYTHONPATH']
 for filename in files:
 p = subprocess.run(
 (
@@ -128,7 +119,22 @@ def main() -> None:
 if shutil.which(linter) is None:
 iotests.notrun(f'{linter} not found')
 
-run_linters()
+files = get_test_files()
+
+iotests.logger.debug('Files to be checked:')
+iotests.logger.debug(', '.join(sorted(files)))
+
+env = os.environ.copy()
+qemu_module_path = os.path.join(os.path.dirname(__file__),
+'..', '..', 'python')
+try:
+env['PYTHONPATH'] += os.pathsep + qemu_module_path
+except KeyError:
+env['PYTHONPATH'] = qemu_module_path
+
+env['MYPYPATH'] = env['PYTHONPATH']
+
+run_linters(files, env=env)
 
 
 iotests.script_main(main)
-- 
2.31.1

[PATCH v3 15/16] python: Add iotest linters to test suite

[John Snow]

As a convenience, since iotests is an extremely prominent user of the
qemu.qmp and qemu.machine packages and already implements a linting
regime, run those tests as well so that it's very hard to miss
regressions caused by changes to the python library.

Signed-off-by: John Snow 
---
 python/tests/iotests.sh | 4 
 1 file changed, 4 insertions(+)
 create mode 100755 python/tests/iotests.sh

diff --git a/python/tests/iotests.sh b/python/tests/iotests.sh
new file mode 100755
index 00..70324540cf
--- /dev/null
+++ b/python/tests/iotests.sh
@@ -0,0 +1,4 @@
+#!/bin/sh -e
+
+cd ../tests/qemu-iotests/
+python3 -m linters
-- 
2.31.1

[PATCH v3 07/16] iotests/297: Don't rely on distro-specific linter binaries

[John Snow]

'pylint-3' is another Fedora-ism. Use "python3 -m pylint" or "python3 -m
mypy" to access these scripts instead. This style of invocation will
prefer the "correct" tool when run in a virtual environment.

Note that we still check for "pylint-3" before the test begins -- this
check is now "overly strict", but shouldn't cause anything that was
already running correctly to start failing.

Signed-off-by: John Snow 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Philippe Mathieu-Daudé 
---
 tests/qemu-iotests/297 | 45 --
 1 file changed, 26 insertions(+), 19 deletions(-)

diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
index b3a56a3a29..01dd8147d5 100755
--- a/tests/qemu-iotests/297
+++ b/tests/qemu-iotests/297
@@ -82,8 +82,11 @@ def run_linters():
 env['PYTHONPATH'] += os.pathsep + qemu_module_path
 except KeyError:
 env['PYTHONPATH'] = qemu_module_path
-subprocess.run(('pylint-3', '--score=n', '--notes=FIXME,XXX', *files),
-   env=env, check=False)
+subprocess.run(
+('python3', '-m', 'pylint', '--score=n', '--notes=FIXME,XXX', *files),
+env=env,
+check=False,
+)
 
 print('=== mypy ===')
 sys.stdout.flush()
@@ -94,23 +97,27 @@ def run_linters():
 # must not both define the __main__ module).
 env['MYPYPATH'] = env['PYTHONPATH']
 for filename in files:
-p = subprocess.run(('mypy',
-'--warn-unused-configs',
-'--disallow-subclassing-any',
-'--disallow-any-generics',
-'--disallow-incomplete-defs',
-'--disallow-untyped-decorators',
-'--no-implicit-optional',
-'--warn-redundant-casts',
-'--warn-unused-ignores',
-'--no-implicit-reexport',
-'--namespace-packages',
-filename),
-   env=env,
-   check=False,
-   stdout=subprocess.PIPE,
-   stderr=subprocess.STDOUT,
-   universal_newlines=True)
+p = subprocess.run(
+(
+'python3', '-m', 'mypy',
+'--warn-unused-configs',
+'--disallow-subclassing-any',
+'--disallow-any-generics',
+'--disallow-incomplete-defs',
+'--disallow-untyped-decorators',
+'--no-implicit-optional',
+'--warn-redundant-casts',
+'--warn-unused-ignores',
+'--no-implicit-reexport',
+'--namespace-packages',
+filename,
+),
+env=env,
+check=False,
+stdout=subprocess.PIPE,
+stderr=subprocess.STDOUT,
+universal_newlines=True
+)
 
 if p.returncode != 0:
 print(p.stdout)
-- 
2.31.1

[PATCH v3 13/16] iotests/linters: Add entry point for Python CI linters

[John Snow]

Add a main() function to linters.py so that the Python CI infrastructure
has something it can run.

Now, linters.py represents an invocation of the linting scripts that
more resembles a "normal" execution of pylint/mypy, like you'd expect to
use if 'qemu' was a bona-fide package you obtained from PyPI.

297, by contrast, now represents the iotests-specific configuration bits
you need to get it to function correctly as a part of iotests, and with
'qemu' as a namespace package that isn't "installed" to the current
environment, but just lives elsewhere in our source tree.

By doing this, we will able to run the same linting configuration from
the Python CI tests without calling iotest logging functions or messing
around with PYTHONPATH / MYPYPATH.

iotest 297 continues to operate in a standalone fashion for now --
presumably, it's convenient for block maintainers and contributors to
run in this manner. We can either remove this functionality at a later
date if everyone is happy with the Python CI, or we can opt to continue
to maintain it. Please let me know how you feel.

See the following commit for how this is used from the Python packaging side.

Signed-off-by: John Snow 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/linters.py | 13 +
 1 file changed, 13 insertions(+)

diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py
index e263b7cbee..4df062a973 100755
--- a/tests/qemu-iotests/linters.py
+++ b/tests/qemu-iotests/linters.py
@@ -115,3 +115,16 @@ def run_linters(
 print(p.stdout)
 
 return ret
+
+
+def main() -> int:
+"""
+Used by the Python CI system as an entry point to run these linters.
+"""
+directory = os.path.dirname(os.path.realpath(__file__))
+files = get_test_files(directory)
+return run_linters(files, directory)
+
+
+if __name__ == '__main__':
+sys.exit(main())
-- 
2.31.1

[PATCH v3 08/16] iotests/297: Create main() function

[John Snow]

Instead of running "run_linters" directly, create a main() function that
will be responsible for environment setup, leaving run_linters()
responsible only for execution of the linters.

(That environment setup will be moved over in forthcoming commits.)

Signed-off-by: John Snow 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/297 | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
index 01dd8147d5..6df952808d 100755
--- a/tests/qemu-iotests/297
+++ b/tests/qemu-iotests/297
@@ -123,8 +123,12 @@ def run_linters():
 print(p.stdout)
 
 
-for linter in ('pylint-3', 'mypy'):
-if shutil.which(linter) is None:
-iotests.notrun(f'{linter} not found')
+def main() -> None:
+for linter in ('pylint-3', 'mypy'):
+if shutil.which(linter) is None:
+iotests.notrun(f'{linter} not found')
 
-iotests.script_main(run_linters)
+run_linters()
+
+
+iotests.script_main(main)
-- 
2.31.1

[PATCH v3 06/16] iotests/297: Add get_files() function

[John Snow]

Split out file discovery into its own method to begin separating out the
"environment setup" and "test execution" phases.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/297 | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
index 3e86f788fc..b3a56a3a29 100755
--- a/tests/qemu-iotests/297
+++ b/tests/qemu-iotests/297
@@ -21,6 +21,7 @@ import re
 import shutil
 import subprocess
 import sys
+from typing import List
 
 import iotests
 
@@ -56,10 +57,15 @@ def is_python_file(filename: str, directory: str = '.') -> 
bool:
 return False
 
 
+def get_test_files(directory: str = '.') -> List[str]:
+named_test_dir = os.path.join(directory, 'tests')
+named_tests = [f"tests/{entry}" for entry in os.listdir(named_test_dir)]
+check_tests = set(os.listdir(directory) + named_tests) - set(SKIP_FILES)
+return list(filter(lambda f: is_python_file(f, directory), check_tests))
+
+
 def run_linters():
-named_tests = [f'tests/{entry}' for entry in os.listdir('tests')]
-check_tests = set(os.listdir('.') + named_tests) - set(SKIP_FILES)
-files = [filename for filename in check_tests if is_python_file(filename)]
+files = get_test_files()
 
 iotests.logger.debug('Files to be checked:')
 iotests.logger.debug(', '.join(sorted(files)))
-- 
2.31.1

[PATCH v3 12/16] iotests/297: split linters.py off from 297

[John Snow]

Split the linter execution itself out from 297, leaving just environment
setup in 297. This is done so that non-iotest code can invoke the
linters without needing to worry about imports of unpackaged iotest
code.

Eventually, it should be possible to replace linters.py with mypy.ini
and pylintrc files that instruct these tools how to run properly in this
directory, but ... not yet, and not in this series.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/297| 105 ++
 tests/qemu-iotests/linters.py | 117 ++
 2 files changed, 121 insertions(+), 101 deletions(-)
 create mode 100755 tests/qemu-iotests/linters.py

diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
index f9ddfb53a0..3d29af5b78 100755
--- a/tests/qemu-iotests/297
+++ b/tests/qemu-iotests/297
@@ -17,110 +17,13 @@
 # along with this program.  If not, see .
 
 import os
-import re
 import shutil
-import subprocess
-import sys
-from typing import List, Mapping, Optional
 
 import iotests
+import linters
 
 
-# TODO: Empty this list!
-SKIP_FILES = (
-'030', '040', '041', '044', '045', '055', '056', '057', '065', '093',
-'096', '118', '124', '132', '136', '139', '147', '148', '149',
-'151', '152', '155', '163', '165', '194', '196', '202',
-'203', '205', '206', '207', '208', '210', '211', '212', '213', '216',
-'218', '219', '224', '228', '234', '235', '236', '237', '238',
-'240', '242', '245', '246', '248', '255', '256', '257', '258', '260',
-'262', '264', '266', '274', '277', '280', '281', '295', '296', '298',
-'299', '302', '303', '304', '307',
-'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py'
-)
-
-
-def is_python_file(filename: str, directory: str = '.') -> bool:
-filepath = os.path.join(directory, filename)
-
-if not os.path.isfile(filepath):
-return False
-
-if filename.endswith('.py'):
-return True
-
-with open(filepath, encoding='utf-8') as f:
-try:
-first_line = f.readline()
-return re.match('^#!.*python', first_line) is not None
-except UnicodeDecodeError:  # Ignore binary files
-return False
-
-
-def get_test_files(directory: str = '.') -> List[str]:
-named_test_dir = os.path.join(directory, 'tests')
-named_tests = [f"tests/{entry}" for entry in os.listdir(named_test_dir)]
-check_tests = set(os.listdir(directory) + named_tests) - set(SKIP_FILES)
-return list(filter(lambda f: is_python_file(f, directory), check_tests))
-
-
-def run_linters(
-files: List[str],
-directory: str = '.',
-env: Optional[Mapping[str, str]] = None,
-) -> int:
-ret = 0
-
-print('=== pylint ===')
-sys.stdout.flush()
-
-# Todo notes are fine, but fixme's or xxx's should probably just be
-# fixed (in tests, at least)
-p = subprocess.run(
-('python3', '-m', 'pylint', '--score=n', '--notes=FIXME,XXX', *files),
-cwd=directory,
-env=env,
-check=False,
-universal_newlines=True,
-)
-ret += p.returncode
-
-print('=== mypy ===')
-sys.stdout.flush()
-
-# We have to call mypy separately for each file.  Otherwise, it
-# will interpret all given files as belonging together (i.e., they
-# may not both define the same classes, etc.; most notably, they
-# must not both define the __main__ module).
-for filename in files:
-p = subprocess.run(
-(
-'python3', '-m', 'mypy',
-'--warn-unused-configs',
-'--disallow-subclassing-any',
-'--disallow-any-generics',
-'--disallow-incomplete-defs',
-'--disallow-untyped-decorators',
-'--no-implicit-optional',
-'--warn-redundant-casts',
-'--warn-unused-ignores',
-'--no-implicit-reexport',
-'--namespace-packages',
-filename,
-),
-cwd=directory,
-env=env,
-check=False,
-stdout=subprocess.PIPE,
-stderr=subprocess.STDOUT,
-universal_newlines=True
-)
-
-ret += p.returncode
-if p.returncode != 0:
-print(p.stdout)
-
-return ret
+# Looking for the list of excluded tests? See linters.py !
 
 
 def main() -> None:
@@ -128,7 +31,7 @@ def main() -> None:
 if shutil.which(linter) is None:
 iotests.notrun(f'{linter} not found')
 
-files = get_test_files()
+files = linters.get_test_files()
 
 iotests.logger.debug('Files to be checked:')
 iotests.logger.debug(', '.join(sorted(files)))
@@ -143,7 +46,7 @@ def main() -> None:
 
 env['MYPYPATH'] = env['PYTHONPATH']
 
-run_linters(files, env=env)
+linters.run_linters(files, env=env)
 
 
 iotests.script_main(main)
diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters

[PATCH v3 10/16] iotests/297: Add 'directory' argument to run_linters

[John Snow]

Allow run_linters to work well if it's executed from a different
directory.

Signed-off-by: John Snow 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/297 | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
index 08d2b87108..e05c99972e 100755
--- a/tests/qemu-iotests/297
+++ b/tests/qemu-iotests/297
@@ -66,6 +66,7 @@ def get_test_files(directory: str = '.') -> List[str]:
 
 def run_linters(
 files: List[str],
+directory: str = '.',
 env: Optional[Mapping[str, str]] = None,
 ) -> None:
 
@@ -76,6 +77,7 @@ def run_linters(
 # fixed (in tests, at least)
 subprocess.run(
 ('python3', '-m', 'pylint', '--score=n', '--notes=FIXME,XXX', *files),
+cwd=directory,
 env=env,
 check=False,
 )
@@ -103,6 +105,7 @@ def run_linters(
 '--namespace-packages',
 filename,
 ),
+cwd=directory,
 env=env,
 check=False,
 stdout=subprocess.PIPE,
-- 
2.31.1

[PATCH v3 05/16] iotests/297: modify is_python_file to work from any CWD

[John Snow]

Add a directory argument to is_python_file to allow it to work correctly
no matter what CWD we happen to run it from. This is done in
anticipation of running the iotests from another directory (./python/).

Signed-off-by: John Snow 
---
 tests/qemu-iotests/297 | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
index b04cba5366..3e86f788fc 100755
--- a/tests/qemu-iotests/297
+++ b/tests/qemu-iotests/297
@@ -39,14 +39,16 @@ SKIP_FILES = (
 )
 
 
-def is_python_file(filename):
-if not os.path.isfile(filename):
+def is_python_file(filename: str, directory: str = '.') -> bool:
+filepath = os.path.join(directory, filename)
+
+if not os.path.isfile(filepath):
 return False
 
 if filename.endswith('.py'):
 return True
 
-with open(filename, encoding='utf-8') as f:
+with open(filepath, encoding='utf-8') as f:
 try:
 first_line = f.readline()
 return re.match('^#!.*python', first_line) is not None
-- 
2.31.1

[PATCH v3 11/16] iotests/297: return error code from run_linters()

[John Snow]

This turns run_linters() into a bit of a hybrid test; returning non-zero
on failed execution while also printing diffable information. This is
done for the benefit of the avocado simple test runner, which will soon
be attempting to execute this test from a different environment.

(Note: universal_newlines is added to the pylint invocation for type
consistency with the mypy run -- it's not strictly necessary, but it
avoids some typing errors caused by our re-use of the 'p' variable.)

Signed-off-by: John Snow 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/297 | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297
index e05c99972e..f9ddfb53a0 100755
--- a/tests/qemu-iotests/297
+++ b/tests/qemu-iotests/297
@@ -68,19 +68,22 @@ def run_linters(
 files: List[str],
 directory: str = '.',
 env: Optional[Mapping[str, str]] = None,
-) -> None:
+) -> int:
+ret = 0
 
 print('=== pylint ===')
 sys.stdout.flush()
 
 # Todo notes are fine, but fixme's or xxx's should probably just be
 # fixed (in tests, at least)
-subprocess.run(
+p = subprocess.run(
 ('python3', '-m', 'pylint', '--score=n', '--notes=FIXME,XXX', *files),
 cwd=directory,
 env=env,
 check=False,
+universal_newlines=True,
 )
+ret += p.returncode
 
 print('=== mypy ===')
 sys.stdout.flush()
@@ -113,9 +116,12 @@ def run_linters(
 universal_newlines=True
 )
 
+ret += p.returncode
 if p.returncode != 0:
 print(p.stdout)
 
+return ret
+
 
 def main() -> None:
 for linter in ('pylint-3', 'mypy'):
-- 
2.31.1

[PATCH v3 04/16] iotests/migrate-bitmaps-test: delint

[John Snow]

Mostly uninteresting stuff. Move the test injections under a function
named main() so that the variables used during that process aren't in
the global scope.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/tests/migrate-bitmaps-test | 50 +++
 1 file changed, 28 insertions(+), 22 deletions(-)

diff --git a/tests/qemu-iotests/tests/migrate-bitmaps-test 
b/tests/qemu-iotests/tests/migrate-bitmaps-test
index dc431c35b3..c23df3d75c 100755
--- a/tests/qemu-iotests/tests/migrate-bitmaps-test
+++ b/tests/qemu-iotests/tests/migrate-bitmaps-test
@@ -19,10 +19,11 @@
 # along with this program.  If not, see .
 #
 
-import os
 import itertools
 import operator
+import os
 import re
+
 import iotests
 from iotests import qemu_img, qemu_img_create, Timeout
 
@@ -224,25 +225,6 @@ def inject_test_case(klass, suffix, method, *args, 
**kwargs):
 setattr(klass, 'test_' + method + suffix, lambda self: mc(self))
 
 
-for cmb in list(itertools.product((True, False), repeat=5)):
-name = ('_' if cmb[0] else '_not_') + 'persistent_'
-name += ('_' if cmb[1] else '_not_') + 'migbitmap_'
-name += '_online' if cmb[2] else '_offline'
-name += '_shared' if cmb[3] else '_nonshared'
-if cmb[4]:
-name += '__pre_shutdown'
-
-inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migration',
- *list(cmb))
-
-for cmb in list(itertools.product((True, False), repeat=2)):
-name = ('_' if cmb[0] else '_not_') + 'persistent_'
-name += ('_' if cmb[1] else '_not_') + 'migbitmap'
-
-inject_test_case(TestDirtyBitmapMigration, name,
- 'do_test_migration_resume_source', *list(cmb))
-
-
 class TestDirtyBitmapBackingMigration(iotests.QMPTestCase):
 def setUp(self):
 qemu_img_create('-f', iotests.imgfmt, base_a, size)
@@ -304,6 +286,30 @@ class TestDirtyBitmapBackingMigration(iotests.QMPTestCase):
 self.assert_qmp(result, 'return', {})
 
 
+def main() -> None:
+for cmb in list(itertools.product((True, False), repeat=5)):
+name = ('_' if cmb[0] else '_not_') + 'persistent_'
+name += ('_' if cmb[1] else '_not_') + 'migbitmap_'
+name += '_online' if cmb[2] else '_offline'
+name += '_shared' if cmb[3] else '_nonshared'
+if cmb[4]:
+name += '__pre_shutdown'
+
+inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migration',
+ *list(cmb))
+
+for cmb in list(itertools.product((True, False), repeat=2)):
+name = ('_' if cmb[0] else '_not_') + 'persistent_'
+name += ('_' if cmb[1] else '_not_') + 'migbitmap'
+
+inject_test_case(TestDirtyBitmapMigration, name,
+ 'do_test_migration_resume_source', *list(cmb))
+
+iotests.main(
+supported_fmts=['qcow2'],
+supported_protocols=['file']
+)
+
+
 if __name__ == '__main__':
-iotests.main(supported_fmts=['qcow2'],
- supported_protocols=['file'])
+main()
-- 
2.31.1

[PATCH v3 02/16] iotests/mirror-top-perms: Adjust imports

[John Snow]

We need to import things from the qemu namespace; importing the
namespace alone doesn't bring the submodules with it -- unless someone
else (like iotests.py) imports them too.

Adjust the imports.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/tests/mirror-top-perms | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/tests/qemu-iotests/tests/mirror-top-perms 
b/tests/qemu-iotests/tests/mirror-top-perms
index 2fc8dd66e0..de18182590 100755
--- a/tests/qemu-iotests/tests/mirror-top-perms
+++ b/tests/qemu-iotests/tests/mirror-top-perms
@@ -25,7 +25,8 @@ from iotests import qemu_img
 
 # Import qemu after iotests.py has amended sys.path
 # pylint: disable=wrong-import-order
-import qemu
+from qemu import qmp
+from qemu.machine import machine
 
 
 image_size = 1 * 1024 * 1024
@@ -47,7 +48,7 @@ class TestMirrorTopPerms(iotests.QMPTestCase):
 def tearDown(self):
 try:
 self.vm.shutdown()
-except qemu.machine.machine.AbnormalShutdown:
+except machine.AbnormalShutdown:
 pass
 
 if self.vm_b is not None:
@@ -102,7 +103,7 @@ class TestMirrorTopPerms(iotests.QMPTestCase):
 self.vm_b.launch()
 print('ERROR: VM B launched successfully, this should not have '
   'happened')
-except qemu.qmp.QMPConnectError:
+except qmp.QMPConnectError:
 assert 'Is another process using the image' in self.vm_b.get_log()
 
 result = self.vm.qmp('block-job-cancel',
-- 
2.31.1

[PATCH v3 01/16] python: Update for pylint 2.10

[John Snow]

A few new annoyances. Of note is the new warning for an unspecified
encoding when opening a text file, which actually does indicate a
potentially real problem; see
https://www.python.org/dev/peps/pep-0597/#motivation

Use LC_CTYPE to determine an encoding to use for interpreting QEMU's
terminal output. Note that Python states: "language code and encoding
may be None if their values cannot be determined" -- use a platform
default as a backup.

Signed-off-by: John Snow 
---
 python/qemu/machine/machine.py | 9 -
 python/setup.cfg   | 1 +
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py
index a7081b1845..51b6e79a13 100644
--- a/python/qemu/machine/machine.py
+++ b/python/qemu/machine/machine.py
@@ -19,6 +19,7 @@
 
 import errno
 from itertools import chain
+import locale
 import logging
 import os
 import shutil
@@ -290,8 +291,14 @@ def get_pid(self) -> Optional[int]:
 return self._subp.pid
 
 def _load_io_log(self) -> None:
+# Assume that the output encoding of QEMU's terminal output
+# is defined by our locale. If indeterminate, use a platform default.
+_, encoding = locale.getlocale()
+if encoding is None:
+encoding = locale.getpreferredencoding(do_setlocale=False)
 if self._qemu_log_path is not None:
-with open(self._qemu_log_path, "r") as iolog:
+with open(self._qemu_log_path, "r",
+  encoding=encoding) as iolog:
 self._iolog = iolog.read()
 
 @property
diff --git a/python/setup.cfg b/python/setup.cfg
index 83909c1c97..0f0cab098f 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -104,6 +104,7 @@ good-names=i,
 [pylint.similarities]
 # Ignore imports when computing similarities.
 ignore-imports=yes
+ignore-signatures=yes
 
 # Minimum lines number of a similarity.
 # TODO: Remove after we opt in to Pylint 2.8.3. See commit msg.
-- 
2.31.1

[PATCH v3 03/16] iotests/migrate-bitmaps-postcopy-test: declare instance variables

[John Snow]

Signed-off-by: John Snow 
---
 tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test 
b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
index 00ebb5c251..9c00ae61c8 100755
--- a/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
+++ b/tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test
@@ -115,6 +115,9 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase):
 self.vm_a_events = []
 self.vm_b_events = []
 
+self.discards1_sha256: str
+self.all_discards_sha256: str
+
 def start_postcopy(self):
 """ Run migration until RESUME event on target. Return this event. """
 for i in range(nb_bitmaps):
-- 
2.31.1

[PATCH v3 00/16] python/iotests: Run iotest linters during Python CI

[John Snow]

GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-package-iotest
CI: https://gitlab.com/jsnow/qemu/-/pipelines/371611883
Based-On: <20210915175318.853225-1-hre...@redhat.com>
  "[PULL 00/32] Block patches"

Since iotests are such a heavy and prominent user of the Python qemu.qmp
and qemu.machine packages, it would be convenient if the Python linting
suite also checked this client for any possible regressions introduced
by shifting around signatures, types, or interfaces in these packages.

(We'd eventually find those problems when iotest 297 ran, but with
increasing distance between Python development and Block development,
the risk of an accidental breakage in this regard increases. I,
personally, know to run iotests (and especially 297) after changing
Python code, but not everyone in the future might. Plus, I am lazy, and
I like only having to push one button.)

Add the ability for the Python CI to run the iotest linters too, which
means that the iotest linters would be checked against:

- Python 3.6, using a frozen set of linting packages at their oldest
  supported versions, using 'pipenv'
- Python 3.6 through Python 3.10 inclusive, using 'tox' and the latest
  versions of mypy/pylint that happen to be installed during test
  time. This CI test is allowed to fail with a warning, and can serve
  as a bellwether for when new incompatible changes may disrupt the
  linters. Testing against old and new Python interpreters alike can
  help surface incompatibility issues we may need to be aware of.)

Here are example outputs of those CI jobs with this series applied:
 - "check-python-pipenv": https://gitlab.com/jsnow/qemu/-/jobs/1377735087
 - "check-python-tox": https://gitlab.com/jsnow/qemu/-/jobs/1377735088

You can also run these same tests locally from ./python, plus one more:

- "make check-dev" to test against whatever python you have.
- "make check-pipenv", if you have Python 3.6 and pipenv installed.
- "make check-tox", to test against multiple python versions you have installed,
from 3.6 to 3.10 inclusive. (CI tests against all 5.)

See the old commit message for more sample output, etc.

https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg07056.html

V3:
 - Added patch 1 which has been submitted separately upstream,
   but was necessary for testing.
 - Rebased on top of hreitz/block, which fixed some linting issues.
 - Added a workaround for a rather nasty mypy bug ... >:(

V2:
 - Added patches 1-5 which do some more delinting.
 - Added patch 8, which scans subdirs for tests to lint.
 - Added patch 17, which improves the speed of mypy analysis.
 - Patch 14 is different because of the new patch 8.

John Snow (16):
  python: Update for pylint 2.10
  iotests/mirror-top-perms: Adjust imports
  iotests/migrate-bitmaps-postcopy-test: declare instance variables
  iotests/migrate-bitmaps-test: delint
  iotests/297: modify is_python_file to work from any CWD
  iotests/297: Add get_files() function
  iotests/297: Don't rely on distro-specific linter binaries
  iotests/297: Create main() function
  iotests/297: Separate environment setup from test execution
  iotests/297: Add 'directory' argument to run_linters
  iotests/297: return error code from run_linters()
  iotests/297: split linters.py off from 297
  iotests/linters: Add entry point for Python CI linters
  iotests/linters: Add workaround for mypy bug #9852
  python: Add iotest linters to test suite
  iotests/linters: check mypy files all at once

 python/qemu/machine/machine.py|   9 +-
 python/setup.cfg  |   1 +
 python/tests/iotests.sh   |   4 +
 tests/qemu-iotests/297|  81 ++-
 tests/qemu-iotests/linters.py | 129 ++
 .../tests/migrate-bitmaps-postcopy-test   |   3 +
 tests/qemu-iotests/tests/migrate-bitmaps-test |  50 ---
 tests/qemu-iotests/tests/mirror-top-perms |   7 +-
 8 files changed, 186 insertions(+), 98 deletions(-)
 create mode 100755 python/tests/iotests.sh
 create mode 100755 tests/qemu-iotests/linters.py

-- 
2.31.1

Re: Re: [RFC v6] virtio/vsock: add two more queues for datagram types

[Jiang Wang .]

On Tue, Sep 14, 2021 at 5:46 AM Stefan Hajnoczi  wrote:
>
> On Mon, Sep 13, 2021 at 10:18:43PM +, Jiang Wang wrote:
> > Datagram sockets are connectionless and unreliable.
> > The sender does not know the capacity of the receiver
> > and may send more packets than the receiver can handle.
> >
> > Add two more dedicate virtqueues for datagram sockets,
> > so that it will not unfairly steal resources from
> > stream and future connection-oriented sockets.
> >
> > Signed-off-by: Jiang Wang 
> > ---
> > v1 -> v2: use qemu cmd option to control number of queues,
> > removed configuration settings for dgram.
> > v2 -> v3: use ioctl to get features and decide number of
> > virt queues, instead of qemu cmd option.
> > v3 -> v4: change DGRAM feature bit value to 2. Add an argument
> > in vhost_vsock_common_realize to indicate dgram is supported or not.
> > v4 -> v5: don't open dev to get vhostfd. Removed leftover definition of
> > enable_dgram
> > v5 -> v6: fix style errors. Imporve error handling of
> > vhost_vsock_dgram_supported. Rename MAX_VQS_WITH_DGRAM and another 
> > one.
> >
> >  hw/virtio/vhost-user-vsock.c  |  2 +-
> >  hw/virtio/vhost-vsock-common.c| 25 --
> >  hw/virtio/vhost-vsock.c   | 34 ++-
> >  include/hw/virtio/vhost-vsock-common.h|  6 ++--
> >  include/hw/virtio/vhost-vsock.h   |  3 ++
> >  include/standard-headers/linux/virtio_vsock.h |  1 +
> >  6 files changed, 64 insertions(+), 7 deletions(-)
> >
> > diff --git a/hw/virtio/vhost-user-vsock.c b/hw/virtio/vhost-user-vsock.c
> > index 6095ed7349..e9ec0e1c00 100644
> > --- a/hw/virtio/vhost-user-vsock.c
> > +++ b/hw/virtio/vhost-user-vsock.c
> > @@ -105,7 +105,7 @@ static void vuv_device_realize(DeviceState *dev, Error 
> > **errp)
> >  return;
> >  }
> >
> > -vhost_vsock_common_realize(vdev, "vhost-user-vsock");
> > +vhost_vsock_common_realize(vdev, "vhost-user-vsock", false);
>
> VIRTIO_VSOCK_F_DGRAM support should work equally well for
> vhost-user-vsock. I don't think there is a need to disable it here.
>
Stefano mentioned in previous email ( V3 ) that I can disable dgram
for vhost-user-vsock for now. I think we need some extra changes to
fully support vhost-vsock-user, like feature discovery?

> >
> >  vhost_dev_set_config_notifier(&vvc->vhost_dev, &vsock_ops);
> >
> > diff --git a/hw/virtio/vhost-vsock-common.c b/hw/virtio/vhost-vsock-common.c
> > index 4ad6e234ad..d94636e04e 100644
> > --- a/hw/virtio/vhost-vsock-common.c
> > +++ b/hw/virtio/vhost-vsock-common.c
> > @@ -17,6 +17,8 @@
> >  #include "hw/virtio/vhost-vsock.h"
> >  #include "qemu/iov.h"
> >  #include "monitor/monitor.h"
> > +#include 
> > +#include 
> >
> >  int vhost_vsock_common_start(VirtIODevice *vdev)
> >  {
> > @@ -196,9 +198,11 @@ int vhost_vsock_common_post_load(void *opaque, int 
> > version_id)
> >  return 0;
> >  }
> >
> > -void vhost_vsock_common_realize(VirtIODevice *vdev, const char *name)
> > +void vhost_vsock_common_realize(VirtIODevice *vdev, const char *name,
> > +   bool enable_dgram)
> >  {
> >  VHostVSockCommon *vvc = VHOST_VSOCK_COMMON(vdev);
> > +int nvqs = VHOST_VSOCK_NVQS;
> >
> >  virtio_init(vdev, name, VIRTIO_ID_VSOCK,
> >  sizeof(struct virtio_vsock_config));
> > @@ -209,12 +213,20 @@ void vhost_vsock_common_realize(VirtIODevice *vdev, 
> > const char *name)
> >  vvc->trans_vq = virtio_add_queue(vdev, VHOST_VSOCK_QUEUE_SIZE,
> > vhost_vsock_common_handle_output);
> >
> > +if (enable_dgram) {
> > +nvqs = VHOST_VSOCK_NVQS_DGRAM;
> > +vvc->dgram_recv_vq = virtio_add_queue(vdev, VHOST_VSOCK_QUEUE_SIZE,
> > +  
> > vhost_vsock_common_handle_output);
> > +vvc->dgram_trans_vq = virtio_add_queue(vdev, 
> > VHOST_VSOCK_QUEUE_SIZE,
> > +  
> > vhost_vsock_common_handle_output);
> > +}
>
> I think the virtqueues can be added unconditionally.
>
OK.
> > +
> >  /* The event queue belongs to QEMU */
> >  vvc->event_vq = virtio_add_queue(vdev, VHOST_VSOCK_QUEUE_SIZE,
> > vhost_vsock_common_handle_output);
> >
> > -vvc->vhost_dev.nvqs = ARRAY_SIZE(vvc->vhost_vqs);
> > -vvc->vhost_dev.vqs = vvc->vhost_vqs;
> > +vvc->vhost_dev.nvqs = nvqs;
> > +vvc->vhost_dev.vqs = g_new0(struct vhost_virtqueue, 
> > vvc->vhost_dev.nvqs);
>
> IIUC the number of virtqueues needs to be the maximum supported number.
> It's okay to have more virtqueues than needed. The feature bit is used
> by the driver to detect the device's support for dgrams, not the number
> of virtqueues.
>
OK. I can just make these changes. But I am not able to test vhost-user-vsock
as of now. I tried to follow the steps on here:
https://patchew.org/QEMU/20200515152

Re: [PATCH v4 04/33] i386: Add 'sgx-epc' device to expose EPC sections to guest

[Yang Zhong]

On Tue, Sep 14, 2021 at 08:36:24AM +0200, Philippe Mathieu-Daudé wrote:
> On 7/19/21 1:21 PM, Yang Zhong wrote:
> > From: Sean Christopherson 
> > 
> > SGX EPC is enumerated through CPUID, i.e. EPC "devices" need to be
> > realized prior to realizing the vCPUs themselves, which occurs long
> > before generic devices are parsed and realized.  Because of this,
> > do not allow 'sgx-epc' devices to be instantiated after vCPUS have
> > been created.
> > 
> > The 'sgx-epc' device is essentially a placholder at this time, it will
> > be fully implemented in a future patch along with a dedicated command
> > to create 'sgx-epc' devices.
> > 
> > Signed-off-by: Sean Christopherson 
> > Signed-off-by: Yang Zhong 
> > ---
> >  hw/i386/meson.build   |   1 +
> >  hw/i386/sgx-epc.c | 161 ++
> >  include/hw/i386/sgx-epc.h |  44 +++
> >  3 files changed, 206 insertions(+)
> >  create mode 100644 hw/i386/sgx-epc.c
> >  create mode 100644 include/hw/i386/sgx-epc.h
> > 
> > diff --git a/hw/i386/meson.build b/hw/i386/meson.build
> > index 80dad29f2b..27476b36bb 100644
> > --- a/hw/i386/meson.build
> > +++ b/hw/i386/meson.build
> > @@ -5,6 +5,7 @@ i386_ss.add(files(
> >'e820_memory_layout.c',
> >'multiboot.c',
> >'x86.c',
> > +  'sgx-epc.c',
> >  ))
> >  
> >  i386_ss.add(when: 'CONFIG_X86_IOMMU', if_true: files('x86-iommu.c'),
> > diff --git a/hw/i386/sgx-epc.c b/hw/i386/sgx-epc.c
> > new file mode 100644
> > index 00..aa487dea79
> > --- /dev/null
> > +++ b/hw/i386/sgx-epc.c
> > @@ -0,0 +1,161 @@
> > +/*
> > + * SGX EPC device
> > + *
> > + * Copyright (C) 2019 Intel Corporation
> > + *
> > + * Authors:
> > + *   Sean Christopherson 
> > + *
> > + * This work is licensed under the terms of the GNU GPL, version 2 or 
> > later.
> > + * See the COPYING file in the top-level directory.
> > + */
> > +#include "qemu/osdep.h"
> > +#include "hw/i386/pc.h"
> > +#include "hw/i386/sgx-epc.h"
> > +#include "hw/mem/memory-device.h"
> > +#include "hw/qdev-properties.h"
> > +#include "monitor/qdev.h"
> 
> Is that include used?
> 
> > +#include "qapi/error.h"
> > +#include "qapi/visitor.h"
> > +#include "qemu/config-file.h"
> 
> Ditto.
> 
> > +#include "qemu/error-report.h"
> 
> Ditto.
> 
> > +#include "qemu/option.h"
> 
> Ditto.
> 
> > +#include "qemu/units.h"
> 
> Ditto.
>
 
  Thanks Philippe, those header files have been removed in the Paolo's
  gitlab(sgx branch), thanks!

  Yang

 
> > +#include "target/i386/cpu.h"
> > +#include "exec/address-spaces.h"

[PATCH v7 6/7] spapr_numa.c: FORM2 NUMA affinity support

[Daniel Henrique Barboza]

The main feature of FORM2 affinity support is the separation of NUMA
distances from ibm,associativity information. This allows for a more
flexible and straightforward NUMA distance assignment without relying on
complex associations between several levels of NUMA via
ibm,associativity matches. Another feature is its extensibility. This base
support contains the facilities for NUMA distance assignment, but in the
future more facilities will be added for latency, performance, bandwidth
and so on.

This patch implements the base FORM2 affinity support as follows:

- the use of FORM2 associativity is indicated by using bit 2 of byte 5
of ibm,architecture-vec-5. A FORM2 aware guest can choose to use FORM1
or FORM2 affinity. Setting both forms will default to FORM2. We're not
advertising FORM2 for pseries-6.1 and older machine versions to prevent
guest visible changes in those;

- ibm,associativity-reference-points has a new semantic. Instead of
being used to calculate distances via NUMA levels, it's now used to
indicate the primary domain index in the ibm,associativity domain of
each resource. In our case it's set to {0x4}, matching the position
where we already place logical_domain_id;

- two new RTAS DT artifacts are introduced: ibm,numa-lookup-index-table
and ibm,numa-distance-table. The index table is used to list all the
NUMA logical domains of the platform, in ascending order, and allows for
spartial NUMA configurations (although QEMU ATM doesn't support that).
ibm,numa-distance-table is an array that contains all the distances from
the first NUMA node to all other nodes, then the second NUMA node
distances to all other nodes and so on;

- get_max_dist_ref_points(), get_numa_assoc_size() and get_associativity()
now checks for OV5_FORM2_AFFINITY and returns FORM2 values if the guest
selected FORM2 affinity during CAS.

Signed-off-by: Daniel Henrique Barboza 
---
 hw/ppc/spapr.c  |   8 ++
 hw/ppc/spapr_numa.c | 148 +++-
 include/hw/ppc/spapr.h  |   9 +++
 include/hw/ppc/spapr_ovec.h |   1 +
 4 files changed, 164 insertions(+), 2 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index ada85ee083..babb662845 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2752,6 +2752,11 @@ static void spapr_machine_init(MachineState *machine)
 
 spapr_ovec_set(spapr->ov5, OV5_FORM1_AFFINITY);
 
+/* Do not advertise FORM2 NUMA support for pseries-6.1 and older */
+if (!smc->pre_6_2_numa_affinity) {
+spapr_ovec_set(spapr->ov5, OV5_FORM2_AFFINITY);
+}
+
 /* advertise support for dedicated HP event source to guests */
 if (spapr->use_hotplug_event_source) {
 spapr_ovec_set(spapr->ov5, OV5_HP_EVT);
@@ -4667,8 +4672,11 @@ DEFINE_SPAPR_MACHINE(6_2, "6.2", true);
  */
 static void spapr_machine_6_1_class_options(MachineClass *mc)
 {
+SpaprMachineClass *smc = SPAPR_MACHINE_CLASS(mc);
+
 spapr_machine_6_2_class_options(mc);
 compat_props_add(mc->compat_props, hw_compat_6_1, hw_compat_6_1_len);
+smc->pre_6_2_numa_affinity = true;
 }
 
 DEFINE_SPAPR_MACHINE(6_1, "6.1", false);
diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c
index 3f7f612b03..659513b405 100644
--- a/hw/ppc/spapr_numa.c
+++ b/hw/ppc/spapr_numa.c
@@ -24,7 +24,10 @@
  */
 static int get_max_dist_ref_points(SpaprMachineState *spapr)
 {
-/* No FORM2 affinity implemented yet */
+if (spapr_ovec_test(spapr->ov5_cas, OV5_FORM2_AFFINITY)) {
+return FORM2_DIST_REF_POINTS;
+}
+
 return FORM1_DIST_REF_POINTS;
 }
 
@@ -33,7 +36,10 @@ static int get_max_dist_ref_points(SpaprMachineState *spapr)
  */
 static int get_numa_assoc_size(SpaprMachineState *spapr)
 {
-/* No FORM2 affinity implemented yet */
+if (spapr_ovec_test(spapr->ov5_cas, OV5_FORM2_AFFINITY)) {
+return FORM2_NUMA_ASSOC_SIZE;
+}
+
 return FORM1_NUMA_ASSOC_SIZE;
 }
 
@@ -54,6 +60,9 @@ static int get_vcpu_assoc_size(SpaprMachineState *spapr)
  */
 static uint32_t *get_associativity(SpaprMachineState *spapr, int node_id)
 {
+if (spapr_ovec_test(spapr->ov5_cas, OV5_FORM2_AFFINITY)) {
+return spapr->FORM2_assoc_array[node_id];
+}
 return spapr->FORM1_assoc_array[node_id];
 }
 
@@ -297,14 +306,50 @@ static void 
spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr,
 spapr_numa_define_FORM1_domains(spapr);
 }
 
+/*
+ * Init NUMA FORM2 machine state data
+ */
+static void spapr_numa_FORM2_affinity_init(SpaprMachineState *spapr)
+{
+int i;
+
+/*
+ * For all resources but CPUs, FORM2 associativity arrays will
+ * be a size 2 array with the following format:
+ *
+ * ibm,associativity = {1, numa_id}
+ *
+ * CPUs will write an additional 'vcpu_id' on top of the arrays
+ * being initialized here. 'numa_id' is represented by the
+ * index 'i' of the loop.
+ *
+ * Given that this initialization is also valid for GPU associativity
+ * arrays, handle everything in one single step by pop

[PATCH v7 4/7] spapr_numa.c: rename numa_assoc_array to FORM1_assoc_array

[Daniel Henrique Barboza]

Introducing a new NUMA affinity, FORM2, requires a new mechanism to
switch between affinity modes after CAS. Also, we want FORM2 data
structures and functions to be completely separated from the existing
FORM1 code, allowing us to avoid adding new code that inherits the
existing complexity of FORM1.

The idea of switching values used by the write_dt() functions in
spapr_numa.c was already introduced in the previous patch, and
the same approach will be used when dealing with the FORM1 and FORM2
arrays.

We can accomplish that by that by renaming the existing numa_assoc_array
to FORM1_assoc_array, which now is used exclusively to handle FORM1 affinity
data. A new helper get_associativity() is then introduced to be used by the
write_dt() functions to retrieve the current ibm,associativity array of
a given node, after considering affinity selection that might have been
done during CAS. All code that was using numa_assoc_array now needs to
retrieve the array by calling this function.

This will allow for an easier plug of FORM2 data later on.

Signed-off-by: Daniel Henrique Barboza 
---
 hw/ppc/spapr_hcall.c   |  1 +
 hw/ppc/spapr_numa.c| 38 +-
 include/hw/ppc/spapr.h |  2 +-
 3 files changed, 27 insertions(+), 14 deletions(-)

diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c
index 0e9a5b2e40..9056644890 100644
--- a/hw/ppc/spapr_hcall.c
+++ b/hw/ppc/spapr_hcall.c
@@ -17,6 +17,7 @@
 #include "kvm_ppc.h"
 #include "hw/ppc/fdt.h"
 #include "hw/ppc/spapr_ovec.h"
+#include "hw/ppc/spapr_numa.h"
 #include "mmu-book3s-v3.h"
 #include "hw/mem/memory-device.h"
 
diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c
index 1755c9d772..39f9a73429 100644
--- a/hw/ppc/spapr_numa.c
+++ b/hw/ppc/spapr_numa.c
@@ -48,6 +48,15 @@ static int get_vcpu_assoc_size(SpaprMachineState *spapr)
 return get_numa_assoc_size(spapr) + 1;
 }
 
+/*
+ * Retrieves the ibm,associativity array of NUMA node 'node_id'
+ * for the current NUMA affinity.
+ */
+static uint32_t *get_associativity(SpaprMachineState *spapr, int node_id)
+{
+return spapr->FORM1_assoc_array[node_id];
+}
+
 static bool spapr_numa_is_symmetrical(MachineState *ms)
 {
 int src, dst;
@@ -126,7 +135,7 @@ static void 
spapr_numa_define_FORM1_domains(SpaprMachineState *spapr)
  */
 for (i = 1; i < nb_numa_nodes; i++) {
 for (j = 1; j < FORM1_DIST_REF_POINTS; j++) {
-spapr->numa_assoc_array[i][j] = cpu_to_be32(i);
+spapr->FORM1_assoc_array[i][j] = cpu_to_be32(i);
 }
 }
 
@@ -178,8 +187,8 @@ static void 
spapr_numa_define_FORM1_domains(SpaprMachineState *spapr)
  * and going up to 0x1.
  */
 for (i = n_level; i > 0; i--) {
-assoc_src = spapr->numa_assoc_array[src][i];
-spapr->numa_assoc_array[dst][i] = assoc_src;
+assoc_src = spapr->FORM1_assoc_array[src][i];
+spapr->FORM1_assoc_array[dst][i] = assoc_src;
 }
 }
 }
@@ -206,8 +215,8 @@ static void 
spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr,
  * 'i' will be a valid node_id set by the user.
  */
 for (i = 0; i < nb_numa_nodes; i++) {
-spapr->numa_assoc_array[i][0] = cpu_to_be32(FORM1_DIST_REF_POINTS);
-spapr->numa_assoc_array[i][FORM1_DIST_REF_POINTS] = cpu_to_be32(i);
+spapr->FORM1_assoc_array[i][0] = cpu_to_be32(FORM1_DIST_REF_POINTS);
+spapr->FORM1_assoc_array[i][FORM1_DIST_REF_POINTS] = cpu_to_be32(i);
 }
 
 /*
@@ -221,15 +230,15 @@ static void 
spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr,
 max_nodes_with_gpus = nb_numa_nodes + NVGPU_MAX_NUM;
 
 for (i = nb_numa_nodes; i < max_nodes_with_gpus; i++) {
-spapr->numa_assoc_array[i][0] = cpu_to_be32(FORM1_DIST_REF_POINTS);
+spapr->FORM1_assoc_array[i][0] = cpu_to_be32(FORM1_DIST_REF_POINTS);
 
 for (j = 1; j < FORM1_DIST_REF_POINTS; j++) {
 uint32_t gpu_assoc = smc->pre_5_1_assoc_refpoints ?
  SPAPR_GPU_NUMA_ID : cpu_to_be32(i);
-spapr->numa_assoc_array[i][j] = gpu_assoc;
+spapr->FORM1_assoc_array[i][j] = gpu_assoc;
 }
 
-spapr->numa_assoc_array[i][FORM1_DIST_REF_POINTS] = cpu_to_be32(i);
+spapr->FORM1_assoc_array[i][FORM1_DIST_REF_POINTS] = cpu_to_be32(i);
 }
 
 /*
@@ -261,8 +270,10 @@ void spapr_numa_associativity_init(SpaprMachineState 
*spapr,
 void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt,
int offset, int nodeid)
 {
+uint32_t *associativity = get_associativity(spapr, nodeid);
+
 _FDT((fdt_setprop(fdt, offset, "ibm,associativity",
-  spapr->numa_assoc_array[nodeid],
+  associativity,
   get_numa_assoc_size(spapr) * sizeof(uint32_t;
 }
 
@@ -272,6 +283,7 @@ static uint32_t 
*spapr_numa_get_vcpu_assoc(SpaprMac

[PATCH v7 7/7] spapr_numa.c: handle auto NUMA node with no distance info

[Daniel Henrique Barboza]

numa_complete_configuration() in hw/core/numa.c always adds a NUMA node
for the pSeries machine if none was specified, but without node distance
information for the single node created. This added node is also not
accounted for in numa_state->num_nodes, which returns zero.

NUMA FORM1 affinity code didn't rely on numa_state information to do its
job, but FORM2 does. As is now, this is the result of a pSeries guest
with NUMA FORM2 affinity when no NUMA nodes is specified:

$ numactl -H available: 1 nodes (0) node 0 cpus: 0 node 0 size: 16222 MB
node 0 free: 15681 MB No distance information available.

This can be amended in spapr_numa_FORM2_write_rtas_tables(). We're
always expecting at least one NUMA node, and we're going to enforce that
the local distance (the distance to the node to itself) is always 10.
This allows for the proper creation of the NUMA distance tables, fixing
the output of 'numactl -H' in the guest:

$ numactl -H available: 1 nodes (0) node 0 cpus: 0 node 0 size: 16222 MB
node 0 free: 15685 MB node distances: node   0
  0:  10

CC: Igor Mammedov 
Signed-off-by: Daniel Henrique Barboza 
---

Igor,

CCing you as a FYI. I wasn't sure whether there is a reason for
numa_complete_configuration() not adding distance info an update 'num_nodes'
for the auto-generated NUMA node, I decided to handle this case in
pseries side instead.



 hw/ppc/spapr_numa.c | 13 -
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c
index 659513b405..d8caf5f6bd 100644
--- a/hw/ppc/spapr_numa.c
+++ b/hw/ppc/spapr_numa.c
@@ -499,7 +499,7 @@ static void 
spapr_numa_FORM2_write_rtas_tables(SpaprMachineState *spapr,
 {
 MachineState *ms = MACHINE(spapr);
 NodeInfo *numa_info = ms->numa_state->nodes;
-int nb_numa_nodes = ms->numa_state->num_nodes;
+int nb_numa_nodes = ms->numa_state->num_nodes ?: 1;
 int distance_table_entries = nb_numa_nodes * nb_numa_nodes;
 g_autofree uint32_t *lookup_index_table = NULL;
 g_autofree uint32_t *distance_table = NULL;
@@ -539,6 +539,17 @@ static void 
spapr_numa_FORM2_write_rtas_tables(SpaprMachineState *spapr,
 
 for (src = 0; src < nb_numa_nodes; src++) {
 for (dst = 0; dst < nb_numa_nodes; dst++) {
+/*
+ * We need to be explicit with the local distance
+ * value to cover the case where the user didn't added any
+ * NUMA nodes, but QEMU adds the default NUMA node without
+ * adding the numa_info to retrieve the info from.
+ */
+if (src == dst) {
+node_distances[i++] = 10;
+continue;
+}
+
 node_distances[i++] = numa_info[src].distance[dst];
 }
 }
-- 
2.31.1

[PATCH v7 5/7] spapr: move FORM1 verifications to post CAS

[Daniel Henrique Barboza]

FORM2 NUMA affinity is prepared to deal with empty (memory/cpu less)
NUMA nodes. This is used by the DAX KMEM driver to locate a PAPR SCM
device that has a different latency than the original NUMA node from the
regular memory. FORM2 is also able  to deal with asymmetric NUMA
distances gracefully, something that our FORM1 implementation doesn't
do.

Move these FORM1 verifications to a new function and wait until after
CAS, when we're sure that we're sticking with FORM1, to enforce them.

Signed-off-by: Daniel Henrique Barboza 
---
 hw/ppc/spapr.c  | 33 ---
 hw/ppc/spapr_hcall.c|  6 +
 hw/ppc/spapr_numa.c | 53 -
 include/hw/ppc/spapr_numa.h |  1 +
 4 files changed, 54 insertions(+), 39 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index d39fd4e644..ada85ee083 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2773,39 +2773,6 @@ static void spapr_machine_init(MachineState *machine)
 /* init CPUs */
 spapr_init_cpus(spapr);
 
-/*
- * check we don't have a memory-less/cpu-less NUMA node
- * Firmware relies on the existing memory/cpu topology to provide the
- * NUMA topology to the kernel.
- * And the linux kernel needs to know the NUMA topology at start
- * to be able to hotplug CPUs later.
- */
-if (machine->numa_state->num_nodes) {
-for (i = 0; i < machine->numa_state->num_nodes; ++i) {
-/* check for memory-less node */
-if (machine->numa_state->nodes[i].node_mem == 0) {
-CPUState *cs;
-int found = 0;
-/* check for cpu-less node */
-CPU_FOREACH(cs) {
-PowerPCCPU *cpu = POWERPC_CPU(cs);
-if (cpu->node_id == i) {
-found = 1;
-break;
-}
-}
-/* memory-less and cpu-less node */
-if (!found) {
-error_report(
-   "Memory-less/cpu-less nodes are not supported (node 
%d)",
- i);
-exit(1);
-}
-}
-}
-
-}
-
 spapr->gpu_numa_id = spapr_numa_initial_nvgpu_numa_id(machine);
 
 /* Init numa_assoc_array */
diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c
index 9056644890..222c1b6bbd 100644
--- a/hw/ppc/spapr_hcall.c
+++ b/hw/ppc/spapr_hcall.c
@@ -1198,6 +1198,12 @@ target_ulong do_client_architecture_support(PowerPCCPU 
*cpu,
 spapr->cas_pre_isa3_guest = !spapr_ovec_test(ov1_guest, OV1_PPC_3_00);
 spapr_ovec_cleanup(ov1_guest);
 
+/*
+ * Check for NUMA affinity conditions now that we know which NUMA
+ * affinity the guest will use.
+ */
+spapr_numa_associativity_check(spapr);
+
 /*
  * Ensure the guest asks for an interrupt mode we support;
  * otherwise terminate the boot.
diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c
index 39f9a73429..3f7f612b03 100644
--- a/hw/ppc/spapr_numa.c
+++ b/hw/ppc/spapr_numa.c
@@ -195,6 +195,48 @@ static void 
spapr_numa_define_FORM1_domains(SpaprMachineState *spapr)
 
 }
 
+static void spapr_numa_FORM1_affinity_check(MachineState *machine)
+{
+int i;
+
+/*
+ * Check we don't have a memory-less/cpu-less NUMA node
+ * Firmware relies on the existing memory/cpu topology to provide the
+ * NUMA topology to the kernel.
+ * And the linux kernel needs to know the NUMA topology at start
+ * to be able to hotplug CPUs later.
+ */
+if (machine->numa_state->num_nodes) {
+for (i = 0; i < machine->numa_state->num_nodes; ++i) {
+/* check for memory-less node */
+if (machine->numa_state->nodes[i].node_mem == 0) {
+CPUState *cs;
+int found = 0;
+/* check for cpu-less node */
+CPU_FOREACH(cs) {
+PowerPCCPU *cpu = POWERPC_CPU(cs);
+if (cpu->node_id == i) {
+found = 1;
+break;
+}
+}
+/* memory-less and cpu-less node */
+if (!found) {
+error_report(
+"Memory-less/cpu-less nodes are not supported with FORM1 NUMA (node %d)", i);
+exit(EXIT_FAILURE);
+}
+}
+}
+}
+
+if (!spapr_numa_is_symmetrical(machine)) {
+error_report(
+"Asymmetrical NUMA topologies aren't supported in the pSeries machine using 
FORM1 NUMA");
+exit(EXIT_FAILURE);
+}
+}
+
 /*
  * Set NUMA machine state data based on FORM1 affinity semantics.
  */
@@ -252,12 +294,6 @@ static void 
spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr,
 return;
 }
 
-if (!spapr_numa_is_symmetrical(machine)) {
-error_report("Asymmetrical NUMA topologies aren't supported "
-  

[PATCH v7 3/7] spapr_numa.c: parametrize FORM1 macros

[Daniel Henrique Barboza]

The next preliminary step to introduce NUMA FORM2 affinity is to make
the existing code independent of FORM1 macros and values, i.e.
MAX_DISTANCE_REF_POINTS, NUMA_ASSOC_SIZE and VCPU_ASSOC_SIZE. This patch
accomplishes that by doing the following:

- move the NUMA related macros from spapr.h to spapr_numa.c where they
are used. spapr.h gets instead a 'NUMA_NODES_MAX_NUM' macro that is used
to refer to the maximum number of NUMA nodes, including GPU nodes, that
the machine can support;

- MAX_DISTANCE_REF_POINTS and NUMA_ASSOC_SIZE are renamed to
FORM1_DIST_REF_POINTS and FORM1_NUMA_ASSOC_SIZE. These FORM1 specific
macros are used in FORM1 init functions;

- code that uses MAX_DISTANCE_REF_POINTS now retrieves the
max_dist_ref_points value using get_max_dist_ref_points().
NUMA_ASSOC_SIZE is replaced by get_numa_assoc_size() and VCPU_ASSOC_SIZE
is replaced by get_vcpu_assoc_size(). These functions are used by the
generic device tree functions and h_home_node_associativity() and will
allow them to switch between FORM1 and FORM2 without changing their core
logic.

Signed-off-by: Daniel Henrique Barboza 
---
 hw/ppc/spapr_numa.c| 76 ++
 include/hw/ppc/spapr.h | 28 
 2 files changed, 69 insertions(+), 35 deletions(-)

diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c
index bf520d42b2..1755c9d772 100644
--- a/hw/ppc/spapr_numa.c
+++ b/hw/ppc/spapr_numa.c
@@ -19,6 +19,35 @@
 /* Moved from hw/ppc/spapr_pci_nvlink2.c */
 #define SPAPR_GPU_NUMA_ID   (cpu_to_be32(1))
 
+/*
+ * Retrieves max_dist_ref_points of the current NUMA affinity.
+ */
+static int get_max_dist_ref_points(SpaprMachineState *spapr)
+{
+/* No FORM2 affinity implemented yet */
+return FORM1_DIST_REF_POINTS;
+}
+
+/*
+ * Retrieves numa_assoc_size of the current NUMA affinity.
+ */
+static int get_numa_assoc_size(SpaprMachineState *spapr)
+{
+/* No FORM2 affinity implemented yet */
+return FORM1_NUMA_ASSOC_SIZE;
+}
+
+/*
+ * Retrieves vcpu_assoc_size of the current NUMA affinity.
+ *
+ * vcpu_assoc_size is the size of ibm,associativity array
+ * for CPUs, which has an extra element (vcpu_id) in the end.
+ */
+static int get_vcpu_assoc_size(SpaprMachineState *spapr)
+{
+return get_numa_assoc_size(spapr) + 1;
+}
+
 static bool spapr_numa_is_symmetrical(MachineState *ms)
 {
 int src, dst;
@@ -96,7 +125,7 @@ static void 
spapr_numa_define_FORM1_domains(SpaprMachineState *spapr)
  * considered a match with associativity domains of node 0.
  */
 for (i = 1; i < nb_numa_nodes; i++) {
-for (j = 1; j < MAX_DISTANCE_REF_POINTS; j++) {
+for (j = 1; j < FORM1_DIST_REF_POINTS; j++) {
 spapr->numa_assoc_array[i][j] = cpu_to_be32(i);
 }
 }
@@ -134,7 +163,7 @@ static void 
spapr_numa_define_FORM1_domains(SpaprMachineState *spapr)
  *
  * The Linux kernel will assume that the distance between src and
  * dst, in this case of no match, is 10 (local distance) doubled
- * for each NUMA it didn't match. We have MAX_DISTANCE_REF_POINTS
+ * for each NUMA it didn't match. We have FORM1_DIST_REF_POINTS
  * levels (4), so this gives us 10*2*2*2*2 = 160.
  *
  * This logic can be seen in the Linux kernel source code, as of
@@ -169,7 +198,7 @@ static void 
spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr,
 
 /*
  * For all associativity arrays: first position is the size,
- * position MAX_DISTANCE_REF_POINTS is always the numa_id,
+ * position FORM1_DIST_REF_POINTS is always the numa_id,
  * represented by the index 'i'.
  *
  * This will break on sparse NUMA setups, when/if QEMU starts
@@ -177,8 +206,8 @@ static void 
spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr,
  * 'i' will be a valid node_id set by the user.
  */
 for (i = 0; i < nb_numa_nodes; i++) {
-spapr->numa_assoc_array[i][0] = cpu_to_be32(MAX_DISTANCE_REF_POINTS);
-spapr->numa_assoc_array[i][MAX_DISTANCE_REF_POINTS] = cpu_to_be32(i);
+spapr->numa_assoc_array[i][0] = cpu_to_be32(FORM1_DIST_REF_POINTS);
+spapr->numa_assoc_array[i][FORM1_DIST_REF_POINTS] = cpu_to_be32(i);
 }
 
 /*
@@ -192,15 +221,15 @@ static void 
spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr,
 max_nodes_with_gpus = nb_numa_nodes + NVGPU_MAX_NUM;
 
 for (i = nb_numa_nodes; i < max_nodes_with_gpus; i++) {
-spapr->numa_assoc_array[i][0] = cpu_to_be32(MAX_DISTANCE_REF_POINTS);
+spapr->numa_assoc_array[i][0] = cpu_to_be32(FORM1_DIST_REF_POINTS);
 
-for (j = 1; j < MAX_DISTANCE_REF_POINTS; j++) {
+for (j = 1; j < FORM1_DIST_REF_POINTS; j++) {
 uint32_t gpu_assoc = smc->pre_5_1_assoc_refpoints ?
  SPAPR_GPU_NUMA_ID : cpu_to_be32(i);
 spapr->numa_assoc_array[i][j] = gpu_assoc;
 }
 
-spapr->numa_assoc_arra

Re: [PATCH v6 3/6] spapr: introduce spapr_numa_associativity_reset()

[Daniel Henrique Barboza]

Greg,


On 9/14/21 16:58, Daniel Henrique Barboza wrote:



On 9/14/21 08:55, Greg Kurz wrote:

On Fri, 10 Sep 2021 16:55:36 -0300
Daniel Henrique Barboza  wrote:



[...]



  }
@@ -167,6 +167,11 @@ static void 
spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr,
  int nb_numa_nodes = machine->numa_state->num_nodes;
  int i, j, max_nodes_with_gpus;
+    /* init FORM1_assoc_array */
+    for (i = 0; i < MAX_NODES + NVGPU_MAX_NUM; i++) {
+    spapr->FORM1_assoc_array[i] = g_new0(uint32_t, NUMA_ASSOC_SIZE);


Why dynamic allocation since you have fixed size ?


If I use static allocation the C compiler complains that I can't assign a
uint32_t** pointer to a uint32_t[MAX_NODES + NVGPU_MAX_NUM][NUMA_ASSOC_SIZE]
array type.

And given that the FORM2 array is a [MAX_NODES + NVGPU_MAX_NUM][2] array, the
way I worked around that here is to use dynamic allocation. Then C considers 
valid
to use numa_assoc_array as an uint32_t** pointer for both FORM1 and FORM2
2D arrays. I'm fairly certain that there might be a way of doing static 
allocation
and keeping the uint32_t** pointer as is, but didn't find any. Tips welcome :D

An alternative that I considered, without the need for this dynamic allocation 
hack,
is to make both FORM1 and FORM2 data structures the same size (i.e.
an [MAX_NODES + NVGPU_MAX_NUM][NUMA_ASSOC_SIZE] uint32_t array) and then 
numa_assoc_array
can be a pointer of the same array type for both. Since we're controlling FORM1 
and
FORM2 sizes separately inside the functions this would work. The downside is 
that
FORM2 2D array would be bigger than necessary.


I took a look and didn't find any neat way of doing a pointer switch
between 2 static arrays without either allocating dynamic mem on the
pointer and then g_memdup'ing it, or make all arrays the same size
(i.e. numa_assoc_array would also be a statically allocated array
with FORM1 size) and then memcpy() the chosen mode.

I just posted a new version in which I'm not relying on 'numa_assoc_array'.
Instead, the DT functions will use a get_associativity() to retrieve
the current associativity array based on CAS choice, in a similar
manner like it is already done with the array sizes. This also allowed
us to get rid of associativity_reset().


Thanks,


Daniel






I don't have strong opinions about which way to do it, so I'm all ears.


Thanks,


Daniel






+    }
+
  /*
   * For all associativity arrays: first position is the size,
   * position MAX_DISTANCE_REF_POINTS is always the numa_id,
@@ -177,8 +182,8 @@ static void 
spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr,
   * 'i' will be a valid node_id set by the user.
   */
  for (i = 0; i < nb_numa_nodes; i++) {
-    spapr->numa_assoc_array[i][0] = cpu_to_be32(MAX_DISTANCE_REF_POINTS);
-    spapr->numa_assoc_array[i][MAX_DISTANCE_REF_POINTS] = cpu_to_be32(i);
+    spapr->FORM1_assoc_array[i][0] = cpu_to_be32(MAX_DISTANCE_REF_POINTS);
+    spapr->FORM1_assoc_array[i][MAX_DISTANCE_REF_POINTS] = cpu_to_be32(i);
  }
  /*
@@ -192,15 +197,15 @@ static void 
spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr,
  max_nodes_with_gpus = nb_numa_nodes + NVGPU_MAX_NUM;
  for (i = nb_numa_nodes; i < max_nodes_with_gpus; i++) {
-    spapr->numa_assoc_array[i][0] = cpu_to_be32(MAX_DISTANCE_REF_POINTS);
+    spapr->FORM1_assoc_array[i][0] = cpu_to_be32(MAX_DISTANCE_REF_POINTS);
  for (j = 1; j < MAX_DISTANCE_REF_POINTS; j++) {
  uint32_t gpu_assoc = smc->pre_5_1_assoc_refpoints ?
   SPAPR_GPU_NUMA_ID : cpu_to_be32(i);
-    spapr->numa_assoc_array[i][j] = gpu_assoc;
+    spapr->FORM1_assoc_array[i][j] = gpu_assoc;
  }
-    spapr->numa_assoc_array[i][MAX_DISTANCE_REF_POINTS] = cpu_to_be32(i);
+    spapr->FORM1_assoc_array[i][MAX_DISTANCE_REF_POINTS] = cpu_to_be32(i);
  }
  /*
@@ -227,14 +232,33 @@ void spapr_numa_associativity_init(SpaprMachineState 
*spapr,
 MachineState *machine)
  {
  spapr_numa_FORM1_affinity_init(spapr, machine);
+
+    /*
+ * Default to FORM1 affinity until CAS. We'll call affinity_reset()
+ * during CAS when we're sure about which NUMA affinity the guest
+ * is going to use.
+ *
+ * This step is a failsafe - guests in the wild were able to read
+ * FORM1 affinity info before CAS for a long time. Since affinity_reset()
+ * is just a pointer switch between data that was already populated
+ * here, this is an acceptable overhead to be on the safer side.
+ */
+    spapr->numa_assoc_array = spapr->FORM1_assoc_array;


The right way to do that is to call spapr_numa_associativity_reset() from
spapr_machine_reset() because we want to revert to FORM1 each time the
guest is rebooted.


+}
+
+void spapr_numa_associativity_reset(SpaprMachineState *spapr)
+{
+    /* No FORM2 affinity implemented yet */


This seems quite obvious at this

[PATCH v7 1/7] spapr_numa.c: split FORM1 code into helpers

[Daniel Henrique Barboza]

The upcoming FORM2 NUMA affinity will support asymmetric NUMA topologies
and doesn't need be concerned with all the legacy support for older
pseries FORM1 guests.

We're also not going to calculate associativity domains based on numa
distance (via spapr_numa_define_associativity_domains) since the
distances will be written directly into new DT properties.

Let's split FORM1 code into its own functions to allow for easier
insertion of FORM2 logic later on.

Reviewed-by: Greg Kurz 
Signed-off-by: Daniel Henrique Barboza 
---
 hw/ppc/spapr_numa.c | 35 +--
 1 file changed, 25 insertions(+), 10 deletions(-)

diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c
index 779f18b994..786def7c73 100644
--- a/hw/ppc/spapr_numa.c
+++ b/hw/ppc/spapr_numa.c
@@ -92,7 +92,7 @@ static uint8_t spapr_numa_get_numa_level(uint8_t distance)
 return 0;
 }
 
-static void spapr_numa_define_associativity_domains(SpaprMachineState *spapr)
+static void spapr_numa_define_FORM1_domains(SpaprMachineState *spapr)
 {
 MachineState *ms = MACHINE(spapr);
 NodeInfo *numa_info = ms->numa_state->nodes;
@@ -155,8 +155,11 @@ static void 
spapr_numa_define_associativity_domains(SpaprMachineState *spapr)
 
 }
 
-void spapr_numa_associativity_init(SpaprMachineState *spapr,
-   MachineState *machine)
+/*
+ * Set NUMA machine state data based on FORM1 affinity semantics.
+ */
+static void spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr,
+   MachineState *machine)
 {
 SpaprMachineClass *smc = SPAPR_MACHINE_GET_CLASS(spapr);
 int nb_numa_nodes = machine->numa_state->num_nodes;
@@ -225,7 +228,13 @@ void spapr_numa_associativity_init(SpaprMachineState 
*spapr,
 exit(EXIT_FAILURE);
 }
 
-spapr_numa_define_associativity_domains(spapr);
+spapr_numa_define_FORM1_domains(spapr);
+}
+
+void spapr_numa_associativity_init(SpaprMachineState *spapr,
+   MachineState *machine)
+{
+spapr_numa_FORM1_affinity_init(spapr, machine);
 }
 
 void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt,
@@ -302,12 +311,8 @@ int spapr_numa_write_assoc_lookup_arrays(SpaprMachineState 
*spapr, void *fdt,
 return ret;
 }
 
-/*
- * Helper that writes ibm,associativity-reference-points and
- * max-associativity-domains in the RTAS pointed by @rtas
- * in the DT @fdt.
- */
-void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rtas)
+static void spapr_numa_FORM1_write_rtas_dt(SpaprMachineState *spapr,
+   void *fdt, int rtas)
 {
 MachineState *ms = MACHINE(spapr);
 SpaprMachineClass *smc = SPAPR_MACHINE_GET_CLASS(spapr);
@@ -365,6 +370,16 @@ void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, 
void *fdt, int rtas)
  maxdomains, sizeof(maxdomains)));
 }
 
+/*
+ * Helper that writes ibm,associativity-reference-points and
+ * max-associativity-domains in the RTAS pointed by @rtas
+ * in the DT @fdt.
+ */
+void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rtas)
+{
+spapr_numa_FORM1_write_rtas_dt(spapr, fdt, rtas);
+}
+
 static target_ulong h_home_node_associativity(PowerPCCPU *cpu,
   SpaprMachineState *spapr,
   target_ulong opcode,
-- 
2.31.1

[PATCH v7 2/7] spapr_numa.c: scrap 'legacy_numa' concept

[Daniel Henrique Barboza]

When first introduced, 'legacy_numa' was a way to refer to guests that
either wouldn't be affected by associativity domain calculations, namely
the ones with only 1 NUMA node, and pre 5.2 guests that shouldn't be
affected by it because it would be an userspace change. Calling these
cases 'legacy_numa' was a convenient way to label these cases.

We're about to introduce a new NUMA affinity, FORM2, and this concept
of 'legacy_numa' is now a bit misleading because, although it is called
'legacy' it is in fact a FORM1 exclusive contraint.

This patch removes spapr_machine_using_legacy_numa() and open code the
conditions in each caller. While we're at it, move the chunk inside
spapr_numa_FORM1_affinity_init() that sets all numa_assoc_array domains
with 'node_id' to spapr_numa_define_FORM1_domains(). This chunk was
being executed if !pre_5_2_numa_associativity and num_nodes => 1, the
same conditions in which spapr_numa_define_FORM1_domains() is called
shortly after.

Reviewed-by: Greg Kurz 
Signed-off-by: Daniel Henrique Barboza 
---
 hw/ppc/spapr_numa.c | 47 +++--
 1 file changed, 20 insertions(+), 27 deletions(-)

diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c
index 786def7c73..bf520d42b2 100644
--- a/hw/ppc/spapr_numa.c
+++ b/hw/ppc/spapr_numa.c
@@ -19,15 +19,6 @@
 /* Moved from hw/ppc/spapr_pci_nvlink2.c */
 #define SPAPR_GPU_NUMA_ID   (cpu_to_be32(1))
 
-static bool spapr_machine_using_legacy_numa(SpaprMachineState *spapr)
-{
-MachineState *machine = MACHINE(spapr);
-SpaprMachineClass *smc = SPAPR_MACHINE_GET_CLASS(machine);
-
-return smc->pre_5_2_numa_associativity ||
-   machine->numa_state->num_nodes <= 1;
-}
-
 static bool spapr_numa_is_symmetrical(MachineState *ms)
 {
 int src, dst;
@@ -97,7 +88,18 @@ static void 
spapr_numa_define_FORM1_domains(SpaprMachineState *spapr)
 MachineState *ms = MACHINE(spapr);
 NodeInfo *numa_info = ms->numa_state->nodes;
 int nb_numa_nodes = ms->numa_state->num_nodes;
-int src, dst, i;
+int src, dst, i, j;
+
+/*
+ * Fill all associativity domains of non-zero NUMA nodes with
+ * node_id. This is required because the default value (0) is
+ * considered a match with associativity domains of node 0.
+ */
+for (i = 1; i < nb_numa_nodes; i++) {
+for (j = 1; j < MAX_DISTANCE_REF_POINTS; j++) {
+spapr->numa_assoc_array[i][j] = cpu_to_be32(i);
+}
+}
 
 for (src = 0; src < nb_numa_nodes; src++) {
 for (dst = src; dst < nb_numa_nodes; dst++) {
@@ -164,7 +166,6 @@ static void 
spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr,
 SpaprMachineClass *smc = SPAPR_MACHINE_GET_CLASS(spapr);
 int nb_numa_nodes = machine->numa_state->num_nodes;
 int i, j, max_nodes_with_gpus;
-bool using_legacy_numa = spapr_machine_using_legacy_numa(spapr);
 
 /*
  * For all associativity arrays: first position is the size,
@@ -178,17 +179,6 @@ static void 
spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr,
 for (i = 0; i < nb_numa_nodes; i++) {
 spapr->numa_assoc_array[i][0] = cpu_to_be32(MAX_DISTANCE_REF_POINTS);
 spapr->numa_assoc_array[i][MAX_DISTANCE_REF_POINTS] = cpu_to_be32(i);
-
-/*
- * Fill all associativity domains of non-zero NUMA nodes with
- * node_id. This is required because the default value (0) is
- * considered a match with associativity domains of node 0.
- */
-if (!using_legacy_numa && i != 0) {
-for (j = 1; j < MAX_DISTANCE_REF_POINTS; j++) {
-spapr->numa_assoc_array[i][j] = cpu_to_be32(i);
-}
-}
 }
 
 /*
@@ -214,11 +204,13 @@ static void 
spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr,
 }
 
 /*
- * Legacy NUMA guests (pseries-5.1 and older, or guests with only
- * 1 NUMA node) will not benefit from anything we're going to do
- * after this point.
+ * Guests pseries-5.1 and older uses zeroed associativity domains,
+ * i.e. no domain definition based on NUMA distance input.
+ *
+ * Same thing with guests that have only one NUMA node.
  */
-if (using_legacy_numa) {
+if (smc->pre_5_2_numa_associativity ||
+machine->numa_state->num_nodes <= 1) {
 return;
 }
 
@@ -334,7 +326,8 @@ static void 
spapr_numa_FORM1_write_rtas_dt(SpaprMachineState *spapr,
 cpu_to_be32(maxdomain)
 };
 
-if (spapr_machine_using_legacy_numa(spapr)) {
+if (smc->pre_5_2_numa_associativity ||
+ms->numa_state->num_nodes <= 1) {
 uint32_t legacy_refpoints[] = {
 cpu_to_be32(0x4),
 cpu_to_be32(0x4),
-- 
2.31.1

[PATCH v7 0/7] pSeries FORM2 affinity support

[Daniel Henrique Barboza]

Hi,

In this version the biggest change is the end of the numa_assoc_array
struct in the machine state. The write_dt() functions are now retrieving
the current NUMA associativity array by checking CAS first.

This change allowed for several simplifications, e.g. we don't need
a reset() function to be called after CAS or in spapr_post_load().
Arrays are now statically allocated.

I also catched a last minute bug during my tests. See patch 7 for more
info.

Changes from v6:
- patch 1:
  * added Greg's r-b
- patch 2:
  * added the missing NUMA nodes number check
  * added Greg's r-b
- patch 3 (former patch 4):
  * no changes.
- former patch 3 (associativity_reset()): dropped
- patch 4 (new):
  * added get_associativity()
  * do not allocate FORM1_assoc_array in the heap
- patch 5:
  * fixed typo
  * added new check function to be called in CAS
- patch 6:
  * do not allocate FORM2_assoc_array in the heap
- patch 7 (new):
  * FORM2 fixes to handle the implicit added QEMU NUMA node when there's
  no NUMA node added by the user.
- v6 link: https://lists.gnu.org/archive/html/qemu-devel/2021-09/msg02892.html


Daniel Henrique Barboza (7):
  spapr_numa.c: split FORM1 code into helpers
  spapr_numa.c: scrap 'legacy_numa' concept
  spapr_numa.c: parametrize FORM1 macros
  spapr_numa.c: rename numa_assoc_array to FORM1_assoc_array
  spapr: move FORM1 verifications to post CAS
  spapr_numa.c: FORM2 NUMA affinity support
  spapr_numa.c: handle auto NUMA node with no distance info

 hw/ppc/spapr.c  |  41 +---
 hw/ppc/spapr_hcall.c|   7 +
 hw/ppc/spapr_numa.c | 380 ++--
 include/hw/ppc/spapr.h  |  35 ++--
 include/hw/ppc/spapr_numa.h |   1 +
 include/hw/ppc/spapr_ovec.h |   1 +
 6 files changed, 354 insertions(+), 111 deletions(-)

-- 
2.31.1

Re: [qemu-web PATCH] Gemfile: Add webrick bundle dependency

[Paolo Bonzini]

On 14/09/21 10:21, Daniel P. Berrangé wrote:

Yes, this is needed to fix Ruby 3.

I didn't propose it myself as I'm not sure if it in turn breaks people
with Ruby 2.x ?


Does it pass the GitLab CI (which uses 2.5)?

Paolo

Re: [PATCH] target/i386: Include 'hw/i386/apic.h' locally

[Paolo Bonzini]

On 02/09/21 17:22, Philippe Mathieu-Daudé wrote:

Instead of including a sysemu-specific header in "cpu.h"
(which is shared with user-mode emulations), include it
locally when required.

Signed-off-by: Philippe Mathieu-Daudé 
---
Based-on: <20210902151715.383678-1-f4...@amsat.org>
"accel/tcg: Restrict TCGCPUOps::cpu_exec_interrupt() to sysemu"


Acked-by: Paolo Bonzini 

Paolo


---
  target/i386/cpu.h| 4 
  hw/i386/kvmvapic.c   | 1 +
  hw/i386/x86.c| 1 +
  target/i386/cpu-sysemu.c | 1 +
  target/i386/cpu.c| 1 +
  target/i386/gdbstub.c| 4 
  target/i386/hax/hax-all.c| 1 +
  target/i386/helper.c | 1 +
  target/i386/hvf/x86_emu.c| 1 +
  target/i386/nvmm/nvmm-all.c  | 1 +
  target/i386/tcg/seg_helper.c | 4 
  target/i386/tcg/sysemu/misc_helper.c | 1 +
  target/i386/whpx/whpx-all.c  | 1 +
  13 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index c7cc65e92d5..eddafd1acd5 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -2046,10 +2046,6 @@ typedef X86CPU ArchCPU;
  #include "exec/cpu-all.h"
  #include "svm.h"
  
-#if !defined(CONFIG_USER_ONLY)

-#include "hw/i386/apic.h"
-#endif
-
  static inline void cpu_get_tb_cpu_state(CPUX86State *env, target_ulong *pc,
  target_ulong *cs_base, uint32_t 
*flags)
  {
diff --git a/hw/i386/kvmvapic.c b/hw/i386/kvmvapic.c
index 43f8a8f679e..7333818bdd1 100644
--- a/hw/i386/kvmvapic.c
+++ b/hw/i386/kvmvapic.c
@@ -16,6 +16,7 @@
  #include "sysemu/hw_accel.h"
  #include "sysemu/kvm.h"
  #include "sysemu/runstate.h"
+#include "hw/i386/apic.h"
  #include "hw/i386/apic_internal.h"
  #include "hw/sysbus.h"
  #include "hw/boards.h"
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 00448ed55aa..e0218f8791f 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -43,6 +43,7 @@
  #include "target/i386/cpu.h"
  #include "hw/i386/topology.h"
  #include "hw/i386/fw_cfg.h"
+#include "hw/i386/apic.h"
  #include "hw/intc/i8259.h"
  #include "hw/rtc/mc146818rtc.h"
  
diff --git a/target/i386/cpu-sysemu.c b/target/i386/cpu-sysemu.c

index 1078e3d157f..65709e41903 100644
--- a/target/i386/cpu-sysemu.c
+++ b/target/i386/cpu-sysemu.c
@@ -30,6 +30,7 @@
  #include "hw/qdev-properties.h"
  
  #include "exec/address-spaces.h"

+#include "hw/i386/apic.h"
  #include "hw/i386/apic_internal.h"
  
  #include "cpu-internal.h"

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 97e250e8760..04f59043804 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -33,6 +33,7 @@
  #include "standard-headers/asm-x86/kvm_para.h"
  #include "hw/qdev-properties.h"
  #include "hw/i386/topology.h"
+#include "hw/i386/apic.h"
  #ifndef CONFIG_USER_ONLY
  #include "exec/address-spaces.h"
  #include "hw/boards.h"
diff --git a/target/i386/gdbstub.c b/target/i386/gdbstub.c
index 098a2ad15a9..5438229c1a9 100644
--- a/target/i386/gdbstub.c
+++ b/target/i386/gdbstub.c
@@ -21,6 +21,10 @@
  #include "cpu.h"
  #include "exec/gdbstub.h"
  
+#ifndef CONFIG_USER_ONLY

+#include "hw/i386/apic.h"
+#endif
+
  #ifdef TARGET_X86_64
  static const int gpr_map[16] = {
  R_EAX, R_EBX, R_ECX, R_EDX, R_ESI, R_EDI, R_EBP, R_ESP,
diff --git a/target/i386/hax/hax-all.c b/target/i386/hax/hax-all.c
index bf65ed6fa92..cd89e3233a9 100644
--- a/target/i386/hax/hax-all.c
+++ b/target/i386/hax/hax-all.c
@@ -32,6 +32,7 @@
  #include "sysemu/reset.h"
  #include "sysemu/runstate.h"
  #include "hw/boards.h"
+#include "hw/i386/apic.h"
  
  #include "hax-accel-ops.h"
  
diff --git a/target/i386/helper.c b/target/i386/helper.c

index 533b29cb91b..874beda98ae 100644
--- a/target/i386/helper.c
+++ b/target/i386/helper.c
@@ -26,6 +26,7 @@
  #ifndef CONFIG_USER_ONLY
  #include "sysemu/hw_accel.h"
  #include "monitor/monitor.h"
+#include "hw/i386/apic.h"
  #endif
  
  void cpu_sync_bndcs_hflags(CPUX86State *env)

diff --git a/target/i386/hvf/x86_emu.c b/target/i386/hvf/x86_emu.c
index 7c8203b21fb..fb3e88959d4 100644
--- a/target/i386/hvf/x86_emu.c
+++ b/target/i386/hvf/x86_emu.c
@@ -45,6 +45,7 @@
  #include "x86_flags.h"
  #include "vmcs.h"
  #include "vmx.h"
+#include "hw/i386/apic.h"
  
  void hvf_handle_io(struct CPUState *cpu, uint16_t port, void *data,

 int direction, int size, uint32_t count);
diff --git a/target/i386/nvmm/nvmm-all.c b/target/i386/nvmm/nvmm-all.c
index 28dee4c5ee3..4c07cfc7194 100644
--- a/target/i386/nvmm/nvmm-all.c
+++ b/target/i386/nvmm/nvmm-all.c
@@ -22,6 +22,7 @@
  #include "qemu/queue.h"
  #include "migration/blocker.h"
  #include "strings.h"
+#include "hw/i386/apic.h"
  
  #include "nvmm-accel-ops.h"
  
diff --git a/target/i386/tcg/seg_helper.c b/target/i386/tcg/seg_helper.c

index 13c6e6ee62e..71fd2bf8d06 100644
--- a/target/i386/tcg/seg_helper.c
+++ b/target/i386/tcg/seg_helper.c
@@ -28,6 +28,10 @@
  #include "helper-tcg.h"
  #incl

Re: [PULL 0/4] Update meson version

[Paolo Bonzini]

On 15/09/21 19:34, Peter Maydell wrote:

Paolo Bonzini (4):
   meson: bump submodule to 0.58.2
   meson: switch minimum meson version to 0.58.2
   hexagon: use env keyword argument to pass PYTHONPATH
   target/xtensa: list cores in a text file

Was this intended to be a pull request ? AFAICT patch 4 at least
has not been seen before on the list for review.


It was intended to be a pull request; but it was not intended to include 
previously unsubmitted patches of course.


I have removed patch 4 from the tag.

Paolo

Re: [PATCH] qemu-storage-daemon: Only display FUSE help when FUSE is built-in

[Philippe Mathieu-Daudé]

ping & Cc'ing qemu-trivial@ (reviewed twice) ...

On 8/16/21 8:04 PM, Philippe Mathieu-Daudé wrote:
> When configuring QEMU with --disable-fuse, the qemu-storage-daemon
> still reports FUSE command line options in its help:
> 
>   $ qemu-storage-daemon -h
>   Usage: qemu-storage-daemon [options]
>   QEMU storage daemon
> 
> --export [type=]fuse,id=,node-name=,mountpoint=
>  [,growable=on|off][,writable=on|off]
>export the specified block node over FUSE
> 
> Remove this help message when FUSE is disabled, to avoid:
> 
>   $ qemu-storage-daemon --export fuse
>   qemu-storage-daemon: --export fuse: Invalid parameter 'fuse'
> 
> Reported-by: Qing Wang 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  storage-daemon/qemu-storage-daemon.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/storage-daemon/qemu-storage-daemon.c 
> b/storage-daemon/qemu-storage-daemon.c
> index fc8b1506297..10a1a33761d 100644
> --- a/storage-daemon/qemu-storage-daemon.c
> +++ b/storage-daemon/qemu-storage-daemon.c
> @@ -98,10 +98,12 @@ static void help(void)
>  " export the specified block node over NBD\n"
>  " (requires --nbd-server)\n"
>  "\n"
> +#ifdef CONFIG_FUSE
>  "  --export [type=]fuse,id=,node-name=,mountpoint=\n"
>  "   [,growable=on|off][,writable=on|off]\n"
>  " export the specified block node over FUSE\n"
>  "\n"
> +#endif /* CONFIG_FUSE */
>  "  --monitor [chardev=]name[,mode=control][,pretty[=on|off]]\n"
>  " configure a QMP monitor\n"
>  "\n"
> 

[PATCH v5 13/16] tcg/s390x: Implement TCG_TARGET_HAS_minmax_vec

[Richard Henderson]

Reviewed-by: David Hildenbrand 
Signed-off-by: Richard Henderson 
---
 tcg/s390x/tcg-target.h |  2 +-
 tcg/s390x/tcg-target.c.inc | 25 +
 2 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h
index d7d204b782..a79f4f187a 100644
--- a/tcg/s390x/tcg-target.h
+++ b/tcg/s390x/tcg-target.h
@@ -156,7 +156,7 @@ extern uint64_t s390_facilities[3];
 #define TCG_TARGET_HAS_shv_vec1
 #define TCG_TARGET_HAS_mul_vec1
 #define TCG_TARGET_HAS_sat_vec0
-#define TCG_TARGET_HAS_minmax_vec 0
+#define TCG_TARGET_HAS_minmax_vec 1
 #define TCG_TARGET_HAS_bitsel_vec 0
 #define TCG_TARGET_HAS_cmpsel_vec 0
 
diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
index a56733e09a..c3bd22b116 100644
--- a/tcg/s390x/tcg-target.c.inc
+++ b/tcg/s390x/tcg-target.c.inc
@@ -282,6 +282,10 @@ typedef enum S390Opcode {
 VRRc_VESRAV = 0xe77a,
 VRRc_VESRLV = 0xe778,
 VRRc_VML= 0xe7a2,
+VRRc_VMN= 0xe7fe,
+VRRc_VMNL   = 0xe7fc,
+VRRc_VMX= 0xe7ff,
+VRRc_VMXL   = 0xe7fd,
 VRRc_VN = 0xe768,
 VRRc_VNC= 0xe769,
 VRRc_VNO= 0xe76b,
@@ -2767,6 +2771,19 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc,
 tcg_out_insn(s, VRRc, VERLLV, a0, a1, a2, vece);
 break;
 
+case INDEX_op_smin_vec:
+tcg_out_insn(s, VRRc, VMN, a0, a1, a2, vece);
+break;
+case INDEX_op_smax_vec:
+tcg_out_insn(s, VRRc, VMX, a0, a1, a2, vece);
+break;
+case INDEX_op_umin_vec:
+tcg_out_insn(s, VRRc, VMNL, a0, a1, a2, vece);
+break;
+case INDEX_op_umax_vec:
+tcg_out_insn(s, VRRc, VMXL, a0, a1, a2, vece);
+break;
+
 case INDEX_op_cmp_vec:
 switch ((TCGCond)args[3]) {
 case TCG_COND_EQ:
@@ -2813,7 +2830,11 @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, 
unsigned vece)
 case INDEX_op_shri_vec:
 case INDEX_op_shrs_vec:
 case INDEX_op_shrv_vec:
+case INDEX_op_smax_vec:
+case INDEX_op_smin_vec:
 case INDEX_op_sub_vec:
+case INDEX_op_umax_vec:
+case INDEX_op_umin_vec:
 case INDEX_op_xor_vec:
 return 1;
 case INDEX_op_cmp_vec:
@@ -3074,6 +3095,10 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode 
op)
 case INDEX_op_shlv_vec:
 case INDEX_op_shrv_vec:
 case INDEX_op_sarv_vec:
+case INDEX_op_smax_vec:
+case INDEX_op_smin_vec:
+case INDEX_op_umax_vec:
+case INDEX_op_umin_vec:
 return C_O1_I2(v, v, v);
 case INDEX_op_rotls_vec:
 case INDEX_op_shls_vec:
-- 
2.25.1

[PATCH v5 11/16] tcg/s390x: Implement TCG_TARGET_HAS_mul_vec

[Richard Henderson]

Reviewed-by: David Hildenbrand 
Signed-off-by: Richard Henderson 
---
 tcg/s390x/tcg-target.h | 2 +-
 tcg/s390x/tcg-target.c.inc | 7 +++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h
index a42074e451..1c581a2f60 100644
--- a/tcg/s390x/tcg-target.h
+++ b/tcg/s390x/tcg-target.h
@@ -154,7 +154,7 @@ extern uint64_t s390_facilities[3];
 #define TCG_TARGET_HAS_shi_vec0
 #define TCG_TARGET_HAS_shs_vec0
 #define TCG_TARGET_HAS_shv_vec0
-#define TCG_TARGET_HAS_mul_vec0
+#define TCG_TARGET_HAS_mul_vec1
 #define TCG_TARGET_HAS_sat_vec0
 #define TCG_TARGET_HAS_minmax_vec 0
 #define TCG_TARGET_HAS_bitsel_vec 0
diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
index fa4a142818..aa75d779be 100644
--- a/tcg/s390x/tcg-target.c.inc
+++ b/tcg/s390x/tcg-target.c.inc
@@ -277,6 +277,7 @@ typedef enum S390Opcode {
 VRRc_VCEQ   = 0xe7f8,   /* we leave the m5 cs field 0 */
 VRRc_VCH= 0xe7fb,   /* " */
 VRRc_VCHL   = 0xe7f9,   /* " */
+VRRc_VML= 0xe7a2,
 VRRc_VN = 0xe768,
 VRRc_VNC= 0xe769,
 VRRc_VNO= 0xe76b,
@@ -2696,6 +2697,9 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc,
 case INDEX_op_andc_vec:
 tcg_out_insn(s, VRRc, VNC, a0, a1, a2, 0);
 break;
+case INDEX_op_mul_vec:
+tcg_out_insn(s, VRRc, VML, a0, a1, a2, vece);
+break;
 case INDEX_op_or_vec:
 tcg_out_insn(s, VRRc, VO, a0, a1, a2, 0);
 break;
@@ -2745,6 +2749,8 @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, 
unsigned vece)
 return 1;
 case INDEX_op_cmp_vec:
 return -1;
+case INDEX_op_mul_vec:
+return vece < MO_64;
 default:
 return 0;
 }
@@ -2981,6 +2987,7 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode 
op)
 case INDEX_op_orc_vec:
 case INDEX_op_xor_vec:
 case INDEX_op_cmp_vec:
+case INDEX_op_mul_vec:
 return C_O1_I2(v, v, v);
 
 default:
-- 
2.25.1

[PATCH v5 14/16] tcg/s390x: Implement TCG_TARGET_HAS_sat_vec

[Richard Henderson]

The unsigned saturations are handled via generic code
using min/max.  The signed saturations are expanded using
double-sized arithmetic and a saturating pack.

Since all operations are done via expansion, do not
actually set TCG_TARGET_HAS_sat_vec.

Signed-off-by: Richard Henderson 
---
 tcg/s390x/tcg-target.opc.h |  3 ++
 tcg/s390x/tcg-target.c.inc | 63 ++
 2 files changed, 66 insertions(+)

diff --git a/tcg/s390x/tcg-target.opc.h b/tcg/s390x/tcg-target.opc.h
index 67afc82a93..0eb2350fb3 100644
--- a/tcg/s390x/tcg-target.opc.h
+++ b/tcg/s390x/tcg-target.opc.h
@@ -10,3 +10,6 @@
  * emitted by tcg_expand_vec_op.  For those familiar with GCC internals,
  * consider these to be UNSPEC with names.
  */
+DEF(s390_vuph_vec, 1, 1, 0, IMPLVEC)
+DEF(s390_vupl_vec, 1, 1, 0, IMPLVEC)
+DEF(s390_vpks_vec, 1, 2, 0, IMPLVEC)
diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
index c3bd22b116..4a56532011 100644
--- a/tcg/s390x/tcg-target.c.inc
+++ b/tcg/s390x/tcg-target.c.inc
@@ -291,7 +291,10 @@ typedef enum S390Opcode {
 VRRc_VNO= 0xe76b,
 VRRc_VO = 0xe76a,
 VRRc_VOC= 0xe76f,
+VRRc_VPKS   = 0xe797,   /* we leave the m5 cs field 0 */
 VRRc_VS = 0xe7f7,
+VRRa_VUPH   = 0xe7d7,
+VRRa_VUPL   = 0xe7d6,
 VRRc_VX = 0xe76d,
 VRRf_VLVGP  = 0xe762,
 
@@ -2800,6 +2803,16 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc,
 }
 break;
 
+case INDEX_op_s390_vuph_vec:
+tcg_out_insn(s, VRRa, VUPH, a0, a1, vece);
+break;
+case INDEX_op_s390_vupl_vec:
+tcg_out_insn(s, VRRa, VUPL, a0, a1, vece);
+break;
+case INDEX_op_s390_vpks_vec:
+tcg_out_insn(s, VRRc, VPKS, a0, a1, a2, vece);
+break;
+
 case INDEX_op_mov_vec:   /* Always emitted via tcg_out_mov.  */
 case INDEX_op_dup_vec:   /* Always emitted via tcg_out_dup_vec.  */
 default:
@@ -2842,6 +2855,9 @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, 
unsigned vece)
 return -1;
 case INDEX_op_mul_vec:
 return vece < MO_64;
+case INDEX_op_ssadd_vec:
+case INDEX_op_sssub_vec:
+return vece < MO_64 ? -1 : 0;
 default:
 return 0;
 }
@@ -2897,6 +2913,43 @@ static void expand_vec_cmp(TCGType type, unsigned vece, 
TCGv_vec v0,
 }
 }
 
+static void expand_vec_sat(TCGType type, unsigned vece, TCGv_vec v0,
+   TCGv_vec v1, TCGv_vec v2, TCGOpcode add_sub_opc)
+{
+TCGv_vec h1 = tcg_temp_new_vec(type);
+TCGv_vec h2 = tcg_temp_new_vec(type);
+TCGv_vec l1 = tcg_temp_new_vec(type);
+TCGv_vec l2 = tcg_temp_new_vec(type);
+
+tcg_debug_assert (vece < MO_64);
+
+/* Unpack with sign-extension. */
+vec_gen_2(INDEX_op_s390_vuph_vec, type, vece,
+  tcgv_vec_arg(h1), tcgv_vec_arg(v1));
+vec_gen_2(INDEX_op_s390_vuph_vec, type, vece,
+  tcgv_vec_arg(h2), tcgv_vec_arg(v2));
+
+vec_gen_2(INDEX_op_s390_vupl_vec, type, vece,
+  tcgv_vec_arg(l1), tcgv_vec_arg(v1));
+vec_gen_2(INDEX_op_s390_vupl_vec, type, vece,
+  tcgv_vec_arg(l2), tcgv_vec_arg(v2));
+
+/* Arithmetic on a wider element size. */
+vec_gen_3(add_sub_opc, type, vece + 1, tcgv_vec_arg(h1),
+  tcgv_vec_arg(h1), tcgv_vec_arg(h2));
+vec_gen_3(add_sub_opc, type, vece + 1, tcgv_vec_arg(l1),
+  tcgv_vec_arg(l1), tcgv_vec_arg(l2));
+
+/* Pack with saturation. */
+vec_gen_3(INDEX_op_s390_vpks_vec, type, vece + 1,
+  tcgv_vec_arg(v0), tcgv_vec_arg(h1), tcgv_vec_arg(l1));
+
+tcg_temp_free_vec(h1);
+tcg_temp_free_vec(h2);
+tcg_temp_free_vec(l1);
+tcg_temp_free_vec(l2);
+}
+
 void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece,
TCGArg a0, ...)
 {
@@ -2920,6 +2973,13 @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, 
unsigned vece,
 tcg_temp_free_vec(t0);
 break;
 
+case INDEX_op_ssadd_vec:
+expand_vec_sat(type, vece, v0, v1, v2, INDEX_op_add_vec);
+break;
+case INDEX_op_sssub_vec:
+expand_vec_sat(type, vece, v0, v1, v2, INDEX_op_sub_vec);
+break;
+
 default:
 g_assert_not_reached();
 }
@@ -3080,6 +3140,8 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode 
op)
 case INDEX_op_sari_vec:
 case INDEX_op_shli_vec:
 case INDEX_op_shri_vec:
+case INDEX_op_s390_vuph_vec:
+case INDEX_op_s390_vupl_vec:
 return C_O1_I1(v, v);
 case INDEX_op_add_vec:
 case INDEX_op_sub_vec:
@@ -3099,6 +3161,7 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode 
op)
 case INDEX_op_smin_vec:
 case INDEX_op_umax_vec:
 case INDEX_op_umin_vec:
+case INDEX_op_s390_vpks_vec:
 return C_O1_I2(v, v, v);
 case INDEX_op_rotls_vec:
 case INDEX_op_shls_vec:
-- 
2.25.1

[PATCH v5 15/16] tcg/s390x: Implement TCG_TARGET_HAS_bitsel_vec

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 tcg/s390x/tcg-target-con-set.h |  1 +
 tcg/s390x/tcg-target.h |  2 +-
 tcg/s390x/tcg-target.c.inc | 20 
 3 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/tcg/s390x/tcg-target-con-set.h b/tcg/s390x/tcg-target-con-set.h
index 49b98f33b9..426dd92e51 100644
--- a/tcg/s390x/tcg-target-con-set.h
+++ b/tcg/s390x/tcg-target-con-set.h
@@ -26,6 +26,7 @@ C_O1_I2(r, r, ri)
 C_O1_I2(r, rZ, r)
 C_O1_I2(v, v, r)
 C_O1_I2(v, v, v)
+C_O1_I3(v, v, v, v)
 C_O1_I4(r, r, ri, r, 0)
 C_O1_I4(r, r, ri, rI, 0)
 C_O2_I2(b, a, 0, r)
diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h
index a79f4f187a..527ada0f63 100644
--- a/tcg/s390x/tcg-target.h
+++ b/tcg/s390x/tcg-target.h
@@ -157,7 +157,7 @@ extern uint64_t s390_facilities[3];
 #define TCG_TARGET_HAS_mul_vec1
 #define TCG_TARGET_HAS_sat_vec0
 #define TCG_TARGET_HAS_minmax_vec 1
-#define TCG_TARGET_HAS_bitsel_vec 0
+#define TCG_TARGET_HAS_bitsel_vec 1
 #define TCG_TARGET_HAS_cmpsel_vec 0
 
 /* used for function call generation */
diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
index 4a56532011..5530c974a6 100644
--- a/tcg/s390x/tcg-target.c.inc
+++ b/tcg/s390x/tcg-target.c.inc
@@ -296,6 +296,7 @@ typedef enum S390Opcode {
 VRRa_VUPH   = 0xe7d7,
 VRRa_VUPL   = 0xe7d6,
 VRRc_VX = 0xe76d,
+VRRe_VSEL   = 0xe78d,
 VRRf_VLVGP  = 0xe762,
 
 VRSa_VERLL  = 0xe733,
@@ -647,6 +648,18 @@ static void tcg_out_insn_VRRc(TCGContext *s, S390Opcode op,
 tcg_out16(s, (op & 0x00ff) | RXB(v1, v2, v3, 0) | (m4 << 12));
 }
 
+static void tcg_out_insn_VRRe(TCGContext *s, S390Opcode op,
+  TCGReg v1, TCGReg v2, TCGReg v3, TCGReg v4)
+{
+tcg_debug_assert(is_vector_reg(v1));
+tcg_debug_assert(is_vector_reg(v2));
+tcg_debug_assert(is_vector_reg(v3));
+tcg_debug_assert(is_vector_reg(v4));
+tcg_out16(s, (op & 0xff00) | ((v1 & 0xf) << 4) | (v2 & 0xf));
+tcg_out16(s, v3 << 12);
+tcg_out16(s, (op & 0x00ff) | RXB(v1, v2, v3, v4) | (v4 << 12));
+}
+
 static void tcg_out_insn_VRRf(TCGContext *s, S390Opcode op,
   TCGReg v1, TCGReg r2, TCGReg r3)
 {
@@ -2787,6 +2800,10 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc,
 tcg_out_insn(s, VRRc, VMXL, a0, a1, a2, vece);
 break;
 
+case INDEX_op_bitsel_vec:
+tcg_out_insn(s, VRRe, VSEL, a0, a1, a2, args[3]);
+break;
+
 case INDEX_op_cmp_vec:
 switch ((TCGCond)args[3]) {
 case TCG_COND_EQ:
@@ -2827,6 +2844,7 @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, 
unsigned vece)
 case INDEX_op_add_vec:
 case INDEX_op_and_vec:
 case INDEX_op_andc_vec:
+case INDEX_op_bitsel_vec:
 case INDEX_op_neg_vec:
 case INDEX_op_not_vec:
 case INDEX_op_or_vec:
@@ -3168,6 +3186,8 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode 
op)
 case INDEX_op_shrs_vec:
 case INDEX_op_sars_vec:
 return C_O1_I2(v, v, r);
+case INDEX_op_bitsel_vec:
+return C_O1_I3(v, v, v, v);
 
 default:
 g_assert_not_reached();
-- 
2.25.1

[PATCH v5 03/16] tcg/s390x: Change FACILITY representation

[Richard Henderson]

We will shortly need to be able to check facilities beyond the
first 64.  Instead of explicitly masking against s390_facilities,
create a HAVE_FACILITY macro that indexes an array.

Reviewed-by: David Hildenbrand 
Signed-off-by: Richard Henderson 
---
v2: Change name to HAVE_FACILITY (david)
---
 tcg/s390x/tcg-target.h | 29 ---
 tcg/s390x/tcg-target.c.inc | 74 +++---
 2 files changed, 52 insertions(+), 51 deletions(-)

diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h
index 2e4ede2ea2..18d0d330e6 100644
--- a/tcg/s390x/tcg-target.h
+++ b/tcg/s390x/tcg-target.h
@@ -55,16 +55,19 @@ typedef enum TCGReg {
 /* A list of relevant facilities used by this translator.  Some of these
are required for proper operation, and these are checked at startup.  */
 
-#define FACILITY_ZARCH_ACTIVE (1ULL << (63 - 2))
-#define FACILITY_LONG_DISP(1ULL << (63 - 18))
-#define FACILITY_EXT_IMM  (1ULL << (63 - 21))
-#define FACILITY_GEN_INST_EXT (1ULL << (63 - 34))
-#define FACILITY_LOAD_ON_COND (1ULL << (63 - 45))
+#define FACILITY_ZARCH_ACTIVE 2
+#define FACILITY_LONG_DISP18
+#define FACILITY_EXT_IMM  21
+#define FACILITY_GEN_INST_EXT 34
+#define FACILITY_LOAD_ON_COND 45
 #define FACILITY_FAST_BCR_SER FACILITY_LOAD_ON_COND
 #define FACILITY_DISTINCT_OPS FACILITY_LOAD_ON_COND
-#define FACILITY_LOAD_ON_COND2(1ULL << (63 - 53))
+#define FACILITY_LOAD_ON_COND253
 
-extern uint64_t s390_facilities;
+extern uint64_t s390_facilities[1];
+
+#define HAVE_FACILITY(X) \
+((s390_facilities[FACILITY_##X / 64] >> (63 - FACILITY_##X % 64)) & 1)
 
 /* optional instructions */
 #define TCG_TARGET_HAS_div2_i32   1
@@ -85,8 +88,8 @@ extern uint64_t s390_facilities;
 #define TCG_TARGET_HAS_clz_i320
 #define TCG_TARGET_HAS_ctz_i320
 #define TCG_TARGET_HAS_ctpop_i32  0
-#define TCG_TARGET_HAS_deposit_i32(s390_facilities & FACILITY_GEN_INST_EXT)
-#define TCG_TARGET_HAS_extract_i32(s390_facilities & FACILITY_GEN_INST_EXT)
+#define TCG_TARGET_HAS_deposit_i32HAVE_FACILITY(GEN_INST_EXT)
+#define TCG_TARGET_HAS_extract_i32HAVE_FACILITY(GEN_INST_EXT)
 #define TCG_TARGET_HAS_sextract_i32   0
 #define TCG_TARGET_HAS_extract2_i32   0
 #define TCG_TARGET_HAS_movcond_i321
@@ -98,7 +101,7 @@ extern uint64_t s390_facilities;
 #define TCG_TARGET_HAS_mulsh_i32  0
 #define TCG_TARGET_HAS_extrl_i64_i32  0
 #define TCG_TARGET_HAS_extrh_i64_i32  0
-#define TCG_TARGET_HAS_direct_jump(s390_facilities & FACILITY_GEN_INST_EXT)
+#define TCG_TARGET_HAS_direct_jumpHAVE_FACILITY(GEN_INST_EXT)
 #define TCG_TARGET_HAS_qemu_st8_i32   0
 
 #define TCG_TARGET_HAS_div2_i64   1
@@ -119,11 +122,11 @@ extern uint64_t s390_facilities;
 #define TCG_TARGET_HAS_eqv_i640
 #define TCG_TARGET_HAS_nand_i64   0
 #define TCG_TARGET_HAS_nor_i640
-#define TCG_TARGET_HAS_clz_i64(s390_facilities & FACILITY_EXT_IMM)
+#define TCG_TARGET_HAS_clz_i64HAVE_FACILITY(EXT_IMM)
 #define TCG_TARGET_HAS_ctz_i640
 #define TCG_TARGET_HAS_ctpop_i64  0
-#define TCG_TARGET_HAS_deposit_i64(s390_facilities & FACILITY_GEN_INST_EXT)
-#define TCG_TARGET_HAS_extract_i64(s390_facilities & FACILITY_GEN_INST_EXT)
+#define TCG_TARGET_HAS_deposit_i64HAVE_FACILITY(GEN_INST_EXT)
+#define TCG_TARGET_HAS_extract_i64HAVE_FACILITY(GEN_INST_EXT)
 #define TCG_TARGET_HAS_sextract_i64   0
 #define TCG_TARGET_HAS_extract2_i64   0
 #define TCG_TARGET_HAS_movcond_i641
diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
index b82cf19f09..f7e4a619e7 100644
--- a/tcg/s390x/tcg-target.c.inc
+++ b/tcg/s390x/tcg-target.c.inc
@@ -66,7 +66,7 @@
We don't need this when we have pc-relative loads with the general
instructions extension facility.  */
 #define TCG_REG_TB  TCG_REG_R12
-#define USE_REG_TB  (!(s390_facilities & FACILITY_GEN_INST_EXT))
+#define USE_REG_TB  (!HAVE_FACILITY(GEN_INST_EXT))
 
 #ifndef CONFIG_SOFTMMU
 #define TCG_GUEST_BASE_REG TCG_REG_R13
@@ -377,7 +377,7 @@ static void * const qemu_st_helpers[16] = {
 #endif
 
 static const tcg_insn_unit *tb_ret_addr;
-uint64_t s390_facilities;
+uint64_t s390_facilities[1];
 
 static bool patch_reloc(tcg_insn_unit *src_rw, int type,
 intptr_t value, intptr_t addend)
@@ -577,7 +577,7 @@ static void tcg_out_movi_int(TCGContext *s, TCGType type, 
TCGReg ret,
 }
 
 /* Try all 48-bit insns that can load it in one go.  */
-if (s390_facilities & FACILITY_EXT_IMM) {
+if (HAVE_FACILITY(EXT_IMM)) {
 if (sval == (int32_t)sval) {
 tcg_out_insn(s, RIL, LGFI, ret, sval);
 return;
@@ -620,7 +620,7 @@ static void tcg_out_movi_int(TCGContext *s, TCGType type, 
TCGReg ret,
 }
 
 /* Otherwise, stuff it in the constant pool.  */
-if (s390_facilities & FACILITY_GEN_INST_EXT) {
+  

[PATCH v5 09/16] tcg/s390x: Implement minimal vector operations

[Richard Henderson]

Implementing add, sub, and, or, xor as the minimal set.
This allows us to actually enable vectors in query_s390_facilities.

Reviewed-by: David Hildenbrand 
Signed-off-by: Richard Henderson 
---
 tcg/s390x/tcg-target.c.inc | 154 -
 1 file changed, 150 insertions(+), 4 deletions(-)

diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
index b9de4dc821..28729b6ffa 100644
--- a/tcg/s390x/tcg-target.c.inc
+++ b/tcg/s390x/tcg-target.c.inc
@@ -271,6 +271,14 @@ typedef enum S390Opcode {
 VRIc_VREP   = 0xe74d,
 
 VRRa_VLR= 0xe756,
+VRRc_VA = 0xe7f3,
+VRRc_VCEQ   = 0xe7f8,   /* we leave the m5 cs field 0 */
+VRRc_VCH= 0xe7fb,   /* " */
+VRRc_VCHL   = 0xe7f9,   /* " */
+VRRc_VN = 0xe768,
+VRRc_VO = 0xe76a,
+VRRc_VS = 0xe7f7,
+VRRc_VX = 0xe76d,
 VRRf_VLVGP  = 0xe762,
 
 VRSb_VLVG   = 0xe722,
@@ -607,6 +615,17 @@ static void tcg_out_insn_VRRa(TCGContext *s, S390Opcode op,
 tcg_out32(s, (op & 0x00ff) | RXB(v1, v2, 0, 0) | (m3 << 12));
 }
 
+static void tcg_out_insn_VRRc(TCGContext *s, S390Opcode op,
+  TCGReg v1, TCGReg v2, TCGReg v3, int m4)
+{
+tcg_debug_assert(is_vector_reg(v1));
+tcg_debug_assert(is_vector_reg(v2));
+tcg_debug_assert(is_vector_reg(v3));
+tcg_out16(s, (op & 0xff00) | ((v1 & 0xf) << 4) | (v2 & 0xf));
+tcg_out16(s, v3 << 12);
+tcg_out16(s, (op & 0x00ff) | RXB(v1, v2, v3, 0) | (m4 << 12));
+}
+
 static void tcg_out_insn_VRRf(TCGContext *s, S390Opcode op,
   TCGReg v1, TCGReg r2, TCGReg r3)
 {
@@ -2636,18 +2655,145 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode 
opc,
unsigned vecl, unsigned vece,
const TCGArg *args, const int *const_args)
 {
-g_assert_not_reached();
+TCGType type = vecl + TCG_TYPE_V64;
+TCGArg a0 = args[0], a1 = args[1], a2 = args[2];
+
+switch (opc) {
+case INDEX_op_ld_vec:
+tcg_out_ld(s, type, a0, a1, a2);
+break;
+case INDEX_op_st_vec:
+tcg_out_st(s, type, a0, a1, a2);
+break;
+case INDEX_op_dupm_vec:
+tcg_out_dupm_vec(s, type, vece, a0, a1, a2);
+break;
+
+case INDEX_op_add_vec:
+tcg_out_insn(s, VRRc, VA, a0, a1, a2, vece);
+break;
+case INDEX_op_sub_vec:
+tcg_out_insn(s, VRRc, VS, a0, a1, a2, vece);
+break;
+case INDEX_op_and_vec:
+tcg_out_insn(s, VRRc, VN, a0, a1, a2, 0);
+break;
+case INDEX_op_or_vec:
+tcg_out_insn(s, VRRc, VO, a0, a1, a2, 0);
+break;
+case INDEX_op_xor_vec:
+tcg_out_insn(s, VRRc, VX, a0, a1, a2, 0);
+break;
+
+case INDEX_op_cmp_vec:
+switch ((TCGCond)args[3]) {
+case TCG_COND_EQ:
+tcg_out_insn(s, VRRc, VCEQ, a0, a1, a2, vece);
+break;
+case TCG_COND_GT:
+tcg_out_insn(s, VRRc, VCH, a0, a1, a2, vece);
+break;
+case TCG_COND_GTU:
+tcg_out_insn(s, VRRc, VCHL, a0, a1, a2, vece);
+break;
+default:
+g_assert_not_reached();
+}
+break;
+
+case INDEX_op_mov_vec:   /* Always emitted via tcg_out_mov.  */
+case INDEX_op_dup_vec:   /* Always emitted via tcg_out_dup_vec.  */
+default:
+g_assert_not_reached();
+}
 }
 
 int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
 {
-return 0;
+switch (opc) {
+case INDEX_op_add_vec:
+case INDEX_op_and_vec:
+case INDEX_op_or_vec:
+case INDEX_op_sub_vec:
+case INDEX_op_xor_vec:
+return 1;
+case INDEX_op_cmp_vec:
+return -1;
+default:
+return 0;
+}
+}
+
+static bool expand_vec_cmp_noinv(TCGType type, unsigned vece, TCGv_vec v0,
+ TCGv_vec v1, TCGv_vec v2, TCGCond cond)
+{
+bool need_swap = false, need_inv = false;
+
+switch (cond) {
+case TCG_COND_EQ:
+case TCG_COND_GT:
+case TCG_COND_GTU:
+break;
+case TCG_COND_NE:
+case TCG_COND_LE:
+case TCG_COND_LEU:
+need_inv = true;
+break;
+case TCG_COND_LT:
+case TCG_COND_LTU:
+need_swap = true;
+break;
+case TCG_COND_GE:
+case TCG_COND_GEU:
+need_swap = need_inv = true;
+break;
+default:
+g_assert_not_reached();
+}
+
+if (need_inv) {
+cond = tcg_invert_cond(cond);
+}
+if (need_swap) {
+TCGv_vec t1;
+t1 = v1, v1 = v2, v2 = t1;
+cond = tcg_swap_cond(cond);
+}
+
+vec_gen_4(INDEX_op_cmp_vec, type, vece, tcgv_vec_arg(v0),
+  tcgv_vec_arg(v1), tcgv_vec_arg(v2), cond);
+
+return need_inv;
+}
+
+static void expand_vec_cmp(TCGType type, unsigned vece, TCGv_vec v0,
+   TCGv_vec v1, TCGv_vec v2, TCGCond cond)
+{
+if (expand_vec_cmp_noinv(type, vece, v0, v1, v2, cond)) {
+

[PATCH v5 02/16] tcg/s390x: Rename from tcg/s390

[Richard Henderson]

This emphasizes that we don't support s390, only 64-bit s390x hosts.

Reviewed-by: David Hildenbrand 
Signed-off-by: Richard Henderson 
---
 meson.build  | 2 --
 tcg/{s390 => s390x}/tcg-target-con-set.h | 0
 tcg/{s390 => s390x}/tcg-target-con-str.h | 0
 tcg/{s390 => s390x}/tcg-target.h | 0
 tcg/{s390 => s390x}/tcg-target.c.inc | 0
 5 files changed, 2 deletions(-)
 rename tcg/{s390 => s390x}/tcg-target-con-set.h (100%)
 rename tcg/{s390 => s390x}/tcg-target-con-str.h (100%)
 rename tcg/{s390 => s390x}/tcg-target.h (100%)
 rename tcg/{s390 => s390x}/tcg-target.c.inc (100%)

diff --git a/meson.build b/meson.build
index 2711cbb789..1cf370ab56 100644
--- a/meson.build
+++ b/meson.build
@@ -265,8 +265,6 @@ if not get_option('tcg').disabled()
 tcg_arch = 'tci'
   elif config_host['ARCH'] == 'sparc64'
 tcg_arch = 'sparc'
-  elif config_host['ARCH'] == 's390x'
-tcg_arch = 's390'
   elif config_host['ARCH'] in ['x86_64', 'x32']
 tcg_arch = 'i386'
   elif config_host['ARCH'] == 'ppc64'
diff --git a/tcg/s390/tcg-target-con-set.h b/tcg/s390x/tcg-target-con-set.h
similarity index 100%
rename from tcg/s390/tcg-target-con-set.h
rename to tcg/s390x/tcg-target-con-set.h
diff --git a/tcg/s390/tcg-target-con-str.h b/tcg/s390x/tcg-target-con-str.h
similarity index 100%
rename from tcg/s390/tcg-target-con-str.h
rename to tcg/s390x/tcg-target-con-str.h
diff --git a/tcg/s390/tcg-target.h b/tcg/s390x/tcg-target.h
similarity index 100%
rename from tcg/s390/tcg-target.h
rename to tcg/s390x/tcg-target.h
diff --git a/tcg/s390/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
similarity index 100%
rename from tcg/s390/tcg-target.c.inc
rename to tcg/s390x/tcg-target.c.inc
-- 
2.25.1

[PATCH v5 16/16] tcg/s390x: Implement TCG_TARGET_HAS_cmpsel_vec

[Richard Henderson]

This is via expansion; don't actually set TCG_TARGET_HAS_cmpsel_vec.

Signed-off-by: Richard Henderson 
---
 tcg/s390x/tcg-target.c.inc | 24 +++-
 1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
index 5530c974a6..aef24d0795 100644
--- a/tcg/s390x/tcg-target.c.inc
+++ b/tcg/s390x/tcg-target.c.inc
@@ -2869,6 +2869,7 @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, 
unsigned vece)
 case INDEX_op_xor_vec:
 return 1;
 case INDEX_op_cmp_vec:
+case INDEX_op_cmpsel_vec:
 case INDEX_op_rotrv_vec:
 return -1;
 case INDEX_op_mul_vec:
@@ -2931,6 +2932,21 @@ static void expand_vec_cmp(TCGType type, unsigned vece, 
TCGv_vec v0,
 }
 }
 
+static void expand_vec_cmpsel(TCGType type, unsigned vece, TCGv_vec v0,
+  TCGv_vec c1, TCGv_vec c2,
+  TCGv_vec v3, TCGv_vec v4, TCGCond cond)
+{
+TCGv_vec t = tcg_temp_new_vec(type);
+
+if (expand_vec_cmp_noinv(type, vece, t, c1, c2, cond)) {
+/* Invert the sense of the compare by swapping arguments.  */
+tcg_gen_bitsel_vec(vece, v0, t, v4, v3);
+} else {
+tcg_gen_bitsel_vec(vece, v0, t, v3, v4);
+}
+tcg_temp_free_vec(t);
+}
+
 static void expand_vec_sat(TCGType type, unsigned vece, TCGv_vec v0,
TCGv_vec v1, TCGv_vec v2, TCGOpcode add_sub_opc)
 {
@@ -2972,7 +2988,7 @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, 
unsigned vece,
TCGArg a0, ...)
 {
 va_list va;
-TCGv_vec v0, v1, v2, t0;
+TCGv_vec v0, v1, v2, v3, v4, t0;
 
 va_start(va, a0);
 v0 = temp_tcgv_vec(arg_temp(a0));
@@ -2984,6 +3000,12 @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, 
unsigned vece,
 expand_vec_cmp(type, vece, v0, v1, v2, va_arg(va, TCGArg));
 break;
 
+case INDEX_op_cmpsel_vec:
+v3 = temp_tcgv_vec(arg_temp(va_arg(va, TCGArg)));
+v4 = temp_tcgv_vec(arg_temp(va_arg(va, TCGArg)));
+expand_vec_cmpsel(type, vece, v0, v1, v2, v3, v4, va_arg(va, TCGArg));
+break;
+
 case INDEX_op_rotrv_vec:
 t0 = tcg_temp_new_vec(type);
 tcg_gen_neg_vec(vece, t0, v2);
-- 
2.25.1

[PATCH v5 07/16] tcg/s390x: Implement tcg_out_mov for vector types

[Richard Henderson]

Reviewed-by: David Hildenbrand 
Signed-off-by: Richard Henderson 
---
 tcg/s390x/tcg-target.c.inc | 72 +++---
 1 file changed, 68 insertions(+), 4 deletions(-)

diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
index ea04aefe98..76061bfd80 100644
--- a/tcg/s390x/tcg-target.c.inc
+++ b/tcg/s390x/tcg-target.c.inc
@@ -265,6 +265,11 @@ typedef enum S390Opcode {
 RX_STC  = 0x42,
 RX_STH  = 0x40,
 
+VRRa_VLR= 0xe756,
+
+VRSb_VLVG   = 0xe722,
+VRSc_VLGV   = 0xe721,
+
 VRX_VL  = 0xe706,
 VRX_VLLEZ   = 0xe704,
 VRX_VST = 0xe70e,
@@ -558,6 +563,39 @@ static int RXB(TCGReg v1, TCGReg v2, TCGReg v3, TCGReg v4)
  | ((v4 & 0x10) << (4 + 0));
 }
 
+static void tcg_out_insn_VRRa(TCGContext *s, S390Opcode op,
+  TCGReg v1, TCGReg v2, int m3)
+{
+tcg_debug_assert(is_vector_reg(v1));
+tcg_debug_assert(is_vector_reg(v2));
+tcg_out16(s, (op & 0xff00) | ((v1 & 0xf) << 4) | (v2 & 0xf));
+tcg_out32(s, (op & 0x00ff) | RXB(v1, v2, 0, 0) | (m3 << 12));
+}
+
+static void tcg_out_insn_VRSb(TCGContext *s, S390Opcode op, TCGReg v1,
+  intptr_t d2, TCGReg b2, TCGReg r3, int m4)
+{
+tcg_debug_assert(is_vector_reg(v1));
+tcg_debug_assert(d2 >= 0 && d2 <= 0xfff);
+tcg_debug_assert(is_general_reg(b2));
+tcg_debug_assert(is_general_reg(r3));
+tcg_out16(s, (op & 0xff00) | ((v1 & 0xf) << 4) | r3);
+tcg_out16(s, b2 << 12 | d2);
+tcg_out16(s, (op & 0x00ff) | RXB(v1, 0, 0, 0) | (m4 << 12));
+}
+
+static void tcg_out_insn_VRSc(TCGContext *s, S390Opcode op, TCGReg r1,
+  intptr_t d2, TCGReg b2, TCGReg v3, int m4)
+{
+tcg_debug_assert(is_general_reg(r1));
+tcg_debug_assert(d2 >= 0 && d2 <= 0xfff);
+tcg_debug_assert(is_general_reg(b2));
+tcg_debug_assert(is_vector_reg(v3));
+tcg_out16(s, (op & 0xff00) | (r1 << 4) | (v3 & 0xf));
+tcg_out16(s, b2 << 12 | d2);
+tcg_out16(s, (op & 0x00ff) | RXB(0, 0, v3, 0) | (m4 << 12));
+}
+
 static void tcg_out_insn_VRX(TCGContext *s, S390Opcode op, TCGReg v1,
  TCGReg b2, TCGReg x2, intptr_t d2, int m3)
 {
@@ -591,12 +629,38 @@ static void tcg_out_sh32(TCGContext* s, S390Opcode op, 
TCGReg dest,
 
 static bool tcg_out_mov(TCGContext *s, TCGType type, TCGReg dst, TCGReg src)
 {
-if (src != dst) {
-if (type == TCG_TYPE_I32) {
+if (src == dst) {
+return true;
+}
+switch (type) {
+case TCG_TYPE_I32:
+if (likely(is_general_reg(dst) && is_general_reg(src))) {
 tcg_out_insn(s, RR, LR, dst, src);
-} else {
-tcg_out_insn(s, RRE, LGR, dst, src);
+break;
 }
+/* fallthru */
+
+case TCG_TYPE_I64:
+if (likely(is_general_reg(dst))) {
+if (likely(is_general_reg(src))) {
+tcg_out_insn(s, RRE, LGR, dst, src);
+} else {
+tcg_out_insn(s, VRSc, VLGV, dst, 0, 0, src, 3);
+}
+break;
+} else if (is_general_reg(src)) {
+tcg_out_insn(s, VRSb, VLVG, dst, 0, 0, src, 3);
+break;
+}
+/* fallthru */
+
+case TCG_TYPE_V64:
+case TCG_TYPE_V128:
+tcg_out_insn(s, VRRa, VLR, dst, src, 0);
+break;
+
+default:
+g_assert_not_reached();
 }
 return true;
 }
-- 
2.25.1

[PATCH v5 12/16] tcg/s390x: Implement vector shift operations

[Richard Henderson]

Reviewed-by: David Hildenbrand 
Signed-off-by: Richard Henderson 
---
 tcg/s390x/tcg-target-con-set.h |  1 +
 tcg/s390x/tcg-target.h | 12 ++---
 tcg/s390x/tcg-target.c.inc | 93 +-
 3 files changed, 99 insertions(+), 7 deletions(-)

diff --git a/tcg/s390x/tcg-target-con-set.h b/tcg/s390x/tcg-target-con-set.h
index cb953896d5..49b98f33b9 100644
--- a/tcg/s390x/tcg-target-con-set.h
+++ b/tcg/s390x/tcg-target-con-set.h
@@ -24,6 +24,7 @@ C_O1_I2(r, 0, rI)
 C_O1_I2(r, 0, rJ)
 C_O1_I2(r, r, ri)
 C_O1_I2(r, rZ, r)
+C_O1_I2(v, v, r)
 C_O1_I2(v, v, v)
 C_O1_I4(r, r, ri, r, 0)
 C_O1_I4(r, r, ri, rI, 0)
diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h
index 1c581a2f60..d7d204b782 100644
--- a/tcg/s390x/tcg-target.h
+++ b/tcg/s390x/tcg-target.h
@@ -148,12 +148,12 @@ extern uint64_t s390_facilities[3];
 #define TCG_TARGET_HAS_not_vec1
 #define TCG_TARGET_HAS_neg_vec1
 #define TCG_TARGET_HAS_abs_vec1
-#define TCG_TARGET_HAS_roti_vec   0
-#define TCG_TARGET_HAS_rots_vec   0
-#define TCG_TARGET_HAS_rotv_vec   0
-#define TCG_TARGET_HAS_shi_vec0
-#define TCG_TARGET_HAS_shs_vec0
-#define TCG_TARGET_HAS_shv_vec0
+#define TCG_TARGET_HAS_roti_vec   1
+#define TCG_TARGET_HAS_rots_vec   1
+#define TCG_TARGET_HAS_rotv_vec   1
+#define TCG_TARGET_HAS_shi_vec1
+#define TCG_TARGET_HAS_shs_vec1
+#define TCG_TARGET_HAS_shv_vec1
 #define TCG_TARGET_HAS_mul_vec1
 #define TCG_TARGET_HAS_sat_vec0
 #define TCG_TARGET_HAS_minmax_vec 0
diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
index aa75d779be..a56733e09a 100644
--- a/tcg/s390x/tcg-target.c.inc
+++ b/tcg/s390x/tcg-target.c.inc
@@ -277,6 +277,10 @@ typedef enum S390Opcode {
 VRRc_VCEQ   = 0xe7f8,   /* we leave the m5 cs field 0 */
 VRRc_VCH= 0xe7fb,   /* " */
 VRRc_VCHL   = 0xe7f9,   /* " */
+VRRc_VERLLV = 0xe773,
+VRRc_VESLV  = 0xe770,
+VRRc_VESRAV = 0xe77a,
+VRRc_VESRLV = 0xe778,
 VRRc_VML= 0xe7a2,
 VRRc_VN = 0xe768,
 VRRc_VNC= 0xe769,
@@ -287,6 +291,10 @@ typedef enum S390Opcode {
 VRRc_VX = 0xe76d,
 VRRf_VLVGP  = 0xe762,
 
+VRSa_VERLL  = 0xe733,
+VRSa_VESL   = 0xe730,
+VRSa_VESRA  = 0xe73a,
+VRSa_VESRL  = 0xe738,
 VRSb_VLVG   = 0xe722,
 VRSc_VLGV   = 0xe721,
 
@@ -643,6 +651,18 @@ static void tcg_out_insn_VRRf(TCGContext *s, S390Opcode op,
 tcg_out16(s, (op & 0x00ff) | RXB(v1, 0, 0, 0));
 }
 
+static void tcg_out_insn_VRSa(TCGContext *s, S390Opcode op, TCGReg v1,
+  intptr_t d2, TCGReg b2, TCGReg v3, int m4)
+{
+tcg_debug_assert(is_vector_reg(v1));
+tcg_debug_assert(d2 >= 0 && d2 <= 0xfff);
+tcg_debug_assert(is_general_reg(b2));
+tcg_debug_assert(is_vector_reg(v3));
+tcg_out16(s, (op & 0xff00) | ((v1 & 0xf) << 4) | (v3 & 0xf));
+tcg_out16(s, b2 << 12 | d2);
+tcg_out16(s, (op & 0x00ff) | RXB(v1, 0, v3, 0) | (m4 << 12));
+}
+
 static void tcg_out_insn_VRSb(TCGContext *s, S390Opcode op, TCGReg v1,
   intptr_t d2, TCGReg b2, TCGReg r3, int m4)
 {
@@ -2710,6 +2730,43 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc,
 tcg_out_insn(s, VRRc, VX, a0, a1, a2, 0);
 break;
 
+case INDEX_op_shli_vec:
+tcg_out_insn(s, VRSa, VESL, a0, a2, TCG_REG_NONE, a1, vece);
+break;
+case INDEX_op_shri_vec:
+tcg_out_insn(s, VRSa, VESRL, a0, a2, TCG_REG_NONE, a1, vece);
+break;
+case INDEX_op_sari_vec:
+tcg_out_insn(s, VRSa, VESRA, a0, a2, TCG_REG_NONE, a1, vece);
+break;
+case INDEX_op_rotli_vec:
+tcg_out_insn(s, VRSa, VERLL, a0, a2, TCG_REG_NONE, a1, vece);
+break;
+case INDEX_op_shls_vec:
+tcg_out_insn(s, VRSa, VESL, a0, 0, a2, a1, vece);
+break;
+case INDEX_op_shrs_vec:
+tcg_out_insn(s, VRSa, VESRL, a0, 0, a2, a1, vece);
+break;
+case INDEX_op_sars_vec:
+tcg_out_insn(s, VRSa, VESRA, a0, 0, a2, a1, vece);
+break;
+case INDEX_op_rotls_vec:
+tcg_out_insn(s, VRSa, VERLL, a0, 0, a2, a1, vece);
+break;
+case INDEX_op_shlv_vec:
+tcg_out_insn(s, VRRc, VESLV, a0, a1, a2, vece);
+break;
+case INDEX_op_shrv_vec:
+tcg_out_insn(s, VRRc, VESRLV, a0, a1, a2, vece);
+break;
+case INDEX_op_sarv_vec:
+tcg_out_insn(s, VRRc, VESRAV, a0, a1, a2, vece);
+break;
+case INDEX_op_rotlv_vec:
+tcg_out_insn(s, VRRc, VERLLV, a0, a1, a2, vece);
+break;
+
 case INDEX_op_cmp_vec:
 switch ((TCGCond)args[3]) {
 case TCG_COND_EQ:
@@ -2744,10 +2801,23 @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, 
unsigned vece)
 case INDEX_op_not_vec:
 case INDEX_op_or_vec:
 case INDEX_op_orc_vec:
+case INDEX_op_rotli_vec:
+case INDEX_op_rotls_vec:
+case INDEX_op_rotlv_vec:
+c

[PATCH v5 00/16] tcg/s390x: host vector support

[Richard Henderson]

Changes for v5:
  * Add is_{general,vector}_reg predicates.
  * Use 0xf not 15 for masking.

r~

Richard Henderson (16):
  tcg: Expand usadd/ussub with umin/umax
  tcg/s390x: Rename from tcg/s390
  tcg/s390x: Change FACILITY representation
  tcg/s390x: Merge TCG_AREG0 and TCG_REG_CALL_STACK into TCGReg
  tcg/s390x: Add host vector framework
  tcg/s390x: Implement tcg_out_ld/st for vector types
  tcg/s390x: Implement tcg_out_mov for vector types
  tcg/s390x: Implement tcg_out_dup*_vec
  tcg/s390x: Implement minimal vector operations
  tcg/s390x: Implement andc, orc, abs, neg, not vector operations
  tcg/s390x: Implement TCG_TARGET_HAS_mul_vec
  tcg/s390x: Implement vector shift operations
  tcg/s390x: Implement TCG_TARGET_HAS_minmax_vec
  tcg/s390x: Implement TCG_TARGET_HAS_sat_vec
  tcg/s390x: Implement TCG_TARGET_HAS_bitsel_vec
  tcg/s390x: Implement TCG_TARGET_HAS_cmpsel_vec

 meson.build  |   2 -
 tcg/{s390 => s390x}/tcg-target-con-set.h |   7 +
 tcg/{s390 => s390x}/tcg-target-con-str.h |   1 +
 tcg/{s390 => s390x}/tcg-target.h |  91 ++-
 tcg/s390x/tcg-target.opc.h   |  15 +
 tcg/tcg-op-vec.c |  37 +-
 tcg/{s390 => s390x}/tcg-target.c.inc | 935 +--
 7 files changed, 993 insertions(+), 95 deletions(-)
 rename tcg/{s390 => s390x}/tcg-target-con-set.h (86%)
 rename tcg/{s390 => s390x}/tcg-target-con-str.h (96%)
 rename tcg/{s390 => s390x}/tcg-target.h (66%)
 create mode 100644 tcg/s390x/tcg-target.opc.h
 rename tcg/{s390 => s390x}/tcg-target.c.inc (73%)

-- 
2.25.1

[PATCH v5 05/16] tcg/s390x: Add host vector framework

[Richard Henderson]

Add registers and function stubs.  The functionality
is disabled via squashing s390_facilities[2] to 0.

We must still include results for the mandatory opcodes in
tcg_target_op_def, as all opcodes are checked during tcg init.

Reviewed-by: David Hildenbrand 
Signed-off-by: Richard Henderson 
---
 tcg/s390x/tcg-target-con-set.h |   4 +
 tcg/s390x/tcg-target-con-str.h |   1 +
 tcg/s390x/tcg-target.h |  35 -
 tcg/s390x/tcg-target.opc.h |  12 +++
 tcg/s390x/tcg-target.c.inc | 137 -
 5 files changed, 184 insertions(+), 5 deletions(-)
 create mode 100644 tcg/s390x/tcg-target.opc.h

diff --git a/tcg/s390x/tcg-target-con-set.h b/tcg/s390x/tcg-target-con-set.h
index 31985e4903..ce9432cfe3 100644
--- a/tcg/s390x/tcg-target-con-set.h
+++ b/tcg/s390x/tcg-target-con-set.h
@@ -13,13 +13,17 @@ C_O0_I1(r)
 C_O0_I2(L, L)
 C_O0_I2(r, r)
 C_O0_I2(r, ri)
+C_O0_I2(v, r)
 C_O1_I1(r, L)
 C_O1_I1(r, r)
+C_O1_I1(v, r)
+C_O1_I1(v, vr)
 C_O1_I2(r, 0, ri)
 C_O1_I2(r, 0, rI)
 C_O1_I2(r, 0, rJ)
 C_O1_I2(r, r, ri)
 C_O1_I2(r, rZ, r)
+C_O1_I2(v, v, v)
 C_O1_I4(r, r, ri, r, 0)
 C_O1_I4(r, r, ri, rI, 0)
 C_O2_I2(b, a, 0, r)
diff --git a/tcg/s390x/tcg-target-con-str.h b/tcg/s390x/tcg-target-con-str.h
index 892d8f8c06..8bb0358ae5 100644
--- a/tcg/s390x/tcg-target-con-str.h
+++ b/tcg/s390x/tcg-target-con-str.h
@@ -10,6 +10,7 @@
  */
 REGS('r', ALL_GENERAL_REGS)
 REGS('L', ALL_GENERAL_REGS & ~SOFTMMU_RESERVE_REGS)
+REGS('v', ALL_VECTOR_REGS)
 /*
  * A (single) even/odd pair for division.
  * TODO: Add something to the register allocator to allow
diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h
index 0174357f1b..5a03c5f2f4 100644
--- a/tcg/s390x/tcg-target.h
+++ b/tcg/s390x/tcg-target.h
@@ -37,11 +37,20 @@ typedef enum TCGReg {
 TCG_REG_R8,  TCG_REG_R9,  TCG_REG_R10, TCG_REG_R11,
 TCG_REG_R12, TCG_REG_R13, TCG_REG_R14, TCG_REG_R15,
 
+TCG_REG_V0 = 32, TCG_REG_V1,  TCG_REG_V2,  TCG_REG_V3,
+TCG_REG_V4,  TCG_REG_V5,  TCG_REG_V6,  TCG_REG_V7,
+TCG_REG_V8,  TCG_REG_V9,  TCG_REG_V10, TCG_REG_V11,
+TCG_REG_V12, TCG_REG_V13, TCG_REG_V14, TCG_REG_V15,
+TCG_REG_V16, TCG_REG_V17, TCG_REG_V18, TCG_REG_V19,
+TCG_REG_V20, TCG_REG_V21, TCG_REG_V22, TCG_REG_V23,
+TCG_REG_V24, TCG_REG_V25, TCG_REG_V26, TCG_REG_V27,
+TCG_REG_V28, TCG_REG_V29, TCG_REG_V30, TCG_REG_V31,
+
 TCG_AREG0 = TCG_REG_R10,
 TCG_REG_CALL_STACK = TCG_REG_R15
 } TCGReg;
 
-#define TCG_TARGET_NB_REGS 16
+#define TCG_TARGET_NB_REGS 64
 
 /* A list of relevant facilities used by this translator.  Some of these
are required for proper operation, and these are checked at startup.  */
@@ -54,8 +63,9 @@ typedef enum TCGReg {
 #define FACILITY_FAST_BCR_SER FACILITY_LOAD_ON_COND
 #define FACILITY_DISTINCT_OPS FACILITY_LOAD_ON_COND
 #define FACILITY_LOAD_ON_COND253
+#define FACILITY_VECTOR   129
 
-extern uint64_t s390_facilities[1];
+extern uint64_t s390_facilities[3];
 
 #define HAVE_FACILITY(X) \
 ((s390_facilities[FACILITY_##X / 64] >> (63 - FACILITY_##X % 64)) & 1)
@@ -128,6 +138,27 @@ extern uint64_t s390_facilities[1];
 #define TCG_TARGET_HAS_muluh_i64  0
 #define TCG_TARGET_HAS_mulsh_i64  0
 
+#define TCG_TARGET_HAS_v64HAVE_FACILITY(VECTOR)
+#define TCG_TARGET_HAS_v128   HAVE_FACILITY(VECTOR)
+#define TCG_TARGET_HAS_v256   0
+
+#define TCG_TARGET_HAS_andc_vec   0
+#define TCG_TARGET_HAS_orc_vec0
+#define TCG_TARGET_HAS_not_vec0
+#define TCG_TARGET_HAS_neg_vec0
+#define TCG_TARGET_HAS_abs_vec0
+#define TCG_TARGET_HAS_roti_vec   0
+#define TCG_TARGET_HAS_rots_vec   0
+#define TCG_TARGET_HAS_rotv_vec   0
+#define TCG_TARGET_HAS_shi_vec0
+#define TCG_TARGET_HAS_shs_vec0
+#define TCG_TARGET_HAS_shv_vec0
+#define TCG_TARGET_HAS_mul_vec0
+#define TCG_TARGET_HAS_sat_vec0
+#define TCG_TARGET_HAS_minmax_vec 0
+#define TCG_TARGET_HAS_bitsel_vec 0
+#define TCG_TARGET_HAS_cmpsel_vec 0
+
 /* used for function call generation */
 #define TCG_TARGET_STACK_ALIGN 8
 #define TCG_TARGET_CALL_STACK_OFFSET   160
diff --git a/tcg/s390x/tcg-target.opc.h b/tcg/s390x/tcg-target.opc.h
new file mode 100644
index 00..67afc82a93
--- /dev/null
+++ b/tcg/s390x/tcg-target.opc.h
@@ -0,0 +1,12 @@
+/*
+ * Copyright (c) 2021 Linaro
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * (at your option) any later version.
+ *
+ * See the COPYING file in the top-level directory for details.
+ *
+ * Target-specific opcodes for host vector expansion.  These will be
+ * emitted by tcg_expand_vec_op.  For those familiar with GCC internals,
+ * consider these to be UNSPEC with names.
+ */
diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
index f7e4a619e7..c438751834 100644
--- a/tcg/s390x/tcg-target.c.inc
+++ b/tcg/s390x/tcg-target.c.inc
@@ -43,6 +43,8 @@
 #define TCG_CT_CONST_ZERO  0x80

[PATCH v5 10/16] tcg/s390x: Implement andc, orc, abs, neg, not vector operations

[Richard Henderson]

These logical and arithmetic operations are optional but trivial.

Signed-off-by: Richard Henderson 
---
 tcg/s390x/tcg-target-con-set.h |  1 +
 tcg/s390x/tcg-target.h | 11 ++-
 tcg/s390x/tcg-target.c.inc | 32 
 3 files changed, 39 insertions(+), 5 deletions(-)

diff --git a/tcg/s390x/tcg-target-con-set.h b/tcg/s390x/tcg-target-con-set.h
index ce9432cfe3..cb953896d5 100644
--- a/tcg/s390x/tcg-target-con-set.h
+++ b/tcg/s390x/tcg-target-con-set.h
@@ -17,6 +17,7 @@ C_O0_I2(v, r)
 C_O1_I1(r, L)
 C_O1_I1(r, r)
 C_O1_I1(v, r)
+C_O1_I1(v, v)
 C_O1_I1(v, vr)
 C_O1_I2(r, 0, ri)
 C_O1_I2(r, 0, rI)
diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h
index 5a03c5f2f4..a42074e451 100644
--- a/tcg/s390x/tcg-target.h
+++ b/tcg/s390x/tcg-target.h
@@ -64,6 +64,7 @@ typedef enum TCGReg {
 #define FACILITY_DISTINCT_OPS FACILITY_LOAD_ON_COND
 #define FACILITY_LOAD_ON_COND253
 #define FACILITY_VECTOR   129
+#define FACILITY_VECTOR_ENH1  135
 
 extern uint64_t s390_facilities[3];
 
@@ -142,11 +143,11 @@ extern uint64_t s390_facilities[3];
 #define TCG_TARGET_HAS_v128   HAVE_FACILITY(VECTOR)
 #define TCG_TARGET_HAS_v256   0
 
-#define TCG_TARGET_HAS_andc_vec   0
-#define TCG_TARGET_HAS_orc_vec0
-#define TCG_TARGET_HAS_not_vec0
-#define TCG_TARGET_HAS_neg_vec0
-#define TCG_TARGET_HAS_abs_vec0
+#define TCG_TARGET_HAS_andc_vec   1
+#define TCG_TARGET_HAS_orc_vecHAVE_FACILITY(VECTOR_ENH1)
+#define TCG_TARGET_HAS_not_vec1
+#define TCG_TARGET_HAS_neg_vec1
+#define TCG_TARGET_HAS_abs_vec1
 #define TCG_TARGET_HAS_roti_vec   0
 #define TCG_TARGET_HAS_rots_vec   0
 #define TCG_TARGET_HAS_rotv_vec   0
diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
index 28729b6ffa..fa4a142818 100644
--- a/tcg/s390x/tcg-target.c.inc
+++ b/tcg/s390x/tcg-target.c.inc
@@ -270,13 +270,18 @@ typedef enum S390Opcode {
 VRIb_VGM= 0xe746,
 VRIc_VREP   = 0xe74d,
 
+VRRa_VLC= 0xe7de,
+VRRa_VLP= 0xe7df,
 VRRa_VLR= 0xe756,
 VRRc_VA = 0xe7f3,
 VRRc_VCEQ   = 0xe7f8,   /* we leave the m5 cs field 0 */
 VRRc_VCH= 0xe7fb,   /* " */
 VRRc_VCHL   = 0xe7f9,   /* " */
 VRRc_VN = 0xe768,
+VRRc_VNC= 0xe769,
+VRRc_VNO= 0xe76b,
 VRRc_VO = 0xe76a,
+VRRc_VOC= 0xe76f,
 VRRc_VS = 0xe7f7,
 VRRc_VX = 0xe76d,
 VRRf_VLVGP  = 0xe762,
@@ -2669,6 +2674,16 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc,
 tcg_out_dupm_vec(s, type, vece, a0, a1, a2);
 break;
 
+case INDEX_op_abs_vec:
+tcg_out_insn(s, VRRa, VLP, a0, a1, vece);
+break;
+case INDEX_op_neg_vec:
+tcg_out_insn(s, VRRa, VLC, a0, a1, vece);
+break;
+case INDEX_op_not_vec:
+tcg_out_insn(s, VRRc, VNO, a0, a1, a1, 0);
+break;
+
 case INDEX_op_add_vec:
 tcg_out_insn(s, VRRc, VA, a0, a1, a2, vece);
 break;
@@ -2678,9 +2693,15 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc,
 case INDEX_op_and_vec:
 tcg_out_insn(s, VRRc, VN, a0, a1, a2, 0);
 break;
+case INDEX_op_andc_vec:
+tcg_out_insn(s, VRRc, VNC, a0, a1, a2, 0);
+break;
 case INDEX_op_or_vec:
 tcg_out_insn(s, VRRc, VO, a0, a1, a2, 0);
 break;
+case INDEX_op_orc_vec:
+tcg_out_insn(s, VRRc, VOC, a0, a1, a2, 0);
+break;
 case INDEX_op_xor_vec:
 tcg_out_insn(s, VRRc, VX, a0, a1, a2, 0);
 break;
@@ -2711,9 +2732,14 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc,
 int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece)
 {
 switch (opc) {
+case INDEX_op_abs_vec:
 case INDEX_op_add_vec:
 case INDEX_op_and_vec:
+case INDEX_op_andc_vec:
+case INDEX_op_neg_vec:
+case INDEX_op_not_vec:
 case INDEX_op_or_vec:
+case INDEX_op_orc_vec:
 case INDEX_op_sub_vec:
 case INDEX_op_xor_vec:
 return 1;
@@ -2943,10 +2969,16 @@ static TCGConstraintSetIndex 
tcg_target_op_def(TCGOpcode op)
 return C_O1_I1(v, r);
 case INDEX_op_dup_vec:
 return C_O1_I1(v, vr);
+case INDEX_op_abs_vec:
+case INDEX_op_neg_vec:
+case INDEX_op_not_vec:
+return C_O1_I1(v, v);
 case INDEX_op_add_vec:
 case INDEX_op_sub_vec:
 case INDEX_op_and_vec:
+case INDEX_op_andc_vec:
 case INDEX_op_or_vec:
+case INDEX_op_orc_vec:
 case INDEX_op_xor_vec:
 case INDEX_op_cmp_vec:
 return C_O1_I2(v, v, v);
-- 
2.25.1

[PATCH v5 06/16] tcg/s390x: Implement tcg_out_ld/st for vector types

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 tcg/s390x/tcg-target.c.inc | 132 +
 1 file changed, 120 insertions(+), 12 deletions(-)

diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
index c438751834..ea04aefe98 100644
--- a/tcg/s390x/tcg-target.c.inc
+++ b/tcg/s390x/tcg-target.c.inc
@@ -265,6 +265,12 @@ typedef enum S390Opcode {
 RX_STC  = 0x42,
 RX_STH  = 0x40,
 
+VRX_VL  = 0xe706,
+VRX_VLLEZ   = 0xe704,
+VRX_VST = 0xe70e,
+VRX_VSTEF   = 0xe70b,
+VRX_VSTEG   = 0xe70a,
+
 NOP = 0x0707,
 } S390Opcode;
 
@@ -412,6 +418,16 @@ static void * const qemu_st_helpers[16] = {
 static const tcg_insn_unit *tb_ret_addr;
 uint64_t s390_facilities[3];
 
+static inline bool is_general_reg(TCGReg r)
+{
+return r <= TCG_REG_R15;
+}
+
+static inline bool is_vector_reg(TCGReg r)
+{
+return r >= TCG_REG_V0 && r <= TCG_REG_V31;
+}
+
 static bool patch_reloc(tcg_insn_unit *src_rw, int type,
 intptr_t value, intptr_t addend)
 {
@@ -529,6 +545,31 @@ static void tcg_out_insn_RSY(TCGContext *s, S390Opcode op, 
TCGReg r1,
 #define tcg_out_insn_RX   tcg_out_insn_RS
 #define tcg_out_insn_RXY  tcg_out_insn_RSY
 
+static int RXB(TCGReg v1, TCGReg v2, TCGReg v3, TCGReg v4)
+{
+/*
+ * Shift bit 4 of each regno to its corresponding bit of RXB.
+ * RXB itself begins at bit 8 of the instruction so 8 - 4 = 4
+ * is the left-shift of the 4th operand.
+ */
+return ((v1 & 0x10) << (4 + 3))
+ | ((v2 & 0x10) << (4 + 2))
+ | ((v3 & 0x10) << (4 + 1))
+ | ((v4 & 0x10) << (4 + 0));
+}
+
+static void tcg_out_insn_VRX(TCGContext *s, S390Opcode op, TCGReg v1,
+ TCGReg b2, TCGReg x2, intptr_t d2, int m3)
+{
+tcg_debug_assert(is_vector_reg(v1));
+tcg_debug_assert(d2 >= 0 && d2 <= 0xfff);
+tcg_debug_assert(is_general_reg(x2));
+tcg_debug_assert(is_general_reg(b2));
+tcg_out16(s, (op & 0xff00) | ((v1 & 0xf) << 4) | x2);
+tcg_out16(s, (b2 << 12) | d2);
+tcg_out16(s, (op & 0x00ff) | RXB(v1, 0, 0, 0) | (m3 << 12));
+}
+
 /* Emit an opcode with "type-checking" of the format.  */
 #define tcg_out_insn(S, FMT, OP, ...) \
 glue(tcg_out_insn_,FMT)(S, glue(glue(FMT,_),OP), ## __VA_ARGS__)
@@ -705,25 +746,92 @@ static void tcg_out_mem(TCGContext *s, S390Opcode opc_rx, 
S390Opcode opc_rxy,
 }
 }
 
+static void tcg_out_vrx_mem(TCGContext *s, S390Opcode opc_vrx,
+TCGReg data, TCGReg base, TCGReg index,
+tcg_target_long ofs, int m3)
+{
+if (ofs < 0 || ofs >= 0x1000) {
+if (ofs >= -0x8 && ofs < 0x8) {
+tcg_out_insn(s, RXY, LAY, TCG_TMP0, base, index, ofs);
+base = TCG_TMP0;
+index = TCG_REG_NONE;
+ofs = 0;
+} else {
+tcg_out_movi(s, TCG_TYPE_PTR, TCG_TMP0, ofs);
+if (index != TCG_REG_NONE) {
+tcg_out_insn(s, RRE, AGR, TCG_TMP0, index);
+}
+index = TCG_TMP0;
+ofs = 0;
+}
+}
+tcg_out_insn_VRX(s, opc_vrx, data, base, index, ofs, m3);
+}
 
 /* load data without address translation or endianness conversion */
-static inline void tcg_out_ld(TCGContext *s, TCGType type, TCGReg data,
-  TCGReg base, intptr_t ofs)
+static void tcg_out_ld(TCGContext *s, TCGType type, TCGReg data,
+   TCGReg base, intptr_t ofs)
 {
-if (type == TCG_TYPE_I32) {
-tcg_out_mem(s, RX_L, RXY_LY, data, base, TCG_REG_NONE, ofs);
-} else {
-tcg_out_mem(s, 0, RXY_LG, data, base, TCG_REG_NONE, ofs);
+switch (type) {
+case TCG_TYPE_I32:
+if (likely(is_general_reg(data))) {
+tcg_out_mem(s, RX_L, RXY_LY, data, base, TCG_REG_NONE, ofs);
+break;
+}
+tcg_out_vrx_mem(s, VRX_VLLEZ, data, base, TCG_REG_NONE, ofs, MO_32);
+break;
+
+case TCG_TYPE_I64:
+if (likely(is_general_reg(data))) {
+tcg_out_mem(s, 0, RXY_LG, data, base, TCG_REG_NONE, ofs);
+break;
+}
+/* fallthru */
+
+case TCG_TYPE_V64:
+tcg_out_vrx_mem(s, VRX_VLLEZ, data, base, TCG_REG_NONE, ofs, MO_64);
+break;
+
+case TCG_TYPE_V128:
+/* Hint quadword aligned.  */
+tcg_out_vrx_mem(s, VRX_VL, data, base, TCG_REG_NONE, ofs, 4);
+break;
+
+default:
+g_assert_not_reached();
 }
 }
 
-static inline void tcg_out_st(TCGContext *s, TCGType type, TCGReg data,
-  TCGReg base, intptr_t ofs)
+static void tcg_out_st(TCGContext *s, TCGType type, TCGReg data,
+   TCGReg base, intptr_t ofs)
 {
-if (type == TCG_TYPE_I32) {
-tcg_out_mem(s, RX_ST, RXY_STY, data, base, TCG_REG_NONE, ofs);
-} else {
-tcg_out_mem(s, 0, RXY_STG, data, base, TCG_REG_NONE, ofs);
+switch (type) {
+case TCG_TYPE_I32:

[PATCH v5 08/16] tcg/s390x: Implement tcg_out_dup*_vec

[Richard Henderson]

Signed-off-by: Richard Henderson 
---
 tcg/s390x/tcg-target.c.inc | 122 -
 1 file changed, 119 insertions(+), 3 deletions(-)

diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc
index 76061bfd80..b9de4dc821 100644
--- a/tcg/s390x/tcg-target.c.inc
+++ b/tcg/s390x/tcg-target.c.inc
@@ -265,13 +265,20 @@ typedef enum S390Opcode {
 RX_STC  = 0x42,
 RX_STH  = 0x40,
 
+VRIa_VGBM   = 0xe744,
+VRIa_VREPI  = 0xe745,
+VRIb_VGM= 0xe746,
+VRIc_VREP   = 0xe74d,
+
 VRRa_VLR= 0xe756,
+VRRf_VLVGP  = 0xe762,
 
 VRSb_VLVG   = 0xe722,
 VRSc_VLGV   = 0xe721,
 
 VRX_VL  = 0xe706,
 VRX_VLLEZ   = 0xe704,
+VRX_VLREP   = 0xe705,
 VRX_VST = 0xe70e,
 VRX_VSTEF   = 0xe70b,
 VRX_VSTEG   = 0xe70a,
@@ -563,6 +570,34 @@ static int RXB(TCGReg v1, TCGReg v2, TCGReg v3, TCGReg v4)
  | ((v4 & 0x10) << (4 + 0));
 }
 
+static void tcg_out_insn_VRIa(TCGContext *s, S390Opcode op,
+  TCGReg v1, uint16_t i2, int m3)
+{
+tcg_debug_assert(is_vector_reg(v1));
+tcg_out16(s, (op & 0xff00) | ((v1 & 0xf) << 4));
+tcg_out16(s, i2);
+tcg_out16(s, (op & 0x00ff) | RXB(v1, 0, 0, 0) | (m3 << 12));
+}
+
+static void tcg_out_insn_VRIb(TCGContext *s, S390Opcode op,
+  TCGReg v1, uint8_t i2, uint8_t i3, int m4)
+{
+tcg_debug_assert(is_vector_reg(v1));
+tcg_out16(s, (op & 0xff00) | ((v1 & 0xf) << 4));
+tcg_out16(s, (i2 << 8) | (i3 & 0xff));
+tcg_out16(s, (op & 0x00ff) | RXB(v1, 0, 0, 0) | (m4 << 12));
+}
+
+static void tcg_out_insn_VRIc(TCGContext *s, S390Opcode op,
+  TCGReg v1, uint16_t i2, TCGReg v3, int m4)
+{
+tcg_debug_assert(is_vector_reg(v1));
+tcg_debug_assert(is_vector_reg(v3));
+tcg_out16(s, (op & 0xff00) | ((v1 & 0xf) << 4) | (v3 & 0xf));
+tcg_out16(s, i2);
+tcg_out16(s, (op & 0x00ff) | RXB(v1, 0, v3, 0) | (m4 << 12));
+}
+
 static void tcg_out_insn_VRRa(TCGContext *s, S390Opcode op,
   TCGReg v1, TCGReg v2, int m3)
 {
@@ -572,6 +607,17 @@ static void tcg_out_insn_VRRa(TCGContext *s, S390Opcode op,
 tcg_out32(s, (op & 0x00ff) | RXB(v1, v2, 0, 0) | (m3 << 12));
 }
 
+static void tcg_out_insn_VRRf(TCGContext *s, S390Opcode op,
+  TCGReg v1, TCGReg r2, TCGReg r3)
+{
+tcg_debug_assert(is_vector_reg(v1));
+tcg_debug_assert(is_general_reg(r2));
+tcg_debug_assert(is_general_reg(r3));
+tcg_out16(s, (op & 0xff00) | ((v1 & 0xf) << 4) | r2);
+tcg_out16(s, r3 << 12);
+tcg_out16(s, (op & 0x00ff) | RXB(v1, 0, 0, 0));
+}
+
 static void tcg_out_insn_VRSb(TCGContext *s, S390Opcode op, TCGReg v1,
   intptr_t d2, TCGReg b2, TCGReg r3, int m4)
 {
@@ -2501,19 +2547,89 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode 
opc,
 static bool tcg_out_dup_vec(TCGContext *s, TCGType type, unsigned vece,
 TCGReg dst, TCGReg src)
 {
-g_assert_not_reached();
+if (is_general_reg(src)) {
+/* Replicate general register into two MO_64. */
+tcg_out_insn(s, VRRf, VLVGP, dst, src, src);
+if (vece == MO_64) {
+return true;
+}
+}
+
+/*
+ * Recall that the "standard" integer, within a vector, is the
+ * rightmost element of the leftmost doubleword, a-la VLLEZ.
+ */
+tcg_out_insn(s, VRIc, VREP, dst, (8 >> vece) - 1, src, vece);
+return true;
 }
 
 static bool tcg_out_dupm_vec(TCGContext *s, TCGType type, unsigned vece,
  TCGReg dst, TCGReg base, intptr_t offset)
 {
-g_assert_not_reached();
+tcg_out_vrx_mem(s, VRX_VLREP, dst, base, TCG_REG_NONE, offset, vece);
+return true;
 }
 
 static void tcg_out_dupi_vec(TCGContext *s, TCGType type, unsigned vece,
  TCGReg dst, int64_t val)
 {
-g_assert_not_reached();
+int i, mask, msb, lsb;
+
+/* Look for int16_t elements.  */
+if (vece <= MO_16 ||
+(vece == MO_32 ? (int32_t)val : val) == (int16_t)val) {
+tcg_out_insn(s, VRIa, VREPI, dst, val, vece);
+return;
+}
+
+/* Look for bit masks.  */
+if (vece == MO_32) {
+if (risbg_mask((int32_t)val)) {
+/* Handle wraparound by swapping msb and lsb.  */
+if ((val & 0x8001u) == 0x8001u) {
+msb = 32 - ctz32(~val);
+lsb = clz32(~val) - 1;
+} else {
+msb = clz32(val);
+lsb = 31 - ctz32(val);
+}
+tcg_out_insn(s, VRIb, VGM, dst, lsb, msb, MO_32);
+return;
+}
+} else {
+if (risbg_mask(val)) {
+/* Handle wraparound by swapping msb and lsb.  */
+if ((val & 0x8001ull) == 0x8001ull) {
+/* Handle wraparound by swapping msb and lsb.  */
+msb = 64 - ctz64(~val);
+

[PATCH v5 04/16] tcg/s390x: Merge TCG_AREG0 and TCG_REG_CALL_STACK into TCGReg

[Richard Henderson]

They are rightly values in the same enumeration.

Reviewed-by: David Hildenbrand 
Signed-off-by: Richard Henderson 
---
 tcg/s390x/tcg-target.h | 28 +++-
 1 file changed, 7 insertions(+), 21 deletions(-)

diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h
index 18d0d330e6..0174357f1b 100644
--- a/tcg/s390x/tcg-target.h
+++ b/tcg/s390x/tcg-target.h
@@ -32,22 +32,13 @@
 #define MAX_CODE_GEN_BUFFER_SIZE  (3 * GiB)
 
 typedef enum TCGReg {
-TCG_REG_R0 = 0,
-TCG_REG_R1,
-TCG_REG_R2,
-TCG_REG_R3,
-TCG_REG_R4,
-TCG_REG_R5,
-TCG_REG_R6,
-TCG_REG_R7,
-TCG_REG_R8,
-TCG_REG_R9,
-TCG_REG_R10,
-TCG_REG_R11,
-TCG_REG_R12,
-TCG_REG_R13,
-TCG_REG_R14,
-TCG_REG_R15
+TCG_REG_R0,  TCG_REG_R1,  TCG_REG_R2,  TCG_REG_R3,
+TCG_REG_R4,  TCG_REG_R5,  TCG_REG_R6,  TCG_REG_R7,
+TCG_REG_R8,  TCG_REG_R9,  TCG_REG_R10, TCG_REG_R11,
+TCG_REG_R12, TCG_REG_R13, TCG_REG_R14, TCG_REG_R15,
+
+TCG_AREG0 = TCG_REG_R10,
+TCG_REG_CALL_STACK = TCG_REG_R15
 } TCGReg;
 
 #define TCG_TARGET_NB_REGS 16
@@ -138,7 +129,6 @@ extern uint64_t s390_facilities[1];
 #define TCG_TARGET_HAS_mulsh_i64  0
 
 /* used for function call generation */
-#define TCG_REG_CALL_STACK TCG_REG_R15
 #define TCG_TARGET_STACK_ALIGN 8
 #define TCG_TARGET_CALL_STACK_OFFSET   160
 
@@ -147,10 +137,6 @@ extern uint64_t s390_facilities[1];
 
 #define TCG_TARGET_DEFAULT_MO (TCG_MO_ALL & ~TCG_MO_ST_LD)
 
-enum {
-TCG_AREG0 = TCG_REG_R10,
-};
-
 static inline void tb_target_set_jmp_target(uintptr_t tc_ptr, uintptr_t jmp_rx,
 uintptr_t jmp_rw, uintptr_t addr)
 {
-- 
2.25.1

[PATCH v5 01/16] tcg: Expand usadd/ussub with umin/umax

[Richard Henderson]

For usadd, we only have to consider overflow.  Since ~B + B == -1,
the maximum value for A that saturates is ~B.

For ussub, we only have to consider underflow.  The minimum value
that saturates to 0 from A - B is B.

Signed-off-by: Richard Henderson 
---
 tcg/tcg-op-vec.c | 37 +++--
 1 file changed, 35 insertions(+), 2 deletions(-)

diff --git a/tcg/tcg-op-vec.c b/tcg/tcg-op-vec.c
index 15e026ae49..7705a49c0b 100644
--- a/tcg/tcg-op-vec.c
+++ b/tcg/tcg-op-vec.c
@@ -119,6 +119,18 @@ bool tcg_can_emit_vecop_list(const TCGOpcode *list,
 continue;
 }
 break;
+case INDEX_op_usadd_vec:
+if (tcg_can_emit_vec_op(INDEX_op_umin_vec, type, vece) ||
+tcg_can_emit_vec_op(INDEX_op_cmp_vec, type, vece)) {
+continue;
+}
+break;
+case INDEX_op_ussub_vec:
+if (tcg_can_emit_vec_op(INDEX_op_umax_vec, type, vece) ||
+tcg_can_emit_vec_op(INDEX_op_cmp_vec, type, vece)) {
+continue;
+}
+break;
 case INDEX_op_cmpsel_vec:
 case INDEX_op_smin_vec:
 case INDEX_op_smax_vec:
@@ -603,7 +615,18 @@ void tcg_gen_ssadd_vec(unsigned vece, TCGv_vec r, TCGv_vec 
a, TCGv_vec b)
 
 void tcg_gen_usadd_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b)
 {
-do_op3_nofail(vece, r, a, b, INDEX_op_usadd_vec);
+if (!do_op3(vece, r, a, b, INDEX_op_usadd_vec)) {
+const TCGOpcode *hold_list = tcg_swap_vecop_list(NULL);
+TCGv_vec t = tcg_temp_new_vec_matching(r);
+
+/* usadd(a, b) = min(a, ~b) + b */
+tcg_gen_not_vec(vece, t, b);
+tcg_gen_umin_vec(vece, t, t, a);
+tcg_gen_add_vec(vece, r, r, b);
+
+tcg_temp_free_vec(t);
+tcg_swap_vecop_list(hold_list);
+}
 }
 
 void tcg_gen_sssub_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b)
@@ -613,7 +636,17 @@ void tcg_gen_sssub_vec(unsigned vece, TCGv_vec r, TCGv_vec 
a, TCGv_vec b)
 
 void tcg_gen_ussub_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b)
 {
-do_op3_nofail(vece, r, a, b, INDEX_op_ussub_vec);
+if (!do_op3(vece, r, a, b, INDEX_op_ussub_vec)) {
+const TCGOpcode *hold_list = tcg_swap_vecop_list(NULL);
+TCGv_vec t = tcg_temp_new_vec_matching(r);
+
+/* ussub(a, b) = max(a, b) - b */
+tcg_gen_umax_vec(vece, t, a, b);
+tcg_gen_sub_vec(vece, r, t, b);
+
+tcg_temp_free_vec(t);
+tcg_swap_vecop_list(hold_list);
+}
 }
 
 static void do_minmax(unsigned vece, TCGv_vec r, TCGv_vec a,
-- 
2.25.1

[PATCH] hw/intc: GIC maintenance interrupt not triggered

[Shashi Mallela]

During sbsa acs level 3 testing,it is seen that the GIC
maintenance interrupts are not triggered and the related test
cases failed.On debugging the cause,found that the value of
MISR register (from maintenance_interrupt_state()) was being
passed to qemu_set_irq() as level.Updated logic to set level
to 1 if any of the maintenance interrupt attributes are set.
Confirmed that the GIC maintanence interrupts are triggered and
sbsa acs test cases passed with this change.

Signed-off-by: Shashi Mallela 
---
 hw/intc/arm_gicv3_cpuif.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c
index 462a35f66e..34691d4fe2 100644
--- a/hw/intc/arm_gicv3_cpuif.c
+++ b/hw/intc/arm_gicv3_cpuif.c
@@ -418,7 +418,9 @@ static void gicv3_cpuif_virt_update(GICv3CPUState *cs)
 }
 
 if (cs->ich_hcr_el2 & ICH_HCR_EL2_EN) {
-maintlevel = maintenance_interrupt_state(cs);
+if (maintenance_interrupt_state(cs)) {
+maintlevel = 1;
+}
 }
 
 trace_gicv3_cpuif_virt_set_irqs(gicv3_redist_affid(cs), fiqlevel,
-- 
2.27.0

Re: [PATCH v2 3/3] qapi: deprecate drive-backup

[Markus Armbruster]

Markus Armbruster  writes:

> Vladimir Sementsov-Ogievskiy  writes:
>
>> 08.06.2021 14:12, Markus Armbruster wrote:
>>> Vladimir Sementsov-Ogievskiy  writes:
>>> 
>>> [...]
>>> 
 TODO: We also need to deprecate drive-backup transaction action..
 But union members in QAPI doesn't support 'deprecated' feature. I tried
 to dig a bit, but failed :/ Markus, could you please help with it? At
 least by advice?
>>> 
>>> There are two closely related things in play here: the union branch and
>>> the corresponding enum value.
>>> 
>>> So far, the QAPI schema language doesn't support tacking feature flags
>>> to either.
>>> 
>>> If an enum value is deprecated, any union branches corresponding to it
>>> must also be deprecated (because their use requires using the deprecated
>>> enum value).
>>> 
>>> The converse is not true, but I can't see a use for deprecating a union
>>> branch without also deprecating the enum member.
>>> 
>>> I think we can implement feature flags just for enum members, then
>>> document that 'deprecated' enum value implies corresponding union
>>> branches are also deprecated.
>>> 
>>> I have unfinished patches implementing feature flags for enum members.
>>> 
>>> Since TransactionAction is a simple union, the corresponding enum is
>>> implicit.  We can make it explicit by converting to a flat union.
>>> Simple unions need to die anyway.
>>
>>
>> Does BlockStatsSpecific from qapi/block-core.json a correct example of flat 
>> union you mean? I can make patch to convert TransactionAction to be similar 
>> if that helps (discriminator field should be called "type", yes?).
>
> From docs/devel/qapi-code-gen.txt:
>
> A simple union can always be re-written as a flat union where the base
> class has a single member named 'type', and where each branch of the
> union has a struct with a single member named 'data'.  That is,
>
>  { 'union': 'Simple', 'data': { 'one': 'str', 'two': 'int' } }
>
> is identical on the wire to:
>
>  { 'enum': 'Enum', 'data': ['one', 'two'] }
>  { 'struct': 'Branch1', 'data': { 'data': 'str' } }
>  { 'struct': 'Branch2', 'data': { 'data': 'int' } }
>  { 'union': 'Flat', 'base': { 'type': 'Enum' }, 'discriminator': 'type',
>'data': { 'one': 'Branch1', 'two': 'Branch2' } }
>
> The generated C isn't identical, but adjusting the code using it should
> be straightforward.
>
>>> Does this make sense?
>>> 
>>
>> Yes if it helps)
>>
>> Did you also look at John's 
>> https://gitlab.com/jsnow/qemu/-/commits/hack-deprecate-union-branches/ ?
>
> Not yet.
>
>> I hope you and John will send patches that you have, I'll help with 
>> reviewing (keep me in CC), and finally we'll get the feature.
>
> Sounds like a plan.  I need to get my post-vacation e-mail pileup under
> control first.

Just sent:

Subject: [PATCH RFC 0/5] Subject: [PATCH RFC 0/5] qapi: Add feature flags to 
enum members
Message-Id: <20210915192425.4104210-1-arm...@redhat.com>

Yes, I mangled the subject %-/

[PATCH RFC 4/5] qapi: Implement deprecated-input={reject, crash} for enum values

[Markus Armbruster]

This copies the code implementing the policy from qapi/qmp-dispatch.c
to qapi/qobject-input-visitor.c.  I hope to avoid that in a future
revision.

Signed-off-by: Markus Armbruster 
---
 qapi/compat.json   |  3 ++-
 include/qapi/util.h|  6 +-
 qapi/qapi-visit-core.c | 18 +++---
 scripts/qapi/types.py  | 17 -
 4 files changed, 38 insertions(+), 6 deletions(-)

diff --git a/qapi/compat.json b/qapi/compat.json
index 1d2b76f00c..74a8493d3d 100644
--- a/qapi/compat.json
+++ b/qapi/compat.json
@@ -42,7 +42,8 @@
 # with feature 'deprecated'.  We may want to extend it to cover
 # semantic aspects, CLI, and experimental features.
 #
-# Limitation: not implemented for deprecated enumeration values.
+# Limitation: deprecated-output policy @hide is not implemented for
+# enumeration values.  They behave the same as with policy @accept.
 #
 # @deprecated-input: how to handle deprecated input (default 'accept')
 # @deprecated-output: how to handle deprecated output (default 'accept')
diff --git a/include/qapi/util.h b/include/qapi/util.h
index d7bfb30e25..257c600f99 100644
--- a/include/qapi/util.h
+++ b/include/qapi/util.h
@@ -11,9 +11,13 @@
 #ifndef QAPI_UTIL_H
 #define QAPI_UTIL_H
 
+/* QEnumLookup flags */
+#define QAPI_ENUM_DEPRECATED 1
+
 typedef struct QEnumLookup {
 const char *const *array;
-int size;
+const unsigned char *const flags;
+const int size;
 } QEnumLookup;
 
 const char *qapi_enum_lookup(const QEnumLookup *lookup, int val);
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index 066f77a26d..49136ae88e 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -393,7 +393,7 @@ static bool input_type_enum(Visitor *v, const char *name, 
int *obj,
 const QEnumLookup *lookup, Error **errp)
 {
 int64_t value;
-char *enum_str;
+g_autofree char *enum_str = NULL;
 
 if (!visit_type_str(v, name, &enum_str, errp)) {
 return false;
@@ -402,11 +402,23 @@ static bool input_type_enum(Visitor *v, const char *name, 
int *obj,
 value = qapi_enum_parse(lookup, enum_str, -1, NULL);
 if (value < 0) {
 error_setg(errp, QERR_INVALID_PARAMETER, enum_str);
-g_free(enum_str);
 return false;
 }
 
-g_free(enum_str);
+if (lookup->flags && (lookup->flags[value] & QAPI_ENUM_DEPRECATED)) {
+switch (v->compat_policy.deprecated_input) {
+case COMPAT_POLICY_INPUT_ACCEPT:
+break;
+case COMPAT_POLICY_INPUT_REJECT:
+error_setg(errp, "Deprecated value '%s' disabled by policy",
+   enum_str);
+return false;
+case COMPAT_POLICY_INPUT_CRASH:
+default:
+abort();
+}
+}
+
 *obj = value;
 return true;
 }
diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
index 831294fe42..ab2441adc9 100644
--- a/scripts/qapi/types.py
+++ b/scripts/qapi/types.py
@@ -38,6 +38,8 @@
 def gen_enum_lookup(name: str,
 members: List[QAPISchemaEnumMember],
 prefix: Optional[str] = None) -> str:
+max_index = c_enum_const(name, '_MAX', prefix)
+flags = ''
 ret = mcgen('''
 
 const QEnumLookup %(c_name)s_lookup = {
@@ -52,13 +54,26 @@ def gen_enum_lookup(name: str,
 ''',
  index=index, name=memb.name)
 ret += memb.ifcond.gen_endif()
+if 'deprecated' in (f.name for f in memb.features):
+flags += mcgen('''
+[%(index)s] = QAPI_ENUM_DEPRECATED,
+''',
+   index=index)
+
+if flags:
+ret += mcgen('''
+},
+.flags = (const unsigned char[%(max_index)s]) {
+''',
+ max_index=max_index)
+ret += flags
 
 ret += mcgen('''
 },
 .size = %(max_index)s
 };
 ''',
- max_index=c_enum_const(name, '_MAX', prefix))
+ max_index=max_index)
 return ret
 
 
-- 
2.31.1

[PATCH RFC 3/5] qapi: Move compat policy from QObject to generic visitor

[Markus Armbruster]

The next commit needs to access compat policy from the generic visitor
core.  Move it there from qobject input and output visitor.

Signed-off-by: Markus Armbruster 
---
 include/qapi/qobject-input-visitor.h  |  4 
 include/qapi/qobject-output-visitor.h |  4 
 include/qapi/visitor-impl.h   |  3 +++
 include/qapi/visitor.h|  9 +
 qapi/qapi-visit-core.c|  9 +
 qapi/qmp-dispatch.c   |  4 ++--
 qapi/qobject-input-visitor.c  | 14 +-
 qapi/qobject-output-visitor.c | 14 +-
 8 files changed, 25 insertions(+), 36 deletions(-)

diff --git a/include/qapi/qobject-input-visitor.h 
b/include/qapi/qobject-input-visitor.h
index 8d69388810..95985e25e5 100644
--- a/include/qapi/qobject-input-visitor.h
+++ b/include/qapi/qobject-input-visitor.h
@@ -15,7 +15,6 @@
 #ifndef QOBJECT_INPUT_VISITOR_H
 #define QOBJECT_INPUT_VISITOR_H
 
-#include "qapi/qapi-types-compat.h"
 #include "qapi/visitor.h"
 
 typedef struct QObjectInputVisitor QObjectInputVisitor;
@@ -59,9 +58,6 @@ typedef struct QObjectInputVisitor QObjectInputVisitor;
  */
 Visitor *qobject_input_visitor_new(QObject *obj);
 
-void qobject_input_visitor_set_policy(Visitor *v,
-  CompatPolicyInput deprecated);
-
 /*
  * Create a QObject input visitor for @obj for use with keyval_parse()
  *
diff --git a/include/qapi/qobject-output-visitor.h 
b/include/qapi/qobject-output-visitor.h
index f2a2f92a00..2b1726baf5 100644
--- a/include/qapi/qobject-output-visitor.h
+++ b/include/qapi/qobject-output-visitor.h
@@ -15,7 +15,6 @@
 #define QOBJECT_OUTPUT_VISITOR_H
 
 #include "qapi/visitor.h"
-#include "qapi/qapi-types-compat.h"
 
 typedef struct QObjectOutputVisitor QObjectOutputVisitor;
 
@@ -54,7 +53,4 @@ typedef struct QObjectOutputVisitor QObjectOutputVisitor;
  */
 Visitor *qobject_output_visitor_new(QObject **result);
 
-void qobject_output_visitor_set_policy(Visitor *v,
-   CompatPolicyOutput deprecated);
-
 #endif
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 3b950f6e3d..72b6537bef 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -122,6 +122,9 @@ struct Visitor
 /* Must be set */
 VisitorType type;
 
+/* Optional */
+struct CompatPolicy compat_policy;
+
 /* Must be set for output visitors, optional otherwise. */
 void (*complete)(Visitor *v, void *opaque);
 
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index b3c9ef7a81..dcb96018a9 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -16,6 +16,7 @@
 #define QAPI_VISITOR_H
 
 #include "qapi/qapi-builtin-types.h"
+#include "qapi/qapi-types-compat.h"
 
 /*
  * The QAPI schema defines both a set of C data types, and a QMP wire
@@ -477,6 +478,14 @@ bool visit_deprecated_accept(Visitor *v, const char *name, 
Error **errp);
  */
 bool visit_deprecated(Visitor *v, const char *name);
 
+/*
+ * Set policy for handling deprecated management interfaces.
+ *
+ * Intended use: call visit_set_policy(v, &compat_policy) when
+ * visiting management interface input or output.
+ */
+void visit_set_policy(Visitor *v, CompatPolicy *policy);
+
 /*
  * Visit an enum value.
  *
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index a641adec51..066f77a26d 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -19,6 +19,10 @@
 #include "qapi/visitor-impl.h"
 #include "trace.h"
 
+/* Zero-initialization must result in default policy */
+QEMU_BUILD_BUG_ON(COMPAT_POLICY_INPUT_ACCEPT || COMPAT_POLICY_OUTPUT_ACCEPT);
+
+
 void visit_complete(Visitor *v, void *opaque)
 {
 assert(v->type != VISITOR_OUTPUT || v->complete);
@@ -153,6 +157,11 @@ bool visit_deprecated(Visitor *v, const char *name)
 return true;
 }
 
+void visit_set_policy(Visitor *v, CompatPolicy *policy)
+{
+v->compat_policy = *policy;
+}
+
 bool visit_is_input(Visitor *v)
 {
 return v->type == VISITOR_INPUT;
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 59600210ce..7e943a0af5 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -32,7 +32,7 @@ Visitor *qobject_input_visitor_new_qmp(QObject *obj)
 {
 Visitor *v = qobject_input_visitor_new(obj);
 
-qobject_input_visitor_set_policy(v, compat_policy.deprecated_input);
+visit_set_policy(v, &compat_policy);
 return v;
 }
 
@@ -40,7 +40,7 @@ Visitor *qobject_output_visitor_new_qmp(QObject **result)
 {
 Visitor *v = qobject_output_visitor_new(result);
 
-qobject_output_visitor_set_policy(v, compat_policy.deprecated_output);
+visit_set_policy(v, &compat_policy);
 return v;
 }
 
diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c
index 04b790412e..71b24a4429 100644
--- a/qapi/qobject-input-visitor.c
+++ b/qapi/qobject-input-visitor.c
@@ -14,7 +14,6 @@
 
 #include "qemu/osdep.h"
 #include 
-#include "qapi/compat-policy.h"
 #include "qapi/

[PATCH RFC 2/5] qapi: Add feature flags to enum members

[Markus Armbruster]

This is quite similar to commit 84ab008687 "qapi: Add feature flags to
struct members", only for enums instead of structs.

Signed-off-by: Markus Armbruster 
---
 docs/devel/qapi-code-gen.rst  |  4 +++-
 qapi/compat.json  |  2 ++
 qapi/introspect.json  |  5 -
 scripts/qapi/expr.py  |  3 ++-
 scripts/qapi/introspect.py|  5 +++--
 scripts/qapi/schema.py| 22 +--
 tests/qapi-schema/doc-good.json   |  5 -
 tests/qapi-schema/doc-good.out|  3 +++
 tests/qapi-schema/doc-good.txt|  3 +++
 .../qapi-schema/enum-dict-member-unknown.err  |  2 +-
 tests/qapi-schema/qapi-schema-test.json   |  3 ++-
 tests/qapi-schema/qapi-schema-test.out|  1 +
 tests/qapi-schema/test-qapi.py|  1 +
 13 files changed, 49 insertions(+), 10 deletions(-)

diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
index b2569de486..00334e9fb8 100644
--- a/docs/devel/qapi-code-gen.rst
+++ b/docs/devel/qapi-code-gen.rst
@@ -200,7 +200,9 @@ Syntax::
  '*if': COND,
  '*features': FEATURES }
 ENUM-VALUE = STRING
-   | { 'name': STRING, '*if': COND }
+   | { 'name': STRING,
+   '*if': COND,
+   '*features': FEATURES }
 
 Member 'enum' names the enum type.
 
diff --git a/qapi/compat.json b/qapi/compat.json
index ae3afc22df..1d2b76f00c 100644
--- a/qapi/compat.json
+++ b/qapi/compat.json
@@ -42,6 +42,8 @@
 # with feature 'deprecated'.  We may want to extend it to cover
 # semantic aspects, CLI, and experimental features.
 #
+# Limitation: not implemented for deprecated enumeration values.
+#
 # @deprecated-input: how to handle deprecated input (default 'accept')
 # @deprecated-output: how to handle deprecated output (default 'accept')
 #
diff --git a/qapi/introspect.json b/qapi/introspect.json
index 250748cd95..e1219914c9 100644
--- a/qapi/introspect.json
+++ b/qapi/introspect.json
@@ -162,10 +162,13 @@
 #
 # @name: the member's name, as defined in the QAPI schema.
 #
+# @features: names of features associated with the member, in no
+#particular order.
+#
 # Since: 6.1
 ##
 { 'struct': 'SchemaInfoEnumMember',
-  'data': { 'name': 'str' } }
+  'data': { 'name': 'str', '*features': [ 'str' ] } }
 
 ##
 # @SchemaInfoArray:
diff --git a/scripts/qapi/expr.py b/scripts/qapi/expr.py
index 819ea6ad97..3cb389e875 100644
--- a/scripts/qapi/expr.py
+++ b/scripts/qapi/expr.py
@@ -472,7 +472,7 @@ def check_enum(expr: _JSONObject, info: QAPISourceInfo) -> 
None:
   for m in members]
 for member in members:
 source = "'data' member"
-check_keys(member, info, source, ['name'], ['if'])
+check_keys(member, info, source, ['name'], ['if', 'features'])
 member_name = member['name']
 check_name_is_str(member_name, info, source)
 source = "%s '%s'" % (source, member_name)
@@ -483,6 +483,7 @@ def check_enum(expr: _JSONObject, info: QAPISourceInfo) -> 
None:
  permit_upper=permissive,
  permit_underscore=permissive)
 check_if(member, info, source)
+check_features(member.get('features'), info)
 
 
 def check_struct(expr: _JSONObject, info: QAPISourceInfo) -> None:
diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 6334546363..67c7d89aae 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -275,12 +275,13 @@ def _gen_tree(self, name: str, mtype: str, obj: Dict[str, 
object],
 obj['features'] = self._gen_features(features)
 self._trees.append(Annotated(obj, ifcond, comment))
 
-@staticmethod
-def _gen_enum_member(member: QAPISchemaEnumMember
+def _gen_enum_member(self, member: QAPISchemaEnumMember
  ) -> Annotated[SchemaInfoEnumMember]:
 obj: SchemaInfoEnumMember = {
 'name': member.name,
 }
+if member.features:
+obj['features'] = self._gen_features(member.features)
 return Annotated(obj, member.ifcond)
 
 def _gen_object_member(self, member: QAPISchemaObjectTypeMember
diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
index 004d7095ff..6d5f46509a 100644
--- a/scripts/qapi/schema.py
+++ b/scripts/qapi/schema.py
@@ -708,6 +708,19 @@ def describe(self, info):
 class QAPISchemaEnumMember(QAPISchemaMember):
 role = 'value'
 
+def __init__(self, name, info, ifcond=None, features=None):
+super().__init__(name, info, ifcond)
+for f in features or []:
+assert isinstance(f, QAPISchemaFeature)
+f.set_defined_in(name)
+self.features = features or []
+
+def connect_doc(self, doc):
+super().connect_doc(doc)
+if doc:
+for f in self.features:
+doc.connect_f

[PATCH RFC 5/5] block: Deprecate transaction type drive-backup

[Markus Armbruster]

Several moons ago, Vladimir posted

Subject: [PATCH v2 3/3] qapi: deprecate drive-backup
Date: Wed,  5 May 2021 16:58:03 +0300
Message-Id: <20210505135803.67896-4-vsement...@virtuozzo.com>
https://lists.gnu.org/archive/html/qemu-devel/2021-05/msg01394.html

with this

TODO: We also need to deprecate drive-backup transaction action..
But union members in QAPI doesn't support 'deprecated' feature. I tried
to dig a bit, but failed :/ Markus, could you please help with it? At
least by advice?

This is one way to resolve it.  Sorry it took so long.

John explored another way, namely adding feature flags to union
branches.  Could also be useful, say to add different features to
branches in multiple unions sharing the same tag enum.

Signed-off-by: Markus Armbruster 
---
 qapi/transaction.json | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/qapi/transaction.json b/qapi/transaction.json
index d7fc73d7df..0be6f83f6d 100644
--- a/qapi/transaction.json
+++ b/qapi/transaction.json
@@ -41,6 +41,9 @@
 ##
 # @TransactionActionKind:
 #
+# Features:
+# @deprecated: Member @drive-backup is deprecated.  Use FIXME instead.
+#
 # Since: 6.1
 ##
 { 'enum': 'TransactionActionKind',
@@ -49,7 +52,7 @@
 'block-dirty-bitmap-disable', 'block-dirty-bitmap-merge',
 'blockdev-backup', 'blockdev-snapshot',
 'blockdev-snapshot-internal-sync', 'blockdev-snapshot-sync',
-'drive-backup' ] }
+{ 'name': 'drive-backup', 'features': [ 'deprecated' ] } ] }
 
 ##
 # @AbortWrapper:
-- 
2.31.1

[PATCH RFC 1/5] qapi: Enable enum member introspection to show more than name

[Markus Armbruster]

The next commit will add feature flags to enum members.  There's a
problem, though: query-qmp-schema shows an enum type's members as an
array of member names (SchemaInfoEnum member @values).  If it showed
an array of objects with a name member, we could simply add more
members to these objects.  Since it's just strings, we can't.

I can see three ways to correct this design mistake:

1. Do it the way we should have done it, plus compatibility goo.

   We want a ['SchemaInfoEnumMember'] member in SchemaInfoEnum.  Since
   changing @values would be a compatibility break, add a new member
   @members instead.

   @values is now redundant.  We should be able to get rid of it
   eventually.

   In my testing, output of qemu-system-x86_64's query-qmp-schema
   grows by 11% (18.5KiB).

2. Like 1, but omit "boring" elements of @member, and empty @member.

   @values does not become redundant.  Output of query-qmp-schema
   grows only as we make enum members non-boring.

3. Versioned query-qmp-schema.

   query-qmp-schema provides either @values or @members.  The QMP
   client can select which version it wants.

This commit implements 1. simply because it's the solution I thought
of first.  I'm prepared to implement one of the others if we decide
that's what we want.

Signed-off-by: Markus Armbruster 
---
 qapi/introspect.json   | 20 ++--
 scripts/qapi/introspect.py | 18 ++
 2 files changed, 32 insertions(+), 6 deletions(-)

diff --git a/qapi/introspect.json b/qapi/introspect.json
index 39bd303778..250748cd95 100644
--- a/qapi/introspect.json
+++ b/qapi/introspect.json
@@ -142,14 +142,30 @@
 #
 # Additional SchemaInfo members for meta-type 'enum'.
 #
-# @values: the enumeration type's values, in no particular order.
+# @members: the enum type's members, in no particular order.
+#
+# @values: the enumeration type's member names, in no particular order.
+#  Redundant with @members.  Just for backward compatibility.
 #
 # Values of this type are JSON string on the wire.
 #
 # Since: 2.5
 ##
 { 'struct': 'SchemaInfoEnum',
-  'data': { 'values': ['str'] } }
+  'data': { 'members': [ 'SchemaInfoEnumMember' ],
+'values': ['str'] } }
+
+##
+# @SchemaInfoEnumMember:
+#
+# An object member.
+#
+# @name: the member's name, as defined in the QAPI schema.
+#
+# Since: 6.1
+##
+{ 'struct': 'SchemaInfoEnumMember',
+  'data': { 'name': 'str' } }
 
 ##
 # @SchemaInfoArray:
diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 4c079ee627..6334546363 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -68,6 +68,7 @@
 # TypedDict constructs, so they are broadly typed here as simple
 # Python Dicts.
 SchemaInfo = Dict[str, object]
+SchemaInfoEnumMember = Dict[str, object]
 SchemaInfoObject = Dict[str, object]
 SchemaInfoObjectVariant = Dict[str, object]
 SchemaInfoObjectMember = Dict[str, object]
@@ -274,8 +275,16 @@ def _gen_tree(self, name: str, mtype: str, obj: Dict[str, 
object],
 obj['features'] = self._gen_features(features)
 self._trees.append(Annotated(obj, ifcond, comment))
 
-def _gen_member(self, member: QAPISchemaObjectTypeMember
-) -> Annotated[SchemaInfoObjectMember]:
+@staticmethod
+def _gen_enum_member(member: QAPISchemaEnumMember
+ ) -> Annotated[SchemaInfoEnumMember]:
+obj: SchemaInfoEnumMember = {
+'name': member.name,
+}
+return Annotated(obj, member.ifcond)
+
+def _gen_object_member(self, member: QAPISchemaObjectTypeMember
+   ) -> Annotated[SchemaInfoObjectMember]:
 obj: SchemaInfoObjectMember = {
 'name': member.name,
 'type': self._use_type(member.type)
@@ -305,7 +314,8 @@ def visit_enum_type(self, name: str, info: 
Optional[QAPISourceInfo],
 prefix: Optional[str]) -> None:
 self._gen_tree(
 name, 'enum',
-{'values': [Annotated(m.name, m.ifcond) for m in members]},
+{'members': [self._gen_enum_member(m) for m in members],
+ 'values': [Annotated(m.name, m.ifcond) for m in members]},
 ifcond, features
 )
 
@@ -322,7 +332,7 @@ def visit_object_type_flat(self, name: str, info: 
Optional[QAPISourceInfo],
members: List[QAPISchemaObjectTypeMember],
variants: Optional[QAPISchemaVariants]) -> None:
 obj: SchemaInfoObject = {
-'members': [self._gen_member(m) for m in members]
+'members': [self._gen_object_member(m) for m in members]
 }
 if variants:
 obj['tag'] = variants.tag_member.name
-- 
2.31.1

[PATCH RFC 0/5] Subject: [PATCH RFC 0/5] qapi: Add feature flags to enum members

[Markus Armbruster]

PATCH 1+2 add feature flags to enum members.  Awkward due to an
introspection design mistake; see PATCH 1 for details.  Feedback
welcome, in particular from management application guys.

PATCH 3+4 implement policy deprecated-input={reject,crash} for enum
values.

Policy deprecated-output=hide is not implemented, because we can't
hide a value without hiding the entire member, which is almost
certainly more than the requester of this policy bargained for.
Perhaps we want a new policy deprecated-output=crash to help us catch
unwanted use of deprecated enum values.  Thoughts?

PATCH 5 puts the new feature flags to use.  It makes sense only on top
of Vladimir's deprecation of drive-backup.  See its commit message for
a reference.

Based on my "[PATCH 00/22] qapi: Remove simple unions from the schema
language".

Based-on: Message-Id: <20210913123932.3306639-1-arm...@redhat.com>

Markus Armbruster (5):
  qapi: Enable enum member introspection to show more than name
  qapi: Add feature flags to enum members
  qapi: Move compat policy from QObject to generic visitor
  qapi: Implement deprecated-input={reject,crash} for enum values
  block: Deprecate transaction type drive-backup

 docs/devel/qapi-code-gen.rst  |  4 ++-
 qapi/compat.json  |  3 +++
 qapi/introspect.json  | 23 ++--
 qapi/transaction.json |  5 +++-
 include/qapi/qobject-input-visitor.h  |  4 ---
 include/qapi/qobject-output-visitor.h |  4 ---
 include/qapi/util.h   |  6 -
 include/qapi/visitor-impl.h   |  3 +++
 include/qapi/visitor.h|  9 +++
 qapi/qapi-visit-core.c| 27 ---
 qapi/qmp-dispatch.c   |  4 +--
 qapi/qobject-input-visitor.c  | 14 +-
 qapi/qobject-output-visitor.c | 14 +-
 scripts/qapi/expr.py  |  3 ++-
 scripts/qapi/introspect.py| 19 ++---
 scripts/qapi/schema.py| 22 +--
 scripts/qapi/types.py | 17 +++-
 tests/qapi-schema/doc-good.json   |  5 +++-
 tests/qapi-schema/doc-good.out|  3 +++
 tests/qapi-schema/doc-good.txt|  3 +++
 .../qapi-schema/enum-dict-member-unknown.err  |  2 +-
 tests/qapi-schema/qapi-schema-test.json   |  3 ++-
 tests/qapi-schema/qapi-schema-test.out|  1 +
 tests/qapi-schema/test-qapi.py|  1 +
 24 files changed, 144 insertions(+), 55 deletions(-)

-- 
2.31.1

Re: [PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server

[John Johnson]

> On Sep 15, 2021, at 6:04 AM, Stefan Hajnoczi  wrote:
> 
> On Wed, Sep 15, 2021 at 12:21:10AM +, John Johnson wrote:
>> 
>> 
>>> On Sep 14, 2021, at 6:06 AM, Stefan Hajnoczi  wrote:
>>> 
>>> On Mon, Sep 13, 2021 at 05:23:33PM +, John Johnson wrote:
>> On Sep 9, 2021, at 10:25 PM, John Johnson  
>> wrote:
>>> On Sep 8, 2021, at 11:29 PM, Stefan Hajnoczi  
>>> wrote:
>>> On Thu, Sep 09, 2021 at 05:11:49AM +, John Johnson wrote:
I did look at coroutines, but they seemed to work when the 
 sender
 is triggering the coroutine on send, not when request packets are 
 arriving
 asynchronously to the sends.
>>> 
>>> This can be done with a receiver coroutine. Its job is to be the only
>>> thing that reads vfio-user messages from the socket. A receiver
>>> coroutine reads messages from the socket and wakes up the waiting
>>> coroutine that yielded from vfio_user_send_recv() or
>>> vfio_user_pci_process_req().
>>> 
>>> (Although vfio_user_pci_process_req() could be called directly from the
>>> receiver coroutine, it seems safer to have a separate coroutine that
>>> processes requests so that the receiver isn't blocked in case
>>> vfio_user_pci_process_req() yields while processing a request.)
>>> 
>>> Going back to what you mentioned above, the receiver coroutine does
>>> something like this:
>>> 
>>> if it's a reply
>>>   reply = find_reply(...)
>>>   qemu_coroutine_enter(reply->co) // instead of signalling reply->cv
>>> else
>>>   QSIMPLEQ_INSERT_TAIL(&pending_reqs, request, next);
>>>   if (pending_reqs_was_empty) {
>>>   qemu_coroutine_enter(process_request_co);
>>>   }
>>> 
>>> The pending_reqs queue holds incoming requests that the
>>> process_request_co coroutine processes.
>>> 
>> 
>> 
>>  How do coroutines work across threads?  There can be multiple vCPU
>> threads waiting for replies, and I think the receiver coroutine will be
>> running in the main loop thread.  Where would a vCPU block waiting for
>> a reply?  I think coroutine_yield() returns to its coroutine_enter() 
>> caller
> 
> 
> 
> A vCPU thread holding the BQL can iterate the event loop if it has
> reached a synchronous point that needs to wait for a reply before
> returning. I think we have this situation when a MemoryRegion is
> accessed on the proxy device.
> 
> For example, block/block-backend.c:blk_prw() kicks off a coroutine and
> then runs the event loop until the coroutine finishes:
> 
> Coroutine *co = qemu_coroutine_create(co_entry, &rwco);
> bdrv_coroutine_enter(blk_bs(blk), co);
> BDRV_POLL_WHILE(blk_bs(blk), rwco.ret == NOT_DONE);
> 
> BDRV_POLL_WHILE() boils down to a loop like this:
> 
> while ((cond)) {
>   aio_poll(ctx, true);
> }
> 
 
I think that would make vCPUs sending requests and the
 receiver coroutine all poll on the same socket.  If the “wrong”
 routine reads the message, I’d need a second level of synchronization
 to pass the message to the “right” one.  e.g., if the vCPU coroutine
 reads a request, it needs to pass it to the receiver; if the receiver
 coroutine reads a reply, it needs to pass it to a vCPU.
 
Avoiding this complexity is one of the reasons I went with
 a separate thread that only reads the socket over the mp-qemu model,
 which does have the sender poll, but doesn’t need to handle incoming
 requests.
>>> 
>>> Only one coroutine reads from the socket, the "receiver" coroutine. In a
>>> previous reply I sketched what the receiver does:
>>> 
>>> if it's a reply
>>> reply = find_reply(...)
>>> qemu_coroutine_enter(reply->co) // instead of signalling reply->cv
>>> else
>>> QSIMPLEQ_INSERT_TAIL(&pending_reqs, request, next);
>>> if (pending_reqs_was_empty) {
>>> qemu_coroutine_enter(process_request_co);
>>> }
>>> 
>> 
>>  Sorry, I was assuming when you said the coroutine will block with
>> aio_poll(), you implied it would also read messages from the socket.
> 
> The vCPU thread calls aio_poll() outside the coroutine, similar to the
> block/block-backend.c:blk_prw() example I posted above:
> 
>  Coroutine *co = qemu_coroutine_create(co_entry, &rwco);
>  bdrv_coroutine_enter(blk_bs(blk), co);
>  BDRV_POLL_WHILE(blk_bs(blk), rwco.ret == NOT_DONE);
> 
> (BDRV_POLL_WHILE() is a aio_poll() loop.)
> 
> The coroutine isn't aware of aio_poll(), it just yields when it needs to
> wait.
> 
>>> The qemu_coroutine_enter(reply->co) call re-enters the coroutine that
>>> was created by the vCPU thread. Is this the "second level of
>>> synchronization" that you described? It's very similar to signalling
>>> reply->cv in the existing patch.
>>> 
>> 
>>  Yes, the only difference is it would be woken on each message,
>> even though it doesn

[PATCH v11 07/10] hvf: arm: Implement PSCI handling

[Alexander Graf]

We need to handle PSCI calls. Most of the TCG code works for us,
but we can simplify it to only handle aa64 mode and we need to
handle SUSPEND differently.

This patch takes the TCG code as template and duplicates it in HVF.

To tell the guest that we support PSCI 0.2 now, update the check in
arm_cpu_initfn() as well.

Signed-off-by: Alexander Graf 
Reviewed-by: Sergio Lopez 

---

v6 -> v7:

  - This patch integrates "arm: Set PSCI to 0.2 for HVF"

v7 -> v8:

  - Do not advance for HVC, PC is already updated by hvf
  - Fix checkpatch error

v8 -> v9:

  - Use new hvf_raise_exception() prototype
  - Make cpu_off function void
  - Add comment about return value, use -1 for "not found"
  - Remove cpu_synchronize_state() when halted

v9 -> v10:

  - Only handle PSCI calls for the current conduit
  - Return true/false

v10 -> v11:

  - Inject UDEF on SMC. Will be changed in TCG+HVF in a later patch set.
---
 target/arm/cpu.c|   4 +-
 target/arm/hvf/hvf.c| 138 ++--
 target/arm/hvf/trace-events |   1 +
 3 files changed, 136 insertions(+), 7 deletions(-)

diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 016df2e1e7..e44c301527 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -1096,8 +1096,8 @@ static void arm_cpu_initfn(Object *obj)
 cpu->psci_version = 1; /* By default assume PSCI v0.1 */
 cpu->kvm_target = QEMU_KVM_ARM_TARGET_NONE;
 
-if (tcg_enabled()) {
-cpu->psci_version = 2; /* TCG implements PSCI 0.2 */
+if (tcg_enabled() || hvf_enabled()) {
+cpu->psci_version = 2; /* TCG and HVF implement PSCI 0.2 */
 }
 }
 
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 025280f4c9..829ff3ff3b 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -25,6 +25,7 @@
 #include "hw/irq.h"
 #include "qemu/main-loop.h"
 #include "sysemu/cpus.h"
+#include "arm-powerctl.h"
 #include "target/arm/cpu.h"
 #include "target/arm/internals.h"
 #include "trace/trace-target_arm_hvf.h"
@@ -48,6 +49,8 @@
 #define TMR_CTL_IMASK   (1 << 1)
 #define TMR_CTL_ISTATUS (1 << 2)
 
+static void hvf_wfi(CPUState *cpu);
+
 typedef struct HVFVTimer {
 /* Vtimer value during migration and paused state */
 uint64_t vtimer_val;
@@ -599,6 +602,116 @@ static void hvf_raise_exception(CPUState *cpu, uint32_t 
excp,
 arm_cpu_do_interrupt(cpu);
 }
 
+static void hvf_psci_cpu_off(ARMCPU *arm_cpu)
+{
+int32_t ret = arm_set_cpu_off(arm_cpu->mp_affinity);
+assert(ret == QEMU_ARM_POWERCTL_RET_SUCCESS);
+}
+
+/*
+ * Handle a PSCI call.
+ *
+ * Returns 0 on success
+ * -1 when the PSCI call is unknown,
+ */
+static bool hvf_handle_psci_call(CPUState *cpu)
+{
+ARMCPU *arm_cpu = ARM_CPU(cpu);
+CPUARMState *env = &arm_cpu->env;
+uint64_t param[4] = {
+env->xregs[0],
+env->xregs[1],
+env->xregs[2],
+env->xregs[3]
+};
+uint64_t context_id, mpidr;
+bool target_aarch64 = true;
+CPUState *target_cpu_state;
+ARMCPU *target_cpu;
+target_ulong entry;
+int target_el = 1;
+int32_t ret = 0;
+
+trace_hvf_psci_call(param[0], param[1], param[2], param[3],
+arm_cpu->mp_affinity);
+
+switch (param[0]) {
+case QEMU_PSCI_0_2_FN_PSCI_VERSION:
+ret = QEMU_PSCI_0_2_RET_VERSION_0_2;
+break;
+case QEMU_PSCI_0_2_FN_MIGRATE_INFO_TYPE:
+ret = QEMU_PSCI_0_2_RET_TOS_MIGRATION_NOT_REQUIRED; /* No trusted OS */
+break;
+case QEMU_PSCI_0_2_FN_AFFINITY_INFO:
+case QEMU_PSCI_0_2_FN64_AFFINITY_INFO:
+mpidr = param[1];
+
+switch (param[2]) {
+case 0:
+target_cpu_state = arm_get_cpu_by_id(mpidr);
+if (!target_cpu_state) {
+ret = QEMU_PSCI_RET_INVALID_PARAMS;
+break;
+}
+target_cpu = ARM_CPU(target_cpu_state);
+
+ret = target_cpu->power_state;
+break;
+default:
+/* Everything above affinity level 0 is always on. */
+ret = 0;
+}
+break;
+case QEMU_PSCI_0_2_FN_SYSTEM_RESET:
+qemu_system_reset_request(SHUTDOWN_CAUSE_GUEST_RESET);
+/* QEMU reset and shutdown are async requests, but PSCI
+ * mandates that we never return from the reset/shutdown
+ * call, so power the CPU off now so it doesn't execute
+ * anything further.
+ */
+hvf_psci_cpu_off(arm_cpu);
+break;
+case QEMU_PSCI_0_2_FN_SYSTEM_OFF:
+qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN);
+hvf_psci_cpu_off(arm_cpu);
+break;
+case QEMU_PSCI_0_1_FN_CPU_ON:
+case QEMU_PSCI_0_2_FN_CPU_ON:
+case QEMU_PSCI_0_2_FN64_CPU_ON:
+mpidr = param[1];
+entry = param[2];
+context_id = param[3];
+ret = arm_set_cpu_on(mpidr, entry, context_id,
+ target_el, target_aarch64);
+break;
+case QEMU_PSCI_0_1_FN_CPU_OFF:

[PATCH v11 08/10] arm: Add Hypervisor.framework build target

[Alexander Graf]

Now that we have all logic in place that we need to handle Hypervisor.framework
on Apple Silicon systems, let's add CONFIG_HVF for aarch64 as well so that we
can build it.

Signed-off-by: Alexander Graf 
Reviewed-by: Roman Bolshakov 
Tested-by: Roman Bolshakov  (x86 only)
Reviewed-by: Peter Maydell 
Reviewed-by: Sergio Lopez 

---

v1 -> v2:

  - Fix build on 32bit arm

v3 -> v4:

  - Remove i386-softmmu target

v6 -> v7:

  - Simplify HVF matching logic in meson build file
---
 meson.build| 7 +++
 target/arm/hvf/meson.build | 3 +++
 target/arm/meson.build | 2 ++
 3 files changed, 12 insertions(+)
 create mode 100644 target/arm/hvf/meson.build

diff --git a/meson.build b/meson.build
index baa28d7d62..15ef4d3c41 100644
--- a/meson.build
+++ b/meson.build
@@ -77,6 +77,13 @@ else
 endif
 
 accelerator_targets = { 'CONFIG_KVM': kvm_targets }
+
+if cpu in ['aarch64']
+  accelerator_targets += {
+'CONFIG_HVF': ['aarch64-softmmu']
+  }
+endif
+
 if cpu in ['x86', 'x86_64', 'arm', 'aarch64']
   # i386 emulator provides xenpv machine type for multiple architectures
   accelerator_targets += {
diff --git a/target/arm/hvf/meson.build b/target/arm/hvf/meson.build
new file mode 100644
index 00..855e6cce5a
--- /dev/null
+++ b/target/arm/hvf/meson.build
@@ -0,0 +1,3 @@
+arm_softmmu_ss.add(when: [hvf, 'CONFIG_HVF'], if_true: files(
+  'hvf.c',
+))
diff --git a/target/arm/meson.build b/target/arm/meson.build
index 25a02bf276..50f152214a 100644
--- a/target/arm/meson.build
+++ b/target/arm/meson.build
@@ -60,5 +60,7 @@ arm_softmmu_ss.add(files(
   'psci.c',
 ))
 
+subdir('hvf')
+
 target_arch += {'arm': arm_ss}
 target_softmmu_arch += {'arm': arm_softmmu_ss}
-- 
2.30.1 (Apple Git-130)

[PATCH v11 00/10] hvf: Implement Apple Silicon Support

[Alexander Graf]

Now that Apple Silicon is widely available, people are obviously excited
to try and run virtualized workloads on them, such as Linux and Windows.

This patch set implements a fully functional version to get the ball
going on that. With this applied, I can successfully run both Linux and
Windows as guests. I am not aware of any limitations specific to
Hypervisor.framework apart from:

  - gdbstub debugging (breakpoints)
  - missing GICv3 support
  - Windows will not work due to UDEF SMC implementation

To use hvf support, please make sure to run -M virt,highmem=off to fit
in M1's physical address space limits and use -cpu host.


Enjoy!

Alex

v1 -> v2:

  - New patch: hvf: Actually set SIG_IPI mask
  - New patch: hvf: Introduce hvf vcpu struct
  - New patch: hvf: arm: Mark CPU as dirty on reset
  - Removed patch: hw/arm/virt: Disable highmem when on hypervisor.framework
  - Removed patch: arm: Synchronize CPU on PSCI on
  - Fix build on 32bit arm
  - Merge vcpu kick function patch into ARM enablement
  - Implement WFI handling (allows vCPUs to sleep)
  - Synchronize system registers (fixes OVMF crashes and reboot)
  - Don't always call cpu_synchronize_state()
  - Use more fine grained iothread locking
  - Populate aa64mmfr0 from hardware
  - Make safe to ctrl-C entitlement application

v2 -> v3:

  - Removed patch: hvf: Actually set SIG_IPI mask
  - New patch: hvf: arm: Add support for GICv3
  - New patch: hvf: arm: Implement -cpu host
  - Advance PC on SMC
  - Use cp list interface for sysreg syncs
  - Do not set current_cpu
  - Fix sysreg isread mask
  - Move sysreg handling to functions
  - Remove WFI logic again
  - Revert to global iothread locking

v3 -> v4:

  - Removed patch: hvf: arm: Mark CPU as dirty on reset
  - New patch: hvf: Simplify post reset/init/loadvm hooks
  - Remove i386-softmmu target (meson.build for hvf target)
  - Combine both if statements (PSCI)
  - Use hv.h instead of Hypervisor.h for 10.15 compat
  - Remove manual inclusion of Hypervisor.h in common .c files
  - No longer include Hypervisor.h in arm hvf .c files
  - Remove unused exe_full variable
  - Reuse exe_name variable

v4 -> v5:

  - Use g_free() on destroy

v5 -> v6:

  - Switch SYSREG() macro order to the same as asm intrinsics

v6 -> v7:

  - Already merged: hvf: Add hypervisor entitlement to output binaries
  - Already merged: hvf: x86: Remove unused definitions
  - Patch split: hvf: Move common code out
-> hvf: Move assert_hvf_ok() into common directory
-> hvf: Move vcpu thread functions into common directory
-> hvf: Move cpu functions into common directory
-> hvf: Move hvf internal definitions into common header
-> hvf: Make hvf_set_phys_mem() static
-> hvf: Remove use of hv_uvaddr_t and hv_gpaddr_t
-> hvf: Split out common code on vcpu init and destroy
-> hvf: Use cpu_synchronize_state()
-> hvf: Make synchronize functions static
-> hvf: Remove hvf-accel-ops.h
  - New patch: hvf: arm: Implement PSCI handling
  - New patch: arm: Enable Windows 10 trusted SMCCC boot call
  - New patch: hvf: arm: Handle Windows 10 SMC call
  - Removed patch: "arm: Set PSCI to 0.2 for HVF" (included above)
  - Removed patch: "hvf: arm: Add support for GICv3" (deferred to later)
  - Remove osdep.h include from hvf_int.h
  - Synchronize SIMD registers as well
  - Prepend 0x for hex values
  - Convert DPRINTF to trace points
  - Use main event loop (fixes gdbstub issues)
  - Remove PSCI support, inject UDEF on HVC/SMC
  - Change vtimer logic to look at ctl.istatus for vtimer mask sync
  - Add kick callback again (fixes remote CPU notification)
  - Move function define to own header
  - Do not propagate SVE features for HVF
  - Remove stray whitespace change
  - Verify that EL0 and EL1 do not allow AArch32 mode
  - Only probe host CPU features once
  - Move WFI into function
  - Improve comment wording
  - Simplify HVF matching logic in meson build file

v7 -> v8:

  - checkpatch fixes
  - Do not advance for HVC, PC is already updated by hvf
(fixes Linux boot)

v8 -> v9:

  - [Merged] hvf: Move assert_hvf_ok() into common directory
  - [Merged] hvf: Move vcpu thread functions into common directory
  - [Merged] hvf: Move cpu functions into common directory
  - [Merged] hvf: Move hvf internal definitions into common header
  - [Merged] hvf: Make hvf_set_phys_mem() static
  - [Merged] hvf: Remove use of hv_uvaddr_t and hv_gpaddr_t
  - [Merged] hvf: Split out common code on vcpu init and destroy
  - [Merged] hvf: Use cpu_synchronize_state()
  - [Merged] hvf: Make synchronize functions static
  - [Merged] hvf: Remove hvf-accel-ops.h
  - [Merged] hvf: Introduce hvf vcpu struct
  - [Merged] hvf: Simplify post reset/init/loadvm hooks
  - [Dropped] arm: Enable Windows 10 trusted SMCCC boot call
  - [Dropped] hvf: arm: Handle Windows 10 SMC call
  - [New] arm: Move PMC register definitions to cpu.h
  - [New] hvf: Add execute to dirty log permission bitmap
  - [New] hvf: Introduce hvf_arch_init() callback

Re: [PATCH v4 00/30] accel: Move has_work() from SysemuCPUOps to AccelOpsClass

[Richard Henderson]

On 9/12/21 10:27 AM, Philippe Mathieu-Daudé wrote:

Philippe Mathieu-Daudé (30):
   accel/tcg: Restrict cpu_handle_halt() to sysemu
   hw/core: Restrict cpu_has_work() to sysemu
   hw/core: Un-inline cpu_has_work()
   sysemu: Introduce AccelOpsClass::has_work()
   accel/kvm: Implement AccelOpsClass::has_work()
   accel/whpx: Implement AccelOpsClass::has_work()
   accel/tcg: Implement AccelOpsClass::has_work() as stub
   target/alpha: Restrict has_work() handler to sysemu
   target/arm: Restrict has_work() handler to sysemu and TCG
   target/avr: Restrict has_work() handler to sysemu
   target/cris: Restrict has_work() handler to sysemu
   target/hexagon: Remove unused has_work() handler
   target/hppa: Restrict has_work() handler to sysemu
   target/i386: Restrict has_work() handler to sysemu and TCG
   target/m68k: Restrict has_work() handler to sysemu
   target/microblaze: Restrict has_work() handler to sysemu
   target/mips: Restrict has_work() handler to sysemu and TCG
   target/nios2: Restrict has_work() handler to sysemu
   target/openrisc: Restrict has_work() handler to sysemu
   target/ppc: Introduce PowerPCCPUClass::has_work()
   target/ppc: Restrict has_work() handlers to sysemu and TCG
   target/riscv: Restrict has_work() handler to sysemu and TCG
   target/rx: Restrict has_work() handler to sysemu
   target/s390x: Restrict has_work() handler to sysemu and TCG
   target/sh4: Restrict has_work() handler to sysemu
   target/sparc: Remove pointless use of CONFIG_TCG definition
   target/sparc: Restrict has_work() handler to sysemu
   target/tricore: Restrict has_work() handler to sysemu
   target/xtensa: Restrict has_work() handler to sysemu
   accel: Add missing AccelOpsClass::has_work() and drop SysemuCPUOps one


Queued, thanks!

r~

[PULL 32/32] qemu-img: Add -F shorthand to convert

[Hanna Reitz]

From: Eric Blake 

Although we have long supported 'qemu-img convert -o
backing_file=foo,backing_fmt=bar', the fact that we have a shortcut -B
for backing_file but none for backing_fmt has made it more likely that
users accidentally run into:

qemu-img: warning: Deprecated use of backing file without explicit backing 
format

when using -B instead of -o.  For similarity with other qemu-img
commands, such as create and compare, add '-F $fmt' as the shorthand
for '-o backing_fmt=$fmt'.  Update iotest 122 for coverage of both
spellings.

Signed-off-by: Eric Blake 
Message-Id: <20210913131735.1948339-1-ebl...@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Maxim Levitsky 
Signed-off-by: Hanna Reitz 
---
 docs/tools/qemu-img.rst |  4 ++--
 qemu-img.c  | 10 +++---
 qemu-img-cmds.hx|  2 +-
 tests/qemu-iotests/122  |  2 +-
 4 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst
index fe6c30d509..d58980aef8 100644
--- a/docs/tools/qemu-img.rst
+++ b/docs/tools/qemu-img.rst
@@ -415,7 +415,7 @@ Command description:
   4
 Error on reading data
 
-.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] 
[--target-is-zero] [--bitmaps [--skip-broken-bitmaps]] [-U] [-C] [-c] [-p] [-q] 
[-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o 
OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-r RATE_LIMIT] [-m 
NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME
+.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] 
[--target-is-zero] [--bitmaps [--skip-broken-bitmaps]] [-U] [-C] [-c] [-p] [-q] 
[-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE [-F 
backing_fmt]] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-r RATE_LIMIT] 
[-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME
 
   Convert the disk image *FILENAME* or a snapshot *SNAPSHOT_PARAM*
   to disk image *OUTPUT_FILENAME* using format *OUTPUT_FMT*. It can
@@ -439,7 +439,7 @@ Command description:
   You can use the *BACKING_FILE* option to force the output image to be
   created as a copy on write image of the specified base image; the
   *BACKING_FILE* should have the same content as the input's base image,
-  however the path, image format, etc may differ.
+  however the path, image format (as given by *BACKING_FMT*), etc may differ.
 
   If a relative path name is given, the backing file is looked up relative to
   the directory containing *OUTPUT_FILENAME*.
diff --git a/qemu-img.c b/qemu-img.c
index e43a71a794..f036a1d428 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -2183,7 +2183,8 @@ static int img_convert(int argc, char **argv)
 int c, bs_i, flags, src_flags = BDRV_O_NO_SHARE;
 const char *fmt = NULL, *out_fmt = NULL, *cache = "unsafe",
*src_cache = BDRV_DEFAULT_CACHE, *out_baseimg = NULL,
-   *out_filename, *out_baseimg_param, *snapshot_name = NULL;
+   *out_filename, *out_baseimg_param, *snapshot_name = NULL,
+   *backing_fmt = NULL;
 BlockDriver *drv = NULL, *proto_drv = NULL;
 BlockDriverInfo bdi;
 BlockDriverState *out_bs;
@@ -2223,7 +2224,7 @@ static int img_convert(int argc, char **argv)
 {"skip-broken-bitmaps", no_argument, 0, OPTION_SKIP_BROKEN},
 {0, 0, 0, 0}
 };
-c = getopt_long(argc, argv, ":hf:O:B:Cco:l:S:pt:T:qnm:WUr:",
+c = getopt_long(argc, argv, ":hf:O:B:CcF:o:l:S:pt:T:qnm:WUr:",
 long_options, NULL);
 if (c == -1) {
 break;
@@ -2253,6 +2254,9 @@ static int img_convert(int argc, char **argv)
 case 'c':
 s.compressed = true;
 break;
+case 'F':
+backing_fmt = optarg;
+break;
 case 'o':
 if (accumulate_options(&options, optarg) < 0) {
 goto fail_getopt;
@@ -2521,7 +2525,7 @@ static int img_convert(int argc, char **argv)
 
 qemu_opt_set_number(opts, BLOCK_OPT_SIZE,
 s.total_sectors * BDRV_SECTOR_SIZE, &error_abort);
-ret = add_old_style_options(out_fmt, opts, out_baseimg, NULL);
+ret = add_old_style_options(out_fmt, opts, out_baseimg, backing_fmt);
 if (ret < 0) {
 goto out;
 }
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index b3620f29e5..4c4d94ab22 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -46,7 +46,7 @@ SRST
 ERST
 
 DEF("convert", img_convert,
-"convert [--object objectdef] [--image-opts] [--target-image-opts] 
[--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t 
cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l 
snapshot_param] [-S sparse_size] [-r rate_limit] [-m num_coroutines] [-W] 
[--salvage] filename [filename2 [...]] output_filename")
+"convert [--object objectdef] [--image-opts] [--target-image-opts] 
[--t

[PATCH v11 09/10] hvf: arm: Add rudimentary PMC support

[Alexander Graf]

We can expose cycle counters on the PMU easily. To be as compatible as
possible, let's do so, but make sure we don't expose any other architectural
counters that we can not model yet.

This allows OSs to work that require PMU support.

Signed-off-by: Alexander Graf 
---
 target/arm/hvf/hvf.c | 179 +++
 1 file changed, 179 insertions(+)

diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 829ff3ff3b..0a4b03c11b 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -42,6 +42,18 @@
 #define SYSREG_OSLSR_EL1  SYSREG(2, 0, 1, 1, 4)
 #define SYSREG_OSDLR_EL1  SYSREG(2, 0, 1, 3, 4)
 #define SYSREG_CNTPCT_EL0 SYSREG(3, 3, 14, 0, 1)
+#define SYSREG_PMCR_EL0   SYSREG(3, 3, 9, 12, 0)
+#define SYSREG_PMUSERENR_EL0  SYSREG(3, 3, 9, 14, 0)
+#define SYSREG_PMCNTENSET_EL0 SYSREG(3, 3, 9, 12, 1)
+#define SYSREG_PMCNTENCLR_EL0 SYSREG(3, 3, 9, 12, 2)
+#define SYSREG_PMINTENCLR_EL1 SYSREG(3, 0, 9, 14, 2)
+#define SYSREG_PMOVSCLR_EL0   SYSREG(3, 3, 9, 12, 3)
+#define SYSREG_PMSWINC_EL0SYSREG(3, 3, 9, 12, 4)
+#define SYSREG_PMSELR_EL0 SYSREG(3, 3, 9, 12, 5)
+#define SYSREG_PMCEID0_EL0SYSREG(3, 3, 9, 12, 6)
+#define SYSREG_PMCEID1_EL0SYSREG(3, 3, 9, 12, 7)
+#define SYSREG_PMCCNTR_EL0SYSREG(3, 3, 9, 13, 0)
+#define SYSREG_PMCCFILTR_EL0  SYSREG(3, 3, 14, 15, 7)
 
 #define WFX_IS_WFE (1 << 0)
 
@@ -723,6 +735,40 @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, 
uint32_t rt)
 val = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) /
   gt_cntfrq_period_ns(arm_cpu);
 break;
+case SYSREG_PMCR_EL0:
+val = env->cp15.c9_pmcr;
+break;
+case SYSREG_PMCCNTR_EL0:
+pmu_op_start(env);
+val = env->cp15.c15_ccnt;
+pmu_op_finish(env);
+break;
+case SYSREG_PMCNTENCLR_EL0:
+val = env->cp15.c9_pmcnten;
+break;
+case SYSREG_PMOVSCLR_EL0:
+val = env->cp15.c9_pmovsr;
+break;
+case SYSREG_PMSELR_EL0:
+val = env->cp15.c9_pmselr;
+break;
+case SYSREG_PMINTENCLR_EL1:
+val = env->cp15.c9_pminten;
+break;
+case SYSREG_PMCCFILTR_EL0:
+val = env->cp15.pmccfiltr_el0;
+break;
+case SYSREG_PMCNTENSET_EL0:
+val = env->cp15.c9_pmcnten;
+break;
+case SYSREG_PMUSERENR_EL0:
+val = env->cp15.c9_pmuserenr;
+break;
+case SYSREG_PMCEID0_EL0:
+case SYSREG_PMCEID1_EL0:
+/* We can't really count anything yet, declare all events invalid */
+val = 0;
+break;
 case SYSREG_OSLSR_EL1:
 val = env->cp15.oslsr_el1;
 break;
@@ -753,6 +799,82 @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, 
uint32_t rt)
 return 0;
 }
 
+static void pmu_update_irq(CPUARMState *env)
+{
+ARMCPU *cpu = env_archcpu(env);
+qemu_set_irq(cpu->pmu_interrupt, (env->cp15.c9_pmcr & PMCRE) &&
+(env->cp15.c9_pminten & env->cp15.c9_pmovsr));
+}
+
+static bool pmu_event_supported(uint16_t number)
+{
+return false;
+}
+
+/* Returns true if the counter (pass 31 for PMCCNTR) should count events using
+ * the current EL, security state, and register configuration.
+ */
+static bool pmu_counter_enabled(CPUARMState *env, uint8_t counter)
+{
+uint64_t filter;
+bool enabled, filtered = true;
+int el = arm_current_el(env);
+
+enabled = (env->cp15.c9_pmcr & PMCRE) &&
+  (env->cp15.c9_pmcnten & (1 << counter));
+
+if (counter == 31) {
+filter = env->cp15.pmccfiltr_el0;
+} else {
+filter = env->cp15.c14_pmevtyper[counter];
+}
+
+if (el == 0) {
+filtered = filter & PMXEVTYPER_U;
+} else if (el == 1) {
+filtered = filter & PMXEVTYPER_P;
+}
+
+if (counter != 31) {
+/*
+ * If not checking PMCCNTR, ensure the counter is setup to an event we
+ * support
+ */
+uint16_t event = filter & PMXEVTYPER_EVTCOUNT;
+if (!pmu_event_supported(event)) {
+return false;
+}
+}
+
+return enabled && !filtered;
+}
+
+static void pmswinc_write(CPUARMState *env, uint64_t value)
+{
+unsigned int i;
+for (i = 0; i < pmu_num_counters(env); i++) {
+/* Increment a counter's count iff: */
+if ((value & (1 << i)) && /* counter's bit is set */
+/* counter is enabled and not filtered */
+pmu_counter_enabled(env, i) &&
+/* counter is SW_INCR */
+(env->cp15.c14_pmevtyper[i] & PMXEVTYPER_EVTCOUNT) == 0x0) {
+/*
+ * Detect if this write causes an overflow since we can't predict
+ * PMSWINC overflows like we can for other events
+ */
+uint32_t new_pmswinc = env->cp15.c14_pmevcntr[i] + 1;
+
+if (env->cp15.c14_pmevcntr[i] & ~new_pmswinc & INT32_MIN) {
+env->cp15.c9_pmovsr |= (1 << i);
+pmu_update_irq(env);
+

[PULL 30/32] qcow2-refcount: check_refcounts_l1(): check reserved bits

[Hanna Reitz]

From: Vladimir Sementsov-Ogievskiy 

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Eric Blake 
Tested-by: Kirill Tkhai 
Reviewed-by: Hanna Reitz 
Message-Id: <20210914122454.141075-10-vsement...@virtuozzo.com>
Signed-off-by: Hanna Reitz 
---
 block/qcow2.h  | 1 +
 block/qcow2-refcount.c | 6 ++
 2 files changed, 7 insertions(+)

diff --git a/block/qcow2.h b/block/qcow2.h
index b8b1093b61..58fd7f1678 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -586,6 +586,7 @@ typedef enum QCow2MetadataOverlap {
 (QCOW2_OL_CACHED | QCOW2_OL_INACTIVE_L2)
 
 #define L1E_OFFSET_MASK 0x00fffe00ULL
+#define L1E_RESERVED_MASK 0x7f0001ffULL
 #define L2E_OFFSET_MASK 0x00fffe00ULL
 #define L2E_STD_RESERVED_MASK 0x3f0001feULL
 
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index 3c89e09599..1c246b9227 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -1902,6 +1902,12 @@ static int check_refcounts_l1(BlockDriverState *bs,
 continue;
 }
 
+if (l1_table[i] & L1E_RESERVED_MASK) {
+fprintf(stderr, "ERROR found L1 entry with reserved bits set: "
+"%" PRIx64 "\n", l1_table[i]);
+res->corruptions++;
+}
+
 l2_offset = l1_table[i] & L1E_OFFSET_MASK;
 
 /* Mark L2 table as used */
-- 
2.31.1

[PATCH v11 06/10] hvf: arm: Implement -cpu host

[Alexander Graf]

Now that we have working system register sync, we push more target CPU
properties into the virtual machine. That might be useful in some
situations, but is not the typical case that users want.

So let's add a -cpu host option that allows them to explicitly pass all
CPU capabilities of their host CPU into the guest.

Signed-off-by: Alexander Graf 
Acked-by: Roman Bolshakov 
Reviewed-by: Sergio Lopez 

---

v6 -> v7:

  - Move function define to own header
  - Do not propagate SVE features for HVF
  - Remove stray whitespace change
  - Verify that EL0 and EL1 do not allow AArch32 mode
  - Only probe host CPU features once

v8 -> v9:

  - Zero-initialize host_isar
  - Use M1 SCTLR reset value

v9 -> v10:

  - Fail -cpu host class creation gracefully
  - Adjust error message on -cpu host realize failure
  - Extend SCTLR comment that hvf returns 0 as default value
  - Return true/false
  - Report errors lazily
---
 target/arm/cpu.c | 13 +--
 target/arm/cpu.h |  2 +
 target/arm/hvf/hvf.c | 91 
 target/arm/hvf_arm.h | 19 +
 target/arm/kvm_arm.h |  2 -
 5 files changed, 121 insertions(+), 6 deletions(-)
 create mode 100644 target/arm/hvf_arm.h

diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index ba0741b20e..016df2e1e7 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -39,6 +39,7 @@
 #include "sysemu/tcg.h"
 #include "sysemu/hw_accel.h"
 #include "kvm_arm.h"
+#include "hvf_arm.h"
 #include "disas/capstone.h"
 #include "fpu/softfloat.h"
 
@@ -1400,8 +1401,8 @@ static void arm_cpu_realizefn(DeviceState *dev, Error 
**errp)
  * this is the first point where we can report it.
  */
 if (cpu->host_cpu_probe_failed) {
-if (!kvm_enabled()) {
-error_setg(errp, "The 'host' CPU type can only be used with KVM");
+if (!kvm_enabled() && !hvf_enabled()) {
+error_setg(errp, "The 'host' CPU type can only be used with KVM or 
HVF");
 } else {
 error_setg(errp, "Failed to retrieve host CPU features");
 }
@@ -2061,15 +2062,19 @@ static void arm_cpu_class_init(ObjectClass *oc, void 
*data)
 #endif /* CONFIG_TCG */
 }
 
-#ifdef CONFIG_KVM
+#if defined(CONFIG_KVM) || defined(CONFIG_HVF)
 static void arm_host_initfn(Object *obj)
 {
 ARMCPU *cpu = ARM_CPU(obj);
 
+#ifdef CONFIG_KVM
 kvm_arm_set_cpu_features_from_host(cpu);
 if (arm_feature(&cpu->env, ARM_FEATURE_AARCH64)) {
 aarch64_add_sve_properties(obj);
 }
+#else
+hvf_arm_set_cpu_features_from_host(cpu);
+#endif
 arm_cpu_post_init(obj);
 }
 
@@ -2129,7 +2134,7 @@ static void arm_cpu_register_types(void)
 {
 type_register_static(&arm_cpu_type_info);
 
-#ifdef CONFIG_KVM
+#if defined(CONFIG_KVM) || defined(CONFIG_HVF)
 type_register_static(&host_arm_cpu_type_info);
 #endif
 }
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 09d9027734..3ed03391a8 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -3015,6 +3015,8 @@ bool write_cpustate_to_list(ARMCPU *cpu, bool kvm_sync);
 #define ARM_CPU_TYPE_NAME(name) (name ARM_CPU_TYPE_SUFFIX)
 #define CPU_RESOLVING_TYPE TYPE_ARM_CPU
 
+#define TYPE_ARM_HOST_CPU "host-" TYPE_ARM_CPU
+
 #define cpu_signal_handler cpu_arm_signal_handler
 #define cpu_list arm_cpu_list
 
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 49f265cc08..025280f4c9 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -17,6 +17,7 @@
 #include "sysemu/hvf.h"
 #include "sysemu/hvf_int.h"
 #include "sysemu/hw_accel.h"
+#include "hvf_arm.h"
 
 #include 
 
@@ -54,6 +55,16 @@ typedef struct HVFVTimer {
 
 static HVFVTimer vtimer;
 
+typedef struct ARMHostCPUFeatures {
+ARMISARegisters isar;
+uint64_t features;
+uint64_t midr;
+uint32_t reset_sctlr;
+const char *dtb_compatible;
+} ARMHostCPUFeatures;
+
+static ARMHostCPUFeatures arm_host_cpu_features;
+
 struct hvf_reg_match {
 int reg;
 uint64_t offset;
@@ -416,6 +427,86 @@ static uint64_t hvf_get_reg(CPUState *cpu, int rt)
 return val;
 }
 
+static bool hvf_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
+{
+ARMISARegisters host_isar = {};
+const struct isar_regs {
+int reg;
+uint64_t *val;
+} regs[] = {
+{ HV_SYS_REG_ID_AA64PFR0_EL1, &host_isar.id_aa64pfr0 },
+{ HV_SYS_REG_ID_AA64PFR1_EL1, &host_isar.id_aa64pfr1 },
+{ HV_SYS_REG_ID_AA64DFR0_EL1, &host_isar.id_aa64dfr0 },
+{ HV_SYS_REG_ID_AA64DFR1_EL1, &host_isar.id_aa64dfr1 },
+{ HV_SYS_REG_ID_AA64ISAR0_EL1, &host_isar.id_aa64isar0 },
+{ HV_SYS_REG_ID_AA64ISAR1_EL1, &host_isar.id_aa64isar1 },
+{ HV_SYS_REG_ID_AA64MMFR0_EL1, &host_isar.id_aa64mmfr0 },
+{ HV_SYS_REG_ID_AA64MMFR1_EL1, &host_isar.id_aa64mmfr1 },
+{ HV_SYS_REG_ID_AA64MMFR2_EL1, &host_isar.id_aa64mmfr2 },
+};
+hv_vcpu_t fd;
+hv_return_t r = HV_SUCCESS;
+hv_vcpu_exit_t *exit;
+int i;
+
+ahcf->dtb_compatible = "arm,arm-v8";
+ahcf->feature

[PATCH v11 10/10] arm: tcg: Adhere to SMCCC 1.3 section 5.2

[Alexander Graf]

The SMCCC 1.3 spec section 5.2 says

  The Unknown SMC Function Identifier is a sign-extended value of (-1)
  that is returned in the R0, W0 or X0 registers. An implementation must
  return this error code when it receives:

* An SMC or HVC call with an unknown Function Identifier
* An SMC or HVC call for a removed Function Identifier
* An SMC64/HVC64 call from AArch32 state

To comply with these statements, let's always return -1 when we encounter
an unknown HVC or SMC call.

Signed-off-by: Alexander Graf 

---

v8 -> v9:

  - Remove Windows specifics and just comply with SMCCC spec

v9 -> v10:

  - Fix comment
---
 target/arm/psci.c | 35 ++-
 1 file changed, 6 insertions(+), 29 deletions(-)

diff --git a/target/arm/psci.c b/target/arm/psci.c
index 6709e28013..b279c0b9a4 100644
--- a/target/arm/psci.c
+++ b/target/arm/psci.c
@@ -27,15 +27,13 @@
 
 bool arm_is_psci_call(ARMCPU *cpu, int excp_type)
 {
-/* Return true if the r0/x0 value indicates a PSCI call and
- * the exception type matches the configured PSCI conduit. This is
- * called before the SMC/HVC instruction is executed, to decide whether
- * we should treat it as a PSCI call or with the architecturally
+/*
+ * Return true if the exception type matches the configured PSCI conduit.
+ * This is called before the SMC/HVC instruction is executed, to decide
+ * whether we should treat it as a PSCI call or with the architecturally
  * defined behaviour for an SMC or HVC (which might be UNDEF or trap
  * to EL2 or to EL3).
  */
-CPUARMState *env = &cpu->env;
-uint64_t param = is_a64(env) ? env->xregs[0] : env->regs[0];
 
 switch (excp_type) {
 case EXCP_HVC:
@@ -52,27 +50,7 @@ bool arm_is_psci_call(ARMCPU *cpu, int excp_type)
 return false;
 }
 
-switch (param) {
-case QEMU_PSCI_0_2_FN_PSCI_VERSION:
-case QEMU_PSCI_0_2_FN_MIGRATE_INFO_TYPE:
-case QEMU_PSCI_0_2_FN_AFFINITY_INFO:
-case QEMU_PSCI_0_2_FN64_AFFINITY_INFO:
-case QEMU_PSCI_0_2_FN_SYSTEM_RESET:
-case QEMU_PSCI_0_2_FN_SYSTEM_OFF:
-case QEMU_PSCI_0_1_FN_CPU_ON:
-case QEMU_PSCI_0_2_FN_CPU_ON:
-case QEMU_PSCI_0_2_FN64_CPU_ON:
-case QEMU_PSCI_0_1_FN_CPU_OFF:
-case QEMU_PSCI_0_2_FN_CPU_OFF:
-case QEMU_PSCI_0_1_FN_CPU_SUSPEND:
-case QEMU_PSCI_0_2_FN_CPU_SUSPEND:
-case QEMU_PSCI_0_2_FN64_CPU_SUSPEND:
-case QEMU_PSCI_0_1_FN_MIGRATE:
-case QEMU_PSCI_0_2_FN_MIGRATE:
-return true;
-default:
-return false;
-}
+return true;
 }
 
 void arm_handle_psci_call(ARMCPU *cpu)
@@ -194,10 +172,9 @@ void arm_handle_psci_call(ARMCPU *cpu)
 break;
 case QEMU_PSCI_0_1_FN_MIGRATE:
 case QEMU_PSCI_0_2_FN_MIGRATE:
+default:
 ret = QEMU_PSCI_RET_NOT_SUPPORTED;
 break;
-default:
-g_assert_not_reached();
 }
 
 err:
-- 
2.30.1 (Apple Git-130)

[PULL 28/32] qcow2-refcount: check_refcounts_l2(): check reserved bits

[Hanna Reitz]

From: Vladimir Sementsov-Ogievskiy 

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Eric Blake 
Tested-by: Kirill Tkhai 
Reviewed-by: Hanna Reitz 
Message-Id: <20210914122454.141075-8-vsement...@virtuozzo.com>
[hreitz: Separated `type` declaration from statements]
Signed-off-by: Hanna Reitz 
---
 block/qcow2.h  |  1 +
 block/qcow2-refcount.c | 14 +-
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/block/qcow2.h b/block/qcow2.h
index c0e1e83796..b8b1093b61 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -587,6 +587,7 @@ typedef enum QCow2MetadataOverlap {
 
 #define L1E_OFFSET_MASK 0x00fffe00ULL
 #define L2E_OFFSET_MASK 0x00fffe00ULL
+#define L2E_STD_RESERVED_MASK 0x3f0001feULL
 
 #define REFT_OFFSET_MASK 0xfe00ULL
 
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index 9a5ae3cac4..bdac7b1780 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -1680,10 +1680,22 @@ static int check_refcounts_l2(BlockDriverState *bs, 
BdrvCheckResult *res,
 for (i = 0; i < s->l2_size; i++) {
 uint64_t coffset;
 int csize;
+QCow2ClusterType type;
+
 l2_entry = get_l2_entry(s, l2_table, i);
 l2_bitmap = get_l2_bitmap(s, l2_table, i);
+type = qcow2_get_cluster_type(bs, l2_entry);
+
+if (type != QCOW2_CLUSTER_COMPRESSED) {
+/* Check reserved bits of Standard Cluster Descriptor */
+if (l2_entry & L2E_STD_RESERVED_MASK) {
+fprintf(stderr, "ERROR found l2 entry with reserved bits set: "
+"%" PRIx64 "\n", l2_entry);
+res->corruptions++;
+}
+}
 
-switch (qcow2_get_cluster_type(bs, l2_entry)) {
+switch (type) {
 case QCOW2_CLUSTER_COMPRESSED:
 /* Compressed clusters don't have QCOW_OFLAG_COPIED */
 if (l2_entry & QCOW_OFLAG_COPIED) {
-- 
2.31.1

[PULL 29/32] qcow2-refcount: improve style of check_refcounts_l1()

[Hanna Reitz]

From: Vladimir Sementsov-Ogievskiy 

 - use g_autofree for l1_table
 - better name for size in bytes variable
 - reduce code blocks nesting
 - whitespaces, braces, newlines

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Hanna Reitz 
Message-Id: <20210914122454.141075-9-vsement...@virtuozzo.com>
Signed-off-by: Hanna Reitz 
---
 block/qcow2-refcount.c | 98 +-
 1 file changed, 50 insertions(+), 48 deletions(-)

diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index bdac7b1780..3c89e09599 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -1862,71 +1862,73 @@ static int check_refcounts_l1(BlockDriverState *bs,
   int flags, BdrvCheckMode fix, bool active)
 {
 BDRVQcow2State *s = bs->opaque;
-uint64_t *l1_table = NULL, l2_offset, l1_size2;
+size_t l1_size_bytes = l1_size * L1E_SIZE;
+g_autofree uint64_t *l1_table = NULL;
+uint64_t l2_offset;
 int i, ret;
 
-l1_size2 = l1_size * L1E_SIZE;
+if (!l1_size) {
+return 0;
+}
 
 /* Mark L1 table as used */
 ret = qcow2_inc_refcounts_imrt(bs, res, refcount_table, 
refcount_table_size,
-   l1_table_offset, l1_size2);
+   l1_table_offset, l1_size_bytes);
 if (ret < 0) {
-goto fail;
+return ret;
+}
+
+l1_table = g_try_malloc(l1_size_bytes);
+if (l1_table == NULL) {
+res->check_errors++;
+return -ENOMEM;
 }
 
 /* Read L1 table entries from disk */
-if (l1_size2 > 0) {
-l1_table = g_try_malloc(l1_size2);
-if (l1_table == NULL) {
-ret = -ENOMEM;
-res->check_errors++;
-goto fail;
-}
-ret = bdrv_pread(bs->file, l1_table_offset, l1_table, l1_size2);
-if (ret < 0) {
-fprintf(stderr, "ERROR: I/O error in check_refcounts_l1\n");
-res->check_errors++;
-goto fail;
-}
-for(i = 0;i < l1_size; i++)
-be64_to_cpus(&l1_table[i]);
+ret = bdrv_pread(bs->file, l1_table_offset, l1_table, l1_size_bytes);
+if (ret < 0) {
+fprintf(stderr, "ERROR: I/O error in check_refcounts_l1\n");
+res->check_errors++;
+return ret;
+}
+
+for (i = 0; i < l1_size; i++) {
+be64_to_cpus(&l1_table[i]);
 }
 
 /* Do the actual checks */
-for(i = 0; i < l1_size; i++) {
-l2_offset = l1_table[i];
-if (l2_offset) {
-/* Mark L2 table as used */
-l2_offset &= L1E_OFFSET_MASK;
-ret = qcow2_inc_refcounts_imrt(bs, res,
-   refcount_table, refcount_table_size,
-   l2_offset, s->cluster_size);
-if (ret < 0) {
-goto fail;
-}
+for (i = 0; i < l1_size; i++) {
+if (!l1_table[i]) {
+continue;
+}
 
-/* L2 tables are cluster aligned */
-if (offset_into_cluster(s, l2_offset)) {
-fprintf(stderr, "ERROR l2_offset=%" PRIx64 ": Table is not "
-"cluster aligned; L1 entry corrupted\n", l2_offset);
-res->corruptions++;
-}
+l2_offset = l1_table[i] & L1E_OFFSET_MASK;
 
-/* Process and check L2 entries */
-ret = check_refcounts_l2(bs, res, refcount_table,
- refcount_table_size, l2_offset, flags,
- fix, active);
-if (ret < 0) {
-goto fail;
-}
+/* Mark L2 table as used */
+ret = qcow2_inc_refcounts_imrt(bs, res,
+   refcount_table, refcount_table_size,
+   l2_offset, s->cluster_size);
+if (ret < 0) {
+return ret;
+}
+
+/* L2 tables are cluster aligned */
+if (offset_into_cluster(s, l2_offset)) {
+fprintf(stderr, "ERROR l2_offset=%" PRIx64 ": Table is not "
+"cluster aligned; L1 entry corrupted\n", l2_offset);
+res->corruptions++;
+}
+
+/* Process and check L2 entries */
+ret = check_refcounts_l2(bs, res, refcount_table,
+ refcount_table_size, l2_offset, flags,
+ fix, active);
+if (ret < 0) {
+return ret;
 }
 }
-g_free(l1_table);
-return 0;
 
-fail:
-g_free(l1_table);
-return ret;
+return 0;
 }
 
 /*
-- 
2.31.1

[PATCH v11 04/10] hvf: Add Apple Silicon support

[Alexander Graf]

With Apple Silicon available to the masses, it's a good time to add support
for driving its virtualization extensions from QEMU.

This patch adds all necessary architecture specific code to get basic VMs
working, including save/restore.

Known limitations:

  - WFI handling is missing (follows in later patch)
  - No watchpoint/breakpoint support

Signed-off-by: Alexander Graf 
Reviewed-by: Roman Bolshakov 
Reviewed-by: Sergio Lopez 

---

v1 -> v2:

  - Merge vcpu kick function patch
  - Implement WFI handling (allows vCPUs to sleep)
  - Synchronize system registers (fixes OVMF crashes and reboot)
  - Don't always call cpu_synchronize_state()
  - Use more fine grained iothread locking
  - Populate aa64mmfr0 from hardware

v2 -> v3:

  - Advance PC on SMC
  - Use cp list interface for sysreg syncs
  - Do not set current_cpu
  - Fix sysreg isread mask
  - Move sysreg handling to functions
  - Remove WFI logic again
  - Revert to global iothread locking
  - Use Hypervisor.h on arm, hv.h does not contain aarch64 definitions

v3 -> v4:

  - No longer include Hypervisor.h

v5 -> v6:

  - Swap sysreg definition order. This way we're in line with asm outputs.

v6 -> v7:

  - Remove osdep.h include from hvf_int.h
  - Synchronize SIMD registers as well
  - Prepend 0x for hex values
  - Convert DPRINTF to trace points
  - Use main event loop (fixes gdbstub issues)
  - Remove PSCI support, inject UDEF on HVC/SMC
  - Change vtimer logic to look at ctl.istatus for vtimer mask sync
  - Add kick callback again (fixes remote CPU notification)

v7 -> v8:

  - Fix checkpatch errors

v8 -> v9:

  - Make kick function non-weak
  - Use arm_cpu_do_interrupt()
  - Remove CNTPCT_EL0 write case
  - Inject UDEF on invalid sysreg access
  - Add support for OS locking sysregs
  - Remove PMCCNTR_EL0 handling
  - Print PC on unhandled sysreg trace
  - Sync SP (x31) based on SP_EL0/SP_EL1
  - Fix SPSR_EL1 mapping
  - Only sync known sysregs, assert when syncing fails
  - Improve error message on unhandled ec
  - Move vtimer sync to post-exit (fixes disable corner case from
kvm-unit-tests)
  - Add vtimer offset, migration and pause logic
  - Flush registers only after EXCP checkers (fixes PSCI on race)

v9 -> v10:

  - Return -1 in x0 on unknown SMC/HVC calls

v10 -> v11:

  - Treat SMC as UDEF. A follow-up patch set will try to change behavior
consistently in TCG as well as HVF.
---
 MAINTAINERS |   5 +
 accel/hvf/hvf-accel-ops.c   |   9 +
 include/sysemu/hvf_int.h|  10 +-
 meson.build |   1 +
 target/arm/hvf/hvf.c| 794 
 target/arm/hvf/trace-events |  10 +
 target/i386/hvf/hvf.c   |   5 +
 7 files changed, 833 insertions(+), 1 deletion(-)
 create mode 100644 target/arm/hvf/hvf.c
 create mode 100644 target/arm/hvf/trace-events

diff --git a/MAINTAINERS b/MAINTAINERS
index 6c20634d63..d7915ec128 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -433,6 +433,11 @@ F: accel/accel-*.c
 F: accel/Makefile.objs
 F: accel/stubs/Makefile.objs
 
+Apple Silicon HVF CPUs
+M: Alexander Graf 
+S: Maintained
+F: target/arm/hvf/
+
 X86 HVF CPUs
 M: Cameron Esfahani 
 M: Roman Bolshakov 
diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c
index 65d431868f..4f75927a8e 100644
--- a/accel/hvf/hvf-accel-ops.c
+++ b/accel/hvf/hvf-accel-ops.c
@@ -60,6 +60,10 @@
 
 HVFState *hvf_state;
 
+#ifdef __aarch64__
+#define HV_VM_DEFAULT NULL
+#endif
+
 /* Memory slots */
 
 hvf_slot *hvf_find_overlap_slot(uint64_t start, uint64_t size)
@@ -376,7 +380,11 @@ static int hvf_init_vcpu(CPUState *cpu)
 pthread_sigmask(SIG_BLOCK, NULL, &set);
 sigdelset(&set, SIG_IPI);
 
+#ifdef __aarch64__
+r = hv_vcpu_create(&cpu->hvf->fd, (hv_vcpu_exit_t **)&cpu->hvf->exit, 
NULL);
+#else
 r = hv_vcpu_create((hv_vcpuid_t *)&cpu->hvf->fd, HV_VCPU_DEFAULT);
+#endif
 cpu->vcpu_dirty = 1;
 assert_hvf_ok(r);
 
@@ -452,6 +460,7 @@ static void hvf_accel_ops_class_init(ObjectClass *oc, void 
*data)
 AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
 
 ops->create_vcpu_thread = hvf_start_vcpu_thread;
+ops->kick_vcpu_thread = hvf_kick_vcpu_thread;
 
 ops->synchronize_post_reset = hvf_cpu_synchronize_post_reset;
 ops->synchronize_post_init = hvf_cpu_synchronize_post_init;
diff --git a/include/sysemu/hvf_int.h b/include/sysemu/hvf_int.h
index 0466106d16..7c245c7b11 100644
--- a/include/sysemu/hvf_int.h
+++ b/include/sysemu/hvf_int.h
@@ -11,7 +11,11 @@
 #ifndef HVF_INT_H
 #define HVF_INT_H
 
+#ifdef __aarch64__
+#include 
+#else
 #include 
+#endif
 
 /* hvf_slot flags */
 #define HVF_SLOT_LOG (1 << 0)
@@ -40,11 +44,14 @@ struct HVFState {
 int num_slots;
 
 hvf_vcpu_caps *hvf_caps;
+uint64_t vtimer_offset;
 };
 extern HVFState *hvf_state;
 
 struct hvf_vcpu_state {
-int fd;
+uint64_t fd;
+void *exit;
+bool vtimer_masked;
 };
 
 void assert_hvf_ok(hv_return_t ret);
@@ -55,5 +62,6 @@ int hvf_vcpu_exec(CPUState *);
 hvf_slot *hvf_find_overla

[PATCH v11 02/10] hvf: Add execute to dirty log permission bitmap

[Alexander Graf]

Hvf's permission bitmap during and after dirty logging does not include
the HV_MEMORY_EXEC permission. At least on Apple Silicon, this leads to
instruction faults once dirty logging was enabled.

Add the bit to make it work properly.

Signed-off-by: Alexander Graf 
---
 accel/hvf/hvf-accel-ops.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c
index d1691be989..71cc2fa70f 100644
--- a/accel/hvf/hvf-accel-ops.c
+++ b/accel/hvf/hvf-accel-ops.c
@@ -239,12 +239,12 @@ static void hvf_set_dirty_tracking(MemoryRegionSection 
*section, bool on)
 if (on) {
 slot->flags |= HVF_SLOT_LOG;
 hv_vm_protect((uintptr_t)slot->start, (size_t)slot->size,
-  HV_MEMORY_READ);
+  HV_MEMORY_READ | HV_MEMORY_EXEC);
 /* stop tracking region*/
 } else {
 slot->flags &= ~HVF_SLOT_LOG;
 hv_vm_protect((uintptr_t)slot->start, (size_t)slot->size,
-  HV_MEMORY_READ | HV_MEMORY_WRITE);
+  HV_MEMORY_READ | HV_MEMORY_WRITE | HV_MEMORY_EXEC);
 }
 }
 
-- 
2.30.1 (Apple Git-130)

[PULL 26/32] qcow2-refcount: fix_l2_entry_by_zero(): also zero L2 entry bitmap

[Hanna Reitz]

From: Vladimir Sementsov-Ogievskiy 

We'll reuse the function to fix wrong L2 entry bitmap. Support it now.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Eric Blake 
Reviewed-by: Hanna Reitz 
Message-Id: <20210914122454.141075-6-vsement...@virtuozzo.com>
Signed-off-by: Hanna Reitz 
---
 block/qcow2-refcount.c | 18 +++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index 184b96ad63..f48c5e1b5d 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -1588,7 +1588,8 @@ enum {
 };
 
 /*
- * Fix L2 entry by making it QCOW2_CLUSTER_ZERO_PLAIN.
+ * Fix L2 entry by making it QCOW2_CLUSTER_ZERO_PLAIN (or making all its 
present
+ * subclusters QCOW2_SUBCLUSTER_ZERO_PLAIN).
  *
  * This function decrements res->corruptions on success, so the caller is
  * responsible to increment res->corruptions prior to the call.
@@ -1605,9 +1606,20 @@ static int fix_l2_entry_by_zero(BlockDriverState *bs, 
BdrvCheckResult *res,
 int idx = l2_index * (l2_entry_size(s) / sizeof(uint64_t));
 uint64_t l2e_offset = l2_offset + (uint64_t)l2_index * l2_entry_size(s);
 int ign = active ? QCOW2_OL_ACTIVE_L2 : QCOW2_OL_INACTIVE_L2;
-uint64_t l2_entry = has_subclusters(s) ? 0 : QCOW_OFLAG_ZERO;
 
-set_l2_entry(s, l2_table, l2_index, l2_entry);
+if (has_subclusters(s)) {
+uint64_t l2_bitmap = get_l2_bitmap(s, l2_table, l2_index);
+
+/* Allocated subclusters become zero */
+l2_bitmap |= l2_bitmap << 32;
+l2_bitmap &= QCOW_L2_BITMAP_ALL_ZEROES;
+
+set_l2_bitmap(s, l2_table, l2_index, l2_bitmap);
+set_l2_entry(s, l2_table, l2_index, 0);
+} else {
+set_l2_entry(s, l2_table, l2_index, QCOW_OFLAG_ZERO);
+}
+
 ret = qcow2_pre_write_overlap_check(bs, ign, l2e_offset, l2_entry_size(s),
 false);
 if (metadata_overlap) {
-- 
2.31.1

[PATCH v11 05/10] arm/hvf: Add a WFI handler

[Alexander Graf]

From: Peter Collingbourne 

Sleep on WFI until the VTIMER is due but allow ourselves to be woken
up on IPI.

In this implementation IPI is blocked on the CPU thread at startup and
pselect() is used to atomically unblock the signal and begin sleeping.
The signal is sent unconditionally so there's no need to worry about
races between actually sleeping and the "we think we're sleeping"
state. It may lead to an extra wakeup but that's better than missing
it entirely.

Signed-off-by: Peter Collingbourne 
[agraf: Remove unused 'set' variable, always advance PC on WFX trap,
support vm stop / continue operations and cntv offsets]
Signed-off-by: Alexander Graf 
Acked-by: Roman Bolshakov 
Reviewed-by: Sergio Lopez 

---

v6 -> v7:

  - Move WFI into function
  - Improve comment wording

v8 -> v9:

  - Add support for cntv offsets
  - Improve code readability
---
 accel/hvf/hvf-accel-ops.c |  5 ++-
 include/sysemu/hvf_int.h  |  1 +
 target/arm/hvf/hvf.c  | 76 +++
 3 files changed, 79 insertions(+), 3 deletions(-)

diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c
index 4f75927a8e..93976f4ece 100644
--- a/accel/hvf/hvf-accel-ops.c
+++ b/accel/hvf/hvf-accel-ops.c
@@ -370,15 +370,14 @@ static int hvf_init_vcpu(CPUState *cpu)
 cpu->hvf = g_malloc0(sizeof(*cpu->hvf));
 
 /* init cpu signals */
-sigset_t set;
 struct sigaction sigact;
 
 memset(&sigact, 0, sizeof(sigact));
 sigact.sa_handler = dummy_signal;
 sigaction(SIG_IPI, &sigact, NULL);
 
-pthread_sigmask(SIG_BLOCK, NULL, &set);
-sigdelset(&set, SIG_IPI);
+pthread_sigmask(SIG_BLOCK, NULL, &cpu->hvf->unblock_ipi_mask);
+sigdelset(&cpu->hvf->unblock_ipi_mask, SIG_IPI);
 
 #ifdef __aarch64__
 r = hv_vcpu_create(&cpu->hvf->fd, (hv_vcpu_exit_t **)&cpu->hvf->exit, 
NULL);
diff --git a/include/sysemu/hvf_int.h b/include/sysemu/hvf_int.h
index 7c245c7b11..6545f7cd61 100644
--- a/include/sysemu/hvf_int.h
+++ b/include/sysemu/hvf_int.h
@@ -52,6 +52,7 @@ struct hvf_vcpu_state {
 uint64_t fd;
 void *exit;
 bool vtimer_masked;
+sigset_t unblock_ipi_mask;
 };
 
 void assert_hvf_ok(hv_return_t ret);
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 8fe008dab5..49f265cc08 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -2,6 +2,7 @@
  * QEMU Hypervisor.framework support for Apple Silicon
 
  * Copyright 2020 Alexander Graf 
+ * Copyright 2020 Google LLC
  *
  * This work is licensed under the terms of the GNU GPL, version 2 or later.
  * See the COPYING file in the top-level directory.
@@ -490,6 +491,7 @@ int hvf_arch_init_vcpu(CPUState *cpu)
 
 void hvf_kick_vcpu_thread(CPUState *cpu)
 {
+cpus_kick_thread(cpu);
 hv_vcpus_exit(&cpu->hvf->fd, 1);
 }
 
@@ -608,6 +610,77 @@ static uint64_t hvf_vtimer_val_raw(void)
 return mach_absolute_time() - hvf_state->vtimer_offset;
 }
 
+static uint64_t hvf_vtimer_val(void)
+{
+if (!runstate_is_running()) {
+/* VM is paused, the vtimer value is in vtimer.vtimer_val */
+return vtimer.vtimer_val;
+}
+
+return hvf_vtimer_val_raw();
+}
+
+static void hvf_wait_for_ipi(CPUState *cpu, struct timespec *ts)
+{
+/*
+ * Use pselect to sleep so that other threads can IPI us while we're
+ * sleeping.
+ */
+qatomic_mb_set(&cpu->thread_kicked, false);
+qemu_mutex_unlock_iothread();
+pselect(0, 0, 0, 0, ts, &cpu->hvf->unblock_ipi_mask);
+qemu_mutex_lock_iothread();
+}
+
+static void hvf_wfi(CPUState *cpu)
+{
+ARMCPU *arm_cpu = ARM_CPU(cpu);
+hv_return_t r;
+uint64_t ctl;
+uint64_t cval;
+int64_t ticks_to_sleep;
+uint64_t seconds;
+uint64_t nanos;
+
+if (cpu->interrupt_request & (CPU_INTERRUPT_HARD | CPU_INTERRUPT_FIQ)) {
+/* Interrupt pending, no need to wait */
+return;
+}
+
+r = hv_vcpu_get_sys_reg(cpu->hvf->fd, HV_SYS_REG_CNTV_CTL_EL0, &ctl);
+assert_hvf_ok(r);
+
+if (!(ctl & 1) || (ctl & 2)) {
+/* Timer disabled or masked, just wait for an IPI. */
+hvf_wait_for_ipi(cpu, NULL);
+return;
+}
+
+r = hv_vcpu_get_sys_reg(cpu->hvf->fd, HV_SYS_REG_CNTV_CVAL_EL0, &cval);
+assert_hvf_ok(r);
+
+ticks_to_sleep = cval - hvf_vtimer_val();
+if (ticks_to_sleep < 0) {
+return;
+}
+
+nanos = ticks_to_sleep * gt_cntfrq_period_ns(arm_cpu);
+seconds = nanos / NANOSECONDS_PER_SECOND;
+nanos -= (seconds * NANOSECONDS_PER_SECOND);
+
+/*
+ * Don't sleep for less than the time a context switch would take,
+ * so that we can satisfy fast timer requests on the same CPU.
+ * Measurements on M1 show the sweet spot to be ~2ms.
+ */
+if (!seconds && nanos < (2 * SCALE_MS)) {
+return;
+}
+
+struct timespec ts = { seconds, nanos };
+hvf_wait_for_ipi(cpu, &ts);
+}
+
 static void hvf_sync_vtimer(CPUState *cpu)
 {
 ARMCPU *arm_cpu = ARM_CPU(cpu);
@@ -728,6 +801,9 @@ int hvf_vcpu_exec(CPUState *