Re: [PATCH v5 04/11] target/riscv: Don't set PMP feature in the cpu init

[Bin Meng]

On Fri, May 29, 2020 at 6:22 AM Alistair Francis
 wrote:
>
> The PMP is enabled by default via the "pmp" property so there is no need
> for us to set it in the init function. As all CPUs have PMP support just
> remove the set_feature() call in the CPU init functions.
>
> Signed-off-by: Alistair Francis 
> ---
>  target/riscv/cpu.c | 7 ---
>  1 file changed, 7 deletions(-)
>

Reviewed-by: Bin Meng 

Re: [PATCH v5 03/11] target/riscv: Disable the MMU correctly

[Bin Meng]

On Fri, May 29, 2020 at 6:22 AM Alistair Francis
 wrote:
>
> Previously if we didn't enable the MMU it would be enabled in the
> realize() function anyway. Let's ensure that if we don't want the MMU we
> disable it. We also don't need to enable the MMU as it will be enalbed

typo: enabled

> in realize() by default.
>
> Signed-off-by: Alistair Francis 
> ---
>  target/riscv/cpu.c | 7 +++
>  1 file changed, 3 insertions(+), 4 deletions(-)
>

Reviewed-by: Bin Meng 

[PATCH] msix: add valid.accepts methods to check address

[P J P]

From: Prasad J Pandit 

While doing msi-x mmio operations, a guest may send an address
that leads to an OOB access issue. Add valid.accepts methods to
ensure that ensuing mmio r/w operation don't go beyond regions.

Reported-by: Ren Ding 
Reported-by: Hanqing Zhao 
Reported-by: Anatoly Trosinenko 
Reported-by: Alexander Bulekov 
Signed-off-by: Prasad J Pandit 
---
 hw/pci/msix.c | 20 
 1 file changed, 20 insertions(+)

diff --git a/hw/pci/msix.c b/hw/pci/msix.c
index 29187898f2..d90d66a3b8 100644
--- a/hw/pci/msix.c
+++ b/hw/pci/msix.c
@@ -193,6 +193,15 @@ static void msix_table_mmio_write(void *opaque, hwaddr 
addr,
 msix_handle_mask_update(dev, vector, was_masked);
 }
 
+static bool msix_table_accepts(void *opaque, hwaddr addr, unsigned size,
+bool is_write, MemTxAttrs attrs)
+{
+PCIDevice *dev = opaque;
+uint16_t tbl_size = dev->msix_entries_nr * PCI_MSIX_ENTRY_SIZE;
+
+return dev->msix_table + addr + 4 <= dev->msix_table + tbl_size;
+}
+
 static const MemoryRegionOps msix_table_mmio_ops = {
 .read = msix_table_mmio_read,
 .write = msix_table_mmio_write,
@@ -200,6 +209,7 @@ static const MemoryRegionOps msix_table_mmio_ops = {
 .valid = {
 .min_access_size = 4,
 .max_access_size = 4,
+.accepts = msix_table_accepts
 },
 };
 
@@ -221,6 +231,15 @@ static void msix_pba_mmio_write(void *opaque, hwaddr addr,
 {
 }
 
+static bool msix_pba_accepts(void *opaque, hwaddr addr, unsigned size,
+bool is_write, MemTxAttrs attrs)
+{
+PCIDevice *dev = opaque;
+uint16_t pba_size = QEMU_ALIGN_UP(dev->msix_entries_nr, 64) / 8;
+
+return dev->msix_pba + addr + 4 <= dev->msix_pba + pba_size;
+}
+
 static const MemoryRegionOps msix_pba_mmio_ops = {
 .read = msix_pba_mmio_read,
 .write = msix_pba_mmio_write,
@@ -228,6 +247,7 @@ static const MemoryRegionOps msix_pba_mmio_ops = {
 .valid = {
 .min_access_size = 4,
 .max_access_size = 4,
+.accepts = msix_pba_accepts
 },
 };
 
-- 
2.26.2

Re: [PATCH v4 4/4] docs: deprecated: Update the -bios documentation

[Bin Meng]

On Fri, May 29, 2020 at 6:24 AM Alistair Francis
 wrote:
>
> Update the -bios deprecation documentation to describe the new
> behaviour.
>
> Signed-off-by: Alistair Francis 
> ---
>  docs/system/deprecated.rst | 28 +---
>  1 file changed, 13 insertions(+), 15 deletions(-)
>

Reviewed-by: Bin Meng 

Re: [PATCH v4 3/4] target/riscv: Drop support for ISA spec version 1.09.1

[Bin Meng]

On Fri, May 29, 2020 at 6:24 AM Alistair Francis
 wrote:
>
> The RISC-V ISA spec version 1.09.1 has been deprecated in QEMU since
> 4.1. It's not commonly used so let's remove support for it.
>
> Signed-off-by: Alistair Francis 
> ---
>  docs/system/deprecated.rst|  20 +--
>  target/riscv/cpu.h|   1 -
>  target/riscv/cpu.c|   2 -
>  target/riscv/cpu_helper.c |  82 ---
>  target/riscv/csr.c| 138 --
>  .../riscv/insn_trans/trans_privileged.inc.c   |  18 +--
>  target/riscv/monitor.c|   5 -
>  target/riscv/op_helper.c  |  17 +--
>  8 files changed, 73 insertions(+), 210 deletions(-)
>

Reviewed-by: Bin Meng 

Re: [PATCH v4 2/4] target/riscv: Remove the deprecated CPUs

[Bin Meng]

On Fri, May 29, 2020 at 6:24 AM Alistair Francis
 wrote:
>
> Signed-off-by: Alistair Francis 
> Reviewed-by: Bin Meng 
> ---
>  docs/system/deprecated.rst  | 33 ++---
>  target/riscv/cpu.h  |  7 ---
>  target/riscv/cpu.c  | 28 
>  tests/qtest/machine-none-test.c |  4 ++--
>  4 files changed, 20 insertions(+), 52 deletions(-)
>
> diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst
> index 50927bad74..a6664bfca9 100644
> --- a/docs/system/deprecated.rst
> +++ b/docs/system/deprecated.rst
> @@ -314,21 +314,6 @@ should be used instead of the 1.09.1 version.
>  System emulator CPUS
>  
>
> -RISC-V ISA CPUs (since 4.1)
> -'''
> -
> -The RISC-V cpus with the ISA version in the CPU name have been depcreated. 
> The
> -four CPUs are: ``rv32gcsu-v1.9.1``, ``rv32gcsu-v1.10.0``, 
> ``rv64gcsu-v1.9.1`` and
> -``rv64gcsu-v1.10.0``. Instead the version can be specified via the CPU 
> ``priv_spec``
> -option when using the ``rv32`` or ``rv64`` CPUs.
> -
> -RISC-V ISA CPUs (since 4.1)

Not sure if this is a typo. Should we say: RISC-V no MMU CPUs ? If
not, since the subsection titles are the same, should we merge the
following description to above?

> -'''
> -
> -The RISC-V no MMU cpus have been depcreated. The two CPUs: 
> ``rv32imacu-nommu`` and
> -``rv64imacu-nommu`` should no longer be used. Instead the MMU status can be 
> specified
> -via the CPU ``mmu`` option when using the ``rv32`` or ``rv64`` CPUs.
> -
>  ``compat`` property of server class POWER CPUs (since 5.0)
>  ''
>
> @@ -486,6 +471,24 @@ The ``hub_id`` parameter of ``hostfwd_add`` / 
> ``hostfwd_remove`` (removed in 5.0
>  The ``[hub_id name]`` parameter tuple of the 'hostfwd_add' and
>  'hostfwd_remove' HMP commands has been replaced by ``netdev_id``.
>
> +System emulator CPUS
> +
> +
> +RISC-V ISA CPUs (removed in 5.1)
> +
> +
> +The RISC-V cpus with the ISA version in the CPU name have been removed. The
> +four CPUs are: ``rv32gcsu-v1.9.1``, ``rv32gcsu-v1.10.0``, 
> ``rv64gcsu-v1.9.1`` and
> +``rv64gcsu-v1.10.0``. Instead the version can be specified via the CPU 
> ``priv_spec``
> +option when using the ``rv32`` or ``rv64`` CPUs.
> +
> +RISC-V ISA CPUs (removed in 5.1)
> +
> +
> +The RISC-V no MMU cpus have been removed. The two CPUs: ``rv32imacu-nommu`` 
> and
> +``rv64imacu-nommu`` can no longer be used. Instead the MMU status can be 
> specified
> +via the CPU ``mmu`` option when using the ``rv32`` or ``rv64`` CPUs.
> +
>  System emulator machines
>  
>
> diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h
> index d0e7f5b9c5..76b98d7a33 100644
> --- a/target/riscv/cpu.h
> +++ b/target/riscv/cpu.h
> @@ -40,13 +40,6 @@
>  #define TYPE_RISCV_CPU_SIFIVE_E51   RISCV_CPU_TYPE_NAME("sifive-e51")
>  #define TYPE_RISCV_CPU_SIFIVE_U34   RISCV_CPU_TYPE_NAME("sifive-u34")
>  #define TYPE_RISCV_CPU_SIFIVE_U54   RISCV_CPU_TYPE_NAME("sifive-u54")
> -/* Deprecated */
> -#define TYPE_RISCV_CPU_RV32IMACU_NOMMU  
> RISCV_CPU_TYPE_NAME("rv32imacu-nommu")
> -#define TYPE_RISCV_CPU_RV32GCSU_V1_09_1 
> RISCV_CPU_TYPE_NAME("rv32gcsu-v1.9.1")
> -#define TYPE_RISCV_CPU_RV32GCSU_V1_10_0 
> RISCV_CPU_TYPE_NAME("rv32gcsu-v1.10.0")
> -#define TYPE_RISCV_CPU_RV64IMACU_NOMMU  
> RISCV_CPU_TYPE_NAME("rv64imacu-nommu")
> -#define TYPE_RISCV_CPU_RV64GCSU_V1_09_1 
> RISCV_CPU_TYPE_NAME("rv64gcsu-v1.9.1")
> -#define TYPE_RISCV_CPU_RV64GCSU_V1_10_0 
> RISCV_CPU_TYPE_NAME("rv64gcsu-v1.10.0")
>
>  #define RV32 ((target_ulong)1 << (TARGET_LONG_BITS - 2))
>  #define RV64 ((target_ulong)2 << (TARGET_LONG_BITS - 2))
> diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
> index 059d71f2c7..112f2e3a2f 100644
> --- a/target/riscv/cpu.c
> +++ b/target/riscv/cpu.c
> @@ -135,16 +135,6 @@ static void riscv_base32_cpu_init(Object *obj)
>  set_misa(env, 0);
>  }
>
> -static void rv32gcsu_priv1_09_1_cpu_init(Object *obj)
> -{
> -CPURISCVState *env = _CPU(obj)->env;
> -set_misa(env, RV32 | RVI | RVM | RVA | RVF | RVD | RVC | RVS | RVU);
> -set_priv_version(env, PRIV_VERSION_1_09_1);
> -set_resetvec(env, DEFAULT_RSTVEC);
> -set_feature(env, RISCV_FEATURE_MMU);
> -set_feature(env, RISCV_FEATURE_PMP);
> -}
> -
>  static void rv32gcsu_priv1_10_0_cpu_init(Object *obj)
>  {
>  CPURISCVState *env = _CPU(obj)->env;
> @@ -182,16 +172,6 @@ static void riscv_base64_cpu_init(Object *obj)
>  set_misa(env, 0);
>  }
>
> -static void rv64gcsu_priv1_09_1_cpu_init(Object *obj)
> -{
> -CPURISCVState *env = _CPU(obj)->env;
> -set_misa(env, RV64 | RVI | RVM | RVA | RVF | RVD | RVC | RVS | RVU);
> -set_priv_version(env, PRIV_VERSION_1_09_1);
> -set_resetvec(env, DEFAULT_RSTVEC);
> -set_feature(env, RISCV_FEATURE_MMU);
> -

[PATCH] migration: Count new_dirty instead of real_dirty

[Keqian Zhu]

DIRTY_LOG_INITIALLY_ALL_SET feature is on the queue. This fixs the
dirty rate calculation for this feature. After introducing this
feature, real_dirty_pages is equal to total memory size at begining.
This causing wrong dirty rate and false positive throttling.

BTW, real dirty rate is not suitable and not very accurate.

1. For not suitable: We mainly concern on the relationship between
   dirty rate and network bandwidth. Net increasement of dirty pages
   makes more sense.
2. For not very accurate: With manual dirty log clear, some dirty pages
   will be cleared during each peroid, our "real dirty rate" is less
   than real "real dirty rate".

Signed-off-by: Keqian Zhu 
---
 include/exec/ram_addr.h | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
index 5e59a3d8d7..af9677e291 100644
--- a/include/exec/ram_addr.h
+++ b/include/exec/ram_addr.h
@@ -443,7 +443,7 @@ static inline
 uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
ram_addr_t start,
ram_addr_t length,
-   uint64_t *real_dirty_pages)
+   uint64_t *accu_dirty_pages)
 {
 ram_addr_t addr;
 unsigned long word = BIT_WORD((start + rb->offset) >> TARGET_PAGE_BITS);
@@ -469,7 +469,6 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
 if (src[idx][offset]) {
 unsigned long bits = atomic_xchg([idx][offset], 0);
 unsigned long new_dirty;
-*real_dirty_pages += ctpopl(bits);
 new_dirty = ~dest[k];
 dest[k] |= bits;
 new_dirty &= bits;
@@ -502,7 +501,6 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
 start + addr + offset,
 TARGET_PAGE_SIZE,
 DIRTY_MEMORY_MIGRATION)) {
-*real_dirty_pages += 1;
 long k = (start + addr) >> TARGET_PAGE_BITS;
 if (!test_and_set_bit(k, dest)) {
 num_dirty++;
@@ -511,6 +509,7 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
 }
 }
 
+*accu_dirty_pages += num_dirty;
 return num_dirty;
 }
 #endif
-- 
2.19.1

Re: [PATCH v1 11/12] util: Added tsan annotate for thread name.

[Emilio G. Cota]

On Fri, May 29, 2020 at 09:23:41 -0400, Robert Foley wrote:
> This allows us to see the name of the thread in tsan
> warning reports such as this:
> 
>   Thread T7 'CPU 1/TCG' (tid=24317, running) created by main thread at:
> 
> Signed-off-by: Robert Foley 

Reviewed-by: Emilio G. Cota 

Thanks,

Emilio

Re: [PATCH v4 05/11] rules.mak: Add base-arch() rule

[Richard Henderson]

On 5/22/20 9:37 AM, Philippe Mathieu-Daudé wrote:
> +   $(if $(call startwith,risc,$1),risc,\

Should be "riscv" not "risc".  Especially the substitution.

> + $(if $(call startwith,aarch64,$1),arm,\
> +   $(if $(call startwith,x86_64,$1),i386,\

We don't have an exact equality function?


r~

Re: Adding a RISC-V board to QEMU

[Alistair Francis]

On Sun, May 31, 2020 at 10:33 AM Yash Jain  wrote:
>
> Hi,
> I'm Yash. I'm a 3rd year college undergraduate student, currently interning 
> in IIT Madras in Project Shakti. We have built RISC-V boards and  I have been 
> assigned the task to add these boards as machines to QEMU.
> I am absolutely new to this and have no idea how to go about this. Please 
> help me out!
> One way I can go about this is by modifying sifive code to fit my 
> requirements. Is that a viable option?

Hey Yash,

That's great news! I'm glad to hear you want to contribute and I am
more then happy to help :)

Your best bet is to find something similar and copy that. For example
if your machine (SoC and board) is similar to the SiFive U, copy the
QEMU sifive_u model and make changes as required.

Let me know how you go and I will help where I can.

Do you have documentation on what your board is and what it supports?

Alistair

>
> Thanks!
> Yash Jain

Re: [RFC v3 8/8] vhost-vdpa: introduce vhost-vdpa net client

[Cindy Lu]

On Fri, May 29, 2020 at 10:23 PM Eric Blake  wrote:
>
> On 5/29/20 9:06 AM, Cindy Lu wrote:
> > From: Tiwei Bie 
> >
> > This patch set introduces a new net client type: vhost-vdpa.
> > vhost-vdpa net client will set up a vDPA device which is specified
> > by a "vhostdev" parameter.
> >
> > Co-authored-by: Lingshan Zhu 
> > Signed-off-by: Cindy Lu 
> > ---
>
> > +static int net_vhost_vdpa_init(NetClientState *peer, const char *device,
> > +   const char *name, const char *vhostdev,
> > +   bool has_fd, char *fd)
> > +{
>
> fd is usually an int, not a string.
>
will fix this
> > +NetClientState *nc = NULL;
> > +VhostVDPAState *s;
> > +int vdpa_device_fd = -1;
> > +Error *err = NULL;
> > +int ret = 0;
> > +assert(name);
> > +
> > +nc = qemu_new_net_client(_vhost_vdpa_info, peer, device, name);
> > +snprintf(nc->info_str, sizeof(nc->info_str), "vhost-vdpa");
> > +nc->queue_index = 0;
> > +
> > +s = DO_UPCAST(VhostVDPAState, nc, nc);
> > +
> > +if (has_fd) {
> > +vdpa_device_fd = monitor_fd_param(cur_mon, fd, );
> > +} else{
> > +vdpa_device_fd = open(vhostdev, O_RDWR);
> > +}
>
> Oh, you're trying to use the old way for passing in fds.  The preferred
> way is to use qemu_open(), at which point you can pass in fds via the
> add-fd QMP command, and then pass the string "/dev/fdset/NNN" as
> vhostdev.  Then you don't need a special fd parameter here.
>
Thanks Eric, I will try this.

> > +++ b/qapi/net.json
> > @@ -428,6 +428,27 @@
> >   '*vhostforce':'bool',
> >   '*queues':'int' } }
> >
> > +##
> > +# @NetdevVhostVDPAOptions:
> > +#
> > +# Vhost-vdpa network backend
> > +#
> > +# @vhostdev: name of a vdpa dev path in sysfs
> > +#(default path:/dev/vhost-vdpa-$ID)
> > +#
> > +# @fd: file descriptor of an already opened vdpa device
> > +#
> > +# @queues: number of queues to be created for multiqueue vhost-vdpa
> > +#  (default: 1)
> > +#
> > +# Since: 5.1
> > +##
> > +{ 'struct': 'NetdevVhostVDPAOptions',
> > +  'data': {
> > +'*vhostdev': 'str',
> > +'*fd':   'str',
> > +'*queues':   'int' } }
>
> Instead of having vhostdev and fd both be optional (but where the user
> has to specify exactly one of them), you should only have vhostdev be
> mandatory, and rely on the /dev/fdset/NNN string as a way to get
> vhostdev to point to a previously-passed fd.
>
will fix this
> --
> Eric Blake, Principal Software Engineer
> Red Hat, Inc.   +1-919-301-3226
> Virtualization:  qemu.org | libvirt.org
>

according what does the qemu know to exit current TB and find helper functions from environment?

[tugouxp]

Hi folks:


   a questions puzzles me during review the qemu code of 5.0.0, take vexpress 
emulation arm A9 on RTOS  for example.
the emulated RTOS has its own "printf" implementations, so during the qemu 
emulations, it would find 
   helper_le_stl_mmu()
 store_helper()
   ..
  pl011_write();
writev();  //syscall
and finally, the helper would call the pl011_write to emulate the actual stdio 
output functions.
but, how did the TCB TB block know this during the translations? there may be 
others different "printf" implement ions on different  target, 
how the tcg knows that a special helper functions need to be called and others 
not, from the binary code disassembly language of the emulated target, there is 
no
obvious sign to make anyone know which instutions is special and need to 
emulation by helper, How get this?


Thank you.
  

Re: ia-32/ia-64 fxsave64 instruction behavior when saving mmx

[Robert Henry]

Here's additional information.

All of the remill tests of the legacy MMX instructions fail. These instructions 
work on 64-bit registers aliased with the lower 64-bits of the x87 fp80 
registers.  The tests fail because remill expects the fxsave64 instruction to 
deliver 16 bits of 1's (infinity or nan prefix) in the fp80 exponent, eg bits 
79:64.  Metal does this, but QEMU does not.

Reading of Intel Software development manual, table 3.44 
(https://www.felixcloutier.com/x86/FXSAVE.html#tbl-3-44) says these 16 bits are 
reserved, but another version of the manual 
(http://math-atlas.sourceforge.net/devel/arch/ia32_arch.pdf) section 9.6.2 
"Transitions between x87 fpu and mmx code" says a write to an MMX register sets 
those 16 bits to all 1s.

In digging through the code for the implementation of the SSE/mmx instruction 
pavgb I see a nice clean implementation in the SSE_HELPER_B macro which takes a 
MMXREG which is an MMREG_UNION which does not provide, to the extent that I can 
figure this out, a handle to bits 79:64 of the aliased-with x87 register.

I find it hard to believe that an apparent bug like this has been here 
"forever". Am I missing something?

Robert Henry

From: Robert Henry
Sent: Friday, May 29, 2020 10:38 AM
To: qemu-devel@nongnu.org 
Subject: ia-32/ia-64 fxsave64 instruction behavior when saving mmx

Background: The ia-32/ia-64 fxsave64 instruction saves fp80 or legacy SSE mmx 
registers. The mmx registers are saved as if they were fp80 values. The lower 
64 bits of the constructed fp80 value is the mmx register.  The upper 16 bits 
of the constructed fp80 value are reserved; see the last row of table 3-44 of 
https://www.felixcloutier.com/x86/fxsave#tbl-3-44

The Intel core i9-9980XE Skylake metal I have puts 0x into these reserved 
16 bits when saving MMX.

QEMU appears to put 0's there.

Does anybody have insight as to what "reserved" really means, or must be, in 
this case?  I take the verb "reserved" to mean something other than "undefined".

I came across this issue when running the remill instruction test engine.  See 
my issue https://github.com/lifting-bits/remill/issues/423 For better or worse, 
remill assumes that those bits are 0x, not 0x

Re: [PULL 0/2] m68k coldfire machine cleanup patches

[Peter Maydell]

On Sat, 30 May 2020 at 08:53, Thomas Huth  wrote:
>
>  Hi Peter,
>
> the following changes since commit c86274bc2e34295764fb44c2aef3cf29623f9b4b:
>
>   Merge remote-tracking branch 
> 'remotes/stsquad/tags/pull-testing-tcg-plugins-270520-1' into staging 
> (2020-05-29 17:41:45 +0100)
>
> are available in the Git repository at:
>
>   https://gitlab.com/huth/qemu.git tags/pull-request-2020-05-30
>
> for you to fetch changes up to b809667808b1f742a85d6cce0d77800be20bcaa0:
>
>   hw/m68k/mcf52xx: Replace hw_error() by qemu_log_mask() (2020-05-30 09:17:46 
> +0200)
>
> 
> * Replace hw_error() with qemu_log_mask() in the m68k coldfire machine code
> 


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
for any user-visible changes.

-- PMM

Re: [PATCH 0/8] hw: Fix some incomplete memory region size

[Peter Maydell]

On Sun, 31 May 2020 at 18:38, Philippe Mathieu-Daudé  wrote:
>
> memory_region_set_size() handle the 16 Exabytes limit by
> special-casing the UINT64_MAX value.
> This is not a problem for the 32-bit maximum, 4 GiB, but
> in some places we incorrectly use UINT32_MAX instead of
> 4 GiB, and end up missing 1 byte in the memory region.
>
> This series fixes the cases I encountered.
> Also included few patches while reviewing, I replaced some
> magic values by the IEC binary prefix equivalent.
>
> Regards,
>
> Phil.
>
> Philippe Mathieu-Daudé (8):
>   hw/arm/aspeed: Correct DRAM container region size
>   hw/pci-host/prep: Correct RAVEN bus bridge memory region size
>   hw/pci/pci_bridge: Correct pci_bridge_io memory region size
>   hw/pci/pci_bridge: Use the IEC binary prefix definitions
>   hw/pci-host: Use the IEC binary prefix definitions
>   hw/hppa/dino: Use the IEC binary prefix definitions
>   hw/i386/xen/xen-hvm: Use the IEC binary prefix definitions
>   target/i386/cpu: Use the IEC binary prefix definitions

whole series:
Reviewed-by: Peter Maydell 

thanks
-- PMM

Re: [PULL v3 00/11] bitmaps patches for 2020-05-26

[Peter Maydell]

On Thu, 28 May 2020 at 19:19, Eric Blake  wrote:
>
> The following changes since commit a20ab81d22300cca80325c284f21eefee99aa740:
>
>   Merge remote-tracking branch 
> 'remotes/huth-gitlab/tags/pull-request-2020-05-28' into staging (2020-05-28 
> 16:18:06 +0100)
>
> are available in the Git repository at:
>
>   https://repo.or.cz/qemu/ericb.git tags/pull-bitmaps-2020-05-26-v3
>
> for you to fetch changes up to cf2d1203dcfc2bf964453d83a2302231ce77f2dc:
>
>   iotests: Add test 291 to for qemu-img bitmap coverage (2020-05-28 13:16:30 
> -0500)
>
> v3: port sed expression to BSD sed
> v2: fix iotest 190 to not be as sensitive to different sparseness of
> qcow2 file on various filesystems, such as FreeBSD (sending only the
> changed patch)
>
> 
> bitmaps patches for 2020-05-26
>
> - fix non-blockdev migration of bitmaps when mirror job is in use
> - add bitmap sizing to 'qemu-img measure'
> - add 'qemu-img convert --bitmaps'
>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
for any user-visible changes.

-- PMM

Re: [PATCH 4/6] exec/cpu-common: Do not restrict CPU to 32-bit memory access maximum

[Peter Maydell]

On Sun, 31 May 2020 at 18:54, Philippe Mathieu-Daudé  wrote:
>
> Most CPUs can do 64-bit operations. Update the CPUReadMemoryFunc
> and CPUWriteMemoryFunc prototypes.
>
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  include/exec/cpu-common.h |  4 ++--
>  hw/usb/hcd-musb.c | 12 ++--
>  2 files changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
> index b47e5630e7..5ac766e3b6 100644
> --- a/include/exec/cpu-common.h
> +++ b/include/exec/cpu-common.h
> @@ -43,8 +43,8 @@ extern ram_addr_t ram_size;
>
>  /* memory API */
>
> -typedef void CPUWriteMemoryFunc(void *opaque, hwaddr addr, uint32_t value);
> -typedef uint32_t CPUReadMemoryFunc(void *opaque, hwaddr addr);
> +typedef void CPUWriteMemoryFunc(void *opaque, hwaddr addr, uint64_t value);
> +typedef uint64_t CPUReadMemoryFunc(void *opaque, hwaddr addr);

I don't think the type of these functions has anything to do with the
CPU's capabilities, does it? The typedefs are a legacy remnant from before
the conversion to MemoryRegions:
 * before MemoryRegions, devices provided separate functions for
   byte/word/long reads and writes (64-bit writes were simply
   impossible with the ancient APIs, which required a 3-element
   function pointer array for read and the same for write)
 * the initial MemoryRegion conversion introduced the new-style
   "one read/write fn for all widths" APIs, but also supported
   old-style six-function devices, for ease of conversion, using
   MemoryRegionOps::old_mmio.
 * in commit 62a0db942dec6ebfe we were finally able to drop the
   old_mmio (having changed over the last devices using old-style).
   (I see I forgot to delete the now-unused MemoryRegionMmio typedef.)

The only remaining user of these typedefs is hw/usb/hcd-musb.c,
which is still not converted to QOM/qdev. It uses them to allow
its one user (hw/usb/tusb6010.c) to perform reads/writes on the
underlying musb registers.

There's no point in changing these typedefs to pass or return
a 64-bit data type, because their sole use is in the musb_read[]
and musb_write[] arrays, which only allow for 1, 2 or 4 byte
accesses, depending on which array element you use.

Possible cleanups here:
Easy:
 * delete the unused MmeoryRegionMmio
 * move these typedefs into include/hw/usb.h and rename them
   to MUSBReadFunc and MUSBWriteFunc, since that's all they're
   used for now
Tricky:
 * convert the hw/usb/hcd-musb.c code to QOM/qdev, which would
   include refactoring the current musb_read/musb_write so that
   instead of the tusb6010.c code calling function entries in these
   arrays the hcd-musb.c code exposed a MemoryRegion; the tusb6010
   code would access it via memory_region_dispatch_read/write

thanks
-- PMM

Re: [PATCH 3/6] hw/sd/allwinner-sdhost: Do DMA accesses via DMA address space

[Philippe Mathieu-Daudé]

On 5/31/20 7:54 PM, Philippe Mathieu-Daudé wrote:
> The DMA operations should not use the CPU address space, but
> the DMA address space. Add support for a DMA address space,
> and replace the cpu_physical_memory API calls by equivalent
> dma_memory_read/write calls.
> 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
>  include/hw/sd/allwinner-sdhost.h |  4 
>  hw/sd/allwinner-sdhost.c | 36 ++--
>  2 files changed, 34 insertions(+), 6 deletions(-)
> 
[...]> @@ -742,6 +747,17 @@ static void allwinner_sdhost_init(Object *obj)
>  sysbus_init_irq(SYS_BUS_DEVICE(s), >irq);
>  }
>  
> +static void allwinner_sdhost_realize(DeviceState *dev, Error **errp)
> +{
> +AwSdHostState *s = AW_SDHOST(dev);
> +
> +if (!s->dma_mr) {
> +error_setg(errp, "\"dma\" property must be provided.");

Oops I forgot to include the part that sets this property in the A10/H3
SoCs.

> +return;
> +}
> +address_space_init(>dma_as, s->dma_mr, "sdhost-dma");
> +}
> +
>  static void allwinner_sdhost_reset(DeviceState *dev)
>  {
>  AwSdHostState *s = AW_SDHOST(dev);
> @@ -787,6 +803,12 @@ static void allwinner_sdhost_reset(DeviceState *dev)
>  s->status_crc = REG_SD_CRC_STA_RST;
>  }
>  
> +static Property allwinner_sdhost_properties[] = {
> +DEFINE_PROP_LINK("dma", AwSdHostState,
> + dma_mr, TYPE_MEMORY_REGION, MemoryRegion *),
> +DEFINE_PROP_END_OF_LIST(),
> +};
> +
>  static void allwinner_sdhost_bus_class_init(ObjectClass *klass, void *data)
>  {
>  SDBusClass *sbc = SD_BUS_CLASS(klass);
> @@ -798,7 +820,9 @@ static void allwinner_sdhost_class_init(ObjectClass 
> *klass, void *data)
>  {
>  DeviceClass *dc = DEVICE_CLASS(klass);
>  
> +device_class_set_props(dc, allwinner_sdhost_properties);
>  dc->reset = allwinner_sdhost_reset;
> +dc->realize = allwinner_sdhost_realize;
>  dc->vmsd = _allwinner_sdhost;
>  }
>  
> 

[Bug 1881506] [NEW] TCG doesn't support a lot of features that should be supported

[Ethin Probst]

Public bug reported:

This is quite odd, and I'm not sure about how to get around it. I'm writing an 
OS in Rust and require APIC support. When I boot my kernel with 
qemu-system-x86_64, however, it dumps out a [lot] of warnings; it claims that 
TCG doesn't support FMA, X2APIC, AVX, F16C, AVX2, RDSEED, SHA-NI, FXSR-OPT, 
misalignsse, 3dnowprefetch, osvw, topoext, perfctr-core, clzero, xsaveerptr, 
ibpb, nrip-save, xsavec, and xsaves, but prints these warnings over 80 times 
before finally doing what I told it to do. Running QEMU 5.0.0 (unknown commit 
hash), as follows:
qemu-system-x86_64 -drive 
format=raw,file=target\x86_64-kernel-none\debug\bootimage-kernel.bin -serial 
stdio -no-reboot -hdb disk.img -s -m 4G -usb -rtc base=utc,clock=host -cpu 
EPYC-v3,+acpi,+apic,+rdrand,+rdseed,+sse,+sse2,+sse4.1,+sse4.2,+sse4a,+ssse3,+syscall,+x2apic
 -smp cpus=8 -soundhw all
I would run using HAXM, but my kernel requires RDRAND, and QEMU does not, to my 
knowledge, automatically support RDRAND (and I don't know how to enable it).

** Affects: qemu
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1881506

Title:
  TCG doesn't support a lot of features that should be supported

Status in QEMU:
  New

Bug description:
  This is quite odd, and I'm not sure about how to get around it. I'm writing 
an OS in Rust and require APIC support. When I boot my kernel with 
qemu-system-x86_64, however, it dumps out a [lot] of warnings; it claims that 
TCG doesn't support FMA, X2APIC, AVX, F16C, AVX2, RDSEED, SHA-NI, FXSR-OPT, 
misalignsse, 3dnowprefetch, osvw, topoext, perfctr-core, clzero, xsaveerptr, 
ibpb, nrip-save, xsavec, and xsaves, but prints these warnings over 80 times 
before finally doing what I told it to do. Running QEMU 5.0.0 (unknown commit 
hash), as follows:
  qemu-system-x86_64 -drive 
format=raw,file=target\x86_64-kernel-none\debug\bootimage-kernel.bin -serial 
stdio -no-reboot -hdb disk.img -s -m 4G -usb -rtc base=utc,clock=host -cpu 
EPYC-v3,+acpi,+apic,+rdrand,+rdseed,+sse,+sse2,+sse4.1,+sse4.2,+sse4a,+ssse3,+syscall,+x2apic
 -smp cpus=8 -soundhw all
  I would run using HAXM, but my kernel requires RDRAND, and QEMU does not, to 
my knowledge, automatically support RDRAND (and I don't know how to enable it).

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1881506/+subscriptions

Re: [RFC PATCH 6/6] memory: Use CPU register size as default access_size_max

[Peter Maydell]

On Sun, 31 May 2020 at 18:54, Philippe Mathieu-Daudé  wrote:
>
> Do not restrict 64-bit CPU to 32-bit max access by default.
>
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
> RFC because this probably require an audit of all devices
> used on 64-bit targets.
> But if we find such problematic devices, they should instead
> enforce their access_size_max = 4 rather than expecting the
> default value to be valid...
> ---
>  memory.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/memory.c b/memory.c
> index fd6f3d6aca..1d6bb5cdb0 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -1370,7 +1370,7 @@ bool memory_region_access_valid(MemoryRegion *mr,
>
>  access_size_max = mr->ops->valid.max_access_size;
>  if (!mr->ops->valid.max_access_size) {
> -access_size_max = 4;
> +access_size_max = TARGET_LONG_SIZE;
>  }

This is definitely not the right approach. TARGET_LONG_SIZE
is a property of the CPU, but memory_region_access_valid()
is testing properties of the MemoryRegion (ie the device
being addressed). One can have devices in a system with a
64-bit CPU which can only handle being accessed at 32-bit
width (indeed, it's pretty common). The behaviour of a device
shouldn't change depending on whether we happened to compile
it into a system with TARGET_LONG_SIZE=4 or 8.

(If you want to argue that we should make all our devices
explicit about the valid.max_access_size rather than relying
on "default means 4" then I wouldn't necessarily disagree.)

thanks
-- PMM

Re: [PATCH 5/6] exec: Restrict 32-bit CPUs to 32-bit address space

[Peter Maydell]

On Sun, 31 May 2020 at 18:54, Philippe Mathieu-Daudé  wrote:
>
> It is pointless to have 32-bit CPUs see a 64-bit address
> space, when they can only address the 32 lower bits.
>
> Only create CPU address space with a size it can address.
> This makes HMP 'info mtree' command easier to understand
> (on 32-bit CPUs).

> diff --git a/exec.c b/exec.c
> index 5162f0d12f..d6809a9447 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -2962,9 +2962,17 @@ static void tcg_commit(MemoryListener *listener)
>
>  static void memory_map_init(void)
>  {
> +uint64_t system_memory_size;
> +
> +#if TARGET_LONG_BITS >= 64
> +system_memory_size = UINT64_MAX;
> +#else
> +system_memory_size = 1ULL << TARGET_LONG_BITS;
> +#endif

TARGET_LONG_BITS is a description of the CPU's virtual
address size; but the size of the system_memory memory
region is related to the CPU's physical address size[*].
In particular, for the Arm Cortex-A15 (and any other
32-bit CPU with LPAE) TARGET_LONG_BITS is 32 but the CPU
can address more than 32 bits of physical memory.

[*] Strictly speaking, it would depend on the
maximum physical address size used by any transaction
master in the system -- in theory you could have a
32-bit-only CPU and a DMA controller that could be
programmed with 64-bit addresses. In practice the
CPU can generally address at least as much of the
physical address space as any other transaction master.

thanks
-- PMM

[Bug 1377095] Re: KVM guest VM does not reattach a throughpassed USB printer from Host after switching printer off and on

[123p]

Thanks Thomas for Your suggestion. Unfortunately I cannot achieve the
reconnection:

KVM runs with following parameters:

-usb -device usb-host,vendorid=0x04e8,productid=0x3242,id=drucker
-monitor unix:qemu-monitor-socket,server,nowait

With the Unix socket, I can now pipe commands to the qemu monitor. This
info command works:

echo 'info status' | socat - unix-connect:/qemu-monitor-socket
QEMU 4.2.0 monitor - type 'help' for more information
(qemu) info status
VM status: running

What I have is this:
- disconnect USB printer (Product Samsung ML-1510_700)
- stop KVM
- connect USB printer
- start KVM

echo 'info usb' | socat - unix-connect:/qemu-monitor-socket
QEMU 4.2.0 monitor - type 'help' for more information
(qemu) info usb
  Device 0.1, Port 1, Speed 12 Mb/s, Product QEMU USB Tablet
  Device 0.2, Port 2, Speed 12 Mb/s, Product QEMU USB Hub
  Device 0.3, Port 2.1, Speed 12 Mb/s, Product Samsung ML-1510_700, ID: drucker

Printer works.
- disconnect printer

echo 'info usb' | socat - unix-connect:/qemu-monitor-socket
QEMU 4.2.0 monitor - type 'help' for more information
(qemu) info usb
  Device 0.2, Port 1, Speed 12 Mb/s, Product QEMU USB Tablet
  Device 0.3, Port 2, Speed 12 Mb/s, Product QEMU USB Hub
  Device 0.4, Port 2.1, Speed 12 Mb/s, Product Samsung ML-1510_700, ID: drucker

USB Info still shows the printer, allthough it is disconnected.

Remove device from KVM:

echo 'device_del drucker' | socat - unix-connect:/qemu-monitor-socket

Printer is removed from the device list:

info usbdrucker' | socat - unix-connect:/qemu-monitor-socket
QEMU 4.2.0 monitor - type 'help' for more information
(qemu) info usb
  Device 0.2, Port 1, Speed 12 Mb/s, Product QEMU USB Tablet
  Device 0.3, Port 2, Speed 12 Mb/s, Product QEMU USB Hub

-connect printer

printer is not shown:

echo 'info usb' | socat - unix-connect:/qemu-monitor-socket
QEMU 4.2.0 monitor - type 'help' for more information
(qemu) info usb
  Device 0.2, Port 1, Speed 12 Mb/s, Product QEMU USB Tablet
  Device 0.3, Port 2, Speed 12 Mb/s, Product QEMU USB Hub

Add printer device:

echo 'device_add usb-host,vendorid=0x04e8,productid=0x3242,id=drucker ' | socat 
- unix-connect:/qemu-monitor-socket
QEMU 4.2.0 monitor - type 'help' for more information
(qemu) device_add usb-host,vendorid=0x04e8,productid=0x3242,id=drucker

echo 'info usb' | socat - unix-connect:/qemu-monitor-socket
QEMU 4.2.0 monitor - type 'help' for more information
(qemu) info usb
  Device 0.2, Port 1, Speed 12 Mb/s, Product QEMU USB Tablet
  Device 0.3, Port 2, Speed 12 Mb/s, Product QEMU USB Hub
  Device 0.0, Port 2.2, Speed 1.5 Mb/s, Product USB Host Device, ID: drucker

Device is shown, but not like when KVM is started with connected
printer: Speed only 1.5 Mb/s, no vendor and product info.

Printer does _not_ work.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1377095

Title:
  KVM guest VM does not reattach a throughpassed USB printer from Host
  after switching printer off and on

Status in QEMU:
  New

Bug description:
  
  Host OS: Gentoo, all packages built 2014-10-01

  qemu version 2.1.0-r1

  Linux kernel 3.14.14   x86_64 Intel(R) Core(TM) i3-3220T CPU @ 2.80GHz
  GenuineIntel GNU/Linux

  
  Guest VM: Debian 7 (Wheezy) Linux 3.2.0 686

  
  Start command:
  /usr/bin/qemu-system-i386 -enable-kvm -name wheezy -k de -serial null 
-parallel null -hda /var/kvm/wheezy.kvm-img -daemonize -net 
nic,macaddr=02:00:00:00:01:31 -net tap,ifname=tap3,script=no,downscript=no -m 
512 -pidfile /var/run/kvm/wheezy.pid -usb -usbdevice tablet -runas myuser -vnc 
127.0.0.1:3 -usbdevice host:04e8:3242

  Problem:
  USB printer pass-through from KVM host to guest vm only works if I start the 
qemu kvm when the USB printer (vendor/product ID 04e8:3242) is switched on and 
therefore shown in lsusb on the host. Then it is available in the started VM.

  But when I switch the usb printer attached to the host off, it
  disappears in lsusb both on the host and the VM (as expected) but when
  I switch the USB printer on again, it is shown on the host and also on
  the QEMU Monitor (Crtl Alt Shift 2 -> info usbhost), but in the VM
  lsusb does not show it again- so USB pass-through / hot plugging does
  not work. It worked with a previous Version of qemu (1.0 or
  something).

  That is very annoying, because every time I want to print something, I need 
to shutdown the VM, start the printer, and then start the VM (which runs cups 
as printer server).
  But after printing, I do not want the printer to keep running, so I switch it 
off.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1377095/+subscriptions

Re: [PATCH 0/6] exec/memory: Rework some address and access size limits

[no-reply]

Patchew URL: https://patchew.org/QEMU/20200531175425.10329-1-f4...@amsat.org/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing 
commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

qemu-system-aarch64: Initialization of device allwinner-sdhost-sun4i failed: 
"dma" property must be provided.
Broken pipe
/tmp/qemu-test/src/tests/qtest/libqtest.c:166: kill_qemu() tried to terminate 
QEMU process but encountered exit status 1 (expected 0)
ERROR - too few tests run (expected 66, got 19)
make: *** [check-qtest-aarch64] Error 1
make: *** Waiting for unfinished jobs
Could not access KVM kernel module: No such file or directory
qemu-system-x86_64: -accel kvm: failed to initialize kvm: No such file or 
directory
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', 
'--label', 'com.qemu.instance.uuid=2f35274b5f7c4771ba16a3557b505e20', '-u', 
'1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', 
'-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 
'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', 
'/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', 
'/var/tmp/patchew-tester-tmp-gxvgqn30/src/docker-src.2020-05-31-14.18.38.23880:/var/tmp/qemu:z,ro',
 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit 
status 2.
filter=--filter=label=com.qemu.instance.uuid=2f35274b5f7c4771ba16a3557b505e20
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-gxvgqn30/src'
make: *** [docker-run-test-quick@centos7] Error 2

real13m18.712s
user0m8.733s


The full log is available at
http://patchew.org/logs/20200531175425.10329-1-f4...@amsat.org/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

[PATCH 5/6] exec: Restrict 32-bit CPUs to 32-bit address space

[Philippe Mathieu-Daudé]

It is pointless to have 32-bit CPUs see a 64-bit address
space, when they can only address the 32 lower bits.

Only create CPU address space with a size it can address.
This makes HMP 'info mtree' command easier to understand
(on 32-bit CPUs).

Signed-off-by: Philippe Mathieu-Daudé 
---
This is particularly helpful with the AVR cores.
---
 exec.c | 10 +-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/exec.c b/exec.c
index 5162f0d12f..d6809a9447 100644
--- a/exec.c
+++ b/exec.c
@@ -2962,9 +2962,17 @@ static void tcg_commit(MemoryListener *listener)
 
 static void memory_map_init(void)
 {
+uint64_t system_memory_size;
+
+#if TARGET_LONG_BITS >= 64
+system_memory_size = UINT64_MAX;
+#else
+system_memory_size = 1ULL << TARGET_LONG_BITS;
+#endif
+
 system_memory = g_malloc(sizeof(*system_memory));
 
-memory_region_init(system_memory, NULL, "system", UINT64_MAX);
+memory_region_init(system_memory, NULL, "system", system_memory_size);
 address_space_init(_space_memory, system_memory, "memory");
 
 system_io = g_malloc(sizeof(*system_io));
-- 
2.21.3

[PATCH 4/6] exec/cpu-common: Do not restrict CPU to 32-bit memory access maximum

[Philippe Mathieu-Daudé]

Most CPUs can do 64-bit operations. Update the CPUReadMemoryFunc
and CPUWriteMemoryFunc prototypes.

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/exec/cpu-common.h |  4 ++--
 hw/usb/hcd-musb.c | 12 ++--
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
index b47e5630e7..5ac766e3b6 100644
--- a/include/exec/cpu-common.h
+++ b/include/exec/cpu-common.h
@@ -43,8 +43,8 @@ extern ram_addr_t ram_size;
 
 /* memory API */
 
-typedef void CPUWriteMemoryFunc(void *opaque, hwaddr addr, uint32_t value);
-typedef uint32_t CPUReadMemoryFunc(void *opaque, hwaddr addr);
+typedef void CPUWriteMemoryFunc(void *opaque, hwaddr addr, uint64_t value);
+typedef uint64_t CPUReadMemoryFunc(void *opaque, hwaddr addr);
 
 void qemu_ram_remap(ram_addr_t addr, ram_addr_t length);
 /* This should not be used by devices.  */
diff --git a/hw/usb/hcd-musb.c b/hw/usb/hcd-musb.c
index c29fbef6fc..4063cbccf8 100644
--- a/hw/usb/hcd-musb.c
+++ b/hw/usb/hcd-musb.c
@@ -1243,7 +1243,7 @@ static void musb_ep_writeh(void *opaque, int ep, int 
addr, uint16_t value)
 }
 
 /* Generic control */
-static uint32_t musb_readb(void *opaque, hwaddr addr)
+static uint64_t musb_readb(void *opaque, hwaddr addr)
 {
 MUSBState *s = (MUSBState *) opaque;
 int ep, i;
@@ -1305,7 +1305,7 @@ static uint32_t musb_readb(void *opaque, hwaddr addr)
 };
 }
 
-static void musb_writeb(void *opaque, hwaddr addr, uint32_t value)
+static void musb_writeb(void *opaque, hwaddr addr, uint64_t value)
 {
 MUSBState *s = (MUSBState *) opaque;
 int ep;
@@ -1392,7 +1392,7 @@ static void musb_writeb(void *opaque, hwaddr addr, 
uint32_t value)
 };
 }
 
-static uint32_t musb_readh(void *opaque, hwaddr addr)
+static uint64_t musb_readh(void *opaque, hwaddr addr)
 {
 MUSBState *s = (MUSBState *) opaque;
 int ep, i;
@@ -1446,7 +1446,7 @@ static uint32_t musb_readh(void *opaque, hwaddr addr)
 };
 }
 
-static void musb_writeh(void *opaque, hwaddr addr, uint32_t value)
+static void musb_writeh(void *opaque, hwaddr addr, uint64_t value)
 {
 MUSBState *s = (MUSBState *) opaque;
 int ep;
@@ -1502,7 +1502,7 @@ static void musb_writeh(void *opaque, hwaddr addr, 
uint32_t value)
 };
 }
 
-static uint32_t musb_readw(void *opaque, hwaddr addr)
+static uint64_t musb_readw(void *opaque, hwaddr addr)
 {
 MUSBState *s = (MUSBState *) opaque;
 int ep;
@@ -1520,7 +1520,7 @@ static uint32_t musb_readw(void *opaque, hwaddr addr)
 };
 }
 
-static void musb_writew(void *opaque, hwaddr addr, uint32_t value)
+static void musb_writew(void *opaque, hwaddr addr, uint64_t value)
 {
 MUSBState *s = (MUSBState *) opaque;
 int ep;
-- 
2.21.3

[PATCH 1/6] target/s390x/mmu_helper: Use address_space_rw() in place

[Philippe Mathieu-Daudé]

In an effort to remove the cpu_physical_memory_rw() API,
update s390_cpu_virt_mem_rw() to use a more recent
address_space_rw() API.

Signed-off-by: Philippe Mathieu-Daudé 
---
 target/s390x/mmu_helper.c | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/target/s390x/mmu_helper.c b/target/s390x/mmu_helper.c
index 7d9f3059cd..632e8a8af4 100644
--- a/target/s390x/mmu_helper.c
+++ b/target/s390x/mmu_helper.c
@@ -529,8 +529,10 @@ int s390_cpu_virt_mem_rw(S390CPU *cpu, vaddr laddr, 
uint8_t ar, void *hostbuf,
 /* Copy data by stepping through the area page by page */
 for (i = 0; i < nr_pages; i++) {
 currlen = MIN(len, TARGET_PAGE_SIZE - (laddr % TARGET_PAGE_SIZE));
-cpu_physical_memory_rw(pages[i] | (laddr & ~TARGET_PAGE_MASK),
-   hostbuf, currlen, is_write);
+address_space_rw(CPU(cpu)->as,
+ pages[i] | (laddr & ~TARGET_PAGE_MASK),
+ MEMTXATTRS_UNSPECIFIED,
+ hostbuf, currlen, is_write);
 laddr += currlen;
 hostbuf += currlen;
 len -= currlen;
-- 
2.21.3

[RFC PATCH 6/6] memory: Use CPU register size as default access_size_max

[Philippe Mathieu-Daudé]

Do not restrict 64-bit CPU to 32-bit max access by default.

Signed-off-by: Philippe Mathieu-Daudé 
---
RFC because this probably require an audit of all devices
used on 64-bit targets.
But if we find such problematic devices, they should instead
enforce their access_size_max = 4 rather than expecting the
default value to be valid...
---
 memory.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/memory.c b/memory.c
index fd6f3d6aca..1d6bb5cdb0 100644
--- a/memory.c
+++ b/memory.c
@@ -1370,7 +1370,7 @@ bool memory_region_access_valid(MemoryRegion *mr,
 
 access_size_max = mr->ops->valid.max_access_size;
 if (!mr->ops->valid.max_access_size) {
-access_size_max = 4;
+access_size_max = TARGET_LONG_SIZE;
 }
 
 access_size = MAX(MIN(size, access_size_max), access_size_min);
-- 
2.21.3

[PATCH 0/6] exec/memory: Rework some address and access size limits

[Philippe Mathieu-Daudé]

These patches are extracted from a bigger series which
- remove generic ISA space, restricting it to the hw
  that really has it (mostly PCI-ISA bridges)
- allow QTest/GDB to use any address space
- make I/O address space target-specific (only X86 and
  AVR have a CPU connected to it)
- better handle Harvard architectures

Various patches only make sense if the AVR arch is merged,
so instead of waiting and keeping rebasing/testing, let's
share what is generic and might be worthwhile.

Currently the QMP/QTest commands only use the 1st CPU
address space, which has already been reported to limit
fuzzing/fault_injection/gdbstub.

I'll probably follow with the PCI-ISA bridge part, but
let's first see the feedback for this batch.

Regards,

Phil.

Philippe Mathieu-Daudé (6):
  target/s390x/mmu_helper: Use address_space_rw() in place
  hw/dma/rc4030: Use DMA address space to do DMA accesses
  hw/sd/allwinner-sdhost: Do DMA accesses via DMA address space
  exec/cpu-common: Do not restrict CPU to 32-bit memory access maximum
  exec: Restrict 32-bit CPUs to 32-bit address space
  memory: Use CPU register size as default access_size_max

 include/exec/cpu-common.h|  4 ++--
 include/hw/sd/allwinner-sdhost.h |  4 
 exec.c   | 10 -
 hw/dma/rc4030.c  |  3 ++-
 hw/sd/allwinner-sdhost.c | 36 ++--
 hw/usb/hcd-musb.c| 12 +--
 memory.c |  2 +-
 target/s390x/mmu_helper.c|  6 --
 8 files changed, 58 insertions(+), 19 deletions(-)

-- 
2.21.3

[PATCH 2/6] hw/dma/rc4030: Use DMA address space to do DMA accesses

[Philippe Mathieu-Daudé]

The DMA device should not use the CPU address space
to do its operation, but its own address space.
Replace cpu_physical_memory_write() by dma_memory_read()
since we already have the DMA address space available.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/dma/rc4030.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/hw/dma/rc4030.c b/hw/dma/rc4030.c
index eefbabd758..c39fe2bb69 100644
--- a/hw/dma/rc4030.c
+++ b/hw/dma/rc4030.c
@@ -24,6 +24,7 @@
 
 #include "qemu/osdep.h"
 #include "qemu/units.h"
+#include "sysemu/dma.h"
 #include "hw/irq.h"
 #include "hw/mips/mips.h"
 #include "hw/sysbus.h"
@@ -301,7 +302,7 @@ static void rc4030_write(void *opaque, hwaddr addr, 
uint64_t data,
 if (s->cache_ltag == 0x8001 && s->cache_bmask == 0xf0f0f0f) {
 hwaddr dest = s->cache_ptag & ~0x1;
 dest += (s->cache_maint & 0x3) << 3;
-cpu_physical_memory_write(dest, , 4);
+dma_memory_read(>dma_as, dest, , 4);
 }
 break;
 /* Remote Speed Registers */
-- 
2.21.3

[PATCH 3/6] hw/sd/allwinner-sdhost: Do DMA accesses via DMA address space

[Philippe Mathieu-Daudé]

The DMA operations should not use the CPU address space, but
the DMA address space. Add support for a DMA address space,
and replace the cpu_physical_memory API calls by equivalent
dma_memory_read/write calls.

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/hw/sd/allwinner-sdhost.h |  4 
 hw/sd/allwinner-sdhost.c | 36 ++--
 2 files changed, 34 insertions(+), 6 deletions(-)

diff --git a/include/hw/sd/allwinner-sdhost.h b/include/hw/sd/allwinner-sdhost.h
index d94606a853..ae1125c026 100644
--- a/include/hw/sd/allwinner-sdhost.h
+++ b/include/hw/sd/allwinner-sdhost.h
@@ -68,6 +68,10 @@ typedef struct AwSdHostState {
 /** Maps I/O registers in physical memory */
 MemoryRegion iomem;
 
+/** DMA physical memory */
+MemoryRegion *dma_mr;
+AddressSpace dma_as;
+
 /** Interrupt output signal to notify CPU */
 qemu_irq irq;
 
diff --git a/hw/sd/allwinner-sdhost.c b/hw/sd/allwinner-sdhost.c
index f404e1fdb4..9a2b5fcaeb 100644
--- a/hw/sd/allwinner-sdhost.c
+++ b/hw/sd/allwinner-sdhost.c
@@ -21,7 +21,10 @@
 #include "qemu/log.h"
 #include "qemu/module.h"
 #include "qemu/units.h"
+#include "qapi/error.h"
 #include "sysemu/blockdev.h"
+#include "sysemu/dma.h"
+#include "hw/qdev-properties.h"
 #include "hw/irq.h"
 #include "hw/sd/allwinner-sdhost.h"
 #include "migration/vmstate.h"
@@ -306,7 +309,7 @@ static uint32_t allwinner_sdhost_process_desc(AwSdHostState 
*s,
 uint8_t buf[1024];
 
 /* Read descriptor */
-cpu_physical_memory_read(desc_addr, desc, sizeof(*desc));
+dma_memory_read(>dma_as, desc_addr, desc, sizeof(*desc));
 if (desc->size == 0) {
 desc->size = klass->max_desc_size;
 } else if (desc->size > klass->max_desc_size) {
@@ -331,8 +334,9 @@ static uint32_t allwinner_sdhost_process_desc(AwSdHostState 
*s,
 
 /* Write to SD bus */
 if (is_write) {
-cpu_physical_memory_read((desc->addr & DESC_SIZE_MASK) + num_done,
-  buf, buf_bytes);
+dma_memory_read(>dma_as,
+(desc->addr & DESC_SIZE_MASK) + num_done,
+buf, buf_bytes);
 
 for (uint32_t i = 0; i < buf_bytes; i++) {
 sdbus_write_data(>sdbus, buf[i]);
@@ -343,15 +347,16 @@ static uint32_t 
allwinner_sdhost_process_desc(AwSdHostState *s,
 for (uint32_t i = 0; i < buf_bytes; i++) {
 buf[i] = sdbus_read_data(>sdbus);
 }
-cpu_physical_memory_write((desc->addr & DESC_SIZE_MASK) + num_done,
-   buf, buf_bytes);
+dma_memory_write(>dma_as,
+ (desc->addr & DESC_SIZE_MASK) + num_done,
+ buf, buf_bytes);
 }
 num_done += buf_bytes;
 }
 
 /* Clear hold flag and flush descriptor */
 desc->status &= ~DESC_STATUS_HOLD;
-cpu_physical_memory_write(desc_addr, desc, sizeof(*desc));
+dma_memory_write(>dma_as, desc_addr, desc, sizeof(*desc));
 
 return num_done;
 }
@@ -742,6 +747,17 @@ static void allwinner_sdhost_init(Object *obj)
 sysbus_init_irq(SYS_BUS_DEVICE(s), >irq);
 }
 
+static void allwinner_sdhost_realize(DeviceState *dev, Error **errp)
+{
+AwSdHostState *s = AW_SDHOST(dev);
+
+if (!s->dma_mr) {
+error_setg(errp, "\"dma\" property must be provided.");
+return;
+}
+address_space_init(>dma_as, s->dma_mr, "sdhost-dma");
+}
+
 static void allwinner_sdhost_reset(DeviceState *dev)
 {
 AwSdHostState *s = AW_SDHOST(dev);
@@ -787,6 +803,12 @@ static void allwinner_sdhost_reset(DeviceState *dev)
 s->status_crc = REG_SD_CRC_STA_RST;
 }
 
+static Property allwinner_sdhost_properties[] = {
+DEFINE_PROP_LINK("dma", AwSdHostState,
+ dma_mr, TYPE_MEMORY_REGION, MemoryRegion *),
+DEFINE_PROP_END_OF_LIST(),
+};
+
 static void allwinner_sdhost_bus_class_init(ObjectClass *klass, void *data)
 {
 SDBusClass *sbc = SD_BUS_CLASS(klass);
@@ -798,7 +820,9 @@ static void allwinner_sdhost_class_init(ObjectClass *klass, 
void *data)
 {
 DeviceClass *dc = DEVICE_CLASS(klass);
 
+device_class_set_props(dc, allwinner_sdhost_properties);
 dc->reset = allwinner_sdhost_reset;
+dc->realize = allwinner_sdhost_realize;
 dc->vmsd = _allwinner_sdhost;
 }
 
-- 
2.21.3

[PATCH 8/8] target/i386/cpu: Use the IEC binary prefix definitions

[Philippe Mathieu-Daudé]

IEC binary prefixes ease code review: the unit is explicit.

Signed-off-by: Philippe Mathieu-Daudé 
---
 target/i386/cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 3733d9a279..33ce4861fb 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6159,7 +6159,7 @@ static void x86_cpu_machine_done(Notifier *n, void 
*unused)
 if (smram) {
 cpu->smram = g_new(MemoryRegion, 1);
 memory_region_init_alias(cpu->smram, OBJECT(cpu), "smram",
- smram, 0, 1ull << 32);
+ smram, 0, 4 * GiB);
 memory_region_set_enabled(cpu->smram, true);
 memory_region_add_subregion_overlap(cpu->cpu_as_root, 0, cpu->smram, 
1);
 }
-- 
2.21.3

[PATCH 7/8] hw/i386/xen/xen-hvm: Use the IEC binary prefix definitions

[Philippe Mathieu-Daudé]

IEC binary prefixes ease code review: the unit is explicit.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/i386/xen/xen-hvm.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/hw/i386/xen/xen-hvm.c b/hw/i386/xen/xen-hvm.c
index 82ece6b9e7..679d74e6a3 100644
--- a/hw/i386/xen/xen-hvm.c
+++ b/hw/i386/xen/xen-hvm.c
@@ -9,6 +9,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/units.h"
 
 #include "cpu.h"
 #include "hw/pci/pci.h"
@@ -230,7 +231,7 @@ static void xen_ram_init(PCMachineState *pcms,
  * Xen does not allocate the memory continuously, it keeps a
  * hole of the size computed above or passed in.
  */
-block_len = (1ULL << 32) + x86ms->above_4g_mem_size;
+block_len = 4 * GiB + x86ms->above_4g_mem_size;
 }
 memory_region_init_ram(_memory, NULL, "xen.ram", block_len,
_fatal);
-- 
2.21.3

[PATCH 2/8] hw/pci-host/prep: Correct RAVEN bus bridge memory region size

[Philippe Mathieu-Daudé]

memory_region_set_size() handle the 16 Exabytes limit by
special-casing the UINT64_MAX value. This is not a problem
for the 32-bit maximum, 4 GiB.
By using the UINT32_MAX value, the bm-raven MemoryRegion
ends up missing 1 byte:

  $ qemu-system-ppc -M prep -S -monitor stdio -usb
  memory-region: bm-raven
-fffe (prio 0, i/o): bm-raven
  -3eff (prio 0, i/o): alias bm-pci-memory 
@pci-memory -3eff
  8000- (prio 0, i/o): alias bm-system @system 
-7fff

Fix by using the correct value. We now have:

  memory-region: bm-raven
- (prio 0, i/o): bm-raven
  -3eff (prio 0, i/o): alias bm-pci-memory 
@pci-memory -3eff
  8000- (prio 0, i/o): alias bm-system @system 
-7fff

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/pci-host/prep.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/pci-host/prep.c b/hw/pci-host/prep.c
index 1a02e9a670..88e2fc66a9 100644
--- a/hw/pci-host/prep.c
+++ b/hw/pci-host/prep.c
@@ -294,7 +294,7 @@ static void raven_pcihost_initfn(Object *obj)
  >pci_memory, >pci_io, 0, TYPE_PCI_BUS);
 
 /* Bus master address space */
-memory_region_init(>bm, obj, "bm-raven", UINT32_MAX);
+memory_region_init(>bm, obj, "bm-raven", 4 * GiB);
 memory_region_init_alias(>bm_pci_memory_alias, obj, "bm-pci-memory",
  >pci_memory, 0,
  memory_region_size(>pci_memory));
-- 
2.21.3

[PATCH 4/8] hw/pci/pci_bridge: Use the IEC binary prefix definitions

[Philippe Mathieu-Daudé]

IEC binary prefixes ease code review: the unit is explicit.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/pci/pci_bridge.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/pci/pci_bridge.c b/hw/pci/pci_bridge.c
index 3ba3203f72..3789c17edc 100644
--- a/hw/pci/pci_bridge.c
+++ b/hw/pci/pci_bridge.c
@@ -423,14 +423,14 @@ int pci_bridge_qemu_reserve_cap_init(PCIDevice *dev, int 
cap_offset,
 }
 
 if (res_reserve.mem_non_pref != (uint64_t)-1 &&
-res_reserve.mem_non_pref >= (1ULL << 32)) {
+res_reserve.mem_non_pref >= 4 * GiB) {
 error_setg(errp,
"PCI resource reserve cap: mem-reserve must be less than 
4G");
 return -EINVAL;
 }
 
 if (res_reserve.mem_pref_32 != (uint64_t)-1 &&
-res_reserve.mem_pref_32 >= (1ULL << 32)) {
+res_reserve.mem_pref_32 >= 4 * GiB) {
 error_setg(errp,
"PCI resource reserve cap: pref32-reserve  must be less 
than 4G");
 return -EINVAL;
-- 
2.21.3

[PATCH 1/8] hw/arm/aspeed: Correct DRAM container region size

[Philippe Mathieu-Daudé]

memory_region_set_size() handle the 16 Exabytes limit by
special-casing the UINT64_MAX value. This is not a problem
for the 32-bit maximum, 4 GiB.
By using the UINT32_MAX value, the aspeed-ram-container
MemoryRegion ends up missing 1 byte:

 $ qemu-system-arm -M ast2600-evb -S -monitor stdio
 (qemu) info mtree

  address-space: aspeed.fmc-ast2600-dma-dram
8000-00017ffe (prio 0, i/o): aspeed-ram-container
  8000-bfff (prio 0, ram): ram
  c000- (prio 0, i/o): max_ram

Fix by using the correct value. We now have:

  address-space: aspeed.fmc-ast2600-dma-dram
8000-00017fff (prio 0, i/o): aspeed-ram-container
  8000-bfff (prio 0, ram): ram
  c000- (prio 0, i/o): max_ram

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/arm/aspeed.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/arm/aspeed.c b/hw/arm/aspeed.c
index 2c23297edf..62344ac6a3 100644
--- a/hw/arm/aspeed.c
+++ b/hw/arm/aspeed.c
@@ -262,7 +262,7 @@ static void aspeed_machine_init(MachineState *machine)
 bmc = g_new0(AspeedBoardState, 1);
 
 memory_region_init(>ram_container, NULL, "aspeed-ram-container",
-   UINT32_MAX);
+   4 * GiB);
 memory_region_add_subregion(>ram_container, 0, machine->ram);
 
 object_initialize_child(OBJECT(machine), "soc", >soc,
-- 
2.21.3

[PATCH 5/8] hw/pci-host: Use the IEC binary prefix definitions

[Philippe Mathieu-Daudé]

IEC binary prefixes ease code review: the unit is explicit.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/pci-host/i440fx.c| 3 ++-
 hw/pci-host/q35.c   | 2 +-
 hw/pci-host/versatile.c | 5 +++--
 3 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/hw/pci-host/i440fx.c b/hw/pci-host/i440fx.c
index 0adbd77553..aefb416c8f 100644
--- a/hw/pci-host/i440fx.c
+++ b/hw/pci-host/i440fx.c
@@ -23,6 +23,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/units.h"
 #include "qemu/range.h"
 #include "hw/i386/pc.h"
 #include "hw/pci/pci.h"
@@ -301,7 +302,7 @@ PCIBus *i440fx_init(const char *host_type, const char 
*pci_type,
 memory_region_set_enabled(>smram_region, true);
 
 /* smram, as seen by SMM CPUs */
-memory_region_init(>smram, OBJECT(d), "smram", 1ull << 32);
+memory_region_init(>smram, OBJECT(d), "smram", 4 * GiB);
 memory_region_set_enabled(>smram, true);
 memory_region_init_alias(>low_smram, OBJECT(d), "smram-low",
  f->ram_memory, 0xa, 0x2);
diff --git a/hw/pci-host/q35.c b/hw/pci-host/q35.c
index 352aeecfa7..b788f17b2c 100644
--- a/hw/pci-host/q35.c
+++ b/hw/pci-host/q35.c
@@ -589,7 +589,7 @@ static void mch_realize(PCIDevice *d, Error **errp)
 memory_region_set_enabled(>open_high_smram, false);
 
 /* smram, as seen by SMM CPUs */
-memory_region_init(>smram, OBJECT(mch), "smram", 1ull << 32);
+memory_region_init(>smram, OBJECT(mch), "smram", 4 * GiB);
 memory_region_set_enabled(>smram, true);
 memory_region_init_alias(>low_smram, OBJECT(mch), "smram-low",
  mch->ram_memory, MCH_HOST_BRIDGE_SMRAM_C_BASE,
diff --git a/hw/pci-host/versatile.c b/hw/pci-host/versatile.c
index cfb9a78ea6..8ddfb8772a 100644
--- a/hw/pci-host/versatile.c
+++ b/hw/pci-host/versatile.c
@@ -8,6 +8,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/units.h"
 #include "hw/sysbus.h"
 #include "migration/vmstate.h"
 #include "hw/irq.h"
@@ -399,8 +400,8 @@ static void pci_vpb_realize(DeviceState *dev, Error **errp)
 pci_map_irq_fn mapfn;
 int i;
 
-memory_region_init(>pci_io_space, OBJECT(s), "pci_io", 1ULL << 32);
-memory_region_init(>pci_mem_space, OBJECT(s), "pci_mem", 1ULL << 32);
+memory_region_init(>pci_io_space, OBJECT(s), "pci_io", 4 * GiB);
+memory_region_init(>pci_mem_space, OBJECT(s), "pci_mem", 4 * GiB);
 
 pci_root_bus_new_inplace(>pci_bus, sizeof(s->pci_bus), dev, "pci",
  >pci_mem_space, >pci_io_space,
-- 
2.21.3

[PATCH 3/8] hw/pci/pci_bridge: Correct pci_bridge_io memory region size

[Philippe Mathieu-Daudé]

memory_region_set_size() handle the 16 Exabytes limit by
special-casing the UINT64_MAX value. This is not a problem
for the 32-bit maximum, 4 GiB.
By using the UINT32_MAX value, the pci_bridge_io MemoryRegion
ends up missing 1 byte:

  (qemu) info mtree
  memory-region: pci_bridge_io
-fffe (prio 0, i/o): pci_bridge_io
  0060-0060 (prio 0, i/o): i8042-data
  0064-0064 (prio 0, i/o): i8042-cmd
  01ce-01d1 (prio 0, i/o): vbe
  0378-037f (prio 0, i/o): parallel
  03b4-03b5 (prio 0, i/o): vga
  ...

Fix by using the correct value. We now have:

  memory-region: pci_bridge_io
- (prio 0, i/o): pci_bridge_io
  0060-0060 (prio 0, i/o): i8042-data
  0064-0064 (prio 0, i/o): i8042-cmd
  ...

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/pci/pci_bridge.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/hw/pci/pci_bridge.c b/hw/pci/pci_bridge.c
index 97967d12eb..3ba3203f72 100644
--- a/hw/pci/pci_bridge.c
+++ b/hw/pci/pci_bridge.c
@@ -30,6 +30,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/units.h"
 #include "hw/pci/pci_bridge.h"
 #include "hw/pci/pci_bus.h"
 #include "qemu/module.h"
@@ -381,7 +382,7 @@ void pci_bridge_initfn(PCIDevice *dev, const char *typename)
 memory_region_init(>address_space_mem, OBJECT(br), "pci_bridge_pci", 
UINT64_MAX);
 sec_bus->address_space_io = >address_space_io;
 memory_region_init(>address_space_io, OBJECT(br), "pci_bridge_io",
-   UINT32_MAX);
+   4 * GiB);
 br->windows = pci_bridge_region_init(br);
 QLIST_INIT(_bus->child);
 QLIST_INSERT_HEAD(>child, sec_bus, sibling);
-- 
2.21.3

[PATCH 6/8] hw/hppa/dino: Use the IEC binary prefix definitions

[Philippe Mathieu-Daudé]

IEC binary prefixes ease code review: the unit is explicit.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/hppa/dino.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c
index 2b1b38c58a..7290f23962 100644
--- a/hw/hppa/dino.c
+++ b/hw/hppa/dino.c
@@ -542,7 +542,7 @@ PCIBus *dino_init(MemoryRegion *addr_space,
 >parent_obj.data_mem);
 
 /* Dino PCI bus memory.  */
-memory_region_init(>pci_mem, OBJECT(s), "pci-memory", 1ull << 32);
+memory_region_init(>pci_mem, OBJECT(s), "pci-memory", 4 * GiB);
 
 b = pci_register_root_bus(dev, "pci", dino_set_irq, dino_pci_map_irq, s,
   >pci_mem, get_system_io(),
@@ -561,7 +561,7 @@ PCIBus *dino_init(MemoryRegion *addr_space,
 }
 
 /* Set up PCI view of memory: Bus master address space.  */
-memory_region_init(>bm, OBJECT(s), "bm-dino", 1ull << 32);
+memory_region_init(>bm, OBJECT(s), "bm-dino", 4 * GiB);
 memory_region_init_alias(>bm_ram_alias, OBJECT(s),
  "bm-system", addr_space, 0,
  0xf000 + DINO_MEM_CHUNK_SIZE);
-- 
2.21.3

[PATCH 0/8] hw: Fix some incomplete memory region size

[Philippe Mathieu-Daudé]

memory_region_set_size() handle the 16 Exabytes limit by
special-casing the UINT64_MAX value.
This is not a problem for the 32-bit maximum, 4 GiB, but
in some places we incorrectly use UINT32_MAX instead of
4 GiB, and end up missing 1 byte in the memory region.

This series fixes the cases I encountered.
Also included few patches while reviewing, I replaced some
magic values by the IEC binary prefix equivalent.

Regards,

Phil.

Philippe Mathieu-Daudé (8):
  hw/arm/aspeed: Correct DRAM container region size
  hw/pci-host/prep: Correct RAVEN bus bridge memory region size
  hw/pci/pci_bridge: Correct pci_bridge_io memory region size
  hw/pci/pci_bridge: Use the IEC binary prefix definitions
  hw/pci-host: Use the IEC binary prefix definitions
  hw/hppa/dino: Use the IEC binary prefix definitions
  hw/i386/xen/xen-hvm: Use the IEC binary prefix definitions
  target/i386/cpu: Use the IEC binary prefix definitions

 hw/arm/aspeed.c | 2 +-
 hw/hppa/dino.c  | 4 ++--
 hw/i386/xen/xen-hvm.c   | 3 ++-
 hw/pci-host/i440fx.c| 3 ++-
 hw/pci-host/prep.c  | 2 +-
 hw/pci-host/q35.c   | 2 +-
 hw/pci-host/versatile.c | 5 +++--
 hw/pci/pci_bridge.c | 7 ---
 target/i386/cpu.c   | 2 +-
 9 files changed, 17 insertions(+), 13 deletions(-)

-- 
2.21.3

Adding a RISC-V board to QEMU

[Yash Jain]

Hi,
I'm Yash. I'm a 3rd year college undergraduate student, currently interning
in IIT Madras in Project Shakti. We have built RISC-V boards and  I have
been assigned the task to add these boards as machines to QEMU.
I am absolutely new to this and have no idea how to go about this. Please
help me out!
One way I can go about this is by modifying sifive code to fit my
requirements. Is that a viable option?

Thanks!
Yash Jain

Re: [PATCH-for-5.0] gdbstub: Use correct address space with Qqemu.PhyMemMode packet

[Peter Maydell]

On Sun, 31 May 2020 at 17:42, Jon Doron  wrote:
>
> On 31/05/2020, Philippe Mathieu-Daudé wrote:
> >On 3/30/20 6:41 PM, Peter Maydell wrote:
> >> PS: do we have any documentation of this new command ?
> >> ab4752ec8d9 has the implementation but no documentation...
> >
> >Jon, do you have documentation on the Qqemu.PhyMemMode packet?

> Hi, there is no documentation for this mode, but in general the idea was
> very simple.
>
> I want to have GDB the option to see the physical memory and examine it
> and have this option toggled.
>
> This was useful to me when I was working on nested virtual machine and I
> wanted to examine different states of the VMCS12 and EPTs.
>
> I used this in the following commands:
> // Enable
> maint packet Qqemu.PhyMemMode:1
>
> // Disable
> maint packet Qqemu.PhyMemMode:0

docs/system/gdb.rst would be the place to document QEMU-specific
extensions to the gdb protocol (there's an "advanced debugging
options" section where we document things like the single-step
stuff you can also change via 'maint packet').

thanks
-- PMM

[PULL 23/25] tests/acceptance: allow console interaction with specific VMs

[Philippe Mathieu-Daudé]

From: Pavel Dovgalyuk 

Console interaction in avocado scripts was possible only with single
default VM.
This patch modifies the function parameters to allow passing a specific
VM as a parameter to interact with it.

Signed-off-by: Pavel Dovgalyuk 
Reviewed-by: Willian Rampazzo 
Reviewed-by: Alex Bennée 
Tested-by: Philippe Mathieu-Daudé 
Message-Id: <159073587933.20809.5122618715976660635.stgit@pasha-ThinkPad-X280>
Signed-off-by: Philippe Mathieu-Daudé 
---
 tests/acceptance/avocado_qemu/__init__.py | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index 59e7b4f763..77d1c1d9ff 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -69,13 +69,15 @@ def pick_default_qemu_bin(arch=None):
 
 
 def _console_interaction(test, success_message, failure_message,
- send_string, keep_sending=False):
+ send_string, keep_sending=False, vm=None):
 assert not keep_sending or send_string
-console = test.vm.console_socket.makefile()
+if vm is None:
+vm = test.vm
+console = vm.console_socket.makefile()
 console_logger = logging.getLogger('console')
 while True:
 if send_string:
-test.vm.console_socket.sendall(send_string.encode())
+vm.console_socket.sendall(send_string.encode())
 if not keep_sending:
 send_string = None # send only once
 msg = console.readline().strip()
@@ -115,7 +117,8 @@ def interrupt_interactive_console_until_pattern(test, 
success_message,
 _console_interaction(test, success_message, failure_message,
  interrupt_string, True)
 
-def wait_for_console_pattern(test, success_message, failure_message=None):
+def wait_for_console_pattern(test, success_message, failure_message=None,
+ vm=None):
 """
 Waits for messages to appear on the console, while logging the content
 
@@ -125,7 +128,7 @@ def wait_for_console_pattern(test, success_message, 
failure_message=None):
 :param success_message: if this message appears, test succeeds
 :param failure_message: if this message appears, test fails
 """
-_console_interaction(test, success_message, failure_message, None)
+_console_interaction(test, success_message, failure_message, None, vm=vm)
 
 def exec_command_and_wait_for_pattern(test, command,
   success_message, failure_message=None):
-- 
2.21.3

Re: [PATCH 00/10] Add RX hardware emulation

[Philippe Mathieu-Daudé]

On 5/31/20 6:24 PM, Yoshinori Sato wrote:
> Hello.
> 
> This series add to hardware emulation module for RX target.
> 
> Details below.
> Interrupt controller, 8bit timer, 16bit comapare match timer and
> SCI is RX62N integrated peripheral.
> rx-virt - RX62N MCU and external RAM. It like gdb simulator.
> 
> The compare match timer has a CPU interface similar to the SH4 timer.
> sh_timer will be deprecated and integrated into this module.
> SCI is also implemented in sh_serial, but the functionality is omitted.
> I implemented the complete one as renesas_sci.
> 
> git repository here.
> git://git.pf.osdn.net/gitroot/y/ys/ysato/qemu.git tags/hw-rx-20200601

Sigh. It is a bit unfortunate and demotivating that we can not
coordinate this. I was going to send a pre-merge for this today.

What is positive is you unified some SH4/RX peripherals.

I'll try to merge both series. If it takes too long, I might send my
current queue for you to review, then go back to this one you just sent.

Regards,

Phil.

[PULL 22/25] tests/acceptance/migration.py: Wait for both sides

[Philippe Mathieu-Daudé]

From: "Dr. David Alan Gilbert" 

When the source finishes migration the destination will still be
receiving the data sent by the source, so it might not have quite
finished yet, so won't quite have reached 'completed'.
This lead to occasional asserts in the next few checks.

After the source has finished, check the destination as well.
(We can't just switch to checking the destination, because it doesn't
give a status until it has started receiving the migration).

Reported-by: Alex Bennée 
Signed-off-by: Dr. David Alan Gilbert 
Tested-by: Alex Bennée 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20200528112404.121972-1-dgilb...@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 tests/acceptance/migration.py | 4 
 1 file changed, 4 insertions(+)

diff --git a/tests/acceptance/migration.py b/tests/acceptance/migration.py
index 0365289cda..792639cb69 100644
--- a/tests/acceptance/migration.py
+++ b/tests/acceptance/migration.py
@@ -35,6 +35,10 @@ def assert_migration(self, src_vm, dst_vm):
   timeout=self.timeout,
   step=0.1,
   args=(src_vm,))
+wait.wait_for(self.migration_finished,
+  timeout=self.timeout,
+  step=0.1,
+  args=(dst_vm,))
 self.assertEqual(src_vm.command('query-migrate')['status'], 
'completed')
 self.assertEqual(dst_vm.command('query-migrate')['status'], 
'completed')
 self.assertEqual(dst_vm.command('query-status')['status'], 'running')
-- 
2.21.3

[PULL 25/25] tests/acceptance: refactor boot_linux to allow code reuse

[Philippe Mathieu-Daudé]

From: Pavel Dovgalyuk 

This patch moves image downloading functions to the separate class to allow
reusing them from record/replay tests.

Signed-off-by: Pavel Dovgalyuk 
Tested-by: Philippe Mathieu-Daudé 
Message-Id: <159073593167.20809.17582679291556188984.stgit@pasha-ThinkPad-X280>
Signed-off-by: Philippe Mathieu-Daudé 
---
 tests/acceptance/boot_linux.py | 49 --
 1 file changed, 29 insertions(+), 20 deletions(-)

diff --git a/tests/acceptance/boot_linux.py b/tests/acceptance/boot_linux.py
index 075a386300..3aa57e88b0 100644
--- a/tests/acceptance/boot_linux.py
+++ b/tests/acceptance/boot_linux.py
@@ -26,22 +26,8 @@
 TCG_NOT_AVAILABLE = ACCEL_NOT_AVAILABLE_FMT % "TCG"
 
 
-class BootLinux(Test):
-"""
-Boots a Linux system, checking for a successful initialization
-"""
-
-timeout = 900
-chksum = None
-
-def setUp(self):
-super(BootLinux, self).setUp()
-self.vm.add_args('-smp', '2')
-self.vm.add_args('-m', '1024')
-self.prepare_boot()
-self.prepare_cloudinit()
-
-def prepare_boot(self):
+class BootLinuxBase(Test):
+def download_boot(self):
 self.log.debug('Looking for and selecting a qemu-img binary to be '
'used to create the bootable snapshot image')
 # If qemu-img has been built, use it, otherwise the system wide one
@@ -60,17 +46,17 @@ def prepare_boot(self):
 if image_arch == 'ppc64':
 image_arch = 'ppc64le'
 try:
-self.boot = vmimage.get(
+boot = vmimage.get(
 'fedora', arch=image_arch, version='31',
 checksum=self.chksum,
 algorithm='sha256',
 cache_dir=self.cache_dirs[0],
 snapshot_dir=self.workdir)
-self.vm.add_args('-drive', 'file=%s' % self.boot.path)
 except:
 self.cancel('Failed to download/prepare boot image')
+return boot.path
 
-def prepare_cloudinit(self):
+def download_cloudinit(self):
 self.log.info('Preparing cloudinit image')
 try:
 cloudinit_iso = os.path.join(self.workdir, 'cloudinit.iso')
@@ -81,9 +67,32 @@ def prepare_cloudinit(self):
   # QEMU's hard coded usermode router address
   phone_home_host='10.0.2.2',
   phone_home_port=self.phone_home_port)
-self.vm.add_args('-drive', 'file=%s,format=raw' % cloudinit_iso)
 except Exception:
 self.cancel('Failed to prepared cloudinit image')
+return cloudinit_iso
+
+class BootLinux(BootLinuxBase):
+"""
+Boots a Linux system, checking for a successful initialization
+"""
+
+timeout = 900
+chksum = None
+
+def setUp(self):
+super(BootLinux, self).setUp()
+self.vm.add_args('-smp', '2')
+self.vm.add_args('-m', '1024')
+self.prepare_boot()
+self.prepare_cloudinit()
+
+def prepare_boot(self):
+path = self.download_boot()
+self.vm.add_args('-drive', 'file=%s' % path)
+
+def prepare_cloudinit(self):
+cloudinit_iso = self.download_cloudinit()
+self.vm.add_args('-drive', 'file=%s,format=raw' % cloudinit_iso)
 
 def launch_and_wait(self):
 self.vm.set_console()
-- 
2.21.3

Re: [PATCH-for-5.0] gdbstub: Use correct address space with Qqemu.PhyMemMode packet

[Jon Doron]

On 31/05/2020, Philippe Mathieu-Daudé wrote:

On 3/30/20 6:41 PM, Peter Maydell wrote:

On Mon, 30 Mar 2020 at 17:21, Philippe Mathieu-Daudé  wrote:

On 3/30/20 6:08 PM, Peter Maydell wrote:

On Mon, 30 Mar 2020 at 16:30, Philippe Mathieu-Daudé  wrote:


Since commit 3f940dc98, we added support for vAttach packet
to select a particular thread/cpu/core. However when using
the GDB physical memory mode, it is not clear which CPU
address space is used.
Since the CPU address space is stored in CPUState::as, use
address_space_rw() instead of cpu_physical_memory_rw().

Fixes: ab4752ec8d9
Signed-off-by: Philippe Mathieu-Daudé 
---
  gdbstub.c | 7 ++-
  1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/gdbstub.c b/gdbstub.c
index 013fb1ac0f..3baaef50e3 100644
--- a/gdbstub.c
+++ b/gdbstub.c
@@ -69,11 +69,8 @@ static inline int target_memory_rw_debug(CPUState *cpu, 
target_ulong addr,

  #ifndef CONFIG_USER_ONLY
  if (phy_memory_mode) {
-if (is_write) {
-cpu_physical_memory_write(addr, buf, len);
-} else {
-cpu_physical_memory_read(addr, buf, len);
-}
+address_space_rw(cpu->as, addr, MEMTXATTRS_UNSPECIFIED,
+ buf, len, is_write);
  return 0;


There's an argument here for using
int asidx = cpu_asidx_from_attrs(cpu, MEMTXATTRS_UNSPECIFIED);
AddressSpace *as = cpu_get_address_space(cpu, asidx);

though it will effectively boil down to the same thing in the end
as there's no way for the gdbstub to specify whether it wanted
eg the Arm secure or non-secure physical address space.


https://static.docs.arm.com/ihi0074/a/debug_interface_v6_0_architecture_specification_IHI0074A.pdf

* Configuration of hypervisor noninvasive debug.

This field can have one of the following values:

- 0b00
Separate controls for hypervisor noninvasive debug are not implemented,
or no hypervisor is implemented. For ARMv7 PEs that implement the
Virtualization Extensions, and for ARMv8 PEs that implement EL2, if
separate controls for hypervisor debug visibility are not implemented,
the hypervisor debug visibility is indicated by the relevant Non-secure
debug visibility fields NSNID and NSID.

OK so for ARM "noninvasive debug is not implemented" and we would use
the core secure address space?


I'm not very familiar with the debug interface (we don't model
it in QEMU), but I think that is the wrong end of it. These
are bits in AUTHSTATUS, which is a read-only register provided
by the CPU to the debugger. It basically says "am I, the CPU,
going to permit you, the debugger, to debug code running
in secure mode, or in EL2". (The CPU gets to decide this:
for security some h/w will not want any random with access
to the jtag debug port to be able to just read out code from
the secure world, for instance.)

What the debugger gets to control is bits in the CSW register
in the "MEM-AP"; it can use these to specify the size of
a memory access it wants to make to the guest, and also
the 'type', which is IMPDEF but typically lets the debugger
say "code access vs data access", "privileged vs usermode"
and "secure vs non-secure".

The equivalent in the QEMU world is that the debugger can
specify the memory transaction attributes. The question is
whether the gdb protocol provides any equivalent of that:
if it doesn't then gdbstub.c has to make up an attribute and
use that.


Instead of MEMTXATTRS_UNSPECIFIED I should use a crafted MemTxAttrs with
.secure = 1, .unspecified = 1?


You shouldn't set 'unspecified = 1', because that indicates
"this is MEMTXATTRS_UNSPECIFIED". The default set of
unspecified-attributes are probably good enough,
though, so you can just use MEMTXATTRS_UNSPECIFIED.


The idea of this command is to use the
CPU AS but not the MMU/MPU, maybe it doesn't make sense...


The trouble is that the command isn't precise enough.
"read/write to physical memory" is fine if the CPU has
exactly one physical address space, but it's ambiguous if the CPU
has more than one physical address space. Either we need the
user to be able to tell us which one they wanted somehow
(which for QEMU more or less means that they should tell
us what tx attributes they wanted), or we need to make an
arbitrary decision.

PS: do we have any documentation of this new command ?
ab4752ec8d9 has the implementation but no documentation...


Jon, do you have documentation on the Qqemu.PhyMemMode packet?



thanks
-- PMM



Hi, there is no documentation for this mode, but in general the idea was 
very simple.


I want to have GDB the option to see the physical memory and examine it 
and have this option toggled.


This was useful to me when I was working on nested virtual machine and I 
wanted to examine different states of the VMCS12 and EPTs.


I used this in the following commands:
// Enable
maint packet Qqemu.PhyMemMode:1

// Disable
maint packet Qqemu.PhyMemMode:0

It was mostly used part of a GDB script I played with to help me find 
the VMCS12 and EPTs.

[PULL 21/25] tests/migration/guestperf: Use Python 3 interpreter

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé 
Reviewed-by: John Snow 
Reviewed-by: Kevin Wolf 
Message-Id: <20200512103238.7078-7-phi...@redhat.com>
---
 tests/migration/guestperf-batch.py | 2 +-
 tests/migration/guestperf-plot.py  | 2 +-
 tests/migration/guestperf.py   | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/tests/migration/guestperf-batch.py 
b/tests/migration/guestperf-batch.py
index cb150ce804..f1e900908d 100755
--- a/tests/migration/guestperf-batch.py
+++ b/tests/migration/guestperf-batch.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 #
 # Migration test batch comparison invokation
 #
diff --git a/tests/migration/guestperf-plot.py 
b/tests/migration/guestperf-plot.py
index d70bb7a557..907151011a 100755
--- a/tests/migration/guestperf-plot.py
+++ b/tests/migration/guestperf-plot.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 #
 # Migration test graph plotting command
 #
diff --git a/tests/migration/guestperf.py b/tests/migration/guestperf.py
index 99b027e8ba..ba1c4bc4ca 100755
--- a/tests/migration/guestperf.py
+++ b/tests/migration/guestperf.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 #
 # Migration test direct invokation command
 #
-- 
2.21.3

[PULL 11/25] python/qemu: delint; add flake8 config

[Philippe Mathieu-Daudé]

From: John Snow 

Mostly, ignore the "no bare except" rule, because flake8 is not
contextual and cannot determine if we re-raise. Pylint can, though, so
always prefer pylint for that.

Signed-off-by: John Snow 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20200528222129.23826-5-js...@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 python/qemu/.flake8|  2 ++
 python/qemu/accel.py   |  9 ++---
 python/qemu/machine.py | 13 +
 python/qemu/qmp.py |  4 ++--
 4 files changed, 19 insertions(+), 9 deletions(-)
 create mode 100644 python/qemu/.flake8

diff --git a/python/qemu/.flake8 b/python/qemu/.flake8
new file mode 100644
index 00..45d8146f3f
--- /dev/null
+++ b/python/qemu/.flake8
@@ -0,0 +1,2 @@
+[flake8]
+extend-ignore = E722  # Pylint handles this, but smarter.
\ No newline at end of file
diff --git a/python/qemu/accel.py b/python/qemu/accel.py
index 36ae85791e..7fabe62920 100644
--- a/python/qemu/accel.py
+++ b/python/qemu/accel.py
@@ -23,11 +23,12 @@
 # Mapping host architecture to any additional architectures it can
 # support which often includes its 32 bit cousin.
 ADDITIONAL_ARCHES = {
-"x86_64" : "i386",
-"aarch64" : "armhf",
-"ppc64le" : "ppc64",
+"x86_64": "i386",
+"aarch64": "armhf",
+"ppc64le": "ppc64",
 }
 
+
 def list_accel(qemu_bin):
 """
 List accelerators enabled in the QEMU binary.
@@ -47,6 +48,7 @@ def list_accel(qemu_bin):
 # Skip the first line which is the header.
 return [acc.strip() for acc in out.splitlines()[1:]]
 
+
 def kvm_available(target_arch=None, qemu_bin=None):
 """
 Check if KVM is available using the following heuristic:
@@ -69,6 +71,7 @@ def kvm_available(target_arch=None, qemu_bin=None):
 return False
 return True
 
+
 def tcg_available(qemu_bin):
 """
 Check if TCG is available.
diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 8e4ecd1837..187790ce9e 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -29,6 +29,7 @@
 
 LOG = logging.getLogger(__name__)
 
+
 class QEMUMachineError(Exception):
 """
 Exception called when an error in QEMUMachine happens.
@@ -62,7 +63,8 @@ class QEMUMachine:
 """
 A QEMU VM
 
-Use this object as a context manager to ensure the QEMU process 
terminates::
+Use this object as a context manager to ensure
+the QEMU process terminates::
 
 with VM(binary) as vm:
 ...
@@ -185,8 +187,10 @@ def send_fd_scm(self, fd=None, file_path=None):
 fd_param.append(str(fd))
 
 devnull = open(os.path.devnull, 'rb')
-proc = subprocess.Popen(fd_param, stdin=devnull, 
stdout=subprocess.PIPE,
-stderr=subprocess.STDOUT, close_fds=False)
+proc = subprocess.Popen(
+fd_param, stdin=devnull, stdout=subprocess.PIPE,
+stderr=subprocess.STDOUT, close_fds=False
+)
 output = proc.communicate()[0]
 if output:
 LOG.debug(output)
@@ -485,7 +489,8 @@ def event_wait(self, name, timeout=60.0, match=None):
 
 def events_wait(self, events, timeout=60.0):
 """
-events_wait waits for and returns a named event from QMP with a 
timeout.
+events_wait waits for and returns a named event
+from QMP with a timeout.
 
 events: a sequence of (name, match_criteria) tuples.
 The match criteria are optional and may be None.
diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py
index d6c9b2f4b1..6ae7693965 100644
--- a/python/qemu/qmp.py
+++ b/python/qemu/qmp.py
@@ -168,8 +168,8 @@ def accept(self, timeout=15.0):
 
 @param timeout: timeout in seconds (nonnegative float number, or
 None). The value passed will set the behavior of the
-underneath QMP socket as described in [1]. Default 
value
-is set to 15.0.
+underneath QMP socket as described in [1].
+Default value is set to 15.0.
 @return QMP greeting dict
 @raise OSError on socket connection errors
 @raise QMPConnectError if the greeting is not received
-- 
2.21.3

[PULL 19/25] tests/vm: Add ability to select QEMU from current build

[Philippe Mathieu-Daudé]

From: Robert Foley 

Added a new special variable QEMU_LOCAL=1, which
will indicate to take the QEMU binary from the current
build.

Signed-off-by: Robert Foley 
Reviewed-by: Peter Puhov 
Reviewed-by: Alex Bennée 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Message-Id: <20200529203458.1038-6-robert.fo...@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé 
---
 tests/vm/Makefile.include |  4 
 tests/vm/basevm.py| 28 +++-
 2 files changed, 27 insertions(+), 5 deletions(-)

diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index 80f7f6bdee..a253aba457 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -41,6 +41,7 @@ endif
@echo "J=[0..9]* - Override the -jN parameter 
for make commands"
@echo "DEBUG=1   - Enable verbose output on 
host and interactive debugging"
@echo "V=1   - Enable verbose ouput on host 
and guest commands"
+   @echo "QEMU_LOCAL=1 - Use QEMU binary local to this 
build."
@echo "QEMU=/path/to/qemu- Change path to QEMU binary"
@echo "QEMU_IMG=/path/to/qemu-img- Change path to qemu-img tool"
 
@@ -57,6 +58,7 @@ $(IMAGES_DIR)/%.img:  $(SRC_PATH)/tests/vm/% \
$(PYTHON) $< \
$(if $(V)$(DEBUG), --debug) \
$(if $(GENISOIMAGE),--genisoimage $(GENISOIMAGE)) \
+   $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \
--image "$@" \
--force \
--build-image $@, \
@@ -71,6 +73,7 @@ vm-build-%: $(IMAGES_DIR)/%.img
$(if $(DEBUG), --interactive) \
$(if $(J),--jobs $(J)) \
$(if $(V),--verbose) \
+   $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \
--image "$<" \
$(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \
--snapshot \
@@ -92,6 +95,7 @@ vm-boot-ssh-%: $(IMAGES_DIR)/%.img
$(PYTHON) $(SRC_PATH)/tests/vm/$* \
$(if $(J),--jobs $(J)) \
$(if $(V)$(DEBUG), --debug) \
+   $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \
--image "$<" \
--interactive \
false, \
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index a2d4054d72..5a3ce42281 100644
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -61,9 +61,11 @@ class BaseVM(object):
 # 4 is arbitrary, but greater than 2,
 # since we found we need to wait more than twice as long.
 tcg_ssh_timeout_multiplier = 4
-def __init__(self, debug=False, vcpus=None, genisoimage=None):
+def __init__(self, debug=False, vcpus=None, genisoimage=None,
+ build_path=None):
 self._guest = None
 self._genisoimage = genisoimage
+self._build_path = build_path
 self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-",
  suffix=".tmp",
  dir="."))
@@ -184,15 +186,15 @@ def boot(self, img, extra_args=[]):
 "-device", "virtio-blk,drive=drive0,bootindex=0"]
 args += self._data_args + extra_args
 logging.debug("QEMU args: %s", " ".join(args))
-qemu_bin = os.environ.get("QEMU", "qemu-system-" + self.arch)
-guest = QEMUMachine(binary=qemu_bin, args=args)
+qemu_path = get_qemu_path(self.arch, self._build_path)
+guest = QEMUMachine(binary=qemu_path, args=args)
 guest.set_machine('pc')
 guest.set_console()
 try:
 guest.launch()
 except:
 logging.error("Failed to launch QEMU, command line:")
-logging.error(" ".join([qemu_bin] + args))
+logging.error(" ".join([qemu_path] + args))
 logging.error("Log:")
 logging.error(guest.get_log())
 logging.error("QEMU version >= 2.10 is required")
@@ -391,6 +393,19 @@ def gen_cloud_init_iso(self):
 
 return os.path.join(cidir, "cloud-init.iso")
 
+def get_qemu_path(arch, build_path=None):
+"""Fetch the path to the qemu binary."""
+# If QEMU environment variable set, it takes precedence
+if "QEMU" in os.environ:
+qemu_path = os.environ["QEMU"]
+elif build_path:
+qemu_path = os.path.join(build_path, arch + "-softmmu")
+qemu_path = os.path.join(qemu_path, "qemu-system-" + arch)
+else:
+# Default is to use system path for qemu.
+qemu_path = "qemu-system-" + arch
+return qemu_path
+
 def parse_args(vmcls):
 
 def get_default_jobs():
@@ -421,6 +436,9 @@ def get_default_jobs():
   help="build QEMU from source in guest")
 parser.add_option("--build-target",
   help="QEMU build target", default="check")
+

[PULL 24/25] tests/acceptance: refactor boot_linux_console test to allow code reuse

[Philippe Mathieu-Daudé]

From: Pavel Dovgalyuk 

This patch splits code in BootLinuxConsole class into two different
classes to allow reusing it by record/replay tests.

Signed-off-by: Pavel Dovgalyuk 
Reviewed-by: Alex Bennée 
Tested-by: Philippe Mathieu-Daudé 
Message-Id: <159073588490.20809.13942096070255577558.stgit@pasha-ThinkPad-X280>
Signed-off-by: Philippe Mathieu-Daudé 
---
 tests/acceptance/boot_linux_console.py | 21 +++--
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/tests/acceptance/boot_linux_console.py 
b/tests/acceptance/boot_linux_console.py
index c6b06a1a13..12725d4529 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -28,19 +28,13 @@
 except CmdNotFoundError:
 P7ZIP_AVAILABLE = False
 
-class BootLinuxConsole(Test):
-"""
-Boots a Linux kernel and checks that the console is operational and the
-kernel command line is properly passed from QEMU to the kernel
-"""
-
-timeout = 90
-
+class LinuxKernelTest(Test):
 KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
 
-def wait_for_console_pattern(self, success_message):
+def wait_for_console_pattern(self, success_message, vm=None):
 wait_for_console_pattern(self, success_message,
- failure_message='Kernel panic - not syncing')
+ failure_message='Kernel panic - not syncing',
+ vm=vm)
 
 def extract_from_deb(self, deb, path):
 """
@@ -79,6 +73,13 @@ def extract_from_rpm(self, rpm, path):
 os.chdir(cwd)
 return os.path.normpath(os.path.join(self.workdir, path))
 
+class BootLinuxConsole(LinuxKernelTest):
+"""
+Boots a Linux kernel and checks that the console is operational and the
+kernel command line is properly passed from QEMU to the kernel
+"""
+timeout = 90
+
 def test_x86_64_pc(self):
 """
 :avocado: tags=arch:x86_64
-- 
2.21.3

[PULL 20/25] tests/vm: allow wait_ssh() to specify command

[Philippe Mathieu-Daudé]

From: Robert Foley 

This allows for waiting for completion of arbitrary commands.

Signed-off-by: Robert Foley 
Reviewed-by: Peter Puhov 
Reviewed-by: Alex Bennée 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Message-Id: <20200529203458.1038-7-robert.fo...@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé 
---
 tests/vm/basevm.py | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 5a3ce42281..a80b616a08 100644
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -320,24 +320,24 @@ def console_sshd_config(self, prompt):
 def print_step(self, text):
 sys.stderr.write("### %s ...\n" % text)
 
-def wait_ssh(self, wait_root=False, seconds=300):
+def wait_ssh(self, wait_root=False, seconds=300, cmd="exit 0"):
 # Allow more time for VM to boot under TCG.
 if not kvm_available(self.arch):
 seconds *= self.tcg_ssh_timeout_multiplier
 starttime = datetime.datetime.now()
 endtime = starttime + datetime.timedelta(seconds=seconds)
-guest_up = False
+cmd_success = False
 while datetime.datetime.now() < endtime:
-if wait_root and self.ssh_root("exit 0") == 0:
-guest_up = True
+if wait_root and self.ssh_root(cmd) == 0:
+cmd_success = True
 break
-elif self.ssh("exit 0") == 0:
-guest_up = True
+elif self.ssh(cmd) == 0:
+cmd_success = True
 break
 seconds = (endtime - datetime.datetime.now()).total_seconds()
 logging.debug("%ds before timeout", seconds)
 time.sleep(1)
-if not guest_up:
+if not cmd_success:
 raise Exception("Timeout while waiting for guest ssh")
 
 def shutdown(self):
-- 
2.21.3

[PULL 08/25] python/qemu/machine: add kill() method

[Philippe Mathieu-Daudé]

From: Vladimir Sementsov-Ogievskiy 

Add method to hard-kill vm, without any quit commands.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Andrey Shinkevich 
Message-Id: <20200217150246.29180-19-vsement...@virtuozzo.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 python/qemu/machine.py | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index b9a98e2c86..d2f531f1b4 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -342,7 +342,7 @@ def wait(self):
 self._load_io_log()
 self._post_shutdown()
 
-def shutdown(self, has_quit=False):
+def shutdown(self, has_quit=False, hard=False):
 """
 Terminate the VM and clean up
 """
@@ -354,7 +354,9 @@ def shutdown(self, has_quit=False):
 self._console_socket = None
 
 if self.is_running():
-if self._qmp:
+if hard:
+self._popen.kill()
+elif self._qmp:
 try:
 if not has_quit:
 self._qmp.cmd('quit')
@@ -368,7 +370,8 @@ def shutdown(self, has_quit=False):
 self._post_shutdown()
 
 exitcode = self.exitcode()
-if exitcode is not None and exitcode < 0:
+if exitcode is not None and exitcode < 0 and \
+not (exitcode == -9 and hard):
 msg = 'qemu received signal %i: %s'
 if self._qemu_full_args:
 command = ' '.join(self._qemu_full_args)
@@ -378,6 +381,9 @@ def shutdown(self, has_quit=False):
 
 self._launched = False
 
+def kill(self):
+self.shutdown(hard=True)
+
 def set_qmp_monitor(self, enabled=True):
 """
 Set the QMP monitor.
-- 
2.21.3

[PULL 18/25] tests/vm: Pass --debug through for vm-boot-ssh

[Philippe Mathieu-Daudé]

From: Robert Foley 

This helps debug issues that occur during the boot sequence.

Signed-off-by: Robert Foley 
Reviewed-by: Peter Puhov 
Reviewed-by: Alex Bennée 
Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 
Message-Id: <20200529203458.1038-5-robert.fo...@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé 
---
 tests/vm/Makefile.include | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index 74ab522c55..80f7f6bdee 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -91,6 +91,7 @@ vm-boot-ssh-%: $(IMAGES_DIR)/%.img
$(call quiet-command, \
$(PYTHON) $(SRC_PATH)/tests/vm/$* \
$(if $(J),--jobs $(J)) \
+   $(if $(V)$(DEBUG), --debug) \
--image "$<" \
--interactive \
false, \
-- 
2.21.3

[PULL 17/25] python/qemu/qtest: Check before accessing _qtest

[Philippe Mathieu-Daudé]

From: John Snow 

It can be None; so add assertions or exceptions where appropriate to
guard the access accordingly.

Signed-off-by: John Snow 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20200514055403.18902-30-js...@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 python/qemu/qtest.py | 14 +++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/python/qemu/qtest.py b/python/qemu/qtest.py
index 4c88590eb0..888c8bd2f6 100644
--- a/python/qemu/qtest.py
+++ b/python/qemu/qtest.py
@@ -121,7 +121,8 @@ def _pre_launch(self):
 super()._pre_launch()
 self._qtest = QEMUQtestProtocol(self._qtest_path, server=True)
 
-def _post_launch(self):
+def _post_launch(self) -> None:
+assert self._qtest is not None
 super()._post_launch()
 self._qtest.accept()
 
@@ -129,6 +130,13 @@ def _post_shutdown(self):
 super()._post_shutdown()
 self._remove_if_exists(self._qtest_path)
 
-def qtest(self, cmd):
-'''Send a qtest command to guest'''
+def qtest(self, cmd: str) -> str:
+"""
+Send a qtest command to the guest.
+
+:param cmd: qtest command to send
+:return: qtest server response
+"""
+if self._qtest is None:
+raise RuntimeError("qtest socket not available")
 return self._qtest.cmd(cmd)
-- 
2.21.3

[PULL 16/25] python/qemu/qmp: assert sockfile is not None

[Philippe Mathieu-Daudé]

From: John Snow 

In truth, if you don't do this, you'll just get a TypeError
exception. Now, you'll get an AssertionError.

Is this tangibly better? No.
Does mypy complain less? Yes.

Signed-off-by: John Snow 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20200514055403.18902-21-js...@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 python/qemu/qmp.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py
index a634c4e26c..e64b6b5faa 100644
--- a/python/qemu/qmp.py
+++ b/python/qemu/qmp.py
@@ -94,6 +94,7 @@ def __negotiate_capabilities(self):
 raise QMPCapabilitiesError
 
 def __json_read(self, only_event=False):
+assert self.__sockfile is not None
 while True:
 data = self.__sockfile.readline()
 if not data:
-- 
2.21.3

[PULL 06/25] scripts/qmp: Fix shebang and imports

[Philippe Mathieu-Daudé]

From: John Snow 

There's more wrong with these scripts; They are in various stages of
disrepair. That's beyond the scope of this current patchset.

This just mechanically corrects the imports and the shebangs, as part of
ensuring that the python/qemu/lib refactoring didn't break anything
needlessly.

Signed-off-by: John Snow 
Message-Id: <20200528222129.23826-2-js...@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Philippe Mathieu-Daudé 
---
 scripts/qmp/qmp  | 4 +++-
 scripts/qmp/qom-fuse | 4 +++-
 scripts/qmp/qom-get  | 4 +++-
 scripts/qmp/qom-list | 4 +++-
 scripts/qmp/qom-set  | 4 +++-
 scripts/qmp/qom-tree | 4 +++-
 6 files changed, 18 insertions(+), 6 deletions(-)

diff --git a/scripts/qmp/qmp b/scripts/qmp/qmp
index 0625fc2aba..8e52e4a54d 100755
--- a/scripts/qmp/qmp
+++ b/scripts/qmp/qmp
@@ -11,7 +11,9 @@
 # See the COPYING file in the top-level directory.
 
 import sys, os
-from qmp import QEMUMonitorProtocol
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp import QEMUMonitorProtocol
 
 def print_response(rsp, prefix=[]):
 if type(rsp) == list:
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index 6bada2c33d..5fa6b3bf64 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -15,7 +15,9 @@ import fuse, stat
 from fuse import Fuse
 import os, posix
 from errno import *
-from qmp import QEMUMonitorProtocol
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp import QEMUMonitorProtocol
 
 fuse.fuse_python_api = (0, 2)
 
diff --git a/scripts/qmp/qom-get b/scripts/qmp/qom-get
index 7c5ede91bb..666df71832 100755
--- a/scripts/qmp/qom-get
+++ b/scripts/qmp/qom-get
@@ -13,7 +13,9 @@
 
 import sys
 import os
-from qmp import QEMUMonitorProtocol
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp import QEMUMonitorProtocol
 
 cmd, args = sys.argv[0], sys.argv[1:]
 socket_path = None
diff --git a/scripts/qmp/qom-list b/scripts/qmp/qom-list
index bb68fd65d4..5074fd939f 100755
--- a/scripts/qmp/qom-list
+++ b/scripts/qmp/qom-list
@@ -13,7 +13,9 @@
 
 import sys
 import os
-from qmp import QEMUMonitorProtocol
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp import QEMUMonitorProtocol
 
 cmd, args = sys.argv[0], sys.argv[1:]
 socket_path = None
diff --git a/scripts/qmp/qom-set b/scripts/qmp/qom-set
index 19881d85e9..240a78187f 100755
--- a/scripts/qmp/qom-set
+++ b/scripts/qmp/qom-set
@@ -13,7 +13,9 @@
 
 import sys
 import os
-from qmp import QEMUMonitorProtocol
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp import QEMUMonitorProtocol
 
 cmd, args = sys.argv[0], sys.argv[1:]
 socket_path = None
diff --git a/scripts/qmp/qom-tree b/scripts/qmp/qom-tree
index fa91147a03..25b0781323 100755
--- a/scripts/qmp/qom-tree
+++ b/scripts/qmp/qom-tree
@@ -15,7 +15,9 @@
 
 import sys
 import os
-from qmp import QEMUMonitorProtocol
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp import QEMUMonitorProtocol
 
 cmd, args = sys.argv[0], sys.argv[1:]
 socket_path = None
-- 
2.21.3

[PULL 13/25] python/qemu: fix socket.makefile() typing

[Philippe Mathieu-Daudé]

From: John Snow 

Note:

A bug in typeshed (https://github.com/python/typeshed/issues/3977)
misinterprets the type of makefile(). Work around this by explicitly
stating that we are opening a text-mode file.

Signed-off-by: John Snow 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20200514055403.18902-13-js...@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 python/qemu/qmp.py   | 10 +++---
 python/qemu/qtest.py | 12 
 2 files changed, 15 insertions(+), 7 deletions(-)

diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py
index 6ae7693965..73d49050ed 100644
--- a/python/qemu/qmp.py
+++ b/python/qemu/qmp.py
@@ -11,6 +11,10 @@
 import errno
 import socket
 import logging
+from typing import (
+Optional,
+TextIO,
+)
 
 
 class QMPError(Exception):
@@ -61,7 +65,7 @@ def __init__(self, address, server=False, nickname=None):
 self.__events = []
 self.__address = address
 self.__sock = self.__get_sock()
-self.__sockfile = None
+self.__sockfile: Optional[TextIO] = None
 self._nickname = nickname
 if self._nickname:
 self.logger = logging.getLogger('QMP').getChild(self._nickname)
@@ -157,7 +161,7 @@ def connect(self, negotiate=True):
 @raise QMPCapabilitiesError if fails to negotiate capabilities
 """
 self.__sock.connect(self.__address)
-self.__sockfile = self.__sock.makefile()
+self.__sockfile = self.__sock.makefile(mode='r')
 if negotiate:
 return self.__negotiate_capabilities()
 return None
@@ -180,7 +184,7 @@ def accept(self, timeout=15.0):
 """
 self.__sock.settimeout(timeout)
 self.__sock, _ = self.__sock.accept()
-self.__sockfile = self.__sock.makefile()
+self.__sockfile = self.__sock.makefile(mode='r')
 return self.__negotiate_capabilities()
 
 def cmd_obj(self, qmp_cmd):
diff --git a/python/qemu/qtest.py b/python/qemu/qtest.py
index 7943487c2b..4c88590eb0 100644
--- a/python/qemu/qtest.py
+++ b/python/qemu/qtest.py
@@ -19,6 +19,7 @@
 
 import socket
 import os
+from typing import Optional, TextIO
 
 from .machine import QEMUMachine
 
@@ -40,7 +41,7 @@ class QEMUQtestProtocol:
 def __init__(self, address, server=False):
 self._address = address
 self._sock = self._get_sock()
-self._sockfile = None
+self._sockfile: Optional[TextIO] = None
 if server:
 self._sock.bind(self._address)
 self._sock.listen(1)
@@ -59,7 +60,7 @@ def connect(self):
 @raise socket.error on socket connection errors
 """
 self._sock.connect(self._address)
-self._sockfile = self._sock.makefile()
+self._sockfile = self._sock.makefile(mode='r')
 
 def accept(self):
 """
@@ -68,7 +69,7 @@ def accept(self):
 @raise socket.error on socket connection errors
 """
 self._sock, _ = self._sock.accept()
-self._sockfile = self._sock.makefile()
+self._sockfile = self._sock.makefile(mode='r')
 
 def cmd(self, qtest_cmd):
 """
@@ -76,6 +77,7 @@ def cmd(self, qtest_cmd):
 
 @param qtest_cmd: qtest command text to be sent
 """
+assert self._sockfile is not None
 self._sock.sendall((qtest_cmd + "\n").encode('utf-8'))
 resp = self._sockfile.readline()
 return resp
@@ -83,7 +85,9 @@ def cmd(self, qtest_cmd):
 def close(self):
 """Close this socket."""
 self._sock.close()
-self._sockfile.close()
+if self._sockfile:
+self._sockfile.close()
+self._sockfile = None
 
 def settimeout(self, timeout):
 """Set a timeout, in seconds."""
-- 
2.21.3

[PULL 15/25] python/qemu/qmp: use True/False for non/blocking modes

[Philippe Mathieu-Daudé]

From: John Snow 

The type system doesn't want integers.

Signed-off-by: John Snow 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20200514055403.18902-15-js...@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 python/qemu/qmp.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py
index b91c9d5c1c..a634c4e26c 100644
--- a/python/qemu/qmp.py
+++ b/python/qemu/qmp.py
@@ -120,14 +120,14 @@ def __get_events(self, wait=False):
 """
 
 # Check for new events regardless and pull them into the cache:
-self.__sock.setblocking(0)
+self.__sock.setblocking(False)
 try:
 self.__json_read()
 except OSError as err:
 if err.errno == errno.EAGAIN:
 # No data available
 pass
-self.__sock.setblocking(1)
+self.__sock.setblocking(True)
 
 # Wait for new events, if needed.
 # if wait is 0.0, this means "no wait" and is also implicitly false.
-- 
2.21.3

[PULL 12/25] python/qemu: remove Python2 style super() calls

[Philippe Mathieu-Daudé]

From: John Snow 

Use the Python3 style instead.

Signed-off-by: John Snow 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20200514055403.18902-12-js...@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 python/qemu/machine.py |  2 +-
 python/qemu/qtest.py   | 15 +++
 2 files changed, 8 insertions(+), 9 deletions(-)

diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 187790ce9e..95a20a17f9 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -55,7 +55,7 @@ def __init__(self, reply):
 desc = reply["error"]["desc"]
 except KeyError:
 desc = reply
-super(MonitorResponseError, self).__init__(desc)
+super().__init__(desc)
 self.reply = reply
 
 
diff --git a/python/qemu/qtest.py b/python/qemu/qtest.py
index 53d814c064..7943487c2b 100644
--- a/python/qemu/qtest.py
+++ b/python/qemu/qtest.py
@@ -101,29 +101,28 @@ def __init__(self, binary, args=None, name=None, 
test_dir="/var/tmp",
 name = "qemu-%d" % os.getpid()
 if sock_dir is None:
 sock_dir = test_dir
-super(QEMUQtestMachine,
-  self).__init__(binary, args, name=name, test_dir=test_dir,
- socket_scm_helper=socket_scm_helper,
- sock_dir=sock_dir)
+super().__init__(binary, args, name=name, test_dir=test_dir,
+ socket_scm_helper=socket_scm_helper,
+ sock_dir=sock_dir)
 self._qtest = None
 self._qtest_path = os.path.join(sock_dir, name + "-qtest.sock")
 
 def _base_args(self):
-args = super(QEMUQtestMachine, self)._base_args()
+args = super()._base_args()
 args.extend(['-qtest', 'unix:path=' + self._qtest_path,
  '-accel', 'qtest'])
 return args
 
 def _pre_launch(self):
-super(QEMUQtestMachine, self)._pre_launch()
+super()._pre_launch()
 self._qtest = QEMUQtestProtocol(self._qtest_path, server=True)
 
 def _post_launch(self):
-super(QEMUQtestMachine, self)._post_launch()
+super()._post_launch()
 self._qtest.accept()
 
 def _post_shutdown(self):
-super(QEMUQtestMachine, self)._post_shutdown()
+super()._post_shutdown()
 self._remove_if_exists(self._qtest_path)
 
 def qtest(self, cmd):
-- 
2.21.3

[PULL 05/25] scripts/modules/module_block: Use Python 3 interpreter & add pseudo-main

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé 
Reviewed-by: John Snow 
Reviewed-by: Kevin Wolf 
Message-Id: <20200512103238.7078-6-phi...@redhat.com>
---
 scripts/modules/module_block.py | 29 +++--
 1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/scripts/modules/module_block.py b/scripts/modules/module_block.py
index f23191fac1..1109df827d 100644
--- a/scripts/modules/module_block.py
+++ b/scripts/modules/module_block.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 #
 # Module information generator
 #
@@ -80,19 +80,20 @@ def print_bottom(fheader):
 #endif
 ''')
 
-# First argument: output file
-# All other arguments: modules source files (.c)
-output_file = sys.argv[1]
-with open(output_file, 'w') as fheader:
-print_top(fheader)
+if __name__ == '__main__':
+# First argument: output file
+# All other arguments: modules source files (.c)
+output_file = sys.argv[1]
+with open(output_file, 'w') as fheader:
+print_top(fheader)
 
-for filename in sys.argv[2:]:
-if os.path.isfile(filename):
-process_file(fheader, filename)
-else:
-print("File " + filename + " does not exist.", file=sys.stderr)
-sys.exit(1)
+for filename in sys.argv[2:]:
+if os.path.isfile(filename):
+process_file(fheader, filename)
+else:
+print("File " + filename + " does not exist.", file=sys.stderr)
+sys.exit(1)
 
-print_bottom(fheader)
+print_bottom(fheader)
 
-sys.exit(0)
+sys.exit(0)
-- 
2.21.3

[PULL 10/25] python/qemu: delint and add pylintrc

[Philippe Mathieu-Daudé]

From: John Snow 

Bring our these files up to speed with pylint 2.5.0.
Add a pylintrc file to formalize which pylint subset
we are targeting.

The similarity ignore is there to suppress similarity
reports across imports, which for typing constants,
are going to trigger this report erroneously.

Signed-off-by: John Snow 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20200528222129.23826-4-js...@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 python/qemu/machine.py |  6 ++---
 python/qemu/pylintrc   | 58 ++
 python/qemu/qtest.py   | 42 +++---
 3 files changed, 88 insertions(+), 18 deletions(-)
 create mode 100644 python/qemu/pylintrc

diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 41554de533..8e4ecd1837 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -58,7 +58,7 @@ def __init__(self, reply):
 self.reply = reply
 
 
-class QEMUMachine(object):
+class QEMUMachine:
 """
 A QEMU VM
 
@@ -239,7 +239,7 @@ def _base_args(self):
  'chardev=mon,mode=control'])
 if self._machine is not None:
 args.extend(['-machine', self._machine])
-for i in range(self._console_index):
+for _ in range(self._console_index):
 args.extend(['-serial', 'null'])
 if self._console_set:
 self._console_address = os.path.join(self._sock_dir,
@@ -374,7 +374,7 @@ def shutdown(self, has_quit=False, hard=False):
 command = ' '.join(self._qemu_full_args)
 else:
 command = ''
-LOG.warning(msg, -exitcode, command)
+LOG.warning(msg, -int(exitcode), command)
 
 self._launched = False
 
diff --git a/python/qemu/pylintrc b/python/qemu/pylintrc
new file mode 100644
index 00..5d6ae7367d
--- /dev/null
+++ b/python/qemu/pylintrc
@@ -0,0 +1,58 @@
+[MASTER]
+
+[MESSAGES CONTROL]
+
+# Disable the message, report, category or checker with the given id(s). You
+# can either give multiple identifiers separated by comma (,) or put this
+# option multiple times (only on the command line, not in the configuration
+# file where it should appear only once). You can also use "--disable=all" to
+# disable everything first and then reenable specific checks. For example, if
+# you want to run only the similarities checker, you can use "--disable=all
+# --enable=similarities". If you want to run only the classes checker, but have
+# no Warning level messages displayed, use "--disable=all --enable=classes
+# --disable=W".
+disable=too-many-arguments,
+too-many-instance-attributes,
+too-many-public-methods,
+
+[REPORTS]
+
+[REFACTORING]
+
+[MISCELLANEOUS]
+
+[LOGGING]
+
+[BASIC]
+
+# Good variable names which should always be accepted, separated by a comma.
+good-names=i,
+   j,
+   k,
+   ex,
+   Run,
+   _,
+   fd,
+
+[VARIABLES]
+
+[STRING]
+
+[SPELLING]
+
+[FORMAT]
+
+[SIMILARITIES]
+
+# Ignore imports when computing similarities.
+ignore-imports=yes
+
+[TYPECHECK]
+
+[CLASSES]
+
+[IMPORTS]
+
+[DESIGN]
+
+[EXCEPTIONS]
diff --git a/python/qemu/qtest.py b/python/qemu/qtest.py
index d24ad04256..53d814c064 100644
--- a/python/qemu/qtest.py
+++ b/python/qemu/qtest.py
@@ -1,5 +1,11 @@
-# QEMU qtest library
-#
+"""
+QEMU qtest library
+
+qtest offers the QEMUQtestProtocol and QEMUQTestMachine classes, which
+offer a connection to QEMU's qtest protocol socket, and a qtest-enabled
+subclass of QEMUMachine, respectively.
+"""
+
 # Copyright (C) 2015 Red Hat Inc.
 #
 # Authors:
@@ -17,19 +23,21 @@
 from .machine import QEMUMachine
 
 
-class QEMUQtestProtocol(object):
-def __init__(self, address, server=False):
-"""
-Create a QEMUQtestProtocol object.
+class QEMUQtestProtocol:
+"""
+QEMUQtestProtocol implements a connection to a qtest socket.
 
-@param address: QEMU address, can be either a unix socket path (string)
-or a tuple in the form ( address, port ) for a TCP
-connection
-@param server: server mode, listens on the socket (bool)
-@raise socket.error on socket connection errors
-@note No connection is established, this is done by the connect() or
-  accept() methods
-"""
+:param address: QEMU address, can be either a unix socket path (string)
+or a tuple in the form ( address, port ) for a TCP
+connection
+:param server: server mode, listens on the socket (bool)
+:raise socket.error: on socket connection errors
+
+.. note::
+   No conection is estabalished by __init__(), this is done
+   by the connect() or accept() methods.
+"""
+def __init__(self, address, server=False):
 self._address = address
 self._sock = self._get_sock()
 self._sockfile = None
@@ -73,15 +81,19 @@ def cmd(self, 

[PULL 14/25] python/qemu: Adjust traceback typing

[Philippe Mathieu-Daudé]

From: John Snow 

mypy considers it incorrect to use `bool` to statically return false,
because it will assume that it could conceivably return True, and gives
different analysis in that case. Use a None return to achieve the same
effect, but make mypy happy.

Note: Pylint considers function signatures as code that might trip the
duplicate-code checker. I'd rather not disable this as it does not
trigger often in practice, so I'm disabling it as a one-off and filed a
change request; see https://github.com/PyCQA/pylint/issues/3619

Signed-off-by: John Snow 
Acked-by: Philippe Mathieu-Daudé 
Message-Id: <20200514055403.18902-14-js...@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 python/qemu/machine.py |  8 ++--
 python/qemu/qmp.py | 10 --
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 95a20a17f9..041c615052 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -24,6 +24,8 @@
 import shutil
 import socket
 import tempfile
+from typing import Optional, Type
+from types import TracebackType
 
 from . import qmp
 
@@ -124,9 +126,11 @@ def __init__(self, binary, args=None, wrapper=None, 
name=None,
 def __enter__(self):
 return self
 
-def __exit__(self, exc_type, exc_val, exc_tb):
+def __exit__(self,
+ exc_type: Optional[Type[BaseException]],
+ exc_val: Optional[BaseException],
+ exc_tb: Optional[TracebackType]) -> None:
 self.shutdown()
-return False
 
 def add_monitor_null(self):
 """
diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py
index 73d49050ed..b91c9d5c1c 100644
--- a/python/qemu/qmp.py
+++ b/python/qemu/qmp.py
@@ -14,7 +14,9 @@
 from typing import (
 Optional,
 TextIO,
+Type,
 )
+from types import TracebackType
 
 
 class QMPError(Exception):
@@ -146,10 +148,14 @@ def __enter__(self):
 # Implement context manager enter function.
 return self
 
-def __exit__(self, exc_type, exc_value, exc_traceback):
+def __exit__(self,
+ # pylint: disable=duplicate-code
+ # see https://github.com/PyCQA/pylint/issues/3619
+ exc_type: Optional[Type[BaseException]],
+ exc_val: Optional[BaseException],
+ exc_tb: Optional[TracebackType]) -> None:
 # Implement context manager exit function.
 self.close()
-return False
 
 def connect(self, negotiate=True):
 """
-- 
2.21.3

[PULL 02/25] scripts/qemu-gdb: Use Python 3 interpreter

[Philippe Mathieu-Daudé]

From: Philippe Mathieu-Daudé 

Signed-off-by: Philippe Mathieu-Daudé 
Reviewed-by: John Snow 
Reviewed-by: Kevin Wolf 
Message-Id: <20200512103238.7078-3-phi...@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 scripts/qemu-gdb.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/qemu-gdb.py b/scripts/qemu-gdb.py
index f2a305c42e..e0bfa7b5a4 100644
--- a/scripts/qemu-gdb.py
+++ b/scripts/qemu-gdb.py
@@ -1,5 +1,5 @@
-#!/usr/bin/python
-
+#!/usr/bin/env python3
+#
 # GDB debugging support
 #
 # Copyright 2012 Red Hat, Inc. and/or its affiliates
-- 
2.21.3

[PULL 04/25] scripts/kvm/vmxcap: Use Python 3 interpreter and add pseudo-main()

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé 
Acked-by: Paolo Bonzini 
Reviewed-by: John Snow 
Reviewed-by: Kevin Wolf 
Message-Id: <20200512103238.7078-5-phi...@redhat.com>
---
 scripts/kvm/vmxcap | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/scripts/kvm/vmxcap b/scripts/kvm/vmxcap
index 971ed0e721..6fe66d5f57 100755
--- a/scripts/kvm/vmxcap
+++ b/scripts/kvm/vmxcap
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 #
 # tool for querying VMX capabilities
 #
@@ -275,5 +275,6 @@ controls = [
 ),
 ]
 
-for c in controls:
-c.show()
+if __name__ == '__main__':
+for c in controls:
+c.show()
-- 
2.21.3

[PULL 07/25] python: remove more instances of sys.version_info

[Philippe Mathieu-Daudé]

From: John Snow 

We guarantee 3.5+ everywhere; remove more dead checks. In general, try
to avoid using version checks and instead prefer to attempt behavior
when possible.

Signed-off-by: John Snow 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20200514035230.25756-1-js...@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 scripts/analyze-migration.py |  5 -
 scripts/decodetree.py| 25 +---
 scripts/qmp/qmp-shell|  3 ---
 tests/docker/docker.py   |  5 +++--
 tests/qemu-iotests/nbd-fault-injector.py |  5 +
 5 files changed, 13 insertions(+), 30 deletions(-)

diff --git a/scripts/analyze-migration.py b/scripts/analyze-migration.py
index 96a31d3974..95838cbff3 100755
--- a/scripts/analyze-migration.py
+++ b/scripts/analyze-migration.py
@@ -25,11 +25,6 @@
 import sys
 
 
-MIN_PYTHON = (3, 2)
-if sys.version_info < MIN_PYTHON:
-sys.exit("Python %s.%s or later is required.\n" % MIN_PYTHON)
-
-
 def mkdir_p(path):
 try:
 os.makedirs(path)
diff --git a/scripts/decodetree.py b/scripts/decodetree.py
index 46ab917807..f9d204aa36 100755
--- a/scripts/decodetree.py
+++ b/scripts/decodetree.py
@@ -75,13 +75,6 @@ def output(*args):
 output_fd.write(a)
 
 
-if sys.version_info >= (3, 4):
-re_fullmatch = re.fullmatch
-else:
-def re_fullmatch(pat, str):
-return re.match('^' + pat + '$', str)
-
-
 def output_autogen():
 output('/* This file is autogenerated by scripts/decodetree.py.  */\n\n')
 
@@ -428,18 +421,18 @@ def parse_field(lineno, name, toks):
 width = 0
 func = None
 for t in toks:
-if re_fullmatch('!function=' + re_ident, t):
+if re.fullmatch('!function=' + re_ident, t):
 if func:
 error(lineno, 'duplicate function')
 func = t.split('=')
 func = func[1]
 continue
 
-if re_fullmatch('[0-9]+:s[0-9]+', t):
+if re.fullmatch('[0-9]+:s[0-9]+', t):
 # Signed field extract
 subtoks = t.split(':s')
 sign = True
-elif re_fullmatch('[0-9]+:[0-9]+', t):
+elif re.fullmatch('[0-9]+:[0-9]+', t):
 # Unsigned field extract
 subtoks = t.split(':')
 sign = False
@@ -488,11 +481,11 @@ def parse_arguments(lineno, name, toks):
 flds = []
 extern = False
 for t in toks:
-if re_fullmatch('!extern', t):
+if re.fullmatch('!extern', t):
 extern = True
 anyextern = True
 continue
-if not re_fullmatch(re_ident, t):
+if not re.fullmatch(re_ident, t):
 error(lineno, 'invalid argument set token "{0}"'.format(t))
 if t in flds:
 error(lineno, 'duplicate argument "{0}"'.format(t))
@@ -621,13 +614,13 @@ def parse_generic(lineno, is_format, name, toks):
 continue
 
 # 'Foo=%Bar' imports a field with a different name.
-if re_fullmatch(re_ident + '=%' + re_ident, t):
+if re.fullmatch(re_ident + '=%' + re_ident, t):
 (fname, iname) = t.split('=%')
 flds = add_field_byname(lineno, flds, fname, iname)
 continue
 
 # 'Foo=number' sets an argument field to a constant value
-if re_fullmatch(re_ident + '=[+-]?[0-9]+', t):
+if re.fullmatch(re_ident + '=[+-]?[0-9]+', t):
 (fname, value) = t.split('=')
 value = int(value)
 flds = add_field(lineno, flds, fname, ConstField(value))
@@ -635,7 +628,7 @@ def parse_generic(lineno, is_format, name, toks):
 
 # Pattern of 0s, 1s, dots and dashes indicate required zeros,
 # required ones, or dont-cares.
-if re_fullmatch('[01.-]+', t):
+if re.fullmatch('[01.-]+', t):
 shift = len(t)
 fms = t.replace('0', '1')
 fms = fms.replace('.', '0')
@@ -652,7 +645,7 @@ def parse_generic(lineno, is_format, name, toks):
 fixedmask = (fixedmask << shift) | fms
 undefmask = (undefmask << shift) | ubm
 # Otherwise, fieldname:fieldwidth
-elif re_fullmatch(re_ident + ':s?[0-9]+', t):
+elif re.fullmatch(re_ident + ':s?[0-9]+', t):
 (fname, flen) = t.split(':')
 sign = False
 if flen[0] == 's':
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index a01d31de1e..c5eef06f3f 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -77,9 +77,6 @@ import re
 sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
 from qemu import qmp
 
-if sys.version_info[0] == 2:
-input = raw_input
-
 class QMPCompleter(list):
 def complete(self, text, state):
 for cmd in self:
diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index d8268c..5a9735db78 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -258,12 +258,13 @@ def _kill_instances(self, 

[PULL 00/25] python-next patches for 2020-05-31

[Philippe Mathieu-Daudé]

The following changes since commit c86274bc2e34295764fb44c2aef3cf29623f9b4b:

  Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-tcg-plugins=
-270520-1' into staging (2020-05-29 17:41:45 +0100)

are available in the Git repository at:

  https://gitlab.com/philmd/qemu.git tags/python-next-20200531

for you to fetch changes up to 1c80c87c8c2489e4318c93c844aa29bc1d014146:

  tests/acceptance: refactor boot_linux to allow code reuse (2020-05-31 18:25=
:31 +0200)


Python queue:

* migration acceptance test fix
* introduce pylintrc & flake8 config
* various cleanups (Python3, style)
* vm-test can set QEMU_LOCAL=3D1 to use locally built binaries
* refactored BootLinuxBase & LinuxKernelTest acceptance classes

https://gitlab.com/philmd/qemu/pipelines/151323210
https://travis-ci.org/github/philmd/qemu/builds/693157969



Dr. David Alan Gilbert (1):
  tests/acceptance/migration.py: Wait for both sides

John Snow (11):
  scripts/qmp: Fix shebang and imports
  python: remove more instances of sys.version_info
  python/qemu/machine: remove logging configuration
  python/qemu: delint and add pylintrc
  python/qemu: delint; add flake8 config
  python/qemu: remove Python2 style super() calls
  python/qemu: fix socket.makefile() typing
  python/qemu: Adjust traceback typing
  python/qemu/qmp: use True/False for non/blocking modes
  python/qemu/qmp: assert sockfile is not None
  python/qemu/qtest: Check before accessing _qtest

Pavel Dovgaluk (3):
  tests/acceptance: allow console interaction with specific VMs
  tests/acceptance: refactor boot_linux_console test to allow code reuse
  tests/acceptance: refactor boot_linux to allow code reuse

Philippe Mathieu-Daud=C3=A9 (6):
  scripts/qemugdb: Remove shebang header
  scripts/qemu-gdb: Use Python 3 interpreter
  scripts/qmp: Use Python 3 interpreter
  scripts/kvm/vmxcap: Use Python 3 interpreter and add pseudo-main()
  scripts/modules/module_block: Use Python 3 interpreter & add
pseudo-main
  tests/migration/guestperf: Use Python 3 interpreter

Robert Foley (3):
  tests/vm: Pass --debug through for vm-boot-ssh
  tests/vm: Add ability to select QEMU from current build
  tests/vm: allow wait_ssh() to specify command

Vladimir Sementsov-Ogievskiy (1):
  python/qemu/machine: add kill() method

 python/qemu/.flake8   |  2 +
 python/qemu/accel.py  |  9 ++-
 python/qemu/machine.py| 44 +++-
 python/qemu/pylintrc  | 58 
 python/qemu/qmp.py| 29 +---
 python/qemu/qtest.py  | 83 +++
 scripts/analyze-migration.py  |  5 --
 scripts/decodetree.py | 25 +++
 scripts/kvm/vmxcap|  7 +-
 scripts/modules/module_block.py   | 29 
 scripts/qemu-gdb.py   |  4 +-
 scripts/qemugdb/__init__.py   |  3 +-
 scripts/qemugdb/aio.py|  3 +-
 scripts/qemugdb/coroutine.py  |  3 +-
 scripts/qemugdb/mtree.py  |  4 +-
 scripts/qemugdb/tcg.py|  1 -
 scripts/qemugdb/timers.py |  1 -
 scripts/qmp/qmp   |  4 +-
 scripts/qmp/qmp-shell |  3 -
 scripts/qmp/qom-fuse  |  4 +-
 scripts/qmp/qom-get   |  6 +-
 scripts/qmp/qom-list  |  6 +-
 scripts/qmp/qom-set   |  6 +-
 scripts/qmp/qom-tree  |  6 +-
 tests/acceptance/avocado_qemu/__init__.py | 13 ++--
 tests/acceptance/boot_linux.py| 49 +++--
 tests/acceptance/boot_linux_console.py| 21 +++---
 tests/acceptance/migration.py |  4 ++
 tests/docker/docker.py|  5 +-
 tests/migration/guestperf-batch.py|  2 +-
 tests/migration/guestperf-plot.py |  2 +-
 tests/migration/guestperf.py  |  2 +-
 tests/qemu-iotests/nbd-fault-injector.py  |  5 +-
 tests/vm/Makefile.include |  5 ++
 tests/vm/basevm.py| 42 
 35 files changed, 317 insertions(+), 178 deletions(-)
 create mode 100644 python/qemu/.flake8
 create mode 100644 python/qemu/pylintrc

--=20
2.21.3

[PULL 03/25] scripts/qmp: Use Python 3 interpreter

[Philippe Mathieu-Daudé]

From: Philippe Mathieu-Daudé 

Signed-off-by: Philippe Mathieu-Daudé 
Reviewed-by: John Snow 
Reviewed-by: Kevin Wolf 
Message-Id: <20200512103238.7078-4-phi...@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 scripts/qmp/qom-get  | 2 +-
 scripts/qmp/qom-list | 2 +-
 scripts/qmp/qom-set  | 2 +-
 scripts/qmp/qom-tree | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/qmp/qom-get b/scripts/qmp/qom-get
index 007b4cd442..7c5ede91bb 100755
--- a/scripts/qmp/qom-get
+++ b/scripts/qmp/qom-get
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 ##
 # QEMU Object Model test tools
 #
diff --git a/scripts/qmp/qom-list b/scripts/qmp/qom-list
index 03bda3446b..bb68fd65d4 100755
--- a/scripts/qmp/qom-list
+++ b/scripts/qmp/qom-list
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 ##
 # QEMU Object Model test tools
 #
diff --git a/scripts/qmp/qom-set b/scripts/qmp/qom-set
index c37fe78b00..19881d85e9 100755
--- a/scripts/qmp/qom-set
+++ b/scripts/qmp/qom-set
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 ##
 # QEMU Object Model test tools
 #
diff --git a/scripts/qmp/qom-tree b/scripts/qmp/qom-tree
index 1c8acf61e7..fa91147a03 100755
--- a/scripts/qmp/qom-tree
+++ b/scripts/qmp/qom-tree
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 ##
 # QEMU Object Model test tools
 #
-- 
2.21.3

[PULL 01/25] scripts/qemugdb: Remove shebang header

[Philippe Mathieu-Daudé]

From: Philippe Mathieu-Daudé 

These scripts are loaded as plugin by GDB (and they don't
have any __main__ entry point). Remove the shebang header.

Signed-off-by: Philippe Mathieu-Daudé 
Acked-by: Alex Bennée 
Reviewed-by: John Snow 
Reviewed-by: Kevin Wolf 
Message-Id: <20200512103238.7078-2-phi...@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 scripts/qemugdb/__init__.py  | 3 +--
 scripts/qemugdb/aio.py   | 3 +--
 scripts/qemugdb/coroutine.py | 3 +--
 scripts/qemugdb/mtree.py | 4 +---
 scripts/qemugdb/tcg.py   | 1 -
 scripts/qemugdb/timers.py| 1 -
 6 files changed, 4 insertions(+), 11 deletions(-)

diff --git a/scripts/qemugdb/__init__.py b/scripts/qemugdb/__init__.py
index 969f552b26..da8ff612e5 100644
--- a/scripts/qemugdb/__init__.py
+++ b/scripts/qemugdb/__init__.py
@@ -1,5 +1,4 @@
-#!/usr/bin/python
-
+#
 # GDB debugging support
 #
 # Copyright (c) 2015 Linaro Ltd
diff --git a/scripts/qemugdb/aio.py b/scripts/qemugdb/aio.py
index 2ba00c..d7c1ba0c28 100644
--- a/scripts/qemugdb/aio.py
+++ b/scripts/qemugdb/aio.py
@@ -1,5 +1,4 @@
-#!/usr/bin/python
-
+#
 # GDB debugging support: aio/iohandler debug
 #
 # Copyright (c) 2015 Red Hat, Inc.
diff --git a/scripts/qemugdb/coroutine.py b/scripts/qemugdb/coroutine.py
index 41e079d0e2..db61389022 100644
--- a/scripts/qemugdb/coroutine.py
+++ b/scripts/qemugdb/coroutine.py
@@ -1,5 +1,4 @@
-#!/usr/bin/python
-
+#
 # GDB debugging support
 #
 # Copyright 2012 Red Hat, Inc. and/or its affiliates
diff --git a/scripts/qemugdb/mtree.py b/scripts/qemugdb/mtree.py
index 3030a60d3f..8fe42c3c12 100644
--- a/scripts/qemugdb/mtree.py
+++ b/scripts/qemugdb/mtree.py
@@ -1,5 +1,4 @@
-#!/usr/bin/python
-
+#
 # GDB debugging support
 #
 # Copyright 2012 Red Hat, Inc. and/or its affiliates
@@ -84,4 +83,3 @@ def print_item(self, ptr, offset = gdb.Value(0), level = 0):
 while not isnull(subregion):
 self.print_item(subregion, addr, level)
 subregion = subregion['subregions_link']['tqe_next']
-
diff --git a/scripts/qemugdb/tcg.py b/scripts/qemugdb/tcg.py
index 18880fc9a7..16c03c06a9 100644
--- a/scripts/qemugdb/tcg.py
+++ b/scripts/qemugdb/tcg.py
@@ -1,4 +1,3 @@
-#!/usr/bin/python
 # -*- coding: utf-8 -*-
 #
 # GDB debugging support, TCG status
diff --git a/scripts/qemugdb/timers.py b/scripts/qemugdb/timers.py
index f0e132d27a..46537b27cf 100644
--- a/scripts/qemugdb/timers.py
+++ b/scripts/qemugdb/timers.py
@@ -1,4 +1,3 @@
-#!/usr/bin/python
 # -*- coding: utf-8 -*-
 # GDB debugging support
 #
-- 
2.21.3

[PULL 09/25] python/qemu/machine: remove logging configuration

[Philippe Mathieu-Daudé]

From: John Snow 

Python 3.5 and above do not print a warning when logging is not
configured. As a library, it's best practice to leave logging
configuration to the client executable.

Signed-off-by: John Snow 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20200514055403.18902-22-js...@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé 
---
 python/qemu/machine.py | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index d2f531f1b4..41554de533 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -119,9 +119,6 @@ def __init__(self, binary, args=None, wrapper=None, 
name=None,
 self._console_socket = None
 self._remove_files = []
 
-# just in case logging wasn't configured by the main script:
-logging.basicConfig()
-
 def __enter__(self):
 return self
 
-- 
2.21.3

[PATCH 04/10] hw/char: Renesas SCI module.

[Yoshinori Sato]

This module supported SCI / SCIa / SCIF.

Hardware manual.
SCI / SCIF
https://www.renesas.com/us/en/doc/products/mpumcu/001/r01uh0457ej0401_sh7751.pdf
SCIa
https://www.renesas.com/us/en/doc/products/mpumcu/doc/rx_family/r01uh0033ej0140_rx62n.pdf

Signed-off-by: Yoshinori Sato 
---
 include/hw/char/renesas_sci.h |  77 
 hw/char/renesas_sci.c | 786 ++
 hw/char/Kconfig   |   3 +
 hw/char/Makefile.objs |   3 +-
 4 files changed, 868 insertions(+), 1 deletion(-)
 create mode 100644 include/hw/char/renesas_sci.h
 create mode 100644 hw/char/renesas_sci.c

diff --git a/include/hw/char/renesas_sci.h b/include/hw/char/renesas_sci.h
new file mode 100644
index 00..9152faff6c
--- /dev/null
+++ b/include/hw/char/renesas_sci.h
@@ -0,0 +1,77 @@
+/*
+ * Renesas Serial Communication Interface
+ *
+ * Copyright (c) 2020 Yoshinori Sato
+ *
+ * This code is licensed under the GPL version 2 or later.
+ *
+ */
+
+#include "chardev/char-fe.h"
+#include "qemu/timer.h"
+#include "qemu/fifo8.h"
+#include "hw/sysbus.h"
+
+#define TYPE_RENESAS_SCI "renesas-sci"
+#define RSCI(obj) OBJECT_CHECK(RSCIState, (obj), TYPE_RENESAS_SCI)
+
+enum {
+ERI = 0,
+RXI = 1,
+TXI = 2,
+TEI = 3,
+BRI = 3,
+SCI_NR_IRQ = 4,
+};
+
+enum {
+SCI_FEAT_SCI = 0x00,
+SCI_FEAT_SCIA = 0x01,
+SCI_FEAT_SCIF = 0x10,
+};
+
+enum {
+RXTOUT,
+RXNEXT,
+TXEMPTY,
+TXEND,
+NR_SCI_EVENT,
+};
+
+typedef struct RSCIState {
+SysBusDevice parent_obj;
+MemoryRegion memory;
+MemoryRegion memory_p4;
+MemoryRegion memory_a7;
+
+/* SCI register */
+uint8_t smr;
+uint8_t brr;
+uint8_t scr;
+uint8_t tdr;
+uint16_t Xsr;
+uint8_t scmr;
+uint8_t semr;
+uint16_t fcr;
+uint16_t sptr;
+uint16_t lsr;
+
+/* internal use */
+uint16_t read_Xsr;
+uint16_t read_lsr;
+int64_t etu;
+int64_t trtime;
+int64_t tx_start_time;
+int tdcnt;
+int regsize;
+struct {
+int64_t time;
+int64_t (*handler)(struct RSCIState *sci);
+} event[NR_SCI_EVENT];
+QEMUTimer *event_timer;
+CharBackend chr;
+uint64_t input_freq;
+int feature;
+qemu_irq irq[SCI_NR_IRQ];
+Fifo8 rxfifo;
+} RSCIState;
diff --git a/hw/char/renesas_sci.c b/hw/char/renesas_sci.c
new file mode 100644
index 00..6b23055a40
--- /dev/null
+++ b/hw/char/renesas_sci.c
@@ -0,0 +1,786 @@
+/*
+ * Renesas Serial Communication Interface (SCI / SCIa / SCIF)
+ *
+ * Datasheet: RX62N Group, RX621 Group User's Manual: Hardware
+ * (Rev.1.40 R01UH0033EJ0140)
+ * And SH7751 Group, SH7751R Group User's Manual: Hardware
+ * (Rev.4.01 R01UH0457EJ0401)
+ *
+ * Copyright (c) 2020 Yoshinori Sato
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qapi/error.h"
+#include "qemu-common.h"
+#include "hw/hw.h"
+#include "hw/irq.h"
+#include "hw/sysbus.h"
+#include "hw/registerfields.h"
+#include "hw/qdev-properties.h"
+#include "hw/char/renesas_sci.h"
+#include "migration/vmstate.h"
+#include "qemu/error-report.h"
+
+/* SCI register map */
+REG8(SMR, 0)
+  FIELD(SMR, CKS,  0, 2)
+  FIELD(SMR, MP,   2, 1)
+  FIELD(SMR, STOP, 3, 1)
+  FIELD(SMR, PM,   4, 1)
+  FIELD(SMR, PE,   5, 1)
+  FIELD(SMR, CHR,  6, 1)
+  FIELD(SMR, CM,   7, 1)
+REG16(BRR, 2)
+REG16(SCR, 4)
+  FIELD(SCR, CKE, 0, 2)
+  FIELD(SCR, TEIE, 2, 1)
+  FIELD(SCR, MPIE, 3, 1)
+  FIELD(SCR, REIE, 3, 1)
+  FIELD(SCR, RE,   4, 1)
+  FIELD(SCR, TE,   5, 1)
+  FIELD(SCR, RIE,  6, 1)
+  FIELD(SCR, TIE,  7, 1)
+REG16(TDR, 6)
+REG16(SSR, 8)
+  FIELD(SSR, MPBT, 0, 1)
+  FIELD(SSR, MPB,  1, 1)
+  FIELD(SSR, TEND, 2, 1)
+  FIELD(SSR, ERR, 3, 3)
+FIELD(SSR, PER,  3, 1)
+FIELD(SSR, FER,  4, 1)
+FIELD(SSR, ORER, 5, 1)
+  FIELD(SSR, RDRF, 6, 1)
+  FIELD(SSR, TDRE, 7, 1)
+REG16(FSR, 8)
+  FIELD(FSR, DR, 0, 1)
+  FIELD(FSR, RDF, 1, 1)
+  FIELD(FSR, RDF_DR, 0, 2)
+  FIELD(FSR, PER, 2, 1)
+  FIELD(FSR, FER, 3, 1)
+  FIELD(FSR, BRK, 4, 1)
+  FIELD(FSR, TDFE, 5, 1)
+  FIELD(FSR, TEND, 6, 1)
+  FIELD(FSR, ER, 7, 1)
+  FIELD(FSR, FERn, 8, 4)
+  FIELD(FSR, PERn, 12, 4)
+REG16(RDR, 10)
+REG16(SCMR, 12)
+  FIELD(SCMR, SMIF, 0, 1)
+  FIELD(SCMR, SINV, 2, 1)
+  FIELD(SCMR, SDIR, 3, 1)
+  FIELD(SCMR, BCP2, 7, 1)
+REG16(FCR, 12)
+  FIELD(FCR, LOOP, 0, 1)
+  FIELD(FCR, RFRST, 1, 1)
+  FIELD(FCR, TFRST, 2, 1)
+  FIELD(FCR, MCE, 3, 1)
+  

[PATCH 08/10] hw/char: remove sh_serial.c

[Yoshinori Sato]

SH4 SCI module convert to renesas_sci.c.
This file is obsolute.

Signed-off-by: Yoshinori Sato 
---
 hw/char/sh_serial.c | 431 
 1 file changed, 431 deletions(-)
 delete mode 100644 hw/char/sh_serial.c

diff --git a/hw/char/sh_serial.c b/hw/char/sh_serial.c
deleted file mode 100644
index 167f4d8cb9..00
--- a/hw/char/sh_serial.c
+++ /dev/null
@@ -1,431 +0,0 @@
-/*
- * QEMU SCI/SCIF serial port emulation
- *
- * Copyright (c) 2007 Magnus Damm
- *
- * Based on serial.c - QEMU 16450 UART emulation
- * Copyright (c) 2003-2004 Fabrice Bellard
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this software and associated documentation files (the "Software"), to 
deal
- * in the Software without restriction, including without limitation the rights
- * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
- * copies of the Software, and to permit persons to whom the Software is
- * furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
- * THE SOFTWARE.
- */
-
-#include "qemu/osdep.h"
-#include "hw/irq.h"
-#include "hw/sh4/sh.h"
-#include "chardev/char-fe.h"
-#include "qapi/error.h"
-#include "qemu/timer.h"
-
-//#define DEBUG_SERIAL
-
-#define SH_SERIAL_FLAG_TEND (1 << 0)
-#define SH_SERIAL_FLAG_TDE  (1 << 1)
-#define SH_SERIAL_FLAG_RDF  (1 << 2)
-#define SH_SERIAL_FLAG_BRK  (1 << 3)
-#define SH_SERIAL_FLAG_DR   (1 << 4)
-
-#define SH_RX_FIFO_LENGTH (16)
-
-typedef struct {
-MemoryRegion iomem;
-MemoryRegion iomem_p4;
-MemoryRegion iomem_a7;
-uint8_t smr;
-uint8_t brr;
-uint8_t scr;
-uint8_t dr; /* ftdr / tdr */
-uint8_t sr; /* fsr / ssr */
-uint16_t fcr;
-uint8_t sptr;
-
-uint8_t rx_fifo[SH_RX_FIFO_LENGTH]; /* frdr / rdr */
-uint8_t rx_cnt;
-uint8_t rx_tail;
-uint8_t rx_head;
-
-int freq;
-int feat;
-int flags;
-int rtrg;
-
-CharBackend chr;
-QEMUTimer *fifo_timeout_timer;
-uint64_t etu; /* Elementary Time Unit (ns) */
-
-qemu_irq eri;
-qemu_irq rxi;
-qemu_irq txi;
-qemu_irq tei;
-qemu_irq bri;
-} sh_serial_state;
-
-static void sh_serial_clear_fifo(sh_serial_state * s)
-{
-memset(s->rx_fifo, 0, SH_RX_FIFO_LENGTH);
-s->rx_cnt = 0;
-s->rx_head = 0;
-s->rx_tail = 0;
-}
-
-static void sh_serial_write(void *opaque, hwaddr offs,
-uint64_t val, unsigned size)
-{
-sh_serial_state *s = opaque;
-unsigned char ch;
-
-#ifdef DEBUG_SERIAL
-printf("sh_serial: write offs=0x%02x val=0x%02x\n",
-   offs, val);
-#endif
-switch(offs) {
-case 0x00: /* SMR */
-s->smr = val & ((s->feat & SH_SERIAL_FEAT_SCIF) ? 0x7b : 0xff);
-return;
-case 0x04: /* BRR */
-s->brr = val;
-return;
-case 0x08: /* SCR */
-/* TODO : For SH7751, SCIF mask should be 0xfb. */
-s->scr = val & ((s->feat & SH_SERIAL_FEAT_SCIF) ? 0xfa : 0xff);
-if (!(val & (1 << 5)))
-s->flags |= SH_SERIAL_FLAG_TEND;
-if ((s->feat & SH_SERIAL_FEAT_SCIF) && s->txi) {
-qemu_set_irq(s->txi, val & (1 << 7));
-}
-if (!(val & (1 << 6))) {
-qemu_set_irq(s->rxi, 0);
-}
-return;
-case 0x0c: /* FTDR / TDR */
-if (qemu_chr_fe_backend_connected(>chr)) {
-ch = val;
-/* XXX this blocks entire thread. Rewrite to use
- * qemu_chr_fe_write and background I/O callbacks */
-qemu_chr_fe_write_all(>chr, , 1);
-}
-s->dr = val;
-s->flags &= ~SH_SERIAL_FLAG_TDE;
-return;
-#if 0
-case 0x14: /* FRDR / RDR */
-ret = 0;
-break;
-#endif
-}
-if (s->feat & SH_SERIAL_FEAT_SCIF) {
-switch(offs) {
-case 0x10: /* FSR */
-if (!(val & (1 << 6)))
-s->flags &= ~SH_SERIAL_FLAG_TEND;
-if (!(val & (1 << 5)))
-s->flags &= ~SH_SERIAL_FLAG_TDE;
-if (!(val & (1 << 4)))
-s->flags &= ~SH_SERIAL_FLAG_BRK;
-if (!(val & (1 << 1)))
-s->flags &= ~SH_SERIAL_FLAG_RDF;
-if (!(val & (1 << 0)))
-s->flags &= ~SH_SERIAL_FLAG_DR;
-
-if (!(val & (1 << 1)) || !(val & (1 << 0))) {
-if (s->rxi) {
-

[PATCH 05/10] hw/rx: RX MCU and target

[Yoshinori Sato]

rx62n - RX62N MCU.
rx-virt - RX QEMU virtual target.
 This has the same specifications as the gdb simulator.

Signed-off-by: Yoshinori Sato 
---
 include/hw/rx/rx.h|   7 ++
 include/hw/rx/rx62n.h |  91 
 hw/rx/rx-virt.c   | 143 +
 hw/rx/rx62n.c | 240 ++
 hw/rx/Kconfig |  13 +++
 hw/rx/Makefile.objs   |   2 +
 6 files changed, 496 insertions(+)
 create mode 100644 include/hw/rx/rx.h
 create mode 100644 include/hw/rx/rx62n.h
 create mode 100644 hw/rx/rx-virt.c
 create mode 100644 hw/rx/rx62n.c
 create mode 100644 hw/rx/Kconfig
 create mode 100644 hw/rx/Makefile.objs

diff --git a/include/hw/rx/rx.h b/include/hw/rx/rx.h
new file mode 100644
index 00..ff5924b81f
--- /dev/null
+++ b/include/hw/rx/rx.h
@@ -0,0 +1,7 @@
+#ifndef QEMU_RX_H
+#define QEMU_RX_H
+/* Definitions for RX board emulation.  */
+
+#include "target/rx/cpu-qom.h"
+
+#endif
diff --git a/include/hw/rx/rx62n.h b/include/hw/rx/rx62n.h
new file mode 100644
index 00..11176eadf3
--- /dev/null
+++ b/include/hw/rx/rx62n.h
@@ -0,0 +1,91 @@
+/*
+ * RX62N MCU Object
+ *
+ * Datasheet: RX62N Group, RX621 Group User's Manual: Hardware
+ * (Rev.1.40 R01UH0033EJ0140)
+ *
+ * Copyright (c) 2019 Yoshinori Sato
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see .
+ */
+
+#ifndef HW_RX_RX62N_H
+#define HW_RX_RX62N_H
+
+#include "hw/sysbus.h"
+#include "hw/intc/rx_icu.h"
+#include "hw/timer/renesas_8timer.h"
+#include "hw/timer/renesas_timer.h"
+#include "hw/char/renesas_sci.h"
+#include "target/rx/cpu.h"
+#include "qemu/units.h"
+
+#define TYPE_RX62N "rx62n"
+#define RX62N(obj) OBJECT_CHECK(RX62NState, (obj), TYPE_RX62N)
+
+#define RX62N_NR_TMR2
+#define RX62N_NR_CMT2
+#define RX62N_NR_SCI6
+
+typedef struct RX62NState {
+SysBusDevice parent_obj;
+
+RXCPU cpu;
+RXICUState icu;
+RTMRState tmr[RX62N_NR_TMR];
+RTIMERState cmt[RX62N_NR_CMT];
+RSCIState sci[RX62N_NR_SCI];
+
+MemoryRegion *sysmem;
+bool kernel;
+
+MemoryRegion iram;
+MemoryRegion iomem1;
+MemoryRegion d_flash;
+MemoryRegion iomem2;
+MemoryRegion iomem3;
+MemoryRegion c_flash;
+qemu_irq irq[NR_IRQS];
+} RX62NState;
+
+/*
+ * RX62N Peripheral Address
+ * See users manual section 5
+ */
+#define RX62N_ICUBASE 0x00087000
+#define RX62N_TMRBASE 0x00088200
+#define RX62N_CMTBASE 0x00088000
+#define RX62N_SCIBASE 0x00088240
+
+/*
+ * RX62N Peripheral IRQ
+ * See users manual section 11
+ */
+#define RX62N_TMR_IRQBASE 174
+#define RX62N_CMT_IRQBASE 28
+#define RX62N_SCI_IRQBASE 214
+
+/*
+ * RX62N Internal Memory
+ * It is the value of R5F562N8.
+ * Please change the size for R5F562N7.
+ */
+#define RX62N_IRAM_BASE 0x
+#define RX62N_IRAM_SIZE (96 * KiB)
+#define RX62N_DFLASH_BASE 0x0010
+#define RX62N_DFLASH_SIZE (32 * KiB)
+#define RX62N_CFLASH_BASE 0xfff8
+#define RX62N_CFLASH_SIZE (512 * KiB)
+
+#define RX62N_PCLK (48 * 1000 * 1000)
+#endif
diff --git a/hw/rx/rx-virt.c b/hw/rx/rx-virt.c
new file mode 100644
index 00..3168a85b83
--- /dev/null
+++ b/hw/rx/rx-virt.c
@@ -0,0 +1,143 @@
+/*
+ * RX QEMU virtual platform
+ *
+ * Copyright (c) 2019 Yoshinori Sato
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu-common.h"
+#include "cpu.h"
+#include "hw/hw.h"
+#include "hw/sysbus.h"
+#include "hw/loader.h"
+#include "hw/rx/rx62n.h"
+#include "sysemu/sysemu.h"
+#include "sysemu/qtest.h"
+#include "sysemu/device_tree.h"
+#include "hw/boards.h"
+
+/* Same address of GDB integrated simulator */
+#define SDRAM_BASE 0x0100
+
+static void rx_load_image(RXCPU *cpu, const char *filename,
+  uint32_t start, uint32_t size)
+{
+static uint32_t extable[32];
+long kernel_size;
+int i;
+
+

[PATCH 02/10] hw/timer: Renesas 8bit timer module.

[Yoshinori Sato]

8bit or 16bit timer.
This implementation support only internal clock mode.

Hardware manual.
https://www.renesas.com/us/en/doc/products/mpumcu/doc/rx_family/r01uh0033ej0140_rx62n.pdf

Signed-off-by: Yoshinori Sato 
---
 include/hw/timer/renesas_8timer.h |  61 
 hw/timer/renesas_8timer.c | 466 ++
 hw/timer/Kconfig  |   3 +
 hw/timer/Makefile.objs|   2 +
 4 files changed, 532 insertions(+)
 create mode 100644 include/hw/timer/renesas_8timer.h
 create mode 100644 hw/timer/renesas_8timer.c

diff --git a/include/hw/timer/renesas_8timer.h 
b/include/hw/timer/renesas_8timer.h
new file mode 100644
index 00..7e05c7be97
--- /dev/null
+++ b/include/hw/timer/renesas_8timer.h
@@ -0,0 +1,61 @@
+/*
+ * Renesas 8bit timer Object
+ *
+ * Copyright (c) 2018 Yoshinori Sato
+ *
+ * This code is licensed under the GPL version 2 or later.
+ *
+ */
+
+#ifndef HW_RENESAS_TMR_H
+#define HW_RENESAS_TMR_H
+
+#include "hw/sysbus.h"
+
+#define TYPE_RENESAS_8TMR "renesas-8tmr"
+#define RTMR(obj) OBJECT_CHECK(RTMRState, (obj), TYPE_RENESAS_8TMR)
+
+enum timer_event {
+cmia, cmib, ovi, wovi,
+TMR_NR_EVENTS,
+};
+
+enum {
+TMR_CH = 2,
+TMR_NR_COR = 2,
+TMR_NR_IRQ = 3,
+};
+
+enum {
+IRQ_CMIA, IRQ_CMIB, IRQ_OVI,
+};
+
+struct RTMRState;
+
+struct channel_8tmr {
+uint16_t cnt;
+uint16_t cor[TMR_NR_COR];
+uint8_t tcr;
+uint8_t tccr;
+uint8_t tcsr;
+qemu_irq irq[TMR_NR_IRQ];
+QEMUTimer *timer;
+int64_t base;
+int64_t next;
+int64_t clk;
+enum timer_event event;
+int id;
+struct RTMRState *tmrp;
+bool word;
+};
+
+typedef struct RTMRState {
+SysBusDevice parent_obj;
+
+uint64_t input_freq;
+MemoryRegion memory;
+
+struct channel_8tmr ch[TMR_CH];
+} RTMRState;
+
+#endif
diff --git a/hw/timer/renesas_8timer.c b/hw/timer/renesas_8timer.c
new file mode 100644
index 00..b545ba0e2b
--- /dev/null
+++ b/hw/timer/renesas_8timer.c
@@ -0,0 +1,466 @@
+/*
+ * Renesas 8bit timer
+ *
+ * Datasheet: RX62N Group, RX621 Group User's Manual: Hardware
+ * (Rev.1.40 R01UH0033EJ0140)
+ *
+ * Copyright (c) 2019 Yoshinori Sato
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "qemu/log.h"
+#include "qapi/error.h"
+#include "qemu/timer.h"
+#include "qemu/bitops.h"
+#include "hw/hw.h"
+#include "hw/irq.h"
+#include "hw/sysbus.h"
+#include "hw/registerfields.h"
+#include "hw/qdev-properties.h"
+#include "hw/timer/renesas_8timer.h"
+#include "migration/vmstate.h"
+#include "qemu/error-report.h"
+
+REG8(TCR, 0)
+  FIELD(TCR, CCLR, 3, 2)
+  FIELD(TCR, OVIE, 5, 1)
+  FIELD(TCR, CMIEA, 6, 1)
+  FIELD(TCR, CMIEB, 7, 1)
+REG8(TCSR, 2)
+  FIELD(TCSR, OSA, 0, 2)
+  FIELD(TCSR, OSB, 2, 2)
+  FIELD(TCSR, ADTE, 4, 1)
+REG8(TCORA, 4)
+REG8(TCORB, 6)
+REG8(TCNT, 8)
+REG8(TCCR, 10)
+  FIELD(TCCR, CKS, 0, 3)
+  FIELD(TCCR, CSS, 3, 2)
+  FIELD(TCCR, TMRIS, 7, 1)
+
+#define CLK_EVT -1
+
+static void update_clk(RTMRState *tmr, int ch)
+{
+int64_t t;
+static const int divlist[] = {1, 2, 8, 32, 64, 1024, 8192, 0};
+switch (FIELD_EX8(tmr->ch[ch].tccr, TCCR, CSS)) {
+case 0:
+qemu_log_mask(LOG_UNIMP,
+  "renesas_8timer: External clock not implemented.");
+tmr->ch[ch].clk = 0;
+break;
+case 1:
+/* Internal clock */
+t = divlist[FIELD_EX8(tmr->ch[ch].tccr, TCCR, CKS)];
+if (t > 0) {
+tmr->ch[ch].clk = tmr->input_freq / t;
+tmr->ch[ch].clk = NANOSECONDS_PER_SECOND / t;
+} else {
+tmr->ch[ch].clk = 0;
+}
+break;
+case 2:
+qemu_log_mask(LOG_UNIMP,
+  "renesas_8timer: CSS undefined.");
+tmr->ch[ch].clk = 0;
+break;
+case 3:
+/* Event count */
+tmr->ch[ch].clk = CLK_EVT;
+break;
+}
+}
+
+static uint16_t catreg(uint8_t hi, uint8_t lo)
+{
+uint16_t ret = 0;
+ret = deposit32(ret, 8, 8, hi);
+ret = deposit32(ret, 0, 8, lo);
+return ret;
+}
+
+static int is_clr(uint8_t tcr, int event)
+{
+return FIELD_EX8(tcr, TCR, CCLR) == (event + 1);
+}
+
+static int is_irq(uint8_t tcr, int event)
+{
+switch (event) {
+case cmia:
+return FIELD_EX8(tcr, TCR, CMIEA);
+case cmib:
+return FIELD_EX8(tcr, TCR, CMIEB);
+case ovi:
+return 

[PATCH 03/10] hw/timer: Renesas TMU/CMT module.

[Yoshinori Sato]

TMU - SH4 Timer module.
CMT - Compare and match timer used by some Renesas MCUs.

The two modules have similar interfaces and have been merged.

Signed-off-by: Yoshinori Sato 
---
 include/hw/timer/renesas_timer.h |  59 +
 hw/timer/renesas_timer.c | 421 +++
 hw/timer/Kconfig |   3 +
 hw/timer/Makefile.objs   |   1 +
 4 files changed, 484 insertions(+)
 create mode 100644 include/hw/timer/renesas_timer.h
 create mode 100644 hw/timer/renesas_timer.c

diff --git a/include/hw/timer/renesas_timer.h b/include/hw/timer/renesas_timer.h
new file mode 100644
index 00..f9a2661203
--- /dev/null
+++ b/include/hw/timer/renesas_timer.h
@@ -0,0 +1,59 @@
+/*
+ * Renesas Timer unit Object
+ *
+ * Copyright (c) 2020 Yoshinori Sato
+ *
+ * This code is licensed under the GPL version 2 or later.
+ *
+ */
+
+#ifndef HW_RENESAS_TIMER_H
+#define HW_RENESAS_TIMER_H
+
+#include "hw/sysbus.h"
+
+#define TYPE_RENESAS_TIMER "renesas-timer"
+#define RTIMER(obj) OBJECT_CHECK(RTIMERState, (obj), TYPE_RENESAS_TIMER)
+
+enum {
+TIMER_CH_CMT = 2,
+/* TMU have 5channels. It separated 0-2 and 3-4. */
+TIMER_CH_TMU = 3,
+};
+
+enum {
+RTIMER_FEAT_CMT,
+RTIMER_FEAT_TMU_LOW,
+RTIMER_FEAT_TMU_HIGH,
+};
+
+struct RTIMERState;
+
+struct channel_rtimer {
+uint32_t cnt;
+uint32_t cor;
+uint16_t ctrl;
+qemu_irq irq;
+int64_t base;
+int64_t next;
+uint64_t clk;
+bool start;
+QEMUTimer *timer;
+struct RTIMERState *tmrp;
+};
+
+typedef struct RTIMERState {
+SysBusDevice parent_obj;
+
+uint64_t input_freq;
+MemoryRegion memory;
+MemoryRegion memory_p4;
+MemoryRegion memory_a7;
+
+uint8_t tocr;
+struct channel_rtimer ch[TIMER_CH_TMU];
+uint32_t feature;
+int num_ch;
+} RTIMERState;
+
+#endif
diff --git a/hw/timer/renesas_timer.c b/hw/timer/renesas_timer.c
new file mode 100644
index 00..459c7a4708
--- /dev/null
+++ b/hw/timer/renesas_timer.c
@@ -0,0 +1,421 @@
+/*
+ * Renesas 16bit Compare-match timer
+ *
+ * Datasheet: RX62N Group, RX621 Group User's Manual: Hardware
+ * (Rev.1.40 R01UH0033EJ0140)
+ *
+ * Copyright (c) 2019 Yoshinori Sato
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "qemu/log.h"
+#include "qapi/error.h"
+#include "qemu/timer.h"
+#include "hw/hw.h"
+#include "hw/irq.h"
+#include "hw/sysbus.h"
+#include "hw/registerfields.h"
+#include "hw/qdev-properties.h"
+#include "hw/timer/renesas_timer.h"
+#include "migration/vmstate.h"
+#include "qemu/error-report.h"
+
+REG32(TOCR, 0)
+  FIELD(TOCR, TCOE, 0, 1)
+REG32(TSTR, 4)
+REG32(TCOR, 8)
+REG32(TCNT, 12)
+REG32(TCR, 16)
+  FIELD(TCR, TPSC, 0, 3)
+  FIELD(TCR, CKEG, 3, 2)
+  FIELD(TCR, UNIE, 5, 1)
+  FIELD(TCR, ICPE, 6, 2)
+  FIELD(TCR, UNF, 8, 1)
+  FIELD(TCR, ICPF, 9, 1)
+REG32(CMCR, 16)
+  FIELD(CMCR, CKS, 0, 2)
+  FIELD(CMCR, CMIE, 6, 1)
+REG32(TCPR, 20)
+
+#define IS_CMT(t) (t->feature == RTIMER_FEAT_CMT)
+
+static int clkdiv(RTIMERState *tmr, int ch)
+{
+if (IS_CMT(tmr)) {
+return 8 << (2 * FIELD_EX16(tmr->ch[ch].ctrl, CMCR, CKS));
+} else {
+if (FIELD_EX16(tmr->ch[ch].ctrl, TCR, TPSC) <= 5) {
+return 4 << (2 * FIELD_EX16(tmr->ch[ch].ctrl, TCR, TPSC));
+} else {
+return 0;
+}
+}
+}
+
+static void set_next_event(struct channel_rtimer *ch, int64_t now)
+{
+int64_t next;
+RTIMERState *tmr = ch->tmrp;
+if (IS_CMT(tmr)) {
+next = ch->cor - ch->cnt;
+} else {
+next = ch->cnt;
+}
+next *= ch->clk;
+ch->base = now;
+ch->next = now + next;
+timer_mod(ch->timer, ch->next);
+}
+
+static void timer_event(void *opaque)
+{
+struct channel_rtimer *ch = opaque;
+RTIMERState *tmr = ch->tmrp;
+
+if (IS_CMT(tmr)) {
+ch->cnt = 0;
+if (FIELD_EX16(ch->ctrl, CMCR, CMIE)) {
+qemu_irq_pulse(ch->irq);
+}
+} else {
+ch->cnt = ch->cor;
+if (!FIELD_EX16(ch->ctrl, TCR, UNF)) {
+ch->ctrl = FIELD_DP16(ch->ctrl, TCR, UNF, 1);
+qemu_set_irq(ch->irq, FIELD_EX16(ch->ctrl, TCR, UNIE));
+}
+}
+set_next_event(ch, ch->next);
+}
+
+static int64_t read_tcnt(RTIMERState *tmr, int ch)
+{
+int64_t delta, now = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+
+if (tmr->ch[ch].clk > 0) {
+delta = (now - 

[PATCH 01/10] hw/intc: RX62N interrupt controller (ICUa)

[Yoshinori Sato]

This implementation supported only ICUa.
Hardware manual.
https://www.renesas.com/us/en/doc/products/mpumcu/doc/rx_family/r01uh0033ej0140_rx62n.pdf

Signed-off-by: Yoshinori Sato 
---
 include/hw/intc/rx_icu.h |  56 ++
 hw/intc/rx_icu.c | 379 +++
 hw/intc/Makefile.objs|   1 +
 3 files changed, 436 insertions(+)
 create mode 100644 include/hw/intc/rx_icu.h
 create mode 100644 hw/intc/rx_icu.c

diff --git a/include/hw/intc/rx_icu.h b/include/hw/intc/rx_icu.h
new file mode 100644
index 00..acfcf06aef
--- /dev/null
+++ b/include/hw/intc/rx_icu.h
@@ -0,0 +1,56 @@
+#ifndef RX_ICU_H
+#define RX_ICU_H
+
+#include "qemu-common.h"
+#include "hw/irq.h"
+
+enum TRG_MODE {
+TRG_LEVEL = 0,
+TRG_NEDGE = 1,  /* Falling */
+TRG_PEDGE = 2,  /* Raising */
+TRG_BEDGE = 3,  /* Both */
+};
+
+struct IRQSource {
+enum TRG_MODE sense;
+int level;
+};
+
+enum {
+/* Software interrupt request */
+SWI = 27,
+NR_IRQS = 256,
+};
+
+struct RXICUState {
+SysBusDevice parent_obj;
+
+MemoryRegion memory;
+struct IRQSource src[NR_IRQS];
+char *icutype;
+uint32_t nr_irqs;
+uint32_t *map;
+uint32_t nr_sense;
+uint32_t *init_sense;
+
+uint8_t ir[NR_IRQS];
+uint8_t dtcer[NR_IRQS];
+uint8_t ier[NR_IRQS / 8];
+uint8_t ipr[142];
+uint8_t dmasr[4];
+uint16_t fir;
+uint8_t nmisr;
+uint8_t nmier;
+uint8_t nmiclr;
+uint8_t nmicr;
+int req_irq;
+qemu_irq _irq;
+qemu_irq _fir;
+qemu_irq _swi;
+};
+typedef struct RXICUState RXICUState;
+
+#define TYPE_RXICU "rx-icu"
+#define RXICU(obj) OBJECT_CHECK(RXICUState, (obj), TYPE_RXICU)
+
+#endif /* RX_ICU_H */
diff --git a/hw/intc/rx_icu.c b/hw/intc/rx_icu.c
new file mode 100644
index 00..ab9a300467
--- /dev/null
+++ b/hw/intc/rx_icu.c
@@ -0,0 +1,379 @@
+/*
+ * RX Interrupt Control Unit
+ *
+ * Warning: Only ICUa is supported.
+ *
+ * Datasheet: RX62N Group, RX621 Group User's Manual: Hardware
+ * (Rev.1.40 R01UH0033EJ0140)
+ *
+ * Copyright (c) 2019 Yoshinori Sato
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see .
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "qemu/log.h"
+#include "qapi/error.h"
+#include "cpu.h"
+#include "hw/hw.h"
+#include "hw/irq.h"
+#include "hw/sysbus.h"
+#include "hw/registerfields.h"
+#include "hw/qdev-properties.h"
+#include "hw/intc/rx_icu.h"
+#include "migration/vmstate.h"
+#include "qemu/error-report.h"
+
+REG8(IR, 0)
+  FIELD(IR, IR,  0, 1)
+REG8(DTCER, 0x100)
+  FIELD(DTCER, DTCE,  0, 1)
+REG8(IER, 0x200)
+REG8(SWINTR, 0x2e0)
+  FIELD(SWINTR, SWINT, 0, 1)
+REG16(FIR, 0x2f0)
+  FIELD(FIR, FVCT, 0, 8)
+  FIELD(FIR, FIEN, 15, 1)
+REG8(IPR, 0x300)
+  FIELD(IPR, IPR, 0, 4)
+REG8(DMRSR, 0x400)
+REG8(IRQCR, 0x500)
+  FIELD(IRQCR, IRQMD, 2, 2)
+REG8(NMISR, 0x580)
+  FIELD(NMISR, NMIST, 0, 1)
+  FIELD(NMISR, LVDST, 1, 1)
+  FIELD(NMISR, OSTST, 2, 1)
+REG8(NMIER, 0x581)
+  FIELD(NMIER, NMIEN, 0, 1)
+  FIELD(NMIER, LVDEN, 1, 1)
+  FIELD(NMIER, OSTEN, 2, 1)
+REG8(NMICLR, 0x582)
+  FIELD(NMICLR, NMICLR, 0, 1)
+  FIELD(NMICLR, OSTCLR, 2, 1)
+REG8(NMICR, 0x583)
+  FIELD(NMICR, NMIMD, 3, 1)
+
+#define request(icu, n) (icu->ipr[icu->map[n]] << 8 | n)
+
+static void set_irq(RXICUState *icu, int n_IRQ, int req)
+{
+if ((icu->fir & R_FIR_FIEN_MASK) &&
+(icu->fir & R_FIR_FVCT_MASK) == n_IRQ) {
+qemu_set_irq(icu->_fir, req);
+} else {
+qemu_set_irq(icu->_irq, req);
+}
+}
+
+static void rxicu_request(RXICUState *icu, int n_IRQ)
+{
+int enable;
+
+enable = icu->ier[n_IRQ / 8] & (1 << (n_IRQ & 7));
+if (n_IRQ > 0 && enable != 0 && atomic_read(>req_irq) < 0) {
+atomic_set(>req_irq, n_IRQ);
+set_irq(icu, n_IRQ, request(icu, n_IRQ));
+}
+}
+
+static void rxicu_set_irq(void *opaque, int n_IRQ, int level)
+{
+RXICUState *icu = opaque;
+struct IRQSource *src;
+int issue;
+
+if (n_IRQ >= NR_IRQS) {
+error_report("%s: IRQ %d out of range", __func__, n_IRQ);
+return;
+}
+
+src = >src[n_IRQ];
+
+level = (level != 0);
+switch (src->sense) {
+case TRG_LEVEL:
+/* level-sensitive irq */
+issue = level;
+src->level = level;
+break;
+case TRG_NEDGE:
+issue = (level == 0 && src->level == 1);
+src->level = level;
+break;
+case TRG_PEDGE:
+issue = (level 

[PATCH 07/10] hw/sh4: Convert renesas_sci.

[Yoshinori Sato]

Using new implementation SCI module.

Signed-off-by: Yoshinori Sato 
---
 include/hw/sh4/sh.h | 11 ---
 hw/sh4/sh7750.c | 45 +
 hw/sh4/Kconfig  |  1 +
 3 files changed, 42 insertions(+), 15 deletions(-)

diff --git a/include/hw/sh4/sh.h b/include/hw/sh4/sh.h
index 767a2df7e2..e184b4b300 100644
--- a/include/hw/sh4/sh.h
+++ b/include/hw/sh4/sh.h
@@ -38,17 +38,6 @@ void tmu012_init(struct MemoryRegion *sysmem, hwaddr base,
 qemu_irq ch2_irq0, qemu_irq ch2_irq1);
 
 
-/* sh_serial.c */
-#define SH_SERIAL_FEAT_SCIF (1 << 0)
-void sh_serial_init(MemoryRegion *sysmem,
-hwaddr base, int feat,
-uint32_t freq, Chardev *chr,
-qemu_irq eri_source,
-qemu_irq rxi_source,
-qemu_irq txi_source,
-qemu_irq tei_source,
-qemu_irq bri_source);
-
 /* sh7750.c */
 qemu_irq sh7750_irl(struct SH7750State *s);
 
diff --git a/hw/sh4/sh7750.c b/hw/sh4/sh7750.c
index d660714443..150d3029f7 100644
--- a/hw/sh4/sh7750.c
+++ b/hw/sh4/sh7750.c
@@ -24,12 +24,15 @@
  */
 
 #include "qemu/osdep.h"
+#include "qapi/error.h"
 #include "hw/irq.h"
 #include "hw/sh4/sh.h"
 #include "sysemu/sysemu.h"
 #include "sh7750_regs.h"
 #include "sh7750_regnames.h"
 #include "hw/sh4/sh_intc.h"
+#include "hw/char/renesas_sci.h"
+#include "hw/qdev-properties.h"
 #include "cpu.h"
 #include "exec/exec-all.h"
 
@@ -752,6 +755,40 @@ static const MemoryRegionOps sh7750_mmct_ops = {
 .endianness = DEVICE_NATIVE_ENDIAN,
 };
 
+static void sh_serial_init(SH7750State *s, MemoryRegion *sysmem,
+hwaddr base, int feat,
+uint32_t freq, Chardev *chr,
+qemu_irq eri_source,
+qemu_irq rxi_source,
+qemu_irq txi_source,
+qemu_irq tei_source,
+qemu_irq bri_source)
+{
+DeviceState *dev;
+SysBusDevice *sci;
+
+dev = qdev_create(NULL, TYPE_RENESAS_SCI);
+
+sci = SYS_BUS_DEVICE(dev);
+
+qdev_prop_set_chr(dev, "chardev", chr);
+qdev_prop_set_uint64(dev, "input-freq", freq);
+qdev_prop_set_int32(dev, "feature", feat);
+qdev_prop_set_int32(dev, "register-size", 32);
+qdev_init_nofail(dev);
+sysbus_mmio_map(sci, 0, base);
+sysbus_mmio_map(sci, 1, P4ADDR(base));
+sysbus_mmio_map(sci, 2, A7ADDR(base));
+sysbus_connect_irq(sci, 0, eri_source);
+sysbus_connect_irq(sci, 1, rxi_source);
+sysbus_connect_irq(sci, 2, txi_source);
+if (feat == SCI_FEAT_SCI) {
+sysbus_connect_irq(sci, 3, tei_source);
+} else {
+sysbus_connect_irq(sci, 3, bri_source);
+}
+}
+
 SH7750State *sh7750_init(SuperHCPU *cpu, MemoryRegion *sysmem)
 {
 SH7750State *s;
@@ -800,15 +837,15 @@ SH7750State *sh7750_init(SuperHCPU *cpu, MemoryRegion 
*sysmem)
 
 cpu->env.intc_handle = >intc;
 
-sh_serial_init(sysmem, 0x1fe0,
-   0, s->periph_freq, serial_hd(0),
+sh_serial_init(s, sysmem, 0x1fe0,
+   SCI_FEAT_SCI, s->periph_freq, serial_hd(0),
s->intc.irqs[SCI1_ERI],
s->intc.irqs[SCI1_RXI],
s->intc.irqs[SCI1_TXI],
s->intc.irqs[SCI1_TEI],
NULL);
-sh_serial_init(sysmem, 0x1fe8,
-   SH_SERIAL_FEAT_SCIF,
+sh_serial_init(s, sysmem, 0x1fe8,
+   SCI_FEAT_SCIF,
s->periph_freq, serial_hd(1),
s->intc.irqs[SCIF_ERI],
s->intc.irqs[SCIF_RXI],
diff --git a/hw/sh4/Kconfig b/hw/sh4/Kconfig
index 4cbce3a0ed..38509b7e65 100644
--- a/hw/sh4/Kconfig
+++ b/hw/sh4/Kconfig
@@ -22,3 +22,4 @@ config SH7750
 config SH4
 bool
 select PTIMER
+select RENESAS_SCI
-- 
2.20.1

[PATCH 06/10] Add rx-softmmu

[Yoshinori Sato]

Tested-by: Philippe Mathieu-Daudé 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Yoshinori Sato 
Message-Id: <20190607091116.49044-17-ys...@users.sourceforge.jp>
Signed-off-by: Richard Henderson 
pick ed65c02993 target/rx: Add RX to SysEmuTarget
pick 01372568ae tests: Add rx to machine-none-test.c
[PMD: Squashed patches from Richard Henderson modifying
  qapi/common.json and tests/machine-none-test.c]
Signed-off-by: Philippe Mathieu-Daudé 
---
 default-configs/rx-softmmu.mak | 1 +
 hw/Kconfig | 1 +
 2 files changed, 2 insertions(+)

diff --git a/default-configs/rx-softmmu.mak b/default-configs/rx-softmmu.mak
index 7c4eb2c1a0..a3eecefb11 100644
--- a/default-configs/rx-softmmu.mak
+++ b/default-configs/rx-softmmu.mak
@@ -1,2 +1,3 @@
 # Default configuration for rx-softmmu
 
+CONFIG_RX_VIRT=y
diff --git a/hw/Kconfig b/hw/Kconfig
index ecf491bf04..62f9ebdc22 100644
--- a/hw/Kconfig
+++ b/hw/Kconfig
@@ -55,6 +55,7 @@ source nios2/Kconfig
 source openrisc/Kconfig
 source ppc/Kconfig
 source riscv/Kconfig
+source rx/Kconfig
 source s390x/Kconfig
 source sh4/Kconfig
 source sparc/Kconfig
-- 
2.20.1

[PATCH 10/10] hw/timer: remove sh_timer.c

[Yoshinori Sato]

SH4 TMU using new module. This file is obsolute.

Signed-off-by: Yoshinori Sato 
---
 hw/timer/sh_timer.c | 341 
 1 file changed, 341 deletions(-)
 delete mode 100644 hw/timer/sh_timer.c

diff --git a/hw/timer/sh_timer.c b/hw/timer/sh_timer.c
deleted file mode 100644
index 13c4051808..00
--- a/hw/timer/sh_timer.c
+++ /dev/null
@@ -1,341 +0,0 @@
-/*
- * SuperH Timer modules.
- *
- * Copyright (c) 2007 Magnus Damm
- * Based on arm_timer.c by Paul Brook
- * Copyright (c) 2005-2006 CodeSourcery.
- *
- * This code is licensed under the GPL.
- */
-
-#include "qemu/osdep.h"
-#include "hw/hw.h"
-#include "hw/irq.h"
-#include "hw/sh4/sh.h"
-#include "qemu/timer.h"
-#include "hw/ptimer.h"
-
-//#define DEBUG_TIMER
-
-#define TIMER_TCR_TPSC  (7 << 0)
-#define TIMER_TCR_CKEG  (3 << 3)
-#define TIMER_TCR_UNIE  (1 << 5)
-#define TIMER_TCR_ICPE  (3 << 6)
-#define TIMER_TCR_UNF   (1 << 8)
-#define TIMER_TCR_ICPF  (1 << 9)
-#define TIMER_TCR_RESERVED  (0x3f << 10)
-
-#define TIMER_FEAT_CAPT   (1 << 0)
-#define TIMER_FEAT_EXTCLK (1 << 1)
-
-#define OFFSET_TCOR   0
-#define OFFSET_TCNT   1
-#define OFFSET_TCR2
-#define OFFSET_TCPR   3
-
-typedef struct {
-ptimer_state *timer;
-uint32_t tcnt;
-uint32_t tcor;
-uint32_t tcr;
-uint32_t tcpr;
-int freq;
-int int_level;
-int old_level;
-int feat;
-int enabled;
-qemu_irq irq;
-} sh_timer_state;
-
-/* Check all active timers, and schedule the next timer interrupt. */
-
-static void sh_timer_update(sh_timer_state *s)
-{
-int new_level = s->int_level && (s->tcr & TIMER_TCR_UNIE);
-
-if (new_level != s->old_level)
-  qemu_set_irq (s->irq, new_level);
-
-s->old_level = s->int_level;
-s->int_level = new_level;
-}
-
-static uint32_t sh_timer_read(void *opaque, hwaddr offset)
-{
-sh_timer_state *s = (sh_timer_state *)opaque;
-
-switch (offset >> 2) {
-case OFFSET_TCOR:
-return s->tcor;
-case OFFSET_TCNT:
-return ptimer_get_count(s->timer);
-case OFFSET_TCR:
-return s->tcr | (s->int_level ? TIMER_TCR_UNF : 0);
-case OFFSET_TCPR:
-if (s->feat & TIMER_FEAT_CAPT)
-return s->tcpr;
-/* fall through */
-default:
-hw_error("sh_timer_read: Bad offset %x\n", (int)offset);
-return 0;
-}
-}
-
-static void sh_timer_write(void *opaque, hwaddr offset,
-uint32_t value)
-{
-sh_timer_state *s = (sh_timer_state *)opaque;
-int freq;
-
-switch (offset >> 2) {
-case OFFSET_TCOR:
-s->tcor = value;
-ptimer_transaction_begin(s->timer);
-ptimer_set_limit(s->timer, s->tcor, 0);
-ptimer_transaction_commit(s->timer);
-break;
-case OFFSET_TCNT:
-s->tcnt = value;
-ptimer_transaction_begin(s->timer);
-ptimer_set_count(s->timer, s->tcnt);
-ptimer_transaction_commit(s->timer);
-break;
-case OFFSET_TCR:
-ptimer_transaction_begin(s->timer);
-if (s->enabled) {
-/* Pause the timer if it is running.  This may cause some
-   inaccuracy dure to rounding, but avoids a whole lot of other
-   messyness.  */
-ptimer_stop(s->timer);
-}
-freq = s->freq;
-/* ??? Need to recalculate expiry time after changing divisor.  */
-switch (value & TIMER_TCR_TPSC) {
-case 0: freq >>= 2; break;
-case 1: freq >>= 4; break;
-case 2: freq >>= 6; break;
-case 3: freq >>= 8; break;
-case 4: freq >>= 10; break;
-   case 6:
-   case 7: if (s->feat & TIMER_FEAT_EXTCLK) break;
-   default: hw_error("sh_timer_write: Reserved TPSC value\n"); break;
-}
-switch ((value & TIMER_TCR_CKEG) >> 3) {
-   case 0: break;
-case 1:
-case 2:
-case 3: if (s->feat & TIMER_FEAT_EXTCLK) break;
-   default: hw_error("sh_timer_write: Reserved CKEG value\n"); break;
-}
-switch ((value & TIMER_TCR_ICPE) >> 6) {
-   case 0: break;
-case 2:
-case 3: if (s->feat & TIMER_FEAT_CAPT) break;
-   default: hw_error("sh_timer_write: Reserved ICPE value\n"); break;
-}
-   if ((value & TIMER_TCR_UNF) == 0)
-s->int_level = 0;
-
-   value &= ~TIMER_TCR_UNF;
-
-   if ((value & TIMER_TCR_ICPF) && (!(s->feat & TIMER_FEAT_CAPT)))
-hw_error("sh_timer_write: Reserved ICPF value\n");
-
-   value &= ~TIMER_TCR_ICPF; /* capture not supported */
-
-   if (value & TIMER_TCR_RESERVED)
-hw_error("sh_timer_write: Reserved TCR bits set\n");
-s->tcr = value;
-ptimer_set_limit(s->timer, s->tcor, 0);
-ptimer_set_freq(s->timer, freq);
-if (s->enabled) {
-/* Restart the timer if still enabled.  */
-ptimer_run(s->timer, 0);
-}
-

[PATCH 00/10] Add RX hardware emulation

[Yoshinori Sato]

Hello.

This series add to hardware emulation module for RX target.

Details below.
Interrupt controller, 8bit timer, 16bit comapare match timer and
SCI is RX62N integrated peripheral.
rx-virt - RX62N MCU and external RAM. It like gdb simulator.

The compare match timer has a CPU interface similar to the SH4 timer.
sh_timer will be deprecated and integrated into this module.
SCI is also implemented in sh_serial, but the functionality is omitted.
I implemented the complete one as renesas_sci.

git repository here.
git://git.pf.osdn.net/gitroot/y/ys/ysato/qemu.git tags/hw-rx-20200601

Yoshinori Sato (10):
  hw/intc: RX62N interrupt controller (ICUa)
  hw/timer: Renesas 8bit timer module.
  hw/timer: Renesas TMU/CMT module.
  hw/char: Renesas SCI module.
  hw/rx: RX MCU and target
  Add rx-softmmu
  hw/sh4: Convert renesas_sci.
  hw/char: remove sh_serial.c
  hw/sh4: Convert to renesas_timer.c
  hw/timer: remove sh_timer.c

 default-configs/rx-softmmu.mak|   1 +
 include/hw/char/renesas_sci.h |  77 +++
 include/hw/intc/rx_icu.h  |  56 +++
 include/hw/rx/rx.h|   7 +
 include/hw/rx/rx62n.h |  91 
 include/hw/sh4/sh.h   |  21 -
 include/hw/timer/renesas_8timer.h |  61 +++
 include/hw/timer/renesas_timer.h  |  59 +++
 hw/char/renesas_sci.c | 786 ++
 hw/char/sh_serial.c   | 431 
 hw/intc/rx_icu.c  | 379 ++
 hw/rx/rx-virt.c   | 143 ++
 hw/rx/rx62n.c | 240 +
 hw/sh4/sh7750.c   |  78 ++-
 hw/timer/renesas_8timer.c | 466 ++
 hw/timer/renesas_timer.c  | 421 
 hw/timer/sh_timer.c   | 341 -
 hw/Kconfig|   1 +
 hw/char/Kconfig   |   3 +
 hw/char/Makefile.objs |   3 +-
 hw/intc/Makefile.objs |   1 +
 hw/rx/Kconfig |  13 +
 hw/rx/Makefile.objs   |   2 +
 hw/sh4/Kconfig|   3 +-
 hw/timer/Kconfig  |   6 +
 hw/timer/Makefile.objs|   4 +-
 26 files changed, 2891 insertions(+), 803 deletions(-)
 create mode 100644 include/hw/char/renesas_sci.h
 create mode 100644 include/hw/intc/rx_icu.h
 create mode 100644 include/hw/rx/rx.h
 create mode 100644 include/hw/rx/rx62n.h
 create mode 100644 include/hw/timer/renesas_8timer.h
 create mode 100644 include/hw/timer/renesas_timer.h
 create mode 100644 hw/char/renesas_sci.c
 delete mode 100644 hw/char/sh_serial.c
 create mode 100644 hw/intc/rx_icu.c
 create mode 100644 hw/rx/rx-virt.c
 create mode 100644 hw/rx/rx62n.c
 create mode 100644 hw/timer/renesas_8timer.c
 create mode 100644 hw/timer/renesas_timer.c
 delete mode 100644 hw/timer/sh_timer.c
 create mode 100644 hw/rx/Kconfig
 create mode 100644 hw/rx/Makefile.objs

-- 
2.20.1

[PATCH 09/10] hw/sh4: Convert to renesas_timer.c

[Yoshinori Sato]

Using unified TMU/CMT module.

Signed-off-by: Yoshinori Sato 
---
 include/hw/sh4/sh.h| 10 -
 hw/sh4/sh7750.c| 47 +-
 hw/sh4/Kconfig |  2 +-
 hw/timer/Makefile.objs |  1 -
 4 files changed, 38 insertions(+), 22 deletions(-)

diff --git a/include/hw/sh4/sh.h b/include/hw/sh4/sh.h
index e184b4b300..248cc052e1 100644
--- a/include/hw/sh4/sh.h
+++ b/include/hw/sh4/sh.h
@@ -28,16 +28,6 @@ typedef struct {
 
 int sh7750_register_io_device(struct SH7750State *s,
  sh7750_io_device * device);
-/* sh_timer.c */
-#define TMU012_FEAT_TOCR   (1 << 0)
-#define TMU012_FEAT_3CHAN  (1 << 1)
-#define TMU012_FEAT_EXTCLK (1 << 2)
-void tmu012_init(struct MemoryRegion *sysmem, hwaddr base,
- int feat, uint32_t freq,
-qemu_irq ch0_irq, qemu_irq ch1_irq,
-qemu_irq ch2_irq0, qemu_irq ch2_irq1);
-
-
 /* sh7750.c */
 qemu_irq sh7750_irl(struct SH7750State *s);
 
diff --git a/hw/sh4/sh7750.c b/hw/sh4/sh7750.c
index 150d3029f7..733c4009e1 100644
--- a/hw/sh4/sh7750.c
+++ b/hw/sh4/sh7750.c
@@ -32,6 +32,7 @@
 #include "sh7750_regnames.h"
 #include "hw/sh4/sh_intc.h"
 #include "hw/char/renesas_sci.h"
+#include "hw/timer/renesas_timer.h"
 #include "hw/qdev-properties.h"
 #include "cpu.h"
 #include "exec/exec-all.h"
@@ -756,13 +757,13 @@ static const MemoryRegionOps sh7750_mmct_ops = {
 };
 
 static void sh_serial_init(SH7750State *s, MemoryRegion *sysmem,
-hwaddr base, int feat,
-uint32_t freq, Chardev *chr,
-qemu_irq eri_source,
-qemu_irq rxi_source,
-qemu_irq txi_source,
-qemu_irq tei_source,
-qemu_irq bri_source)
+   hwaddr base, int feat,
+   uint32_t freq, Chardev *chr,
+   qemu_irq eri_source,
+   qemu_irq rxi_source,
+   qemu_irq txi_source,
+   qemu_irq tei_source,
+   qemu_irq bri_source)
 {
 DeviceState *dev;
 SysBusDevice *sci;
@@ -789,6 +790,31 @@ static void sh_serial_init(SH7750State *s, MemoryRegion 
*sysmem,
 }
 }
 
+static void tmu012_init(SH7750State *s, MemoryRegion *sysmem, hwaddr base,
+int feat, uint32_t freq,
+qemu_irq ch0_irq, qemu_irq ch1_irq,
+qemu_irq ch2_irq0, qemu_irq ch2_irq1)
+{
+DeviceState *dev;
+SysBusDevice *tmu;
+
+dev = qdev_create(NULL, TYPE_RENESAS_TIMER);
+
+tmu = SYS_BUS_DEVICE(dev);
+
+qdev_prop_set_uint64(dev, "input-freq", freq);
+qdev_prop_set_int32(dev, "feature", feat);
+qdev_init_nofail(dev);
+sysbus_mmio_map(tmu, 0, base);
+sysbus_mmio_map(tmu, 1, P4ADDR(base));
+sysbus_mmio_map(tmu, 2, A7ADDR(base));
+sysbus_connect_irq(tmu, 0, ch0_irq);
+sysbus_connect_irq(tmu, 1, ch1_irq);
+if (ch2_irq0) {
+sysbus_connect_irq(tmu, 2, ch2_irq0);
+}
+}
+
 SH7750State *sh7750_init(SuperHCPU *cpu, MemoryRegion *sysmem)
 {
 SH7750State *s;
@@ -853,8 +879,8 @@ SH7750State *sh7750_init(SuperHCPU *cpu, MemoryRegion 
*sysmem)
NULL,
s->intc.irqs[SCIF_BRI]);
 
-tmu012_init(sysmem, 0x1fd8,
-   TMU012_FEAT_TOCR | TMU012_FEAT_3CHAN | TMU012_FEAT_EXTCLK,
+tmu012_init(s, sysmem, 0x1fd8,
+RTIMER_FEAT_TMU_LOW,
s->periph_freq,
s->intc.irqs[TMU0],
s->intc.irqs[TMU1],
@@ -877,7 +903,8 @@ SH7750State *sh7750_init(SuperHCPU *cpu, MemoryRegion 
*sysmem)
 sh_intc_register_sources(>intc,
 _INTC_ARRAY(vectors_tmu34),
 NULL, 0);
-tmu012_init(sysmem, 0x1e10, 0, s->periph_freq,
+tmu012_init(s, sysmem, 0x1e10,
+RTIMER_FEAT_TMU_HIGH, s->periph_freq,
s->intc.irqs[TMU3],
s->intc.irqs[TMU4],
NULL, NULL);
diff --git a/hw/sh4/Kconfig b/hw/sh4/Kconfig
index 38509b7e65..ac21830bb5 100644
--- a/hw/sh4/Kconfig
+++ b/hw/sh4/Kconfig
@@ -21,5 +21,5 @@ config SH7750
 
 config SH4
 bool
-select PTIMER
+select RENESAS_TIMER
 select RENESAS_SCI
diff --git a/hw/timer/Makefile.objs b/hw/timer/Makefile.objs
index 289b6b03ab..f3918836e4 100644
--- a/hw/timer/Makefile.objs
+++ b/hw/timer/Makefile.objs
@@ -22,7 +22,6 @@ common-obj-$(CONFIG_EXYNOS4) += exynos4210_pwm.o
 common-obj-$(CONFIG_OMAP) += omap_gptimer.o
 common-obj-$(CONFIG_OMAP) += omap_synctimer.o
 common-obj-$(CONFIG_PXA2XX) += pxa2xx_timer.o
-common-obj-$(CONFIG_SH4) += sh_timer.o
 common-obj-$(CONFIG_DIGIC) += digic-timer.o
 common-obj-$(CONFIG_MIPS_CPS) += mips_gictimer.o
 
-- 
2.20.1

Re: [PATCH-for-5.0] gdbstub: Use correct address space with Qqemu.PhyMemMode packet

[Philippe Mathieu-Daudé]

On 3/30/20 6:41 PM, Peter Maydell wrote:
> On Mon, 30 Mar 2020 at 17:21, Philippe Mathieu-Daudé  
> wrote:
>> On 3/30/20 6:08 PM, Peter Maydell wrote:
>>> On Mon, 30 Mar 2020 at 16:30, Philippe Mathieu-Daudé  
>>> wrote:

 Since commit 3f940dc98, we added support for vAttach packet
 to select a particular thread/cpu/core. However when using
 the GDB physical memory mode, it is not clear which CPU
 address space is used.
 Since the CPU address space is stored in CPUState::as, use
 address_space_rw() instead of cpu_physical_memory_rw().

 Fixes: ab4752ec8d9
 Signed-off-by: Philippe Mathieu-Daudé 
 ---
   gdbstub.c | 7 ++-
   1 file changed, 2 insertions(+), 5 deletions(-)

 diff --git a/gdbstub.c b/gdbstub.c
 index 013fb1ac0f..3baaef50e3 100644
 --- a/gdbstub.c
 +++ b/gdbstub.c
 @@ -69,11 +69,8 @@ static inline int target_memory_rw_debug(CPUState *cpu, 
 target_ulong addr,

   #ifndef CONFIG_USER_ONLY
   if (phy_memory_mode) {
 -if (is_write) {
 -cpu_physical_memory_write(addr, buf, len);
 -} else {
 -cpu_physical_memory_read(addr, buf, len);
 -}
 +address_space_rw(cpu->as, addr, MEMTXATTRS_UNSPECIFIED,
 + buf, len, is_write);
   return 0;
>>>
>>> There's an argument here for using
>>> int asidx = cpu_asidx_from_attrs(cpu, MEMTXATTRS_UNSPECIFIED);
>>> AddressSpace *as = cpu_get_address_space(cpu, asidx);
>>>
>>> though it will effectively boil down to the same thing in the end
>>> as there's no way for the gdbstub to specify whether it wanted
>>> eg the Arm secure or non-secure physical address space.
>>
>> https://static.docs.arm.com/ihi0074/a/debug_interface_v6_0_architecture_specification_IHI0074A.pdf
>>
>> * Configuration of hypervisor noninvasive debug.
>>
>> This field can have one of the following values:
>>
>> - 0b00
>> Separate controls for hypervisor noninvasive debug are not implemented,
>> or no hypervisor is implemented. For ARMv7 PEs that implement the
>> Virtualization Extensions, and for ARMv8 PEs that implement EL2, if
>> separate controls for hypervisor debug visibility are not implemented,
>> the hypervisor debug visibility is indicated by the relevant Non-secure
>> debug visibility fields NSNID and NSID.
>>
>> OK so for ARM "noninvasive debug is not implemented" and we would use
>> the core secure address space?
> 
> I'm not very familiar with the debug interface (we don't model
> it in QEMU), but I think that is the wrong end of it. These
> are bits in AUTHSTATUS, which is a read-only register provided
> by the CPU to the debugger. It basically says "am I, the CPU,
> going to permit you, the debugger, to debug code running
> in secure mode, or in EL2". (The CPU gets to decide this:
> for security some h/w will not want any random with access
> to the jtag debug port to be able to just read out code from
> the secure world, for instance.)
> 
> What the debugger gets to control is bits in the CSW register
> in the "MEM-AP"; it can use these to specify the size of
> a memory access it wants to make to the guest, and also
> the 'type', which is IMPDEF but typically lets the debugger
> say "code access vs data access", "privileged vs usermode"
> and "secure vs non-secure".
> 
> The equivalent in the QEMU world is that the debugger can
> specify the memory transaction attributes. The question is
> whether the gdb protocol provides any equivalent of that:
> if it doesn't then gdbstub.c has to make up an attribute and
> use that.
> 
>> Instead of MEMTXATTRS_UNSPECIFIED I should use a crafted MemTxAttrs with
>> .secure = 1, .unspecified = 1?
> 
> You shouldn't set 'unspecified = 1', because that indicates
> "this is MEMTXATTRS_UNSPECIFIED". The default set of
> unspecified-attributes are probably good enough,
> though, so you can just use MEMTXATTRS_UNSPECIFIED.
> 
>> The idea of this command is to use the
>> CPU AS but not the MMU/MPU, maybe it doesn't make sense...
> 
> The trouble is that the command isn't precise enough.
> "read/write to physical memory" is fine if the CPU has
> exactly one physical address space, but it's ambiguous if the CPU
> has more than one physical address space. Either we need the
> user to be able to tell us which one they wanted somehow
> (which for QEMU more or less means that they should tell
> us what tx attributes they wanted), or we need to make an
> arbitrary decision.
> 
> PS: do we have any documentation of this new command ?
> ab4752ec8d9 has the implementation but no documentation...

Jon, do you have documentation on the Qqemu.PhyMemMode packet?

> 
> thanks
> -- PMM
> 

Re: [PATCH v3 00/11] Record/replay acceptance tests

[Philippe Mathieu-Daudé]

On 5/29/20 9:04 AM, Pavel Dovgalyuk wrote:
> The following series adds record/replay tests to the acceptance group.
> Test pass successfully with the latest submitted record/replay fixes:
>  - replay: notify the main loop when there are no instructions
>  - replay: synchronize on every virtual timer callback
> 
> The provided tests perform kernel boot and disk image boot scenarios.
> For all of them recording and replaying phases are executed.
> Tests were borrowed from existing boot_linux*.py tests. But some
> of the platforms and images were excluded, because icount for them
> still has some issues.
> 
> Tested-by: Philippe Mathieu-Daude 
> 
> v3 changes:
>  - Added record/replay logging (suggested by Philippe Mathieu-Daudé)
>  - Changed the console pattern to get longer kernel execution (suggested by 
> Alex Bennée)
>  - Coding style fixes
>  - Other minor changes
> v2 changes:
>  - Some test structure refactoring (suggested by Willian Rampazzo)
> 
> ---
> 
> Pavel Dovgaluk (11):
>   tests/acceptance: allow console interaction with specific VMs
>   tests/acceptance: refactor boot_linux_console test to allow code reuse
>   tests/acceptance: add base class record/replay kernel tests
>   tests/acceptance: add kernel record/replay test for x86_64
>   tests/acceptance: add record/replay test for aarch64
>   tests/acceptance: add record/replay test for arm
>   tests/acceptance: add record/replay test for ppc64
>   tests/acceptance: add record/replay test for m68k
>   tests/acceptance: record/replay tests with advcal images
>   tests/acceptance: refactor boot_linux to allow code reuse
>   tests/acceptance: Linux boot test for record/replay

Thanks, patches 1/2/10 applied to my python-next tree:
https://gitlab.com/philmd/qemu/commits/python-next

- tests/acceptance: allow console interaction with specific VMs
- tests/acceptance: refactor boot_linux_console test to allow code reuse
- tests/acceptance: refactor boot_linux to allow code reuse

[PATCH v2 0/1] Check for page crossings in use_goto_tb() for rx target

[Ahmed Karaman]

Hi,

After discussion on v1 of this series, the conclusion was that page
crossings must be done for all targets and for both user and system
mode. This series deals with rx target, that is the only target that
does not perform this check in system mode.

In version two of this series, the original use_goto_tb() function in
the hppa target is left unchanged. For the rx target, it's modified to
check for the page crossings in both modes along with other minor
fixes.

First version of the series:
https://lists.nongnu.org/archive/html/qemu-devel/2020-05/msg05426.html

Best regards,
Ahmed Karaman

v1->v2:
- Skip the patch related to the use_goto_tb() of the hppa target.
- Apply the page crossings check in use_goto_tb() in both modes for
  the rx target.
- Add appropriate comments in rx use_goto_tb().

Ahmed Karaman (1):
  target/rx: Check for page crossings in use_goto_tb()

 target/rx/translate.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

-- 
2.17.1

[PATCH v2 1/1] target/rx: Check for page crossings in use_goto_tb()

[Ahmed Karaman]

Add the page crossings check in use_goto_tb(). If this check is not
applied, a number of bugs may occasionally occur during target rx
system mode emulation.
Also, this check is needed in user mode related to emulation of system
call mmap(). rx target does not currently support user mode, but it is
better to prepare use_goto_tb() in that sense in advance.

Rename parameter dc of type DisasContext* to the more common name ctx,
to keep consistency with other targets.

Add detailed comments.

Buglink: https://bugs.launchpad.net/qemu/+bug/1880763
Signed-off-by: Ahmed Karaman 
---
 target/rx/translate.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/target/rx/translate.c b/target/rx/translate.c
index 61e86653a4..85a884c7dd 100644
--- a/target/rx/translate.c
+++ b/target/rx/translate.c
@@ -143,12 +143,15 @@ void rx_cpu_dump_state(CPUState *cs, FILE *f, int flags)
 }
 }
 
-static bool use_goto_tb(DisasContext *dc, target_ulong dest)
+static bool use_goto_tb(DisasContext *ctx, target_ulong dest)
 {
-if (unlikely(dc->base.singlestep_enabled)) {
+/* No direct translation block linking in singlestep */
+if (unlikely(ctx->base.singlestep_enabled)) {
 return false;
 } else {
-return true;
+/* Directly link translation blocks only within the same guest page */
+return (ctx->base.tb->pc & TARGET_PAGE_MASK) ==
+   (dest & TARGET_PAGE_MASK);
 }
 }
 
-- 
2.17.1

[Bug 1881450] Re: Emulation of a math function fails for m68k Linux user mode

[Laurent Vivier]

Fix available.

Execution doesn't fail anymore:

  Profiling function expm1f():
Elapsed time: 41 ms
Control result: 71805.108342

Control result matches real hardware one:

  Profiling function expm1f():
Elapsed time: 2152 ms
Control result: 71805.108342


** Changed in: qemu
   Status: New => In Progress

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1881450

Title:
  Emulation of a math function fails for m68k Linux user mode

Status in QEMU:
  In Progress

Bug description:
  Please check the attached math-example.c file.
  When running the m68k executable under QEMU, it results in an 
  "Illegal instruction" error.
  Other targets don't produce this error.

  Steps to reproduce the bug:

  1. Download the math-example.c attached file.
  2. Compile it by running:
     m68k-linux-gnu-gcc -O2 -static math-example.c -o math-example-m68k -lm
  3. Run the executable with QEMU:
     /build/qemu-5.0.0/build-gcc/m68k-linux-user/qemu-m68k math-example-m68k

  The output of execution is:
     Profiling function expm1f():
     qemu: uncaught target signal 4 (Illegal instruction) - core dumped
     Illegal instruction (core dumped)

  Output when running on other targets:
     Profiling function expm1f():
   Elapsed time: 47 ms
   Control result: 71804.953125

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1881450/+subscriptions

[Bug 1881450] Re: Emulation of a math function fails for m68k Linux user mode

[Laurent Vivier]

** Patch added: "0001-target-m68k-implement-opcode-fetoxm1x.patch"
   
https://bugs.launchpad.net/qemu/+bug/1881450/+attachment/5379012/+files/0001-target-m68k-implement-opcode-fetoxm1x.patch

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1881450

Title:
  Emulation of a math function fails for m68k Linux user mode

Status in QEMU:
  In Progress

Bug description:
  Please check the attached math-example.c file.
  When running the m68k executable under QEMU, it results in an 
  "Illegal instruction" error.
  Other targets don't produce this error.

  Steps to reproduce the bug:

  1. Download the math-example.c attached file.
  2. Compile it by running:
     m68k-linux-gnu-gcc -O2 -static math-example.c -o math-example-m68k -lm
  3. Run the executable with QEMU:
     /build/qemu-5.0.0/build-gcc/m68k-linux-user/qemu-m68k math-example-m68k

  The output of execution is:
     Profiling function expm1f():
     qemu: uncaught target signal 4 (Illegal instruction) - core dumped
     Illegal instruction (core dumped)

  Output when running on other targets:
     Profiling function expm1f():
   Elapsed time: 47 ms
   Control result: 71804.953125

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1881450/+subscriptions

[PATCH] target/m68k: implement opcode fetoxm1

[Laurent Vivier]

Example provided in the launchpad bug fails with:

   qemu: uncaught target signal 4 (Illegal instruction) - core dumped
   Illegal instruction (core dumped)

It appears fetoxm1 is not implemented:

   IN: expm1f
   0x85cc: fetoxm1x %fp2,%fp0
   Disassembler disagrees with translator over instruction decoding
   Please report this to qemu-devel@nongnu.org

   (gdb) x/2hx 0x85cc
   0x85cc: 0xf200 0x0808

This patch adds the instruction.

Bug: https://bugs.launchpad.net/qemu/+bug/1881450
Signed-off-by: Laurent Vivier 
---
 target/m68k/helper.h | 1 +
 target/m68k/fpu_helper.c | 5 +
 target/m68k/translate.c  | 3 +++
 3 files changed, 9 insertions(+)

diff --git a/target/m68k/helper.h b/target/m68k/helper.h
index feee7be62655..77808497a95c 100644
--- a/target/m68k/helper.h
+++ b/target/m68k/helper.h
@@ -83,6 +83,7 @@ DEF_HELPER_3(fatan, void, env, fp, fp)
 DEF_HELPER_3(fasin, void, env, fp, fp)
 DEF_HELPER_3(facos, void, env, fp, fp)
 DEF_HELPER_3(fatanh, void, env, fp, fp)
+DEF_HELPER_3(fetoxm1, void, env, fp, fp)
 DEF_HELPER_3(ftanh, void, env, fp, fp)
 DEF_HELPER_3(fsinh, void, env, fp, fp)
 DEF_HELPER_3(fcosh, void, env, fp, fp)
diff --git a/target/m68k/fpu_helper.c b/target/m68k/fpu_helper.c
index 36e6c704d1d0..9acf60dfd443 100644
--- a/target/m68k/fpu_helper.c
+++ b/target/m68k/fpu_helper.c
@@ -641,6 +641,11 @@ void HELPER(fatanh)(CPUM68KState *env, FPReg *res, FPReg 
*val)
 res->d = floatx80_atanh(val->d, >fp_status);
 }
 
+void HELPER(fetoxm1)(CPUM68KState *env, FPReg *res, FPReg *val)
+{
+res->d = floatx80_etoxm1(val->d, >fp_status);
+}
+
 void HELPER(ftanh)(CPUM68KState *env, FPReg *res, FPReg *val)
 {
 res->d = floatx80_tanh(val->d, >fp_status);
diff --git a/target/m68k/translate.c b/target/m68k/translate.c
index 0f80888203d3..db4e69bac195 100644
--- a/target/m68k/translate.c
+++ b/target/m68k/translate.c
@@ -5146,6 +5146,9 @@ DISAS_INSN(fpu)
 case 0x06: /* flognp1 */
 gen_helper_flognp1(cpu_env, cpu_dest, cpu_src);
 break;
+case 0x08: /* fetoxm1 */
+gen_helper_fetoxm1(cpu_env, cpu_dest, cpu_src);
+break;
 case 0x09: /* ftanh */
 gen_helper_ftanh(cpu_env, cpu_dest, cpu_src);
 break;
-- 
2.26.2

Re: [PATCH] target/m68k: implement fmove.l #,FPCR

[John Paul Adrian Glaubitz]

Hi Laurent!

On 5/31/20 2:09 PM, Laurent Vivier wrote:
> I guess you are using my q800-dev branch?

That's what I initially did, then I pulled from upstream.

> In this branch, there is an attempt to manage unnormalized numbers that
> seems to trigger this lock up.
> 
> You can either use master + this patch or update your q800-dev branch
> from my repo.

However, I then still copied the compiled binary from the qemu-m68k where
I first tested the patch. Guess I should be more careful when using the
bash history with absolute paths ;).

So, I can confirm it works for me.

Tested-by: John Paul Adrian Glaubitz 

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

[Bug 1881450] Re: Emulation of a math function fails for m68k Linux user mode

[Laurent Vivier]

Tracing gives me:

IN: expm1f
0x85cc:  fetoxm1x %fp2,%fp0
Disassembler disagrees with translator over instruction decoding
Please report this to qemu-devel@nongnu.org

(gdb) x/2hx 0x85cc
0x85cc: 0xf200  0x0808

The instruction is not implemented in qemu. I fix that.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1881450

Title:
  Emulation of a math function fails for m68k Linux user mode

Status in QEMU:
  New

Bug description:
  Please check the attached math-example.c file.
  When running the m68k executable under QEMU, it results in an 
  "Illegal instruction" error.
  Other targets don't produce this error.

  Steps to reproduce the bug:

  1. Download the math-example.c attached file.
  2. Compile it by running:
     m68k-linux-gnu-gcc -O2 -static math-example.c -o math-example-m68k -lm
  3. Run the executable with QEMU:
     /build/qemu-5.0.0/build-gcc/m68k-linux-user/qemu-m68k math-example-m68k

  The output of execution is:
     Profiling function expm1f():
     qemu: uncaught target signal 4 (Illegal instruction) - core dumped
     Illegal instruction (core dumped)

  Output when running on other targets:
     Profiling function expm1f():
   Elapsed time: 47 ms
   Control result: 71804.953125

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1881450/+subscriptions

Re: [PATCH] target/m68k: implement fmove.l #,FPCR

[Laurent Vivier]

Le 31/05/2020 à 13:23, John Paul Adrian Glaubitz a écrit :
> Hi Laurent!
> 
> On 5/31/20 1:02 PM, Laurent Vivier wrote:
>> The immediate value mode was ignored and instruction execution
>> ends to an invalid access mode.
>>
>> This was found running 'R' that set FPSR to 0 at startup with
>> a 'fmove.l #0,FPSR' in qemu-system-m68k emulation and triggers a
>> kernel crash:
>> (...)
>> Reported-by: John Paul Adrian Glaubitz 
>> Signed-off-by: Laurent Vivier 
>> ---
>>  target/m68k/translate.c | 14 ++
>>  1 file changed, 14 insertions(+)
> Thanks for the fix.
> 
> I applied the patch, but I'm getting a lock-up now as you previously reported
> in the other discussion on the Linux/m68k mailing list:
> 
> root@pacman:~# R
> [   68.42] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [R:650]
> [   68.42] Modules linked in: sg evdev mac_hid ip_tables x_tables 
> sha1_generic hmac ipv6 nf_defrag_ipv6 autofs4 ext4 crc16 mbcache jbd2 
> crc32c_generic sd_mod t10_pi crc_t10dif sr_mod cdrom crct10dif_generic 
> crct10dif_common mac_esp macsonic esp_scsi
> [   68.42] Format 00  Vector: 0064  PC: 0002df9c  Status: 2008Not 
> tainted
> [   68.42] ORIG_D0:   D0:   A2: c02e239a  A1: ffa1
> [   68.42] A0: 3c9adf29  D5: 000d  D4: 8002ce30
> [   68.42] D3: 8002b418  D2: 8002b4b4  D1: 
> 
> Is this related or a different bug?
> 
> I have not seen these lockups on real hardware.
I guess you are using my q800-dev branch?

In this branch, there is an attempt to manage unnormalized numbers that
seems to trigger this lock up.

You can either use master + this patch or update your q800-dev branch
from my repo.

Thanks,
Laurent

Re: [PATCH v3 10/11] tests/acceptance: refactor boot_linux to allow code reuse

[Philippe Mathieu-Daudé]

On 5/29/20 9:05 AM, Pavel Dovgalyuk wrote:
> This patch moves image downloading functions to the separate class to allow
> reusing them from record/replay tests.
> 
> Signed-off-by: Pavel Dovgalyuk 
> ---
>  0 files changed
> 
> diff --git a/tests/acceptance/boot_linux.py b/tests/acceptance/boot_linux.py
> index 075a386300..3aa57e88b0 100644
> --- a/tests/acceptance/boot_linux.py
> +++ b/tests/acceptance/boot_linux.py
> @@ -26,22 +26,8 @@ KVM_NOT_AVAILABLE = ACCEL_NOT_AVAILABLE_FMT % "KVM"
>  TCG_NOT_AVAILABLE = ACCEL_NOT_AVAILABLE_FMT % "TCG"
>  
>  
> -class BootLinux(Test):
> -"""
> -Boots a Linux system, checking for a successful initialization
> -"""
> -
> -timeout = 900
> -chksum = None
> -
> -def setUp(self):
> -super(BootLinux, self).setUp()
> -self.vm.add_args('-smp', '2')
> -self.vm.add_args('-m', '1024')
> -self.prepare_boot()
> -self.prepare_cloudinit()
> -
> -def prepare_boot(self):
> +class BootLinuxBase(Test):
> +def download_boot(self):
>  self.log.debug('Looking for and selecting a qemu-img binary to be '
> 'used to create the bootable snapshot image')
>  # If qemu-img has been built, use it, otherwise the system wide one
> @@ -60,17 +46,17 @@ class BootLinux(Test):
>  if image_arch == 'ppc64':
>  image_arch = 'ppc64le'
>  try:
> -self.boot = vmimage.get(
> +boot = vmimage.get(
>  'fedora', arch=image_arch, version='31',
>  checksum=self.chksum,
>  algorithm='sha256',
>  cache_dir=self.cache_dirs[0],
>  snapshot_dir=self.workdir)
> -self.vm.add_args('-drive', 'file=%s' % self.boot.path)
>  except:
>  self.cancel('Failed to download/prepare boot image')
> +return boot.path
>  
> -def prepare_cloudinit(self):
> +def download_cloudinit(self):
>  self.log.info('Preparing cloudinit image')
>  try:
>  cloudinit_iso = os.path.join(self.workdir, 'cloudinit.iso')
> @@ -81,9 +67,32 @@ class BootLinux(Test):
># QEMU's hard coded usermode router address
>phone_home_host='10.0.2.2',
>phone_home_port=self.phone_home_port)
> -self.vm.add_args('-drive', 'file=%s,format=raw' % cloudinit_iso)
>  except Exception:
>  self.cancel('Failed to prepared cloudinit image')
> +return cloudinit_iso
> +
> +class BootLinux(BootLinuxBase):
> +"""
> +Boots a Linux system, checking for a successful initialization
> +"""
> +
> +timeout = 900
> +chksum = None
> +
> +def setUp(self):
> +super(BootLinux, self).setUp()
> +self.vm.add_args('-smp', '2')
> +self.vm.add_args('-m', '1024')
> +self.prepare_boot()
> +self.prepare_cloudinit()
> +
> +def prepare_boot(self):
> +path = self.download_boot()
> +self.vm.add_args('-drive', 'file=%s' % path)
> +
> +def prepare_cloudinit(self):
> +cloudinit_iso = self.download_cloudinit()
> +self.vm.add_args('-drive', 'file=%s,format=raw' % cloudinit_iso)
>  
>  def launch_and_wait(self):
>  self.vm.set_console()
> 

Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 

Re: [PATCH] tests/acceptance/migration.py: Wait for both sides

[Philippe Mathieu-Daudé]

On 5/28/20 1:24 PM, Dr. David Alan Gilbert (git) wrote:
> From: "Dr. David Alan Gilbert" 
> 
> When the source finishes migration the destination will still be
> receiving the data sent by the source, so it might not have quite
> finished yet, so won't quite have reached 'completed'.
> This lead to occasional asserts in the next few checks.
> 
> After the source has finished, check the destination as well.
> (We can't just switch to checking the destination, because it doesn't
> give a status until it has started receiving the migration).
> 
> Reported-by: Alex Bennée 
> Signed-off-by: Dr. David Alan Gilbert 
> Tested-by: Alex Bennée 
> ---
>  tests/acceptance/migration.py | 4 
>  1 file changed, 4 insertions(+)
> 
> diff --git a/tests/acceptance/migration.py b/tests/acceptance/migration.py
> index 0365289cda..792639cb69 100644
> --- a/tests/acceptance/migration.py
> +++ b/tests/acceptance/migration.py
> @@ -35,6 +35,10 @@ class Migration(Test):
>timeout=self.timeout,
>step=0.1,
>args=(src_vm,))
> +wait.wait_for(self.migration_finished,
> +  timeout=self.timeout,
> +  step=0.1,
> +  args=(dst_vm,))
>  self.assertEqual(src_vm.command('query-migrate')['status'], 
> 'completed')
>  self.assertEqual(dst_vm.command('query-migrate')['status'], 
> 'completed')
>  self.assertEqual(dst_vm.command('query-status')['status'], 'running')
> 

Thanks, applied to my python-next tree:
https://gitlab.com/philmd/qemu/commits/python-next

Re: [PATCH v8 00/12] tests/vm: Add support for aarch64 VMs

[Philippe Mathieu-Daudé]

On 5/29/20 10:34 PM, Robert Foley wrote:
> This is version 8 of the patch series to
> add support for aarch64 VMs in the vm-build infrastructure.
>  - Ubuntu 18.04 aarch64 VM
>  - CentOS 8 aarch64 VM
> 
> v7: https://lists.gnu.org/archive/html/qemu-devel/2020-05/msg05286.html
> 
> Changes in v8:
> - Added Ubuntu common module in tests/vm.
> - Changed ubuntu.i386 and ubuntu.aarch64 to use new common module.
> - Split out ConsoleSocket addition (python/qemu) to separate patch
>   from changes to use it in tests/vm.
> - Adjustments in configure when checking for aarch64 efi images.
> - Remove use of QEMU_LOCAL in basevm.py.  We will use the
>   presence of the --build-path argument instead.
> 
> Robert Foley (12):
>   tests/vm: pass args through to BaseVM's __init__
>   tests/vm: Add configuration to basevm.py
>   tests/vm: Added configuration file support
>   tests/vm: Pass --debug through for vm-boot-ssh.
>   tests/vm: Add ability to select QEMU from current build.
>   tests/vm: allow wait_ssh() to specify command
>   tests/vm: Add common Ubuntu python module
>   tests/vm: Added a new script for ubuntu.aarch64.
>   tests/vm: Added a new script for centos.aarch64.
>   tests/vm: change scripts to use self._config
>   python/qemu: Add ConsoleSocket for optional use in QEMUMachine
>   tests/vm: Add workaround to consume console

Thanks, patches 4/5/6 applied to my python-next tree:
https://gitlab.com/philmd/qemu/commits/python-next

- tests/vm: Pass --debug through for vm-boot-ssh.
- tests/vm: Add ability to select QEMU from current build.
- tests/vm: allow wait_ssh() to specify command

Re: [PATCH v8 12/12] tests/vm: Add workaround to consume console

[Philippe Mathieu-Daudé]

On 5/29/20 10:34 PM, Robert Foley wrote:
> This adds support to basevm.py so that we always
> drain the console chars.  This makes use of
> support added in an earlier commit that allows
> QEMUMachine to use the ConsoleSocket.
> 
> This is a workaround we found was needed since
> there is a known issue where QEMU will hang waiting
> for console characters to be consumed.
> 
> We also added the option of logging the console to a file.
> LOG_CONSOLE=1 will now log the output to a file.
> 
> Signed-off-by: Robert Foley 
> Reviewed-by: Peter Puhov 
> Acked-by: Alex Bennée 
> ---
>  tests/vm/Makefile.include |  4 
>  tests/vm/basevm.py| 17 +++--
>  2 files changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
> index 8cccfaf95d..ad35c6e7a1 100644
> --- a/tests/vm/Makefile.include
> +++ b/tests/vm/Makefile.include
> @@ -49,6 +49,7 @@ endif
>   @echo 'EXTRA_CONFIGURE_OPTS="..."'
>   @echo "J=[0..9]* - Override the -jN parameter 
> for make commands"
>   @echo "DEBUG=1   - Enable verbose output on 
> host and interactive debugging"
> + @echo "LOG_CONSOLE=1 - Log console to file in: 
> ~/.cache/qemu-vm "
>   @echo "V=1   - Enable verbose ouput on host 
> and guest commands"
>   @echo "QEMU_LOCAL=1 - Use QEMU binary local to this 
> build."
>   @echo "QEMU=/path/to/qemu- Change path to QEMU binary"
> @@ -75,6 +76,7 @@ $(IMAGES_DIR)/%.img:$(SRC_PATH)/tests/vm/% \
>   $(if $(GENISOIMAGE),--genisoimage $(GENISOIMAGE)) \
>   $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \
>   $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \
> + $(if $(LOG_CONSOLE),--log-console) \
>   --image "$@" \
>   --force \
>   --build-image $@, \
> @@ -91,6 +93,7 @@ vm-build-%: $(IMAGES_DIR)/%.img
>   $(if $(V),--verbose) \
>   $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \
>   $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \
> + $(if $(LOG_CONSOLE),--log-console) \
>   --image "$<" \
>   $(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \
>   --snapshot \
> @@ -114,6 +117,7 @@ vm-boot-ssh-%: $(IMAGES_DIR)/%.img
>   $(if $(V)$(DEBUG), --debug) \
>   $(if $(QEMU_LOCAL),--build-path $(BUILD_DIR)) \
>   $(if $(EFI_AARCH64),--efi-aarch64 $(EFI_AARCH64)) \
> + $(if $(LOG_CONSOLE),--log-console) \
>   --image "$<" \
>   --interactive \
>   false, \
> diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
> index b9d828423b..64dbe64326 100644
> --- a/tests/vm/basevm.py
> +++ b/tests/vm/basevm.py
> @@ -117,6 +117,11 @@ class BaseVM(object):
>   "w").write(self._config['ssh_pub_key'])
>  
>  self.debug = args.debug
> +self._console_log_path = None
> +if args.log_console:
> +self._console_log_path = \
> + os.path.join(os.path.expanduser("~/.cache/qemu-vm"),
> +  "{}.install.log".format(self.name))
>  self._stderr = sys.stderr
>  self._devnull = open(os.devnull, "w")
>  if self.debug:
> @@ -271,7 +276,9 @@ class BaseVM(object):
>  args += self._data_args + extra_args + self._config['extra_args']
>  logging.debug("QEMU args: %s", " ".join(args))
>  qemu_path = get_qemu_path(self.arch, self._build_path)
> -guest = QEMUMachine(binary=qemu_path, args=args)
> +guest = QEMUMachine(binary=qemu_path, args=args,
> +console_log=self._console_log_path,
> +drain_console=True)

Are you sure you need to set drain_console here? Isn't it implied by
self._console_log_path?

>  guest.set_machine(self._config['machine'])
>  guest.set_console()
>  try:
> @@ -285,6 +292,8 @@ class BaseVM(object):
>  raise
>  atexit.register(self.shutdown)
>  self._guest = guest
> +# Init console so we can start consuming the chars.
> +self.console_init()
>  usernet_info = guest.qmp("human-monitor-command",
>   command_line="info usernet")
>  self.ssh_port = None
> @@ -296,7 +305,9 @@ class BaseVM(object):
>  raise Exception("Cannot find ssh port from 'info usernet':\n%s" 
> % \
>  usernet_info)
>  
> -def console_init(self, timeout = 120):
> +def console_init(self, timeout = None):
> +if timeout == None:
> +timeout = self.socket_timeout
>  vm = self._guest
>  vm.console_socket.settimeout(timeout)
>  self.console_raw_path = os.path.join(vm._temp_dir,
> @@ 

Re: [PATCH v8 06/12] tests/vm: allow wait_ssh() to specify command

[Philippe Mathieu-Daudé]

On 5/29/20 10:34 PM, Robert Foley wrote:
> This allows for waiting for completion of arbitrary commands.
> 
> Signed-off-by: Robert Foley 
> Reviewed-by: Peter Puhov 
> Reviewed-by: Alex Bennée 
> ---
>  tests/vm/basevm.py | 14 +++---
>  1 file changed, 7 insertions(+), 7 deletions(-)
> 
> diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
> index 75a7ac2bd3..1aab9e3a24 100644
> --- a/tests/vm/basevm.py
> +++ b/tests/vm/basevm.py
> @@ -411,24 +411,24 @@ class BaseVM(object):
>  def print_step(self, text):
>  sys.stderr.write("### %s ...\n" % text)
>  
> -def wait_ssh(self, wait_root=False, seconds=300):
> +def wait_ssh(self, wait_root=False, seconds=300, cmd="exit 0"):
>  # Allow more time for VM to boot under TCG.
>  if not kvm_available(self.arch):
>  seconds *= self.tcg_ssh_timeout_multiplier
>  starttime = datetime.datetime.now()
>  endtime = starttime + datetime.timedelta(seconds=seconds)
> -guest_up = False
> +cmd_success = False
>  while datetime.datetime.now() < endtime:
> -if wait_root and self.ssh_root("exit 0") == 0:
> -guest_up = True
> +if wait_root and self.ssh_root(cmd) == 0:
> +cmd_success = True
>  break
> -elif self.ssh("exit 0") == 0:
> -guest_up = True
> +elif self.ssh(cmd) == 0:
> +cmd_success = True
>  break
>  seconds = (endtime - datetime.datetime.now()).total_seconds()
>  logging.debug("%ds before timeout", seconds)
>  time.sleep(1)
> -if not guest_up:
> +if not cmd_success:
>  raise Exception("Timeout while waiting for guest ssh")
>  
>  def shutdown(self):
> 

Reviewed-by: Philippe Mathieu-Daudé 
Tested-by: Philippe Mathieu-Daudé 

Re: [PATCH] target/m68k: implement fmove.l #,FPCR

[John Paul Adrian Glaubitz]

Hi Laurent!

On 5/31/20 1:02 PM, Laurent Vivier wrote:
> The immediate value mode was ignored and instruction execution
> ends to an invalid access mode.
> 
> This was found running 'R' that set FPSR to 0 at startup with
> a 'fmove.l #0,FPSR' in qemu-system-m68k emulation and triggers a
> kernel crash:
> (...)
> Reported-by: John Paul Adrian Glaubitz 
> Signed-off-by: Laurent Vivier 
> ---
>  target/m68k/translate.c | 14 ++
>  1 file changed, 14 insertions(+)
Thanks for the fix.

I applied the patch, but I'm getting a lock-up now as you previously reported
in the other discussion on the Linux/m68k mailing list:

root@pacman:~# R
[   68.42] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [R:650]
[   68.42] Modules linked in: sg evdev mac_hid ip_tables x_tables 
sha1_generic hmac ipv6 nf_defrag_ipv6 autofs4 ext4 crc16 mbcache jbd2 
crc32c_generic sd_mod t10_pi crc_t10dif sr_mod cdrom crct10dif_generic 
crct10dif_common mac_esp macsonic esp_scsi
[   68.42] Format 00  Vector: 0064  PC: 0002df9c  Status: 2008Not 
tainted
[   68.42] ORIG_D0:   D0:   A2: c02e239a  A1: ffa1
[   68.42] A0: 3c9adf29  D5: 000d  D4: 8002ce30
[   68.42] D3: 8002b418  D2: 8002b4b4  D1: 

Is this related or a different bug?

I have not seen these lockups on real hardware.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: [PATCH v8 11/12] python/qemu: Add ConsoleSocket for optional use in QEMUMachine

[Philippe Mathieu-Daudé]

On 5/29/20 10:34 PM, Robert Foley wrote:
> We add the ConsoleSocket object, which has a socket interface
> and which will consume all arriving characters on the
> socket, placing them into an in memory buffer.
> This will also provide those chars via recv() as
> would a regular socket.
> ConsoleSocket also has the option of dumping
> the console bytes to a log file.
> 
> We also give QEMUMachine the option of using ConsoleSocket
> to drain and to use for logging console to a file.
> By default QEMUMachine does not use ConsoleSocket.
> 
> This is added in preparation for use by basevm.py in a later commit.
> This is a workaround we found was needed for basevm.py since
> there is a known issue where QEMU will hang waiting
> for console characters to be consumed.
> 
> Cc: Eduardo Habkost 
> Cc: Cleber Rosa 
> Signed-off-by: Robert Foley 
> Reviewed-by: Peter Puhov 
> Acked-by: Alex Bennée 
> ---
>  python/qemu/console_socket.py | 118 ++
>  python/qemu/machine.py|  23 +--
>  2 files changed, 137 insertions(+), 4 deletions(-)
>  create mode 100644 python/qemu/console_socket.py
> 
> diff --git a/python/qemu/console_socket.py b/python/qemu/console_socket.py
> new file mode 100644
> index 00..2fe1c20d64
> --- /dev/null
> +++ b/python/qemu/console_socket.py
> @@ -0,0 +1,118 @@
> +#!/usr/bin/env python3
> +#
> +# This python module implements a ConsoleSocket object which is
> +# designed always drain the socket itself, and place
> +# the bytes into a in memory buffer for later processing.
> +#
> +# Optionally a file path can be passed in and we will also
> +# dump the characters to this file for debug.
> +#
> +# Copyright 2020 Linaro
> +#
> +# Authors:
> +#  Robert Foley 
> +#
> +# This code is licensed under the GPL version 2 or later.  See
> +# the COPYING file in the top-level directory.
> +#
> +import asyncore
> +import socket
> +import threading
> +import io
> +import os
> +import sys
> +from collections import deque
> +import time
> +import traceback
> +
> +class ConsoleSocket(asyncore.dispatcher):
> +
> +def __init__(self, address, file=None):
> +self._recv_timeout_sec = 300
> +self._buffer = deque()
> +self._asyncore_thread = None
> +self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
> +self._sock.connect(address)
> +self._logfile = None
> +if file:
> +self._logfile = open(file, "w")
> +asyncore.dispatcher.__init__(self, sock=self._sock)
> +self._thread_start()
> +self._open = True

Please set _open before starting the thread (invert last 2 lines).

> +
> +def _thread_start(self):
> +"""Kick off a thread to wait on the asyncore.loop"""
> +if self._asyncore_thread is not None:
> +return
> +self._asyncore_thread = threading.Thread(target=asyncore.loop,
> + kwargs={'timeout':1})
> +self._asyncore_thread.daemon = True
> +self._asyncore_thread.start()
> +
> +def handle_close(self):
> +"""redirect close to base class"""
> +# Call the base class close, but not self.close() since
> +# handle_close() occurs in the context of the thread which
> +# self.close() attempts to join.
> +asyncore.dispatcher.close(self)
> +
> +def close(self):
> +"""Close the base object and wait for the thread to terminate"""
> +if self._open:
> +self._open = False
> +asyncore.dispatcher.close(self)
> +if self._asyncore_thread is not None:
> +thread, self._asyncore_thread = self._asyncore_thread, None
> +thread.join()
> +if self._logfile:
> +self._logfile.close()
> +self._logfile = None
> +
> +def handle_read(self):
> +"""process arriving characters into in memory _buffer"""
> +try:
> +data = asyncore.dispatcher.recv(self, 1)
> +# latin1 is needed since there are some chars
> +# we are receiving that cannot be encoded to utf-8
> +# such as 0xe2, 0x80, 0xA6.
> +string = data.decode("latin1")
> +except:
> +print("Exception seen.")
> +traceback.print_exc()
> +return
> +if self._logfile:
> +self._logfile.write("{}".format(string))
> +self._logfile.flush()
> +for c in string:
> +self._buffer.append(c)

   self._buffer.extend(string) ?

> +
> +def recv(self, n=1):

Please make a sleep_delay_s argument:

   def recv(self, n=1, sleep_delay_s=0.1):

> +"""Return chars from in memory buffer"""
> +start_time = time.time()
> +while len(self._buffer) < n:
> +time.sleep(0.1)

   time.sleep(sleep_delay_s)

> +elapsed_sec = time.time() - start_time
> +if elapsed_sec 

[PATCH] target/m68k: implement fmove.l #,FPCR

[Laurent Vivier]

The immediate value mode was ignored and instruction execution
ends to an invalid access mode.

This was found running 'R' that set FPSR to 0 at startup with
a 'fmove.l #0,FPSR' in qemu-system-m68k emulation and triggers a
kernel crash:

[   56.64] *** ADDRESS ERROR ***   FORMAT=2
[   56.64] Current process id is 728
[   56.64] BAD KERNEL TRAP: 
[   56.64] Modules linked in: sg evdev mac_hid ip_tables x_tables 
sha1_generic hmac ipv6 nf_defrag_ipv6 autofs4 ext4 crc16 mbcache jbd2 
crc32c_generic sd_mod t10_pi crc_t10dif crct10dif_generic crct10dif_common 
sr_mod cdrom mac_esp macsonic esp_scsi
[   56.64] PC: [<00016a2c>] X_UNSUPP+0x2c/0x3c
[   56.64] SR: 2004  SP: 3eb5e68c  a2: c02e239a
[   56.64] d0: 0040d1: 0002d2: 8002adecd3: 8002ad50
[   56.64] d4: 8002c768d5: 000da0: ffc2a1: ffc1
[   56.64] Process R (pid: 728, task=a3dfda5d)
[   56.64] Frame format=2 instr addr=
[   56.65] Stack from 3a4d9f30:
[   56.65] 4100 0002 0002 ffc2 ffc1 1fff 
8000 
[   56.65] 3fbf 8000   2000  
7fff 
[   56.65]   00050008  867c c02c2000 
efffee20 02d8
[   56.65] 2a28 3a4d9f98 0002 0014 fffe 8002c768 
0002 0041
[   56.65] 0002 c041fc58 c0743758   0008c075 
2b24 0012
[   56.65] 07d0 0024 0002 c05bef04 c05bef04 005e 
0077 c28aca70
[   56.65] Call Trace: [<00050008>] copy_overflow+0x10/0x28
[   56.65]  [<2a28>] buserr+0x20/0x28
[   56.65]  [<0008c075>] bpf_check+0x57f/0x1cfa
[   56.65]  [<2b24>] syscall+0x8/0xc
[   56.65]  [] dn_sched_init+0x75/0x88
[   56.65] Code: 1017 0200 00f0 0c00 0040 66ff  05ac  8800  
 f23c 9000   222e ff84 082e 0005 ff1c 6600 000a 0281
[   56.65] Disabling lock debugging due to kernel taint
...

Reported-by: John Paul Adrian Glaubitz 
Signed-off-by: Laurent Vivier 
---
 target/m68k/translate.c | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/target/m68k/translate.c b/target/m68k/translate.c
index 0f80888203d3..c093f6c683e8 100644
--- a/target/m68k/translate.c
+++ b/target/m68k/translate.c
@@ -4936,6 +4936,20 @@ static void gen_op_fmove_fcr(CPUM68KState *env, 
DisasContext *s,
 gen_store_fcr(s, AREG(insn, 0), mask);
 }
 return;
+case 7: /* Immediate */
+if (REG(insn, 0) == 4) {
+if (is_write ||
+(mask != M68K_FPIAR && mask != M68K_FPSR &&
+ mask != M68K_FPCR)) {
+gen_exception(s, s->base.pc_next, EXCP_ILLEGAL);
+return;
+}
+tmp = tcg_const_i32(read_im32(env, s));
+gen_store_fcr(s, tmp, mask);
+tcg_temp_free(tmp);
+return;
+}
+break;
 default:
 break;
 }
-- 
2.26.2

Re: [PATCH v8 08/12] tests/vm: Added a new script for ubuntu.aarch64.

[Alex Bennée]

Philippe Mathieu-Daudé  writes:

> Hi Robert,
>
> On 5/29/20 10:34 PM, Robert Foley wrote:

>> diff --git a/configure b/configure
>> index d82de47fdd..8a286e75a5 100755
>> --- a/configure
>> +++ b/configure
>> @@ -411,6 +411,7 @@ prefix="/usr/local"
>>  mandir="\${prefix}/share/man"
>>  datadir="\${prefix}/share"
>>  firmwarepath="\${prefix}/share/qemu-firmware"
>> +efi_aarch64=""
>>  qemu_docdir="\${prefix}/share/doc/qemu"
>>  bindir="\${prefix}/bin"
>>  libdir="\${prefix}/lib"
>> @@ -1099,6 +1100,8 @@ for opt do
>>;;
>>--firmwarepath=*) firmwarepath="$optarg"
>>;;
>> +  --efi-aarch64=*) efi_aarch64="$optarg"
>> +  ;;
>>--host=*|--build=*|\
>>--disable-dependency-tracking|\
>>--sbindir=*|--sharedstatedir=*|\
>> @@ -1753,6 +1756,7 @@ Advanced options (experts only):
>>--sysconfdir=PATHinstall config in PATH$confsuffix
>>--localstatedir=PATH install local state in PATH (set at runtime on 
>> win32)
>>--firmwarepath=PATH  search PATH for firmware files
>> +  --efi-aarch64=PATH   PATH of efi file to use for aarch64 VMs.
>>--with-confsuffix=SUFFIX suffix for QEMU data inside 
>> datadir/libdir/sysconfdir [$confsuffix]
>>--with-pkgversion=VERS   use specified string as sub-version of the 
>> package
>>--enable-debug   enable common debug build options
>> @@ -3548,6 +3552,20 @@ EOF
>>fi
>>  fi
>>  
>> +
>> +# efi-aarch64 probe
>> +# Check for efi files needed by aarch64 VMs.
>> +# By default we will use the efi included with QEMU.
>> +# Allow user to override the path for efi also.
>> +if ! test -f "$efi_aarch64"; then
>> +if test -f $source_path/pc-bios/edk2-aaarch64-code.fd.bz2; then
>> +# valid after build
>> +efi_aarch64=$PWD/pc-bios/edk2-aarch64-code.fd
>> +else
>> +efi_aarch64=""
>> +fi
>> +fi
>> +

>>  
>>  IMAGES_DIR := $(HOME)/.cache/qemu-vm/images
>> @@ -23,6 +26,11 @@ vm-help vm-test:
>>  ifneq ($(GENISOIMAGE),)
>>  @echo "  vm-build-centos - Build QEMU in CentOS VM, 
>> with Docker"
>>  @echo "  vm-build-ubuntu.i386- Build QEMU in ubuntu i386 VM"
>> +ifneq ($(EFI_AARCH64),)
>> +@echo "  vm-build-ubuntu.aarch64 - Build QEMU in ubuntu aarch64 
>> VM"
>> +else
>> +@echo "  (install qemu-efi-aarch64 to build centos/ubuntu aarch64 
>> images.)"
>
> I'm not sure your test is working well, I have qemu-efi-aarch64
> installed and it not automatically discovered.
>
> # apt install qemu-efi-aarch64
> qemu-efi-aarch64 is already the newest version
> (0~20191122.bd85bf54-2ubuntu3).
>
> Hint: I'm using out-of-tree builds.

I think efi_aarch64="" needs to be the default path, which can then be
overridden by the command line or finally fall back to the built in
image.

-- 
Alex Bennée

Re: [PATCH v8 07/12] tests/vm: Add common Ubuntu python module

[Philippe Mathieu-Daudé]

On 5/29/20 10:34 PM, Robert Foley wrote:
> Add a common Ubuntu python module and make use of
> it with the ubuntu.i386 script.
> This is preparation for adding an Ubuntu script
> ubuntu.aarch64.  Splitting out the common
> logic such as build_image() will reduce duplication.
> 
> Signed-off-by: Robert Foley 
> ---
>  tests/vm/ubuntu.i386 | 46 +--
>  tests/vm/ubuntuvm.py | 58 
>  2 files changed, 70 insertions(+), 34 deletions(-)
>  create mode 100644 tests/vm/ubuntuvm.py
> 
> diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
> index 1570775335..c699eaf8d7 100755
> --- a/tests/vm/ubuntu.i386
> +++ b/tests/vm/ubuntu.i386
> @@ -11,15 +11,22 @@
>  # the COPYING file in the top-level directory.
>  #
>  
> -import os
>  import sys
> -import subprocess
>  import basevm
> -import time
> +import ubuntuvm
>  
> -class UbuntuX86VM(basevm.BaseVM):
> +DEFAULT_CONFIG = {
> +'install_cmds' : "apt-get update,"\
> + "apt-get build-dep -y qemu,"\
> + "apt-get install -y libfdt-dev flex bison 
> language-pack-en",
> +}
> +
> +class UbuntuX86VM(ubuntuvm.UbuntuVM):
>  name = "ubuntu.i386"
>  arch = "i386"
> +image_link="https://cloud-images.ubuntu.com/releases/bionic/"\
> +   "release-20191114/ubuntu-18.04-server-cloudimg-i386.img"
> +
> image_sha256="28969840626d1ea80bb249c08eef1a4533e8904aa51a327b40f37ac4b4ff04ef"
>  BUILD_SCRIPT = """
>  set -e;
>  cd $(mktemp -d);
> @@ -29,34 +36,5 @@ class UbuntuX86VM(basevm.BaseVM):
>  make --output-sync {target} -j{jobs} {verbose};
>  """
>  
> -def build_image(self, img):
> -cimg = self._download_with_cache(
> -
> "https://cloud-images.ubuntu.com/releases/bionic/release-20191114/ubuntu-18.04-server-cloudimg-i386.img;,
> -
> sha256sum="28969840626d1ea80bb249c08eef1a4533e8904aa51a327b40f37ac4b4ff04ef")
> -img_tmp = img + ".tmp"
> -subprocess.check_call(["cp", "-f", cimg, img_tmp])
> -self.exec_qemu_img("resize", img_tmp, "50G")
> -self.boot(img_tmp, extra_args = [
> -"-device", "VGA",
> -"-cdrom", self.gen_cloud_init_iso()
> -])
> -self.wait_ssh()
> -self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")
> -self.ssh_root_check("apt-get update")
> -self.ssh_root_check("apt-get install -y cloud-initramfs-growroot")
> -# Don't check the status in case the guest hang up too quickly
> -self.ssh_root("sync && reboot")
> -time.sleep(5)
> -self.wait_ssh()
> -# The previous update sometimes doesn't survive a reboot, so do it 
> again
> -self.ssh_root_check("sed -ie s/^#\ deb-src/deb-src/g 
> /etc/apt/sources.list")
> -self.ssh_root_check("apt-get update")
> -self.ssh_root_check("apt-get build-dep -y qemu")
> -self.ssh_root_check("apt-get install -y libfdt-dev flex bison 
> language-pack-en")
> -self.ssh_root("poweroff")
> -self.wait()
> -os.rename(img_tmp, img)
> -return 0
> -
>  if __name__ == "__main__":
> -sys.exit(basevm.main(UbuntuX86VM))
> +sys.exit(basevm.main(UbuntuX86VM, DEFAULT_CONFIG))
> diff --git a/tests/vm/ubuntuvm.py b/tests/vm/ubuntuvm.py
> new file mode 100644
> index 00..96f29dcc15
> --- /dev/null
> +++ b/tests/vm/ubuntuvm.py
> @@ -0,0 +1,58 @@
> +#!/usr/bin/env python3
> +#
> +# Ubuntu VM testing library
> +#

Maybe:

   # Copyright 2017 Red Hat Inc.

> +# Copyright 2020 Linaro
> +#
> +# Authors:

And:

   #  Fam Zheng 

or a line "copied from ..."?

> +#  Robert Foley 
> +#
> +# This code is licensed under the GPL version 2 or later.  See
> +# the COPYING file in the top-level directory.
> +
> +import os
> +import subprocess
> +import basevm
> +
> +class UbuntuVM(basevm.BaseVM):
> +
> +def __init__(self, args, config=None):
> +self.login_prompt = "ubuntu-{}-guest login:".format(self.arch)
> +basevm.BaseVM.__init__(self, args, config)
> +
> +def build_image(self, img):
> +"""Build an Ubuntu VM image.  The child class will
> +   define the install_cmds to init the VM."""
> +os_img = self._download_with_cache(self.image_link,
> +   sha256sum=self.image_sha256)
> +img_tmp = img + ".tmp"
> +subprocess.check_call(["cp", "-f", os_img, img_tmp])
> +self.exec_qemu_img("resize", img_tmp, "+50G")
> +ci_img = self.gen_cloud_init_iso()
> +
> +self.boot(img_tmp, extra_args = [ "-device", "VGA", "-cdrom", 
> ci_img, ])
> +
> +# First command we issue is fix for slow ssh login.
> +self.wait_ssh(wait_root=True,
> +  cmd="chmod -x /etc/update-motd.d/*")
> +# Wait for cloud init to finish
> +self.wait_ssh(wait_root=True,
> +  cmd="ls