[Qemu-devel] [PATCH v12 00/27] IOMMU: Enable interrupt remapping for Intel IOMMU

[Peter Xu]

This is v12 for Intel IR. Rebased to lastest master, with some tiny
tweaks from v11. Please check changelog below.

Michael, please choose either v11 or v12 if we are going to merge it.
v12 is preferred though. Thanks!

Online repo:

  https://github.com/xzpeter/qemu vtd-intr-v12

v12 changes (using v11 patch index):
- patch 4: dropping this one. Originally, this patch was going to
  remove direct call from Q35 codes to VT-d codes, then provide
  find_add_as() as generic interface for x86 iommus. However after
  Marcel's patches (also, AMD patches are using pci_setup_iommu()
  directly in realize), we should not need this any more. [Jan, David]
- patch 14: fix a bad pointer in vtd_mem_ir_read() by empty the
  function directly, since no one should be using it. [David]
- patch 28: we should allow ir=off and kernel-irqchip=on [David]

v11 changes (using v10 patch index):
- patch 2: splitted into two patches, one to rename VTD_* macros, the
  other to provide x86_iommu_get_default(). [mst]
- patch 5: removing DMAR_REPORT_F_INTR, with comments [mst]
- patch 8: use size_t for ioapic_scope_size, removing
  ACPI_DMAR_DEV_SCOPE_TYPE_IOAPIC [mst]
- patch 13: better handle "subhandle" field: this is a problem I found
  in v10 when testing with rhel guests.
- patch 18: tiny tweaks to let QEMU build with --disable-kvm ("#ifdef
  CONFIG_KVM")
- patch 24: still using v10 of this patch, and dropping v10.2 version
  (so this patch content is unmodified from v10)
- new patch added: throw error when kernel irqchip=on is specified.
- put all new trace-events into specific directories
- add acked-by for Paolo (patch 16, 21-24, 26, 27)

v10 changes:
- Fix issue when specify more than 1 vcpus.  This is introduced in v9
  after rebased to Marcel's patches.  The problem is that, before
  Marcel's patch, we will first create IOMMU then IOAPIC, while the
  order is switched after Marcel's changes.  This affects patch 18
  ("register IOMMU IEC notifier for ioapic") and I need to do the
  registration after IOAPIC realization.
- Display readable error message if user specify more than one x86
  vIOMMU, rather than an assertion fail. (patch 2)
- Correct vtd iec notifier "global" parameter: if granularity bit is
  clear (not set), then it's a global invalidation (patch 17,
  inverted meaning for granularity).
- added one more patch (patch 26) to add some trace events for irqchip
  msi routes operations.
- rebase to latest master

v9 changes:
- addressed several possible acpi issue with BE machines, and comment
  fix [Igor]
- removed patch 16 in v8 since it's useless after rebasing to Marcel's
  patches
- move vtd_svt_mask into vtd_irte_get() and declare it as constant.
- rebase to latest master, with Marcel's "-device intel-iommu" patch v2
  - re-arrange patch order, moving x86-iommu to the beginning (so that
I can add "intremap" property for it, which can be further shared
by future AMD IOMMUs)
  - add device property "intremap" for X86 IOMMU device (new patch 4
in v9)
  - replace all existing references of MachineState.iommu_intr to
device property X86IOMMUState.intr_supported, removing
MachineState.iommu_intr
  - some other minor changes due to the rebase

v8 changes:
- rebase to latest master
- patch 7
  - remove VTD_IR_IOAPICEntry, which is useless now
  - fix possible issue on big endian machines for VTD_IRTE,
VTD_IR_MSIAddress
- patch 12
  - fix endianess issue with bit-field defines: fix BE issue with
VTD_MSIMessage, do cpu_to_*() or reverse when necessary on
bit-field uses.
- patch 19
  - used le32_to_cpu() for dest_id, and added my s-o-b line beneath
Jan's.

v7 changes (using v6 patch index):
- patch 10: trivial change in debug string (remove one more "\n")
- patch 17-18: ioapic remote irr patches, sent seperately
  already. So removed from this series.
- patch 24: 
  - fix commit message: only irqfd msi routes are maintained, not
all msi routes.
  - skip all IOAPIC msi entries (dev == NULL). We only need to
housekeep irqfd users.
- added patches
  - pick up Radim's patch on adding MHMV ecap bits [Radim]
- remove all vtd_* patches, instead, use x86-iommu ones at the first
  place. This introduced lots of patch order changes and content
  changes, which affected from original patch 8 to the end. Sorry!
  [Jan]

v6 changes:
- patch 10: use write_with_attrs() rather than write(), preparing
  for SID verification [Jan]
- patch 17-18: add r-b line from Radim [Radim]
- new patch 19: put together Jan's EIM patch [Jan]
- new patch 20: add SID validation process
- new patch 21-22: introduce X86IOMMU class, which is the parent of
  IntelIOMMU class. Patch 21 only introduce the class and did
  nothing, patch 22 cleaned up all the vtd_*() hooks into x86
  ones. This is only a start. In the future, we can abstract more
  things into X86IOMMU class, like iotlb, address spaces mgmt,
  etc. [Jan]
- new patch 23-25: this is to do IEC notify to all irqfd consumers
  like vhost/vfio. patch 23 

Re: [Qemu-devel] [RFC 02/13] migration: Allow the migrate command to work on file: urls

[Hailiang Zhang]

On 2016/7/14 0:12, Dr. David Alan Gilbert wrote:

* zhanghailiang (zhang.zhanghaili...@huawei.com) wrote:

Usage:
(qemu) migrate file:/path/to/vm_statefile

Signed-off-by: zhanghailiang 
Signed-off-by: Benoit Canet 
---
- With this patch, we can easily test memory snapshot
- Rebase on qemu 2.5
---
  include/migration/migration.h |  6 +-
  migration/fd.c| 19 +--
  migration/migration.c |  4 +++-
  3 files changed, 25 insertions(+), 4 deletions(-)


Even if the rest of this series takes some time to finish; it might be
best to post this patch separately - it just a nice bit of simplification.
Of course now you have to rewrite using qio_channel_file_new_path
but I guess it's simple.

(I've tended to use migrate "exec:cat > file" but a file:  would be nice)



OK, if you need this, i will send it as a single patch.

Thanks.


Dave


diff --git a/include/migration/migration.h b/include/migration/migration.h
index 4c80939..bf4f8e9 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -193,7 +193,11 @@ void unix_start_outgoing_migration(MigrationState *s, 
const char *path, Error **

  void fd_start_incoming_migration(const char *path, Error **errp);

-void fd_start_outgoing_migration(MigrationState *s, const char *fdname, Error 
**errp);
+void fd_start_outgoing_migration(MigrationState *s, const char *fdname,
+ int outfd, Error **errp);
+
+void file_start_outgoing_migration(MigrationState *s, const char *filename,
+   Error **errp);

  void rdma_start_outgoing_migration(void *opaque, const char *host_port, Error 
**errp);

diff --git a/migration/fd.c b/migration/fd.c
index 3e4bed0..b62161f 100644
--- a/migration/fd.c
+++ b/migration/fd.c
@@ -42,9 +42,10 @@ static bool fd_is_socket(int fd)
  return S_ISSOCK(stat.st_mode);
  }

-void fd_start_outgoing_migration(MigrationState *s, const char *fdname, Error 
**errp)
+void fd_start_outgoing_migration(MigrationState *s, const char *fdname,
+ int outfd, Error **errp)
  {
-int fd = monitor_get_fd(cur_mon, fdname, errp);
+int fd = fdname ? monitor_get_fd(cur_mon, fdname, errp) : outfd;
  if (fd == -1) {
  return;
  }
@@ -58,6 +59,20 @@ void fd_start_outgoing_migration(MigrationState *s, const 
char *fdname, Error **
  migrate_fd_connect(s);
  }

+void file_start_outgoing_migration(MigrationState *s, const char *filename,
+   Error **errp)
+{
+int fd;
+
+fd = qemu_open(filename, O_CREAT | O_TRUNC | O_WRONLY, S_IRUSR | S_IWUSR);
+if (fd < 0) {
+error_setg_errno(errp, errno, "Failed to open file: %s", filename);
+return;
+}
+fd_start_outgoing_migration(s, NULL, fd, errp);
+}
+
+
  static void fd_accept_incoming_migration(void *opaque)
  {
  QEMUFile *f = opaque;
diff --git a/migration/migration.c b/migration/migration.c
index c842499..3ec3b85 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1021,7 +1021,9 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk,
  } else if (strstart(uri, "unix:", )) {
  unix_start_outgoing_migration(s, p, _err);
  } else if (strstart(uri, "fd:", )) {
-fd_start_outgoing_migration(s, p, _err);
+fd_start_outgoing_migration(s, p, -1, _err);
+} else if (strstart(uri, "file:", )) {
+file_start_outgoing_migration(s, p,  _err);
  #endif
  } else {
  error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "uri",
--
1.8.3.1



--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

.

Re: [Qemu-devel] [RFC 1/6] target-ppc: Introduce Power9 family

[Bharata B Rao]

On Tue, Jul 12, 2016 at 11:33 PM, Nikunj A Dadhania
 wrote:
> From: "Aneesh Kumar K.V" 
>
> Signed-off-by: Aneesh Kumar K.V 
> [ rebased and added POWER9 alias ]
> Signed-off-by: Nikunj A Dadhania 
> ---
>  target-ppc/cpu-models.c |  5 +++
>  target-ppc/cpu-models.h |  2 ++
>  target-ppc/cpu-qom.h|  7 
>  target-ppc/mmu_helper.c |  3 +-
>  target-ppc/translate_init.c | 85 
> -
>  5 files changed, 100 insertions(+), 2 deletions(-)
>
> diff --git a/target-ppc/cpu-models.c b/target-ppc/cpu-models.c
> index 5209e63..901cf40 100644
> --- a/target-ppc/cpu-models.c
> +++ b/target-ppc/cpu-models.c
> @@ -1147,6 +1147,10 @@
>  "POWER8NVL v1.0")
>  POWERPC_DEF("970_v2.2",  CPU_POWERPC_970_v22,970,
>  "PowerPC 970 v2.2")
> +
> +POWERPC_DEF("POWER9_v1.0",   CPU_POWERPC_POWER9_BASE,POWER9,
> +"POWER9 v1.0")
> +

I realize it is still early days, but eventually remember to add a
POWER9 core definition in spapr_cpu_core.c

Regards,
Bharata.

Re: [Qemu-devel] [PATCH v11 14/28] intel_iommu: Add support for PCI MSI remap

[Peter Xu]

On Wed, Jul 13, 2016 at 04:17:01PM +0300, David Kiarie wrote:

[...]

> > +static MemTxResult vtd_mem_ir_read(void *opaque, hwaddr addr,
> > +   uint64_t *data, unsigned size,
> > +   MemTxAttrs attrs)
> > +{
> > +addr += VTD_INTERRUPT_ADDR_FIRST;
> > +
> > +VTD_DPRINTF(IR, "read mem_ir addr 0x%"PRIx64 " size %u",
> > +addr, size);
> > +
> > +if (dma_memory_read(_space_memory, addr, , size)) {
> > +VTD_DPRINTF(GENERAL, "error: fail to access 0x%"PRIx64, addr);
> > +return MEMTX_ERROR;
> > +}
> > +
> > +return MEMTX_OK;
> > +}
> 
> I'm looking at this and wondering whether dma_memory_read expected a
> double pointer as the third argument. (??)

Right. It's dangerous and should never be touched. Maybe I should keep
this an empty function, just like what APIC is doing. Read at
address_space_memory is odd here in all cases...

Thanks, David.

-- peterx

Re: [Qemu-devel] [Qemu-ppc] [PATCH] ppc: exit if compat mode is unknown

[Greg Kurz]

On Thu, 14 Jul 2016 12:05:42 +1000
David Gibson  wrote:

> On Wed, Jul 13, 2016 at 02:34:32PM +0200, Greg Kurz wrote:
> > On Wed, 13 Jul 2016 11:10:21 +0200
> > Greg Kurz  wrote:
> >   
> > > Now that properties are set with -global, if we pass a bogus value for
> > > compat mode, a warning is printed and the property is simply ignored.
> > > For example, if the host is POWER8 and we pass compat=POWER7 instead of
> > > compat=power7 the guest will see a POWER8 cpu.
> > > 
> > > While here, this patch also changes the error message to be more explicit.
> > > 
> > > Signed-off-by: Greg Kurz 
> > > ---  
> > 
> > Of course this causes qom-set to exit as well... not sure this is
> > what we want.  
> 
> Yeah, that seems wrong.  If I'm reading this right this is obsoleted
> by your later generic patch adding an _fatal to the generic
> code, yes?
> 

Yes it is indeed.

> >   
> > >  target-ppc/translate_init.c |6 +-
> > >  1 file changed, 5 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/target-ppc/translate_init.c b/target-ppc/translate_init.c
> > > index 8f257fb74aa7..55688db9c9db 100644
> > > --- a/target-ppc/translate_init.c
> > > +++ b/target-ppc/translate_init.c
> > > @@ -8475,7 +8475,11 @@ static void powerpc_set_compat(Object *obj, 
> > > Visitor *v, const char *name,
> > >  } else if (strcmp(value, "power8") == 0) {
> > >  *max_compat = CPU_POWERPC_LOGICAL_2_07;
> > >  } else {
> > > -error_setg(errp, "Invalid compatibility mode \"%s\"", value);
> > > +/* This is called from -global, which does not exit on error. 
> > > Since we
> > > + * don't want to start with a wrong compat mode, we exit now.
> > > + */
> > > +error_report("PPC does not support compatibility mode \"%s\"", 
> > > value);
> > > +exit(1);
> > >  }
> > >  
> > >  g_free(value);
> > > 
> > >   
> >   
> 



pgp1MdD_g4ETo.pgp
Description: OpenPGP digital signature

[Qemu-devel] [PATCH v11 8/9] target-avr: instruction decoder generator

[Michael Rolnik]

Signed-off-by: Michael Rolnik 
---
 target-avr/cpugen/CMakeLists.txt   |  38 +++
 target-avr/cpugen/README.md|  17 ++
 target-avr/cpugen/cpu/avr.yaml | 214 ++
 target-avr/cpugen/src/CMakeLists.txt   |  63 
 target-avr/cpugen/src/cpugen.cpp   | 458 +
 target-avr/cpugen/src/utils.cpp|  27 ++
 target-avr/cpugen/src/utils.h  |  79 +
 target-avr/cpugen/xsl/decode.c.xsl | 103 +++
 target-avr/cpugen/xsl/translate-inst.h.xsl | 118 
 target-avr/cpugen/xsl/utils.xsl| 108 +++
 10 files changed, 1225 insertions(+)
 create mode 100644 target-avr/cpugen/CMakeLists.txt
 create mode 100644 target-avr/cpugen/README.md
 create mode 100644 target-avr/cpugen/cpu/avr.yaml
 create mode 100644 target-avr/cpugen/src/CMakeLists.txt
 create mode 100644 target-avr/cpugen/src/cpugen.cpp
 create mode 100644 target-avr/cpugen/src/utils.cpp
 create mode 100644 target-avr/cpugen/src/utils.h
 create mode 100644 target-avr/cpugen/xsl/decode.c.xsl
 create mode 100644 target-avr/cpugen/xsl/translate-inst.h.xsl
 create mode 100644 target-avr/cpugen/xsl/utils.xsl

diff --git a/target-avr/cpugen/CMakeLists.txt b/target-avr/cpugen/CMakeLists.txt
new file mode 100644
index 000..ded391c
--- /dev/null
+++ b/target-avr/cpugen/CMakeLists.txt
@@ -0,0 +1,38 @@
+cmake_minimum_required(VERSION 2.8)
+
+project(cpugen)
+
+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g -ggdb -g3")
+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++0x")
+
+set(Boost_USE_STATIC_LIBS   ON)
+find_package(
+Boost 1.60.0
+REQUIRED
+COMPONENTS
+system
+regex)
+#set(BUILD_SHARED_LIBS   OFF)
+#set(BUILD_STATIC_LIBS   ON)
+add_subdirectory(tinyxml2)
+add_subdirectory(yaml-cpp)
+
+include_directories(
+${CMAKE_CURRENT_SOURCE_DIR}
+${CMAKE_CURRENT_SOURCE_DIR}/..
+${CMAKE_CURRENT_SOURCE_DIR}/../yaml-cpp/include
+${Boost_INCLUDE_DIRS}
+)
+
+add_executable(
+cpugen
+src/cpugen.cpp
+src/utils.cpp
+)
+
+target_link_libraries(
+cpugen
+yaml-cpp
+tinyxml2
+${Boost_LIBRARIES}
+)
diff --git a/target-avr/cpugen/README.md b/target-avr/cpugen/README.md
new file mode 100644
index 000..f0caa8b
--- /dev/null
+++ b/target-avr/cpugen/README.md
@@ -0,0 +1,17 @@
+# CPUGEN
+## How to build
+within ```cpugen``` directory do
+```
+git clone https://github.com/leethomason/tinyxml2
+git clone https://github.com/jbeder/yaml-cpp
+mkdir build
+cd build
+cmake ..
+make
+```
+## How to use
+```
+cpugen ../cpu/avr.yaml
+xsltproc ../xsl/decode.c.xsl output.xml > ../../decode.c
+xsltproc ../xsl/translate-inst.h.xsl output.xml > ../../translate-inst.h
+```
diff --git a/target-avr/cpugen/cpu/avr.yaml b/target-avr/cpugen/cpu/avr.yaml
new file mode 100644
index 000..a626859
--- /dev/null
+++ b/target-avr/cpugen/cpu/avr.yaml
@@ -0,0 +1,214 @@
+cpu:
+name: avr
+instructions:
+- ADC:
+opcode: 0001 11 hRr[1] Rd[5] lRr[4]
+- ADD:
+opcode:  11 hRr[1] Rd[5] lRr[4]
+- ADIW:
+opcode: 1001 0110 hImm[2] Rd[2] lImm[4]
+- AND:
+opcode: 0010 00 hRr[1] Rd[5] lRr[4]
+- ANDI:
+opcode: 0111 hImm[4] Rd[4] lImm[4]
+- ASR:
+opcode: 1001 010 Rd[5] 0101
+- BCLR:
+opcode: 1001 0100 1 Bit[3] 1000
+- BLD:
+opcode:  100 Rd[5] 0 Bit[3]
+- BRBC:
+opcode:  01 Imm[7] Bit[3]
+- BRBS:
+opcode:  00 Imm[7] Bit[3]
+- BREAK:
+opcode: 1001 0101 1001 1000
+- BSET:
+opcode: 1001 0100 0 Bit[3] 1000
+- BST:
+opcode:  101 Rd[5] 0 Bit[3]
+- CALL:
+opcode: 1001 010 hImm[5] 111 lImm[17]
+- CBI:
+opcode: 1001 1000 Imm[5] Bit[3]
+- COM:
+opcode: 1001 010 Rd[5] 
+- CP:
+opcode: 0001 01 hRr[1] Rd[5] lRr[4]
+- CPC:
+opcode:  01 hRr[1] Rd[5] lRr[4]
+- CPI:
+opcode: 0011 hImm[4] Rd[4] lImm[4]
+- CPSE:
+opcode: 0001 00 hRr[1] Rd[5] lRr[4]
+- DEC:
+opcode: 1001 010 Rd[5] 1010
+- DES:
+opcode: 1001 0100 Imm[4] 1011
+- EICALL:
+opcode: 1001 0101 0001 1001
+- EIJMP:
+opcode: 1001 0100 0001 1001
+- ELPM1:
+opcode: 1001 0101 1101 1000
+- ELPM2:
+opcode: 1001 000 Rd[5] 0110
+- ELPMX:
+opcode: 1001 000 Rd[5] 0111
+- EOR:
+opcode: 0010 01 hRr[1] Rd[5] lRr[4]
+- FMUL:
+opcode:  0011 0 Rd[3] 1 Rr[3]
+- FMULS:
+opcode:  0011 1 Rd[3] 0 Rr[3]
+- FMULSU:
+opcode:  0011 1 Rd[3] 1 Rr[3]
+- ICALL:
+opcode: 1001 0101  1001
+- IJMP:
+

[Qemu-devel] [PATCH v11 7/9] target-avr: adding instruction translation

[Michael Rolnik]

Signed-off-by: Michael Rolnik 
---
 target-avr/Makefile.objs|2 +-
 target-avr/translate-inst.c | 2622 +++
 target-avr/translate.h  |1 +
 3 files changed, 2624 insertions(+), 1 deletion(-)
 create mode 100644 target-avr/translate-inst.c

diff --git a/target-avr/Makefile.objs b/target-avr/Makefile.objs
index 2a10104..fdafcd5 100644
--- a/target-avr/Makefile.objs
+++ b/target-avr/Makefile.objs
@@ -18,6 +18,6 @@
 #  
 #
 
-obj-y   += translate.o cpu.o helper.o
+obj-y   += translate.o cpu.o helper.o translate-inst.o
 obj-y   += gdbstub.o
 obj-$(CONFIG_SOFTMMU) += machine.o
diff --git a/target-avr/translate-inst.c b/target-avr/translate-inst.c
new file mode 100644
index 000..d1dce42
--- /dev/null
+++ b/target-avr/translate-inst.c
@@ -0,0 +1,2622 @@
+/*
+ *  QEMU AVR CPU
+ *
+ *  Copyright (c) 2016 Michael Rolnik
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Lesser General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2.1 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Lesser General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Lesser General Public
+ *  License along with this library; if not, see
+ *  
+ */
+
+#include "translate.h"
+#include "translate-inst.h"
+#include "qemu/bitops.h"
+
+static void gen_add_CHf(TCGv R, TCGv Rd, TCGv Rr)
+{
+TCGv t1 = tcg_temp_new_i32();
+TCGv t2 = tcg_temp_new_i32();
+TCGv t3 = tcg_temp_new_i32();
+
+tcg_gen_and_tl(t1, Rd, Rr); /*  t1 = Rd & Rr  */
+tcg_gen_andc_tl(t2, Rd, R); /*  t2 = Rd & ~R  */
+tcg_gen_andc_tl(t3, Rr, R); /*  t3 = Rr & ~R  */
+tcg_gen_or_tl(t1, t1, t2);  /*  t1 = t1 | t2 | t3  */
+tcg_gen_or_tl(t1, t1, t3);
+
+tcg_gen_shri_tl(cpu_Cf, t1, 7); /*  Cf = t1(7)  */
+tcg_gen_shri_tl(cpu_Hf, t1, 3); /*  Hf = t1(3)  */
+tcg_gen_andi_tl(cpu_Hf, cpu_Hf, 1);
+
+tcg_temp_free_i32(t3);
+tcg_temp_free_i32(t2);
+tcg_temp_free_i32(t1);
+}
+
+static void gen_add_Vf(TCGv R, TCGv Rd, TCGv Rr)
+{
+TCGv t1 = tcg_temp_new_i32();
+TCGv t2 = tcg_temp_new_i32();
+
+/*  t1 = Rd & Rr & ~R | ~Rd & ~Rr & R = (Rd ^ R) & ~(Rd ^ Rr)   */
+tcg_gen_xor_tl(t1, Rd, R);
+tcg_gen_xor_tl(t2, Rd, Rr);
+tcg_gen_andc_tl(t1, t1, t2);
+
+tcg_gen_shri_tl(cpu_Vf, t1, 7); /*  Vf = t1(7)  */
+
+tcg_temp_free_i32(t2);
+tcg_temp_free_i32(t1);
+}
+
+static void gen_sub_CHf(TCGv R, TCGv Rd, TCGv Rr)
+{
+TCGv t1 = tcg_temp_new_i32();
+TCGv t2 = tcg_temp_new_i32();
+TCGv t3 = tcg_temp_new_i32();
+
+/*  Cf & Hf  */
+tcg_gen_not_tl(t1, Rd); /*  t1 = ~Rd  */
+tcg_gen_and_tl(t2, t1, Rr); /*  t2 = ~Rd & Rr  */
+tcg_gen_or_tl(t3, t1, Rr);  /*  t3 = (~Rd | Rr) & R  */
+tcg_gen_and_tl(t3, t3, R);
+tcg_gen_or_tl(t2, t2, t3);  /*  t2 = ~Rd & Rr | ~Rd & R | R & Rr  
*/
+tcg_gen_shri_tl(cpu_Cf, t2, 7); /*  Cf = t2(7)  */
+tcg_gen_shri_tl(cpu_Hf, t2, 3); /*  Hf = t2(3)  */
+tcg_gen_andi_tl(cpu_Hf, cpu_Hf, 1);
+
+tcg_temp_free_i32(t3);
+tcg_temp_free_i32(t2);
+tcg_temp_free_i32(t1);
+}
+
+static void gen_sub_Vf(TCGv R, TCGv Rd, TCGv Rr)
+{
+TCGv t1 = tcg_temp_new_i32();
+TCGv t2 = tcg_temp_new_i32();
+
+/*  Vf  */
+/*  t1 = Rd & ~Rr & ~R | ~Rd & Rr & R  = (Rd ^ R) & (Rd ^ R)*/
+tcg_gen_xor_tl(t1, Rd, R);
+tcg_gen_xor_tl(t2, Rd, Rr);
+tcg_gen_and_tl(t1, t1, t2);
+tcg_gen_shri_tl(cpu_Vf, t1, 7); /*  Vf = t1(7)  */
+
+tcg_temp_free_i32(t2);
+tcg_temp_free_i32(t1);
+}
+
+static void gen_NSf(TCGv R)
+{
+tcg_gen_shri_tl(cpu_Nf, R, 7);  /*  Nf = R(7)  */
+tcg_gen_xor_tl(cpu_Sf, cpu_Nf, cpu_Vf); /*  Sf = Nf ^ Vf  */
+}
+
+static void gen_ZNSf(TCGv R)
+{
+tcg_gen_mov_tl(cpu_Zf, R);  /*  Zf = R  */
+tcg_gen_shri_tl(cpu_Nf, R, 7);  /*  Nf = R(7)  */
+tcg_gen_xor_tl(cpu_Sf, cpu_Nf, cpu_Vf); /*  Sf = Nf ^ Vf  */
+}
+
+static void gen_push_ret(CPUAVRState *env, int ret)
+{
+if (avr_feature(env, AVR_FEATURE_1_BYTE_PC)) {
+
+TCGv t0 = tcg_const_i32((ret & 0xff));
+
+tcg_gen_qemu_st_tl(t0, cpu_sp, MMU_DATA_IDX, MO_UB);
+tcg_gen_subi_tl(cpu_sp, cpu_sp, 1);
+
+tcg_temp_free_i32(t0);
+} else if (avr_feature(env, AVR_FEATURE_2_BYTE_PC)) {
+
+TCGv t0 = tcg_const_i32((ret & 0x00));
+
+tcg_gen_subi_tl(cpu_sp, cpu_sp, 1);
+tcg_gen_qemu_st_tl(t0, cpu_sp, MMU_DATA_IDX, MO_BEUW);
+tcg_gen_subi_tl(cpu_sp, 

[Qemu-devel] [PATCH v11 6/9] target-avr: adding helpers for IN, OUT, SLEEP, WBR & unsupported instructions

[Michael Rolnik]

Signed-off-by: Michael Rolnik 
---
 target-avr/cpu.h   |  13 ++-
 target-avr/helper.c| 284 ++---
 target-avr/helper.h|   8 +-
 target-avr/translate.c |   9 ++
 4 files changed, 299 insertions(+), 15 deletions(-)

diff --git a/target-avr/cpu.h b/target-avr/cpu.h
index 4bb3d6d..f362c26 100644
--- a/target-avr/cpu.h
+++ b/target-avr/cpu.h
@@ -140,6 +140,7 @@ struct CPUAVRState {
 uint32_tsp; /*   16 bits*/
 
 uint64_tintsrc; /*  interrupt sources   */
+boolfullacc;/*  CPU/MEM if true MEM only otherwise  */
 
 uint32_tfeatures;
 
@@ -182,12 +183,22 @@ int avr_cpu_handle_mmu_fault(CPUState *cpu, vaddr 
address, int rw,
 int avr_cpu_memory_rw_debug(CPUState *cs, vaddr address, uint8_t *buf,
 int len, bool is_write);
 
+enum {
+TB_FLAGS_FULL_ACCESS = 1,
+};
 static inline void cpu_get_tb_cpu_state(CPUAVRState *env, target_ulong *pc,
 target_ulong *cs_base, uint32_t *pflags)
 {
+uint32_t flags = 0;
+
 *pc = env->pc_w * 2;
 *cs_base = 0;
-*pflags = 0;
+
+if (env->fullacc) {
+flags |= TB_FLAGS_FULL_ACCESS;
+}
+
+*pflags = flags;
 }
 
 static inline int cpu_interrupts_enabled(CPUAVRState *env1)
diff --git a/target-avr/helper.c b/target-avr/helper.c
index 3e61193..d1f254d 100644
--- a/target-avr/helper.c
+++ b/target-avr/helper.c
@@ -28,6 +28,7 @@
 #include "exec/cpu_ldst.h"
 #include "qemu/host-utils.h"
 #include "exec/helper-proto.h"
+#include "exec/ioport.h"
 
 bool avr_cpu_exec_interrupt(CPUState *cs, int interrupt_request)
 {
@@ -42,14 +43,14 @@ bool avr_cpu_exec_interrupt(CPUState *cs, int 
interrupt_request)
 cs->exception_index = EXCP_RESET;
 cc->do_interrupt(cs);
 
-cs->interrupt_request   &= ~CPU_INTERRUPT_RESET;
+cs->interrupt_request &= ~CPU_INTERRUPT_RESET;
 
 ret = true;
 }
 }
 if (interrupt_request & CPU_INTERRUPT_HARD) {
 if (cpu_interrupts_enabled(env) && env->intsrc != 0) {
-int index = ctz32(env->intsrc);
+int index = ctz32(env->intsrc);
 cs->exception_index = EXCP_INT(index);
 cc->do_interrupt(cs);
 
@@ -64,8 +65,8 @@ bool avr_cpu_exec_interrupt(CPUState *cs, int 
interrupt_request)
 
 void avr_cpu_do_interrupt(CPUState *cs)
 {
-AVRCPU *cpu = AVR_CPU(cs);
-CPUAVRState*env = >env;
+AVRCPU *cpu = AVR_CPU(cs);
+CPUAVRState *env = >env;
 
 uint32_t ret = env->pc_w;
 int vector = 0;
@@ -79,14 +80,14 @@ void avr_cpu_do_interrupt(CPUState *cs)
 }
 
 if (avr_feature(env, AVR_FEATURE_3_BYTE_PC)) {
-stb_phys(cs->as, env->sp--, (ret & 0xff));
-stb_phys(cs->as, env->sp--, (ret & 0x00ff00) >>  8);
-stb_phys(cs->as, env->sp--, (ret & 0xff) >> 16);
+cpu_stb_data(env, env->sp--, (ret & 0xff));
+cpu_stb_data(env, env->sp--, (ret & 0x00ff00) >>  8);
+cpu_stb_data(env, env->sp--, (ret & 0xff) >> 16);
 } else if (avr_feature(env, AVR_FEATURE_2_BYTE_PC)) {
-stb_phys(cs->as, env->sp--, (ret & 0xff));
-stb_phys(cs->as, env->sp--, (ret & 0x00ff00) >>  8);
+cpu_stb_data(env, env->sp--, (ret & 0xff));
+cpu_stb_data(env, env->sp--, (ret & 0x00ff00) >>  8);
 } else {
-stb_phys(cs->as, env->sp--, (ret & 0xff));
+cpu_stb_data(env, env->sp--, (ret & 0xff));
 }
 
 env->pc_w = base + vector * size;
@@ -124,15 +125,55 @@ void tlb_fill(CPUState *cs, target_ulong vaddr, 
MMUAccessType access_type,
 vaddr &= TARGET_PAGE_MASK;
 
 if (mmu_idx == MMU_CODE_IDX) {
-paddr = PHYS_BASE_CODE + vaddr;
+paddr = PHYS_BASE_CODE + vaddr - VIRT_BASE_CODE;
 prot = PAGE_READ | PAGE_EXEC;
 } else {
-paddr = PHYS_BASE_DATA + vaddr;
-prot = PAGE_READ | PAGE_WRITE;
+#if VIRT_BASE_REGS == 0
+if (vaddr < VIRT_BASE_REGS + AVR_REGS) {
+#else
+if (VIRT_BASE_REGS <= vaddr && vaddr < VIRT_BASE_REGS + SIZE_REGS) {
+#endif
+/*
+this is a write into CPU registers, exit and rebuilt this TB
+to use full write
+*/
+AVRCPU *cpu = AVR_CPU(cs);
+CPUAVRState *env = >env;
+env->fullacc = 1;
+cpu_loop_exit_restore(cs, retaddr);
+} else {
+/*
+this is a write into memory. nothing special
+*/
+paddr = PHYS_BASE_DATA + vaddr - VIRT_BASE_DATA;
+prot = PAGE_READ | PAGE_WRITE;
+}
 }
 
 tlb_set_page_with_attrs(cs, vaddr, paddr, attrs, prot, mmu_idx, page_size);
 }
+void helper_sleep(CPUAVRState *env)
+{
+CPUState *cs = CPU(avr_env_get_cpu(env));
+
+cs->exception_index = EXCP_HLT;
+cpu_loop_exit(cs);
+}
+void 

[Qemu-devel] [PATCH v11 3/9] target-avr: adding a sample AVR board

[Michael Rolnik]

Signed-off-by: Michael Rolnik 
---
 MAINTAINERS  |   6 ++
 hw/avr/Makefile.objs |  21 ++
 hw/avr/sample-io.c   | 176 +++
 hw/avr/sample.c  | 137 +++
 4 files changed, 340 insertions(+)
 create mode 100644 hw/avr/Makefile.objs
 create mode 100644 hw/avr/sample-io.c
 create mode 100644 hw/avr/sample.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 1d0e2c3..ecf10da 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -110,6 +110,12 @@ F: disas/arm.c
 F: disas/arm-a64.cc
 F: disas/libvixl/
 
+AVR
+M: Michael Rolnik 
+S: Maintained
+F: target-avr/
+F: hw/avr/
+
 CRIS
 M: Edgar E. Iglesias 
 S: Maintained
diff --git a/hw/avr/Makefile.objs b/hw/avr/Makefile.objs
new file mode 100644
index 000..c080e4e
--- /dev/null
+++ b/hw/avr/Makefile.objs
@@ -0,0 +1,21 @@
+#
+#  QEMU AVR CPU
+#
+#  Copyright (c) 2016 Michael Rolnik
+#
+#  This library is free software; you can redistribute it and/or
+#  modify it under the terms of the GNU Lesser General Public
+#  License as published by the Free Software Foundation; either
+#  version 2.1 of the License, or (at your option) any later version.
+#
+#  This library is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+#  Lesser General Public License for more details.
+#
+#  You should have received a copy of the GNU Lesser General Public
+#  License along with this library; if not, see
+#  
+#
+
+obj-y   += sample.o sample-io.o
diff --git a/hw/avr/sample-io.c b/hw/avr/sample-io.c
new file mode 100644
index 000..fd657aa
--- /dev/null
+++ b/hw/avr/sample-io.c
@@ -0,0 +1,176 @@
+/*
+ *  QEMU AVR CPU
+ *
+ *  Copyright (c) 2016 Michael Rolnik
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Lesser General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2.1 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Lesser General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Lesser General Public
+ *  License along with this library; if not, see
+ *  
+ */
+
+/*
+NOTE:
+This is not a real AVR device !!! This is an example !!!
+
+This example can be used to build a real AVR device.
+
+AVR has the following layout of data memory
+
+LD/ST(addr) IN/OUT(addr)
+--- 
+
+#---#
+   .|   |
+   .|   32 CPU registers|
+001f|   |
+0020#---#   
+   .|   |   .
+   .|   64 CPU IO registers |   .
+005f|   |   003f
+0060#---#
+   .|   |
+   .|  160 EXT IO registers |
+00ff|   |
+0100#---#
+|   |
+|  Internal RAM |
+|   |
+#---#
+|   |
+|  External RAM |
+|   |
+#---#
+
+Current AVR/CPU implementation assumes that IO device responsible to
+implement functionality of IO and EXT IO registers is a memory mapped
+device, mapped to addresses in the range [0x0020 .. 0x0100)
+
+IN/OUT are implemented as an alias to LD/ST instructions
+
+Some of CPU IO registers are implemented within the CPU itself, any
+access to them either by LD/ST or IN/OUT won't be routed to the device.
+
+*/
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu-common.h"
+#include "cpu.h"
+#include "include/hw/sysbus.h"
+
+#define TYPE_SAMPLEIO   "SampleIO"
+#define SAMPLEIO(obj)   OBJECT_CHECK(SAMPLEIOState, (obj), TYPE_SAMPLEIO)
+
+#ifndef DEBUG_SAMPLEIO
+#define DEBUG_SAMPLEIO  0
+#endif
+
+#define DPRINTF(fmt, args...) \
+do {  \
+if (DEBUG_SAMPLEIO) { \
+fprintf(stderr, "[%s]%s: " fmt , TYPE_SAMPLEIO, __func__, ##args);\
+

[Qemu-devel] [PATCH v11 1/9] target-avr: AVR cores support is added.

[Michael Rolnik]

1. basic CPU structure
2. registers
3. no instructions
4. saving sreg, rampD, rampX, rampY, rampD, eind in HW representation

Signed-off-by: Michael Rolnik 
---
 arch_init.c |   2 +
 configure   |   5 +
 default-configs/avr-softmmu.mak |  21 +++
 include/disas/bfd.h |   6 +
 include/sysemu/arch_init.h  |   1 +
 target-avr/Makefile.objs|  23 +++
 target-avr/cpu-qom.h|  85 +++
 target-avr/cpu.c| 307 
 target-avr/cpu.h| 179 +++
 target-avr/gdbstub.c|  85 +++
 target-avr/helper.c |  87 
 target-avr/helper.h |  22 +++
 target-avr/machine.c| 117 +++
 target-avr/machine.h|  22 +++
 target-avr/translate.c  | 283 
 target-avr/translate.h  | 116 +++
 16 files changed, 1361 insertions(+)
 create mode 100644 default-configs/avr-softmmu.mak
 create mode 100644 target-avr/Makefile.objs
 create mode 100644 target-avr/cpu-qom.h
 create mode 100644 target-avr/cpu.c
 create mode 100644 target-avr/cpu.h
 create mode 100644 target-avr/gdbstub.c
 create mode 100644 target-avr/helper.c
 create mode 100644 target-avr/helper.h
 create mode 100644 target-avr/machine.c
 create mode 100644 target-avr/machine.h
 create mode 100644 target-avr/translate.c
 create mode 100644 target-avr/translate.h

diff --git a/arch_init.c b/arch_init.c
index fa05973..be6e6de 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -80,6 +80,8 @@ int graphic_depth = 32;
 #define QEMU_ARCH QEMU_ARCH_UNICORE32
 #elif defined(TARGET_TRICORE)
 #define QEMU_ARCH QEMU_ARCH_TRICORE
+#elif defined(TARGET_AVR)
+#define QEMU_ARCH QEMU_ARCH_AVR
 #endif
 
 const uint32_t arch_type = QEMU_ARCH;
diff --git a/configure b/configure
index 5ada56d..2515124 100755
--- a/configure
+++ b/configure
@@ -5636,6 +5636,8 @@ case "$target_name" in
   x86_64)
 TARGET_BASE_ARCH=i386
   ;;
+  avr)
+  ;;
   alpha)
   ;;
   arm|armeb)
@@ -5832,6 +5834,9 @@ disas_config() {
 
 for i in $ARCH $TARGET_BASE_ARCH ; do
   case "$i" in
+  avr)
+disas_config "AVR"
+  ;;
   alpha)
 disas_config "ALPHA"
   ;;
diff --git a/default-configs/avr-softmmu.mak b/default-configs/avr-softmmu.mak
new file mode 100644
index 000..003465d
--- /dev/null
+++ b/default-configs/avr-softmmu.mak
@@ -0,0 +1,21 @@
+#
+#  QEMU AVR CPU
+#
+#  Copyright (c) 2016 Michael Rolnik
+#
+#  This library is free software; you can redistribute it and/or
+#  modify it under the terms of the GNU Lesser General Public
+#  License as published by the Free Software Foundation; either
+#  version 2.1 of the License, or (at your option) any later version.
+#
+#  This library is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+#  Lesser General Public License for more details.
+#
+#  You should have received a copy of the GNU Lesser General Public
+#  License along with this library; if not, see
+#  
+#
+
+# Default configuration for avr-softmmu
diff --git a/include/disas/bfd.h b/include/disas/bfd.h
index 5fbff00..5578cb7 100644
--- a/include/disas/bfd.h
+++ b/include/disas/bfd.h
@@ -213,6 +213,12 @@ enum bfd_architecture
 #define bfd_mach_m32r  0  /* backwards compatibility */
   bfd_arch_mn10200,/* Matsushita MN10200 */
   bfd_arch_mn10300,/* Matsushita MN10300 */
+  bfd_arch_avr,   /* Atmel AVR microcontrollers.  */
+#define bfd_mach_avr1  1
+#define bfd_mach_avr2  2
+#define bfd_mach_avr3  3
+#define bfd_mach_avr4  4
+#define bfd_mach_avr5  5
   bfd_arch_cris,   /* Axis CRIS */
 #define bfd_mach_cris_v0_v10   255
 #define bfd_mach_cris_v32  32
diff --git a/include/sysemu/arch_init.h b/include/sysemu/arch_init.h
index d690dfa..8c75777 100644
--- a/include/sysemu/arch_init.h
+++ b/include/sysemu/arch_init.h
@@ -23,6 +23,7 @@ enum {
 QEMU_ARCH_UNICORE32 = (1 << 14),
 QEMU_ARCH_MOXIE = (1 << 15),
 QEMU_ARCH_TRICORE = (1 << 16),
+QEMU_ARCH_AVR = (1 << 17),
 };
 
 extern const uint32_t arch_type;
diff --git a/target-avr/Makefile.objs b/target-avr/Makefile.objs
new file mode 100644
index 000..2a10104
--- /dev/null
+++ b/target-avr/Makefile.objs
@@ -0,0 +1,23 @@
+#
+#  QEMU AVR CPU
+#
+#  Copyright (c) 2016 Michael Rolnik
+#
+#  This library is free software; you can redistribute it and/or
+#  modify it under the terms of the GNU Lesser General Public
+#  License as published by the Free Software Foundation; either
+#  version 2.1 of the License, or (at your option) any later version.
+#
+#  This library is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied 

[Qemu-devel] [PATCH] qemu-img: add the 'dd' subcommand

[Reda Sallahi]

This patch adds a basic dd subcommand analogous to dd(1) to qemu-img.

For the start, this implements the bs, if, of and count options and requires
both if and of to be specified (no stdin/stdout if not specified) and doesn't
support tty, pipes, etc.

The image format must be specified with -O for the output if the raw format
is not the intended one.

get_size() was needed for the size syntax dd(1) supports which is different
from qemu_strtosz_suffix().

Two tests are added to test qemu-img dd.

Signed-off-by: Reda Sallahi 
---
 qemu-img-cmds.hx   |   6 +
 qemu-img.c | 645 -
 tests/qemu-iotests/158 |  53 
 tests/qemu-iotests/158.out |  15 ++
 tests/qemu-iotests/159 |  56 
 tests/qemu-iotests/159.out |  87 ++
 tests/qemu-iotests/group   |   2 +
 7 files changed, 863 insertions(+), 1 deletion(-)
 create mode 100755 tests/qemu-iotests/158
 create mode 100644 tests/qemu-iotests/158.out
 create mode 100755 tests/qemu-iotests/159
 create mode 100644 tests/qemu-iotests/159.out

diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index 7e95b2d..03bdd7a 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -45,6 +45,12 @@ STEXI
 @item convert [--object @var{objectdef}] [--image-opts] [-c] [-p] [-q] [-n] 
[-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-o 
@var{options}] [-s @var{snapshot_id_or_name}] [-l @var{snapshot_param}] [-S 
@var{sparse_size}] @var{filename} [@var{filename2} [...]] @var{output_filename}
 ETEXI
 
+DEF("dd", img_dd,
+"dd [--image-opts] [-f fmt] [-O output_fmt] [bs=block_size] [count=blocks] 
if=input of=output")
+STEXI
+@item dd [--image-opts] [-f @var{fmt}] [-O @var{output_fmt}] 
[bs=@var{block_size}] [count=@var{blocks}] if=@var{input} of=@var{output}
+ETEXI
+
 DEF("info", img_info,
 "info [--object objectdef] [--image-opts] [-f fmt] [--output=ofmt] 
[--backing-chain] filename")
 STEXI
diff --git a/qemu-img.c b/qemu-img.c
index ea5970b..d7f134d 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -166,7 +166,14 @@ static void QEMU_NORETURN help(void)
"Parameters to compare subcommand:\n"
"  '-f' first image format\n"
"  '-F' second image format\n"
-   "  '-s' run in Strict mode - fail on different image size or sector 
allocation\n";
+   "  '-s' run in Strict mode - fail on different image size or sector 
allocation\n"
+   "\n"
+   "Parameters to dd subcommand:\n"
+   "  'bs=BYTES' read and write up to BYTES bytes at a time "
+   "(default: 512)\n"
+   "  'count=N' copy only N input blocks\n"
+   "  'if=FILE' read from FILE instead of stdin\n"
+   "  'of=FILE' write to FILE instead of stdout\n";
 
 printf("%s\nSupported formats:", help_msg);
 bdrv_iterate_format(format_print, NULL);
@@ -3794,6 +3801,642 @@ out:
 return 0;
 }
 
+#define C_BS  01
+#define C_CBS 02
+#define C_CONV04
+#define C_COUNT   010
+#define C_IBS 020
+#define C_IF  040
+#define C_IFLAG   0100
+#define C_OBS 0200
+#define C_OF  0400
+#define C_OFLAG   01000
+#define C_SEEK02000
+#define C_SKIP04000
+#define C_STATUS  01
+
+struct DdEss {
+unsigned int flags;
+unsigned int status;
+unsigned int conv;
+size_t count;
+size_t cbsz; /* Conversion block size */
+};
+
+struct DdIo {
+size_t bsz;/* Block size */
+off_t offset;
+const char *filename;
+unsigned int flags;
+uint8_t *buf;
+};
+
+struct DdOpts {
+const char *name;
+int (*f)(const char *, struct DdIo *, struct DdIo *, struct DdEss *);
+unsigned int flag;
+};
+
+static size_t get_size(const char *str)
+{
+/* XXX: handle {k,m,g}B notations */
+unsigned long num;
+size_t res = 0;
+const char *buf;
+int ret;
+
+errno = 0;
+if (strchr(str, '-')) {
+error_report("invalid number: '%s'", str);
+errno = EINVAL;
+return res;
+}
+ret = qemu_strtoul(str, , 0, );
+
+if (ret < 0) {
+error_report("invalid number: '%s'", str);
+return res;
+}
+
+switch (*buf) {
+case '\0':
+case 'c':
+res = num;
+break;
+case 'w':
+res = num * 2;
+break;
+case 'b':
+res = num * 512;
+break;
+case 'K':
+res = num * 1024;
+break;
+case 'M':
+res = num * 1024 * 1024;
+break;
+case 'G':
+res = num * 1024 * 1024 * 1024;
+break;
+case 'T':
+res = num * 1024 * 1024 * 1024 * 1024;
+break;
+case 'P':
+res = num * 1024 * 1024 * 1024 * 1024 * 1024;
+break;
+case 'E':
+res = num * 1024 * 1024 * 1024 * 1024 * 1024 * 1024;
+break;
+case 'Z':
+res = num * 1024 * 1024 * 1024 * 1024 * 1024 * 1024 * 1024;
+break;
+case 'Y':
+res = num * 1024 * 1024 * 1024 * 1024 * 1024 * 1024 

[Qemu-devel] [PATCH v11 9/9] target-avr: adding instruction decoder

[Michael Rolnik]

Signed-off-by: Michael Rolnik 
---
 target-avr/Makefile.objs |   1 +
 target-avr/decode.c  | 693 +++
 target-avr/helper.c  |   2 +
 target-avr/helper.h  |   1 +
 target-avr/translate.c   |  12 -
 5 files changed, 697 insertions(+), 12 deletions(-)
 create mode 100644 target-avr/decode.c

diff --git a/target-avr/Makefile.objs b/target-avr/Makefile.objs
index fdafcd5..993a3a9 100644
--- a/target-avr/Makefile.objs
+++ b/target-avr/Makefile.objs
@@ -20,4 +20,5 @@
 
 obj-y   += translate.o cpu.o helper.o translate-inst.o
 obj-y   += gdbstub.o
+obj-y   += decode.o
 obj-$(CONFIG_SOFTMMU) += machine.o
diff --git a/target-avr/decode.c b/target-avr/decode.c
new file mode 100644
index 000..41e16e3
--- /dev/null
+++ b/target-avr/decode.c
@@ -0,0 +1,693 @@
+/*
+ * QEMU AVR CPU
+ *
+ * Copyright (c) 2016 Michael Rolnik
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see
+ * 
+ */
+
+#include 
+#include "translate.h"
+
+void avr_decode(uint32_t pc, uint32_t *l, uint32_t c, translate_function_t *t)
+{
+uint32_t opc  = extract32(c, 0, 16);
+switch (opc & 0xd000) {
+case 0x: {
+switch (opc & 0x2c00) {
+case 0x: {
+switch (opc & 0x0300) {
+case 0x: {
+*l = 16;
+*t = _translate_NOP;
+break;
+}
+case 0x0100: {
+*l = 16;
+*t = _translate_MOVW;
+break;
+}
+case 0x0200: {
+*l = 16;
+*t = _translate_MULS;
+break;
+}
+case 0x0300: {
+switch (opc & 0x0088) {
+case 0x: {
+*l = 16;
+*t = _translate_MULSU;
+break;
+}
+case 0x0008: {
+*l = 16;
+*t = _translate_FMUL;
+break;
+}
+case 0x0080: {
+*l = 16;
+*t = _translate_FMULS;
+break;
+}
+case 0x0088: {
+*l = 16;
+*t = _translate_FMULSU;
+break;
+}
+}
+break;
+}
+}
+break;
+}
+case 0x0400: {
+*l = 16;
+*t = _translate_CPC;
+break;
+}
+case 0x0800: {
+*l = 16;
+*t = _translate_SBC;
+break;
+}
+case 0x0c00: {
+*l = 16;
+*t = _translate_ADD;
+break;
+}
+case 0x2000: {
+*l = 16;
+*t = _translate_AND;
+break;
+}
+case 0x2400: {
+*l = 16;
+*t = _translate_EOR;
+break;
+}
+case 0x2800: {
+*l = 16;
+*t = _translate_OR;
+break;
+}
+case 0x2c00: {
+*l = 16;
+*t = _translate_MOV;
+break;
+}
+}
+break;
+}
+case 0x1000: {
+switch (opc & 0x2000) {
+case 0x: {
+   

[Qemu-devel] [PATCH v11 2/9] target-avr: adding AVR CPU features/flavors

[Michael Rolnik]

Signed-off-by: Michael Rolnik 
---
 target-avr/cpu.c | 313 ++-
 target-avr/cpu.h |  48 
 target-avr/machine.c |   1 +
 3 files changed, 361 insertions(+), 1 deletion(-)

diff --git a/target-avr/cpu.c b/target-avr/cpu.c
index 7e8d34b..0ee4930 100644
--- a/target-avr/cpu.c
+++ b/target-avr/cpu.c
@@ -204,6 +204,302 @@ static void avr_cpu_class_init(ObjectClass *oc, void 
*data)
 dc->cannot_destroy_with_object_finalize_yet = true;
 }
 
+static void avr_avr1_initfn(Object *obj)
+{
+AVRCPU *cpu = AVR_CPU(obj);
+CPUAVRState *env = >env;
+
+avr_set_feature(env, AVR_FEATURE_LPM);
+avr_set_feature(env, AVR_FEATURE_2_BYTE_SP);
+avr_set_feature(env, AVR_FEATURE_2_BYTE_PC);
+}
+
+static void avr_avr2_initfn(Object *obj)
+{
+AVRCPU *cpu = AVR_CPU(obj);
+CPUAVRState *env = >env;
+
+avr_set_feature(env, AVR_FEATURE_LPM);
+avr_set_feature(env, AVR_FEATURE_IJMP_ICALL);
+avr_set_feature(env, AVR_FEATURE_ADIW_SBIW);
+avr_set_feature(env, AVR_FEATURE_SRAM);
+avr_set_feature(env, AVR_FEATURE_BREAK);
+
+avr_set_feature(env, AVR_FEATURE_2_BYTE_PC);
+avr_set_feature(env, AVR_FEATURE_2_BYTE_SP);
+}
+
+static void avr_avr25_initfn(Object *obj)
+{
+AVRCPU *cpu = AVR_CPU(obj);
+CPUAVRState *env = >env;
+
+avr_set_feature(env, AVR_FEATURE_LPM);
+avr_set_feature(env, AVR_FEATURE_IJMP_ICALL);
+avr_set_feature(env, AVR_FEATURE_ADIW_SBIW);
+avr_set_feature(env, AVR_FEATURE_SRAM);
+avr_set_feature(env, AVR_FEATURE_BREAK);
+
+avr_set_feature(env, AVR_FEATURE_2_BYTE_PC);
+avr_set_feature(env, AVR_FEATURE_2_BYTE_SP);
+avr_set_feature(env, AVR_FEATURE_LPMX);
+avr_set_feature(env, AVR_FEATURE_MOVW);
+}
+
+static void avr_avr3_initfn(Object *obj)
+{
+AVRCPU *cpu = AVR_CPU(obj);
+CPUAVRState *env = >env;
+
+avr_set_feature(env, AVR_FEATURE_LPM);
+avr_set_feature(env, AVR_FEATURE_IJMP_ICALL);
+avr_set_feature(env, AVR_FEATURE_ADIW_SBIW);
+avr_set_feature(env, AVR_FEATURE_SRAM);
+avr_set_feature(env, AVR_FEATURE_BREAK);
+
+avr_set_feature(env, AVR_FEATURE_2_BYTE_PC);
+avr_set_feature(env, AVR_FEATURE_2_BYTE_SP);
+avr_set_feature(env, AVR_FEATURE_JMP_CALL);
+}
+
+static void avr_avr31_initfn(Object *obj)
+{
+AVRCPU *cpu = AVR_CPU(obj);
+CPUAVRState *env = >env;
+
+avr_set_feature(env, AVR_FEATURE_LPM);
+avr_set_feature(env, AVR_FEATURE_IJMP_ICALL);
+avr_set_feature(env, AVR_FEATURE_ADIW_SBIW);
+avr_set_feature(env, AVR_FEATURE_SRAM);
+avr_set_feature(env, AVR_FEATURE_BREAK);
+
+avr_set_feature(env, AVR_FEATURE_2_BYTE_PC);
+avr_set_feature(env, AVR_FEATURE_2_BYTE_SP);
+avr_set_feature(env, AVR_FEATURE_RAMPZ);
+avr_set_feature(env, AVR_FEATURE_ELPM);
+avr_set_feature(env, AVR_FEATURE_JMP_CALL);
+}
+
+static void avr_avr35_initfn(Object *obj)
+{
+AVRCPU *cpu = AVR_CPU(obj);
+CPUAVRState *env = >env;
+
+avr_set_feature(env, AVR_FEATURE_LPM);
+avr_set_feature(env, AVR_FEATURE_IJMP_ICALL);
+avr_set_feature(env, AVR_FEATURE_ADIW_SBIW);
+avr_set_feature(env, AVR_FEATURE_SRAM);
+avr_set_feature(env, AVR_FEATURE_BREAK);
+
+avr_set_feature(env, AVR_FEATURE_2_BYTE_PC);
+avr_set_feature(env, AVR_FEATURE_2_BYTE_SP);
+avr_set_feature(env, AVR_FEATURE_JMP_CALL);
+avr_set_feature(env, AVR_FEATURE_LPMX);
+avr_set_feature(env, AVR_FEATURE_MOVW);
+}
+
+static void avr_avr4_initfn(Object *obj)
+{
+AVRCPU *cpu = AVR_CPU(obj);
+CPUAVRState *env = >env;
+
+avr_set_feature(env, AVR_FEATURE_LPM);
+avr_set_feature(env, AVR_FEATURE_IJMP_ICALL);
+avr_set_feature(env, AVR_FEATURE_ADIW_SBIW);
+avr_set_feature(env, AVR_FEATURE_SRAM);
+avr_set_feature(env, AVR_FEATURE_BREAK);
+
+avr_set_feature(env, AVR_FEATURE_2_BYTE_PC);
+avr_set_feature(env, AVR_FEATURE_2_BYTE_SP);
+avr_set_feature(env, AVR_FEATURE_LPMX);
+avr_set_feature(env, AVR_FEATURE_MOVW);
+avr_set_feature(env, AVR_FEATURE_MUL);
+}
+
+static void avr_avr5_initfn(Object *obj)
+{
+AVRCPU *cpu = AVR_CPU(obj);
+CPUAVRState *env = >env;
+
+avr_set_feature(env, AVR_FEATURE_LPM);
+avr_set_feature(env, AVR_FEATURE_IJMP_ICALL);
+avr_set_feature(env, AVR_FEATURE_ADIW_SBIW);
+avr_set_feature(env, AVR_FEATURE_SRAM);
+avr_set_feature(env, AVR_FEATURE_BREAK);
+
+avr_set_feature(env, AVR_FEATURE_2_BYTE_PC);
+avr_set_feature(env, AVR_FEATURE_2_BYTE_SP);
+avr_set_feature(env, AVR_FEATURE_JMP_CALL);
+avr_set_feature(env, AVR_FEATURE_LPMX);
+avr_set_feature(env, AVR_FEATURE_MOVW);
+avr_set_feature(env, AVR_FEATURE_MUL);
+}
+
+static void avr_avr51_initfn(Object *obj)
+{
+AVRCPU *cpu = AVR_CPU(obj);
+CPUAVRState *env = >env;
+
+avr_set_feature(env, AVR_FEATURE_LPM);
+avr_set_feature(env, AVR_FEATURE_IJMP_ICALL);
+avr_set_feature(env, AVR_FEATURE_ADIW_SBIW);
+avr_set_feature(env, AVR_FEATURE_SRAM);
+

[Qemu-devel] [PATCH v11 5/9] target-avr: adding AVR interrupt handling

[Michael Rolnik]

Signed-off-by: Michael Rolnik 
---
 target-avr/helper.c | 59 -
 1 file changed, 58 insertions(+), 1 deletion(-)

diff --git a/target-avr/helper.c b/target-avr/helper.c
index ffc9378..3e61193 100644
--- a/target-avr/helper.c
+++ b/target-avr/helper.c
@@ -31,11 +31,68 @@
 
 bool avr_cpu_exec_interrupt(CPUState *cs, int interrupt_request)
 {
-return  false;
+CPUClass *cc = CPU_GET_CLASS(cs);
+AVRCPU *cpu = AVR_CPU(cs);
+CPUAVRState *env = >env;
+
+bool ret = false;
+
+if (interrupt_request & CPU_INTERRUPT_RESET) {
+if (cpu_interrupts_enabled(env)) {
+cs->exception_index = EXCP_RESET;
+cc->do_interrupt(cs);
+
+cs->interrupt_request   &= ~CPU_INTERRUPT_RESET;
+
+ret = true;
+}
+}
+if (interrupt_request & CPU_INTERRUPT_HARD) {
+if (cpu_interrupts_enabled(env) && env->intsrc != 0) {
+int index = ctz32(env->intsrc);
+cs->exception_index = EXCP_INT(index);
+cc->do_interrupt(cs);
+
+env->intsrc &= env->intsrc - 1; /* clear the interrupt */
+cs->interrupt_request &= ~CPU_INTERRUPT_HARD;
+
+ret = true;
+}
+}
+return ret;
 }
 
 void avr_cpu_do_interrupt(CPUState *cs)
 {
+AVRCPU *cpu = AVR_CPU(cs);
+CPUAVRState*env = >env;
+
+uint32_t ret = env->pc_w;
+int vector = 0;
+int size = avr_feature(env, AVR_FEATURE_JMP_CALL) ? 2 : 1;
+int base = 0;/* TODO: where to get it */
+
+if (cs->exception_index == EXCP_RESET) {
+vector = 0;
+} else if (env->intsrc != 0) {
+vector = ctz32(env->intsrc) + 1;
+}
+
+if (avr_feature(env, AVR_FEATURE_3_BYTE_PC)) {
+stb_phys(cs->as, env->sp--, (ret & 0xff));
+stb_phys(cs->as, env->sp--, (ret & 0x00ff00) >>  8);
+stb_phys(cs->as, env->sp--, (ret & 0xff) >> 16);
+} else if (avr_feature(env, AVR_FEATURE_2_BYTE_PC)) {
+stb_phys(cs->as, env->sp--, (ret & 0xff));
+stb_phys(cs->as, env->sp--, (ret & 0x00ff00) >>  8);
+} else {
+stb_phys(cs->as, env->sp--, (ret & 0xff));
+}
+
+env->pc_w = base + vector * size;
+env->sregI = 0;/*  clear Global Interrupt Flag */
+
+cs->exception_index = -1;
 }
 
 int avr_cpu_memory_rw_debug(CPUState *cs, vaddr addr, uint8_t *buf,
-- 
2.4.9 (Apple Git-60)

[Qemu-devel] [PATCH v11 4/9] target-avr: adding instructions encodings

[Michael Rolnik]

Signed-off-by: Michael Rolnik 
---
 target-avr/translate-inst.h | 805 
 1 file changed, 805 insertions(+)
 create mode 100644 target-avr/translate-inst.h

diff --git a/target-avr/translate-inst.h b/target-avr/translate-inst.h
new file mode 100644
index 000..7114d83
--- /dev/null
+++ b/target-avr/translate-inst.h
@@ -0,0 +1,805 @@
+/*
+ *  QEMU AVR CPU
+ *
+ *  Copyright (c) 2016 Michael Rolnik
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Lesser General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2.1 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Lesser General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Lesser General Public
+ *  License along with this library; if not, see
+ *  
+ */
+
+#ifndef AVR_TRANSLATE_INST_H_
+#define AVR_TRANSLATE_INST_H_
+
+typedef struct DisasContextDisasContext;
+
+int avr_translate_NOP(CPUAVRState *env, DisasContext* ctx, uint32_t opcode);
+
+int avr_translate_MOVW(CPUAVRState *env, DisasContext* ctx, uint32_t opcode);
+static inline uint32_t MOVW_Rr(uint32_t opcode)
+{
+return extract32(opcode, 0, 4);
+}
+
+static inline uint32_t MOVW_Rd(uint32_t opcode)
+{
+return extract32(opcode, 4, 4);
+}
+
+int avr_translate_MULS(CPUAVRState *env, DisasContext* ctx, uint32_t opcode);
+static inline uint32_t MULS_Rr(uint32_t opcode)
+{
+return extract32(opcode, 0, 4);
+}
+
+static inline uint32_t MULS_Rd(uint32_t opcode)
+{
+return extract32(opcode, 4, 4);
+}
+
+int avr_translate_MULSU(CPUAVRState *env, DisasContext* ctx, uint32_t opcode);
+static inline uint32_t MULSU_Rr(uint32_t opcode)
+{
+return extract32(opcode, 0, 3);
+}
+
+static inline uint32_t MULSU_Rd(uint32_t opcode)
+{
+return extract32(opcode, 4, 3);
+}
+
+int avr_translate_FMUL(CPUAVRState *env, DisasContext* ctx, uint32_t opcode);
+static inline uint32_t FMUL_Rr(uint32_t opcode)
+{
+return extract32(opcode, 0, 3);
+}
+
+static inline uint32_t FMUL_Rd(uint32_t opcode)
+{
+return extract32(opcode, 4, 3);
+}
+
+int avr_translate_FMULS(CPUAVRState *env, DisasContext* ctx, uint32_t opcode);
+static inline uint32_t FMULS_Rr(uint32_t opcode)
+{
+return extract32(opcode, 0, 3);
+}
+
+static inline uint32_t FMULS_Rd(uint32_t opcode)
+{
+return extract32(opcode, 4, 3);
+}
+
+int avr_translate_FMULSU(CPUAVRState *env, DisasContext* ctx, uint32_t opcode);
+static inline uint32_t FMULSU_Rr(uint32_t opcode)
+{
+return extract32(opcode, 0, 3);
+}
+
+static inline uint32_t FMULSU_Rd(uint32_t opcode)
+{
+return extract32(opcode, 4, 3);
+}
+
+int avr_translate_CPC(CPUAVRState *env, DisasContext* ctx, uint32_t opcode);
+static inline uint32_t CPC_Rd(uint32_t opcode)
+{
+return extract32(opcode, 4, 5);
+}
+
+static inline uint32_t CPC_Rr(uint32_t opcode)
+{
+return (extract32(opcode, 9, 1) << 4) |
+(extract32(opcode, 0, 4));
+}
+
+int avr_translate_SBC(CPUAVRState *env, DisasContext* ctx, uint32_t opcode);
+static inline uint32_t SBC_Rd(uint32_t opcode)
+{
+return extract32(opcode, 4, 5);
+}
+
+static inline uint32_t SBC_Rr(uint32_t opcode)
+{
+return (extract32(opcode, 9, 1) << 4) |
+(extract32(opcode, 0, 4));
+}
+
+int avr_translate_ADD(CPUAVRState *env, DisasContext* ctx, uint32_t opcode);
+static inline uint32_t ADD_Rd(uint32_t opcode)
+{
+return extract32(opcode, 4, 5);
+}
+
+static inline uint32_t ADD_Rr(uint32_t opcode)
+{
+return (extract32(opcode, 9, 1) << 4) |
+(extract32(opcode, 0, 4));
+}
+
+int avr_translate_AND(CPUAVRState *env, DisasContext* ctx, uint32_t opcode);
+static inline uint32_t AND_Rd(uint32_t opcode)
+{
+return extract32(opcode, 4, 5);
+}
+
+static inline uint32_t AND_Rr(uint32_t opcode)
+{
+return (extract32(opcode, 9, 1) << 4) |
+(extract32(opcode, 0, 4));
+}
+
+int avr_translate_EOR(CPUAVRState *env, DisasContext* ctx, uint32_t opcode);
+static inline uint32_t EOR_Rd(uint32_t opcode)
+{
+return extract32(opcode, 4, 5);
+}
+
+static inline uint32_t EOR_Rr(uint32_t opcode)
+{
+return (extract32(opcode, 9, 1) << 4) |
+(extract32(opcode, 0, 4));
+}
+
+int avr_translate_OR(CPUAVRState *env, DisasContext* ctx, uint32_t opcode);
+static inline uint32_t OR_Rd(uint32_t opcode)
+{
+return extract32(opcode, 4, 5);
+}
+
+static inline uint32_t OR_Rr(uint32_t opcode)
+{
+return (extract32(opcode, 9, 1) << 4) |
+(extract32(opcode, 0, 4));
+}
+
+int avr_translate_MOV(CPUAVRState *env, DisasContext* ctx, uint32_t opcode);
+static inline uint32_t MOV_Rd(uint32_t opcode)
+{
+return 

[Qemu-devel] [PATCH v11 0/9] 8bit AVR cores

[Michael Rolnik]

This series of patches adds 8bit AVR cores to QEMU.
All instruction, except BREAK/DES/SPM/SPMX, are implemented. Not fully tested 
yet.
However I was able to execute simple code with functions. e.g fibonacci 
calculation.
This series of patches include a non real, sample board.
No fuses support yet. PC is set to 0 at reset.

the patches include the following
1. just a basic 8bit AVR CPU, without instruction decoding or translation
2. CPU features which allow define the following 8bit AVR cores
 avr1
 avr2 avr25
 avr3 avr31 avr35
 avr4
 avr5 avr51
 avr6
 xmega2 xmega4 xmega5 xmega6 xmega7
3. a definition of sample machine with SRAM, FLASH and CPU which allows to 
execute simple code
4. encoding for all AVR instructions
5. interrupt handling
6. helpers for IN, OUT, SLEEP, WBR & unsupported instructions
7. a decoder which given an opcode decides what istruction it is
8. translation of AVR instruction into TCG
9. all features together

changes since v3
1. rampD/X/Y/Z registers are encoded as 0x00ff (instead of 0x00ff) for 
faster address manipulaton
2. ffs changed to ctz32
3. duplicate code removed at avr_cpu_do_interrupt
4. using andc instead of not + and
5. fixing V flag calculation in varios instructions
6. freeing local variables in PUSH
7. tcg_const_local_i32 -> tcg_const_i32
8. using sextract32 instead of my implementation
9. fixing BLD instruction
10.xor(r) instead of 0xff - r at COM
11.fixing MULS/MULSU not to modify inputs' content
12.using SUB for NEG
13.fixing tcg_gen_qemu_ld/st call in XCH

changes since v4
1. target is now defined as big endian in order to optimize push_ret/pop_ret
2. all style warnings are fixed
3. adding cpu_set/get_sreg functions
4. simplifying gen_goto_tb as there is no real paging
5. env->pc -> env->pc_w
6. making flag dump more compact
7. more spacing
8. renaming CODE/DATA_INDEX -> MMU_CODE/DATA_IDX
9. removing avr_set_feature
10. SPL/SPH set bug fix
11. switching stb_phys to cpu_stb_data
12. cleaning up avr_decode
13. saving sreg, rampD/X/Y/Z, eind in HW format (savevm)
14. saving CPU features (savevm)

changes since v5
1. BLD bug fix
2. decoder generator is added

chages since v6
1. using cpu_get_sreg/cpu_set_sreg in 
avr_cpu_gdb_read_register/avr_cpu_gdb_write_register
2. configure the target as little endian because otherwise GDB does not work
3. fixing and testing gen_push_ret/gen_pop_ret

changes since v7
1. folding back v6 
2. logging at helper_outb and helper_inb are done for non supported yet 
registers only
3. MAINTAINERS updated

changes since v8
1. removing hw/avr from hw/Makefile.obj as it should not be built for all
2. making linux compilable
3. testing on
a. Mac, Apple LLVM version 7.0.0
b. Ubuntu 12.04, gcc 4.9.2
c. Fedora 23, gcc 5.3.1
4. folding back some patches
5. translation bug fixes for ORI, CPI, XOR instructions
6. propper handling of cpu register writes though memory

changes since v9
1. removing forward declarations of static functions
2. disabling debug prints
3. switching to case range instead of if else if ...
4. LD/ST IN/OUT accessing CPU maintainder registers are not routed to any device
5. commenst about sample board and sample IO device added
6. sample board description is more descriptive now
7. memory_region_allocate_system_memory is used to create RAM
8. now there are helper_fullrd & helper_fullwr when LD/ST try to access 
registers

changes since v10
1. movig back fullwr & fullrd into the commit where outb and inb were introduced
2. changing tlb_fill function signature
3. adding empty line between functions
4. adding newline on the last line of the file
5. using tb->flags to generae full access ST/LD instructions
6. fixing SBRC bug
7. folding back 10th commit
8. whenever a new file is introduced it's added to Makefile.objs

Michael Rolnik (9):
  target-avr: AVR cores support is added.
  target-avr: adding AVR CPU features/flavors
  target-avr: adding a sample AVR board
  target-avr: adding instructions encodings
  target-avr: adding AVR interrupt handling
  target-avr: adding helpers for IN, OUT, SLEEP, WBR & unsupported
instructions
  target-avr: adding instruction translation
  target-avr: instruction decoder generator
  target-avr: adding instruction decoder

 MAINTAINERS|6 +
 arch_init.c|2 +
 configure  |5 +
 default-configs/avr-softmmu.mak|   21 +
 hw/avr/Makefile.objs   |   21 +
 hw/avr/sample-io.c |  176 ++
 hw/avr/sample.c|  137 ++
 include/disas/bfd.h|6 +
 include/sysemu/arch_init.h |1 +
 target-avr/Makefile.objs   |   24 +
 target-avr/cpu-qom.h   |   85 +
 target-avr/cpu.c   |  618 +++
 target-avr/cpu.h   |  238 +++
 target-avr/cpugen/CMakeLists.txt   

Re: [Qemu-devel] [PATCH v7 4/7] qom: support arbitrary non-scalar properties with -object

[Eric Blake]

On 07/05/2016 07:11 AM, Daniel P. Berrange wrote:
> The current -object command line syntax only allows for
> creation of objects with scalar properties, or a list
> with a fixed scalar element type. Objects which have
> properties that are represented as structs in the QAPI
> schema cannot be created using -object.
> 

> Thus -object can now support non-scalar properties,
> for example the QMP object
> 
>   {
> "execute": "object-add",
> "arguments": {
>   "qom-type": "demo",
>   "id": "demo0",
>   "parameters": {
> "foo": [
>   { "bar": "one", "wizz": "1" },
>   { "bar": "two", "wizz": "2" }
> ]
>   }
> }
>   }
> 
> Would be creatable via the CLI now using
> 
> $QEMU \
>   -object demo,id=demo0,\
>   foo.0.bar=one,foo.0.wizz=1,\
>   foo.1.bar=two,foo.1.wizz=2

Overall, I like it.

> 
> Notice that this syntax is intentionally compatible
> with that currently used by block drivers.

And this gives me hope that someday we may replace blockdev-add from
QemuOpts munging to direct QAPI type handling. Maybe :)  But certainly
not for 2.7.


> @@ -1721,20 +1721,12 @@ void hmp_netdev_del(Monitor *mon, const QDict *qdict)
>  void hmp_object_add(Monitor *mon, const QDict *qdict)
>  {
>  Error *err = NULL;
> -QemuOpts *opts;
> -OptsVisitor *ov;
> +QmpInputVisitor *qiv;

Will need rebasing now that some of my qapi cleanups have landed in
master, and also depending on whether my adoption of your patch 3/7 in
my 'qapi subset F' series also lands first, since I tweaked your string
input visitor interface slightly.  But the idea is right on track.

>  Object *obj = NULL;
>  
> -opts = qemu_opts_from_qdict(qemu_find_opts("object"), qdict, );
> -if (err) {
> -hmp_handle_error(mon, );
> -return;
> -}
> -
> -ov = opts_visitor_new(opts);
> -obj = user_creatable_add(qdict, opts_get_visitor(ov), );
> -opts_visitor_cleanup(ov);
> -qemu_opts_del(opts);
> +qiv = qmp_string_input_visitor_new((QObject *)qdict, true);

While this cast happens to work (because QDict currently has QObject as
its first member), it is not necessarily safe to future layout changes,
and you should probably be using QOBJECT(qdict) instead.

> +obj = user_creatable_add(qdict, qmp_input_get_visitor(qiv), );
> +qmp_input_visitor_cleanup(qiv);

That looks so much simpler, even if it does need rebasing!

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PATCH v9 12/17] qapi: Change Netdev into a flat union

[Eric Blake]

This is a mostly-mechanical conversion that creates a new flat
union 'Netdev' QAPI type that covers all the branches of the
former 'NetClientOptions' simple union, where the branches are
now listed in a new 'NetClientDriver' enum rather than generated
from the simple union.  The existence of a flat union has no
change to the command line syntax accepted for new code, and
will make it possible for a future patch to switch the QMP
command to parse a boxed union for no change to valid QMP; but
it does have some ripple effect on the C code when dealing with
the new types.

While making the conversion, note that the 'NetLegacy' type
remains unchanged: it applies only to legacy command line options,
and will not be ported to QMP, so it should remain a wrapper
around a simple union; to avoid confusion, the type named
'NetClientOptions' is now gone, and we introduce 'NetLegacyOptions'
in its place.  Then, in the C code, we convert from NetLegacy to
Netdev as soon as possible, so that the bulk of the net stack
only has to deal with one QAPI type, not two.  Note that since
the old legacy code always rejected 'hubport', we can just omit
that branch from the new 'NetLegacyOptions' simple union.

Based on an idea originally by Zoltán Kővágó :
Message-Id: 
<01a527fbf1a5de880091f98cf011616a78ad.1441627176.git.dirty.ice...@gmail.com>
although the sed script in that patch no longer applies due to
other changes in the tree since then, and I also did some manual
cleanups (such as fixing whitespace to keep checkpatch happy).

Signed-off-by: Eric Blake 

---
v8: rewrite commit message, claim authorship, rebase to latest
v7: rebase to latest master
v6: rebase to latest master
---
 qapi-schema.json |  49 ++-
 include/net/net.h|   4 +-
 hw/arm/musicpal.c|   2 +-
 hw/core/qdev-properties-system.c |   2 +-
 hw/net/allwinner_emac.c  |   2 +-
 hw/net/cadence_gem.c |   2 +-
 hw/net/dp8393x.c |   2 +-
 hw/net/e1000.c   |   2 +-
 hw/net/e1000e.c  |   2 +-
 hw/net/eepro100.c|   2 +-
 hw/net/etraxfs_eth.c |   2 +-
 hw/net/fsl_etsec/etsec.c |   2 +-
 hw/net/imx_fec.c |   2 +-
 hw/net/lan9118.c |   2 +-
 hw/net/lance.c   |   2 +-
 hw/net/mcf_fec.c |   2 +-
 hw/net/milkymist-minimac2.c  |   2 +-
 hw/net/mipsnet.c |   2 +-
 hw/net/ne2000-isa.c  |   2 +-
 hw/net/ne2000.c  |   2 +-
 hw/net/opencores_eth.c   |   2 +-
 hw/net/pcnet-pci.c   |   2 +-
 hw/net/rocker/rocker_fp.c|   2 +-
 hw/net/rtl8139.c |   2 +-
 hw/net/smc91c111.c   |   2 +-
 hw/net/spapr_llan.c  |   2 +-
 hw/net/stellaris_enet.c  |   2 +-
 hw/net/vhost_net.c   |  20 +++---
 hw/net/virtio-net.c  |  10 +--
 hw/net/vmxnet3.c |   2 +-
 hw/net/xen_nic.c |   2 +-
 hw/net/xgmac.c   |   2 +-
 hw/net/xilinx_axienet.c  |   2 +-
 hw/net/xilinx_ethlite.c  |   2 +-
 hw/usb/dev-network.c |   2 +-
 monitor.c|  14 ++---
 net/dump.c   |   6 +-
 net/filter.c |   2 +-
 net/hub.c|  22 +++
 net/l2tpv3.c |   6 +-
 net/net.c| 133 +--
 net/netmap.c |   4 +-
 net/slirp.c  |   6 +-
 net/socket.c |   8 +--
 net/tap-win32.c  |   6 +-
 net/tap.c|  24 +++
 net/vde.c|   6 +-
 net/vhost-user.c |  20 +++---
 48 files changed, 230 insertions(+), 172 deletions(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index d2d6506..5658723 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2809,16 +2809,32 @@
 '*queues':'int' } }

 ##
-# @NetClientOptions
+# @NetClientDriver
 #
-# A discriminated record of network device traits.
+# Available netdev drivers.
+#
+# Since 2.7
+##
+{ 'enum': 'NetClientDriver',
+  'data': [ 'none', 'nic', 'user', 'tap', 'l2tpv3', 'socket', 'vde', 'dump',
+'bridge', 'hubport', 'netmap', 'vhost-user' ] }
+
+##
+# @Netdev
+#
+# Captures the configuration of a network device.
+#
+# @id: identifier for monitor commands.
+#
+# @type: Specify the driver used for interpreting remaining arguments.
 #
 # Since 1.2
 #
 # 'l2tpv3' - since 2.1
-#
 ##
-{ 'union': 'NetClientOptions',
+{ 'union': 'Netdev',
+  'base': { 'id': 'str', 'type': 'NetClientDriver' },
+  'discriminator': 'type',
   'data': {
 'none': 'NetdevNoneOptions',
 'nic':  'NetLegacyNicOptions',
@@ -2853,23 +2869,28 @@
 '*vlan': 'int32',
 '*id':   'str',
 '*name': 'str',
-'opts':  

[Qemu-devel] [PATCH v9 15/17] option: make parse_option_bool/number non-static

[Eric Blake]

From: "Daniel P. Berrange" 

The opts-visitor.c opts_type_bool() method has code for
parsing a string to set a bool value, as does the
qemu-option.c parse_option_bool() method, except it
handles fewer cases.

To enable consistency across the codebase, extend
parse_option_bool() to handle "yes", "no", "y" and
"n", and make it non-static. Convert the opts
visitor to call this method directly.

Also make parse_option_number() non-static to allow
for similar reuse later.

Reviewed-by: Marc-André Lureau 
Signed-off-by: Daniel P. Berrange 
Message-Id: <1467724312-9378-3-git-send-email-berra...@redhat.com>
Signed-off-by: Eric Blake 

---
v9: reuse in my series
Previously in Dan's v7 series: Provide a QOM-based authorization API
---
 include/qemu/option.h |  4 
 qapi/opts-visitor.c   | 19 +--
 util/qemu-option.c| 27 ---
 3 files changed, 21 insertions(+), 29 deletions(-)

diff --git a/include/qemu/option.h b/include/qemu/option.h
index 1f9e3f9..2a5266f 100644
--- a/include/qemu/option.h
+++ b/include/qemu/option.h
@@ -37,6 +37,10 @@ int get_param_value(char *buf, int buf_size,
 const char *tag, const char *str);


+void parse_option_bool(const char *name, const char *value, bool *ret,
+   Error **errp);
+void parse_option_number(const char *name, const char *value,
+ uint64_t *ret, Error **errp);
 void parse_option_size(const char *name, const char *value,
uint64_t *ret, Error **errp);
 bool has_help_option(const char *param);
diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
index 1048bbc..084f7cc 100644
--- a/qapi/opts-visitor.c
+++ b/qapi/opts-visitor.c
@@ -334,7 +334,6 @@ opts_type_str(Visitor *v, const char *name, char **obj, 
Error **errp)
 }


-/* mimics qemu-option.c::parse_option_bool() */
 static void
 opts_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
 {
@@ -346,23 +345,7 @@ opts_type_bool(Visitor *v, const char *name, bool *obj, 
Error **errp)
 return;
 }

-if (opt->str) {
-if (strcmp(opt->str, "on") == 0 ||
-strcmp(opt->str, "yes") == 0 ||
-strcmp(opt->str, "y") == 0) {
-*obj = true;
-} else if (strcmp(opt->str, "off") == 0 ||
-strcmp(opt->str, "no") == 0 ||
-strcmp(opt->str, "n") == 0) {
-*obj = false;
-} else {
-error_setg(errp, QERR_INVALID_PARAMETER_VALUE, opt->name,
-   "on|yes|y|off|no|n");
-return;
-}
-} else {
-*obj = true;
-}
+parse_option_bool(opt->name, opt->str, obj, errp);

 processed(ov, name);
 }
diff --git a/util/qemu-option.c b/util/qemu-option.c
index 3467dc2..6fc9ccb 100644
--- a/util/qemu-option.c
+++ b/util/qemu-option.c
@@ -125,25 +125,30 @@ int get_param_value(char *buf, int buf_size,
 return get_next_param_value(buf, buf_size, tag, );
 }

-static void parse_option_bool(const char *name, const char *value, bool *ret,
-  Error **errp)
+void parse_option_bool(const char *name, const char *value, bool *ret,
+   Error **errp)
 {
 if (value != NULL) {
-if (!strcmp(value, "on")) {
-*ret = 1;
-} else if (!strcmp(value, "off")) {
-*ret = 0;
+if (strcmp(value, "on") == 0 ||
+strcmp(value, "yes") == 0 ||
+strcmp(value, "y") == 0) {
+*ret = true;
+} else if (strcmp(value, "off") == 0 ||
+strcmp(value, "no") == 0 ||
+strcmp(value, "n") == 0) {
+*ret = false;
 } else {
-error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
-   name, "'on' or 'off'");
+error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name,
+   "on|yes|y|off|no|n");
+return;
 }
 } else {
-*ret = 1;
+*ret = true;
 }
 }

-static void parse_option_number(const char *name, const char *value,
-uint64_t *ret, Error **errp)
+void parse_option_number(const char *name, const char *value,
+ uint64_t *ret, Error **errp)
 {
 char *postfix;
 uint64_t number;
-- 
2.5.5

[Qemu-devel] [PATCH v9 17/17] qapi: Restore autocast behavior in 'netdev_add'

[Eric Blake]

When we converted 'netdev_add' to a fully-introspectible QMP,
we temporarily lost the ability to parse strings in place of
integers, the way the old QemuOpts code had done.  But now that
we have autocast, we can restore that behavior, by adding a
new 'autocast':true marker in QAPI.

Signed-off-by: Eric Blake 

---
v9: new patch

If we like this patch, one alternative would be to rearrange
the series to get autocast working first, prior to converting
netdev_add to drop 'gen':false, so that we don't have any
temporary regression.
---
 qapi-schema.json   |  6 +-
 scripts/qapi.py| 16 ++--
 scripts/qapi-commands.py   | 11 ++-
 scripts/qapi-introspect.py |  2 +-
 docs/qapi-code-gen.txt | 10 +-
 tests/qapi-schema/test-qapi.py |  2 +-
 6 files changed, 32 insertions(+), 15 deletions(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index 7ec01e1..4dccc9b 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2367,12 +2367,16 @@
 #
 # @id: the name of the new network backend
 #
+# For backward compatibility, this command accepts strings where introspection
+# says that integers are expected; however, clients should not rely on this
+# and should obey the types visible in introspection.
+#
 # Since: 0.14.0
 #
 # Returns: Nothing on success
 #  If @type is not a valid network backend, DeviceNotFound
 ##
-{ 'command': 'netdev_add', 'data': 'Netdev', 'boxed': true }
+{ 'command': 'netdev_add', 'data': 'Netdev', 'boxed': true, 'autocast': true }

 ##
 # @netdev_del:
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 21bc32f..0e33f72 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -702,7 +702,7 @@ def check_keys(expr_elem, meta, required, optional=[]):
 raise QAPIExprError(info,
 "'%s' of %s '%s' should only use false value"
 % (key, meta, name))
-if key == 'boxed' and value is not True:
+if (key == 'boxed' or key == 'autocast') and value is not True:
 raise QAPIExprError(info,
 "'%s' of %s '%s' should only use true value"
 % (key, meta, name))
@@ -737,7 +737,8 @@ def check_exprs(exprs):
 add_struct(expr, info)
 elif 'command' in expr:
 check_keys(expr_elem, 'command', [],
-   ['data', 'returns', 'gen', 'success-response', 'boxed'])
+   ['data', 'returns', 'gen', 'success-response',
+'boxed', 'autocast'])
 add_name(expr['command'], info, 'command')
 elif 'event' in expr:
 check_keys(expr_elem, 'event', [], ['data', 'boxed'])
@@ -838,7 +839,7 @@ class QAPISchemaVisitor(object):
 pass

 def visit_command(self, name, info, arg_type, ret_type,
-  gen, success_response, boxed):
+  gen, success_response, boxed, autocast):
 pass

 def visit_event(self, name, info, arg_type, boxed):
@@ -1181,7 +1182,7 @@ class QAPISchemaAlternateType(QAPISchemaType):

 class QAPISchemaCommand(QAPISchemaEntity):
 def __init__(self, name, info, arg_type, ret_type, gen, success_response,
- boxed):
+ boxed, autocast):
 QAPISchemaEntity.__init__(self, name, info)
 assert not arg_type or isinstance(arg_type, str)
 assert not ret_type or isinstance(ret_type, str)
@@ -1192,6 +1193,7 @@ class QAPISchemaCommand(QAPISchemaEntity):
 self.gen = gen
 self.success_response = success_response
 self.boxed = boxed
+self.autocast = autocast

 def check(self, schema):
 if self._arg_type_name:
@@ -1216,7 +1218,8 @@ class QAPISchemaCommand(QAPISchemaEntity):
 def visit(self, visitor):
 visitor.visit_command(self.name, self.info,
   self.arg_type, self.ret_type,
-  self.gen, self.success_response, self.boxed)
+  self.gen, self.success_response, self.boxed,
+  self.autocast)


 class QAPISchemaEvent(QAPISchemaEntity):
@@ -1422,6 +1425,7 @@ class QAPISchema(object):
 gen = expr.get('gen', True)
 success_response = expr.get('success-response', True)
 boxed = expr.get('boxed', False)
+autocast = expr.get('autocast', False)
 if isinstance(data, OrderedDict):
 data = self._make_implicit_object_type(
 name, info, 'arg', self._make_members(data, info))
@@ -1429,7 +1433,7 @@ class QAPISchema(object):
 assert len(rets) == 1
 rets = self._make_array_type(rets[0], info)
 self._def_entity(QAPISchemaCommand(name, info, data, rets, gen,
-   success_response, boxed))
+   success_response, boxed, autocast))

 def 

[Qemu-devel] [PATCH v9 11/17] block: Simplify drive-mirror

[Eric Blake]

Now that we can support boxed commands, use it to greatly
reduce the number of parameters (and likelihood of getting
out of sync) when adjusting drive-mirror parameters.

Signed-off-by: Eric Blake 
Reviewed-by: John Snow 

---
v9: s/box/boxed/, trivial enough to keep R-b
v8: rebase, drop stale sentence in docs, don't rearrange initialiation
v7: new patch
---
 qapi/block-core.json | 20 +++---
 blockdev.c   | 76 +++-
 hmp.c| 25 -
 3 files changed, 60 insertions(+), 61 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index cbf4410..a085c85 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1108,6 +1108,21 @@
 #
 # Start mirroring a block device's writes to a new destination.
 #
+# See DriveMirror for parameter descriptions
+#
+# Returns: nothing on success
+#  If @device is not a valid block device, DeviceNotFound
+#
+# Since 1.3
+##
+{ 'command': 'drive-mirror', 'boxed': true,
+  'data': 'DriveMirror' }
+
+##
+# DriveMirror
+#
+# A set of parameters describing drive mirror setup.
+#
 # @device:  the name of the device whose writes should be mirrored.
 #
 # @target: the target of the new image. If the file exists, or if it
@@ -1154,12 +1169,9 @@
 # written. Both will result in identical contents.
 # Default is true. (Since 2.4)
 #
-# Returns: nothing on success
-#  If @device is not a valid block device, DeviceNotFound
-#
 # Since 1.3
 ##
-{ 'command': 'drive-mirror',
+{ 'struct': 'DriveMirror',
   'data': { 'device': 'str', 'target': 'str', '*format': 'str',
 '*node-name': 'str', '*replaces': 'str',
 'sync': 'MirrorSyncMode', '*mode': 'NewImageMode',
diff --git a/blockdev.c b/blockdev.c
index ddf30e1..f23bf99 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3458,19 +3458,7 @@ static void blockdev_mirror_common(BlockDriverState *bs,
  block_job_cb, bs, errp);
 }

-void qmp_drive_mirror(const char *device, const char *target,
-  bool has_format, const char *format,
-  bool has_node_name, const char *node_name,
-  bool has_replaces, const char *replaces,
-  enum MirrorSyncMode sync,
-  bool has_mode, enum NewImageMode mode,
-  bool has_speed, int64_t speed,
-  bool has_granularity, uint32_t granularity,
-  bool has_buf_size, int64_t buf_size,
-  bool has_on_source_error, BlockdevOnError 
on_source_error,
-  bool has_on_target_error, BlockdevOnError 
on_target_error,
-  bool has_unmap, bool unmap,
-  Error **errp)
+void qmp_drive_mirror(DriveMirror *arg, Error **errp)
 {
 BlockDriverState *bs;
 BlockBackend *blk;
@@ -3481,11 +3469,12 @@ void qmp_drive_mirror(const char *device, const char 
*target,
 QDict *options = NULL;
 int flags;
 int64_t size;
+const char *format = arg->format;

-blk = blk_by_name(device);
+blk = blk_by_name(arg->device);
 if (!blk) {
 error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
-  "Device '%s' not found", device);
+  "Device '%s' not found", arg->device);
 return;
 }

@@ -3493,24 +3482,25 @@ void qmp_drive_mirror(const char *device, const char 
*target,
 aio_context_acquire(aio_context);

 if (!blk_is_available(blk)) {
-error_setg(errp, QERR_DEVICE_HAS_NO_MEDIUM, device);
+error_setg(errp, QERR_DEVICE_HAS_NO_MEDIUM, arg->device);
 goto out;
 }
 bs = blk_bs(blk);
-if (!has_mode) {
-mode = NEW_IMAGE_MODE_ABSOLUTE_PATHS;
+if (!arg->has_mode) {
+arg->mode = NEW_IMAGE_MODE_ABSOLUTE_PATHS;
 }

-if (!has_format) {
-format = mode == NEW_IMAGE_MODE_EXISTING ? NULL : bs->drv->format_name;
+if (!arg->has_format) {
+format = (arg->mode == NEW_IMAGE_MODE_EXISTING
+  ? NULL : bs->drv->format_name);
 }

 flags = bs->open_flags | BDRV_O_RDWR;
 source = backing_bs(bs);
-if (!source && sync == MIRROR_SYNC_MODE_TOP) {
-sync = MIRROR_SYNC_MODE_FULL;
+if (!source && arg->sync == MIRROR_SYNC_MODE_TOP) {
+arg->sync = MIRROR_SYNC_MODE_FULL;
 }
-if (sync == MIRROR_SYNC_MODE_NONE) {
+if (arg->sync == MIRROR_SYNC_MODE_NONE) {
 source = bs;
 }

@@ -3520,18 +3510,18 @@ void qmp_drive_mirror(const char *device, const char 
*target,
 goto out;
 }

-if (has_replaces) {
+if (arg->has_replaces) {
 BlockDriverState *to_replace_bs;
 AioContext *replace_aio_context;
 int64_t replace_size;

-if (!has_node_name) {
+if (!arg->has_node_name) {
 error_setg(errp, "a node-name must be provided when replacing a"
  

[Qemu-devel] [PATCH v9 16/17] qapi: Tweak QmpInputVisitor to optionally do string conversion

[Eric Blake]

Currently the QmpInputVisitor assumes that all scalar
values are directly represented as their final types.
ie it assumes an 'int' is using QInt, and a 'bool' is
using QBool.

This adds an alternative mode where a QString can also
be parsed in place of the native type, by adding a parameter
and updating all callers.  For now, only the testsuite sets
the new autocast parameter.

This makes it possible to use QmpInputVisitor with a QDict
produced from QemuOpts, where everything is in string format.
It also makes it possible for the next patch to support
back-compat in legacy commands like 'netdev_add' that no
longer use QemuOpts, but must parse the same set of QMP
constructs that QemuOpts would historically allow.

Based heavily on a patch by Daniel P. Berrange :
Message-Id: <1467724312-9378-4-git-send-email-berra...@redhat.com>

Signed-off-by: Eric Blake 

---
v9: new patch (but heavily draws from 3/7 in Dan's v7 series,
Provide a QOM-based authorization API)
---
 scripts/qapi-commands.py   |   2 +-
 include/qapi/qmp-input-visitor.h   |  24 --
 qapi/qmp-input-visitor.c   | 100 ++---
 qmp.c  |   2 +-
 qom/qom-qobject.c  |   2 +-
 tests/check-qnull.c|   2 +-
 tests/test-qmp-commands.c  |   2 +-
 tests/test-qmp-input-strict.c  |   2 +-
 tests/test-qmp-input-visitor.c | 150 -
 tests/test-visitor-serialization.c |   2 +-
 docs/qapi-code-gen.txt |   2 +-
 11 files changed, 264 insertions(+), 26 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index a06a2c4..f526c13 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -117,7 +117,7 @@ def gen_marshal(name, arg_type, boxed, ret_type):
 Visitor *v;
 %(c_name)s arg = {0};

-v = qmp_input_visitor_new(QOBJECT(args), true);
+v = qmp_input_visitor_new(QOBJECT(args), true, false);
 visit_start_struct(v, NULL, NULL, 0, );
 if (err) {
 goto out;
diff --git a/include/qapi/qmp-input-visitor.h b/include/qapi/qmp-input-visitor.h
index f3ff5f3..275a167 100644
--- a/include/qapi/qmp-input-visitor.h
+++ b/include/qapi/qmp-input-visitor.h
@@ -19,12 +19,26 @@

 typedef struct QmpInputVisitor QmpInputVisitor;

-/*
- * Return a new input visitor that converts QMP to QAPI.
+/**
+ * qmp_input_visitor_new:
+ * @obj: the input object to visit
+ * @strict: whether to require that all input keys are consumed
+ * @autocast: whether to allow strings in place of other scalars
  *
- * Set @strict to reject a parse that doesn't consume all keys of a
- * dictionary; otherwise excess input is ignored.
+ * Create a new input visitor that converts QMP to QAPI.
+ *
+ * If @strict is set to true, then an error will be reported if any
+ * dict keys are not consumed during visitation.
+ *
+ * When @autocast is false, any scalar values in the @obj input data
+ * structure should be in the required type already. ie if visiting a
+ * bool, the value should already be a QBool instance.  Otherwise, a
+ * string value that can be parsed as the scalar is acceptable;
+ * however, it is not possible to parse QAPI alternates in autocast
+ * mode.
+ *
+ * Returns: a new input visitor
  */
-Visitor *qmp_input_visitor_new(QObject *obj, bool strict);
+Visitor *qmp_input_visitor_new(QObject *obj, bool strict, bool autocast);

 #endif
diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index 64dd392..7a1b93a 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -20,6 +20,7 @@
 #include "qemu-common.h"
 #include "qapi/qmp/types.h"
 #include "qapi/qmp/qerror.h"
+#include "qemu/cutils.h"

 #define QIV_STACK_SIZE 1024

@@ -47,6 +48,9 @@ struct QmpInputVisitor

 /* True to reject parse in visit_end_struct() if unvisited keys remain. */
 bool strict;
+
+/* True to allow strings in place of native scalars. */
+bool autocast;
 };

 static QmpInputVisitor *to_qiv(Visitor *v)
@@ -234,6 +238,8 @@ static void qmp_input_start_alternate(Visitor *v, const 
char *name,
 QmpInputVisitor *qiv = to_qiv(v);
 QObject *qobj = qmp_input_get_object(qiv, name, false);

+assert(!qiv->autocast);
+
 if (!qobj) {
 *obj = NULL;
 error_setg(errp, QERR_MISSING_PARAMETER, name ? name : "null");
@@ -250,11 +256,21 @@ static void qmp_input_type_int64(Visitor *v, const char 
*name, int64_t *obj,
  Error **errp)
 {
 QmpInputVisitor *qiv = to_qiv(v);
-QInt *qint = qobject_to_qint(qmp_input_get_object(qiv, name, true));
+QObject *qobj = qmp_input_get_object(qiv, name, true);
+QInt *qint = qobject_to_qint(qobj);

 if (!qint) {
-error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-   "integer");
+QString *qstr = qobject_to_qstring(qobj);
+
+if (qiv->autocast && qstr) {
+uint64_t ret 

[Qemu-devel] [PATCH v9 13/17] net: Use correct type for bool flag

[Eric Blake]

is_netdev is only used as a bool, so make it one.

Signed-off-by: Eric Blake 

---
v9: no change
v8: no change
v7: no change
v6: rebase to latest context
---
 include/net/net.h|  2 +-
 hw/usb/dev-network.c |  2 +-
 net/net.c| 12 ++--
 3 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/include/net/net.h b/include/net/net.h
index 48382c3..e8d9e9e 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -203,7 +203,7 @@ extern const char *host_net_devices[];
 extern const char *legacy_tftp_prefix;
 extern const char *legacy_bootp_filename;

-int net_client_init(QemuOpts *opts, int is_netdev, Error **errp);
+int net_client_init(QemuOpts *opts, bool is_netdev, Error **errp);
 int net_client_parse(QemuOptsList *opts_list, const char *str);
 int net_init_clients(void);
 void net_check_clients(void);
diff --git a/hw/usb/dev-network.c b/hw/usb/dev-network.c
index 3ced4ba..c0f1193 100644
--- a/hw/usb/dev-network.c
+++ b/hw/usb/dev-network.c
@@ -1396,7 +1396,7 @@ static USBDevice *usb_net_init(USBBus *bus, const char 
*cmdline)
 qemu_opt_set(opts, "type", "nic", _abort);
 qemu_opt_set(opts, "model", "usb", _abort);

-idx = net_client_init(opts, 0, _err);
+idx = net_client_init(opts, false, _err);
 if (local_err) {
 error_report_err(local_err);
 return NULL;
diff --git a/net/net.c b/net/net.c
index 4a7f392..c124b11 100644
--- a/net/net.c
+++ b/net/net.c
@@ -960,7 +960,7 @@ static int (* const 
net_client_init_fun[NET_CLIENT_DRIVER__MAX])(
 };


-static int net_client_init1(const void *object, int is_netdev, Error **errp)
+static int net_client_init1(const void *object, bool is_netdev, Error **errp)
 {
 Netdev legacy = {0};
 const Netdev *netdev;
@@ -1060,7 +1060,7 @@ static int net_client_init1(const void *object, int 
is_netdev, Error **errp)
 }


-int net_client_init(QemuOpts *opts, int is_netdev, Error **errp)
+int net_client_init(QemuOpts *opts, bool is_netdev, Error **errp)
 {
 void *object = NULL;
 Error *err = NULL;
@@ -1153,7 +1153,7 @@ void hmp_host_net_add(Monitor *mon, const QDict *qdict)

 qemu_opt_set(opts, "type", device, _abort);

-net_client_init(opts, 0, _err);
+net_client_init(opts, false, _err);
 if (local_err) {
 error_report_err(local_err);
 monitor_printf(mon, "adding host network device %s failed\n", device);
@@ -1183,7 +1183,7 @@ void hmp_host_net_remove(Monitor *mon, const QDict *qdict)

 void netdev_add(QemuOpts *opts, Error **errp)
 {
-net_client_init(opts, 1, errp);
+net_client_init(opts, true, errp);
 }

 void qmp_netdev_add(QDict *qdict, QObject **ret, Error **errp)
@@ -1481,7 +1481,7 @@ static int net_init_client(void *dummy, QemuOpts *opts, 
Error **errp)
 {
 Error *local_err = NULL;

-net_client_init(opts, 0, _err);
+net_client_init(opts, false, _err);
 if (local_err) {
 error_report_err(local_err);
 return -1;
@@ -1495,7 +1495,7 @@ static int net_init_netdev(void *dummy, QemuOpts *opts, 
Error **errp)
 Error *local_err = NULL;
 int ret;

-ret = net_client_init(opts, 1, _err);
+ret = net_client_init(opts, true, _err);
 if (local_err) {
 error_report_err(local_err);
 return -1;
-- 
2.5.5

[Qemu-devel] [PATCH v9 06/17] qapi: Drop useless gen_err_check()

[Eric Blake]

Ever since commit 12f254f removed the last parameterization
of gen_err_check(), it no longer makes sense to hide the three
lines of generated C code behind a macro call. Just inline it
into the remaining users.

No change to generated code.

Signed-off-by: Eric Blake 

---
v9: new patch
v8: new patch
---
 scripts/qapi.py  | 8 
 scripts/qapi-commands.py | 4 +++-
 scripts/qapi-visit.py| 8 ++--
 3 files changed, 9 insertions(+), 11 deletions(-)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 27284be..9c48f6d 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1656,14 +1656,6 @@ def gen_params(arg_type, extra):
 return ret


-def gen_err_check():
-return mcgen('''
-if (err) {
-goto out;
-}
-''')
-
-
 #
 # Common command line parsing
 #
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index c93470c..333a46f 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -46,8 +46,10 @@ def gen_call(name, arg_type, ret_type):
 ''',
 c_name=c_name(name), args=argstr, lhs=lhs)
 if ret_type:
-ret += gen_err_check()
 ret += mcgen('''
+if (err) {
+goto out;
+}

 qmp_marshal_output_%(c_name)s(retval, ret, );
 ''',
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 0b9e298..96f2491 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -47,9 +47,11 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s 
*obj, Error **errp)
 if base:
 ret += mcgen('''
 visit_type_%(c_type)s_members(v, (%(c_type)s *)obj, );
+if (err) {
+goto out;
+}
 ''',
  c_type=base.c_name())
-ret += gen_err_check()

 for memb in members:
 if memb.optional:
@@ -60,10 +62,12 @@ void visit_type_%(c_name)s_members(Visitor *v, %(c_name)s 
*obj, Error **errp)
 push_indent()
 ret += mcgen('''
 visit_type_%(c_type)s(v, "%(name)s", >%(c_name)s, );
+if (err) {
+goto out;
+}
 ''',
  c_type=memb.type.c_name(), name=memb.name,
  c_name=c_name(memb.name))
-ret += gen_err_check()
 if memb.optional:
 pop_indent()
 ret += mcgen('''
-- 
2.5.5

[Qemu-devel] [PATCH v9 08/17] qapi: Plumb in 'boxed' to qapi generator lower levels

[Eric Blake]

The next patch will add support for passing a qapi union type
as the 'data' of a command.  But to do that, the user function
for implementing the command, as called by the generated
marshal command, must take the corresponding C struct as a
single boxed pointer, rather than a breakdown into one
parameter per member.  Even without a union, being able to use
a C struct rather than a list of parameters can make it much
easier to handle coding with QAPI.

This patch adds the internal plumbing of a 'boxed' flag
associated with each command and event.  In several cases,
this means adding indentation, with one new dead branch and
the remaining branch being the original code more deeply
nested; this was done so that the new implementation in the
next patch is easier to review without also being mixed with
indentation changes.

For this patch, no behavior or generated output changes, other
than the testsuite outputting the value of the new flag
(always False for now).

Signed-off-by: Eric Blake 

---
v9: change title, s/box/boxed/
v8: improve commit message, defer some implementation, rebase without
gen_err_check()
v7: rebase to master
v6: rebase to earlier changes
---
 scripts/qapi.py | 43 +
 scripts/qapi-commands.py| 20 ---
 scripts/qapi-event.py   | 18 +++---
 scripts/qapi-introspect.py  |  4 +--
 tests/qapi-schema/event-case.out|  1 +
 tests/qapi-schema/ident-with-escape.out |  2 +-
 tests/qapi-schema/indented-expr.out |  4 +--
 tests/qapi-schema/qapi-schema-test.out  | 19 +--
 tests/qapi-schema/test-qapi.py  |  8 +++---
 9 files changed, 70 insertions(+), 49 deletions(-)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 9c48f6d..0c6159c 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -826,10 +826,10 @@ class QAPISchemaVisitor(object):
 pass

 def visit_command(self, name, info, arg_type, ret_type,
-  gen, success_response):
+  gen, success_response, boxed):
 pass

-def visit_event(self, name, info, arg_type):
+def visit_event(self, name, info, arg_type, boxed):
 pass


@@ -1165,7 +1165,8 @@ class QAPISchemaAlternateType(QAPISchemaType):


 class QAPISchemaCommand(QAPISchemaEntity):
-def __init__(self, name, info, arg_type, ret_type, gen, success_response):
+def __init__(self, name, info, arg_type, ret_type, gen, success_response,
+ boxed):
 QAPISchemaEntity.__init__(self, name, info)
 assert not arg_type or isinstance(arg_type, str)
 assert not ret_type or isinstance(ret_type, str)
@@ -1175,12 +1176,14 @@ class QAPISchemaCommand(QAPISchemaEntity):
 self.ret_type = None
 self.gen = gen
 self.success_response = success_response
+self.boxed = boxed

 def check(self, schema):
 if self._arg_type_name:
 self.arg_type = schema.lookup_type(self._arg_type_name)
 assert isinstance(self.arg_type, QAPISchemaObjectType)
 assert not self.arg_type.variants   # not implemented
+assert not self.boxed   # not implemented
 if self._ret_type_name:
 self.ret_type = schema.lookup_type(self._ret_type_name)
 assert isinstance(self.ret_type, QAPISchemaType)
@@ -1188,24 +1191,26 @@ class QAPISchemaCommand(QAPISchemaEntity):
 def visit(self, visitor):
 visitor.visit_command(self.name, self.info,
   self.arg_type, self.ret_type,
-  self.gen, self.success_response)
+  self.gen, self.success_response, self.boxed)


 class QAPISchemaEvent(QAPISchemaEntity):
-def __init__(self, name, info, arg_type):
+def __init__(self, name, info, arg_type, boxed):
 QAPISchemaEntity.__init__(self, name, info)
 assert not arg_type or isinstance(arg_type, str)
 self._arg_type_name = arg_type
 self.arg_type = None
+self.boxed = boxed

 def check(self, schema):
 if self._arg_type_name:
 self.arg_type = schema.lookup_type(self._arg_type_name)
 assert isinstance(self.arg_type, QAPISchemaObjectType)
 assert not self.arg_type.variants   # not implemented
+assert not self.boxed   # not implemented

 def visit(self, visitor):
-visitor.visit_event(self.name, self.info, self.arg_type)
+visitor.visit_event(self.name, self.info, self.arg_type, self.boxed)


 class QAPISchema(object):
@@ -1381,6 +1386,7 @@ class QAPISchema(object):
 rets = expr.get('returns')
 gen = expr.get('gen', True)
 success_response = expr.get('success-response', True)
+boxed = expr.get('boxed', False)
 if isinstance(data, OrderedDict):
 data = self._make_implicit_object_type(
   

[Qemu-devel] [PATCH v9 10/17] block: Simplify block_set_io_throttle

[Eric Blake]

Now that we can support boxed commands, use it to greatly
reduce the number of parameters (and likelihood of getting
out of sync) when adjusting throttle parameters.

Signed-off-by: Eric Blake 
Reviewed-by: Alberto Garcia 

---
v9: s/box/boxed/, trivial enough to keep R-b
v8: tweak doc wording
v7: new patch
---
 qapi/block-core.json |  20 --
 blockdev.c   | 111 +++
 hmp.c|  45 +
 3 files changed, 66 insertions(+), 110 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index ac8f5f6..cbf4410 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1312,6 +1312,21 @@
 # the device will be removed from its group and the rest of its
 # members will not be affected. The 'group' parameter is ignored.
 #
+# See BlockIOThrottle for parameter descriptions.
+#
+# Returns: Nothing on success
+#  If @device is not a valid block device, DeviceNotFound
+#
+# Since: 1.1
+##
+{ 'command': 'block_set_io_throttle', 'boxed': true,
+  'data': 'BlockIOThrottle' }
+
+##
+# BlockIOThrottle
+#
+# A set of parameters describing block throttling.
+#
 # @device: The name of the device
 #
 # @bps: total throughput limit in bytes per second
@@ -1378,12 +1393,9 @@
 #
 # @group: #optional throttle group name (Since 2.4)
 #
-# Returns: Nothing on success
-#  If @device is not a valid block device, DeviceNotFound
-#
 # Since: 1.1
 ##
-{ 'command': 'block_set_io_throttle',
+{ 'struct': 'BlockIOThrottle',
   'data': { 'device': 'str', 'bps': 'int', 'bps_rd': 'int', 'bps_wr': 'int',
 'iops': 'int', 'iops_rd': 'int', 'iops_wr': 'int',
 '*bps_max': 'int', '*bps_rd_max': 'int',
diff --git a/blockdev.c b/blockdev.c
index 0f8065c..ddf30e1 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2631,49 +2631,17 @@ fail:
 }

 /* throttling disk I/O limits */
-void qmp_block_set_io_throttle(const char *device, int64_t bps, int64_t bps_rd,
-   int64_t bps_wr,
-   int64_t iops,
-   int64_t iops_rd,
-   int64_t iops_wr,
-   bool has_bps_max,
-   int64_t bps_max,
-   bool has_bps_rd_max,
-   int64_t bps_rd_max,
-   bool has_bps_wr_max,
-   int64_t bps_wr_max,
-   bool has_iops_max,
-   int64_t iops_max,
-   bool has_iops_rd_max,
-   int64_t iops_rd_max,
-   bool has_iops_wr_max,
-   int64_t iops_wr_max,
-   bool has_bps_max_length,
-   int64_t bps_max_length,
-   bool has_bps_rd_max_length,
-   int64_t bps_rd_max_length,
-   bool has_bps_wr_max_length,
-   int64_t bps_wr_max_length,
-   bool has_iops_max_length,
-   int64_t iops_max_length,
-   bool has_iops_rd_max_length,
-   int64_t iops_rd_max_length,
-   bool has_iops_wr_max_length,
-   int64_t iops_wr_max_length,
-   bool has_iops_size,
-   int64_t iops_size,
-   bool has_group,
-   const char *group, Error **errp)
+void qmp_block_set_io_throttle(BlockIOThrottle *arg, Error **errp)
 {
 ThrottleConfig cfg;
 BlockDriverState *bs;
 BlockBackend *blk;
 AioContext *aio_context;

-blk = blk_by_name(device);
+blk = blk_by_name(arg->device);
 if (!blk) {
 error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
-  "Device '%s' not found", device);
+  "Device '%s' not found", arg->device);
 return;
 }

@@ -2682,59 +2650,59 @@ void qmp_block_set_io_throttle(const char *device, 
int64_t bps, int64_t bps_rd,

 bs = blk_bs(blk);
 if (!bs) {
-error_setg(errp, "Device '%s' has no medium", device);
+error_setg(errp, "Device '%s' has no medium", arg->device);
 goto out;
 }

 throttle_config_init();
-cfg.buckets[THROTTLE_BPS_TOTAL].avg = bps;
-cfg.buckets[THROTTLE_BPS_READ].avg  = bps_rd;
-cfg.buckets[THROTTLE_BPS_WRITE].avg = bps_wr;
+cfg.buckets[THROTTLE_BPS_TOTAL].avg = arg->bps;
+cfg.buckets[THROTTLE_BPS_READ].avg  = arg->bps_rd;
+cfg.buckets[THROTTLE_BPS_WRITE].avg = arg->bps_wr;

-cfg.buckets[THROTTLE_OPS_TOTAL].avg = iops;
-cfg.buckets[THROTTLE_OPS_READ].avg  = iops_rd;
-

[Qemu-devel] [PATCH v9 14/17] net: Complete qapi-fication of netdev_add

[Eric Blake]

We finally have all the required pieces for doing a type-safe
representation of netdev_add as a flat union, where the
discriminator 'type' now selects which additional members may
appear in the "arguments" JSON object sent over QMP, and exposes
those types through introspection, and without breaking command
line parsing.

Inline the function netdev_add() into its lone remaining caller.

There are a few places where the QMP 'netdev_add' command is now
more strict: anywhere that the QAPI lists an integer member, we
now require a strict JSON integer (previously, we allowed both
integers and strings, because the conversion from QMP to QemuOpts
back to QObject collapsed them into integers).  For example,
pre-patch, both of these examples succeed, but post-patch, the
second example fails:

{'execute':'netdev_add',
  'arguments':{'id':'net1', 'type':'hubport', 'hubid':1}}
{"return": {}}
{'execute':'netdev_add',
  'arguments':{'id':'net2', 'type':'hubport', 'hubid':"2"}}
{"error": {"class": "GenericError", "desc": "Invalid parameter type for 
'hubid', expected: integer"}}

A later patch will resolve the problem by adding a loose parsing mode
to qmp-input-visitor (teaching it to accept a string encoding of an
integer in place of an actual integer).

In qmp_netdev_add(), we still have to create a QemuOpts object
so that qmp_netdev_del() will be able to remove a hotplugged
network device; but the opts->head remains empty since we now
manage all parsing through the QAPI object rather than QemuOpts.

Signed-off-by: Eric Blake 

---
v9: s/box/boxed/, tweak commit message
v8: improve commit message, call out a change in QMP behavior,
inline netdev_add()
v7: no change
v6: don't lose qemu_opts handling

If you like the approach of the later patches, we can rearrange
things in the series to avoid the temporary regression
---
 qapi-schema.json  | 15 +++
 include/net/net.h |  2 --
 hmp.c |  2 +-
 net/net.c | 11 +++
 qmp-commands.hx   |  2 +-
 5 files changed, 8 insertions(+), 24 deletions(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index 5658723..7ec01e1 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2362,26 +2362,17 @@
 #
 # Add a network backend.
 #
-# @type: the type of network backend.  Current valid values are 'user', 'tap',
-#'vde', 'socket', 'dump' and 'bridge'
+# @type: the type of network backend; determines which additional arguments
+#are valid. See Netdev documentation for more details.
 #
 # @id: the name of the new network backend
 #
-# Additional arguments depend on the type.
-#
-# TODO This command effectively bypasses QAPI completely due to its
-# "additional arguments" business.  It shouldn't have been added to
-# the schema in this form.  It should be qapified properly, or
-# replaced by a properly qapified command.
-#
 # Since: 0.14.0
 #
 # Returns: Nothing on success
 #  If @type is not a valid network backend, DeviceNotFound
 ##
-{ 'command': 'netdev_add',
-  'data': {'type': 'str', 'id': 'str'},
-  'gen': false }# so we can get the additional arguments
+{ 'command': 'netdev_add', 'data': 'Netdev', 'boxed': true }

 ##
 # @netdev_del:
diff --git a/include/net/net.h b/include/net/net.h
index e8d9e9e..820f880 100644
--- a/include/net/net.h
+++ b/include/net/net.h
@@ -210,8 +210,6 @@ void net_check_clients(void);
 void net_cleanup(void);
 void hmp_host_net_add(Monitor *mon, const QDict *qdict);
 void hmp_host_net_remove(Monitor *mon, const QDict *qdict);
-void netdev_add(QemuOpts *opts, Error **errp);
-void qmp_netdev_add(QDict *qdict, QObject **ret, Error **errp);

 int net_hub_id_for_client(NetClientState *nc, int *id);
 NetClientState *net_hub_port_find(int hub_id);
diff --git a/hmp.c b/hmp.c
index 4819abc..3502abf 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1672,7 +1672,7 @@ void hmp_netdev_add(Monitor *mon, const QDict *qdict)
 goto out;
 }

-netdev_add(opts, );
+net_client_init(opts, true, );
 if (err) {
 qemu_opts_del(opts);
 }
diff --git a/net/net.c b/net/net.c
index c124b11..1bd7936 100644
--- a/net/net.c
+++ b/net/net.c
@@ -1181,12 +1181,7 @@ void hmp_host_net_remove(Monitor *mon, const QDict 
*qdict)
 qemu_del_net_client(nc);
 }

-void netdev_add(QemuOpts *opts, Error **errp)
-{
-net_client_init(opts, true, errp);
-}
-
-void qmp_netdev_add(QDict *qdict, QObject **ret, Error **errp)
+void qmp_netdev_add(Netdev *netdev, Error **errp)
 {
 Error *local_err = NULL;
 QemuOptsList *opts_list;
@@ -1197,12 +1192,12 @@ void qmp_netdev_add(QDict *qdict, QObject **ret, Error 
**errp)
 goto out;
 }

-opts = qemu_opts_from_qdict(opts_list, qdict, _err);
+opts = qemu_opts_create(opts_list, netdev->id, 1, _err);
 if (local_err) {
 goto out;
 }

-netdev_add(opts, _err);
+net_client_init1(netdev, true, _err);
 if (local_err) {
 qemu_opts_del(opts);
 goto out;
diff --git a/qmp-commands.hx 

[Qemu-devel] [PATCH v9 09/17] qapi: Implement boxed types for commands/events

[Eric Blake]

Turn on the ability to pass command and event arguments in
a single boxed parameter, which must name a non-empty type
(although the type can be a struct with all optional members).
For structs, it makes it possible to pass a single qapi type
instead of a breakout of all struct members (useful if the
arguments are already in a struct or if the number of members
is large); for other complex types, it is now possible to use
a union or alternate as the data for a command or event.

The empty type may be technically feasible if needed down the
road, but it's easier to forbid it now and relax things to allow
it later, than it is to allow it now and have to special case
how the generated 'q_empty' type is handled (see commit 7ce106a9
for reasons why nothing is generated for the empty type).  An
alternate type is never considered empty, but now that a boxed
type can be either an object or an alternate, we have to provide
a trivial QAPISchemaAlternateType.is_empty().  The new call to
arg_type.is_empty() during QAPISchemaCommand.check() requires
that we first check the type in question; but there is no chance
of introducing a cycle since objects do not refer back to commands.

We still have a split in syntax checking between ad-hoc parsing
up front (merely validates that 'boxed' has a sane value) and
during .check() methods (if 'boxed' is set, then 'data' must name
a non-empty user-defined type).

Generated code is unchanged, as long as no client uses the
new feature.

Signed-off-by: Eric Blake 

---
v9: s/box/boxed/, formatting tweak, commit message improvements,
restore assertions rather than relying on implicit checking
v8: forbid empty type, allow alternates, improve docs
v7: rebase to latest
v6: retitle, rebase, and merge v5 40/46 and 41/46 into one patch
---
 scripts/qapi.py | 63 ++---
 scripts/qapi-commands.py|  3 +-
 scripts/qapi-event.py   |  5 ++-
 tests/test-qmp-commands.c   |  8 +
 docs/qapi-code-gen.txt  | 27 --
 tests/Makefile.include  |  5 +++
 tests/qapi-schema/args-bad-box.err  |  1 +
 tests/qapi-schema/args-bad-box.exit |  1 +
 tests/qapi-schema/args-bad-box.json |  2 ++
 tests/qapi-schema/args-bad-box.out  |  0
 tests/qapi-schema/args-box-anon.err |  1 +
 tests/qapi-schema/args-box-anon.exit|  1 +
 tests/qapi-schema/args-box-anon.json|  2 ++
 tests/qapi-schema/args-box-anon.out |  0
 tests/qapi-schema/args-box-empty.err|  1 +
 tests/qapi-schema/args-box-empty.exit   |  1 +
 tests/qapi-schema/args-box-empty.json   |  3 ++
 tests/qapi-schema/args-box-empty.out|  0
 tests/qapi-schema/args-box-string.err   |  1 +
 tests/qapi-schema/args-box-string.exit  |  1 +
 tests/qapi-schema/args-box-string.json  |  2 ++
 tests/qapi-schema/args-box-string.out   |  0
 tests/qapi-schema/args-union.err|  2 +-
 tests/qapi-schema/args-union.json   |  3 +-
 tests/qapi-schema/event-box-empty.err   |  1 +
 tests/qapi-schema/event-box-empty.exit  |  1 +
 tests/qapi-schema/event-box-empty.json  |  2 ++
 tests/qapi-schema/event-box-empty.out   |  0
 tests/qapi-schema/qapi-schema-test.json |  4 +++
 tests/qapi-schema/qapi-schema-test.out  |  8 +
 30 files changed, 129 insertions(+), 20 deletions(-)
 create mode 100644 tests/qapi-schema/args-bad-box.err
 create mode 100644 tests/qapi-schema/args-bad-box.exit
 create mode 100644 tests/qapi-schema/args-bad-box.json
 create mode 100644 tests/qapi-schema/args-bad-box.out
 create mode 100644 tests/qapi-schema/args-box-anon.err
 create mode 100644 tests/qapi-schema/args-box-anon.exit
 create mode 100644 tests/qapi-schema/args-box-anon.json
 create mode 100644 tests/qapi-schema/args-box-anon.out
 create mode 100644 tests/qapi-schema/args-box-empty.err
 create mode 100644 tests/qapi-schema/args-box-empty.exit
 create mode 100644 tests/qapi-schema/args-box-empty.json
 create mode 100644 tests/qapi-schema/args-box-empty.out
 create mode 100644 tests/qapi-schema/args-box-string.err
 create mode 100644 tests/qapi-schema/args-box-string.exit
 create mode 100644 tests/qapi-schema/args-box-string.json
 create mode 100644 tests/qapi-schema/args-box-string.out
 create mode 100644 tests/qapi-schema/event-box-empty.err
 create mode 100644 tests/qapi-schema/event-box-empty.exit
 create mode 100644 tests/qapi-schema/event-box-empty.json
 create mode 100644 tests/qapi-schema/event-box-empty.out

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 0c6159c..21bc32f 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -522,10 +522,14 @@ def check_type(expr_info, source, value, 
allow_array=False,

 def check_command(expr, expr_info):
 name = expr['command']
+boxed = expr.get('boxed', False)

+args_meta = ['struct']
+if boxed:
+args_meta += ['union', 'alternate']
 check_type(expr_info, "'data' for command '%s'" % name,
-   expr.get('data'), 

[Qemu-devel] [PATCH v9 07/17] qapi-event: Simplify visit of non-implicit data

[Eric Blake]

Commit 7ce106a9 documented why we don't generated a visit_type_FOO()
for implicit types; and therefore events with an anonymous type for
'data' have to open-code a visit.  Note that the open-coded visit in
qapi-event.c is slightly different from what is done in
qapi-visit.c for normal types, in part because we don't have to
check for *obj being NULL or free things on error.  But where the
type is not implicit, it is nicer to reuse the normal visit instead
of open-coding a duplicate.

At the moment, the only event with a non-implicit 'data' is in the
testsuite, where test-qapi-event.c changes as follows:

|@@ -155,6 +155,7 @@ void qapi_event_send___org_qemu_x_event(
| __org_qemu_x_Struct param = {
| __org_qemu_x_member1, (char *)__org_qemu_x_member2, has_q_wchar_t, 
q_wchar_t
| };
|+__org_qemu_x_Struct *arg = 
|
| emit = qmp_event_get_func_emit();
| if (!emit) {
|@@ -164,16 +165,7 @@ void qapi_event_send___org_qemu_x_event(
| qmp = qmp_event_build_dict("__ORG.QEMU_X-EVENT");
|
| v = qmp_output_visitor_new();
|-
|-visit_start_struct(v, "__ORG.QEMU_X-EVENT", NULL, 0, );
|-if (err) {
|-goto out;
|-}
|-visit_type___org_qemu_x_Struct_members(v, , );
|-if (!err) {
|-if (!err) {
|-visit_check_struct(v, );
|-}
|-visit_end_struct(v, NULL);
|+visit_type___org_qemu_x_Struct(v, "__ORG.QEMU_X-EVENT", , );
| if (err) {
| goto out;
| }

Signed-off-by: Eric Blake 

---
v9: fix pep8 warning
v8: new patch
---
 scripts/qapi-event.py | 19 +--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 09c0a2a..b9c6b6e 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -49,6 +49,11 @@ def gen_param_var(typ):

 };
 ''')
+if not typ.is_implicit():
+ret += mcgen('''
+%(c_name)s *arg = 
+''',
+ c_name=typ.c_name())
 return ret


@@ -91,6 +96,14 @@ def gen_event_send(name, arg_type):
 if arg_type and not arg_type.is_empty():
 ret += mcgen('''
 v = qmp_output_visitor_new();
+''')
+if not arg_type.is_implicit():
+ret += mcgen('''
+visit_type_%(c_name)s(v, "%(name)s", , );
+''',
+ name=name, c_name=arg_type.c_name())
+else:
+ret += mcgen('''

 visit_start_struct(v, "%(name)s", NULL, 0, );
 if (err) {
@@ -101,14 +114,16 @@ def gen_event_send(name, arg_type):
 visit_check_struct(v, );
 }
 visit_end_struct(v, NULL);
+''',
+ name=name, c_name=arg_type.c_name())
+ret += mcgen('''
 if (err) {
 goto out;
 }

 visit_complete(v, );
 qdict_put_obj(qmp, "data", obj);
-''',
- name=name, c_name=arg_type.c_name())
+''')

 ret += mcgen('''
 emit(%(c_enum)s, qmp, );
-- 
2.5.5

[Qemu-devel] [PATCH v9 04/17] qapi: Hide tag_name data member of variants

[Eric Blake]

Clean up the only remaining external use of the tag_name field of
QAPISchemaObjectTypeVariants, by explicitly listing the generated
'type' tag for all variants in the testsuite (you can still tell
simple unions by the -wrapper types).  Then we can mark the
tag_name field as private by adding a leading underscore to prevent
any further use.

Signed-off-by: Eric Blake 

---
v9: no change
v8: tweak commit wording
v7: no change
v6: rebase to earlier changes
---
 scripts/qapi.py|  8 
 tests/qapi-schema/qapi-schema-test.out | 10 ++
 tests/qapi-schema/test-qapi.py |  3 +--
 3 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 68ee319..400c4ef 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1093,7 +1093,7 @@ class QAPISchemaObjectTypeVariants(object):
 assert len(variants) > 0
 for v in variants:
 assert isinstance(v, QAPISchemaObjectTypeVariant)
-self.tag_name = tag_name
+self._tag_name = tag_name
 self.tag_member = tag_member
 self.variants = variants

@@ -1103,8 +1103,8 @@ class QAPISchemaObjectTypeVariants(object):

 def check(self, schema, seen):
 if not self.tag_member:# flat union
-self.tag_member = seen[c_name(self.tag_name)]
-assert self.tag_name == self.tag_member.name
+self.tag_member = seen[c_name(self._tag_name)]
+assert self._tag_name == self.tag_member.name
 assert isinstance(self.tag_member.type, QAPISchemaEnumType)
 for v in self.variants:
 v.check(schema)
@@ -1134,7 +1134,7 @@ class QAPISchemaAlternateType(QAPISchemaType):
 def __init__(self, name, info, variants):
 QAPISchemaType.__init__(self, name, info)
 assert isinstance(variants, QAPISchemaObjectTypeVariants)
-assert not variants.tag_name
+assert variants.tag_member
 variants.set_owner(name)
 variants.tag_member.set_owner(self.name)
 self.variants = variants
diff --git a/tests/qapi-schema/qapi-schema-test.out 
b/tests/qapi-schema/qapi-schema-test.out
index 19cd214..f34ecc7 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -1,19 +1,25 @@
 alternate AltIntNum
+tag type
 case i: int
 case n: number
 alternate AltNumInt
+tag type
 case n: number
 case i: int
 alternate AltNumStr
+tag type
 case n: number
 case s: str
 alternate AltStrBool
+tag type
 case s: str
 case b: bool
 alternate AltStrInt
+tag type
 case s: str
 case i: int
 alternate AltStrNum
+tag type
 case s: str
 case n: number
 event EVENT_A None
@@ -50,6 +56,7 @@ object UserDefA
 member boolean: bool optional=False
 member a_b: int optional=True
 alternate UserDefAlternate
+tag type
 case udfu: UserDefFlatUnion
 case s: str
 case i: int
@@ -72,6 +79,7 @@ object UserDefFlatUnion2
 case value2: UserDefB
 object UserDefNativeListUnion
 member type: UserDefNativeListUnionKind optional=False
+tag type
 case integer: q_obj_intList-wrapper
 case s8: q_obj_int8List-wrapper
 case s16: q_obj_int16List-wrapper
@@ -117,6 +125,7 @@ object WrapAlternate
 member alt: UserDefAlternate optional=False
 event __ORG.QEMU_X-EVENT __org.qemu_x-Struct
 alternate __org.qemu_x-Alt
+tag type
 case __org.qemu_x-branch: str
 case b: __org.qemu_x-Base
 object __org.qemu_x-Base
@@ -130,6 +139,7 @@ object __org.qemu_x-Struct2
 member array: __org.qemu_x-Union1List optional=False
 object __org.qemu_x-Union1
 member type: __org.qemu_x-Union1Kind optional=False
+tag type
 case __org.qemu_x-branch: q_obj_str-wrapper
 enum __org.qemu_x-Union1Kind ['__org.qemu_x-branch']
 object __org.qemu_x-Union2
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index 649677e..bedd145 100644
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -47,8 +47,7 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
 @staticmethod
 def _print_variants(variants):
 if variants:
-if variants.tag_name:
-print 'tag %s' % variants.tag_name
+print 'tag %s' % variants.tag_member.name
 for v in variants.variants:
 print 'case %s: %s' % (v.name, v.type.name)

-- 
2.5.5

[Qemu-devel] [PATCH for-2.7 v9 00/17] qapi netdev_add introspection (post-introspection cleanups subset F)

[Eric Blake]

It's time to expose QMP 'netdev_add' through introspection, and
to add boxed commands/events so that we can drastically reduce
the number of C parameters needed to implement a command that
matches a large QAPI type.

Prerequistes:
Markus' qapi-next branch (weak; series also applies on master)

available as a tag at:
git fetch git://repo.or.cz/qemu/ericb.git qapi-cleanupv9f
or as part of my qapi branch:
git fetch git://repo.or.cz/qemu/ericb.git qapi

v8 was here:
https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg00302.html

Since then, I've addressed Markus' comments:
- s/box/boxed/
- fix a useless Python override of c_name()
- comment and formatting tweaks
- defer event collision prevention until later; for 2.7, we are
focusing on the bare minimum needed to take advantage of boxing
- implement a promised followup that allows netdev_add to still
parse strings the way it did with QemuOpts (thanks Dan for starting
that effort)

The diffstat from v8 looks big, but a lot of it is mechanical
due to the spelling change.  It's late at night, and we're close
to hard freeze, so to maximize review time I'm posting now; if I
weren't feeling quite as rushed, I might would have swapped the
order of 14 vs. 15-17 to avoid even a temporary behavior change
in netdev_add.  On the other hand, 14 has had review, while 15-17
are effectively new, and it is feasible that we may be comfortable
enough with the new type-safety constraints of 14 to not want
to bother with 15-17 loosening things back up, given that we are
already in soft freeze.

001/17:[] [--] 'net: use Netdev instead of NetClientOptions in client init'
002/17:[] [--] 'qapi: Require all branches of flat union enum to be covered'
003/17:[down] 'qapi: Special case c_name() for empty type'
004/17:[] [--] 'qapi: Hide tag_name data member of variants'
005/17:[] [-C] 'qapi: Add type.is_empty() helper'
006/17:[] [--] 'qapi: Drop useless gen_err_check()'
007/17:[0002] [FC] 'qapi-event: Simplify visit of non-implicit data'
008/17:[down] 'qapi: Plumb in 'boxed' to qapi generator lower levels'
009/17:[0100] [FC] 'qapi: Implement boxed types for commands/events'
010/17:[0002] [FC] 'block: Simplify block_set_io_throttle'
011/17:[0002] [FC] 'block: Simplify drive-mirror'
012/17:[] [--] 'qapi: Change Netdev into a flat union'
013/17:[] [--] 'net: Use correct type for bool flag'
014/17:[0002] [FC] 'net: Complete qapi-fication of netdev_add'
015/17:[down] 'option: make parse_option_bool/number non-static'
016/17:[down] 'qapi: Tweak QmpInputVisitor to optionally do string conversion'
017/17:[down] 'qapi: Restore autocast behavior in 'netdev_add''

Daniel P. Berrange (1):
  option: make parse_option_bool/number non-static

Eric Blake (15):
  qapi: Require all branches of flat union enum to be covered
  qapi: Special case c_name() for empty type
  qapi: Hide tag_name data member of variants
  qapi: Add type.is_empty() helper
  qapi: Drop useless gen_err_check()
  qapi-event: Simplify visit of non-implicit data
  qapi: Plumb in 'boxed' to qapi generator lower levels
  qapi: Implement boxed types for commands/events
  block: Simplify block_set_io_throttle
  block: Simplify drive-mirror
  qapi: Change Netdev into a flat union
  net: Use correct type for bool flag
  net: Complete qapi-fication of netdev_add
  qapi: Tweak QmpInputVisitor to optionally do string conversion
  qapi: Restore autocast behavior in 'netdev_add'

Kővágó, Zoltán (1):
  net: use Netdev instead of NetClientOptions in client init

 qapi-schema.json   |  66 +---
 qapi/block-core.json   |  40 -
 scripts/qapi.py| 131 +++
 scripts/qapi-commands.py   |  36 ++--
 scripts/qapi-event.py  |  48 --
 scripts/qapi-introspect.py |   4 +-
 scripts/qapi-types.py  |   2 +-
 scripts/qapi-visit.py  |   8 +-
 include/net/net.h  |   8 +-
 include/qapi/qmp-input-visitor.h   |  24 ++-
 include/qemu/option.h  |   4 +
 net/clients.h  |  20 +--
 blockdev.c | 187 -
 hmp.c  |  72 +++-
 hw/arm/musicpal.c  |   2 +-
 hw/core/qdev-properties-system.c   |   2 +-
 hw/net/allwinner_emac.c|   2 +-
 hw/net/cadence_gem.c   |   2 +-
 hw/net/dp8393x.c   |   2 +-
 hw/net/e1000.c |   2 +-
 hw/net/e1000e.c|   2 +-
 hw/net/eepro100.c  |   2 +-
 hw/net/etraxfs_eth.c   |   2 +-
 hw/net/fsl_etsec/etsec.c 

[Qemu-devel] [PATCH v9 01/17] net: use Netdev instead of NetClientOptions in client init

[Eric Blake]

From: Kővágó, Zoltán 

This way we no longer need NetClientOptions and can convert Netdev
into a flat union.

Signed-off-by: Kővágó, Zoltán 
Reviewed-by: Eric Blake 
Message-Id: 
<93ffdfed7054529635e6acb935150d95dc173a12.1441627176.git.dirty.ice...@gmail.com>

[rework net_client_init1() to pass Netdev by copying from NetdevLegacy,
rather than merging the two types - which means that we still need
NetClientOptions after all.  Rebase to qapi changes. The bulk of the
patch is mechanical, replacing 'opts' by 'netdev->opts', while
net_client_init1() takes care of converting between legacy and modern
types.]

Signed-off-by: Eric Blake 

---
v9: no change
v8: tweak commit message
v7: rebase to master
v6: rebase
---
 net/clients.h| 20 ++--
 net/dump.c   |  6 +++---
 net/hub.c|  6 +++---
 net/l2tpv3.c |  6 +++---
 net/net.c| 18 +++---
 net/netmap.c |  4 ++--
 net/slirp.c  |  6 +++---
 net/socket.c |  6 +++---
 net/tap-win32.c  |  6 +++---
 net/tap.c| 12 ++--
 net/vde.c|  6 +++---
 net/vhost-user.c |  6 +++---
 12 files changed, 53 insertions(+), 49 deletions(-)

diff --git a/net/clients.h b/net/clients.h
index d47530e..5cae479 100644
--- a/net/clients.h
+++ b/net/clients.h
@@ -27,39 +27,39 @@
 #include "net/net.h"
 #include "qapi-types.h"

-int net_init_dump(const NetClientOptions *opts, const char *name,
+int net_init_dump(const Netdev *netdev, const char *name,
   NetClientState *peer, Error **errp);

 #ifdef CONFIG_SLIRP
-int net_init_slirp(const NetClientOptions *opts, const char *name,
+int net_init_slirp(const Netdev *netdev, const char *name,
NetClientState *peer, Error **errp);
 #endif

-int net_init_hubport(const NetClientOptions *opts, const char *name,
+int net_init_hubport(const Netdev *netdev, const char *name,
  NetClientState *peer, Error **errp);

-int net_init_socket(const NetClientOptions *opts, const char *name,
+int net_init_socket(const Netdev *netdev, const char *name,
 NetClientState *peer, Error **errp);

-int net_init_tap(const NetClientOptions *opts, const char *name,
+int net_init_tap(const Netdev *netdev, const char *name,
  NetClientState *peer, Error **errp);

-int net_init_bridge(const NetClientOptions *opts, const char *name,
+int net_init_bridge(const Netdev *netdev, const char *name,
 NetClientState *peer, Error **errp);

-int net_init_l2tpv3(const NetClientOptions *opts, const char *name,
+int net_init_l2tpv3(const Netdev *netdev, const char *name,
 NetClientState *peer, Error **errp);
 #ifdef CONFIG_VDE
-int net_init_vde(const NetClientOptions *opts, const char *name,
+int net_init_vde(const Netdev *netdev, const char *name,
  NetClientState *peer, Error **errp);
 #endif

 #ifdef CONFIG_NETMAP
-int net_init_netmap(const NetClientOptions *opts, const char *name,
+int net_init_netmap(const Netdev *netdev, const char *name,
 NetClientState *peer, Error **errp);
 #endif

-int net_init_vhost_user(const NetClientOptions *opts, const char *name,
+int net_init_vhost_user(const Netdev *netdev, const char *name,
 NetClientState *peer, Error **errp);

 #endif /* QEMU_NET_CLIENTS_H */
diff --git a/net/dump.c b/net/dump.c
index 41f7673..f8a500f 100644
--- a/net/dump.c
+++ b/net/dump.c
@@ -179,7 +179,7 @@ static NetClientInfo net_dump_info = {
 .cleanup = dumpclient_cleanup,
 };

-int net_init_dump(const NetClientOptions *opts, const char *name,
+int net_init_dump(const Netdev *netdev, const char *name,
   NetClientState *peer, Error **errp)
 {
 int len, rc;
@@ -189,8 +189,8 @@ int net_init_dump(const NetClientOptions *opts, const char 
*name,
 NetClientState *nc;
 DumpNetClient *dnc;

-assert(opts->type == NET_CLIENT_OPTIONS_KIND_DUMP);
-dump = opts->u.dump.data;
+assert(netdev->opts->type == NET_CLIENT_OPTIONS_KIND_DUMP);
+dump = netdev->opts->u.dump.data;

 assert(peer);

diff --git a/net/hub.c b/net/hub.c
index 6d90c6e..ec4626f 100644
--- a/net/hub.c
+++ b/net/hub.c
@@ -281,14 +281,14 @@ int net_hub_id_for_client(NetClientState *nc, int *id)
 return 0;
 }

-int net_init_hubport(const NetClientOptions *opts, const char *name,
+int net_init_hubport(const Netdev *netdev, const char *name,
  NetClientState *peer, Error **errp)
 {
 const NetdevHubPortOptions *hubport;

-assert(opts->type == NET_CLIENT_OPTIONS_KIND_HUBPORT);
+assert(netdev->opts->type == NET_CLIENT_OPTIONS_KIND_HUBPORT);
 assert(!peer);
-hubport = opts->u.hubport.data;
+hubport = netdev->opts->u.hubport.data;

 net_hub_add_port(hubport->hubid, name);
 return 0;
diff --git a/net/l2tpv3.c b/net/l2tpv3.c
index 5c668f7..df02f5b 100644
--- a/net/l2tpv3.c
+++ 

[Qemu-devel] [PATCH v9 05/17] qapi: Add type.is_empty() helper

[Eric Blake]

In the near future, we want to lift our artificial restriction of
no variants at the top level of an event, at which point the
currently open-coded check for empty members will become
insufficient.  Factor it out into a new helper method is_empty()
now, and future-proof it by checking variants, too, along with an
assert that it is not used prior to the completion of .check().
Update places that were checking for (non-)empty .members to use
the new helper.

All of the current callers assert that there are no variants (either
directly, or by qapi.py asserting that base types have no variants),
so this is not a semantic change.

No change to generated code.

Signed-off-by: Eric Blake 

---
v8: commit message tweaks, also fix qapi-commands.py to use it
v7: rebase to latest
[Previously posted as part of "easier unboxed visits/qapi implicit types"]
v2: no change
v1: add some asserts
[Previously posted as part of "qapi cleanup subset E"]
v9: improve commit message
v8: no change
v7: rebase to context change
v6: new patch
---
 scripts/qapi.py  | 4 
 scripts/qapi-commands.py | 6 +++---
 scripts/qapi-event.py| 6 +++---
 scripts/qapi-types.py| 2 +-
 4 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 400c4ef..27284be 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -999,6 +999,10 @@ class QAPISchemaObjectType(QAPISchemaType):
 # _def_predefineds()
 return self.name.startswith('q_')

+def is_empty(self):
+assert self.members is not None
+return not self.members and not self.variants
+
 def c_name(self):
 assert self.name != 'q_empty'
 return QAPISchemaType.c_name(self)
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 34b6a3a..c93470c 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -107,7 +107,7 @@ def gen_marshal(name, arg_type, ret_type):
 ''',
  c_type=ret_type.c_type())

-if arg_type and arg_type.members:
+if arg_type and not arg_type.is_empty():
 ret += mcgen('''
 Visitor *v;
 %(c_name)s arg = {0};
@@ -137,7 +137,7 @@ def gen_marshal(name, arg_type, ret_type):
 ret += gen_call(name, arg_type, ret_type)

 # 'goto out' produced above for arg_type, and by gen_call() for ret_type
-if (arg_type and arg_type.members) or ret_type:
+if (arg_type and not arg_type.is_empty()) or ret_type:
 ret += mcgen('''

 out:
@@ -145,7 +145,7 @@ out:
 ret += mcgen('''
 error_propagate(errp, err);
 ''')
-if arg_type and arg_type.members:
+if arg_type and not arg_type.is_empty():
 ret += mcgen('''
 visit_free(v);
 v = qapi_dealloc_visitor_new();
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 9c88627..09c0a2a 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -69,7 +69,7 @@ def gen_event_send(name, arg_type):
 ''',
 proto=gen_event_send_proto(name, arg_type))

-if arg_type and arg_type.members:
+if arg_type and not arg_type.is_empty():
 ret += mcgen('''
 QObject *obj;
 Visitor *v;
@@ -88,7 +88,7 @@ def gen_event_send(name, arg_type):
 ''',
  name=name)

-if arg_type and arg_type.members:
+if arg_type and not arg_type.is_empty():
 ret += mcgen('''
 v = qmp_output_visitor_new();

@@ -116,7 +116,7 @@ def gen_event_send(name, arg_type):
 ''',
  c_enum=c_enum_const(event_enum_name, name))

-if arg_type and arg_type.members:
+if arg_type and not arg_type.is_empty():
 ret += mcgen('''
 out:
 visit_free(v);
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 5ace2cf..dabc42e 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -91,7 +91,7 @@ struct %(c_name)s {
 # potential issues with attempting to malloc space for zero-length
 # structs in C, and also incompatibility with C++ (where an empty
 # struct is size 1).
-if not (base and base.members) and not members and not variants:
+if (not base or base.is_empty()) and not members and not variants:
 ret += mcgen('''
 char qapi_dummy_for_empty_struct;
 ''')
-- 
2.5.5

[Qemu-devel] [PATCH v9 02/17] qapi: Require all branches of flat union enum to be covered

[Eric Blake]

We were previously enforcing that all flat union branches were
found in the corresponding enum, but not that all enum values
were covered by branches.  The resulting generated code would
abort() if the user passes the uncovered enum value.

We don't automatically treat non-present branches in a flat
union as empty types, for symmetry with simple unions (there,
the enum type is generated from the list of all branches, so
there is no way to omit a branch but still have it be part of
the union).

A later patch will add shorthand so that branches that are empty
in flat unions can be declared as 'branch':{} instead of
'branch':'Empty', to avoid the need for an otherwise useless
explicit empty type.  [Such shorthand for simple unions is a bit
harder to justify, since we would still have to generate a
wrapper type that parses 'data':{}, rather than truly being an
empty branch with no additional siblings to the 'type' member.]

Signed-off-by: Eric Blake 

---
v9: no change
v8: enhance commit message
v7: new patch
---
 scripts/qapi.py | 8 
 tests/Makefile.include  | 1 +
 tests/qapi-schema/flat-union-incomplete-branch.err  | 1 +
 tests/qapi-schema/flat-union-incomplete-branch.exit | 1 +
 tests/qapi-schema/flat-union-incomplete-branch.json | 9 +
 tests/qapi-schema/flat-union-incomplete-branch.out  | 0
 6 files changed, 20 insertions(+)
 create mode 100644 tests/qapi-schema/flat-union-incomplete-branch.err
 create mode 100644 tests/qapi-schema/flat-union-incomplete-branch.exit
 create mode 100644 tests/qapi-schema/flat-union-incomplete-branch.json
 create mode 100644 tests/qapi-schema/flat-union-incomplete-branch.out

diff --git a/scripts/qapi.py b/scripts/qapi.py
index b13ae47..0635bbb 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -612,6 +612,14 @@ def check_union(expr, expr_info):
 "enum '%s'" %
 (key, enum_define["enum_name"]))

+# If discriminator is user-defined, ensure all values are covered
+if enum_define:
+for value in enum_define['enum_values']:
+if value not in members.keys():
+raise QAPIExprError(expr_info,
+"Union '%s' data missing '%s' branch"
+% (name, value))
+

 def check_alternate(expr, expr_info):
 name = expr['alternate']
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 2010b11..a04c199 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -326,6 +326,7 @@ qapi-schema += flat-union-base-any.json
 qapi-schema += flat-union-base-union.json
 qapi-schema += flat-union-clash-member.json
 qapi-schema += flat-union-empty.json
+qapi-schema += flat-union-incomplete-branch.json
 qapi-schema += flat-union-inline.json
 qapi-schema += flat-union-int-branch.json
 qapi-schema += flat-union-invalid-branch-key.json
diff --git a/tests/qapi-schema/flat-union-incomplete-branch.err 
b/tests/qapi-schema/flat-union-incomplete-branch.err
new file mode 100644
index 000..e826bf0
--- /dev/null
+++ b/tests/qapi-schema/flat-union-incomplete-branch.err
@@ -0,0 +1 @@
+tests/qapi-schema/flat-union-incomplete-branch.json:6: Union 'TestUnion' data 
missing 'value2' branch
diff --git a/tests/qapi-schema/flat-union-incomplete-branch.exit 
b/tests/qapi-schema/flat-union-incomplete-branch.exit
new file mode 100644
index 000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/flat-union-incomplete-branch.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/flat-union-incomplete-branch.json 
b/tests/qapi-schema/flat-union-incomplete-branch.json
new file mode 100644
index 000..25a411b
--- /dev/null
+++ b/tests/qapi-schema/flat-union-incomplete-branch.json
@@ -0,0 +1,9 @@
+# we require all branches of the union to be covered
+{ 'enum': 'TestEnum',
+  'data': [ 'value1', 'value2' ] }
+{ 'struct': 'TestTypeA',
+  'data': { 'string': 'str' } }
+{ 'union': 'TestUnion',
+  'base': { 'type': 'TestEnum' },
+  'discriminator': 'type',
+  'data': { 'value1': 'TestTypeA' } }
diff --git a/tests/qapi-schema/flat-union-incomplete-branch.out 
b/tests/qapi-schema/flat-union-incomplete-branch.out
new file mode 100644
index 000..e69de29
-- 
2.5.5

[Qemu-devel] [PATCH v9 03/17] qapi: Special case c_name() for empty type

[Eric Blake]

Commit 7ce106a rendered QAPISchemaObjectType.c_name() redundant,
since it now does nothing more than delegate to its superclass.
However, rather than deleting it, we can restore part of the
assertion that was removed in that commit, to prove that we never
emit the empty type directly in generated code, but rather
special-case it as a built-in that makes other aspects of code
generation easier to reason about.

Reported-by: Markus Armbruster 
Signed-off-by: Eric Blake 

---
v9: new patch
---
 scripts/qapi.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 0635bbb..68ee319 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1000,6 +1000,7 @@ class QAPISchemaObjectType(QAPISchemaType):
 return self.name.startswith('q_')

 def c_name(self):
+assert self.name != 'q_empty'
 return QAPISchemaType.c_name(self)

 def c_type(self):
-- 
2.5.5

Re: [Qemu-devel] [Qemu-ppc] [PATCH] ppc: exit if compat mode is unknown

[David Gibson]

On Wed, Jul 13, 2016 at 02:34:32PM +0200, Greg Kurz wrote:
> On Wed, 13 Jul 2016 11:10:21 +0200
> Greg Kurz  wrote:
> 
> > Now that properties are set with -global, if we pass a bogus value for
> > compat mode, a warning is printed and the property is simply ignored.
> > For example, if the host is POWER8 and we pass compat=POWER7 instead of
> > compat=power7 the guest will see a POWER8 cpu.
> > 
> > While here, this patch also changes the error message to be more explicit.
> > 
> > Signed-off-by: Greg Kurz 
> > ---
> 
> Of course this causes qom-set to exit as well... not sure this is
> what we want.

Yeah, that seems wrong.  If I'm reading this right this is obsoleted
by your later generic patch adding an _fatal to the generic
code, yes?

> 
> >  target-ppc/translate_init.c |6 +-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> > 
> > diff --git a/target-ppc/translate_init.c b/target-ppc/translate_init.c
> > index 8f257fb74aa7..55688db9c9db 100644
> > --- a/target-ppc/translate_init.c
> > +++ b/target-ppc/translate_init.c
> > @@ -8475,7 +8475,11 @@ static void powerpc_set_compat(Object *obj, Visitor 
> > *v, const char *name,
> >  } else if (strcmp(value, "power8") == 0) {
> >  *max_compat = CPU_POWERPC_LOGICAL_2_07;
> >  } else {
> > -error_setg(errp, "Invalid compatibility mode \"%s\"", value);
> > +/* This is called from -global, which does not exit on error. 
> > Since we
> > + * don't want to start with a wrong compat mode, we exit now.
> > + */
> > +error_report("PPC does not support compatibility mode \"%s\"", 
> > value);
> > +exit(1);
> >  }
> >  
> >  g_free(value);
> > 
> > 
> 

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH] ppc: abort if compat property contains an unknown value

[David Gibson]

On Wed, Jul 13, 2016 at 12:00:17PM +0200, Greg Kurz wrote:
> It is not possible to set the compat property to an unknown value with
> powerpc_set_compat(). Something must have gone terribly wrong in QEMU,
> if we detect an "Internal error" in powerpc_get_compat(). Let's abort then.
> 
> This patch also drops the "max_compat ? *max_compat : -1" construct. It is
> useless since max_compat is dereferenced a few lines above.
> 
> Signed-off-by: Greg Kurz 

Applied to ppc-for-2.7, thanks.

> ---
>  target-ppc/translate_init.c |4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/target-ppc/translate_init.c b/target-ppc/translate_init.c
> index 55688db9c9db..79c86c477878 100644
> --- a/target-ppc/translate_init.c
> +++ b/target-ppc/translate_init.c
> @@ -8446,8 +8446,8 @@ static void powerpc_get_compat(Object *obj, Visitor *v, 
> const char *name,
>  case 0:
>  break;
>  default:
> -error_setg(errp, "Internal error: compat is set to %x",
> -   max_compat ? *max_compat : -1);
> +error_report("Internal error: compat is set to %x", *max_compat);
> +abort();
>  break;
>  }
>  
> 

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH] vl: exit if a bad property value is passed to -global

[David Gibson]

On Wed, Jul 13, 2016 at 08:05:03PM +0200, Greg Kurz wrote:
> When passing '-global driver=host-powerpc64-cpu,property=compat,value=foo'
> on the command line, without this patch, we get the following warning per
> device (which means many lines if the guests has many cpus):
> 
> qemu-system-ppc64: Warning: can't apply global host-powerpc64-cpu.compat=foo:
> Invalid compatibility mode "foo"
> 
> ... and QEMU continues execution, ignoring the property.
> 
> With this patch, we get a single line:
> 
> qemu-system-ppc64: can't apply global host-powerpc64-cpu.compat=foo:
> Invalid compatibility mode "foo"
> 
> ... and QEMU exits.
> 
> Signed-off-by: Greg Kurz 

Reviewed-by: David Gibson 

Not really up to me to decide if this is the right approach for global
option handling, but it seems like the way to go to me.

> ---
>  vl.c |3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/vl.c b/vl.c
> index 356713ea075c..a7cc92781687 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -2922,6 +2922,7 @@ static int global_init_func(void *opaque, QemuOpts 
> *opts, Error **errp)
>  g->property = qemu_opt_get(opts, "property");
>  g->value= qemu_opt_get(opts, "value");
>  g->user_provided = true;
> +g->errp = errp;
>  qdev_prop_register_global(g);
>  return 0;
>  }
> @@ -4451,7 +4452,7 @@ int main(int argc, char **argv, char **envp)
>  machine_register_compat_props(current_machine);
>  
>  qemu_opts_foreach(qemu_find_opts("global"),
> -  global_init_func, NULL, NULL);
> +  global_init_func, NULL, _fatal);
>  
>  /* This checkpoint is required by replay to separate prior clock
> reading from the other reads, because timer polling functions query
> 

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v7 3/7] qapi: add a QmpInputVisitor that does string conversion

[Eric Blake]

On 07/05/2016 07:11 AM, Daniel P. Berrange wrote:
> Currently the QmpInputVisitor assumes that all scalar
> values are directly represented as their final types.
> ie it assumes an 'int' is using QInt, and a 'bool' is
> using QBool.
> 
> This adds an alternative constructor for QmpInputVisitor
> that will set it up such that it expects a QString for
> all scalar types instead.
> 
> This makes it possible to use QmpInputVisitor with a
> QDict produced from QemuOpts, where everything is in
> string format.

This is almost what I need for my netdev_add fallback on my qapi subset
F series, except that I want to have a hybrid mode that accepts BOTH
native type and string.  I'll take this patch and tweak it for my
series, and we can compare the difference.

> 
> Reviewed-by: Marc-André Lureau 
> Signed-off-by: Daniel P. Berrange 
> ---


-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v7 2/7] option: make parse_option_bool/number non-static

[Eric Blake]

On 07/05/2016 07:11 AM, Daniel P. Berrange wrote:
> The opts-visitor.c opts_type_bool() method has code for
> parsing a string to set a bool value, as does the
> qemu-option.c parse_option_bool() method, except it
> handles fewer cases.
> 
> To enable consistency across the codebase, extend
> parse_option_bool() to handle "yes", "no", "y" and
> "n", and make it non-static. Convert the opts
> visitor to call this method directly.
> 
> Also make parse_option_number() non-static to allow
> for similar reuse later.
> 
> Reviewed-by: Marc-André Lureau 
> Signed-off-by: Daniel P. Berrange 
> ---
>  include/qemu/option.h |  4 
>  qapi/opts-visitor.c   | 19 +--
>  util/qemu-option.c| 27 ---
>  3 files changed, 21 insertions(+), 29 deletions(-)
> 

Reviewed-by: Eric Blake 

I'm going to include this in my proposed fallback patches on my qapi
subset F series, where we may need to "fix" netdev_add to care about
back-compat parsing of a string where an integer is expected.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 1/1] spapr: Ensure CPU cores are added contiguously and removed in LIFO order

[David Gibson]

On Wed, Jul 13, 2016 at 12:20:20PM +0530, Bharata B Rao wrote:
> If CPU core addition or removal is allowed in random order leading to
> holes in the core id range (and hence in the cpu_index range), migration
> can fail as migration with holes in cpu_index range isn't yet handled
> correctly.
> 
> Prevent this situation by enforcing the addition in contiguous order
> and removal in LIFO order so that we never end up with holes in
> cpu_index range.
> 
> Signed-off-by: Bharata B Rao 
> ---
> While there is work in progress to support migration when there are holes
> in cpu_index range resulting from out-of-order plug or unplug, this patch
> is intended as a last resort if no easy, risk-free and elegant solution
> emerges before 2.7 dev cycle ends.

Applied to ppc-for-2.7.  We can revert it once the problems with
cpu_index are sorted out.

> 
>  hw/ppc/spapr_cpu_core.c | 21 -
>  1 file changed, 20 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> index bc52b3c..4bfc96b 100644
> --- a/hw/ppc/spapr_cpu_core.c
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -126,12 +126,23 @@ static void spapr_core_release(DeviceState *dev, void 
> *opaque)
>  void spapr_core_unplug(HotplugHandler *hotplug_dev, DeviceState *dev,
> Error **errp)
>  {
> +sPAPRMachineState *spapr = SPAPR_MACHINE(OBJECT(hotplug_dev));
>  CPUCore *cc = CPU_CORE(dev);
>  sPAPRDRConnector *drc =
>  spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, cc->core_id);
>  sPAPRDRConnectorClass *drck;
>  Error *local_err = NULL;
> +int smt = kvmppc_smt_threads();
> +int index = cc->core_id / smt;
> +int spapr_max_cores = max_cpus / smp_threads;
> +int i;
>  
> +for (i = spapr_max_cores - 1; i > index; i--) {
> +if (spapr->cores[i]) {
> +error_setg(errp, "core-id %d should be removed first", i * smt);
> +return;
> +}
> +}
>  g_assert(drc);
>  
>  drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
> @@ -214,7 +225,7 @@ void spapr_core_pre_plug(HotplugHandler *hotplug_dev, 
> DeviceState *dev,
>  sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(OBJECT(hotplug_dev));
>  sPAPRMachineState *spapr = SPAPR_MACHINE(OBJECT(hotplug_dev));
>  int spapr_max_cores = max_cpus / smp_threads;
> -int index;
> +int index, i;
>  int smt = kvmppc_smt_threads();
>  Error *local_err = NULL;
>  CPUCore *cc = CPU_CORE(dev);
> @@ -252,6 +263,14 @@ void spapr_core_pre_plug(HotplugHandler *hotplug_dev, 
> DeviceState *dev,
>  goto out;
>  }
>  
> +for (i = 0; i < index; i++) {
> +if (!spapr->cores[i]) {
> +error_setg(_err, "core-id %d should be added first",
> +   i * smt);
> +goto out;
> +}
> +}
> +
>  out:
>  g_free(base_core_type);
>  error_propagate(errp, local_err);

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v3 10/19] pc: register created initial and hotpluged CPUs in one place pc_cpu_plug()

[Bandan Das]

Eduardo Habkost  writes:

> On Wed, Jul 13, 2016 at 06:59:21PM -0400, Bandan Das wrote:
>> Eduardo Habkost  writes:
>> 
>> > On Wed, Jul 13, 2016 at 06:32:27PM -0400, Bandan Das wrote:
>> >> Igor Mammedov  writes:
>> >> 
>> >> > consolidate possible_cpus array management in pc_cpu_plug()
>> >> > for smp_cpus, coldplugged with -device and hotplugged with
>> >> > device_add.
>> >> 
>> >> So, this takes care of the hotplug case and 09/19 took care of the
>> >> coldplug case, right ? If yes, we should probably modify this commit
>> >> message a little.
>> >
>> > This makes pc_cpu_plug() take care of both: now it will handle
>> > the coldplug case also (in addition to the hotplug case, that it
>> > already handled). I don't understand what you mean.
>> >
>> > Are you talking about the rtc_set_memory() call, only? This seems
>> > to be the only hotplug-specific code that still remains, in this
>> 
>> Right, I thought the rtc_set_memory() call in 09/19 takes care of the
>> case when the user boots with "-device" and this takes care of the case
>> when user invokes device_add, no ?
>
> Yes for rtc_set_memory(). But the main purpose of this patch is
> to reuse all the rest of pc_cpu_plug() (i.e. everything except
> for the rtc_set_memory() call) for the coldplug CPUs too.
>
> Note that I didn't review the patch completely, yet. The
> replacement for the possible_cpus code looks OK, but I didn't
> check if it is safe to call
> HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev)->plug() for coldplugged
> CPUs too.

Owing to "HOTPLUG_HANDLER_GET_CLASS" name, I think I incorrectly assumed that
pc_cpu_plug() is only invoked for the hotplug case. I understand the hotplug
cpu path which iiuc is via pc_hot_add_cpu()->realized->pc_cpu_plug but I am not
sure of "-device" case. I will take a closer look.

Anyway, if this patch takes care of both cases, the commit message makes sense.

Thanks,
Bandan


>> 
>> > patch.
>> >
>> >> 
>> >> Bandan
>> >> > Signed-off-by: Igor Mammedov 
>> >> > ---
>> >> >  hw/i386/pc.c | 25 +
>> >> >  1 file changed, 9 insertions(+), 16 deletions(-)
>> >> >
>> >> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
>> >> > index 3206572..0f85b56 100644
>> >> > --- a/hw/i386/pc.c
>> >> > +++ b/hw/i386/pc.c
>> >> > @@ -1142,7 +1142,6 @@ void pc_cpus_init(PCMachineState *pcms)
>> >> >  if (i < smp_cpus) {
>> >> >  cpu = pc_new_cpu(typename, x86_cpu_apic_id_from_index(i),
>> >> >   _fatal);
>> >> > -pcms->possible_cpus->cpus[i].cpu = CPU(cpu);
>> >> >  object_unref(OBJECT(cpu));
>> >> >  }
>> >> >  }
>> >> > @@ -1697,25 +1696,19 @@ static void pc_cpu_plug(HotplugHandler 
>> >> > *hotplug_dev,
>> >> >  Error *local_err = NULL;
>> >> >  PCMachineState *pcms = PC_MACHINE(hotplug_dev);
>> >> >  
>> >> > -if (!dev->hotplugged) {
>> >> > -goto out;
>> >> > -}
>> >> > -
>> >> > -if (!pcms->acpi_dev) {
>> >> > -error_setg(_err,
>> >> > -   "cpu hotplug is not enabled: missing acpi device");
>> >> > -goto out;
>> >> > +if (pcms->acpi_dev) {
>> >> > +hhc = HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev);
>> >> > +hhc->plug(HOTPLUG_HANDLER(pcms->acpi_dev), dev, _err);
>> >> > +if (local_err) {
>> >> > +goto out;
>> >> > +}
>> >> >  }
>> >> >  
>> >> > -hhc = HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev);
>> >> > -hhc->plug(HOTPLUG_HANDLER(pcms->acpi_dev), dev, _err);
>> >> > -if (local_err) {
>> >> > -goto out;
>> >> > +if (dev->hotplugged) {
>> >> > +/* increment the number of CPUs */
>> >> > +rtc_set_memory(pcms->rtc, 0x5f, rtc_get_memory(pcms->rtc, 
>> >> > 0x5f) + 1);
>> >> >  }
>> >> >  
>> >> > -/* increment the number of CPUs */
>> >> > -rtc_set_memory(pcms->rtc, 0x5f, rtc_get_memory(pcms->rtc, 0x5f) + 
>> >> > 1);
>> >> > -
>> >> >  found_cpu = pc_find_cpu_slot(pcms, CPU(dev), NULL);
>> >> >  found_cpu->cpu = CPU(dev);
>> >> >  out:

[Qemu-devel] [PATCH v9 8/8] docs: Add a generic loader explanation document

[Alistair Francis]

Signed-off-by: Alistair Francis 
---
V9:
 - Clarify the image loading options
V8:
 - Improve documentation
V6:
 - Fixup documentation
V4:
 - Re-write to be more comprehensive

 docs/generic-loader.txt | 63 +
 1 file changed, 63 insertions(+)
 create mode 100644 docs/generic-loader.txt

diff --git a/docs/generic-loader.txt b/docs/generic-loader.txt
new file mode 100644
index 000..16c11ee
--- /dev/null
+++ b/docs/generic-loader.txt
@@ -0,0 +1,63 @@
+Copyright (c) 2016 Xilinx Inc.
+
+This work is licensed under the terms of the GNU GPL, version 2 or later.  See
+the COPYING file in the top-level directory.
+
+
+The 'loader' device allows the user to load multiple images or values into
+QEMU at startup.
+
+Loading Memory Values
+-
+The loader device allows memory values to be set from the command line. This
+can be done by following the syntax below:
+
+-device loader,addr=,data=,data-len=
+-device loader,addr=,cpu-num=
+
+  - The address to store the data or the value to use as the
+  CPU's PC.
+  - The value to be written to the address. The maximum size of
+  the data is 8 bytes.
+  - The length of the data in bytes. This argument must be
+  included if the data argument is.
+   - Set to true if the data to be stored on the guest should be
+  written as big endian data. The default is to write little
+  endian data.
+   - This will cause the CPU to be reset and the PC to be set to
+  the value of addr.
+
+For all values both hex and decimal values are allowed. By default the values
+will be parsed as decimal. To use hex values the user should prefix the number
+with a '0x'.
+
+An example of loading value 0x800e to address 0xfd1a0104 is:
+-device loader,addr=0xfd1a0104,data=0x800e,data-len=4
+
+Loading Files
+-
+The loader device also allows files to be loaded into memory. This can be done
+similarly to setting memory values. The syntax is shown below:
+
+-device loader,file=,addr=,cpu-num=,force-raw=
+
+  - A file to be loaded into memory
+  - The addr in memory that the file should be loaded. This is
+  ignored if you are using an ELF (unless force-raw is true).
+  This is required if you aren't loading an ELF.
+   - This specifies the CPU that should be used. This is an
+  optional argument and will cause the CPU's PC to be set to
+  where the image is stored or in the case of an ELF file to
+  the value in the header. This option should only be used
+  for the boot image.
+  This will also cause the image to be written to the specified
+  CPUs address space.
+ - Forces the file to be treated as a raw image. This can be
+  used to specify the load address of ELF files.
+
+For all values both hex and decimal values are allowed. By default the values
+will be parsed as decimal. To use hex values the user should prefix the number
+with a '0x'.
+
+An example of loading an ELF file which CPU0 will boot is shown below:
+-device loader,file=./images/boot.elf,cpu-num=0
-- 
2.7.4

[Qemu-devel] [PATCH v9 4/8] loader: Add AddressSpace loading support to ELFs

[Alistair Francis]

Add a new function load_elf_as() that allows the caller to specify an
AddressSpace to use when loading the ELF. The original load_elf()
function doesn't have any change in functionality.

Signed-off-by: Alistair Francis 
---
V8:
 - Introduce an RFC version of AddressSpace support

 hw/core/loader.c | 16 ++--
 include/hw/elf_ops.h |  5 +++--
 include/hw/loader.h  | 16 +++-
 3 files changed, 32 insertions(+), 5 deletions(-)

diff --git a/hw/core/loader.c b/hw/core/loader.c
index a024133..0f69894 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -417,6 +417,18 @@ int load_elf(const char *filename, uint64_t 
(*translate_fn)(void *, uint64_t),
  uint64_t *highaddr, int big_endian, int elf_machine,
  int clear_lsb, int data_swab)
 {
+return load_elf_as(filename, translate_fn, translate_opaque, pentry,
+   lowaddr, highaddr, big_endian, elf_machine, clear_lsb,
+   data_swab, NULL);
+}
+
+/* return < 0 if error, otherwise the number of bytes loaded in memory */
+int load_elf_as(const char *filename,
+uint64_t (*translate_fn)(void *, uint64_t),
+void *translate_opaque, uint64_t *pentry, uint64_t *lowaddr,
+uint64_t *highaddr, int big_endian, int elf_machine,
+int clear_lsb, int data_swab, AddressSpace *as)
+{
 int fd, data_order, target_data_order, must_swab, ret = ELF_LOAD_FAILED;
 uint8_t e_ident[EI_NIDENT];
 
@@ -455,11 +467,11 @@ int load_elf(const char *filename, uint64_t 
(*translate_fn)(void *, uint64_t),
 if (e_ident[EI_CLASS] == ELFCLASS64) {
 ret = load_elf64(filename, fd, translate_fn, translate_opaque, 
must_swab,
  pentry, lowaddr, highaddr, elf_machine, clear_lsb,
- data_swab);
+ data_swab, as);
 } else {
 ret = load_elf32(filename, fd, translate_fn, translate_opaque, 
must_swab,
  pentry, lowaddr, highaddr, elf_machine, clear_lsb,
- data_swab);
+ data_swab, as);
 }
 
  fail:
diff --git a/include/hw/elf_ops.h b/include/hw/elf_ops.h
index 1339677..3b8c9e9 100644
--- a/include/hw/elf_ops.h
+++ b/include/hw/elf_ops.h
@@ -263,7 +263,8 @@ static int glue(load_elf, SZ)(const char *name, int fd,
   void *translate_opaque,
   int must_swab, uint64_t *pentry,
   uint64_t *lowaddr, uint64_t *highaddr,
-  int elf_machine, int clear_lsb, int data_swab)
+  int elf_machine, int clear_lsb, int data_swab,
+  AddressSpace *as)
 {
 struct elfhdr ehdr;
 struct elf_phdr *phdr = NULL, *ph;
@@ -405,7 +406,7 @@ static int glue(load_elf, SZ)(const char *name, int fd,
 snprintf(label, sizeof(label), "phdr #%d: %s", i, name);
 
 /* rom_add_elf_program() seize the ownership of 'data' */
-rom_add_elf_program(label, data, file_size, mem_size, addr, NULL);
+rom_add_elf_program(label, data, file_size, mem_size, addr, as);
 
 total_size += mem_size;
 if (addr < low)
diff --git a/include/hw/loader.h b/include/hw/loader.h
index a701423..36a16cc 100644
--- a/include/hw/loader.h
+++ b/include/hw/loader.h
@@ -45,7 +45,7 @@ int load_image_gzipped(const char *filename, hwaddr addr, 
uint64_t max_sz);
 #define ELF_LOAD_WRONG_ENDIAN -4
 const char *load_elf_strerror(int error);
 
-/** load_elf:
+/** load_elf_as:
  * @filename: Path of ELF file
  * @translate_fn: optional function to translate load addresses
  * @translate_opaque: opaque data passed to @translate_fn
@@ -59,6 +59,8 @@ const char *load_elf_strerror(int error);
  * @data_swab: Set to order of byte swapping for data. 0 for no swap, 1
  * for swapping bytes within halfwords, 2 for bytes within
  * words and 3 for within doublewords.
+ * @as: The AddressSpace to load the ELF to. The value of address_space_memory
+ *  is used if nothing is supplied here.
  *
  * Load an ELF file's contents to the emulated system's address space.
  * Clients may optionally specify a callback to perform address
@@ -73,6 +75,16 @@ const char *load_elf_strerror(int error);
  * machine type.
  */
 
+int load_elf_as(const char *filename,
+uint64_t (*translate_fn)(void *, uint64_t),
+void *translate_opaque, uint64_t *pentry, uint64_t *lowaddr,
+uint64_t *highaddr, int big_endian, int elf_machine,
+int clear_lsb, int data_swab, AddressSpace *as);
+
+/** load_elf:
+ * Same as above, but doesn't allow the caller to specify an AddressSpace
+ */
+
 int load_elf(const char *filename, uint64_t (*translate_fn)(void *, uint64_t),
  void *translate_opaque, uint64_t *pentry, uint64_t *lowaddr,
  

Re: [Qemu-devel] [PATCH v3 12/19] apic: move MAX_APICS check to 'apic' class

[Bandan Das]

Eduardo Habkost  writes:

> On Wed, Jul 13, 2016 at 06:47:20PM -0400, Bandan Das wrote:
>> Igor Mammedov  writes:
>> 
>> > MAX_APICS is only used by child 'apic' class and not
>> > by its parent TYPE_APIC_COMMON or any other derived
>> > class.
>> > Move check into end user 'apic' class so it won't
>> > get in the way of other APIC implementations
>> > if they support more then MAX_APICS.
>> >
>> > Signed-off-by: Igor Mammedov 
>> > ---
>> >  hw/intc/apic.c  | 10 ++
>> >  hw/intc/apic_common.c   |  8 
>> >  include/hw/i386/apic_internal.h |  4 +---
>> >  3 files changed, 11 insertions(+), 11 deletions(-)
>> >
>> > diff --git a/hw/intc/apic.c b/hw/intc/apic.c
>> > index e1ab935..b0d237b 100644
>> > --- a/hw/intc/apic.c
>> > +++ b/hw/intc/apic.c
>> > @@ -28,7 +28,9 @@
>> >  #include "trace.h"
>> >  #include "hw/i386/pc.h"
>> >  #include "hw/i386/apic-msidef.h"
>> > +#include "qapi/error.h"
>> >  
>> > +#define MAX_APICS 255
>> >  #define MAX_APIC_WORDS 8
>> >  
>> >  #define SYNC_FROM_VAPIC 0x1
>> > @@ -869,6 +871,14 @@ static const MemoryRegionOps apic_io_ops = {
>> >  static void apic_realize(DeviceState *dev, Error **errp)
>> >  {
>> >  APICCommonState *s = APIC_COMMON(dev);
>> > +static int apic_no;
>> 
>> Can this be a global ? I understand there are no other users but
>> maybe it fits alongside local_apics.
>
> The variable is removed by the next patch. :)
>
>> 
>> > +if (apic_no >= MAX_APICS) {
>> > +error_setg(errp, "%s initialization failed.",
>> > +   object_get_typename(OBJECT(dev)));
>> > +return;
>> > +}
>> > +s->idx = apic_no++;
>> 
>> Is there a possibility of a race here with the apic removal path ?
>
> If there's that possibility: 1) it was already in
> apic_common_realize(); 2) it is removed by patch 13/19.

Ugh! Stupid me :) Yep, thanks for pointing that out.

>> 
>> Bandan
>> 
>> >  memory_region_init_io(>io_memory, OBJECT(s), _io_ops, s, 
>> > "apic-msi",
>> >APIC_SPACE_SIZE);
>> > diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
>> > index e6eb694..fd425d1 100644
>> > --- a/hw/intc/apic_common.c
>> > +++ b/hw/intc/apic_common.c
>> > @@ -299,14 +299,6 @@ static void apic_common_realize(DeviceState *dev, 
>> > Error **errp)
>> >  APICCommonState *s = APIC_COMMON(dev);
>> >  APICCommonClass *info;
>> >  static DeviceState *vapic;
>> > -static int apic_no;
>> > -
>> > -if (apic_no >= MAX_APICS) {
>> > -error_setg(errp, "%s initialization failed.",
>> > -   object_get_typename(OBJECT(dev)));
>> > -return;
>> > -}
>> > -s->idx = apic_no++;
>> >  
>> >  info = APIC_COMMON_GET_CLASS(s);
>> >  info->realize(dev, errp);
>> > diff --git a/include/hw/i386/apic_internal.h 
>> > b/include/hw/i386/apic_internal.h
>> > index 74fe935..5d3be9a 100644
>> > --- a/include/hw/i386/apic_internal.h
>> > +++ b/include/hw/i386/apic_internal.h
>> > @@ -120,8 +120,6 @@
>> >  #define VAPIC_ENABLE_BIT0
>> >  #define VAPIC_ENABLE_MASK   (1 << VAPIC_ENABLE_BIT)
>> >  
>> > -#define MAX_APICS 255
>> > -
>> >  typedef struct APICCommonState APICCommonState;
>> >  
>> >  #define TYPE_APIC_COMMON "apic-common"
>> > @@ -175,7 +173,7 @@ struct APICCommonState {
>> >  uint32_t initial_count;
>> >  int64_t initial_count_load_time;
>> >  int64_t next_time;
>> > -int idx;
>> > +int idx; /* not actually common, used only by 'apic' derived class */
>> >  QEMUTimer *timer;
>> >  int64_t timer_expiry;
>> >  int sipi_vector;

[Qemu-devel] [PATCH v9 6/8] loader: Add AddressSpace loading support to targphys

[Alistair Francis]

Add a new function load_image_targphys_as() that allows the caller
to specify an AddressSpace to use when loading a targphys. The
original load_image_targphys() function doesn't have any change in
functionality.

Signed-off-by: Alistair Francis 
---

 hw/core/loader.c| 10 --
 include/hw/loader.h |  5 +
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/hw/core/loader.c b/hw/core/loader.c
index 861dbc2..31a2d4a 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -133,10 +133,16 @@ ssize_t read_targphys(const char *name,
 return did;
 }
 
-/* return the size or -1 if error */
 int load_image_targphys(const char *filename,
 hwaddr addr, uint64_t max_sz)
 {
+return load_image_targphys_as(filename, addr, max_sz, NULL);
+}
+
+/* return the size or -1 if error */
+int load_image_targphys_as(const char *filename,
+   hwaddr addr, uint64_t max_sz, AddressSpace *as)
+{
 int size;
 
 size = get_image_size(filename);
@@ -144,7 +150,7 @@ int load_image_targphys(const char *filename,
 return -1;
 }
 if (size > 0) {
-rom_add_file_fixed(filename, addr, -1);
+rom_add_file_fixed_as(filename, addr, -1, as);
 }
 return size;
 }
diff --git a/include/hw/loader.h b/include/hw/loader.h
index ede98f6..1a9053f 100644
--- a/include/hw/loader.h
+++ b/include/hw/loader.h
@@ -16,6 +16,9 @@ int load_image(const char *filename, uint8_t *addr); /* 
deprecated */
 ssize_t load_image_size(const char *filename, void *addr, size_t size);
 int load_image_targphys(const char *filename, hwaddr,
 uint64_t max_sz);
+int load_image_targphys_as(const char *filename,
+   hwaddr addr, uint64_t max_sz, AddressSpace *as);
+
 /**
  * load_image_mr: load an image into a memory region
  * @filename: Path to the image file
@@ -163,6 +166,8 @@ void hmp_info_roms(Monitor *mon, const QDict *qdict);
 
 #define rom_add_file_as(_f, _as, _i)\
 rom_add_file(_f, NULL, 0, _i, false, NULL, _as)
+#define rom_add_file_fixed_as(_f, _a, _i, _as)  \
+rom_add_file(_f, NULL, _a, _i, false, NULL, _as)
 #define rom_add_blob_fixed_as(_f, _b, _l, _a, _as)  \
 rom_add_blob(_f, _b, _l, _l, _a, NULL, NULL, _as)
 
-- 
2.7.4

[Qemu-devel] [PATCH v9 7/8] generic-loader: Add a generic loader

[Alistair Francis]

Add a generic loader to QEMU which can be used to load images or set
memory values.

Signed-off-by: Alistair Francis 
---
V9:
 - Fix error messages
 - Updated some incorrect logic
 - Add address space loading support for all image types
 - Explain why the reset is manually registered
V8:
 - Code corrections
 - Rebase
V7:
 - Rebase
V6:
 - Add error checking
V5:
 - Rebase
V4:
 - Allow the loader to work with every architecture
 - Move the file to hw/core
 - Increase the maximum number of CPUs
 - Make the CPU operations conditional
 - Convert the cpu option to cpu-num
 - Require the user to specify endianess
V3:
 - Pass the ram_size to load_image_targphys()
V2:
 - Add maintainers entry
 - Perform bounds checking
 - Register and unregister the reset in the realise/unrealise
Changes since RFC:
 - Add BE support

 MAINTAINERS  |   6 ++
 hw/core/Makefile.objs|   2 +
 hw/core/generic-loader.c | 185 +++
 include/hw/core/generic-loader.h |  45 ++
 4 files changed, 238 insertions(+)
 create mode 100644 hw/core/generic-loader.c
 create mode 100644 include/hw/core/generic-loader.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 1d0e2c3..ea3a3b5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -993,6 +993,12 @@ M: Dmitry Fleytman 
 S: Maintained
 F: hw/net/e1000e*
 
+Generic Loader
+M: Alistair Francis 
+S: Maintained
+F: hw/core/generic-loader.c
+F: include/hw/core/generic-loader.h
+
 Subsystems
 --
 Audio
diff --git a/hw/core/Makefile.objs b/hw/core/Makefile.objs
index cfd4840..939c94e 100644
--- a/hw/core/Makefile.objs
+++ b/hw/core/Makefile.objs
@@ -17,3 +17,5 @@ common-obj-$(CONFIG_SOFTMMU) += loader.o
 common-obj-$(CONFIG_SOFTMMU) += qdev-properties-system.o
 common-obj-$(CONFIG_SOFTMMU) += register.o
 common-obj-$(CONFIG_PLATFORM_BUS) += platform-bus.o
+
+obj-$(CONFIG_SOFTMMU) += generic-loader.o
diff --git a/hw/core/generic-loader.c b/hw/core/generic-loader.c
new file mode 100644
index 000..3fbb954
--- /dev/null
+++ b/hw/core/generic-loader.c
@@ -0,0 +1,185 @@
+/*
+ * Generic Loader
+ *
+ * Copyright (C) 2014 Li Guang
+ * Copyright (C) 2016 Xilinx Inc.
+ * Written by Li Guang 
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+ * for more details.
+ */
+
+#include "qemu/osdep.h"
+#include "qom/cpu.h"
+#include "hw/sysbus.h"
+#include "sysemu/dma.h"
+#include "hw/loader.h"
+#include "qapi/error.h"
+#include "hw/core/generic-loader.h"
+
+#define CPU_NONE 0x
+
+static void generic_loader_reset(void *opaque)
+{
+GenericLoaderState *s = GENERIC_LOADER(opaque);
+
+if (s->cpu) {
+CPUClass *cc = CPU_GET_CLASS(s->cpu);
+cpu_reset(s->cpu);
+if (cc) {
+cc->set_pc(s->cpu, s->addr);
+}
+}
+
+if (s->data_len) {
+assert(s->data_len < sizeof(s->data));
+dma_memory_write(first_cpu->as, s->addr, >data,
+ s->data_len);
+}
+}
+
+static void generic_loader_realize(DeviceState *dev, Error **errp)
+{
+GenericLoaderState *s = GENERIC_LOADER(dev);
+hwaddr entry;
+int big_endian;
+int size = 0;
+
+/* Perform some error checking on the user's options */
+if (s->data || s->data_len  || s->data_be) {
+/* User is loading memory values */
+if (s->file) {
+error_setg(errp, "Specifying a file is not supported when loading "
+   "memory values");
+return;
+} else if (s->force_raw) {
+error_setg(errp, "Specifying force-raw is not supported when "
+   "loading memory values");
+return;
+} else if (!s->data || !s->data_len) {
+error_setg(errp, "Both data and data-len must be specified");
+return;
+} else if (s->cpu_num != CPU_NONE) {
+error_setg(errp, "Setting data and cpu-num is not supported");
+return;
+}
+} else if (s->file || s->force_raw)  {
+/* User is loading an image */
+if (s->data || s->data_len || s->data_be) {
+error_setg(errp, "data can not be specified when loading an "
+   "image");
+return;
+}
+} else if (s->data_len) {
+if (s->data_len > 8) {
+error_setg(errp, "data-len cannot be greater then 8 bytes");
+return;
+} else if (s->data_len > sizeof(s->data)) {
+

[Qemu-devel] [PATCH v9 3/8] loader: Allow a custom AddressSpace when loading ROMs

[Alistair Francis]

When loading ROMs allow the caller to specify an AddressSpace to use for
the load.

Signed-off-by: Alistair Francis 
---
V9:
 - Fixup the ROM ordering
 - Don't allow address space and memory region to be specified
V8:
 - Introduce an RFC version of AddressSpace loading support

 hw/core/loader.c | 39 ---
 include/hw/elf_ops.h |  2 +-
 include/hw/loader.h  | 10 ++
 3 files changed, 35 insertions(+), 16 deletions(-)

diff --git a/hw/core/loader.c b/hw/core/loader.c
index 6b61f29..a024133 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -777,6 +777,7 @@ struct Rom {
 
 uint8_t *data;
 MemoryRegion *mr;
+AddressSpace *as;
 int isrom;
 char *fw_dir;
 char *fw_file;
@@ -796,12 +797,15 @@ static void rom_insert(Rom *rom)
 hw_error ("ROM images must be loaded at startup\n");
 }
 
-/* list is ordered by load address */
+/* List is ordered by load address in the same address space */
 QTAILQ_FOREACH(item, , next) {
-if (rom->addr >= item->addr)
-continue;
-QTAILQ_INSERT_BEFORE(item, rom, next);
-return;
+if (rom->addr >= item->addr && rom->as == item->as) {
+QTAILQ_INSERT_AFTER(, item, rom, next);
+return;
+} else if (rom->addr <= item->addr && rom->as == item->as) {
+QTAILQ_INSERT_BEFORE(item, rom, next);
+return;
+}
 }
 QTAILQ_INSERT_TAIL(, rom, next);
 }
@@ -833,16 +837,25 @@ static void *rom_set_mr(Rom *rom, Object *owner, const 
char *name)
 
 int rom_add_file(const char *file, const char *fw_dir,
  hwaddr addr, int32_t bootindex,
- bool option_rom, MemoryRegion *mr)
+ bool option_rom, MemoryRegion *mr,
+ AddressSpace *as)
 {
 MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
 Rom *rom;
 int rc, fd = -1;
 char devpath[100];
 
+if (as && mr) {
+fprintf(stderr, "Specifying an Address Space and Memory Region is " \
+"not valid when loading a rom\n");
+/* We haven't allocated anything so we don't need any cleanup */
+return -1;
+}
+
 rom = g_malloc0(sizeof(*rom));
 rom->name = g_strdup(file);
 rom->path = qemu_find_file(QEMU_FILE_TYPE_BIOS, rom->name);
+rom->as = as;
 if (rom->path == NULL) {
 rom->path = g_strdup(file);
 }
@@ -969,7 +982,7 @@ MemoryRegion *rom_add_blob(const char *name, const void 
*blob, size_t len,
  * memory ownership of "data", so we don't have to allocate and copy the 
buffer.
  */
 int rom_add_elf_program(const char *name, void *data, size_t datasize,
-size_t romsize, hwaddr addr)
+size_t romsize, hwaddr addr, AddressSpace *as)
 {
 Rom *rom;
 
@@ -979,18 +992,19 @@ int rom_add_elf_program(const char *name, void *data, 
size_t datasize,
 rom->datasize = datasize;
 rom->romsize  = romsize;
 rom->data = data;
+rom->as   = as;
 rom_insert(rom);
 return 0;
 }
 
 int rom_add_vga(const char *file)
 {
-return rom_add_file(file, "vgaroms", 0, -1, true, NULL);
+return rom_add_file(file, "vgaroms", 0, -1, true, NULL, NULL);
 }
 
 int rom_add_option(const char *file, int32_t bootindex)
 {
-return rom_add_file(file, "genroms", 0, bootindex, true, NULL);
+return rom_add_file(file, "genroms", 0, bootindex, true, NULL, NULL);
 }
 
 static void rom_reset(void *unused)
@@ -1008,7 +1022,8 @@ static void rom_reset(void *unused)
 void *host = memory_region_get_ram_ptr(rom->mr);
 memcpy(host, rom->data, rom->datasize);
 } else {
-cpu_physical_memory_write_rom(_space_memory,
+cpu_physical_memory_write_rom(rom->as ? rom->as :
+_space_memory,
   rom->addr, rom->data, rom->datasize);
 }
 if (rom->isrom) {
@@ -1031,12 +1046,13 @@ int rom_check_and_register_reset(void)
 hwaddr addr = 0;
 MemoryRegionSection section;
 Rom *rom;
+AddressSpace *as = NULL;
 
 QTAILQ_FOREACH(rom, , next) {
 if (rom->fw_file) {
 continue;
 }
-if (addr > rom->addr) {
+if ((addr > rom->addr) && (as == rom->as)) {
 fprintf(stderr, "rom: requested regions overlap "
 "(rom %s. free=0x" TARGET_FMT_plx
 ", addr=0x" TARGET_FMT_plx ")\n",
@@ -1049,6 +1065,7 @@ int rom_check_and_register_reset(void)
  rom->addr, 1);
 rom->isrom = int128_nz(section.size) && 
memory_region_is_rom(section.mr);
 memory_region_unref(section.mr);
+as = rom->as;
 }
 qemu_register_reset(rom_reset, NULL);
 roms_loaded = 1;
diff --git a/include/hw/elf_ops.h b/include/hw/elf_ops.h
index db70c11..1339677 100644
--- 

[Qemu-devel] [PATCH v9 5/8] loader: Add AddressSpace loading support to uImages

[Alistair Francis]

Add a new function load_uimage_as() that allows the caller to
specify an AddressSpace to use when loading the uImage. The
original load_uimage() function doesn't have any change in
functionality.

Signed-off-by: Alistair Francis 
---

 hw/core/loader.c| 17 +
 include/hw/loader.h |  6 ++
 2 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/hw/core/loader.c b/hw/core/loader.c
index 0f69894..861dbc2 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -581,7 +581,7 @@ static ssize_t gunzip(void *dst, size_t dstlen, uint8_t 
*src,
 static int load_uboot_image(const char *filename, hwaddr *ep, hwaddr *loadaddr,
 int *is_linux, uint8_t image_type,
 uint64_t (*translate_fn)(void *, uint64_t),
-void *translate_opaque)
+void *translate_opaque, AddressSpace *as)
 {
 int fd;
 int size;
@@ -682,7 +682,7 @@ static int load_uboot_image(const char *filename, hwaddr 
*ep, hwaddr *loadaddr,
 hdr->ih_size = bytes;
 }
 
-rom_add_blob_fixed(filename, data, hdr->ih_size, address);
+rom_add_blob_fixed_as(filename, data, hdr->ih_size, address, as);
 
 ret = hdr->ih_size;
 
@@ -698,14 +698,23 @@ int load_uimage(const char *filename, hwaddr *ep, hwaddr 
*loadaddr,
 void *translate_opaque)
 {
 return load_uboot_image(filename, ep, loadaddr, is_linux, IH_TYPE_KERNEL,
-translate_fn, translate_opaque);
+translate_fn, translate_opaque, NULL);
+}
+
+int load_uimage_as(const char *filename, hwaddr *ep, hwaddr *loadaddr,
+   int *is_linux,
+   uint64_t (*translate_fn)(void *, uint64_t),
+   void *translate_opaque, AddressSpace *as)
+{
+return load_uboot_image(filename, ep, loadaddr, is_linux, IH_TYPE_KERNEL,
+translate_fn, translate_opaque, as);
 }
 
 /* Load a ramdisk.  */
 int load_ramdisk(const char *filename, hwaddr addr, uint64_t max_sz)
 {
 return load_uboot_image(filename, NULL, , NULL, IH_TYPE_RAMDISK,
-NULL, NULL);
+NULL, NULL, NULL);
 }
 
 /* Load a gzip-compressed kernel to a dynamically allocated buffer. */
diff --git a/include/hw/loader.h b/include/hw/loader.h
index 36a16cc..ede98f6 100644
--- a/include/hw/loader.h
+++ b/include/hw/loader.h
@@ -108,6 +108,10 @@ int load_uimage(const char *filename, hwaddr *ep,
 hwaddr *loadaddr, int *is_linux,
 uint64_t (*translate_fn)(void *, uint64_t),
 void *translate_opaque);
+int load_uimage_as(const char *filename, hwaddr *ep,
+   hwaddr *loadaddr, int *is_linux,
+   uint64_t (*translate_fn)(void *, uint64_t),
+   void *translate_opaque, AddressSpace *as);
 
 /**
  * load_ramdisk:
@@ -159,6 +163,8 @@ void hmp_info_roms(Monitor *mon, const QDict *qdict);
 
 #define rom_add_file_as(_f, _as, _i)\
 rom_add_file(_f, NULL, 0, _i, false, NULL, _as)
+#define rom_add_blob_fixed_as(_f, _b, _l, _a, _as)  \
+rom_add_blob(_f, _b, _l, _l, _a, NULL, NULL, _as)
 
 #define PC_ROM_MIN_VGA 0xc
 #define PC_ROM_MIN_OPTION  0xc8000
-- 
2.7.4

[Qemu-devel] [PATCH v9 2/8] loader: Use the specified MemoryRegion

[Alistair Francis]

Prevously the specified MemoryRegion was ignored during the rom register
reset. This patch uses the rom MemoryRegion is avaliable.

Signed-off-by: Alistair Francis 
---

 hw/core/loader.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/hw/core/loader.c b/hw/core/loader.c
index 53e0e41..6b61f29 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -1045,7 +1045,8 @@ int rom_check_and_register_reset(void)
 }
 addr  = rom->addr;
 addr += rom->romsize;
-section = memory_region_find(get_system_memory(), rom->addr, 1);
+section = memory_region_find(rom->mr ? rom->mr : get_system_memory(),
+ rom->addr, 1);
 rom->isrom = int128_nz(section.size) && 
memory_region_is_rom(section.mr);
 memory_region_unref(section.mr);
 }
-- 
2.7.4

[Qemu-devel] [PATCH v9 0/8] Add a generic loader

[Alistair Francis]

This work is based on the original work by Li Guang with extra
features added by Peter C and myself.

The idea of this loader is to allow the user to load multiple images
or values into QEMU at startup.

Memory values can be loaded like this: -device 
loader,addr=0xfd1a0104,data=0x800e,data-len=4

Images can be loaded like this: -device loader,file=./images/u-boot.elf,cpu=0

This can be useful and we use it a lot in Xilinx to load multiple images
into a machine at creation (ATF, Kernel and DTB for example).

It can also be used to set registers.

This patch series makes the load_elf() function more generic by not
requiring an architecture. It also adds new functions load_elf_as(),
load_uimage_as and load_image_targphys_as which allows custom
AddressSpaces when loading images.

V9:
 - Logic and documentation corrections
 - Add address space loading support for uImages and targphys
V8:
 - Allow custom AddressSpaces when loading images
 - Move ELF architecture handling code
 - Rebase
 - Corrections to loading code
 - Corrections to documentation
V7:
 - Fix typo in comment
 - Rebase
V6:
 - Add error checking
V5:
 - Rebase
V4:
 - Re-write documentation
 - Allow the loader to work with every architecture
 - Move the file to hw/core
 - Increase the maximum number of CPUs
 - Make the CPU operations conditional
 - Convert the cpu option to cpu-num
 - Require the user to specify endianess
V2:
 - Add an entry to the maintainers file
 - Add some documentation
 - Perform bounds checking on the data_len
 - Register and unregister the reset in the realise/unrealise
Changes since RFC:
 - Add support for BE


Alistair Francis (8):
  loader: Allow ELF loader to auto-detect the ELF arch
  loader: Use the specified MemoryRegion
  loader: Allow a custom AddressSpace when loading ROMs
  loader: Add AddressSpace loading support to ELFs
  loader: Add AddressSpace loading support to uImages
  loader: Add AddressSpace loading support to targphys
  generic-loader: Add a generic loader
  docs: Add a generic loader explanation document

 MAINTAINERS  |   6 ++
 docs/generic-loader.txt  |  63 +
 hw/core/Makefile.objs|   2 +
 hw/core/generic-loader.c | 185 +++
 hw/core/loader.c |  85 +-
 include/hw/core/generic-loader.h |  45 ++
 include/hw/elf_ops.h |  10 ++-
 include/hw/loader.h  |  40 +++--
 8 files changed, 409 insertions(+), 27 deletions(-)
 create mode 100644 docs/generic-loader.txt
 create mode 100644 hw/core/generic-loader.c
 create mode 100644 include/hw/core/generic-loader.h

-- 
2.7.4

[Qemu-devel] [PATCH v9 1/8] loader: Allow ELF loader to auto-detect the ELF arch

[Alistair Francis]

If the caller didn't specify an architecture for the ELF machine
the load_elf() function will auto detect it based on the ELF file.

Signed-off-by: Alistair Francis 
---
V9:
 - Update documentation
V8:
 - Move into load_elf64/load_elf32
V7:
 - Fix typo

 include/hw/elf_ops.h | 5 +
 include/hw/loader.h  | 3 +++
 2 files changed, 8 insertions(+)

diff --git a/include/hw/elf_ops.h b/include/hw/elf_ops.h
index f510e7e..db70c11 100644
--- a/include/hw/elf_ops.h
+++ b/include/hw/elf_ops.h
@@ -280,6 +280,11 @@ static int glue(load_elf, SZ)(const char *name, int fd,
 glue(bswap_ehdr, SZ)();
 }
 
+if (elf_machine < 1) {
+/* The caller didn't specify an ARCH, we can figure it out */
+elf_machine = ehdr.e_machine;
+}
+
 switch (elf_machine) {
 case EM_PPC64:
 if (ehdr.e_machine != EM_PPC64) {
diff --git a/include/hw/loader.h b/include/hw/loader.h
index 4879b63..fd540fc 100644
--- a/include/hw/loader.h
+++ b/include/hw/loader.h
@@ -68,6 +68,9 @@ const char *load_elf_strerror(int error);
  * load will fail if the target ELF does not match. Some architectures
  * have some architecture-specific behaviours that come into effect when
  * their particular values for @elf_machine are set.
+ * If no @elf_machine is provided the machine will default to the value
+ * in the ELFs header and no checks will be carried out against the
+ * machine type.
  */
 
 int load_elf(const char *filename, uint64_t (*translate_fn)(void *, uint64_t),
-- 
2.7.4

Re: [Qemu-devel] [PATCH v3 12/19] apic: move MAX_APICS check to 'apic' class

[Eduardo Habkost]

On Wed, Jul 13, 2016 at 06:47:20PM -0400, Bandan Das wrote:
> Igor Mammedov  writes:
> 
> > MAX_APICS is only used by child 'apic' class and not
> > by its parent TYPE_APIC_COMMON or any other derived
> > class.
> > Move check into end user 'apic' class so it won't
> > get in the way of other APIC implementations
> > if they support more then MAX_APICS.
> >
> > Signed-off-by: Igor Mammedov 
> > ---
> >  hw/intc/apic.c  | 10 ++
> >  hw/intc/apic_common.c   |  8 
> >  include/hw/i386/apic_internal.h |  4 +---
> >  3 files changed, 11 insertions(+), 11 deletions(-)
> >
> > diff --git a/hw/intc/apic.c b/hw/intc/apic.c
> > index e1ab935..b0d237b 100644
> > --- a/hw/intc/apic.c
> > +++ b/hw/intc/apic.c
> > @@ -28,7 +28,9 @@
> >  #include "trace.h"
> >  #include "hw/i386/pc.h"
> >  #include "hw/i386/apic-msidef.h"
> > +#include "qapi/error.h"
> >  
> > +#define MAX_APICS 255
> >  #define MAX_APIC_WORDS 8
> >  
> >  #define SYNC_FROM_VAPIC 0x1
> > @@ -869,6 +871,14 @@ static const MemoryRegionOps apic_io_ops = {
> >  static void apic_realize(DeviceState *dev, Error **errp)
> >  {
> >  APICCommonState *s = APIC_COMMON(dev);
> > +static int apic_no;
> 
> Can this be a global ? I understand there are no other users but
> maybe it fits alongside local_apics.

The variable is removed by the next patch. :)

> 
> > +if (apic_no >= MAX_APICS) {
> > +error_setg(errp, "%s initialization failed.",
> > +   object_get_typename(OBJECT(dev)));
> > +return;
> > +}
> > +s->idx = apic_no++;
> 
> Is there a possibility of a race here with the apic removal path ?

If there's that possibility: 1) it was already in
apic_common_realize(); 2) it is removed by patch 13/19.

> 
> Bandan
> 
> >  memory_region_init_io(>io_memory, OBJECT(s), _io_ops, s, 
> > "apic-msi",
> >APIC_SPACE_SIZE);
> > diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
> > index e6eb694..fd425d1 100644
> > --- a/hw/intc/apic_common.c
> > +++ b/hw/intc/apic_common.c
> > @@ -299,14 +299,6 @@ static void apic_common_realize(DeviceState *dev, 
> > Error **errp)
> >  APICCommonState *s = APIC_COMMON(dev);
> >  APICCommonClass *info;
> >  static DeviceState *vapic;
> > -static int apic_no;
> > -
> > -if (apic_no >= MAX_APICS) {
> > -error_setg(errp, "%s initialization failed.",
> > -   object_get_typename(OBJECT(dev)));
> > -return;
> > -}
> > -s->idx = apic_no++;
> >  
> >  info = APIC_COMMON_GET_CLASS(s);
> >  info->realize(dev, errp);
> > diff --git a/include/hw/i386/apic_internal.h 
> > b/include/hw/i386/apic_internal.h
> > index 74fe935..5d3be9a 100644
> > --- a/include/hw/i386/apic_internal.h
> > +++ b/include/hw/i386/apic_internal.h
> > @@ -120,8 +120,6 @@
> >  #define VAPIC_ENABLE_BIT0
> >  #define VAPIC_ENABLE_MASK   (1 << VAPIC_ENABLE_BIT)
> >  
> > -#define MAX_APICS 255
> > -
> >  typedef struct APICCommonState APICCommonState;
> >  
> >  #define TYPE_APIC_COMMON "apic-common"
> > @@ -175,7 +173,7 @@ struct APICCommonState {
> >  uint32_t initial_count;
> >  int64_t initial_count_load_time;
> >  int64_t next_time;
> > -int idx;
> > +int idx; /* not actually common, used only by 'apic' derived class */
> >  QEMUTimer *timer;
> >  int64_t timer_expiry;
> >  int sipi_vector;

-- 
Eduardo

Re: [Qemu-devel] [PATCH v3 10/19] pc: register created initial and hotpluged CPUs in one place pc_cpu_plug()

[Eduardo Habkost]

On Wed, Jul 13, 2016 at 06:59:21PM -0400, Bandan Das wrote:
> Eduardo Habkost  writes:
> 
> > On Wed, Jul 13, 2016 at 06:32:27PM -0400, Bandan Das wrote:
> >> Igor Mammedov  writes:
> >> 
> >> > consolidate possible_cpus array management in pc_cpu_plug()
> >> > for smp_cpus, coldplugged with -device and hotplugged with
> >> > device_add.
> >> 
> >> So, this takes care of the hotplug case and 09/19 took care of the
> >> coldplug case, right ? If yes, we should probably modify this commit
> >> message a little.
> >
> > This makes pc_cpu_plug() take care of both: now it will handle
> > the coldplug case also (in addition to the hotplug case, that it
> > already handled). I don't understand what you mean.
> >
> > Are you talking about the rtc_set_memory() call, only? This seems
> > to be the only hotplug-specific code that still remains, in this
> 
> Right, I thought the rtc_set_memory() call in 09/19 takes care of the
> case when the user boots with "-device" and this takes care of the case
> when user invokes device_add, no ?

Yes for rtc_set_memory(). But the main purpose of this patch is
to reuse all the rest of pc_cpu_plug() (i.e. everything except
for the rtc_set_memory() call) for the coldplug CPUs too.

Note that I didn't review the patch completely, yet. The
replacement for the possible_cpus code looks OK, but I didn't
check if it is safe to call
HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev)->plug() for coldplugged
CPUs too.

> 
> > patch.
> >
> >> 
> >> Bandan
> >> > Signed-off-by: Igor Mammedov 
> >> > ---
> >> >  hw/i386/pc.c | 25 +
> >> >  1 file changed, 9 insertions(+), 16 deletions(-)
> >> >
> >> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> >> > index 3206572..0f85b56 100644
> >> > --- a/hw/i386/pc.c
> >> > +++ b/hw/i386/pc.c
> >> > @@ -1142,7 +1142,6 @@ void pc_cpus_init(PCMachineState *pcms)
> >> >  if (i < smp_cpus) {
> >> >  cpu = pc_new_cpu(typename, x86_cpu_apic_id_from_index(i),
> >> >   _fatal);
> >> > -pcms->possible_cpus->cpus[i].cpu = CPU(cpu);
> >> >  object_unref(OBJECT(cpu));
> >> >  }
> >> >  }
> >> > @@ -1697,25 +1696,19 @@ static void pc_cpu_plug(HotplugHandler 
> >> > *hotplug_dev,
> >> >  Error *local_err = NULL;
> >> >  PCMachineState *pcms = PC_MACHINE(hotplug_dev);
> >> >  
> >> > -if (!dev->hotplugged) {
> >> > -goto out;
> >> > -}
> >> > -
> >> > -if (!pcms->acpi_dev) {
> >> > -error_setg(_err,
> >> > -   "cpu hotplug is not enabled: missing acpi device");
> >> > -goto out;
> >> > +if (pcms->acpi_dev) {
> >> > +hhc = HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev);
> >> > +hhc->plug(HOTPLUG_HANDLER(pcms->acpi_dev), dev, _err);
> >> > +if (local_err) {
> >> > +goto out;
> >> > +}
> >> >  }
> >> >  
> >> > -hhc = HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev);
> >> > -hhc->plug(HOTPLUG_HANDLER(pcms->acpi_dev), dev, _err);
> >> > -if (local_err) {
> >> > -goto out;
> >> > +if (dev->hotplugged) {
> >> > +/* increment the number of CPUs */
> >> > +rtc_set_memory(pcms->rtc, 0x5f, rtc_get_memory(pcms->rtc, 0x5f) 
> >> > + 1);
> >> >  }
> >> >  
> >> > -/* increment the number of CPUs */
> >> > -rtc_set_memory(pcms->rtc, 0x5f, rtc_get_memory(pcms->rtc, 0x5f) + 
> >> > 1);
> >> > -
> >> >  found_cpu = pc_find_cpu_slot(pcms, CPU(dev), NULL);
> >> >  found_cpu->cpu = CPU(dev);
> >> >  out:

-- 
Eduardo

Re: [Qemu-devel] [PATCH v3 10/19] pc: register created initial and hotpluged CPUs in one place pc_cpu_plug()

[Bandan Das]

Eduardo Habkost  writes:

> On Wed, Jul 13, 2016 at 06:32:27PM -0400, Bandan Das wrote:
>> Igor Mammedov  writes:
>> 
>> > consolidate possible_cpus array management in pc_cpu_plug()
>> > for smp_cpus, coldplugged with -device and hotplugged with
>> > device_add.
>> 
>> So, this takes care of the hotplug case and 09/19 took care of the
>> coldplug case, right ? If yes, we should probably modify this commit
>> message a little.
>
> This makes pc_cpu_plug() take care of both: now it will handle
> the coldplug case also (in addition to the hotplug case, that it
> already handled). I don't understand what you mean.
>
> Are you talking about the rtc_set_memory() call, only? This seems
> to be the only hotplug-specific code that still remains, in this

Right, I thought the rtc_set_memory() call in 09/19 takes care of the
case when the user boots with "-device" and this takes care of the case
when user invokes device_add, no ?

> patch.
>
>> 
>> Bandan
>> > Signed-off-by: Igor Mammedov 
>> > ---
>> >  hw/i386/pc.c | 25 +
>> >  1 file changed, 9 insertions(+), 16 deletions(-)
>> >
>> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
>> > index 3206572..0f85b56 100644
>> > --- a/hw/i386/pc.c
>> > +++ b/hw/i386/pc.c
>> > @@ -1142,7 +1142,6 @@ void pc_cpus_init(PCMachineState *pcms)
>> >  if (i < smp_cpus) {
>> >  cpu = pc_new_cpu(typename, x86_cpu_apic_id_from_index(i),
>> >   _fatal);
>> > -pcms->possible_cpus->cpus[i].cpu = CPU(cpu);
>> >  object_unref(OBJECT(cpu));
>> >  }
>> >  }
>> > @@ -1697,25 +1696,19 @@ static void pc_cpu_plug(HotplugHandler 
>> > *hotplug_dev,
>> >  Error *local_err = NULL;
>> >  PCMachineState *pcms = PC_MACHINE(hotplug_dev);
>> >  
>> > -if (!dev->hotplugged) {
>> > -goto out;
>> > -}
>> > -
>> > -if (!pcms->acpi_dev) {
>> > -error_setg(_err,
>> > -   "cpu hotplug is not enabled: missing acpi device");
>> > -goto out;
>> > +if (pcms->acpi_dev) {
>> > +hhc = HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev);
>> > +hhc->plug(HOTPLUG_HANDLER(pcms->acpi_dev), dev, _err);
>> > +if (local_err) {
>> > +goto out;
>> > +}
>> >  }
>> >  
>> > -hhc = HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev);
>> > -hhc->plug(HOTPLUG_HANDLER(pcms->acpi_dev), dev, _err);
>> > -if (local_err) {
>> > -goto out;
>> > +if (dev->hotplugged) {
>> > +/* increment the number of CPUs */
>> > +rtc_set_memory(pcms->rtc, 0x5f, rtc_get_memory(pcms->rtc, 0x5f) + 
>> > 1);
>> >  }
>> >  
>> > -/* increment the number of CPUs */
>> > -rtc_set_memory(pcms->rtc, 0x5f, rtc_get_memory(pcms->rtc, 0x5f) + 1);
>> > -
>> >  found_cpu = pc_find_cpu_slot(pcms, CPU(dev), NULL);
>> >  found_cpu->cpu = CPU(dev);
>> >  out:

Re: [Qemu-devel] [PATCH v3 07/19] pc: set APIC ID based on socket/core/thread ids if it's not been set yet

[Bandan Das]

Eduardo Habkost  writes:

> On Wed, Jul 13, 2016 at 06:24:17PM -0400, Bandan Das wrote:
>> Igor Mammedov  writes:
>> 
>> > CPU added with device_add help won't have APIC ID set,
>> > so set it according to socket/core/thread ids provided
>> > with device_add command.
>> >
>> > Signed-off-by: Igor Mammedov 
>> > ---
>> > v3:
>> >  - use %u for printing topo ids
>> > v2:
>> >  - add validity checks for socket-id/core-id/thread-id values
>> > ---
>> >  hw/i386/pc.c | 44 +---
>> >  1 file changed, 41 insertions(+), 3 deletions(-)
>> >
>> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
>> > index 24231ca..29da2d4 100644
>> > --- a/hw/i386/pc.c
>> > +++ b/hw/i386/pc.c
>> > @@ -1763,14 +1763,52 @@ static void pc_cpu_pre_plug(HotplugHandler 
>> > *hotplug_dev,
>> >  DeviceState *dev, Error **errp)
>> >  {
>> >  int idx;
>> > +CPUArchId *cpu_slot;
>> >  X86CPUTopoInfo topo;
>> >  X86CPU *cpu = X86_CPU(dev);
>> >  PCMachineState *pcms = PC_MACHINE(hotplug_dev);
>> > -CPUArchId *cpu_slot = pc_find_cpu_slot(pcms, CPU(dev), );
>> >  
>> > +/* if APIC ID is not set, set it based on socket/core/thread 
>> > properties */
>> > +if (cpu->apic_id == UNASSIGNED_APIC_ID) {
>> > +int max_socket = (max_cpus - 1) / smp_threads / smp_cores;
>> > +
>> > +if (cpu->socket_id < 0) {
>> > +error_setg(errp, "CPU socket-id is not set");
>> > +return;
>> > +} else if (cpu->socket_id > max_socket) {
>> > +error_setg(errp, "Invalid CPU socket-id: %u must be in range 
>> > 0:%u",
>> > +   cpu->socket_id, max_socket);
>> > +return;
>> > +}
>> > +if (cpu->core_id < 0) {
>> > +error_setg(errp, "CPU core-id is not set");
>> > +return;
>> > +} else if (cpu->core_id > (smp_cores - 1)) {
>> > +error_setg(errp, "Invalid CPU core-id: %u must be in range 
>> > 0:%u",
>> > +   cpu->core_id, smp_cores - 1);
>> > +return;
>> > +}
>> > +if (cpu->thread_id < 0) {
>> > +error_setg(errp, "CPU thread-id is not set");
>> > +return;
>> > +} else if (cpu->thread_id > (smp_threads - 1)) {
>> > +error_setg(errp, "Invalid CPU thread-id: %u must be in range 
>> > 0:%u",
>> > +   cpu->thread_id, smp_threads - 1);
>> > +return;
>> > +}
>> 
>> Just curoious, when any of these values < 0, the only other values that they
>> can take is -1, right ?
>
> No, the user might have set thread=-2 explicitly, and this is
> where we validate the user-provided values.
>
>> I am wondering why decided to do the check differently
>> for in the preceeding patch.
>
> Because on both cases we want thread_id <= -2 to generate an
> error message. On patch 06/19, thread_id == -1 is one case where
> the error message will be skipped (but thread_id == -2 will
> generate an error). In this patch, all negative values should
> generate an error.

Understood, this makes sense. Thanks.

>> 
>> Bandan
>> > +topo.pkg_id = cpu->socket_id;
>> > +topo.core_id = cpu->core_id;
>> > +topo.smt_id = cpu->thread_id;
>> > +cpu->apic_id = apicid_from_topo_ids(smp_cores, smp_threads, 
>> > );
>> > +}
>> > +
>> > +cpu_slot = pc_find_cpu_slot(pcms, CPU(dev), );
>> >  if (!cpu_slot) {
>> > -error_setg(errp, "Invalid CPU index with APIC ID (%" PRIu32
>> > -   "), valid range 0:%d", cpu->apic_id,
>> > +x86_topo_ids_from_apicid(cpu->apic_id, smp_cores, smp_threads, 
>> > );
>> > +error_setg(errp, "Invalid CPU[socket: %d, core: %d, thread: %d] 
>> > with"
>> > +  " APIC ID (%" PRIu32 "), valid index range 0:%d",
>> > +   topo.pkg_id, topo.core_id, topo.smt_id, cpu->apic_id,
>> > pcms->possible_cpus->len - 1);
>> >  return;
>> >  }

Re: [Qemu-devel] [PATCH v3 17/19] target-i386: fix apic object leak when CPU is deleted

[Bandan Das]

Igor Mammedov  writes:

> Signed-off-by: Igor Mammedov 
> ---
>  target-i386/cpu.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index 04c0b79..2fa445d 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -2765,6 +2765,7 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error 
> **errp)
>  
>  object_property_add_child(OBJECT(cpu), "lapic",
>OBJECT(cpu->apic_state), _abort);
> +object_unref(OBJECT(cpu->apic_state));

Just about to write down my concern but I noticed you guys have already sorted 
this
out :)

>  qdev_prop_set_uint8(cpu->apic_state, "id", cpu->apic_id);
>  /* TODO: convert to link<> */

Re: [Qemu-devel] [PATCH v3 12/19] apic: move MAX_APICS check to 'apic' class

[Bandan Das]

Igor Mammedov  writes:

> MAX_APICS is only used by child 'apic' class and not
> by its parent TYPE_APIC_COMMON or any other derived
> class.
> Move check into end user 'apic' class so it won't
> get in the way of other APIC implementations
> if they support more then MAX_APICS.
>
> Signed-off-by: Igor Mammedov 
> ---
>  hw/intc/apic.c  | 10 ++
>  hw/intc/apic_common.c   |  8 
>  include/hw/i386/apic_internal.h |  4 +---
>  3 files changed, 11 insertions(+), 11 deletions(-)
>
> diff --git a/hw/intc/apic.c b/hw/intc/apic.c
> index e1ab935..b0d237b 100644
> --- a/hw/intc/apic.c
> +++ b/hw/intc/apic.c
> @@ -28,7 +28,9 @@
>  #include "trace.h"
>  #include "hw/i386/pc.h"
>  #include "hw/i386/apic-msidef.h"
> +#include "qapi/error.h"
>  
> +#define MAX_APICS 255
>  #define MAX_APIC_WORDS 8
>  
>  #define SYNC_FROM_VAPIC 0x1
> @@ -869,6 +871,14 @@ static const MemoryRegionOps apic_io_ops = {
>  static void apic_realize(DeviceState *dev, Error **errp)
>  {
>  APICCommonState *s = APIC_COMMON(dev);
> +static int apic_no;

Can this be a global ? I understand there are no other users but
maybe it fits alongside local_apics.

> +if (apic_no >= MAX_APICS) {
> +error_setg(errp, "%s initialization failed.",
> +   object_get_typename(OBJECT(dev)));
> +return;
> +}
> +s->idx = apic_no++;

Is there a possibility of a race here with the apic removal path ?

Bandan

>  memory_region_init_io(>io_memory, OBJECT(s), _io_ops, s, 
> "apic-msi",
>APIC_SPACE_SIZE);
> diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
> index e6eb694..fd425d1 100644
> --- a/hw/intc/apic_common.c
> +++ b/hw/intc/apic_common.c
> @@ -299,14 +299,6 @@ static void apic_common_realize(DeviceState *dev, Error 
> **errp)
>  APICCommonState *s = APIC_COMMON(dev);
>  APICCommonClass *info;
>  static DeviceState *vapic;
> -static int apic_no;
> -
> -if (apic_no >= MAX_APICS) {
> -error_setg(errp, "%s initialization failed.",
> -   object_get_typename(OBJECT(dev)));
> -return;
> -}
> -s->idx = apic_no++;
>  
>  info = APIC_COMMON_GET_CLASS(s);
>  info->realize(dev, errp);
> diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
> index 74fe935..5d3be9a 100644
> --- a/include/hw/i386/apic_internal.h
> +++ b/include/hw/i386/apic_internal.h
> @@ -120,8 +120,6 @@
>  #define VAPIC_ENABLE_BIT0
>  #define VAPIC_ENABLE_MASK   (1 << VAPIC_ENABLE_BIT)
>  
> -#define MAX_APICS 255
> -
>  typedef struct APICCommonState APICCommonState;
>  
>  #define TYPE_APIC_COMMON "apic-common"
> @@ -175,7 +173,7 @@ struct APICCommonState {
>  uint32_t initial_count;
>  int64_t initial_count_load_time;
>  int64_t next_time;
> -int idx;
> +int idx; /* not actually common, used only by 'apic' derived class */
>  QEMUTimer *timer;
>  int64_t timer_expiry;
>  int sipi_vector;

Re: [Qemu-devel] [PATCH v3 10/19] pc: register created initial and hotpluged CPUs in one place pc_cpu_plug()

[Eduardo Habkost]

On Wed, Jul 13, 2016 at 06:32:27PM -0400, Bandan Das wrote:
> Igor Mammedov  writes:
> 
> > consolidate possible_cpus array management in pc_cpu_plug()
> > for smp_cpus, coldplugged with -device and hotplugged with
> > device_add.
> 
> So, this takes care of the hotplug case and 09/19 took care of the
> coldplug case, right ? If yes, we should probably modify this commit
> message a little.

This makes pc_cpu_plug() take care of both: now it will handle
the coldplug case also (in addition to the hotplug case, that it
already handled). I don't understand what you mean.

Are you talking about the rtc_set_memory() call, only? This seems
to be the only hotplug-specific code that still remains, in this
patch.

> 
> Bandan
> > Signed-off-by: Igor Mammedov 
> > ---
> >  hw/i386/pc.c | 25 +
> >  1 file changed, 9 insertions(+), 16 deletions(-)
> >
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index 3206572..0f85b56 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -1142,7 +1142,6 @@ void pc_cpus_init(PCMachineState *pcms)
> >  if (i < smp_cpus) {
> >  cpu = pc_new_cpu(typename, x86_cpu_apic_id_from_index(i),
> >   _fatal);
> > -pcms->possible_cpus->cpus[i].cpu = CPU(cpu);
> >  object_unref(OBJECT(cpu));
> >  }
> >  }
> > @@ -1697,25 +1696,19 @@ static void pc_cpu_plug(HotplugHandler *hotplug_dev,
> >  Error *local_err = NULL;
> >  PCMachineState *pcms = PC_MACHINE(hotplug_dev);
> >  
> > -if (!dev->hotplugged) {
> > -goto out;
> > -}
> > -
> > -if (!pcms->acpi_dev) {
> > -error_setg(_err,
> > -   "cpu hotplug is not enabled: missing acpi device");
> > -goto out;
> > +if (pcms->acpi_dev) {
> > +hhc = HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev);
> > +hhc->plug(HOTPLUG_HANDLER(pcms->acpi_dev), dev, _err);
> > +if (local_err) {
> > +goto out;
> > +}
> >  }
> >  
> > -hhc = HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev);
> > -hhc->plug(HOTPLUG_HANDLER(pcms->acpi_dev), dev, _err);
> > -if (local_err) {
> > -goto out;
> > +if (dev->hotplugged) {
> > +/* increment the number of CPUs */
> > +rtc_set_memory(pcms->rtc, 0x5f, rtc_get_memory(pcms->rtc, 0x5f) + 
> > 1);
> >  }
> >  
> > -/* increment the number of CPUs */
> > -rtc_set_memory(pcms->rtc, 0x5f, rtc_get_memory(pcms->rtc, 0x5f) + 1);
> > -
> >  found_cpu = pc_find_cpu_slot(pcms, CPU(dev), NULL);
> >  found_cpu->cpu = CPU(dev);
> >  out:

-- 
Eduardo

Re: [Qemu-devel] [PATCH v3 07/19] pc: set APIC ID based on socket/core/thread ids if it's not been set yet

[Eduardo Habkost]

On Wed, Jul 13, 2016 at 06:24:17PM -0400, Bandan Das wrote:
> Igor Mammedov  writes:
> 
> > CPU added with device_add help won't have APIC ID set,
> > so set it according to socket/core/thread ids provided
> > with device_add command.
> >
> > Signed-off-by: Igor Mammedov 
> > ---
> > v3:
> >  - use %u for printing topo ids
> > v2:
> >  - add validity checks for socket-id/core-id/thread-id values
> > ---
> >  hw/i386/pc.c | 44 +---
> >  1 file changed, 41 insertions(+), 3 deletions(-)
> >
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index 24231ca..29da2d4 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -1763,14 +1763,52 @@ static void pc_cpu_pre_plug(HotplugHandler 
> > *hotplug_dev,
> >  DeviceState *dev, Error **errp)
> >  {
> >  int idx;
> > +CPUArchId *cpu_slot;
> >  X86CPUTopoInfo topo;
> >  X86CPU *cpu = X86_CPU(dev);
> >  PCMachineState *pcms = PC_MACHINE(hotplug_dev);
> > -CPUArchId *cpu_slot = pc_find_cpu_slot(pcms, CPU(dev), );
> >  
> > +/* if APIC ID is not set, set it based on socket/core/thread 
> > properties */
> > +if (cpu->apic_id == UNASSIGNED_APIC_ID) {
> > +int max_socket = (max_cpus - 1) / smp_threads / smp_cores;
> > +
> > +if (cpu->socket_id < 0) {
> > +error_setg(errp, "CPU socket-id is not set");
> > +return;
> > +} else if (cpu->socket_id > max_socket) {
> > +error_setg(errp, "Invalid CPU socket-id: %u must be in range 
> > 0:%u",
> > +   cpu->socket_id, max_socket);
> > +return;
> > +}
> > +if (cpu->core_id < 0) {
> > +error_setg(errp, "CPU core-id is not set");
> > +return;
> > +} else if (cpu->core_id > (smp_cores - 1)) {
> > +error_setg(errp, "Invalid CPU core-id: %u must be in range 
> > 0:%u",
> > +   cpu->core_id, smp_cores - 1);
> > +return;
> > +}
> > +if (cpu->thread_id < 0) {
> > +error_setg(errp, "CPU thread-id is not set");
> > +return;
> > +} else if (cpu->thread_id > (smp_threads - 1)) {
> > +error_setg(errp, "Invalid CPU thread-id: %u must be in range 
> > 0:%u",
> > +   cpu->thread_id, smp_threads - 1);
> > +return;
> > +}
> 
> Just curoious, when any of these values < 0, the only other values that they
> can take is -1, right ?

No, the user might have set thread=-2 explicitly, and this is
where we validate the user-provided values.

> I am wondering why decided to do the check differently
> for in the preceeding patch.

Because on both cases we want thread_id <= -2 to generate an
error message. On patch 06/19, thread_id == -1 is one case where
the error message will be skipped (but thread_id == -2 will
generate an error). In this patch, all negative values should
generate an error.

> 
> Bandan
> > +topo.pkg_id = cpu->socket_id;
> > +topo.core_id = cpu->core_id;
> > +topo.smt_id = cpu->thread_id;
> > +cpu->apic_id = apicid_from_topo_ids(smp_cores, smp_threads, );
> > +}
> > +
> > +cpu_slot = pc_find_cpu_slot(pcms, CPU(dev), );
> >  if (!cpu_slot) {
> > -error_setg(errp, "Invalid CPU index with APIC ID (%" PRIu32
> > -   "), valid range 0:%d", cpu->apic_id,
> > +x86_topo_ids_from_apicid(cpu->apic_id, smp_cores, smp_threads, 
> > );
> > +error_setg(errp, "Invalid CPU[socket: %d, core: %d, thread: %d] 
> > with"
> > +  " APIC ID (%" PRIu32 "), valid index range 0:%d",
> > +   topo.pkg_id, topo.core_id, topo.smt_id, cpu->apic_id,
> > pcms->possible_cpus->len - 1);
> >  return;
> >  }

-- 
Eduardo

Re: [Qemu-devel] [PATCH v3 10/19] pc: register created initial and hotpluged CPUs in one place pc_cpu_plug()

[Bandan Das]

Igor Mammedov  writes:

> consolidate possible_cpus array management in pc_cpu_plug()
> for smp_cpus, coldplugged with -device and hotplugged with
> device_add.

So, this takes care of the hotplug case and 09/19 took care of the
coldplug case, right ? If yes, we should probably modify this commit
message a little.

Bandan
> Signed-off-by: Igor Mammedov 
> ---
>  hw/i386/pc.c | 25 +
>  1 file changed, 9 insertions(+), 16 deletions(-)
>
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 3206572..0f85b56 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -1142,7 +1142,6 @@ void pc_cpus_init(PCMachineState *pcms)
>  if (i < smp_cpus) {
>  cpu = pc_new_cpu(typename, x86_cpu_apic_id_from_index(i),
>   _fatal);
> -pcms->possible_cpus->cpus[i].cpu = CPU(cpu);
>  object_unref(OBJECT(cpu));
>  }
>  }
> @@ -1697,25 +1696,19 @@ static void pc_cpu_plug(HotplugHandler *hotplug_dev,
>  Error *local_err = NULL;
>  PCMachineState *pcms = PC_MACHINE(hotplug_dev);
>  
> -if (!dev->hotplugged) {
> -goto out;
> -}
> -
> -if (!pcms->acpi_dev) {
> -error_setg(_err,
> -   "cpu hotplug is not enabled: missing acpi device");
> -goto out;
> +if (pcms->acpi_dev) {
> +hhc = HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev);
> +hhc->plug(HOTPLUG_HANDLER(pcms->acpi_dev), dev, _err);
> +if (local_err) {
> +goto out;
> +}
>  }
>  
> -hhc = HOTPLUG_HANDLER_GET_CLASS(pcms->acpi_dev);
> -hhc->plug(HOTPLUG_HANDLER(pcms->acpi_dev), dev, _err);
> -if (local_err) {
> -goto out;
> +if (dev->hotplugged) {
> +/* increment the number of CPUs */
> +rtc_set_memory(pcms->rtc, 0x5f, rtc_get_memory(pcms->rtc, 0x5f) + 1);
>  }
>  
> -/* increment the number of CPUs */
> -rtc_set_memory(pcms->rtc, 0x5f, rtc_get_memory(pcms->rtc, 0x5f) + 1);
> -
>  found_cpu = pc_find_cpu_slot(pcms, CPU(dev), NULL);
>  found_cpu->cpu = CPU(dev);
>  out:

Re: [Qemu-devel] [PATCH v3 07/19] pc: set APIC ID based on socket/core/thread ids if it's not been set yet

[Bandan Das]

Igor Mammedov  writes:

> CPU added with device_add help won't have APIC ID set,
> so set it according to socket/core/thread ids provided
> with device_add command.
>
> Signed-off-by: Igor Mammedov 
> ---
> v3:
>  - use %u for printing topo ids
> v2:
>  - add validity checks for socket-id/core-id/thread-id values
> ---
>  hw/i386/pc.c | 44 +---
>  1 file changed, 41 insertions(+), 3 deletions(-)
>
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 24231ca..29da2d4 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -1763,14 +1763,52 @@ static void pc_cpu_pre_plug(HotplugHandler 
> *hotplug_dev,
>  DeviceState *dev, Error **errp)
>  {
>  int idx;
> +CPUArchId *cpu_slot;
>  X86CPUTopoInfo topo;
>  X86CPU *cpu = X86_CPU(dev);
>  PCMachineState *pcms = PC_MACHINE(hotplug_dev);
> -CPUArchId *cpu_slot = pc_find_cpu_slot(pcms, CPU(dev), );
>  
> +/* if APIC ID is not set, set it based on socket/core/thread properties 
> */
> +if (cpu->apic_id == UNASSIGNED_APIC_ID) {
> +int max_socket = (max_cpus - 1) / smp_threads / smp_cores;
> +
> +if (cpu->socket_id < 0) {
> +error_setg(errp, "CPU socket-id is not set");
> +return;
> +} else if (cpu->socket_id > max_socket) {
> +error_setg(errp, "Invalid CPU socket-id: %u must be in range 
> 0:%u",
> +   cpu->socket_id, max_socket);
> +return;
> +}
> +if (cpu->core_id < 0) {
> +error_setg(errp, "CPU core-id is not set");
> +return;
> +} else if (cpu->core_id > (smp_cores - 1)) {
> +error_setg(errp, "Invalid CPU core-id: %u must be in range 0:%u",
> +   cpu->core_id, smp_cores - 1);
> +return;
> +}
> +if (cpu->thread_id < 0) {
> +error_setg(errp, "CPU thread-id is not set");
> +return;
> +} else if (cpu->thread_id > (smp_threads - 1)) {
> +error_setg(errp, "Invalid CPU thread-id: %u must be in range 
> 0:%u",
> +   cpu->thread_id, smp_threads - 1);
> +return;
> +}

Just curoious, when any of these values < 0, the only other values that they
can take is -1, right ? I am wondering why decided to do the check differently
for in the preceeding patch.

Bandan
> +topo.pkg_id = cpu->socket_id;
> +topo.core_id = cpu->core_id;
> +topo.smt_id = cpu->thread_id;
> +cpu->apic_id = apicid_from_topo_ids(smp_cores, smp_threads, );
> +}
> +
> +cpu_slot = pc_find_cpu_slot(pcms, CPU(dev), );
>  if (!cpu_slot) {
> -error_setg(errp, "Invalid CPU index with APIC ID (%" PRIu32
> -   "), valid range 0:%d", cpu->apic_id,
> +x86_topo_ids_from_apicid(cpu->apic_id, smp_cores, smp_threads, 
> );
> +error_setg(errp, "Invalid CPU[socket: %d, core: %d, thread: %d] with"
> +  " APIC ID (%" PRIu32 "), valid index range 0:%d",
> +   topo.pkg_id, topo.core_id, topo.smt_id, cpu->apic_id,
> pcms->possible_cpus->len - 1);
>  return;
>  }

Re: [Qemu-devel] [PATCH v3 06/19] target-i386: add socket/core/thread properties to X86CPU

[Bandan Das]

Igor Mammedov  writes:

> these properties will be used by as address where to plug
> CPU with help -device/device_add commands.
>
> Signed-off-by: Igor Mammedov 
> ---
> v3:
>   - use %u for printing topo ids
>   - add to error message topo ids from set apic_id
> v2:
>   - rename socket/core/thread properties to socket-id/core-id/thread-id
>   - add mismatch checks for apic_id and socket-id/core-id/thread-id
> in case both are set
> ---
>  hw/i386/pc.c  | 29 +
>  target-i386/cpu.c |  6 ++
>  target-i386/cpu.h |  4 
>  3 files changed, 39 insertions(+)
>
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 99dfbe0..24231ca 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -1763,6 +1763,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
>  DeviceState *dev, Error **errp)
>  {
>  int idx;
> +X86CPUTopoInfo topo;
>  X86CPU *cpu = X86_CPU(dev);
>  PCMachineState *pcms = PC_MACHINE(hotplug_dev);
>  CPUArchId *cpu_slot = pc_find_cpu_slot(pcms, CPU(dev), );
> @@ -1780,6 +1781,34 @@ static void pc_cpu_pre_plug(HotplugHandler 
> *hotplug_dev,
> cpu->apic_id);
>  return;
>  }
> +
> +/* if 'address' properties socket-id/core-id/thread-id are not set, set 
> them
> + * so that query_hotpluggable_cpus would show correct values
> + */
> +/* TODO: move socket_id/core_id/thread_id checks into x86_cpu_realizefn()
> + * once -smp refactoring is complete and there will be CPU private
> + * CPUState::nr_cores and CPUState::nr_threads fields instead of globals 
> */
> +x86_topo_ids_from_apicid(cpu->apic_id, smp_cores, smp_threads, );
> +if (cpu->socket_id != -1 && cpu->socket_id != topo.pkg_id) {
> +error_setg(errp, "property socket-id: %u doesn't match set apic-id:"
> +" 0x%x (socket-id: %u)", cpu->socket_id, cpu->apic_id, 
> topo.pkg_id);
> +return;
> +}
> +cpu->socket_id = topo.pkg_id;
> +
> +if (cpu->core_id != -1 && cpu->core_id != topo.core_id) {
> +error_setg(errp, "property core-id: %u doesn't match set apic-id:"
> +" 0x%x (core-id: %u)", cpu->core_id, cpu->apic_id, topo.core_id);
> +return;
> +}
> +cpu->core_id = topo.core_id;
> +
> +if (cpu->thread_id != -1 && cpu->thread_id != topo.smt_id) {
> +error_setg(errp, "property thread-id: %u doesn't match set apic-id:"
> +" 0x%x (thread-id: %u)", cpu->thread_id, cpu->apic_id, 
> topo.smt_id);
> +return;
> +}
> +cpu->thread_id = topo.smt_id;
>  }

What is the case where these fields are already populated ? When the cpus are 
online
at system boot followed by an unplug and then a hotplug ?

>  static void pc_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index 9294b3d..1ec40a0 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -3164,8 +3164,14 @@ static Property x86_cpu_properties[] = {
>  #ifdef CONFIG_USER_ONLY
>  /* apic_id = 0 by default for *-user, see commit 9886e834 */
>  DEFINE_PROP_UINT32("apic-id", X86CPU, apic_id, 0),
> +DEFINE_PROP_INT32("thread-id", X86CPU, thread_id, 0),
> +DEFINE_PROP_INT32("core-id", X86CPU, core_id, 0),
> +DEFINE_PROP_INT32("socket-id", X86CPU, socket_id, 0),
>  #else
>  DEFINE_PROP_UINT32("apic-id", X86CPU, apic_id, UNASSIGNED_APIC_ID),
> +DEFINE_PROP_INT32("thread-id", X86CPU, thread_id, -1),
> +DEFINE_PROP_INT32("core-id", X86CPU, core_id, -1),
> +DEFINE_PROP_INT32("socket-id", X86CPU, socket_id, -1),

Are these values (including the UNASSIGNED_APIC_ID) mandated by spec
or just convenient values ?

Bandan

>  #endif
>  DEFINE_PROP_BOOL("pmu", X86CPU, enable_pmu, false),
>  { .name  = "hv-spinlocks", .info  = _prop_spinlocks },
> diff --git a/target-i386/cpu.h b/target-i386/cpu.h
> index 00de199..0612181 100644
> --- a/target-i386/cpu.h
> +++ b/target-i386/cpu.h
> @@ -1193,6 +1193,10 @@ struct X86CPU {
>  Notifier machine_done;
>  
>  struct kvm_msrs *kvm_msr_buf;
> +
> +int32_t socket_id;
> +int32_t core_id;
> +int32_t thread_id;
>  };
>  
>  static inline X86CPU *x86_env_get_cpu(CPUX86State *env)

Re: [Qemu-devel] [PATCH v8 4/5] generic-loader: Add a generic loader

[Alistair Francis]

On Wed, Jul 13, 2016 at 2:09 PM, Peter Maydell  wrote:
> On 13 July 2016 at 21:30, Alistair Francis  
> wrote:
>> On Wed, Jul 13, 2016 at 12:44 PM, Peter Maydell
>>  wrote:
>>> On 13 July 2016 at 18:45, Alistair Francis  
>>> wrote:
 On Tue, Jul 12, 2016 at 9:39 AM, Peter Maydell  
 wrote:
> On 2 July 2016 at 02:07, Alistair Francis  
> wrote:
>> +qemu_register_reset(generic_loader_reset, dev);
>
> What's wrong with a device reset function set via dc->reset ?

 This allows setting values from the HMP command line interface once
 the machine is running. The dc->reset isn't applied in that case.
>>>
>>> I don't understand this -- could you explain in a bit
>>> more detail, please?
>>
>> So this loading system is just a device, which means that it can be
>> loaded whenever a normal device would be loaded. The more interesting
>> option is using this via the command line but it is also possible to
>> use the QEMU Monitor once QEMU is running to call this.
>>
>> A user can use the device_add function to add set values once QEMU is
>> running. The problem with that though is that the qdev_device_add()
>> function doesn't connect the reset function. I initially had a patch
>> to do this "qdev-monitor.c: Register reset function if the device has
>> one" but it was decided to just register the reset function in the
>> realise instead.
>
> I'm really surprised that device_add doesn't wire up the device's
> reset method. This sounds to me like it's a bug -- adding a device
> dynamically via the monitor shouldn't behave any differently
> to adding it via the command line.

I thought the same as you, but there was a general consensus that
registering the reset there was not correct as there are issues with
bus-less devices. I think this comes back to the infrastructure not
being ready to connect all devices via the device_add, hopefully in
the future this can be improved on.

>
> If we are doing this to avoid a bug (and it's too risky
> to fix that bug at this point) then we should have a comment
> explaining why we're not using dc->reset, so we can fix this
> when we do fix device_add.

I wouldn't call it a bug, I would call it more of a limitation. I'll
add a comment.

Thanks,

Alistair

>
> thanks
> -- PMM
>

Re: [Qemu-devel] [PATCH v3 04/19] pc: cpu: consolidate apic-id validity checks in pc_cpu_pre_plug()

[Bandan Das]

Igor Mammedov  writes:

> Machine code knows about all possible APIC IDs so use that
> instead of hack which does O(n^2) complexity duplicate
> checks, interating over global CPUs list.
> As result duplicate check is done only once with O(log n) complexity.
>
> Signed-off-by: Igor Mammedov 
> ---
>  hw/i386/pc.c  | 44 
>  target-i386/cpu.c | 13 -
>  2 files changed, 32 insertions(+), 25 deletions(-)
>
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 3c42d84..99dfbe0 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -1071,18 +1071,6 @@ void pc_hot_add_cpu(const int64_t id, Error **errp)
>  return;
>  }
>  
> -if (cpu_exists(apic_id)) {
> -error_setg(errp, "Unable to add CPU: %" PRIi64
> -   ", it already exists", id);
> -return;
> -}
> -
> -if (id >= max_cpus) {
> -error_setg(errp, "Unable to add CPU: %" PRIi64
> -   ", max allowed: %d", id, max_cpus - 1);
> -return;
> -}
> -
>  if (apic_id >= ACPI_CPU_HOTPLUG_ID_LIMIT) {
>  error_setg(errp, "Unable to add CPU: %" PRIi64
> ", resulting APIC ID (%" PRIi64 ") is too large",
> @@ -1771,6 +1759,37 @@ static void pc_cpu_unplug_cb(HotplugHandler 
> *hotplug_dev,
>  error_propagate(errp, local_err);
>  }
>  
> +static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
> +DeviceState *dev, Error **errp)
> +{
> +int idx;
> +X86CPU *cpu = X86_CPU(dev);
> +PCMachineState *pcms = PC_MACHINE(hotplug_dev);
> +CPUArchId *cpu_slot = pc_find_cpu_slot(pcms, CPU(dev), );
> +
> +if (!cpu_slot) {
> +error_setg(errp, "Invalid CPU index with APIC ID (%" PRIu32
> +   "), valid range 0:%d", cpu->apic_id,
> +   pcms->possible_cpus->len - 1);
> +return;
> +}
> +
> +if (cpu_slot->cpu) {
> +error_setg(errp, "CPU[%ld] with APIC ID %" PRIu32 " exists",
> +   cpu_slot - pcms->possible_cpus->cpus,
> +   cpu->apic_id);
> +return;
> +}
> +}
> +
> +static void pc_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
> +  DeviceState *dev, Error **errp)
> +{
> +if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> +pc_cpu_pre_plug(hotplug_dev, dev, errp);
> +}
> +}
>

Will this path be invoked for non TYPE_CPU callbacks too ? I was just wondering
if there should be an "else" error message.

Bandan

>  static void pc_machine_device_plug_cb(HotplugHandler *hotplug_dev,
>DeviceState *dev, Error **errp)
>  {
> @@ -2068,6 +2087,7 @@ static void pc_machine_class_init(ObjectClass *oc, void 
> *data)
>  mc->hot_add_cpu = pc_hot_add_cpu;
>  mc->max_cpus = 255;
>  mc->reset = pc_machine_reset;
> +hc->pre_plug = pc_machine_device_pre_plug_cb;
>  hc->plug = pc_machine_device_plug_cb;
>  hc->unplug_request = pc_machine_device_unplug_request_cb;
>  hc->unplug = pc_machine_device_unplug_cb;
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index e7319e3..9511474 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -1838,8 +1838,6 @@ static void x86_cpuid_set_apic_id(Object *obj, Visitor 
> *v, const char *name,
>  {
>  X86CPU *cpu = X86_CPU(obj);
>  DeviceState *dev = DEVICE(obj);
> -const int64_t min = 0;
> -const int64_t max = UINT32_MAX;
>  Error *error = NULL;
>  int64_t value;
>  
> @@ -1854,17 +1852,6 @@ static void x86_cpuid_set_apic_id(Object *obj, Visitor 
> *v, const char *name,
>  error_propagate(errp, error);
>  return;
>  }
> -if (value < min || value > max) {
> -error_setg(errp, "Property %s.%s doesn't take value %" PRId64
> -   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
> -   object_get_typename(obj), name, value, min, max);
> -return;
> -}
> -
> -if ((value != cpu->apic_id) && cpu_exists(value)) {
> -error_setg(errp, "CPU with APIC ID %" PRIi64 " exists", value);
> -return;
> -}
>  cpu->apic_id = value;
>  }

Re: [Qemu-devel] [PATCH v3 01/19] target-i386: cpu: use uint32_t for X86CPU.apic_id

[Bandan Das]

I know some of these have already been pulled. I just have some minor
questions/comments that shouldn't conflict.

Igor Mammedov  writes:

> Redo 9886e834 (target-i386: Require APIC ID to be explicitly set before
> CPU realize) in another way that doesn't use int64_t to detect
> if apic-id property has been set.
>
> Use the fact that 0x is the broadcast

I noticed this was UINT32_MAX in v2. Why is UINT32_MAX
not appropriate ?

> value that a CPU can't have and set default
> uint32_t apic_id to it instead of using int64_t.
>
> Later uint32_t apic_id will be used to drop custom
> property setter/getter in favor of static property.
>
> Signed-off-by: Igor Mammedov 
> ---
>  target-i386/cpu.c | 4 ++--
>  target-i386/cpu.h | 7 ++-
>  2 files changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index 5c69c43..e7319e3 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -2883,7 +2883,7 @@ static void x86_cpu_realizefn(DeviceState *dev, Error 
> **errp)
>  goto out;
>  }
>  
> -if (cpu->apic_id < 0) {
> +if (cpu->apic_id == UNASSIGNED_APIC_ID) {
>  error_setg(errp, "apic-id property was not initialized properly");
>  return;
>  }
> @@ -3154,7 +3154,7 @@ static void x86_cpu_initfn(Object *obj)
>  
>  #ifndef CONFIG_USER_ONLY
>  /* Any code creating new X86CPU objects have to set apic-id explicitly */
> -cpu->apic_id = -1;
> +cpu->apic_id = UNASSIGNED_APIC_ID;
>  #endif
>  
>  for (w = 0; w < FEATURE_WORDS; w++) {
> diff --git a/target-i386/cpu.h b/target-i386/cpu.h
> index 738958e..00de199 100644
> --- a/target-i386/cpu.h
> +++ b/target-i386/cpu.h
> @@ -835,6 +835,11 @@ typedef struct {
>  
>  #define NB_OPMASK_REGS 8
>  
> +/* CPU can't have 0x APIC ID, use that value to distinguish
> + * that APIC ID hasn't been set yet
> + */
> +#define UNASSIGNED_APIC_ID 0x
> +
>  typedef union X86LegacyXSaveArea {
>  struct {
>  uint16_t fcw;
> @@ -1163,7 +1168,7 @@ struct X86CPU {
>  bool expose_kvm;
>  bool migratable;
>  bool host_features;
> -int64_t apic_id;
> +uint32_t apic_id;
>  
>  /* if true the CPUID code directly forward host cache leaves to the 
> guest */
>  bool cache_info_passthrough;

Re: [Qemu-devel] [PATCH v8 4/5] generic-loader: Add a generic loader

[Peter Maydell]

On 13 July 2016 at 21:30, Alistair Francis  wrote:
> On Wed, Jul 13, 2016 at 12:44 PM, Peter Maydell
>  wrote:
>> On 13 July 2016 at 18:45, Alistair Francis  
>> wrote:
>>> On Tue, Jul 12, 2016 at 9:39 AM, Peter Maydell  
>>> wrote:
 On 2 July 2016 at 02:07, Alistair Francis  
 wrote:
> +qemu_register_reset(generic_loader_reset, dev);

 What's wrong with a device reset function set via dc->reset ?
>>>
>>> This allows setting values from the HMP command line interface once
>>> the machine is running. The dc->reset isn't applied in that case.
>>
>> I don't understand this -- could you explain in a bit
>> more detail, please?
>
> So this loading system is just a device, which means that it can be
> loaded whenever a normal device would be loaded. The more interesting
> option is using this via the command line but it is also possible to
> use the QEMU Monitor once QEMU is running to call this.
>
> A user can use the device_add function to add set values once QEMU is
> running. The problem with that though is that the qdev_device_add()
> function doesn't connect the reset function. I initially had a patch
> to do this "qdev-monitor.c: Register reset function if the device has
> one" but it was decided to just register the reset function in the
> realise instead.

I'm really surprised that device_add doesn't wire up the device's
reset method. This sounds to me like it's a bug -- adding a device
dynamically via the monitor shouldn't behave any differently
to adding it via the command line.

If we are doing this to avoid a bug (and it's too risky
to fix that bug at this point) then we should have a comment
explaining why we're not using dc->reset, so we can fix this
when we do fix device_add.

thanks
-- PMM

[Qemu-devel] [PATCH v3 12/12] tcg: Make tb_flush() thread safe

[Sergey Fedorov]

From: Sergey Fedorov 

Use async_safe_run_on_cpu() to make tb_flush() thread safe.

Signed-off-by: Sergey Fedorov 
Signed-off-by: Sergey Fedorov 

---
Changes in v3:
 - 'tb_flushed' removed
Changes in v2:
 - stale comment about unsafe tb_flush() removed
---
 cpu-exec.c| 14 +-
 include/qom/cpu.h |  2 --
 translate-all.c   | 14 --
 3 files changed, 9 insertions(+), 21 deletions(-)

diff --git a/cpu-exec.c b/cpu-exec.c
index b840e1d2dd41..0b7614ffcc9b 100644
--- a/cpu-exec.c
+++ b/cpu-exec.c
@@ -203,20 +203,16 @@ static void cpu_exec_nocache(CPUState *cpu, int 
max_cycles,
  TranslationBlock *orig_tb, bool ignore_icount)
 {
 TranslationBlock *tb;
-bool old_tb_flushed;
 
 /* Should never happen.
We only end up here when an existing TB is too long.  */
 if (max_cycles > CF_COUNT_MASK)
 max_cycles = CF_COUNT_MASK;
 
-old_tb_flushed = cpu->tb_flushed;
-cpu->tb_flushed = false;
 tb = tb_gen_code(cpu, orig_tb->pc, orig_tb->cs_base, orig_tb->flags,
  max_cycles | CF_NOCACHE
  | (ignore_icount ? CF_IGNORE_ICOUNT : 0));
-tb->orig_tb = cpu->tb_flushed ? NULL : orig_tb;
-cpu->tb_flushed |= old_tb_flushed;
+tb->orig_tb = orig_tb;
 /* execute the generated code */
 trace_exec_tb_nocache(tb, tb->pc);
 cpu_tb_exec(cpu, tb);
@@ -338,13 +334,6 @@ static inline TranslationBlock *tb_find_fast(CPUState *cpu,
  tb->flags != flags)) {
 tb = tb_find_slow(cpu, pc, cs_base, flags);
 }
-if (cpu->tb_flushed) {
-/* Ensure that no TB jump will be modified as the
- * translation buffer has been flushed.
- */
-*last_tb = NULL;
-cpu->tb_flushed = false;
-}
 #ifndef CONFIG_USER_ONLY
 /* We don't take care of direct jumps when address mapping changes in
  * system emulation. So it's not safe to make a direct jump to a TB
@@ -619,7 +608,6 @@ int cpu_exec(CPUState *cpu)
 }
 
 last_tb = NULL; /* forget the last executed TB after exception */
-cpu->tb_flushed = false; /* reset before first TB lookup */
 for(;;) {
 cpu_handle_interrupt(cpu, _tb);
 tb = tb_find_fast(cpu, _tb, tb_exit);
diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index ab67bf2ba19f..9af4420d2bb5 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -258,7 +258,6 @@ struct qemu_work_item {
  * @crash_occurred: Indicates the OS reported a crash (panic) for this CPU
  * @tcg_exit_req: Set to force TCG to stop executing linked TBs for this
  *   CPU and return to its top level loop.
- * @tb_flushed: Indicates the translation buffer has been flushed.
  * @singlestep_enabled: Flags for single-stepping.
  * @icount_extra: Instructions until next timer event.
  * @icount_decr: Number of cycles left, with interrupt flag in high bit.
@@ -310,7 +309,6 @@ struct CPUState {
 bool unplug;
 bool crash_occurred;
 bool exit_request;
-bool tb_flushed;
 uint32_t interrupt_request;
 int singlestep_enabled;
 int64_t icount_extra;
diff --git a/translate-all.c b/translate-all.c
index 0d47c1c0cf82..b04e9113c18c 100644
--- a/translate-all.c
+++ b/translate-all.c
@@ -831,8 +831,7 @@ static void page_flush_tb(void)
 }
 
 /* flush all the translation blocks */
-/* XXX: tb_flush is currently not thread safe */
-void tb_flush(CPUState *cpu)
+static void do_tb_flush(CPUState *cpu, void *data)
 {
 #if defined(DEBUG_FLUSH)
 printf("qemu: flush code_size=%ld nb_tbs=%d avg_tb_size=%ld\n",
@@ -849,7 +848,6 @@ void tb_flush(CPUState *cpu)
 
 CPU_FOREACH(cpu) {
 memset(cpu->tb_jmp_cache, 0, sizeof(cpu->tb_jmp_cache));
-cpu->tb_flushed = true;
 }
 
 qht_reset_size(_ctx.tb_ctx.htable, CODE_GEN_HTABLE_SIZE);
@@ -861,6 +859,11 @@ void tb_flush(CPUState *cpu)
 tcg_ctx.tb_ctx.tb_flush_count++;
 }
 
+void tb_flush(CPUState *cpu)
+{
+async_safe_run_on_cpu(cpu, do_tb_flush, NULL);
+}
+
 #ifdef DEBUG_TB_CHECK
 
 static void
@@ -1163,9 +1166,8 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
  buffer_overflow:
 /* flush must be done */
 tb_flush(cpu);
-/* cannot fail at this point */
-tb = tb_alloc(pc);
-assert(tb != NULL);
+mmap_unlock();
+cpu_loop_exit(cpu);
 }
 
 gen_code_buf = tcg_ctx.code_gen_ptr;
-- 
1.9.1

[Qemu-devel] [PATCH v3 08/12] linux-user: Add qemu_cpu_is_self() and qemu_cpu_kick()

[Sergey Fedorov]

From: Sergey Fedorov 

Signed-off-by: Sergey Fedorov 
Signed-off-by: Sergey Fedorov 
---
 linux-user/main.c | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/linux-user/main.c b/linux-user/main.c
index 5ff0b20bad89..a8790ac63f68 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -3785,6 +3785,16 @@ void cpu_loop(CPUTLGState *env)
 
 THREAD CPUState *thread_cpu;
 
+bool qemu_cpu_is_self(CPUState *cpu)
+{
+return thread_cpu == cpu;
+}
+
+void qemu_cpu_kick(CPUState *cpu)
+{
+cpu_exit(cpu);
+}
+
 void task_settid(TaskState *ts)
 {
 if (ts->ts_tid == 0) {
-- 
1.9.1

[Qemu-devel] [PATCH v3 09/12] linux-user: Support CPU work queue

[Sergey Fedorov]

From: Sergey Fedorov 

Make CPU work core functions common between system and user-mode
emulation. User-mode does not have BQL, so process_queued_cpu_work() is
protected by 'exclusive_lock'.

Signed-off-by: Sergey Fedorov 
Signed-off-by: Sergey Fedorov 

---
Changes in v2:
 - 'qemu_work_cond' definition moved to cpu-exec-common.c
 - documentation commend for new public API added
---
 cpu-exec-common.c   | 85 
 cpus.c  | 86 +
 include/exec/exec-all.h | 17 ++
 linux-user/main.c   |  8 +
 4 files changed, 111 insertions(+), 85 deletions(-)

diff --git a/cpu-exec-common.c b/cpu-exec-common.c
index 0cb4ae60eff9..a233f0124559 100644
--- a/cpu-exec-common.c
+++ b/cpu-exec-common.c
@@ -77,3 +77,88 @@ void cpu_loop_exit_restore(CPUState *cpu, uintptr_t pc)
 }
 siglongjmp(cpu->jmp_env, 1);
 }
+
+QemuCond qemu_work_cond;
+
+static void queue_work_on_cpu(CPUState *cpu, struct qemu_work_item *wi)
+{
+qemu_mutex_lock(>work_mutex);
+if (cpu->queued_work_first == NULL) {
+cpu->queued_work_first = wi;
+} else {
+cpu->queued_work_last->next = wi;
+}
+cpu->queued_work_last = wi;
+wi->next = NULL;
+wi->done = false;
+qemu_mutex_unlock(>work_mutex);
+
+qemu_cpu_kick(cpu);
+}
+
+void run_on_cpu(CPUState *cpu, run_on_cpu_func func, void *data)
+{
+struct qemu_work_item wi;
+
+if (qemu_cpu_is_self(cpu)) {
+func(cpu, data);
+return;
+}
+
+wi.func = func;
+wi.data = data;
+wi.free = false;
+
+queue_work_on_cpu(cpu, );
+while (!atomic_mb_read()) {
+CPUState *self_cpu = current_cpu;
+
+qemu_cond_wait(_work_cond, qemu_get_cpu_work_mutex());
+current_cpu = self_cpu;
+}
+}
+
+void async_run_on_cpu(CPUState *cpu, run_on_cpu_func func, void *data)
+{
+struct qemu_work_item *wi;
+
+if (qemu_cpu_is_self(cpu)) {
+func(cpu, data);
+return;
+}
+
+wi = g_malloc0(sizeof(struct qemu_work_item));
+wi->func = func;
+wi->data = data;
+wi->free = true;
+
+queue_work_on_cpu(cpu, wi);
+}
+
+void process_queued_cpu_work(CPUState *cpu)
+{
+struct qemu_work_item *wi;
+
+if (cpu->queued_work_first == NULL) {
+return;
+}
+
+qemu_mutex_lock(>work_mutex);
+while (cpu->queued_work_first != NULL) {
+wi = cpu->queued_work_first;
+cpu->queued_work_first = wi->next;
+if (!cpu->queued_work_first) {
+cpu->queued_work_last = NULL;
+}
+qemu_mutex_unlock(>work_mutex);
+wi->func(cpu, wi->data);
+qemu_mutex_lock(>work_mutex);
+if (wi->free) {
+g_free(wi);
+} else {
+atomic_mb_set(>done, true);
+}
+}
+qemu_mutex_unlock(>work_mutex);
+qemu_cond_broadcast(_work_cond);
+}
diff --git a/cpus.c b/cpus.c
index 51fd8c18b4c8..282d7e399902 100644
--- a/cpus.c
+++ b/cpus.c
@@ -896,7 +896,6 @@ static QemuThread io_thread;
 static QemuCond qemu_cpu_cond;
 /* system init */
 static QemuCond qemu_pause_cond;
-static QemuCond qemu_work_cond;
 
 void qemu_init_cpu_loop(void)
 {
@@ -910,66 +909,11 @@ void qemu_init_cpu_loop(void)
 qemu_thread_get_self(_thread);
 }
 
-static QemuMutex *qemu_get_cpu_work_mutex(void)
+QemuMutex *qemu_get_cpu_work_mutex(void)
 {
 return _global_mutex;
 }
 
-static void queue_work_on_cpu(CPUState *cpu, struct qemu_work_item *wi)
-{
-qemu_mutex_lock(>work_mutex);
-if (cpu->queued_work_first == NULL) {
-cpu->queued_work_first = wi;
-} else {
-cpu->queued_work_last->next = wi;
-}
-cpu->queued_work_last = wi;
-wi->next = NULL;
-wi->done = false;
-qemu_mutex_unlock(>work_mutex);
-
-qemu_cpu_kick(cpu);
-}
-
-void run_on_cpu(CPUState *cpu, run_on_cpu_func func, void *data)
-{
-struct qemu_work_item wi;
-
-if (qemu_cpu_is_self(cpu)) {
-func(cpu, data);
-return;
-}
-
-wi.func = func;
-wi.data = data;
-wi.free = false;
-
-queue_work_on_cpu(cpu, );
-while (!atomic_mb_read()) {
-CPUState *self_cpu = current_cpu;
-
-qemu_cond_wait(_work_cond, qemu_get_cpu_work_mutex());
-current_cpu = self_cpu;
-}
-}
-
-void async_run_on_cpu(CPUState *cpu, run_on_cpu_func func, void *data)
-{
-struct qemu_work_item *wi;
-
-if (qemu_cpu_is_self(cpu)) {
-func(cpu, data);
-return;
-}
-
-wi = g_malloc0(sizeof(struct qemu_work_item));
-wi->func = func;
-wi->data = data;
-wi->free = true;
-
-queue_work_on_cpu(cpu, wi);
-}
-
 static void qemu_kvm_destroy_vcpu(CPUState *cpu)
 {
 if (kvm_destroy_vcpu(cpu) < 0) {
@@ -982,34 +926,6 @@ static void qemu_tcg_destroy_vcpu(CPUState *cpu)
 {
 }
 
-static void process_queued_cpu_work(CPUState *cpu)
-{
-struct qemu_work_item *wi;
-
-

[Qemu-devel] [PATCH v3 03/12] cpus: Move common code out of {async_, }run_on_cpu()

[Sergey Fedorov]

From: Sergey Fedorov 

Move the code common between run_on_cpu() and async_run_on_cpu() into a
new function queue_work_on_cpu().

Signed-off-by: Sergey Fedorov 
Signed-off-by: Sergey Fedorov 
Reviewed-by: Alex Bennée 
---
 cpus.c | 42 ++
 1 file changed, 18 insertions(+), 24 deletions(-)

diff --git a/cpus.c b/cpus.c
index 049c2d04e150..04687c85bcd4 100644
--- a/cpus.c
+++ b/cpus.c
@@ -910,6 +910,22 @@ void qemu_init_cpu_loop(void)
 qemu_thread_get_self(_thread);
 }
 
+static void queue_work_on_cpu(CPUState *cpu, struct qemu_work_item *wi)
+{
+qemu_mutex_lock(>work_mutex);
+if (cpu->queued_work_first == NULL) {
+cpu->queued_work_first = wi;
+} else {
+cpu->queued_work_last->next = wi;
+}
+cpu->queued_work_last = wi;
+wi->next = NULL;
+wi->done = false;
+qemu_mutex_unlock(>work_mutex);
+
+qemu_cpu_kick(cpu);
+}
+
 void run_on_cpu(CPUState *cpu, run_on_cpu_func func, void *data)
 {
 struct qemu_work_item wi;
@@ -923,18 +939,7 @@ void run_on_cpu(CPUState *cpu, run_on_cpu_func func, void 
*data)
 wi.data = data;
 wi.free = false;
 
-qemu_mutex_lock(>work_mutex);
-if (cpu->queued_work_first == NULL) {
-cpu->queued_work_first = 
-} else {
-cpu->queued_work_last->next = 
-}
-cpu->queued_work_last = 
-wi.next = NULL;
-wi.done = false;
-qemu_mutex_unlock(>work_mutex);
-
-qemu_cpu_kick(cpu);
+queue_work_on_cpu(cpu, );
 while (!atomic_mb_read()) {
 CPUState *self_cpu = current_cpu;
 
@@ -957,18 +962,7 @@ void async_run_on_cpu(CPUState *cpu, run_on_cpu_func func, 
void *data)
 wi->data = data;
 wi->free = true;
 
-qemu_mutex_lock(>work_mutex);
-if (cpu->queued_work_first == NULL) {
-cpu->queued_work_first = wi;
-} else {
-cpu->queued_work_last->next = wi;
-}
-cpu->queued_work_last = wi;
-wi->next = NULL;
-wi->done = false;
-qemu_mutex_unlock(>work_mutex);
-
-qemu_cpu_kick(cpu);
+queue_work_on_cpu(cpu, wi);
 }
 
 static void qemu_kvm_destroy_vcpu(CPUState *cpu)
-- 
1.9.1

Re: [Qemu-devel] [PATCH v8 11/16] qapi-event: Reduce chance of collision with event data

[Eric Blake]

On 07/07/2016 05:37 AM, Markus Armbruster wrote:
> Eric Blake  writes:
> 
>> When an unboxed event has accompanying data, we are exposing all
>> of its members alongside our local variables in the generated
>> qapi_event_send_FOO() function.  So far, we haven't hit a
>> collision, but it may be a matter of time before someone wants
>> to name a QMP data element 'err' or similar.  We can separate
>> the names by making the public function (qapi_event_send_FOO())
>> a shell that boxes things up without having to worry about
>> collisions, then calls into a new worker function
>> (do_qapi_event_send_FOO()) that gets generated to look like what
>> we already output for boxed events.
>>

>> +++ b/scripts/qapi.py
>> @@ -1019,7 +1019,6 @@ class QAPISchemaObjectType(QAPISchemaType):
>def c_name(self):
>>  return QAPISchemaType.c_name(self)
> 
> Aside: this method became pointless in commit 7ce106a.
> 

Sounds like an easy separate cleanup.

>>
>>  def c_type(self):
>> -assert not self.is_implicit()
>>  return c_name(self.name) + pointer_suffix
> 
> Correct if we emit a C type with this name for *any* object type.  I
> don't think we do for the empty object type.  Any others?
> 
> The assertion should be relaxed to reject exactly the object types for
> which we don't emit a C type.

Will fix.

> 
>>
>>  def c_unboxed_type(self):
>> diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
>> index 2cab588..b2da0a9 100644
>> --- a/scripts/qapi-event.py
>> +++ b/scripts/qapi-event.py
>> @@ -14,8 +14,13 @@
>>  from qapi import *
>>
>>
>> -def gen_event_send_proto(name, arg_type, box):
>> -return 'void qapi_event_send_%(c_name)s(%(param)s)' % {
>> +def gen_event_send_proto(name, arg_type, box, impl=False):
> 
> The function has changed from one that does just one thing to one that
> can do many things depending on arg_type, box and impl.  It's not even
> obvious which combinations are valid.  Either split it into simple,
> self-documenting functions, or explain this complex one with a function
> comment.

At this point, this cleanup is separate from the main work to get
'boxed' commands working, so I'll focus on getting the important first
half of the series posted today, and leave this particular patch for
when I have more time to get the refactoring done right.


> 
> This function spans 95 lines, and you have to read to the end to see
> that it first emits a function that can either be the
> qapi_event_send_FOO() function or a helper, and if it's the latter, then
> emits the former.  Either split it up into parts that can be more easily
> understood, or add suitable comments to the complex function.

Indeed.  The changes were all made piecemeal with minimal churn, but the
end result is messier than if we step back and do it right.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PATCH v3 06/12] linux-user: Use QemuMutex and QemuCond

[Sergey Fedorov]

From: Sergey Fedorov 

Convert pthread_mutex_t and pthread_cond_t to QemuMutex and QemuCond.
This will allow to make some locks and conditional variables common
between user and system mode emulation.

Signed-off-by: Sergey Fedorov 
Signed-off-by: Sergey Fedorov 
Reviewed-by: Alex Bennée 
---
 linux-user/main.c | 53 +++--
 1 file changed, 31 insertions(+), 22 deletions(-)

diff --git a/linux-user/main.c b/linux-user/main.c
index 617a179f14a4..bdbda693cc5f 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -108,17 +108,25 @@ int cpu_get_pic_interrupt(CPUX86State *env)
We don't require a full sync, only that no cpus are executing guest code.
The alternative is to map target atomic ops onto host equivalents,
which requires quite a lot of per host/target work.  */
-static pthread_mutex_t cpu_list_mutex = PTHREAD_MUTEX_INITIALIZER;
-static pthread_mutex_t exclusive_lock = PTHREAD_MUTEX_INITIALIZER;
-static pthread_cond_t exclusive_cond = PTHREAD_COND_INITIALIZER;
-static pthread_cond_t exclusive_resume = PTHREAD_COND_INITIALIZER;
+static QemuMutex cpu_list_mutex;
+static QemuMutex exclusive_lock;
+static QemuCond exclusive_cond;
+static QemuCond exclusive_resume;
 static int pending_cpus;
 
+void qemu_init_cpu_loop(void)
+{
+qemu_mutex_init(_list_mutex);
+qemu_mutex_init(_lock);
+qemu_cond_init(_cond);
+qemu_cond_init(_resume);
+}
+
 /* Make sure everything is in a consistent state for calling fork().  */
 void fork_start(void)
 {
 qemu_mutex_lock(_ctx.tb_ctx.tb_lock);
-pthread_mutex_lock(_lock);
+qemu_mutex_lock(_lock);
 mmap_fork_start();
 }
 
@@ -135,14 +143,14 @@ void fork_end(int child)
 }
 }
 pending_cpus = 0;
-pthread_mutex_init(_lock, NULL);
-pthread_mutex_init(_list_mutex, NULL);
-pthread_cond_init(_cond, NULL);
-pthread_cond_init(_resume, NULL);
+qemu_mutex_init(_lock);
+qemu_mutex_init(_list_mutex);
+qemu_cond_init(_cond);
+qemu_cond_init(_resume);
 qemu_mutex_init(_ctx.tb_ctx.tb_lock);
 gdbserver_fork(thread_cpu);
 } else {
-pthread_mutex_unlock(_lock);
+qemu_mutex_unlock(_lock);
 qemu_mutex_unlock(_ctx.tb_ctx.tb_lock);
 }
 }
@@ -152,7 +160,7 @@ void fork_end(int child)
 static inline void exclusive_idle(void)
 {
 while (pending_cpus) {
-pthread_cond_wait(_resume, _lock);
+qemu_cond_wait(_resume, _lock);
 }
 }
 
@@ -162,7 +170,7 @@ static inline void start_exclusive(void)
 {
 CPUState *other_cpu;
 
-pthread_mutex_lock(_lock);
+qemu_mutex_lock(_lock);
 exclusive_idle();
 
 pending_cpus = 1;
@@ -174,7 +182,7 @@ static inline void start_exclusive(void)
 }
 }
 if (pending_cpus > 1) {
-pthread_cond_wait(_cond, _lock);
+qemu_cond_wait(_cond, _lock);
 }
 }
 
@@ -182,42 +190,42 @@ static inline void start_exclusive(void)
 static inline void __attribute__((unused)) end_exclusive(void)
 {
 pending_cpus = 0;
-pthread_cond_broadcast(_resume);
-pthread_mutex_unlock(_lock);
+qemu_cond_broadcast(_resume);
+qemu_mutex_unlock(_lock);
 }
 
 /* Wait for exclusive ops to finish, and begin cpu execution.  */
 static inline void cpu_exec_start(CPUState *cpu)
 {
-pthread_mutex_lock(_lock);
+qemu_mutex_lock(_lock);
 exclusive_idle();
 cpu->running = true;
-pthread_mutex_unlock(_lock);
+qemu_mutex_unlock(_lock);
 }
 
 /* Mark cpu as not executing, and release pending exclusive ops.  */
 static inline void cpu_exec_end(CPUState *cpu)
 {
-pthread_mutex_lock(_lock);
+qemu_mutex_lock(_lock);
 cpu->running = false;
 if (pending_cpus > 1) {
 pending_cpus--;
 if (pending_cpus == 1) {
-pthread_cond_signal(_cond);
+qemu_cond_signal(_cond);
 }
 }
 exclusive_idle();
-pthread_mutex_unlock(_lock);
+qemu_mutex_unlock(_lock);
 }
 
 void cpu_list_lock(void)
 {
-pthread_mutex_lock(_list_mutex);
+qemu_mutex_lock(_list_mutex);
 }
 
 void cpu_list_unlock(void)
 {
-pthread_mutex_unlock(_list_mutex);
+qemu_mutex_unlock(_list_mutex);
 }
 
 
@@ -4210,6 +4218,7 @@ int main(int argc, char **argv, char **envp)
 int ret;
 int execfd;
 
+qemu_init_cpu_loop();
 module_call_init(MODULE_INIT_QOM);
 
 if ((envlist = envlist_create()) == NULL) {
-- 
1.9.1

[Qemu-devel] [PATCH v3 05/12] cpus: Rename flush_queued_work()

[Sergey Fedorov]

From: Sergey Fedorov 

To avoid possible confusion, rename flush_queued_work() to
process_queued_cpu_work().

Signed-off-by: Sergey Fedorov 
Signed-off-by: Sergey Fedorov 
Reviewed-by: Alex Bennée 
---
 cpus.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/cpus.c b/cpus.c
index f80ed2aeefdd..51fd8c18b4c8 100644
--- a/cpus.c
+++ b/cpus.c
@@ -982,7 +982,7 @@ static void qemu_tcg_destroy_vcpu(CPUState *cpu)
 {
 }
 
-static void flush_queued_work(CPUState *cpu)
+static void process_queued_cpu_work(CPUState *cpu)
 {
 struct qemu_work_item *wi;
 
@@ -1017,7 +1017,7 @@ static void qemu_wait_io_event_common(CPUState *cpu)
 cpu->stopped = true;
 qemu_cond_broadcast(_pause_cond);
 }
-flush_queued_work(cpu);
+process_queued_cpu_work(cpu);
 cpu->thread_kicked = false;
 }
 
-- 
1.9.1

[Qemu-devel] [PATCH v3 07/12] linux-user: Rework exclusive operation mechanism

[Sergey Fedorov]

From: Sergey Fedorov 

A single variable 'pending_cpus' was used for both counting currently
running CPUs and for signalling the pending exclusive operation request.

To prepare for supporting operations which requires a quiescent state,
like translation buffer flush, it is useful to keep a counter of
currently running CPUs always up to date.

Use a separate variable 'tcg_pending_threads' to count for currently
running CPUs and a separate variable 'exclusive_pending' to indicate
that there's an exclusive operation pending.

Signed-off-by: Sergey Fedorov 
Signed-off-by: Sergey Fedorov 

---
Changes in v2:
 - Rename 'tcg_pending_cpus' to 'tcg_pending_threads'
---
 linux-user/main.c | 24 
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/linux-user/main.c b/linux-user/main.c
index bdbda693cc5f..5ff0b20bad89 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -112,7 +112,8 @@ static QemuMutex cpu_list_mutex;
 static QemuMutex exclusive_lock;
 static QemuCond exclusive_cond;
 static QemuCond exclusive_resume;
-static int pending_cpus;
+static bool exclusive_pending;
+static int tcg_pending_threads;
 
 void qemu_init_cpu_loop(void)
 {
@@ -142,7 +143,8 @@ void fork_end(int child)
 QTAILQ_REMOVE(, cpu, node);
 }
 }
-pending_cpus = 0;
+tcg_pending_threads = 0;
+exclusive_pending = false;
 qemu_mutex_init(_lock);
 qemu_mutex_init(_list_mutex);
 qemu_cond_init(_cond);
@@ -159,7 +161,7 @@ void fork_end(int child)
must be held.  */
 static inline void exclusive_idle(void)
 {
-while (pending_cpus) {
+while (exclusive_pending) {
 qemu_cond_wait(_resume, _lock);
 }
 }
@@ -173,15 +175,14 @@ static inline void start_exclusive(void)
 qemu_mutex_lock(_lock);
 exclusive_idle();
 
-pending_cpus = 1;
+exclusive_pending = true;
 /* Make all other cpus stop executing.  */
 CPU_FOREACH(other_cpu) {
 if (other_cpu->running) {
-pending_cpus++;
 cpu_exit(other_cpu);
 }
 }
-if (pending_cpus > 1) {
+while (tcg_pending_threads) {
 qemu_cond_wait(_cond, _lock);
 }
 }
@@ -189,7 +190,7 @@ static inline void start_exclusive(void)
 /* Finish an exclusive operation.  */
 static inline void __attribute__((unused)) end_exclusive(void)
 {
-pending_cpus = 0;
+exclusive_pending = false;
 qemu_cond_broadcast(_resume);
 qemu_mutex_unlock(_lock);
 }
@@ -200,6 +201,7 @@ static inline void cpu_exec_start(CPUState *cpu)
 qemu_mutex_lock(_lock);
 exclusive_idle();
 cpu->running = true;
+tcg_pending_threads++;
 qemu_mutex_unlock(_lock);
 }
 
@@ -208,11 +210,9 @@ static inline void cpu_exec_end(CPUState *cpu)
 {
 qemu_mutex_lock(_lock);
 cpu->running = false;
-if (pending_cpus > 1) {
-pending_cpus--;
-if (pending_cpus == 1) {
-qemu_cond_signal(_cond);
-}
+tcg_pending_threads--;
+if (!tcg_pending_threads) {
+qemu_cond_signal(_cond);
 }
 exclusive_idle();
 qemu_mutex_unlock(_lock);
-- 
1.9.1

[Qemu-devel] [PATCH v3 11/12] cpu-exec-common: Introduce async_safe_run_on_cpu()

[Sergey Fedorov]

From: Sergey Fedorov 

This patch is based on the ideas found in work of KONRAD Frederic [1],
Alex Bennée [2], and Alvise Rigo [3].

This mechanism allows to perform an operation safely in a quiescent
state. Quiescent state means: (1) no vCPU is running and (2) BQL in
system-mode or 'exclusive_lock' in user-mode emulation is held while
performing the operation. This functionality is required e.g. for
performing translation buffer flush safely in multi-threaded user-mode
emulation.

The existing CPU work queue is used to schedule such safe operations. A
new 'safe' flag is added into struct qemu_work_item to designate the
special requirements of the safe work. An operation in a quiescent sate
can be scheduled by using async_safe_run_on_cpu() function which is
actually the same as sync_run_on_cpu() except that it marks the queued
work item with the 'safe' flag set to true. Given this flag set
queue_work_on_cpu() atomically increments 'safe_work_pending' global
counter and kicks all the CPUs instead of just the target CPU as in case
of normal CPU work. This allows to force other CPUs to exit their
execution loops and wait in wait_safe_cpu_work() function for the safe
work to finish. When a CPU drains its work queue, if it encounters a
work item marked as safe, it first waits for other CPUs to exit their
execution loops, then called the work item function, and finally
decrements 'safe_work_pending' counter with signalling other CPUs to let
them continue execution as soon as all pending safe work items have been
processed. The 'tcg_pending_threads' protected by 'exclusive_lock' in
user-mode or by 'qemu_global_mutex' in system-mode emulation is used to
determine if there is any CPU run and wait for it to exit the execution
loop. The fairness of all the CPU work queues is ensured by draining all
the pending safe work items before any CPU can run.

[1] http://lists.nongnu.org/archive/html/qemu-devel/2015-08/msg01128.html
[2] http://lists.nongnu.org/archive/html/qemu-devel/2016-04/msg02531.html
[3] http://lists.nongnu.org/archive/html/qemu-devel/2016-05/msg04792.html

Signed-off-by: Sergey Fedorov 
Signed-off-by: Sergey Fedorov 

---
Changes in v3:
 - bsd-user supported
Changes in v2:
 - some conditional varialbes moved to cpu-exec-common.c
 - documentation commend for new public API added
---
 bsd-user/main.c |  3 ++-
 cpu-exec-common.c   | 49 -
 cpus.c  | 20 
 include/exec/exec-all.h | 14 ++
 include/qom/cpu.h   | 14 ++
 linux-user/main.c   | 13 +++--
 6 files changed, 105 insertions(+), 8 deletions(-)

diff --git a/bsd-user/main.c b/bsd-user/main.c
index f738dd64d691..5433bca0fca6 100644
--- a/bsd-user/main.c
+++ b/bsd-user/main.c
@@ -66,9 +66,10 @@ int cpu_get_pic_interrupt(CPUX86State *env)
 void qemu_init_cpu_loop(void)
 {
 /* We need to do this becuase process_queued_cpu_work() calls
- * qemu_cond_broadcast() on it
+ * qemu_cond_broadcast() on them
  */
 qemu_cond_init(_work_cond);
+qemu_cond_init(_safe_work_cond);
 }
 
 QemuMutex *qemu_get_cpu_work_mutex(void)
diff --git a/cpu-exec-common.c b/cpu-exec-common.c
index a233f0124559..6f278d6d3b70 100644
--- a/cpu-exec-common.c
+++ b/cpu-exec-common.c
@@ -25,6 +25,7 @@
 
 bool exit_request;
 CPUState *tcg_current_cpu;
+int tcg_pending_threads;
 
 /* exit the current TB, but without causing any exception to be raised */
 void cpu_loop_exit_noexc(CPUState *cpu)
@@ -79,6 +80,17 @@ void cpu_loop_exit_restore(CPUState *cpu, uintptr_t pc)
 }
 
 QemuCond qemu_work_cond;
+QemuCond qemu_safe_work_cond;
+QemuCond qemu_exclusive_cond;
+
+static int safe_work_pending;
+
+void wait_safe_cpu_work(void)
+{
+while (atomic_mb_read(_work_pending) > 0) {
+qemu_cond_wait(_safe_work_cond, qemu_get_cpu_work_mutex());
+}
+}
 
 static void queue_work_on_cpu(CPUState *cpu, struct qemu_work_item *wi)
 {
@@ -91,9 +103,18 @@ static void queue_work_on_cpu(CPUState *cpu, struct 
qemu_work_item *wi)
 cpu->queued_work_last = wi;
 wi->next = NULL;
 wi->done = false;
+if (wi->safe) {
+atomic_inc(_work_pending);
+}
 qemu_mutex_unlock(>work_mutex);
 
-qemu_cpu_kick(cpu);
+if (!wi->safe) {
+qemu_cpu_kick(cpu);
+} else {
+CPU_FOREACH(cpu) {
+qemu_cpu_kick(cpu);
+}
+}
 }
 
 void run_on_cpu(CPUState *cpu, run_on_cpu_func func, void *data)
@@ -108,6 +129,7 @@ void run_on_cpu(CPUState *cpu, run_on_cpu_func func, void 
*data)
 wi.func = func;
 wi.data = data;
 wi.free = false;
+wi.safe = false;
 
 queue_work_on_cpu(cpu, );
 while (!atomic_mb_read()) {
@@ -131,6 +153,20 @@ void async_run_on_cpu(CPUState *cpu, run_on_cpu_func func, 
void *data)
 wi->func = func;
 wi->data = data;
 wi->free = true;
+wi->safe = false;
+
+

[Qemu-devel] [PATCH v3 04/12] cpus: Wrap mutex used to protect CPU work

[Sergey Fedorov]

From: Sergey Fedorov 

This will be useful to enable CPU work on user mode emulation.

Signed-off-by: Sergey Fedorov 
Signed-off-by: Sergey Fedorov 
Reviewed-by: Alex Bennée 
---
 cpus.c | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/cpus.c b/cpus.c
index 04687c85bcd4..f80ed2aeefdd 100644
--- a/cpus.c
+++ b/cpus.c
@@ -910,6 +910,11 @@ void qemu_init_cpu_loop(void)
 qemu_thread_get_self(_thread);
 }
 
+static QemuMutex *qemu_get_cpu_work_mutex(void)
+{
+return _global_mutex;
+}
+
 static void queue_work_on_cpu(CPUState *cpu, struct qemu_work_item *wi)
 {
 qemu_mutex_lock(>work_mutex);
@@ -943,7 +948,7 @@ void run_on_cpu(CPUState *cpu, run_on_cpu_func func, void 
*data)
 while (!atomic_mb_read()) {
 CPUState *self_cpu = current_cpu;
 
-qemu_cond_wait(_work_cond, _global_mutex);
+qemu_cond_wait(_work_cond, qemu_get_cpu_work_mutex());
 current_cpu = self_cpu;
 }
 }
-- 
1.9.1

[Qemu-devel] [PATCH v3 00/12] cpu-exec: Safe work in quiescent state

[Sergey Fedorov]

From: Sergey Fedorov 

Hi,

This is a v3 for the RFC series [1]. This is not RFC anymore because
bsd-user is supported now. However I wasn't able to even compile-test
bsd-user so some help on this would be really appreciated.

This series is available at a public git repository:

https://github.com/sergefdrv/qemu.git safe-cpu-work-v3

Summary of changes in v3:
 - bsd-user support added
 - 'tb_flushed' removed
Summary of changes in v2:
 - atomic_dec_fetch() used to decrement 'safe_work_pending'
 - more work to use/fix passing CPUState to run_on_cpu helpers
 - instead of wrapping conditional variables access, use QemuMutex and
   QemuCond in linux-user and just wrap getting of the relevant mutex.
 - document new public API
 - Rename 'tcg_pending_cpus' to 'tcg_pending_threads'

Kind regards,
Sergey

[1] http://thread.gmane.org/gmane.comp.emulators.qemu/425242

Alex Bennée (2):
  atomic: introduce atomic_dec_fetch.
  cpus: pass CPUState to run_on_cpu helpers

Sergey Fedorov (10):
  cpus: Move common code out of {async_,}run_on_cpu()
  cpus: Wrap mutex used to protect CPU work
  cpus: Rename flush_queued_work()
  linux-user: Use QemuMutex and QemuCond
  linux-user: Rework exclusive operation mechanism
  linux-user: Add qemu_cpu_is_self() and qemu_cpu_kick()
  linux-user: Support CPU work queue
  bsd-user: Support CPU work queue
  cpu-exec-common: Introduce async_safe_run_on_cpu()
  tcg: Make tb_flush() thread safe

 bsd-user/main.c|  16 ++
 cpu-exec-common.c  | 132 +
 cpus.c | 106 +++-
 hw/i386/kvm/apic.c |   3 +-
 hw/i386/kvmvapic.c |   6 +--
 hw/ppc/ppce500_spin.c  |  31 ---
 hw/ppc/spapr.c |   6 +--
 hw/ppc/spapr_hcall.c   |  17 +++---
 include/exec/exec-all.h|  31 +++
 include/qemu/atomic.h  |   4 ++
 include/qom/cpu.h  |  22 ++--
 kvm-all.c  |  21 +++-
 linux-user/main.c  |  94 
 target-i386/helper.c   |  19 +++
 target-i386/kvm.c  |   6 +--
 target-s390x/cpu.c |   4 +-
 target-s390x/cpu.h |   7 +--
 target-s390x/kvm.c |  98 -
 target-s390x/misc_helper.c |   4 +-
 translate-all.c|  13 +++--
 20 files changed, 387 insertions(+), 253 deletions(-)

-- 
1.9.1

[Qemu-devel] [PATCH v3 02/12] cpus: pass CPUState to run_on_cpu helpers

[Sergey Fedorov]

From: Alex Bennée 

CPUState is a fairly common pointer to pass to these helpers. This means
if you need other arguments for the async_run_on_cpu case you end up
having to do a g_malloc to stuff additional data into the routine. For
the current users this isn't a massive deal but for MTTCG this gets
cumbersome when the only other parameter is often an address.

This adds the typedef run_on_cpu_func for helper functions which has an
explicit CPUState * passed as the first parameter. All the users of
run_on_cpu and async_run_on_cpu have had their helpers updated to use
CPUState where available.

Signed-off-by: Alex Bennée 
[Sergey Fedorov:
 - eliminate more CPUState in user data;
 - remove unnecessary user data passing;
 - fix target-s390x/kvm.c and target-s390x/misc_helper.c]
Signed-off-by: Sergey Fedorov 
Acked-by: David Gibson  (ppc parts)
Reviewed-by: Christian Borntraeger  (s390 parts)

---
Changes in v2:
 - eliminate more CPUState in user data
 - remove unnecessary user data passing
 - fix target-s390x/kvm.c and target-s390x/misc_helper.c
---
 cpus.c | 15 ---
 hw/i386/kvm/apic.c |  3 +-
 hw/i386/kvmvapic.c |  6 +--
 hw/ppc/ppce500_spin.c  | 31 +--
 hw/ppc/spapr.c |  6 +--
 hw/ppc/spapr_hcall.c   | 17 
 include/qom/cpu.h  |  8 ++--
 kvm-all.c  | 21 --
 target-i386/helper.c   | 19 -
 target-i386/kvm.c  |  6 +--
 target-s390x/cpu.c |  4 +-
 target-s390x/cpu.h |  7 +---
 target-s390x/kvm.c | 98 +++---
 target-s390x/misc_helper.c |  4 +-
 14 files changed, 108 insertions(+), 137 deletions(-)

diff --git a/cpus.c b/cpus.c
index 84c3520d446f..049c2d04e150 100644
--- a/cpus.c
+++ b/cpus.c
@@ -551,9 +551,8 @@ static const VMStateDescription vmstate_timers = {
 }
 };
 
-static void cpu_throttle_thread(void *opaque)
+static void cpu_throttle_thread(CPUState *cpu, void *opaque)
 {
-CPUState *cpu = opaque;
 double pct;
 double throttle_ratio;
 long sleeptime_ns;
@@ -583,7 +582,7 @@ static void cpu_throttle_timer_tick(void *opaque)
 }
 CPU_FOREACH(cpu) {
 if (!atomic_xchg(>throttle_thread_scheduled, 1)) {
-async_run_on_cpu(cpu, cpu_throttle_thread, cpu);
+async_run_on_cpu(cpu, cpu_throttle_thread, NULL);
 }
 }
 
@@ -911,12 +910,12 @@ void qemu_init_cpu_loop(void)
 qemu_thread_get_self(_thread);
 }
 
-void run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data)
+void run_on_cpu(CPUState *cpu, run_on_cpu_func func, void *data)
 {
 struct qemu_work_item wi;
 
 if (qemu_cpu_is_self(cpu)) {
-func(data);
+func(cpu, data);
 return;
 }
 
@@ -944,12 +943,12 @@ void run_on_cpu(CPUState *cpu, void (*func)(void *data), 
void *data)
 }
 }
 
-void async_run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data)
+void async_run_on_cpu(CPUState *cpu, run_on_cpu_func func, void *data)
 {
 struct qemu_work_item *wi;
 
 if (qemu_cpu_is_self(cpu)) {
-func(data);
+func(cpu, data);
 return;
 }
 
@@ -1000,7 +999,7 @@ static void flush_queued_work(CPUState *cpu)
 cpu->queued_work_last = NULL;
 }
 qemu_mutex_unlock(>work_mutex);
-wi->func(wi->data);
+wi->func(cpu, wi->data);
 qemu_mutex_lock(>work_mutex);
 if (wi->free) {
 g_free(wi);
diff --git a/hw/i386/kvm/apic.c b/hw/i386/kvm/apic.c
index c5983c79be47..9b66e741d4b4 100644
--- a/hw/i386/kvm/apic.c
+++ b/hw/i386/kvm/apic.c
@@ -125,10 +125,9 @@ static void kvm_apic_vapic_base_update(APICCommonState *s)
 }
 }
 
-static void do_inject_external_nmi(void *data)
+static void do_inject_external_nmi(CPUState *cpu, void *data)
 {
 APICCommonState *s = data;
-CPUState *cpu = CPU(s->cpu);
 uint32_t lvt;
 int ret;
 
diff --git a/hw/i386/kvmvapic.c b/hw/i386/kvmvapic.c
index 3bf1ddd97612..1bc02fb2f1a1 100644
--- a/hw/i386/kvmvapic.c
+++ b/hw/i386/kvmvapic.c
@@ -483,7 +483,7 @@ typedef struct VAPICEnableTPRReporting {
 bool enable;
 } VAPICEnableTPRReporting;
 
-static void vapic_do_enable_tpr_reporting(void *data)
+static void vapic_do_enable_tpr_reporting(CPUState *cpu, void *data)
 {
 VAPICEnableTPRReporting *info = data;
 
@@ -734,10 +734,10 @@ static void vapic_realize(DeviceState *dev, Error **errp)
 nb_option_roms++;
 }
 
-static void do_vapic_enable(void *data)
+static void do_vapic_enable(CPUState *cs, void *data)
 {
 VAPICROMState *s = data;
-X86CPU *cpu = X86_CPU(first_cpu);
+X86CPU *cpu = X86_CPU(cs);
 
 static const uint8_t enabled = 1;
 cpu_physical_memory_write(s->vapic_paddr + offsetof(VAPICState, enabled),
diff --git a/hw/ppc/ppce500_spin.c b/hw/ppc/ppce500_spin.c
index 

[Qemu-devel] [PATCH v3 10/12] bsd-user: Support CPU work queue

[Sergey Fedorov]

From: Sergey Fedorov 

It is a minimalistic support because bsd-linux claims to be _not_
threadsafe.

Signed-off-by: Sergey Fedorov 
Signed-off-by: Sergey Fedorov 
---
 bsd-user/main.c | 15 +++
 1 file changed, 15 insertions(+)

diff --git a/bsd-user/main.c b/bsd-user/main.c
index 4819b9ec6333..f738dd64d691 100644
--- a/bsd-user/main.c
+++ b/bsd-user/main.c
@@ -63,6 +63,19 @@ int cpu_get_pic_interrupt(CPUX86State *env)
 }
 #endif
 
+void qemu_init_cpu_loop(void)
+{
+/* We need to do this becuase process_queued_cpu_work() calls
+ * qemu_cond_broadcast() on it
+ */
+qemu_cond_init(_work_cond);
+}
+
+QemuMutex *qemu_get_cpu_work_mutex(void)
+{
+return NULL; /* it will never be used */
+}
+
 /* These are no-ops because we are not threadsafe.  */
 static inline void cpu_exec_start(CPUArchState *env)
 {
@@ -70,6 +83,7 @@ static inline void cpu_exec_start(CPUArchState *env)
 
 static inline void cpu_exec_end(CPUArchState *env)
 {
+process_queued_cpu_work(cpu);
 }
 
 static inline void start_exclusive(void)
@@ -740,6 +754,7 @@ int main(int argc, char **argv)
 if (argc <= 1)
 usage();
 
+qemu_init_cpu_loop();
 module_call_init(MODULE_INIT_QOM);
 
 if ((envlist = envlist_create()) == NULL) {
-- 
1.9.1

[Qemu-devel] [PATCH v3 01/12] atomic: introduce atomic_dec_fetch.

[Sergey Fedorov]

From: Alex Bennée 

Useful for counting down.

Signed-off-by: Alex Bennée 
Signed-off-by: Sergey Fedorov 
---
 include/qemu/atomic.h | 4 
 1 file changed, 4 insertions(+)

diff --git a/include/qemu/atomic.h b/include/qemu/atomic.h
index 7a590969b59f..8ac9ca7f457a 100644
--- a/include/qemu/atomic.h
+++ b/include/qemu/atomic.h
@@ -162,6 +162,8 @@
 #define atomic_fetch_and(ptr, n) __atomic_fetch_and(ptr, n, __ATOMIC_SEQ_CST)
 #define atomic_fetch_or(ptr, n)  __atomic_fetch_or(ptr, n, __ATOMIC_SEQ_CST)
 
+#define atomic_dec_fetch(ptr)  __atomic_sub_fetch(ptr, 1, __ATOMIC_SEQ_CST)
+
 /* And even shorter names that return void.  */
 #define atomic_inc(ptr)((void) __atomic_fetch_add(ptr, 1, 
__ATOMIC_SEQ_CST))
 #define atomic_dec(ptr)((void) __atomic_fetch_sub(ptr, 1, 
__ATOMIC_SEQ_CST))
@@ -357,6 +359,8 @@
 #define atomic_fetch_or__sync_fetch_and_or
 #define atomic_cmpxchg __sync_val_compare_and_swap
 
+#define atomic_dec_fetch(ptr)  __sync_sub_and_fetch(ptr, 1)
+
 /* And even shorter names that return void.  */
 #define atomic_inc(ptr)((void) __sync_fetch_and_add(ptr, 1))
 #define atomic_dec(ptr)((void) __sync_fetch_and_add(ptr, -1))
-- 
1.9.1

Re: [Qemu-devel] [PATCH v8 4/5] generic-loader: Add a generic loader

[Alistair Francis]

On Wed, Jul 13, 2016 at 12:44 PM, Peter Maydell
 wrote:
> On 13 July 2016 at 18:45, Alistair Francis  
> wrote:
>> On Tue, Jul 12, 2016 at 9:39 AM, Peter Maydell  
>> wrote:
>>> On 2 July 2016 at 02:07, Alistair Francis  
>>> wrote:
 Add a generic loader to QEMU which can be used to load images or set
 memory values.

 Signed-off-by: Alistair Francis 
 ---
>
 +if (s->cpu) {
 +CPUClass *cc = CPU_GET_CLASS(s->cpu);
 +cpu_reset(s->cpu);
>>>
>>> If something else sets the PC later on this gets lost, which is
>>> ugly and fragile. It's kinda-sorta OK that we bodge this for things
>>> like the hw/arm/boot.c code because all the pieces are inside QEMU,
>>> but when you start adding in command line devices I'm a bit
>>> nervous. Maybe we should actually figure out our reset order
>>> woes properly ?
>>
>> This is up to the user to specify the commands in the order they want
>> them applied. What other method should we be using?
>
> Well, if it works it works, I guess, but one day we
> will need to figure out an actual model for reset order
> rather than having it work by chance (in this case,
> because command line devices get reset after ones in
> the board model proper, I think.)

Agreed, but I think that is out of scope of this series. That would
require a overhaul of the reset infrastructure to allow a more
configurable system or at least some stricter ordering rules to
follow. I thought I saw some patches on the list to start to improve
the reset functionality?

>
 +qemu_register_reset(generic_loader_reset, dev);
>>>
>>> What's wrong with a device reset function set via dc->reset ?
>>
>> This allows setting values from the HMP command line interface once
>> the machine is running. The dc->reset isn't applied in that case.
>
> I don't understand this -- could you explain in a bit
> more detail, please?

So this loading system is just a device, which means that it can be
loaded whenever a normal device would be loaded. The more interesting
option is using this via the command line but it is also possible to
use the QEMU Monitor once QEMU is running to call this.

A user can use the device_add function to add set values once QEMU is
running. The problem with that though is that the qdev_device_add()
function doesn't connect the reset function. I initially had a patch
to do this "qdev-monitor.c: Register reset function if the device has
one" but it was decided to just register the reset function in the
realise instead.

At the moment rom loading is only supported at startup, but in the
future maybe that could be supported as well, adding even more value
to this.

>
>>> Changing that after this goes into the tree would be a command
>>> line compatibility break, so this inclines me to saying that this
>>> isn't baked enough for 2.7 and we should aim to put it into 2.8.
>>
>> That's fair, but upsetting.
>
> I'm not really happy about pushing this back yet another
> release either, and there are two weeks left til hard
> freeze. So it's not impossible that we could fit it in,
> but we're still discussing the semantics of the device
> at this point, so if it did get into 2.7 I think it
> would be ending up going in really fairly late, and I
> tend by preference to be conservative about freezes.
> Wider review (ie somebody else as well as me) would help
> in providing reassurance that the interface is correct.

That is fair, I can see that it is risky.

I have the next version ready to send, I'll send it by the end of
today (PST time) if no other issues come up. Maybe it can still make
it in.

Thanks,

Alistair

>
> thanks
> -- PMM
>

Re: [Qemu-devel] [PATCH] machine: add comment to abort path in machine_set_kernel_irqchip

[Eduardo Habkost]

On Wed, Jul 13, 2016 at 08:11:45PM +0200, Greg Kurz wrote:
> We're not supposed to abort when the user passes a bogus value.
> Since the ckecing is done in visit_type_OnOffSplit(), the call to
> abort() is legitimate. Let's add a comment to make it explicit.
> 
> Signed-off-by: Greg Kurz 

Applied to machine-next, thanks. Although I believe the property
should be changed to use object_property_add_enum() instead.

-- 
Eduardo

Re: [Qemu-devel] [PATCH v8 4/5] generic-loader: Add a generic loader

[Peter Maydell]

On 13 July 2016 at 18:45, Alistair Francis  wrote:
> On Tue, Jul 12, 2016 at 9:39 AM, Peter Maydell  
> wrote:
>> On 2 July 2016 at 02:07, Alistair Francis  
>> wrote:
>>> Add a generic loader to QEMU which can be used to load images or set
>>> memory values.
>>>
>>> Signed-off-by: Alistair Francis 
>>> ---

>>> +if (s->cpu) {
>>> +CPUClass *cc = CPU_GET_CLASS(s->cpu);
>>> +cpu_reset(s->cpu);
>>
>> If something else sets the PC later on this gets lost, which is
>> ugly and fragile. It's kinda-sorta OK that we bodge this for things
>> like the hw/arm/boot.c code because all the pieces are inside QEMU,
>> but when you start adding in command line devices I'm a bit
>> nervous. Maybe we should actually figure out our reset order
>> woes properly ?
>
> This is up to the user to specify the commands in the order they want
> them applied. What other method should we be using?

Well, if it works it works, I guess, but one day we
will need to figure out an actual model for reset order
rather than having it work by chance (in this case,
because command line devices get reset after ones in
the board model proper, I think.)

>>> +qemu_register_reset(generic_loader_reset, dev);
>>
>> What's wrong with a device reset function set via dc->reset ?
>
> This allows setting values from the HMP command line interface once
> the machine is running. The dc->reset isn't applied in that case.

I don't understand this -- could you explain in a bit
more detail, please?

>> Changing that after this goes into the tree would be a command
>> line compatibility break, so this inclines me to saying that this
>> isn't baked enough for 2.7 and we should aim to put it into 2.8.
>
> That's fair, but upsetting.

I'm not really happy about pushing this back yet another
release either, and there are two weeks left til hard
freeze. So it's not impossible that we could fit it in,
but we're still discussing the semantics of the device
at this point, so if it did get into 2.7 I think it
would be ending up going in really fairly late, and I
tend by preference to be conservative about freezes.
Wider review (ie somebody else as well as me) would help
in providing reassurance that the interface is correct.

thanks
-- PMM

Re: [Qemu-devel] [PATCH] machine: add comment to abort path in machine_set_kernel_irqchip

[Greg Kurz]

On Wed, 13 Jul 2016 20:11:45 +0200
Greg Kurz  wrote:

> We're not supposed to abort when the user passes a bogus value.
> Since the ckecing is done in visit_type_OnOffSplit(), the call to

s/ckecing/checking :)

> abort() is legitimate. Let's add a comment to make it explicit.
> 
> Signed-off-by: Greg Kurz 
> ---
>  hw/core/machine.c |3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index 2fe6ff6f3007..e5a456f21dd9 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -65,6 +65,9 @@ static void machine_set_kernel_irqchip(Object *obj, Visitor 
> *v,
>  ms->kernel_irqchip_split = true;
>  break;
>  default:
> +/* The value was checked in visit_type_OnOffSplit() above. If
> + * we get here, then something is wrong in QEMU.
> + */
>  abort();
>  }
>  }
> 
> 

Re: [Qemu-devel] tests/ide-test hangs with gthread

[John Snow]

On 07/13/2016 12:30 PM, Marc-André Lureau wrote:
> Hi,
> 
> Since 4840f10 "memory: let address_space_rw/ld*/st* run outside the
> BQL", the ide-test /x86_64/ide/bmdma/no_busmaster hangs when qemu is
> compiled with gthread coroutine . Is this a known issue?
> 

News to me.

What's your configure string look like?

> (fwiw, running the test alone also crashes, with -p
> /x86_64/ide/bmdma/no_busmaster, lacking /x86_64/ide/bmdma/setup I
> suppose)

Yes, I think the ide-test is naughty in this way.

> 
> thanks
> 

Does ahci-test work alright?

--js

Re: [Qemu-devel] [RFC 00/13] Live memory snapshot based on userfaultfd

[Dr. David Alan Gilbert]

* zhanghailiang (zhang.zhanghaili...@huawei.com) wrote:
> For now, we still didn't support live memory snapshot, we have discussed
> a scheme which based on userfaultfd long time ago.
> You can find the discussion by the follow link:
> https://lists.nongnu.org/archive/html/qemu-devel/2014-11/msg01779.html
> 
> The scheme is based on userfaultfd's write-protect capability.
> The userfaultfd write protection feature is available here:
> http://www.spinics.net/lists/linux-mm/msg97422.html

I've (finally!) had a brief look through this, I like the idea.
I've not bothered with minor cleanup like comments on them;
I'm sure those will happen later; some larger scale things to think
about are:
  a) I wonder if it's really best to put that much code into the postcopy
 function; it might be but I can see other userfault uses as well.
  b) I worry a bit about the size of the copies you create during setup
 and I don't really understand why you can't start sending those pages
 immediately - but then I worry aobut the relative order of when pages
 data should be sent compared to the state of devices view of RAM.
  c) Have you considered also using userfault for loading the snapshot - I
know there was someone on #qemu a while ago who was talking about using
it as a way to quickly reload from a migration image.

Dave

> 
> The process of this live memory scheme is like bellow:
> 1. Pause VM
> 2. Enable write-protect fault notification by using userfaultfd to
>mark VM's memory to write-protect (readonly).
> 3. Save VM's static state (here is device state) to snapshot file
> 4. Resume VM, VM is going to run.
> 5. Snapshot thread begins to save VM's live state (here is RAM) into
>snapshot file.
> 6. During this time, all the actions of writing VM's memory will be blocked
>   by kernel, and kernel will wakeup the fault treating thread in qemu to
>   process this write-protect fault. The fault treating thread will deliver 
> this
>   page's address to snapshot thread.
> 7. snapshot thread gets this address, save this page into snasphot file,
>and then remove the write-protect by using userfaultfd API, after that,
>the actions of writing will be recovered. 
> 8. Repeat step 5~7 until all VM's memory is saved to snapshot file
> 
> Compared with the feature of 'migrate VM's state to file',
> the main difference for live memory snapshot is it has little time delay for
> catching VM's state. It just captures the VM's state while got users snapshot
> command, just like take a photo of VM's state.
> 
> For now, we only support tcg accelerator, since userfaultfd is not supporting
> tracking write faults for KVM.
> 
> Usage:
> 1. Take a snapshot
> #x86_64-softmmu/qemu-system-x86_64 -machine pc-i440fx-2.5,accel=tcg,usb=off 
> -drive 
> file=/mnt/windows/win7_install.qcow2.bak,if=none,id=drive-ide0-0-1,format=qcow2,cache=none
>  -device ide-hd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1  -vnc :7 -m 
> 8192 -smp 1 -netdev tap,id=bn0 -device virtio-net-pci,id=net-pci0,netdev=bn0  
> --monitor stdio
> Issue snapshot command:
> (qemu)migrate -d file:/home/Snapshot
> 2. Revert to the snapshot
> #x86_64-softmmu/qemu-system-x86_64 -machine pc-i440fx-2.5,accel=tcg,usb=off 
> -drive 
> file=/mnt/windows/win7_install.qcow2.bak,if=none,id=drive-ide0-0-1,format=qcow2,cache=none
>  -device ide-hd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1  -vnc :7 -m 
> 8192 -smp 1 -netdev tap,id=bn0 -device virtio-net-pci,id=net-pci0,netdev=bn0  
> --monitor stdio -incoming file:/home/Snapshot
> 
> NOTE:
> The userfaultfd write protection feature does not support THP for now,
> Before taking snapshot, please disable THP by:
> echo never > /sys/kernel/mm/transparent_hugepage/enabled
> 
> TODO:
> - Reduce the influence for VM while taking snapshot
> 
> zhanghailiang (13):
>   postcopy/migration: Split fault related state into struct
> UserfaultState
>   migration: Allow the migrate command to work on file: urls
>   migration: Allow -incoming to work on file: urls
>   migration: Create a snapshot thread to realize saving memory snapshot
>   migration: implement initialization work for snapshot
>   QEMUSizedBuffer: Introduce two help functions for qsb
>   savevm: Split qemu_savevm_state_complete_precopy() into two helper
> functions
>   snapshot: Save VM's device state into snapshot file
>   migration/postcopy-ram: fix some helper functions to support
> userfaultfd write-protect
>   snapshot: Enable the write-protect notification capability for VM's
> RAM
>   snapshot/migration: Save VM's RAM into snapshot file
>   migration/ram: Fix some helper functions' parameter to use
> PageSearchStatus
>   snapshot: Remove page's write-protect and copy the content during
> setup stage
> 
>  include/migration/migration.h |  41 +--
>  include/migration/postcopy-ram.h  |   9 +-
>  include/migration/qemu-file.h |   3 +-
>  include/qemu/typedefs.h   |   1 +
>  include/sysemu/sysemu.h  

[Qemu-devel] [PATCH] vl: exit if a bad property value is passed to -global

[Greg Kurz]

When passing '-global driver=host-powerpc64-cpu,property=compat,value=foo'
on the command line, without this patch, we get the following warning per
device (which means many lines if the guests has many cpus):

qemu-system-ppc64: Warning: can't apply global host-powerpc64-cpu.compat=foo:
Invalid compatibility mode "foo"

... and QEMU continues execution, ignoring the property.

With this patch, we get a single line:

qemu-system-ppc64: can't apply global host-powerpc64-cpu.compat=foo:
Invalid compatibility mode "foo"

... and QEMU exits.

Signed-off-by: Greg Kurz 
---
 vl.c |3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/vl.c b/vl.c
index 356713ea075c..a7cc92781687 100644
--- a/vl.c
+++ b/vl.c
@@ -2922,6 +2922,7 @@ static int global_init_func(void *opaque, QemuOpts *opts, 
Error **errp)
 g->property = qemu_opt_get(opts, "property");
 g->value= qemu_opt_get(opts, "value");
 g->user_provided = true;
+g->errp = errp;
 qdev_prop_register_global(g);
 return 0;
 }
@@ -4451,7 +4452,7 @@ int main(int argc, char **argv, char **envp)
 machine_register_compat_props(current_machine);
 
 qemu_opts_foreach(qemu_find_opts("global"),
-  global_init_func, NULL, NULL);
+  global_init_func, NULL, _fatal);
 
 /* This checkpoint is required by replay to separate prior clock
reading from the other reads, because timer polling functions query

[Qemu-devel] [PATCH] machine: add comment to abort path in machine_set_kernel_irqchip

[Greg Kurz]

We're not supposed to abort when the user passes a bogus value.
Since the ckecing is done in visit_type_OnOffSplit(), the call to
abort() is legitimate. Let's add a comment to make it explicit.

Signed-off-by: Greg Kurz 
---
 hw/core/machine.c |3 +++
 1 file changed, 3 insertions(+)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index 2fe6ff6f3007..e5a456f21dd9 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -65,6 +65,9 @@ static void machine_set_kernel_irqchip(Object *obj, Visitor 
*v,
 ms->kernel_irqchip_split = true;
 break;
 default:
+/* The value was checked in visit_type_OnOffSplit() above. If
+ * we get here, then something is wrong in QEMU.
+ */
 abort();
 }
 }

Re: [Qemu-devel] [PATCH v3 00/11] Reduce lock contention on TCG hot-path

[Sergey Fedorov]

On 13/07/16 10:39, Paolo Bonzini wrote:
> If anything, for historical reasons one might rename tb_find_physical to
> tb_find_slow and leave the tb_find_fast name, but I think the patch is
> good as is.

I think tb_find_htable() or tb_find_global_htable() could also be good
options if we're going to rename tb_find_physical().

Kind regards,
Sergey

Re: [Qemu-devel] [PATCH v3 01/11] util/qht: Document memory ordering assumptions

[Sergey Fedorov]

On 13/07/16 14:13, Paolo Bonzini wrote:
>
> On 12/07/2016 22:13, Sergey Fedorov wrote:
>> diff --git a/include/qemu/qht.h b/include/qemu/qht.h
>> index 70bfc68b8d67..5f633e5d8100 100644
>> --- a/include/qemu/qht.h
>> +++ b/include/qemu/qht.h
>> @@ -69,6 +69,9 @@ void qht_destroy(struct qht *ht);
>>   * Attempting to insert a NULL @p is a bug.
>>   * Inserting the same pointer @p with different @hash values is a bug.
>>   *
>> + * In case of successful operation, smp_wmb() is implied before the pointer 
>> is
>> + * inserted into the hash table.
>> + *
>>   * Returns true on sucess.
>>   * Returns false if the @p-@hash pair already exists in the hash table.
>>   */
>> @@ -86,6 +89,9 @@ bool qht_insert(struct qht *ht, void *p, uint32_t hash);
>>   * The user-provided @func compares pointers in QHT against @userp.
>>   * If the function returns true, a match has been found.
>>   *
>> + * smp_rmb() is implied before and after the pointer is looked up and 
>> retrieved
>> + * from the hash table.
> Do we really need to guarantee smp_rmb(), or is smp_read_barrier_depends()
> aka atomic_rcu_read() enough?

The idea was something like: qht_lookup() can be "paired" with either
qht_insert() or qht_remove(). The intention was to guarantee independent
tb_jmp_cache lookup performed before qht_lookup() in tb_find_physical().

>
> Likewise, perhaps only an implicit smp_wmb() before the insert is
> "interesting" to qht_insert__locked callers .

I see the idea behind this patch is worthwhile so I spend more time on
refining it and I must look at your patch carefully ;)

Thanks,
Sergey

>
> Something like:
>
> diff --git a/include/qemu/qht.h b/include/qemu/qht.h
> index 70bfc68..f4f1d55 100644
> --- a/include/qemu/qht.h
> +++ b/include/qemu/qht.h
> @@ -69,6 +69,9 @@ void qht_destroy(struct qht *ht);
>   * Attempting to insert a NULL @p is a bug.
>   * Inserting the same pointer @p with different @hash values is a bug.
>   *
> + * In case of successful operation, smp_wmb() is implied before the pointer 
> is
> + * inserted into the hash table.
> + *
>   * Returns true on sucess.
>   * Returns false if the @p-@hash pair already exists in the hash table.
>   */
> @@ -83,6 +86,8 @@ bool qht_insert(struct qht *ht, void *p, uint32_t hash);
>   *
>   * Needs to be called under an RCU read-critical section.
>   *
> + * smp_read_barrier_depends() is implied before the call to @func.
> + *
>   * The user-provided @func compares pointers in QHT against @userp.
>   * If the function returns true, a match has been found.
>   *
> @@ -105,6 +110,10 @@ void *qht_lookup(struct qht *ht, qht_lookup_func_t func, 
> const void *userp,
>   * This guarantees that concurrent lookups will always compare against valid
>   * data.
>   *
> + * In case of successful operation, a smp_wmb() barrier is implied before and
> + * after the pointer is removed from the hash table.  In other words,
> + * a successful qht_remove acts as a bidirectional write barrier.
> + *
>   * Returns true on success.
>   * Returns false if the @p-@hash pair was not found.
>   */
> diff --git a/util/qht.c b/util/qht.c
> index 40d6e21..d38948e 100644
> --- a/util/qht.c
> +++ b/util/qht.c
> @@ -445,7 +445,11 @@ void *qht_do_lookup(struct qht_bucket *head, 
> qht_lookup_func_t func,
>  do {
>  for (i = 0; i < QHT_BUCKET_ENTRIES; i++) {
>  if (b->hashes[i] == hash) {
> -void *p = atomic_read(>pointers[i]);
> +/* The pointer is dereferenced before seqlock_read_retry,
> + * so (unlike qht_insert__locked) we need to use
> + * atomic_rcu_read here.
> + */
> +void *p = atomic_rcu_read(>pointers[i]);
>  
>  if (likely(p) && likely(func(p, userp))) {
>  return p;
> @@ -535,6 +539,7 @@ static bool qht_insert__locked(struct qht *ht, struct 
> qht_map *map,
>  atomic_rcu_set(>next, b);
>  }
>  b->hashes[i] = hash;
> +/* smp_wmb() implicit in seqlock_write_begin.  */
>  atomic_set(>pointers[i], p);
>  seqlock_write_end(>sequence);
>  return true;
> @@ -659,6 +664,9 @@ bool qht_remove__locked(struct qht_map *map, struct 
> qht_bucket *head,
>  }
>  if (q == p) {
>  qht_debug_assert(b->hashes[i] == hash);
> +/* seqlock_write_begin and seqlock_write_end provide write
> + * memory barrier semantics to callers of qht_remove.
> + */
>  seqlock_write_begin(>sequence);
>  qht_bucket_remove_entry(b, i);
>  seqlock_write_end(>sequence);

Re: [Qemu-devel] [RFC 13/13] snapshot: Remove page's write-protect and copy the content during setup stage

[Dr. David Alan Gilbert]

* zhanghailiang (zhang.zhanghaili...@huawei.com) wrote:
> If we modify VM's RAM (pages) during setup stage after enable write-protect
> notification in snapshot thread, the modification action will get stuck 
> because
> we only remove the page's write-protect in savevm process, it blocked by 
> itself.
> 
> To fix this bug, we will remove page's write-protect in fault thread during
> the setup stage. Besides, we should not try to get global lock after setup 
> stage,
> or there maybe an deadlock error.

Hmm this complicates things a bit more doesn't it.
What's the order of:
   a) setup
   b) savings devices
   c) Being able to transmit the pages?

Are these pages that are being modified during setup, being modified
as part of the device state save?

Dave

> 
> Signed-off-by: zhanghailiang 
> ---
>  include/migration/migration.h |  4 ++--
>  migration/migration.c |  2 +-
>  migration/postcopy-ram.c  | 17 -
>  migration/ram.c   | 37 +++--
>  4 files changed, 50 insertions(+), 10 deletions(-)
> 
> diff --git a/include/migration/migration.h b/include/migration/migration.h
> index ef4c071..435de31 100644
> --- a/include/migration/migration.h
> +++ b/include/migration/migration.h
> @@ -127,7 +127,7 @@ struct MigrationSrcPageRequest {
>  RAMBlock *rb;
>  hwaddroffset;
>  hwaddrlen;
> -
> +uint8_t *pages_copy_addr;
>  QSIMPLEQ_ENTRY(MigrationSrcPageRequest) next_req;
>  };
>  
> @@ -333,7 +333,7 @@ void global_state_store_running(void);
>  
>  void flush_page_queue(MigrationState *ms);
>  int ram_save_queue_pages(MigrationState *ms, const char *rbname,
> - ram_addr_t start, ram_addr_t len);
> + ram_addr_t start, ram_addr_t len, bool copy_pages);
>  
>  PostcopyState postcopy_state_get(void);
>  /* Set the state and return the old state */
> diff --git a/migration/migration.c b/migration/migration.c
> index 3765c3b..bf4c7a1 100644
> --- a/migration/migration.c
> +++ b/migration/migration.c
> @@ -1248,7 +1248,7 @@ static void migrate_handle_rp_req_pages(MigrationState 
> *ms, const char* rbname,
>  return;
>  }
>  
> -if (ram_save_queue_pages(ms, rbname, start, len)) {
> +if (ram_save_queue_pages(ms, rbname, start, len, false)) {
>  mark_source_rp_bad(ms);
>  }
>  }
> diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
> index 61392d3..2cf477d 100644
> --- a/migration/postcopy-ram.c
> +++ b/migration/postcopy-ram.c
> @@ -543,13 +543,28 @@ static void *postcopy_ram_fault_thread(void *opaque)
>  MigrationState *ms = container_of(us, MigrationState,
>userfault_state);
>  ret = ram_save_queue_pages(ms, qemu_ram_get_idstr(rb), rb_offset,
> -   hostpagesize);
> +   hostpagesize, true);
>  
>  if (ret < 0) {
>  error_report("%s: Save: %"PRIx64 " failed!",
>   __func__, (uint64_t)msg.arg.pagefault.address);
>  break;
>  }
> +
> +/* Note: In the setup process, snapshot_thread may modify VM's
> +* write-protected pages, we should not block it there, or there
> +* will be an deadlock error.
> +*/
> +if (migration_in_setup(ms)) {
> +uint64_t host = msg.arg.pagefault.address;
> +
> +host &= ~(hostpagesize - 1);
> +ret = ram_set_pages_wp(host, getpagesize(), true,
> +   us->userfault_fd);
> +if (ret < 0) {
> +error_report("Remove page's write-protect failed");
> +}
> +}
>  }
>  }
>  trace_postcopy_ram_fault_thread_exit();
> diff --git a/migration/ram.c b/migration/ram.c
> index 8656719..747f9aa 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -233,6 +233,7 @@ struct PageSearchStatus {
>  ram_addr_t   offset;
>  /* Set once we wrap around */
>  bool complete_round;
> +uint8_t *pages_copy;
>  };
>  typedef struct PageSearchStatus PageSearchStatus;
>  
> @@ -742,7 +743,12 @@ static int ram_save_page(QEMUFile *f, PageSearchStatus 
> *pss,
>  RAMBlock *block = pss->block;
>  ram_addr_t offset = pss->offset;
>  
> -p = block->host + offset;
> +/* If we have a copy of this page, use the backup page first */
> +if (pss->pages_copy) {
> +p = pss->pages_copy;
> +} else {
> +p = block->host + offset;
> +}
>  
>  /* In doubt sent page as normal */
>  bytes_xmit = 0;
> @@ -926,7 +932,12 @@ static int ram_save_compressed_page(QEMUFile *f, 
> PageSearchStatus *pss,
>  RAMBlock *block = pss->block;
>  ram_addr_t offset = pss->offset;
>  
> -p = block->host + offset;
> +/* If we 

Re: [Qemu-devel] [PATCH v3 01/11] util/qht: Document memory ordering assumptions

[Sergey Fedorov]

On 13/07/16 10:36, Paolo Bonzini wrote:
>
> On 13/07/2016 01:19, Emilio G. Cota wrote:
>> I wouldn't put those comments in the source--seqlock callers should
>> know what they're doing, and what barriers seqlocks imply.
> In general I'd agree with you, however in this case the "begin" calls
> are what implements QHT's guarantee *for the caller*, so I think it's
> worth having the comments.  In other words, if for any reason you do
> anything before the read_begin and write_begin you still have to provide
> barrier semantics.  It's not an explanation, it's a protection against
> future mistakes.

Exactly :)

> There's no need for such comment at read_retry and write_end callsites,
> though.

Why?

> Also, it's spelled "guarantee". :)

Hmm, I can't see where the spelling isn't correct.

Thanks,
Sergey

Re: [Qemu-devel] [PATCH v8 5/5] docs: Add a generic loader explanation document

[Alistair Francis]

On Tue, Jul 12, 2016 at 9:31 AM, Peter Maydell  wrote:
> On 2 July 2016 at 02:07, Alistair Francis  wrote:
>> Signed-off-by: Alistair Francis 
>> ---
>> V8:
>>  - Improve documentation
>> V6:
>>  - Fixup documentation
>> V4:
>>  - Re-write to be more comprehensive
>>
>>  docs/generic-loader.txt | 60 
>> +
>>  1 file changed, 60 insertions(+)
>>  create mode 100644 docs/generic-loader.txt
>
>> diff --git a/docs/generic-loader.txt b/docs/generic-loader.txt
>> new file mode 100644
>> index 000..34684fc
>> --- /dev/null
>> +++ b/docs/generic-loader.txt
>> @@ -0,0 +1,60 @@
>> +Copyright (c) 2016 Xilinx Inc.
>> +
>> +This work is licensed under the terms of the GNU GPL, version 2 or later.  
>> See
>> +the COPYING file in the top-level directory.
>> +
>> +
>> +The 'loader' device allows the user to load multiple images or values into
>> +QEMU at startup.
>> +
>> +Loading Memory Values
>> +-
>> +The loader device allows memory values to be set from the command line. This
>> +can be done by following the syntax below:
>> +
>> +-device loader,addr=,data=,data-len=
>> +-device loader,addr=,cpu-num=
>> +
>> +  - The address to store the data or the value to use as the
>> +  CPU's PC.
>> +  - The value to be written to the address. The maximum size 
>> of
>> +  the data is 8 bytes.
>> +  - The length of the data in bytes. This argument must be
>> +  included if the data argument is.
>> +   - Set to true if the data to be stored on the guest should 
>> be
>> +  written as big endian data. The default is to write little
>> +  endian data.
>> +   - This will cause the CPU to be reset and the PC to be set 
>> to
>> +  the value of addr.
>> +
>> +For all values both hex and decimal values are allowed. By default the 
>> values
>> +will be parsed as decimal. To use hex values the user should prefix the 
>> number
>> +with a '0x'.
>> +
>> +An example of loading value 0x800e to address 0xfd1a0104 is:
>> +-device loader,addr=0xfd1a0104,data=0x800e,data-len=4
>> +
>> +Loading Files
>> +-
>> +The loader device also allows files to be loaded into memory. This can be 
>> done
>> +similarly to setting memory values. The syntax is shown below:
>> +
>> +-device loader,file=,addr=,cpu-num=,force-raw=
>> +
>> +  - A file to be loaded into memory
>> +  - The addr in memory that the file should be loaded. This is
>> +  ignored if you are using an ELF (unless force-raw is 
>> true).
>> +  This is required if you aren't loading an ELF.
>> +   - This specifies the CPU that should be used. This is an
>> +  optional argument and will cause the CPU's PC to be set to
>> +  where the image is stored. This option should only be used
>> +  for the boot image.
>
> For an ELF file we use the start address specified in the ELF header, right?

Yes, I have specified that now in the docs.

>
> We also end up writing to that CPU's address space, but only for ELF files.

Yes, this will apply to all images in the next version.

>
>> + - Forces the file to be treated as a raw image. This can be
>> +  used to specify the load address of ELF files.
>
> Presumably it results in the ELF file being dumped into memory as a raw
> image without any regard to what the segment headers say about loading,
> offsets, etc?

Correct, I have made that more clear.

Thanks,

Alistair

>
>> +
>> +For all values both hex and decimal values are allowed. By default the 
>> values
>> +will be parsed as decimal. To use hex values the user should prefix the 
>> number
>> +with a '0x'.
>> +
>> +An example of loading an ELF file which CPU0 will boot is shown below:
>> +-device loader,file=./images/boot.elf,cpu-num=0
>
> thanks
> -- PMM
>

Re: [Qemu-devel] [PATCH v8 4/5] generic-loader: Add a generic loader

[Alistair Francis]

On Tue, Jul 12, 2016 at 9:39 AM, Peter Maydell  wrote:
> On 2 July 2016 at 02:07, Alistair Francis  wrote:
>> Add a generic loader to QEMU which can be used to load images or set
>> memory values.
>>
>> Signed-off-by: Alistair Francis 
>> ---
>> V8:
>>  - Code corrections
>>  - Rebase
>> V7:
>>  - Rebase
>> V6:
>>  - Add error checking
>> V5:
>>  - Rebase
>> V4:
>>  - Allow the loader to work with every architecture
>>  - Move the file to hw/core
>>  - Increase the maximum number of CPUs
>>  - Make the CPU operations conditional
>>  - Convert the cpu option to cpu-num
>>  - Require the user to specify endianess
>> V3:
>>  - Pass the ram_size to load_image_targphys()
>> V2:
>>  - Add maintainers entry
>>  - Perform bounds checking
>>  - Register and unregister the reset in the realise/unrealise
>> Changes since RFC:
>>  - Add BE support
>>
>>  MAINTAINERS  |   6 ++
>>  hw/core/Makefile.objs|   2 +
>>  hw/core/generic-loader.c | 177 
>> +++
>>  include/hw/core/generic-loader.h |  45 ++
>>  4 files changed, 230 insertions(+)
>>  create mode 100644 hw/core/generic-loader.c
>>  create mode 100644 include/hw/core/generic-loader.h
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 2ab6e3b..0077e22 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -992,6 +992,12 @@ M: Dmitry Fleytman 
>>  S: Maintained
>>  F: hw/net/e1000e*
>>
>> +Generic Loader
>> +M: Alistair Francis 
>> +S: Maintained
>> +F: hw/core/generic-loader.c
>> +F: include/hw/core/generic-loader.h
>> +
>>  Subsystems
>>  --
>>  Audio
>> diff --git a/hw/core/Makefile.objs b/hw/core/Makefile.objs
>> index 82a9ef8..ab238fa 100644
>> --- a/hw/core/Makefile.objs
>> +++ b/hw/core/Makefile.objs
>> @@ -16,3 +16,5 @@ common-obj-$(CONFIG_SOFTMMU) += null-machine.o
>>  common-obj-$(CONFIG_SOFTMMU) += loader.o
>>  common-obj-$(CONFIG_SOFTMMU) += qdev-properties-system.o
>>  common-obj-$(CONFIG_PLATFORM_BUS) += platform-bus.o
>> +
>> +obj-$(CONFIG_SOFTMMU) += generic-loader.o
>> diff --git a/hw/core/generic-loader.c b/hw/core/generic-loader.c
>> new file mode 100644
>> index 000..c9b0572
>> --- /dev/null
>> +++ b/hw/core/generic-loader.c
>> @@ -0,0 +1,177 @@
>> +/*
>> + * Generic Loader
>> + *
>> + * Copyright (C) 2014 Li Guang
>> + * Copyright (C) 2016 Xilinx Inc.
>> + * Written by Li Guang 
>> + *
>> + * This program is free software; you can redistribute it and/or modify it
>> + * under the terms of the GNU General Public License as published by the
>> + * Free Software Foundation; either version 2 of the License, or
>> + * (at your option) any later version.
>> + *
>> + * This program is distributed in the hope that it will be useful, but 
>> WITHOUT
>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>> + * for more details.
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "qom/cpu.h"
>> +#include "hw/sysbus.h"
>> +#include "sysemu/dma.h"
>> +#include "hw/loader.h"
>> +#include "qapi/error.h"
>> +#include "hw/core/generic-loader.h"
>> +
>> +#define CPU_NONE 0x
>
> This should be updated now we've widened the type from 16 bits.

Fixed

>
>> +
>> +static void generic_loader_reset(void *opaque)
>> +{
>> +GenericLoaderState *s = GENERIC_LOADER(opaque);
>> +
>> +if (s->cpu) {
>> +CPUClass *cc = CPU_GET_CLASS(s->cpu);
>> +cpu_reset(s->cpu);
>> +if (cc) {
>> +cc->set_pc(s->cpu, s->addr);
>
> If something else sets the PC later on this gets lost, which is
> ugly and fragile. It's kinda-sorta OK that we bodge this for things
> like the hw/arm/boot.c code because all the pieces are inside QEMU,
> but when you start adding in command line devices I'm a bit
> nervous. Maybe we should actually figure out our reset order
> woes properly ?

This is up to the user to specify the commands in the order they want
them applied. What other method should we be using?

>
>> +}
>> +}
>> +
>> +if (s->data_len) {
>> +assert(s->data_len < sizeof(s->data));
>> +dma_memory_write((s->cpu ? s->cpu : first_cpu)->as, s->addr, 
>> >data,
>> + s->data_len);
>
> We rule out specifying cpu_num below, so when is s->cpu non-NULL?

I have removed this.

>
>> +}
>> +}
>> +
>> +static void generic_loader_realize(DeviceState *dev, Error **errp)
>> +{
>> +GenericLoaderState *s = GENERIC_LOADER(dev);
>> +hwaddr entry;
>> +int big_endian;
>> +int size = 0;
>> +
>> +/* Perform some error checking on the user's options */
>> +if (s->data || s->data_len  || s->data_be) {
>> +/* User is loading memory values */
>> +if (s->file) {
>> +error_setg(errp, "Specifying a file is not supported when 

Re: [Qemu-devel] [PATCH v5 02/10] HBitmap: Introduce "meta" bitmap to track bit changes

[John Snow]

On 06/22/2016 11:22 AM, Max Reitz wrote:
> On 03.06.2016 06:32, Fam Zheng wrote:
>> Upon each bit toggle, the corresponding bit in the meta bitmap will be
>> set.
>>
>> Signed-off-by: Fam Zheng 
>> Reviewed-by: John Snow 
>> ---
>>  block/dirty-bitmap.c   |  2 +-
>>  include/qemu/hbitmap.h | 17 +
>>  util/hbitmap.c | 69 
>> +++---
>>  3 files changed, 72 insertions(+), 16 deletions(-)
>>
>> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
>> index ec073ee..628b77c 100644
>> --- a/block/dirty-bitmap.c
>> +++ b/block/dirty-bitmap.c
>> @@ -231,7 +231,7 @@ static void 
>> bdrv_do_release_matching_dirty_bitmap(BlockDriverState *bs,
>>  BdrvDirtyBitmap *bm, *next;
>>  QLIST_FOREACH_SAFE(bm, >dirty_bitmaps, list, next) {
>>  if ((!bitmap || bm == bitmap) && (!only_named || bm->name)) {
>> -assert(!bitmap->active_iterators);
>> +assert(!bm->active_iterators);
> 
> Same comment as for v4, this should be squashed into the previous patch.
> 
> Max
> 

Ah, replied too soon on #1. Thanks :)

Re: [Qemu-devel] [PATCH v8 2/5] loader: All a custom SddressSpace when loading ROMs

[Alistair Francis]

On Tue, Jul 12, 2016 at 9:56 AM, Peter Maydell  wrote:
> On 2 July 2016 at 02:07, Alistair Francis  wrote:
>> When loading ROMs allow the caller to specify an AddressSpace to use for
>> the load.
>>
>> Signed-off-by: Alistair Francis 
>> ---
>> V8:
>>  - Introduce an RFC version of AddressSpace loading support
>>
>>  hw/core/loader.c | 18 --
>>  include/hw/elf_ops.h |  2 +-
>>  include/hw/loader.h  | 10 ++
>>  3 files changed, 19 insertions(+), 11 deletions(-)
>>
>> diff --git a/hw/core/loader.c b/hw/core/loader.c
>> index 53e0e41..fcbcfbf 100644
>> --- a/hw/core/loader.c
>> +++ b/hw/core/loader.c
>> @@ -777,6 +777,7 @@ struct Rom {
>>
>>  uint8_t *data;
>>  MemoryRegion *mr;
>> +AddressSpace *as;
>>  int isrom;
>>  char *fw_dir;
>>  char *fw_file;
>> @@ -833,7 +834,8 @@ static void *rom_set_mr(Rom *rom, Object *owner, const 
>> char *name)
>>
>>  int rom_add_file(const char *file, const char *fw_dir,
>>   hwaddr addr, int32_t bootindex,
>> - bool option_rom, MemoryRegion *mr)
>> + bool option_rom, MemoryRegion *mr,
>> + AddressSpace *as)
>
> We seem to add this argument to the function but never use it?

I have fixed that.

>
> I think specifying both mr and as should be an error.

Agreed, it is now an error.

>
>>  {
>>  MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
>>  Rom *rom;
>> @@ -969,7 +971,7 @@ MemoryRegion *rom_add_blob(const char *name, const void 
>> *blob, size_t len,
>>   * memory ownership of "data", so we don't have to allocate and copy the 
>> buffer.
>>   */
>>  int rom_add_elf_program(const char *name, void *data, size_t datasize,
>> -size_t romsize, hwaddr addr)
>> +size_t romsize, hwaddr addr, AddressSpace *as)
>>  {
>>  Rom *rom;
>>
>> @@ -979,18 +981,19 @@ int rom_add_elf_program(const char *name, void *data, 
>> size_t datasize,
>>  rom->datasize = datasize;
>>  rom->romsize  = romsize;
>>  rom->data = data;
>> +rom->as   = as;
>>  rom_insert(rom);
>>  return 0;
>>  }
>>
>>  int rom_add_vga(const char *file)
>>  {
>> -return rom_add_file(file, "vgaroms", 0, -1, true, NULL);
>> +return rom_add_file(file, "vgaroms", 0, -1, true, NULL, NULL);
>>  }
>>
>>  int rom_add_option(const char *file, int32_t bootindex)
>>  {
>> -return rom_add_file(file, "genroms", 0, bootindex, true, NULL);
>> +return rom_add_file(file, "genroms", 0, bootindex, true, NULL, NULL);
>>  }
>>
>>  static void rom_reset(void *unused)
>> @@ -1008,7 +1011,8 @@ static void rom_reset(void *unused)
>>  void *host = memory_region_get_ram_ptr(rom->mr);
>>  memcpy(host, rom->data, rom->datasize);
>>  } else {
>> -cpu_physical_memory_write_rom(_space_memory,
>> +cpu_physical_memory_write_rom(rom->as ? rom->as :
>> +_space_memory,
>>rom->addr, rom->data, 
>> rom->datasize);
>>  }
>>  if (rom->isrom) {
>> @@ -1031,12 +1035,13 @@ int rom_check_and_register_reset(void)
>>  hwaddr addr = 0;
>>  MemoryRegionSection section;
>>  Rom *rom;
>> +AddressSpace *as = NULL;
>>
>>  QTAILQ_FOREACH(rom, , next) {
>>  if (rom->fw_file) {
>>  continue;
>>  }
>> -if (addr > rom->addr) {
>> +if ((addr > rom->addr) && (as == rom->as)) {
>>  fprintf(stderr, "rom: requested regions overlap "
>>  "(rom %s. free=0x" TARGET_FMT_plx
>>  ", addr=0x" TARGET_FMT_plx ")\n",
>> @@ -1048,6 +1053,7 @@ int rom_check_and_register_reset(void)
>>  section = memory_region_find(get_system_memory(), rom->addr, 1);
>
> (An unrelated bug but I've just noticed that this code doesn't
> make sense for ROMs which go into a memory region rather than
> at an address in the system address space.)

I'll add a patch to fix this as well.

>
>>  rom->isrom = int128_nz(section.size) && 
>> memory_region_is_rom(section.mr);
>>  memory_region_unref(section.mr);
>> +as = rom->as;
>>  }
>
> I don't think this attempt at checking is going to actually
> catch all the overlap cases if you allow multiple address
> spaces. The rom_insert() code orders roms in this
> list purely by load address, so you could get cases like
>  [AS A, 0..0x1000], [AS B, 0..0x1000], [AS A, 0x500..0x1000]
> where entries 1 and 3 overlap but won't get caught.
> You probably need to order the list by AS first and then
> by address, and then adjust this code accordingly.

I have changed the ordering now to be by AddressSpace and load address.

Thanks,

Alistair

>
>>  qemu_register_reset(rom_reset, NULL);
>>  roms_loaded = 1;
>> diff --git a/include/hw/elf_ops.h 

Re: [Qemu-devel] [PATCH 6/9] Convert cpu_memory_rw_debug to use MMUAccessType

[Andrey Smirnov]

On Wed, Jul 13, 2016 at 2:52 AM, Peter Maydell  wrote:
> On 13 July 2016 at 01:54, David Gibson  wrote:
>> On Tue, Jul 12, 2016 at 09:05:24AM -0700, Andrey Smirnov wrote:
>>> Please let me know how I should proceed with this series.
>>
>> My preference would be to start the series with something which
>> renames MMUAccessType and its individual values to something more
>> generic, including changing the existing users.
>>
>> But I can't speak to what Peter would prefer.
>
> Yes, if we're going to rename the type then that's the way to do it.

Any suggestions on naming?

Thanks,
Andrey

Re: [Qemu-devel] [PATCH v3 00/11] Reduce lock contention on TCG hot-path

[Sergey Fedorov]

On 13/07/16 10:39, Paolo Bonzini wrote:
> On 12/07/2016 22:13, Sergey Fedorov wrote:
>> From: Sergey Fedorov 
>>
>> Hi,
>>
>> This is my respin of Alex's v2 series [1].
>>
>> The first 8 patches are preparation for the patch 9, the subject matter
>> of this series, which enables lockless translation block lookup. The
>> main change here is that Paolo's suggestion is implemented: TBs are
>> marked with invalid CPU state early during invalidation. This allows to
>> make lockless lookup safe from races on 'tb_jmp_cache' and direct block
>> chaining.
> Thanks for looking at the suggestion again and especially for perfecting it!
>
>> The patch 10 is a simple solution to avoid unnecessary bouncing on
>> 'tb_lock' between tb_gen_code() and tb_add_jump(). A local variable is
>> used to keep track of whether 'tb_lock' has already been taken.
>>
>> The last patch is my attempt to restructure tb_find_{fast,slow}() into a
>> single function tb_find(). I think it will be easier to follow the
>> locking scheme this way. However, I am afraid this last patch can be
>> controversial, so it can be simply dropped.
> Actually I agree entirely with it.
>
> If anything, for historical reasons one might rename tb_find_physical to
> tb_find_slow and leave the tb_find_fast name, but I think the patch is
> good as is.

Nice to hear that :)

>
> Have you measured performance with the series?  In any case, it's nice
> to see MTTCG finally taking shape!

No, I didn't measured the performance. Maybe Alex can help me with this?

Thanks,
Sergey

Re: [Qemu-devel] [PULL] OpenBIOS: switch over to official OpenBIOS git repo

[Peter Maydell]

On 12 July 2016 at 19:17, Mark Cave-Ayland
 wrote:
> Hi Peter,
>
> This commit simply updates the OpenBIOS submodule to point to the same HEAD 
> as the new official
> repository hosted on github. Please pull.
>
>
> ATB,
>
> Mark.
>
>
> The following changes since commit ca3d87d4c84032f19478010b5604cac88b045c25:
>
>   Merge remote-tracking branch 'remotes/armbru/tags/pull-include-2016-07-12' 
> into staging (2016-07-12 16:04:36 +0100)
>
> are available in the git repository at:
>
>
>   https://github.com/mcayland/qemu.git tags/qemu-openbios-signed
>
> for you to fetch changes up to fa3007e7ec3a1e19a85912482ded024f95b48dc8:
>
>   OpenBIOS: switch over to official OpenBIOS git repo (2016-07-12 19:07:56 
> +0100)
>
> 
> OpenBIOS: switch over to official OpenBIOS git repo

Applied, thanks.

-- PMM