Re: [Qemu-devel] [Qemu-block] [PATCH] throttle: test that snapshots move the throttling configuration

[Alberto Garcia]

On Fri 18 Sep 2015 05:54:36 PM CEST, Max Reitz  wrote:

>>  create mode 100644 tests/qemu-iotests/096
>>  create mode 100644 tests/qemu-iotests/096.out
>
> Looks good, I'd just like to throw in that 096 is in use by my
> looks-dead-but-actually-is-not and
> only-waiting-for-the-blockbackend-and-media-series-to-get-merged series
> "block: Rework bdrv_close_all()":
> http://lists.nongnu.org/archive/html/qemu-block/2015-03/msg00048.html

Ah, I don't care about the number, if someone wants to change it when
applying my patch I have no problem with that.

Berto

Re: [Qemu-devel] [PATCH v18 00/21] Deterministic replay core

[Pavel Dovgaluk]

Hi!

Paolo, have you reviewed these patches?

Pavel Dovgalyuk

> -Original Message-
> From: Pavel Dovgalyuk [mailto:pavel.dovga...@ispras.ru]
> Sent: Thursday, September 17, 2015 7:23 PM
> To: qemu-devel@nongnu.org
> Cc: edgar.igles...@xilinx.com; peter.mayd...@linaro.org; 
> igor.rubi...@gmail.com;
> ebl...@redhat.com; mark.bur...@greensocs.com; r...@ispras.ru; 
> batuz...@ispras.ru;
> maria.klimushenk...@ispras.ru; pavel.dovga...@ispras.ru; pbonz...@redhat.com; 
> hi...@cert.org;
> alex.ben...@linaro.org; fred.kon...@greensocs.com
> Subject: [PATCH v18 00/21] Deterministic replay core
> 
> This set of patches is related to the reverse execution and deterministic
> replay of qemu execution. This implementation of deterministic replay can
> be used for deterministic debugging of guest code through gdb remote
> interface.
> 
> Core set of patches does not include support for reverse debugging commands
> of gdb, block devices' operations, USB replay support.
> 
> These patches include only core function of the replay,
> excluding the support for replaying serial, audio, network, and USB devices'
> operations. Reverse debugging and monitor commands were also excluded to
> be submitted later as separate patches.
> 
> Execution recording writes non-deterministic events log, which can be later
> used for replaying the execution anywhere and for unlimited number of times.
> It also supports checkpointing for faster rewinding during reverse debugging.
> Execution replaying reads the log and replays all non-deterministic events
> including external input, hardware clocks, and interrupts.
> 
> Full version of deterministic replay has the following features:
>  * Deterministically replays whole system execution and all contents of the 
> memory,
>state of the hadrware devices, clocks, and screen of the VM.
>  * Writes execution log into the file for latter replaying for multiple times
>on different machines.
>  * Supports i386, x86_64, ARM, PowerPC, and MIPS hardware platforms.
>  * Performs deterministic replay of all operations with keyboard and mouse
>input devices.
>  * Supports auto-checkpointing for convenient reverse debugging.
> 
> Usage of the record/replay core:
>  * First, record the execution, by adding the following string to the command 
> line:
>'-icount shift=7,rr=record,rrfile=replay.bin -net none'.
>Block devices' images are not actually changed in the recording mode,
>because all of the changes are written to the temporary overlay file.
>  * Then you can replay it for the multiple times by using another command
>line option: '-icount shift=7,rr=replay,rrfile=replay.bin -net none'
>  * '-net none' option should also be specified if network replay patches
>are not applied.
>  * Do not add any disk images to VM, because they are not supported by
>the core patches.
> 
> Papers with description of deterministic replay implementation:
> http://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html
> http://dl.acm.org/citation.cfm?id=2786805.2803179
> 
> Public repository with current version of the patches:
> https://github.com/Dovgalyuk/qemu/tree/rr-17
> 
> Modifications of qemu include:
>  * wrappers for clock and time functions to save their return values in the 
> log
>  * saving different asynchronous events (e.g. system shutdown) into the log
>  * synchronization of the threads from thread pool
>  * recording/replaying user input (mouse and keyboard)
>  * adding internal events for cpu and io synchronization
> 
> v18 changes:
>  * Patches were updated to match upstream version
>  * Added missed replay-user.c file
> 
> v17 changes:
>  * Removed useless stub functions (as suggested by Paolo Bonzini)
>  * Refined checkpoint-related code (as suggested by Paolo Bonzini)
>  * Improved icount processing (as suggested by Paolo Bonzini)
>  * Added checkpoint for suspend event (as suggested by Paolo Bonzini)
>  * Fixed linux-user configurations build
>  * Minor fixes
> 
> v16 changes:
>  * Several warnings were fixed
> 
> v15 changes:
>  * Tested record/replay with MIPS and PowerPC guests
>  * Published the patches on github
>  * Fixed replay mutex operation in icount mode
>  * Fixed timers processing in record/replay mode
> 
> v14 changes:
>  * Minor fixes
> 
> v13 changes:
>  * Introduced "ptimer trigger" event (as suggested by Paolo Bonzini)
> 
> v12 changes:
>  * Removed block patches from the core patches set.
> 
> v11 changes:
>  * Fixed instructions event processing.
>  * Added some mutex protection calls for replay.
>  * Fixed replaying read operations for qcow2.
>  * Fixed rtc reads on initializations stage.
>  * Eliminated some warnings in replay module.
>  * Fixed misprints in documentation for replay (as suggested by Eric Blake)
> 
> v10 changes:
>  * Fixed queue processing for bottom halves (as suggested by Paolo Bonzini)
>  * Rewritten several replay functions (as suggested by Paolo Bonzini)
>  * Some minor fixes.
> 
> v9 changes:
>  * 

Re: [Qemu-devel] [PATCH v4 09/22] vhost: use a function for each call

[Thibaut Collet]

On Sat, Sep 19, 2015 at 12:12 PM,   wrote:
> From: Marc-André Lureau 
>
> Replace the generic vhost_call() by specific functions for each
> function call to help with type safety and changing arguments.
>
> Signed-off-by: Marc-André Lureau 
> ---
>  hw/net/vhost_net.c|  12 +-
>  hw/scsi/vhost-scsi.c  |   7 +-
>  hw/virtio/vhost-backend.c | 140 +++--
>  hw/virtio/vhost-user.c| 402 
> ++
>  hw/virtio/vhost.c |  34 ++--
>  include/hw/virtio/vhost-backend.h |  59 +-
>  6 files changed, 484 insertions(+), 170 deletions(-)
>
> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
> index 9d32d76..d116fb3 100644
> --- a/hw/net/vhost_net.c
> +++ b/hw/net/vhost_net.c
> @@ -243,8 +243,7 @@ static int vhost_net_start_one(struct vhost_net *net,
>  file.fd = net->backend;
>  for (file.index = 0; file.index < net->dev.nvqs; ++file.index) {
>  const VhostOps *vhost_ops = net->dev.vhost_ops;
> -r = vhost_ops->vhost_call(&net->dev, VHOST_NET_SET_BACKEND,
> -  &file);
> +r = vhost_ops->vhost_net_set_backend(&net->dev, &file);
>  if (r < 0) {
>  r = -errno;
>  goto fail;
> @@ -257,8 +256,7 @@ fail:
>  if (net->nc->info->type == NET_CLIENT_OPTIONS_KIND_TAP) {
>  while (file.index-- > 0) {
>  const VhostOps *vhost_ops = net->dev.vhost_ops;
> -int r = vhost_ops->vhost_call(&net->dev, VHOST_NET_SET_BACKEND,
> -  &file);
> +int r = vhost_ops->vhost_net_set_backend(&net->dev, &file);
>  assert(r >= 0);
>  }
>  }
> @@ -280,15 +278,13 @@ static void vhost_net_stop_one(struct vhost_net *net,
>  if (net->nc->info->type == NET_CLIENT_OPTIONS_KIND_TAP) {
>  for (file.index = 0; file.index < net->dev.nvqs; ++file.index) {
>  const VhostOps *vhost_ops = net->dev.vhost_ops;
> -int r = vhost_ops->vhost_call(&net->dev, VHOST_NET_SET_BACKEND,
> -  &file);
> +int r = vhost_ops->vhost_net_set_backend(&net->dev, &file);
>  assert(r >= 0);
>  }
>  } else if (net->nc->info->type == NET_CLIENT_OPTIONS_KIND_VHOST_USER) {
>  for (file.index = 0; file.index < net->dev.nvqs; ++file.index) {
>  const VhostOps *vhost_ops = net->dev.vhost_ops;
> -int r = vhost_ops->vhost_call(&net->dev, VHOST_RESET_OWNER,
> -  NULL);
> +int r = vhost_ops->vhost_reset_owner(&net->dev);
>  assert(r >= 0);
>  }
>  }
> diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c
> index bac9ddb..a0034ab 100644
> --- a/hw/scsi/vhost-scsi.c
> +++ b/hw/scsi/vhost-scsi.c
> @@ -45,7 +45,7 @@ static int vhost_scsi_set_endpoint(VHostSCSI *s)
>
>  memset(&backend, 0, sizeof(backend));
>  pstrcpy(backend.vhost_wwpn, sizeof(backend.vhost_wwpn), vs->conf.wwpn);
> -ret = vhost_ops->vhost_call(&s->dev, VHOST_SCSI_SET_ENDPOINT, &backend);
> +ret = vhost_ops->vhost_scsi_set_endpoint(&s->dev, &backend);
>  if (ret < 0) {
>  return -errno;
>  }
> @@ -60,7 +60,7 @@ static void vhost_scsi_clear_endpoint(VHostSCSI *s)
>
>  memset(&backend, 0, sizeof(backend));
>  pstrcpy(backend.vhost_wwpn, sizeof(backend.vhost_wwpn), vs->conf.wwpn);
> -vhost_ops->vhost_call(&s->dev, VHOST_SCSI_CLEAR_ENDPOINT, &backend);
> +vhost_ops->vhost_scsi_clear_endpoint(&s->dev, &backend);
>  }
>
>  static int vhost_scsi_start(VHostSCSI *s)
> @@ -76,8 +76,7 @@ static int vhost_scsi_start(VHostSCSI *s)
>  return -ENOSYS;
>  }
>
> -ret = vhost_ops->vhost_call(&s->dev,
> -VHOST_SCSI_GET_ABI_VERSION, &abi_version);
> +ret = vhost_ops->vhost_scsi_get_abi_version(&s->dev, &abi_version);
>  if (ret < 0) {
>  return -errno;
>  }
> diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c
> index 4d68a27..bf2d1d4 100644
> --- a/hw/virtio/vhost-backend.c
> +++ b/hw/virtio/vhost-backend.c
> @@ -11,19 +11,10 @@
>  #include "hw/virtio/vhost.h"
>  #include "hw/virtio/vhost-backend.h"
>  #include "qemu/error-report.h"
> +#include "linux/vhost.h"
>
>  #include 
>
> -static int vhost_kernel_call(struct vhost_dev *dev, unsigned long int 
> request,
> - void *arg)
> -{
> -int fd = (uintptr_t) dev->opaque;
> -
> -assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_KERNEL);
> -
> -return ioctl(fd, request, arg);
> -}
> -
>  static int vhost_kernel_init(struct vhost_dev *dev, void *opaque)
>  {
>  assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_KERNEL);
> @@ -42,11 +33,136 @@ static int vhost_kernel_cleanup(struct vhost_dev *dev)
>  return close(fd);
>  }
>
> +static int vhost_kernel_call(struct vh

Re: [Qemu-devel] [Qemu-stable] Recent patches for 2.4

[Peter Lieven]

Am 07.09.2015 um 21:14 schrieb Paolo Bonzini:


On 07/09/2015 21:11, Peter Lieven wrote:

It helps, but I have a small issue when my backport of the patch
is applied.

I launch qemu witch a cmdline like this to probe for enforcable CPU types.

echo quit | qemu -enable-kvm -monitor stdio -nodefaults -nographic -cpu 
Haswell,enforce -S

Qemu hangs when entering the quit command. If I omit the -S it works. And 
without the
fix it also works with -S.

My backport (Linux only and without tests etc.) of the original fix is here:
https://github.com/plieven/qemu/commit/0ddcdc62a85f705017df16421d769e82b70f9b37

Could you be missing edec47c?


Indeed this one fixed the deadlock with -S in cmdline.
Are there other important fixes from the series?

I currently have these two:
 main-loop: fix qemu_notify_event for aio_notify optimization
 AioContext: fix broken ctx->dispatching optimization

Thanks,
Peter

Re: [Qemu-devel] [PATCH v5 14/38] block: Remove wr_highest_sector from BlockAcctStats

[Kevin Wolf]

Am 18.09.2015 um 18:59 hat Eric Blake geschrieben:
> On 09/18/2015 09:22 AM, Max Reitz wrote:
> > BlockAcctStats contains statistics about the data transferred from and
> > to the device; wr_highest_sector does not fit in with the rest.
> > 
> > Furthermore, those statistics are supposed to be specific for a certain
> > device and not necessarily for a BDS (see the comment above
> > bdrv_get_stats()); on the other hand, wr_highest_sector may be a rather
> > important information to know for each BDS. When BlockAcctStats is
> > finally removed from the BDS, we will want to keep wr_highest_sector in
> > the BDS.
> > 
> > Finally, wr_highest_sector is renamed to wr_highest_offset and given the
> > appropriate meaning. Externally, it is represented as an offset so there
> > is no point in doing something different internally. Its definition is
> > changed to match that in qapi/block-core.json which is "the offset after
> > the greatest byte written to". Doing so should not cause any harm since
> > if external programs tried to calculate the volume usage by
> > (wr_highest_offset + 512) / volume_size, after this patch they will just
> > assume the volume to be full slightly earlier than before.
> > 
> > Signed-off-by: Max Reitz 
> > Reviewed-by: Eric Blake 
> > Reviewed-by: Alberto Garcia 
> 
> Still holds, but...
> 
> > +++ b/qmp-commands.hx
> > @@ -2481,8 +2481,8 @@ Each json-object contain the following:
> >  - "wr_total_time_ns": total time spend on writes in nano-seconds 
> > (json-int)
> >  - "rd_total_time_ns": total time spend on reads in nano-seconds 
> > (json-int)
> >  - "flush_total_time_ns": total time spend on cache flushes in 
> > nano-seconds (json-int)
> > -- "wr_highest_offset": Highest offset of a sector written since the
> > -   BlockDriverState has been opened (json-int)
> > +- "wr_highest_offset": The offset after the greatest byte written to 
> > the
> > +   BlockDriverState since it has been opened 
> > (json-int)
> 
> ...someday, I'd really like to have this stat show as non-zero even when
> first opening the device (before writing to it). Right now, you have no
> clue how full a backing device is prior to starting a block-commit; you
> have to start writing to it to get a feel for its current usage.

With which value would it start? You don't want the file/device size
because for block devices that's more than is actually used yet.

What you really want to know is a number for the parent node, which
isn't readily available for qcow2 (you would have to scan the refcounts
in the last refcount block on startup) and doesn't really exist for most
other formats (very few of them can be used on block devices, most rely
on the file size).

Kevin


pgpJwnKFZPzag.pgp
Description: PGP signature

Re: [Qemu-devel] [PATCH v4] ppc/spapr: Implement H_RANDOM hypercall in QEMU

[Greg Kurz]

On Mon, 21 Sep 2015 12:10:00 +1000
David Gibson  wrote:

> On Fri, Sep 18, 2015 at 11:05:52AM +0200, Greg Kurz wrote:
> > On Thu, 17 Sep 2015 10:49:41 +0200
> > Thomas Huth  wrote:
> > 
> > > The PAPR interface defines a hypercall to pass high-quality
> > > hardware generated random numbers to guests. Recent kernels can
> > > already provide this hypercall to the guest if the right hardware
> > > random number generator is available. But in case the user wants
> > > to use another source like EGD, or QEMU is running with an older
> > > kernel, we should also have this call in QEMU, so that guests that
> > > do not support virtio-rng yet can get good random numbers, too.
> > > 
> > > This patch now adds a new pseudo-device to QEMU that either
> > > directly provides this hypercall to the guest or is able to
> > > enable the in-kernel hypercall if available. The in-kernel
> > > hypercall can be enabled with the use-kvm property, e.g.:
> > > 
> > >  qemu-system-ppc64 -device spapr-rng,use-kvm=true
> > > 
> > > For handling the hypercall in QEMU instead, a "RngBackend" is
> > > required since the hypercall should provide "good" random data
> > > instead of pseudo-random (like from a "simple" library function
> > > like rand() or g_random_int()). Since there are multiple RngBackends
> > > available, the user must select an appropriate back-end via the
> > > "rng" property of the device, e.g.:
> > > 
> > >  qemu-system-ppc64 -object rng-random,filename=/dev/hwrng,id=gid0 \
> > >-device spapr-rng,rng=gid0 ...
> > > 
> > > See http://wiki.qemu-project.org/Features-Done/VirtIORNG for
> > > other example of specifying RngBackends.
> > > 
> > > Signed-off-by: Thomas Huth 
> > > ---
> > 
> > It is a good thing that the user can choose between in-kernel and backend,
> > and this patch does the work.
> > 
> > This being said, I am not sure about the use case where a user has a hwrng
> > capable platform and wants to run guests without any hwrng support at all is
> > an appropriate default behavior... I guess we will find more users that want
> > in-kernel being the default if it is available.
> > 
> > The patch below modifies yours to do just this: the pseudo-device is only
> > created if hwrng is present and not already created.
> 
> I have mixed feelings about this.  On the one hand, I agree that it
> would be nice to allow H_RANDOM support by default.  On the other hand
> the patch below leaves no way to turn it off for testing purposes.  It

This could be handled with a new spapr property, say 'use-hwrng', defaulting
to true.

> also adds another place where the guest hardware depends on the host
> configuration, which adds to the already substantial mess of ensuring
> that source and destination hardware configuration matches for
> migration.
> 

Yeah, describing the guest hw is really essential for migration... this
is best addressed at the libvirt level with a full XML description of
the machine... but FWIW if we are talking about running pseries on a
POWER8 or newer host, I am not aware about "hwrng-less" boards... but
I am probably missing something :)

Back to Thomas' patch, it does the job and brings H_RANDOM, which is
currently missing.

Acked-by: Greg Kurz 

I could test both use-kvm and backend flavors (including migration).

Tested-by: Greg Kurz 

Thanks.

--
Greg


pgp6suFnxthfL.pgp
Description: OpenPGP digital signature

[Qemu-devel] [PULL 04/26] qapi: New QAPISchemaVisitor

[Markus Armbruster]

The visitor will help keeping the code generation code simple and
reasonably separated from QAPISchema details.

Signed-off-by: Markus Armbruster 
Message-Id: <1442401589-24189-5-git-send-email-arm...@redhat.com>
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 scripts/qapi.py | 64 +
 1 file changed, 64 insertions(+)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 401b87d..36e0702 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -772,6 +772,39 @@ class QAPISchemaEntity(object):
 def check(self, schema):
 pass
 
+def visit(self, visitor):
+pass
+
+
+class QAPISchemaVisitor(object):
+def visit_begin(self, schema):
+pass
+
+def visit_end(self):
+pass
+
+def visit_builtin_type(self, name, info, json_type):
+pass
+
+def visit_enum_type(self, name, info, values, prefix):
+pass
+
+def visit_array_type(self, name, info, element_type):
+pass
+
+def visit_object_type(self, name, info, base, members, variants):
+pass
+
+def visit_alternate_type(self, name, info, variants):
+pass
+
+def visit_command(self, name, info, arg_type, ret_type,
+  gen, success_response):
+pass
+
+def visit_event(self, name, info, arg_type):
+pass
+
 
 class QAPISchemaType(QAPISchemaEntity):
 def c_type(self, is_param=False):
@@ -818,6 +851,9 @@ class QAPISchemaBuiltinType(QAPISchemaType):
 def json_type(self):
 return self._json_type_name
 
+def visit(self, visitor):
+visitor.visit_builtin_type(self.name, self.info, self.json_type())
+
 
 class QAPISchemaEnumType(QAPISchemaType):
 def __init__(self, name, info, values, prefix):
@@ -841,6 +877,10 @@ class QAPISchemaEnumType(QAPISchemaType):
 def json_type(self):
 return 'string'
 
+def visit(self, visitor):
+visitor.visit_enum_type(self.name, self.info,
+self.values, self.prefix)
+
 
 class QAPISchemaArrayType(QAPISchemaType):
 def __init__(self, name, info, element_type):
@@ -856,6 +896,9 @@ class QAPISchemaArrayType(QAPISchemaType):
 def json_type(self):
 return 'array'
 
+def visit(self, visitor):
+visitor.visit_array_type(self.name, self.info, self.element_type)
+
 
 class QAPISchemaObjectType(QAPISchemaType):
 def __init__(self, name, info, base, local_members, variants):
@@ -904,6 +947,10 @@ class QAPISchemaObjectType(QAPISchemaType):
 def json_type(self):
 return 'object'
 
+def visit(self, visitor):
+visitor.visit_object_type(self.name, self.info,
+  self.base, self.local_members, self.variants)
+
 
 class QAPISchemaObjectTypeMember(object):
 def __init__(self, name, typ, optional):
@@ -971,6 +1018,9 @@ class QAPISchemaAlternateType(QAPISchemaType):
 def json_type(self):
 return 'value'
 
+def visit(self, visitor):
+visitor.visit_alternate_type(self.name, self.info, self.variants)
+
 
 class QAPISchemaCommand(QAPISchemaEntity):
 def __init__(self, name, info, arg_type, ret_type, gen, success_response):
@@ -993,6 +1043,11 @@ class QAPISchemaCommand(QAPISchemaEntity):
 self.ret_type = schema.lookup_type(self._ret_type_name)
 assert isinstance(self.ret_type, QAPISchemaType)
 
+def visit(self, visitor):
+visitor.visit_command(self.name, self.info,
+  self.arg_type, self.ret_type,
+  self.gen, self.success_response)
+
 
 class QAPISchemaEvent(QAPISchemaEntity):
 def __init__(self, name, info, arg_type):
@@ -1007,6 +1062,9 @@ class QAPISchemaEvent(QAPISchemaEntity):
 assert isinstance(self.arg_type, QAPISchemaObjectType)
 assert not self.arg_type.variants   # not implemented
 
+def visit(self, visitor):
+visitor.visit_event(self.name, self.info, self.arg_type)
+
 
 class QAPISchema(object):
 def __init__(self, fname):
@@ -1204,6 +1262,12 @@ class QAPISchema(object):
 for ent in self._entity_dict.values():
 ent.check(self)
 
+def visit(self, visitor):
+visitor.visit_begin(self)
+for name in sorted(self._entity_dict.keys()):
+self._entity_dict[name].visit(visitor)
+visitor.visit_end()
+
 
 #
 # Code generation helpers
-- 
2.4.3

[Qemu-devel] [PULL 01/26] qapi: Rename class QAPISchema to QAPISchemaParser

[Markus Armbruster]

I want to name a new class QAPISchema.

While there, make it a new-style class.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-2-git-send-email-arm...@redhat.com>
---
 scripts/qapi.py | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index c4423b7..2a0f465 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -103,7 +103,7 @@ class QAPIExprError(Exception):
 return error_path(self.info['parent']) + \
 "%s:%d: %s" % (self.info['file'], self.info['line'], self.msg)
 
-class QAPISchema:
+class QAPISchemaParser(object):
 
 def __init__(self, fp, previously_included = [], incl_info = None):
 abs_fname = os.path.abspath(fp.name)
@@ -149,8 +149,8 @@ class QAPISchema:
 except IOError, e:
 raise QAPIExprError(expr_info,
 '%s: %s' % (e.strerror, include))
-exprs_include = QAPISchema(fobj, previously_included,
-   expr_info)
+exprs_include = QAPISchemaParser(fobj, previously_included,
+ expr_info)
 self.exprs.extend(exprs_include.exprs)
 else:
 expr_elem = {'expr': expr,
@@ -755,7 +755,7 @@ def check_exprs(exprs):
 
 def parse_schema(fname):
 try:
-schema = QAPISchema(open(fname, "r"))
+schema = QAPISchemaParser(open(fname, "r"))
 return check_exprs(schema.exprs)
 except (QAPISchemaError, QAPIExprError), e:
 print >>sys.stderr, e
-- 
2.4.3

[Qemu-devel] [PULL 03/26] qapi: QAPISchema code generation helper methods

[Markus Armbruster]

New methods c_name(), c_type(), c_null(), json_type(),
alternate_qtype().

Signed-off-by: Markus Armbruster 
Message-Id: <1442401589-24189-4-git-send-email-arm...@redhat.com>
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 scripts/qapi.py | 94 -
 1 file changed, 87 insertions(+), 7 deletions(-)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index d0dfabe..401b87d 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -766,17 +766,57 @@ class QAPISchemaEntity(object):
 self.name = name
 self.info = info
 
+def c_name(self):
+return c_name(self.name)
+
 def check(self, schema):
 pass
 
 
 class QAPISchemaType(QAPISchemaEntity):
-pass
+def c_type(self, is_param=False):
+return c_name(self.name) + pointer_suffix
+
+def c_null(self):
+return 'NULL'
+
+def json_type(self):
+pass
+
+def alternate_qtype(self):
+json2qtype = {
+'string':  'QTYPE_QSTRING',
+'number':  'QTYPE_QFLOAT',
+'int': 'QTYPE_QINT',
+'boolean': 'QTYPE_QBOOL',
+'object':  'QTYPE_QDICT'
+}
+return json2qtype.get(self.json_type())
 
 
 class QAPISchemaBuiltinType(QAPISchemaType):
-def __init__(self, name):
+def __init__(self, name, json_type, c_type, c_null):
 QAPISchemaType.__init__(self, name, None)
+assert not c_type or isinstance(c_type, str)
+assert json_type in ('string', 'number', 'int', 'boolean', 'null',
+ 'value')
+self._json_type_name = json_type
+self._c_type_name = c_type
+self._c_null_val = c_null
+
+def c_name(self):
+return self.name
+
+def c_type(self, is_param=False):
+if is_param and self.name == 'str':
+return 'const ' + self._c_type_name
+return self._c_type_name
+
+def c_null(self):
+return self._c_null_val
+
+def json_type(self):
+return self._json_type_name
 
 
 class QAPISchemaEnumType(QAPISchemaType):
@@ -791,6 +831,16 @@ class QAPISchemaEnumType(QAPISchemaType):
 def check(self, schema):
 assert len(set(self.values)) == len(self.values)
 
+def c_type(self, is_param=False):
+return c_name(self.name)
+
+def c_null(self):
+return c_enum_const(self.name, (self.values + ['MAX'])[0],
+self.prefix)
+
+def json_type(self):
+return 'string'
+
 
 class QAPISchemaArrayType(QAPISchemaType):
 def __init__(self, name, info, element_type):
@@ -803,6 +853,9 @@ class QAPISchemaArrayType(QAPISchemaType):
 self.element_type = schema.lookup_type(self._element_type_name)
 assert self.element_type
 
+def json_type(self):
+return 'array'
+
 
 class QAPISchemaObjectType(QAPISchemaType):
 def __init__(self, name, info, base, local_members, variants):
@@ -840,6 +893,17 @@ class QAPISchemaObjectType(QAPISchemaType):
 self.variants.check(schema, members, seen)
 self.members = members
 
+def c_name(self):
+assert self.info
+return QAPISchemaType.c_name(self)
+
+def c_type(self, is_param=False):
+assert self.info
+return QAPISchemaType.c_type(self)
+
+def json_type(self):
+return 'object'
+
 
 class QAPISchemaObjectTypeMember(object):
 def __init__(self, name, typ, optional):
@@ -904,6 +968,9 @@ class QAPISchemaAlternateType(QAPISchemaType):
 def check(self, schema):
 self.variants.check(schema, [], {})
 
+def json_type(self):
+return 'value'
+
 
 class QAPISchemaCommand(QAPISchemaEntity):
 def __init__(self, name, info, arg_type, ret_type, gen, success_response):
@@ -969,15 +1036,28 @@ class QAPISchema(object):
 def lookup_type(self, name):
 return self.lookup_entity(name, QAPISchemaType)
 
-def _def_builtin_type(self, name):
-self._def_entity(QAPISchemaBuiltinType(name))
+def _def_builtin_type(self, name, json_type, c_type, c_null):
+self._def_entity(QAPISchemaBuiltinType(name, json_type,
+   c_type, c_null))
 if name != '**':
 self._make_array_type(name) # TODO really needed?
 
 def _def_predefineds(self):
-for t in ['str', 'number', 'int', 'int8', 'int16', 'int32', 'int64',
-  'uint8', 'uint16', 'uint32', 'uint64', 'size', 'bool', '**']:
-self._def_builtin_type(t)
+for t in [('str','string',  'char' + pointer_suffix, 'NULL'),
+  ('number', 'number',  'double',   '0'),
+  ('int','int', 'int64_t',  '0'),
+  ('int8',   'int', 'int8_t',   '0'),
+  ('int16',  'int', 'int16_t',  '0'),
+  ('int32',  'int', 'int32_t',  '0'),
+  ('int64',  'int', 'int64_t',  '0'),
+   

[Qemu-devel] [PULL 00/26] qapi: QMP introspection

[Markus Armbruster]

The following changes since commit 18640989a9f5e4d2e84b566c52ff1fccfa0dbf4a:

  Merge remote-tracking branch 'remotes/afaerber/tags/qom-devices-for-peter' 
into staging (2015-09-19 15:59:52 +0100)

are available in the git repository at:

  git://repo.or.cz/qemu/armbru.git tags/pull-qapi-2015-09-21

for you to fetch changes up to 1a9a507b2e3e90aa719c96b4c092e7fad7215f21:

  qapi-introspect: Hide type names (2015-09-21 09:56:49 +0200)


qapi: QMP introspection


Markus Armbruster (26):
  qapi: Rename class QAPISchema to QAPISchemaParser
  qapi: New QAPISchema intermediate reperesentation
  qapi: QAPISchema code generation helper methods
  qapi: New QAPISchemaVisitor
  tests/qapi-schema: Convert test harness to QAPISchemaVisitor
  qapi-types: Convert to QAPISchemaVisitor, fixing flat unions
  qapi-visit: Convert to QAPISchemaVisitor, fixing bugs
  qapi-commands: Convert to QAPISchemaVisitor
  qapi: De-duplicate enum code generation
  qapi-event: Eliminate global variable event_enum_value
  qapi-event: Convert to QAPISchemaVisitor, fixing data with base
  qapi: Replace dirty is_c_ptr() by method c_null()
  qapi: Clean up after recent conversions to QAPISchemaVisitor
  qapi-visit: Rearrange code a bit
  qapi-commands: Rearrange code
  qapi: Rename qmp_marshal_input_FOO() to qmp_marshal_FOO()
  qapi: De-duplicate parameter list generation
  qapi-commands: De-duplicate output marshaling functions
  qapi: Improve built-in type documentation
  qapi: Make output visitor return qnull() instead of NULL
  qapi: Introduce a first class 'any' type
  qom: Don't use 'gen': false for qom-get, qom-set, object-add
  qapi-schema: Fix up misleading specification of netdev_add
  qapi: Pseudo-type '**' is now unused, drop it
  qapi: New QMP command query-qmp-schema for QMP introspection
  qapi-introspect: Hide type names

 .gitignore |   1 +
 Makefile   |   9 +-
 Makefile.objs  |   4 +-
 docs/qapi-code-gen.txt | 355 +--
 docs/writing-qmp-commands.txt  |   8 +-
 include/monitor/monitor.h  |   3 -
 include/qapi/visitor-impl.h|   2 +
 include/qapi/visitor.h |   1 +
 monitor.c  |  18 +-
 qapi-schema.json   |  25 +-
 qapi/introspect.json   | 273 
 qapi/qapi-dealloc-visitor.c|   9 +
 qapi/qapi-visit-core.c |   6 +
 qapi/qmp-input-visitor.c   |  11 +
 qapi/qmp-output-visitor.c  |  17 +-
 qmp-commands.hx| 269 
 qmp.c  |  27 +-
 scripts/qapi-commands.py   | 286 +
 scripts/qapi-event.py  | 243 +++
 scripts/qapi-introspect.py | 213 +++
 scripts/qapi-types.py  | 375 +--
 scripts/qapi-visit.py  | 369 +--
 scripts/qapi.py| 702 ++---
 tests/.gitignore   |   1 +
 tests/Makefile |  15 +-
 tests/qapi-schema/alternate-good.out   |  16 +-
 tests/qapi-schema/args-any.err |   1 +
 .../{type-bypass-no-gen.exit => args-any.exit} |   0
 tests/qapi-schema/args-any.json|   2 +
 .../qapi-schema/{type-bypass.err => args-any.out}  |   0
 tests/qapi-schema/args-member-array.out|  14 +-
 tests/qapi-schema/comments.out |   5 +-
 tests/qapi-schema/empty.out|   4 +-
 tests/qapi-schema/enum-empty.out   |   5 +-
 tests/qapi-schema/event-case.out   |   5 +-
 tests/qapi-schema/flat-union-base-any.err  |   1 +
 ...ion-base-star.exit => flat-union-base-any.exit} |   0
 ...ion-base-star.json => flat-union-base-any.json} |   2 +-
 ...e-bypass-no-gen.out => flat-union-base-any.out} |   0
 tests/qapi-schema/flat-union-base-star.err |   1 -
 tests/qapi-schema/flat-union-base-star.out |   0
 tests/qapi-schema/flat-union-reverse-define.out|  22 +-
 tests/qapi-schema/ident-with-escape.out|   8 +-
 tests/qapi-schema/include-relpath.out  |   5 +-
 tests/qapi-schema/include-repetition.out   |   5 +-
 tests/qapi-schema/include-simple.out   |   5 +-
 tests/qapi-schema/indented-expr.out|   8 +-
 tests/qapi-schema/qa

[Qemu-devel] [PULL 12/26] qapi: Replace dirty is_c_ptr() by method c_null()

[Markus Armbruster]

is_c_ptr() looks whether the end of the C text for the type looks like
a pointer.  Works, but is fragile.

We now have a better tool: use QAPISchemaType method c_null().  The
initializers for non-pointers become prettier: 0, false or the
enumeration constant with the value 0 instead of {0}.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-13-git-send-email-arm...@redhat.com>
---
 scripts/qapi-commands.py | 16 +---
 scripts/qapi.py  |  3 ---
 2 files changed, 5 insertions(+), 14 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index a68517a..cbff356 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -91,18 +91,12 @@ def gen_visitor_input_vars_decl(args):
 bool has_%(argname)s = false;
 ''',
  argname=c_name(memb.name))
-if is_c_ptr(memb.type.c_type()):
-ret += mcgen('''
-%(argtype)s %(argname)s = NULL;
+ret += mcgen('''
+%(c_type)s %(c_name)s = %(c_null)s;
 ''',
- argname=c_name(memb.name),
- argtype=memb.type.c_type())
-else:
-ret += mcgen('''
-%(argtype)s %(argname)s = {0};
-''',
- argname=c_name(memb.name),
- argtype=memb.type.c_type())
+ c_name=c_name(memb.name),
+ c_type=memb.type.c_type(),
+ c_null=memb.type.c_null())
 
 pop_indent()
 return ret
diff --git a/scripts/qapi.py b/scripts/qapi.py
index ba32aac..0ffd02d 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1441,9 +1441,6 @@ def c_type(value, is_param=False):
 assert isinstance(value, str) and value != ""
 return c_name(value) + pointer_suffix
 
-def is_c_ptr(value):
-return value.endswith(pointer_suffix)
-
 def genindent(count):
 ret = ""
 for i in range(count):
-- 
2.4.3

[Qemu-devel] [PULL 08/26] qapi-commands: Convert to QAPISchemaVisitor

[Markus Armbruster]

Output unchanged apart from reordering and white-space.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Message-Id: <1442401589-24189-9-git-send-email-arm...@redhat.com>
Reviewed-by: Daniel P. Berrange 
---
 scripts/qapi-commands.py | 159 +++
 scripts/qapi.py  |   2 +-
 2 files changed, 91 insertions(+), 70 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 12bdc4c..a68517a 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -12,21 +12,22 @@
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-from ordereddict import OrderedDict
 from qapi import *
 import re
 
 def generate_command_decl(name, args, ret_type):
 arglist=""
-for argname, argtype, optional in parse_args(args):
-argtype = c_type(argtype, is_param=True)
-if optional:
-arglist += "bool has_%s, " % c_name(argname)
-arglist += "%s %s, " % (argtype, c_name(argname))
+if args:
+for memb in args.members:
+argtype = memb.type.c_type(is_param=True)
+if memb.optional:
+arglist += "bool has_%s, " % c_name(memb.name)
+arglist += "%s %s, " % (argtype, c_name(memb.name))
 return mcgen('''
 %(ret_type)s qmp_%(name)s(%(args)sError **errp);
 ''',
- ret_type=c_type(ret_type), name=c_name(name),
+ ret_type=(ret_type and ret_type.c_type()) or 'void',
+ name=c_name(name),
  args=arglist)
 
 def gen_err_check(err):
@@ -45,10 +46,11 @@ def gen_sync_call(name, args, ret_type):
 retval=""
 if ret_type:
 retval = "retval = "
-for argname, argtype, optional in parse_args(args):
-if optional:
-arglist += "has_%s, " % c_name(argname)
-arglist += "%s, " % (c_name(argname))
+if args:
+for memb in args.members:
+if memb.optional:
+arglist += "has_%s, " % c_name(memb.name)
+arglist += "%s, " % c_name(memb.name)
 push_indent()
 ret = mcgen('''
 %(retval)sqmp_%(name)s(%(args)s&local_err);
@@ -68,7 +70,7 @@ def gen_visitor_input_containers_decl(args):
 ret = ""
 
 push_indent()
-if len(args) > 0:
+if args:
 ret += mcgen('''
 QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
 QapiDeallocVisitor *md;
@@ -81,22 +83,26 @@ Visitor *v;
 def gen_visitor_input_vars_decl(args):
 ret = ""
 push_indent()
-for argname, argtype, optional in parse_args(args):
-if optional:
-ret += mcgen('''
+
+if args:
+for memb in args.members:
+if memb.optional:
+ret += mcgen('''
 bool has_%(argname)s = false;
 ''',
- argname=c_name(argname))
-if is_c_ptr(argtype):
-ret += mcgen('''
+ argname=c_name(memb.name))
+if is_c_ptr(memb.type.c_type()):
+ret += mcgen('''
 %(argtype)s %(argname)s = NULL;
 ''',
- argname=c_name(argname), argtype=c_type(argtype))
-else:
-ret += mcgen('''
+ argname=c_name(memb.name),
+ argtype=memb.type.c_type())
+else:
+ret += mcgen('''
 %(argtype)s %(argname)s = {0};
 ''',
- argname=c_name(argname), argtype=c_type(argtype))
+ argname=c_name(memb.name),
+ argtype=memb.type.c_type())
 
 pop_indent()
 return ret
@@ -106,7 +112,7 @@ def gen_visitor_input_block(args, dealloc=False):
 errparg = '&local_err'
 errarg = 'local_err'
 
-if len(args) == 0:
+if not args:
 return ret
 
 push_indent()
@@ -124,25 +130,26 @@ v = qapi_dealloc_get_visitor(md);
 v = qmp_input_get_visitor(mi);
 ''')
 
-for argname, argtype, optional in parse_args(args):
-if optional:
+for memb in args.members:
+if memb.optional:
 ret += mcgen('''
 visit_optional(v, &has_%(c_name)s, "%(name)s", %(errp)s);
 ''',
- c_name=c_name(argname), name=argname, errp=errparg)
+ c_name=c_name(memb.name), name=memb.name,
+ errp=errparg)
 ret += gen_err_check(errarg)
 ret += mcgen('''
 if (has_%(c_name)s) {
 ''',
- c_name=c_name(argname))
+ c_name=c_name(memb.name))
 push_indent()
 ret += mcgen('''
 visit_type_%(visitor)s(v, &%(c_name)s, "%(name)s", %(errp)s);
 ''',
- c_name=c_name(argname), name=argname, argtype=argtype,
- visitor=type_name(argtype), errp=errparg)
+ c_name=c_name(memb.name), name=memb.name,
+ visitor=memb.type.c_name(), errp=errparg)

[Qemu-devel] [PULL 09/26] qapi: De-duplicate enum code generation

[Markus Armbruster]

Duplicated in commit 21cd70d.  Yes, we can't import qapi-types, but
that's no excuse.  Move the helpers from qapi-types.py to qapi.py, and
replace the duplicates in qapi-event.py.

The generated event enumeration type's lookup table becomes
const-correct (see commit 2e4450f), and uses explicit indexes instead
of relying on order (see commit 912ae9c).

Signed-off-by: Markus Armbruster 
Message-Id: <1442401589-24189-10-git-send-email-arm...@redhat.com>
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt |  9 ---
 scripts/qapi-event.py  | 67 +++---
 scripts/qapi-types.py  | 55 -
 scripts/qapi.py| 55 +
 4 files changed, 64 insertions(+), 122 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index c9e21fc..81e87d2 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -829,9 +829,9 @@ Example:
 QDECREF(qmp);
 }
 
-const char *example_QAPIEvent_lookup[] = {
-"MY_EVENT",
-NULL,
+const char *const example_QAPIEvent_lookup[] = {
+[EXAMPLE_QAPI_EVENT_MY_EVENT] = "MY_EVENT",
+[EXAMPLE_QAPI_EVENT_MAX] = NULL,
 };
 $ cat qapi-generated/example-qapi-event.h
 [Uninteresting stuff omitted...]
@@ -846,10 +846,11 @@ Example:
 
 void qapi_event_send_my_event(Error **errp);
 
-extern const char *example_QAPIEvent_lookup[];
 typedef enum example_QAPIEvent {
 EXAMPLE_QAPI_EVENT_MY_EVENT = 0,
 EXAMPLE_QAPI_EVENT_MAX = 1,
 } example_QAPIEvent;
 
+extern const char *const example_QAPIEvent_lookup[];
+
 #endif
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index aec2d32..aed45d6 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -153,63 +153,6 @@ def generate_event_implement(api_name, event_name, params):
 
 return ret
 
-
-# Following are the functions that generate an enum type for all defined
-# events, similar to qapi-types.py. Here we already have enum name and
-# values which were generated before and recorded in event_enum_*. It also
-# works around the issue that "import qapi-types" can't work.
-
-def generate_event_enum_decl(event_enum_name, event_enum_values):
-lookup_decl = mcgen('''
-
-extern const char *%(event_enum_name)s_lookup[];
-''',
-event_enum_name = event_enum_name)
-
-enum_decl = mcgen('''
-typedef enum %(event_enum_name)s {
-''',
-  event_enum_name = event_enum_name)
-
-# append automatically generated _MAX value
-enum_max_value = c_enum_const(event_enum_name, "MAX")
-enum_values = event_enum_values + [ enum_max_value ]
-
-i = 0
-for value in enum_values:
-enum_decl += mcgen('''
-%(value)s = %(i)d,
-''',
- value = value,
- i = i)
-i += 1
-
-enum_decl += mcgen('''
-} %(event_enum_name)s;
-''',
-   event_enum_name = event_enum_name)
-
-return lookup_decl + enum_decl
-
-def generate_event_enum_lookup(event_enum_name, event_enum_strings):
-ret = mcgen('''
-
-const char *%(event_enum_name)s_lookup[] = {
-''',
-event_enum_name = event_enum_name)
-
-for string in event_enum_strings:
-ret += mcgen('''
-"%(string)s",
-''',
- string = string)
-
-ret += mcgen('''
-NULL,
-};
-''')
-return ret
-
 (input_file, output_dir, do_c, do_h, prefix, dummy) = parse_command_line()
 
 c_comment = '''
@@ -266,8 +209,7 @@ fdecl.write(mcgen('''
 exprs = QAPISchema(input_file).get_exprs()
 
 event_enum_name = c_name(prefix + "QAPIEvent", protect=False)
-event_enum_values = []
-event_enum_strings = []
+event_names = []
 
 for expr in exprs:
 if expr.has_key('event'):
@@ -286,12 +228,11 @@ for expr in exprs:
 fdef.write(ret)
 
 # Record it, and generate enum later
-event_enum_values.append(event_enum_value)
-event_enum_strings.append(event_name)
+event_names.append(event_name)
 
-ret = generate_event_enum_decl(event_enum_name, event_enum_values)
+ret = generate_enum(event_enum_name, event_names)
 fdecl.write(ret)
-ret = generate_event_enum_lookup(event_enum_name, event_enum_strings)
+ret = generate_enum_lookup(event_enum_name, event_names)
 fdef.write(ret)
 
 close_output(fdef, fdecl)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index d78f529..76c82d9 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -80,61 +80,6 @@ struct %(name)s {
 
 return ret
 
-def generate_enum_lookup(name, values, prefix=None):
-ret = mcgen('''
-
-const char *const %(name)s_lookup[] = {
-''',
-name=c_name(name))
-for value in values:
-index = c_enum_const(name, value, prefix)
-ret += mcgen('''
-[%(index)s] = "%(value)s",
-''',
- index = index, value = value)
-
-max_

[Qemu-devel] [PULL 05/26] tests/qapi-schema: Convert test harness to QAPISchemaVisitor

[Markus Armbruster]

The old code prints the result of parsing (list of expression
dictionaries), and partial results of semantic analysis (list of enum
dictionaries, list of struct dictionaries).

The new code prints a trace of a schema visit, i.e. what the back-ends
are going to use.  Built-in and array types are omitted, because
they're boring.

Signed-off-by: Markus Armbruster 
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 tests/qapi-schema/alternate-good.out|  15 +-
 tests/qapi-schema/args-member-array.out |  13 +-
 tests/qapi-schema/comments.out  |   4 +-
 tests/qapi-schema/empty.out |   3 -
 tests/qapi-schema/enum-empty.out|   4 +-
 tests/qapi-schema/event-case.out|   4 +-
 tests/qapi-schema/flat-union-reverse-define.out |  21 ++-
 tests/qapi-schema/ident-with-escape.out |   7 +-
 tests/qapi-schema/include-relpath.out   |   4 +-
 tests/qapi-schema/include-repetition.out|   4 +-
 tests/qapi-schema/include-simple.out|   4 +-
 tests/qapi-schema/indented-expr.out |   7 +-
 tests/qapi-schema/qapi-schema-test.out  | 218 +---
 tests/qapi-schema/returns-int.out   |   5 +-
 tests/qapi-schema/test-qapi.py  |  47 -
 tests/qapi-schema/type-bypass.out   |   7 +-
 16 files changed, 249 insertions(+), 118 deletions(-)

diff --git a/tests/qapi-schema/alternate-good.out 
b/tests/qapi-schema/alternate-good.out
index 99848ee..3d765ff 100644
--- a/tests/qapi-schema/alternate-good.out
+++ b/tests/qapi-schema/alternate-good.out
@@ -1,6 +1,9 @@
-[OrderedDict([('struct', 'Data'), ('data', OrderedDict([('*number', 'int'), 
('*name', 'str')]))]),
- OrderedDict([('enum', 'Enum'), ('data', ['hello', 'world'])]),
- OrderedDict([('alternate', 'Alt'), ('data', OrderedDict([('value', 'int'), 
('string', 'Enum'), ('struct', 'Data')]))])]
-[{'enum_name': 'Enum', 'enum_values': ['hello', 'world']},
- {'enum_name': 'AltKind', 'enum_values': None}]
-[OrderedDict([('struct', 'Data'), ('data', OrderedDict([('*number', 'int'), 
('*name', 'str')]))])]
+alternate Alt
+case value: int
+case string: Enum
+case struct: Data
+enum AltKind ['value', 'string', 'struct']
+object Data
+member number: int optional=True
+member name: str optional=True
+enum Enum ['hello', 'world']
diff --git a/tests/qapi-schema/args-member-array.out 
b/tests/qapi-schema/args-member-array.out
index c39fa25..b67384c 100644
--- a/tests/qapi-schema/args-member-array.out
+++ b/tests/qapi-schema/args-member-array.out
@@ -1,5 +1,8 @@
-[OrderedDict([('enum', 'abc'), ('data', ['a', 'b', 'c'])]),
- OrderedDict([('struct', 'def'), ('data', OrderedDict([('array', ['abc'])]))]),
- OrderedDict([('command', 'okay'), ('data', OrderedDict([('member1', ['int']), 
('member2', ['def'])]))])]
-[{'enum_name': 'abc', 'enum_values': ['a', 'b', 'c']}]
-[OrderedDict([('struct', 'def'), ('data', OrderedDict([('array', ['abc'])]))])]
+object :obj-okay-arg
+member member1: intList optional=False
+member member2: defList optional=False
+enum abc ['a', 'b', 'c']
+object def
+member array: abcList optional=False
+command okay :obj-okay-arg -> None
+   gen=True success_response=True
diff --git a/tests/qapi-schema/comments.out b/tests/qapi-schema/comments.out
index 4ce3dcf..6161b90 100644
--- a/tests/qapi-schema/comments.out
+++ b/tests/qapi-schema/comments.out
@@ -1,3 +1 @@
-[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
-[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
-[]
+enum Status ['good', 'bad', 'ugly']
diff --git a/tests/qapi-schema/empty.out b/tests/qapi-schema/empty.out
index b7f89a4..e69de29 100644
--- a/tests/qapi-schema/empty.out
+++ b/tests/qapi-schema/empty.out
@@ -1,3 +0,0 @@
-[]
-[]
-[]
diff --git a/tests/qapi-schema/enum-empty.out b/tests/qapi-schema/enum-empty.out
index 3b75c16..e09b00f 100644
--- a/tests/qapi-schema/enum-empty.out
+++ b/tests/qapi-schema/enum-empty.out
@@ -1,3 +1 @@
-[OrderedDict([('enum', 'MyEnum'), ('data', [])])]
-[{'enum_name': 'MyEnum', 'enum_values': []}]
-[]
+enum MyEnum []
diff --git a/tests/qapi-schema/event-case.out b/tests/qapi-schema/event-case.out
index 3764bc7..b5ae4c2 100644
--- a/tests/qapi-schema/event-case.out
+++ b/tests/qapi-schema/event-case.out
@@ -1,3 +1 @@
-[OrderedDict([('event', 'oops')])]
-[]
-[]
+event oops None
diff --git a/tests/qapi-schema/flat-union-reverse-define.out 
b/tests/qapi-schema/flat-union-reverse-define.out
index 1ed7b8a..477fb31 100644
--- a/tests/qapi-schema/flat-union-reverse-define.out
+++ b/tests/qapi-schema/flat-union-reverse-define.out
@@ -1,9 +1,12 @@
-[OrderedDict([('union', 'TestUnion'), ('base', 'TestBase'), ('discriminator', 
'enum1'), ('data', OrderedDict([('value1', 'TestTypeA'), ('value2', 
'TestTypeB')]))]),
- OrderedDict([('struct', 'TestBase'), ('data', OrderedDict([('enum1', 
'TestEnum')]))]),
- OrderedDict([('enum', 'TestEnum'), 

[Qemu-devel] [PULL 14/26] qapi-visit: Rearrange code a bit

[Markus Armbruster]

Move gen_visit_decl() to a better place.  Inline
generate_visit_struct_body().

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-15-git-send-email-arm...@redhat.com>
---
 scripts/qapi-visit.py | 50 --
 1 file changed, 20 insertions(+), 30 deletions(-)

diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index fae70e2..97343cf 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -19,6 +19,16 @@ implicit_structs_seen = set()
 struct_fields_seen = set()
 
 
+def gen_visit_decl(name, scalar=False):
+c_type = c_name(name) + ' *'
+if not scalar:
+c_type += '*'
+return mcgen('''
+void visit_type_%(c_name)s(Visitor *m, %(c_type)sobj, const char *name, Error 
**errp);
+''',
+ c_name=c_name(name), c_type=c_type)
+
+
 def gen_visit_implicit_struct(typ):
 if typ in implicit_structs_seen:
 return ''
@@ -117,12 +127,17 @@ out:
 return ret
 
 
-def gen_visit_struct_body(name):
+def gen_visit_struct(name, base, members):
+ret = gen_visit_struct_fields(name, base, members)
+
 # FIXME: if *obj is NULL on entry, and visit_start_struct() assigns to
 # *obj, but then visit_type_FOO_fields() fails, we should clean up *obj
 # rather than leaving it non-NULL. As currently written, the caller must
 # call qapi_free_FOO() to avoid a memory leak of the partial FOO.
-ret = mcgen('''
+ret += mcgen('''
+
+void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, 
Error **errp)
+{
 Error *err = NULL;
 
 visit_start_struct(m, (void **)obj, "%(name)s", name, sizeof(%(c_name)s), 
&err);
@@ -133,25 +148,10 @@ def gen_visit_struct_body(name):
 visit_end_struct(m, &err);
 }
 error_propagate(errp, err);
-''',
-name=name, c_name=c_name(name))
-return ret
-
-
-def gen_visit_struct(name, base, members):
-ret = gen_visit_struct_fields(name, base, members)
-ret += mcgen('''
-
-void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, 
Error **errp)
-{
-''',
- c_name=c_name(name))
-
-ret += gen_visit_struct_body(name)
-
-ret += mcgen('''
 }
-''')
+''',
+ name=name, c_name=c_name(name))
+
 return ret
 
 
@@ -342,16 +342,6 @@ out:
 return ret
 
 
-def gen_visit_decl(name, scalar=False):
-c_type = c_name(name) + ' *'
-if not scalar:
-c_type += '*'
-return mcgen('''
-void visit_type_%(c_name)s(Visitor *m, %(c_type)sobj, const char *name, Error 
**errp);
-''',
- c_name=c_name(name), c_type=c_type)
-
-
 class QAPISchemaGenVisitVisitor(QAPISchemaVisitor):
 def __init__(self):
 self.decl = None
-- 
2.4.3

[Qemu-devel] [PULL 10/26] qapi-event: Eliminate global variable event_enum_value

[Markus Armbruster]

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-11-git-send-email-arm...@redhat.com>
---
 scripts/qapi-event.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index aed45d6..537da17 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -137,7 +137,7 @@ def generate_event_implement(api_name, event_name, params):
 emit(%(event_enum_value)s, qmp, &local_err);
 
 """,
- event_enum_value = event_enum_value)
+ event_enum_value = c_enum_const(event_enum_name, event_name))
 
 # step 5: clean up
 if params:
@@ -223,7 +223,6 @@ for expr in exprs:
 fdecl.write(ret)
 
 # We need an enum value per event
-event_enum_value = c_enum_const(event_enum_name, event_name)
 ret = generate_event_implement(api_name, event_name, params)
 fdef.write(ret)
 
-- 
2.4.3

[Qemu-devel] [PULL 11/26] qapi-event: Convert to QAPISchemaVisitor, fixing data with base

[Markus Armbruster]

Fixes events whose data is struct with base to include the struct's
base members.  Test case is qapi-schema-test.json's event
__org.qemu_x-command:

{ 'event': '__ORG.QEMU_X-EVENT', 'data': '__org.qemu_x-Struct' }

{ 'struct': '__org.qemu_x-Struct', 'base': '__org.qemu_x-Base',
  'data': { '__org.qemu_x-member2': 'str' } }

{ 'struct': '__org.qemu_x-Base',
  'data': { '__org.qemu_x-member1': '__org.qemu_x-Enum' } }

Patch's effect on generated qapi_event_send___org_qemu_x_event():

-void qapi_event_send___org_qemu_x_event(const char *__org_qemu_x_member2,
+void qapi_event_send___org_qemu_x_event(__org_qemu_x_Enum 
__org_qemu_x_member1,
+const char *__org_qemu_x_member2,
 Error **errp)
 {
 QDict *qmp;
@@ -224,6 +225,10 @@ void qapi_event_send___org_qemu_x_event(
 goto clean;
 }

+visit_type___org_qemu_x_Enum(v, &__org_qemu_x_member1, 
"__org.qemu_x-member1", &local_err);
+if (local_err) {
+goto clean;
+}
 visit_type_str(v, (char **)&__org_qemu_x_member2, 
"__org.qemu_x-member2", &local_err);
 if (local_err) {
 goto clean;

Code is generated in a different order now, but that doesn't matter.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
---
 scripts/qapi-event.py   | 89 +
 tests/qapi-schema/qapi-schema-test.json |  3 --
 2 files changed, 47 insertions(+), 45 deletions(-)

diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 537da17..d19acda 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -2,14 +2,15 @@
 # QAPI event generator
 #
 # Copyright (c) 2014 Wenchao Xia
+# Copyright (c) 2015 Red Hat Inc.
 #
 # Authors:
 #  Wenchao Xia 
+#  Markus Armbruster 
 #
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-from ordereddict import OrderedDict
 from qapi import *
 
 def _generate_event_api_name(event_name, params):
@@ -17,13 +18,13 @@ def _generate_event_api_name(event_name, params):
 l = len(api_name)
 
 if params:
-for argname, argentry, optional in parse_args(params):
-if optional:
-api_name += "bool has_%s,\n" % c_name(argname)
+for m in params.members:
+if m.optional:
+api_name += "bool has_%s,\n" % c_name(m.name)
 api_name += "".ljust(l)
 
-api_name += "%s %s,\n" % (c_type(argentry, is_param=True),
-  c_name(argname))
+api_name += "%s %s,\n" % (m.type.c_type(is_param=True),
+  c_name(m.name))
 api_name += "".ljust(l)
 
 api_name += "Error **errp)"
@@ -51,7 +52,7 @@ def generate_event_implement(api_name, event_name, params):
 """,
 api_name = api_name)
 
-if params:
+if params and params.members:
 ret += mcgen("""
 QmpOutputVisitor *qov;
 Visitor *v;
@@ -72,7 +73,7 @@ def generate_event_implement(api_name, event_name, params):
  event_name = event_name)
 
 # step 3: visit the params if params != None
-if params:
+if params and params.members:
 ret += mcgen("""
 qov = qmp_output_visitor_new();
 g_assert(qov);
@@ -89,15 +90,15 @@ def generate_event_implement(api_name, event_name, params):
 """,
 event_name = event_name)
 
-for argname, argentry, optional in parse_args(params):
-if optional:
+for memb in params.members:
+if memb.optional:
 ret += mcgen("""
 if (has_%(var)s) {
 """,
- var = c_name(argname))
+ var=c_name(memb.name))
 push_indent()
 
-if argentry == "str":
+if memb.type.name == "str":
 var_type = "(char **)"
 else:
 var_type = ""
@@ -109,11 +110,11 @@ def generate_event_implement(api_name, event_name, 
params):
 }
 """,
  var_type = var_type,
- var = c_name(argname),
- type = type_name(argentry),
- name = argname)
+ var=c_name(memb.name),
+ type=memb.type.c_name(),
+ name=memb.name)
 
-if optional:
+if memb.optional:
 pop_indent()
 ret += mcgen("""
 }
@@ -140,7 +141,7 @@ def generate_event_implement(api_name, event_name, params):
  event_enum_value = c_enum_const(event_enum_name, event_name))
 
 # step 5: clean up
-if params:
+if params and params.members:
 ret += mcgen("""
  clean:
 qmp_output_visitor_cleanup(qov);
@@ -153,6 +

[Qemu-devel] [PULL 26/26] qapi-introspect: Hide type names

[Markus Armbruster]

To eliminate the temptation for clients to look up types by name
(which are not ABI), replace all type names by meaningless strings.

Reduces output of query-schema by 13 out of 85KiB.

As a debugging aid, provide option -u to suppress the hiding.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Message-Id: <1442401589-24189-27-git-send-email-arm...@redhat.com>
---
 docs/qapi-code-gen.txt | 36 +---
 qapi/introspect.json   | 12 
 scripts/qapi-introspect.py | 41 +++--
 3 files changed, 56 insertions(+), 33 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index f321d4b..b1c8361 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -530,13 +530,16 @@ additional variant members depending on the value of 
meta-type.
 Each SchemaInfo object describes a wire ABI entity of a certain
 meta-type: a command, event or one of several kinds of type.
 
-SchemaInfo for entities defined in the QAPI schema have the same name
-as in the schema.  This is the case for all commands and events, and
-most types.
+SchemaInfo for commands and events have the same name as in the QAPI
+schema.
 
 Command and event names are part of the wire ABI, but type names are
-not.  Therefore, looking up a type by its name in the QAPI schema is
-wrong.  Look up the command or event, then follow references by name.
+not.  Therefore, the SchemaInfo for types have auto-generated
+meaningless names.  For readability, the examples in this section use
+meaningful type names instead.
+
+To examine a type, start with a command or event using it, then follow
+references by name.
 
 QAPI schema definitions not reachable that way are omitted.
 
@@ -567,8 +570,7 @@ object type without members.  The event may not have a data 
member on
 the wire then.
 
 Each command or event defined with dictionary-valued 'data' in the
-QAPI schema implicitly defines an object type called ":obj-NAME-arg",
-where NAME is the command or event's name.
+QAPI schema implicitly defines an object type.
 
 Example: the SchemaInfo for EVENT_C from section Events
 
@@ -623,12 +625,9 @@ Note that base types are "flattened": its members are 
included in the
 
 A simple union implicitly defines an enumeration type for its implicit
 discriminator (called "type" on the wire, see section Union types).
-Such a type's name is made by appending "Kind" to the simple union's
-name.
 
 A simple union implicitly defines an object type for each of its
-variants.  The type's name is ":obj-NAME-wrapper", where NAME is the
-name of the name of the variant's type.
+variants.
 
 Example: the SchemaInfo for simple union BlockdevOptions from section
 Union types
@@ -659,8 +658,7 @@ Example: the SchemaInfo for BlockRef from section Alternate 
types
 
 The SchemaInfo for an array type has meta-type "array", and variant
 member "element-type", which names the array's element type.  Array
-types are implicitly defined.  An array type's name is made by
-appending "List" to its element type's name.
+types are implicitly defined.
 
 Example: the SchemaInfo for ['str']
 
@@ -1067,13 +1065,13 @@ Example:
 [Uninteresting stuff omitted...]
 
 const char example_qmp_schema_json[] = "["
-"{\"arg-type\": \":empty\", \"meta-type\": \"event\", \"name\": 
\"MY_EVENT\"}, "
+"{\"arg-type\": \"0\", \"meta-type\": \"event\", \"name\": 
\"MY_EVENT\"}, "
+"{\"arg-type\": \"1\", \"meta-type\": \"command\", \"name\": 
\"my-command\", \"ret-type\": \"2\"}, "
+"{\"members\": [], \"meta-type\": \"object\", \"name\": \"0\"}, "
+"{\"members\": [{\"name\": \"arg1\", \"type\": \"2\"}], \"meta-type\": 
\"object\", \"name\": \"1\"}, "
+"{\"members\": [{\"name\": \"integer\", \"type\": \"int\"}, {\"name\": 
\"string\", \"type\": \"str\"}], \"meta-type\": \"object\", \"name\": \"2\"}, "
 "{\"json-type\": \"int\", \"meta-type\": \"builtin\", \"name\": 
\"int\"}, "
-"{\"json-type\": \"string\", \"meta-type\": \"builtin\", \"name\": 
\"str\"}, "
-"{\"members\": [], \"meta-type\": \"object\", \"name\": \":empty\"}, "
-"{\"members\": [{\"name\": \"arg1\", \"type\": \"UserDefOne\"}], 
\"meta-type\": \"object\", \"name\": \":obj-my-command-arg\"}, "
-"{\"members\": [{\"name\": \"integer\", \"type\": \"int\"}, {\"name\": 
\"string\", \"type\": \"str\"}], \"meta-type\": \"object\", \"name\": 
\"UserDefOne\"}, "
-"{\"arg-type\": \":obj-my-command-arg\", \"meta-type\": \"command\", 
\"name\": \"my-command\", \"ret-type\": \"UserDefOne\"}]";
+"{\"json-type\": \"string\", \"meta-type\": \"builtin\", \"name\": 
\"str\"}]";
 $ cat qapi-generated/example-qmp-introspect.h
 [Uninteresting stuff omitted...]
 
diff --git a/qapi/introspect.json b/qapi/introspect.json
index 9c8ad53..cc50dc6 100644
--- a/qapi/introspect.json
+++ b/qapi/introspect.json
@@ -75,17 +75,13 @@
 # @SchemaInfo
 #
 # @name: the entity's name,

[Qemu-devel] [PULL 20/26] qapi: Make output visitor return qnull() instead of NULL

[Markus Armbruster]

Before commit 1d10b44, it crashed.  Since then, it returns NULL, with
a FIXME comment.  The FIXME is valid: code that assumes QObject *
can't be null exists.  I'm not aware of a way to feed this problematic
return value to code that actually chokes on null in the current code,
but the next few commits will create one, failing "make check".

Commit 481b002 solved a very similar problem by introducing a special
null QObject.  Using this special null QObject is clearly the right
way to resolve this FIXME, so do that, and update the test
accordingly.

However, the patch isn't quite right: it messes up the reference
counting.  After about SIZE_MAX visits, the reference counter
overflows, failing the assertion in qnull_destroy_obj().  Because
that's many orders of magnitude more visits of nulls than we expect,
we take this patch despite its flaws, to get the QMP introspection
stuff in without further delay.  We'll want to fix it for real before
the release.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-21-git-send-email-arm...@redhat.com>
---
 qapi/qmp-output-visitor.c   | 8 ++--
 tests/test-qmp-output-visitor.c | 3 ++-
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/qapi/qmp-output-visitor.c b/qapi/qmp-output-visitor.c
index efc19d5..8dce087 100644
--- a/qapi/qmp-output-visitor.c
+++ b/qapi/qmp-output-visitor.c
@@ -66,9 +66,13 @@ static QObject *qmp_output_first(QmpOutputVisitor *qov)
 {
 QStackEntry *e = QTAILQ_LAST(&qov->stack, QStack);
 
-/* FIXME - find a better way to deal with NULL values */
+/*
+ * FIXME Wrong, because qmp_output_get_qobject() will increment
+ * the refcnt *again*.  We need to think through how visitors
+ * handle null.
+ */
 if (!e) {
-return NULL;
+return qnull();
 }
 
 return e->value;
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 338ada0..a48ae72 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -485,7 +485,8 @@ static void test_visitor_out_empty(TestOutputVisitorData 
*data,
 QObject *arg;
 
 arg = qmp_output_get_qobject(data->qov);
-g_assert(!arg);
+g_assert(qobject_type(arg) == QTYPE_QNULL);
+qobject_decref(arg);
 }
 
 static void init_native_list(UserDefNativeListUnion *cvalue)
-- 
2.4.3

[Qemu-devel] [PULL 23/26] qapi-schema: Fix up misleading specification of netdev_add

[Markus Armbruster]

It doesn't take a 'props' argument, let alone one in the format
"NAME=VALUE,..."

The bogus arguments specification doesn't matter due to 'gen': false.
Clean it up to be incomplete rather than wrong, and document the
incompleteness.

While there, improve netdev_add usage example in the manual: add a
device option to show how it's done.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-24-git-send-email-arm...@redhat.com>
---
 docs/qapi-code-gen.txt |  2 +-
 qapi-schema.json   | 13 +++--
 qmp-commands.hx|  4 +++-
 3 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 70fdae7..4e331a0 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -471,7 +471,7 @@ try to avoid adding new commands that rely on this, and 
instead use
 type-safe unions.  For an example of bypass usage:
 
  { 'command': 'netdev_add',
-   'data': {'type': 'str', 'id': 'str', '*props': '**'},
+   'data': {'type': 'str', 'id': 'str'},
'gen': false }
 
 Normally, the QAPI schema is used to describe synchronous exchanges,
diff --git a/qapi-schema.json b/qapi-schema.json
index 756ed9e..3ff9fec 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2079,11 +2079,12 @@
 #
 # @id: the name of the new network backend
 #
-# @props: #optional a list of properties to be passed to the backend in
-# the format 'name=value', like 'ifname=tap0,script=no'
+# Additional arguments depend on the type.
 #
-# Notes: The semantics of @props is not well defined.  Future commands will be
-#introduced that provide stronger typing for backend creation.
+# TODO This command effectively bypasses QAPI completely due to its
+# "additional arguments" business.  It shouldn't have been added to
+# the schema in this form.  It should be qapified properly, or
+# replaced by a properly qapified command.
 #
 # Since: 0.14.0
 #
@@ -2091,8 +2092,8 @@
 #  If @type is not a valid network backend, DeviceNotFound
 ##
 { 'command': 'netdev_add',
-  'data': {'type': 'str', 'id': 'str', '*props': '**'},
-  'gen': false }
+  'data': {'type': 'str', 'id': 'str'},
+  'gen': false }# so we can get the additional arguments
 
 ##
 # @netdev_del:
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 7c74e20..5a54406 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -917,7 +917,9 @@ Arguments:
 
 Example:
 
--> { "execute": "netdev_add", "arguments": { "type": "user", "id": "netdev1" } 
}
+-> { "execute": "netdev_add",
+ "arguments": { "type": "user", "id": "netdev1",
+"dnssearch": "example.org" } }
 <- { "return": {} }
 
 Note: The supported device options are the same ones supported by the '-netdev'
-- 
2.4.3

[Qemu-devel] [PULL 02/26] qapi: New QAPISchema intermediate reperesentation

[Markus Armbruster]

The QAPI code generators work with a syntax tree (nested dictionaries)
plus a few symbol tables (also dictionaries) on the side.

They have clearly outgrown these simple data structures.  There's lots
of rummaging around in dictionaries, and information is recomputed on
the fly.  For the work I'm going to do, I want more clearly defined
and more convenient interfaces.

Going forward, I also want less coupling between the back-ends and the
syntax tree, to make messing with the syntax easier.

Create a bunch of classes to represent QAPI schemata.

Have the QAPISchema initializer call the parser, then walk the syntax
tree to create the new internal representation, and finally perform
semantic analysis.

Shortcut: the semantic analysis still relies on existing check_exprs()
to do the actual semantic checking.  All this code needs to move into
the classes.  Mark as TODO.

Simple unions are lowered to flat unions.  Flat unions and structs are
represented as a more general object type.

Catching name collisions in generated code would be nice.  Mark as
TODO.

We generate array types eagerly, even though most of them aren't used.
Mark as TODO.

Nothing uses the new intermediate representation just yet, thus no
change to generated files.

Signed-off-by: Markus Armbruster 
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 scripts/qapi-commands.py   |   2 +-
 scripts/qapi-event.py  |   2 +-
 scripts/qapi-types.py  |   2 +-
 scripts/qapi-visit.py  |   2 +-
 scripts/qapi.py| 382 -
 tests/qapi-schema/test-qapi.py |   2 +-
 6 files changed, 378 insertions(+), 14 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 890ce5d..12bdc4c 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -272,7 +272,7 @@ for o, a in opts:
 if o in ("-m", "--middle"):
 middle_mode = True
 
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
 commands = filter(lambda expr: expr.has_key('command'), exprs)
 commands = filter(lambda expr: not expr.has_key('gen'), commands)
 
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 7f238df..aec2d32 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -263,7 +263,7 @@ fdecl.write(mcgen('''
 ''',
   prefix=prefix))
 
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
 
 event_enum_name = c_name(prefix + "QAPIEvent", protect=False)
 event_enum_values = []
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index a8453d1..23e9505 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -336,7 +336,7 @@ fdecl.write(mcgen('''
 #include 
 '''))
 
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
 
 fdecl.write(guardstart("QAPI_TYPES_BUILTIN_STRUCT_DECL"))
 for typename in builtin_types.keys():
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 3cd662b..c493964 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -446,7 +446,7 @@ fdecl.write(mcgen('''
 ''',
   prefix=prefix))
 
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
 
 # to avoid header dependency hell, we always generate declarations
 # for built-in types in our header files and simply guard them
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 2a0f465..d0dfabe 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -302,6 +302,8 @@ class QAPISchemaParser(object):
 
 #
 # Semantic analysis of schema expressions
+# TODO fold into QAPISchema
+# TODO catching name collisions in generated code would be nice
 #
 
 def find_base_fields(base):
@@ -751,15 +753,377 @@ def check_exprs(exprs):
 else:
 assert False, 'unexpected meta type'
 
-return map(lambda expr_elem: expr_elem['expr'], exprs)
-
-def parse_schema(fname):
-try:
-schema = QAPISchemaParser(open(fname, "r"))
-return check_exprs(schema.exprs)
-except (QAPISchemaError, QAPIExprError), e:
-print >>sys.stderr, e
-exit(1)
+return exprs
+
+
+#
+# Schema compiler frontend
+#
+
+class QAPISchemaEntity(object):
+def __init__(self, name, info):
+assert isinstance(name, str)
+self.name = name
+self.info = info
+
+def check(self, schema):
+pass
+
+
+class QAPISchemaType(QAPISchemaEntity):
+pass
+
+
+class QAPISchemaBuiltinType(QAPISchemaType):
+def __init__(self, name):
+QAPISchemaType.__init__(self, name, None)
+
+
+class QAPISchemaEnumType(QAPISchemaType):
+def __init__(self, name, info, values, prefix):
+QAPISchemaType.__init__(self, name, info)
+for v in values:
+assert isinstance(v, str)
+assert prefix is None or isinstance(prefix, str)
+self.values = values
+self.prefix = prefix
+
+def check(self, schema):
+assert len(set(self.values)) == len(self.v

[Qemu-devel] [PULL 15/26] qapi-commands: Rearrange code

[Markus Armbruster]

Rename gen_marshal_input() to gen_marshal(), because the generated
function marshals both arguments and results.

Rename gen_visitor_input_containers_decl() to gen_marshal_vars(), and
move the other variable declarations there, too.

Rename gen_visitor_input_block() to gen_marshal_input_visit(), and
rearrange its code slightly.

Rename gen_marshal_input_decl() to gen_marshal_proto(), because the
result isn't a full declaration, unlike gen_command_decl()'s.

New gen_marshal_decl() actually returns a full declaration.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-16-git-send-email-arm...@redhat.com>
---
 scripts/qapi-commands.py | 87 ++--
 1 file changed, 39 insertions(+), 48 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 0501582..6a52c88 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -59,6 +59,7 @@ def gen_call(name, arg_type, ret_type):
 
 push_indent()
 ret = mcgen('''
+
 %(lhs)sqmp_%(c_name)s(%(args)s&local_err);
 ''',
 c_name=c_name(name), args=argstr, lhs=lhs)
@@ -73,26 +74,26 @@ qmp_marshal_output_%(c_name)s(retval, ret, &local_err);
 return ret
 
 
-def gen_visitor_input_containers_decl(arg_type):
-ret = ''
+def gen_marshal_vars(arg_type, ret_type):
+ret = mcgen('''
+Error *local_err = NULL;
+''')
 
 push_indent()
+
+if ret_type:
+ret += mcgen('''
+%(c_type)s retval;
+''',
+ c_type=ret_type.c_type())
+
 if arg_type:
 ret += mcgen('''
 QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
 QapiDeallocVisitor *md;
 Visitor *v;
 ''')
-pop_indent()
 
-return ret
-
-
-def gen_visitor_input_vars_decl(arg_type):
-ret = ''
-push_indent()
-
-if arg_type:
 for memb in arg_type.members:
 if memb.optional:
 ret += mcgen('''
@@ -105,15 +106,19 @@ bool has_%(c_name)s = false;
  c_name=c_name(memb.name),
  c_type=memb.type.c_type(),
  c_null=memb.type.c_null())
+ret += '\n'
+else:
+ret += mcgen('''
+
+(void)args;
+''')
 
 pop_indent()
 return ret
 
 
-def gen_visitor_input_block(arg_type, dealloc=False):
+def gen_marshal_input_visit(arg_type, dealloc=False):
 ret = ''
-errparg = '&local_err'
-errarg = 'local_err'
 
 if not arg_type:
 return ret
@@ -129,6 +134,8 @@ md = qapi_dealloc_visitor_new();
 v = qapi_dealloc_get_visitor(md);
 ''')
 else:
+errparg = '&local_err'
+errarg = 'local_err'
 ret += mcgen('''
 v = qmp_input_get_visitor(mi);
 ''')
@@ -167,10 +174,7 @@ qapi_dealloc_visitor_cleanup(md);
 
 
 def gen_marshal_output(name, ret_type):
-if not ret_type:
-return ''
-
-ret = mcgen('''
+return mcgen('''
 
 static void qmp_marshal_output_%(c_cmd_name)s(%(c_type)s ret_in, QObject 
**ret_out, Error **errp)
 {
@@ -195,47 +199,34 @@ out:
 qapi_dealloc_visitor_cleanup(md);
 }
 ''',
-c_type=ret_type.c_type(), c_cmd_name=c_name(name),
-c_name=ret_type.c_name())
+ c_type=ret_type.c_type(), c_cmd_name=c_name(name),
+ c_name=ret_type.c_name())
 
-return ret
 
-
-def gen_marshal_input_decl(name):
+def gen_marshal_proto(name):
 ret = 'void qmp_marshal_input_%s(QDict *args, QObject **ret, Error 
**errp)' % c_name(name)
 if not middle_mode:
 ret = 'static ' + ret
 return ret
 
 
-def gen_marshal_input(name, arg_type, ret_type):
-hdr = gen_marshal_input_decl(name)
+def gen_marshal_decl(name):
+return mcgen('''
+%(proto)s;
+''',
+ proto=gen_marshal_proto(name))
 
+
+def gen_marshal(name, arg_type, ret_type):
 ret = mcgen('''
 
-%(header)s
+%(proto)s
 {
-Error *local_err = NULL;
 ''',
-header=hdr)
-
-if ret_type:
-ret += mcgen('''
-%(c_type)s retval;
-''',
- c_type=ret_type.c_type())
-
-if arg_type:
-ret += gen_visitor_input_containers_decl(arg_type)
-ret += gen_visitor_input_vars_decl(arg_type) + '\n'
-ret += gen_visitor_input_block(arg_type) + '\n'
-else:
-ret += mcgen('''
-
-(void)args;
-
-''')
+proto=gen_marshal_proto(name))
 
+ret += gen_marshal_vars(arg_type, ret_type)
+ret += gen_marshal_input_visit(arg_type)
 ret += gen_call(name, arg_type, ret_type)
 
 if re.search('^ *goto out;', ret, re.MULTILINE):
@@ -246,7 +237,7 @@ out:
 ret += mcgen('''
 error_propagate(errp, local_err);
 ''')
-ret += gen_visitor_input_block(arg_type, dealloc=True)
+ret += gen_marshal_input_visit(arg_type, dealloc=True)
 ret += mcgen('''
 }
 ''')
@@ -307,8 +298,8 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
 if ret_type:
 self.defn += gen_marshal_

[Qemu-devel] [PULL 25/26] qapi: New QMP command query-qmp-schema for QMP introspection

[Markus Armbruster]

qapi/introspect.json defines the introspection schema.  It's designed
for QMP introspection, but should do for similar uses, such as QGA.

The introspection schema does not reflect all the rules and
restrictions that apply to QAPI schemata.  A valid QAPI schema has an
introspection value conforming to the introspection schema, but the
converse is not true.

Introspection lowers away a number of schema details, and makes
implicit things explicit:

* The built-in types are declared with their JSON type.

  All integer types are mapped to 'int', because how many bits we use
  internally is an implementation detail.  It could be pressed into
  external interface service as very approximate range information,
  but that's a bad idea.  If we need range information, we better do
  it properly.

* Implicit type definitions are made explicit, and given
  auto-generated names:

  - Array types, named by appending "List" to the name of their
element type, like in generated C.

  - The enumeration types implicitly defined by simple union types,
named by appending "Kind" to the name of their simple union type,
like in generated C.

  - Types that don't occur in generated C.  Their names start with ':'
so they don't clash with the user's names.

* All type references are by name.

* The struct and union types are generalized into an object type.

* Base types are flattened.

* Commands take a single argument and return a single result.

  Dictionary argument or list result is an implicit type definition.

  The empty object type is used when a command takes no arguments or
  produces no results.

  The argument is always of object type, but the introspection schema
  doesn't reflect that.

  The 'gen': false directive is omitted as implementation detail.

  The 'success-response' directive is omitted as well for now, even
  though it's not an implementation detail, because it's not used by
  QMP.

* Events carry a single data value.

  Implicit type definition and empty object type use, just like for
  commands.

  The value is of object type, but the introspection schema doesn't
  reflect that.

* Types not used by commands or events are omitted.

  Indirect use counts as use.

* Optional members have a default, which can only be null right now

  Instead of a mandatory "optional" flag, we have an optional default.
  No default means mandatory, default null means optional without
  default value.  Non-null is available for optional with default
  (possible future extension).

* Clients should *not* look up types by name, because type names are
  not ABI.  Look up the command or event you're interested in, then
  follow the references.

  TODO Should we hide the type names to eliminate the temptation?

New generator scripts/qapi-introspect.py computes an introspection
value for its input, and generates a C variable holding it.

It can generate awfully long lines.  Marked TODO.

A new test-qmp-input-visitor test case feeds its result for both
tests/qapi-schema/qapi-schema-test.json and qapi-schema.json to a
QmpInputVisitor to verify it actually conforms to the schema.

New QMP command query-qmp-schema takes its return value from that
variable.  Its reply is some 85KiBytes for me right now.

If this turns out to be too much, we have a couple of options:

* We can use shorter names in the JSON.  Not the QMP style.

* Optionally return the sub-schema for commands and events given as
  arguments.

  Right now qmp_query_schema() sends the string literal computed by
  qmp-introspect.py.  To compute sub-schema at run time, we'd have to
  duplicate parts of qapi-introspect.py in C.  Unattractive.

* Let clients cache the output of query-qmp-schema.

  It changes only on QEMU upgrades, i.e. rarely.  Provide a command
  query-qmp-schema-hash.  Clients can have a cache indexed by hash,
  and re-query the schema only when they don't have it cached.  Even
  simpler: put the hash in the QMP greeting.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 .gitignore  |   1 +
 Makefile|   9 +-
 Makefile.objs   |   4 +-
 docs/qapi-code-gen.txt  | 237 +++-
 monitor.c   |  16 ++
 qapi-schema.json|   3 +
 qapi/introspect.json| 277 
 qmp-commands.hx |  17 ++
 scripts/qapi-introspect.py  | 184 
 scripts/qapi.py |  13 +-
 tests/.gitignore|   1 +
 tests/Makefile  |  10 +-
 tests/qapi-schema/alternate-good.out|   1 +
 tests/qapi-schema/args-member-array.out |   1 +
 tests/qapi-schema/comments.out  |   1 +
 tests/qapi-schema/empty.out |   1 +

[Qemu-devel] [PULL 06/26] qapi-types: Convert to QAPISchemaVisitor, fixing flat unions

[Markus Armbruster]

Fixes flat unions to get the base's base members.  Test case is from
commit 2fc0043, in qapi-schema-test.json:

{ 'union': 'UserDefFlatUnion',
  'base': 'UserDefUnionBase',
  'discriminator': 'enum1',
  'data': { 'value1' : 'UserDefA',
'value2' : 'UserDefB',
'value3' : 'UserDefB' } }

{ 'struct': 'UserDefUnionBase',
  'base': 'UserDefZero',
  'data': { 'string': 'str', 'enum1': 'EnumOne' } }

{ 'struct': 'UserDefZero',
  'data': { 'integer': 'int' } }

Patch's effect on UserDefFlatUnion:

 struct UserDefFlatUnion {
 /* Members inherited from UserDefUnionBase: */
+int64_t integer;
 char *string;
 EnumOne enum1;
 /* Own members: */
 union { /* union tag is @enum1 */
 void *data;
 UserDefA *value1;
 UserDefB *value2;
 UserDefB *value3;
 };
 };

Flat union visitors remain broken.  They'll be fixed next.

Code is generated in a different order now, but that doesn't matter.

The two guards QAPI_TYPES_BUILTIN_STRUCT_DECL and
QAPI_TYPES_BUILTIN_CLEANUP_DECL are replaced by just
QAPI_TYPES_BUILTIN.

Two ugly special cases for simple unions now stand out like sore
thumbs:

1. The type tag is named 'type' everywhere, except in generated C,
   where it's 'kind'.

2. QAPISchema lowers simple unions to semantically equivalent flat
   unions.  However, the C generated for a simple unions differs from
   the C generated for its equivalent flat union, and we therefore
   need special code to preserve that pointless difference for now.

Mark both TODO.

Signed-off-by: Markus Armbruster 
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt  |  51 +++---
 scripts/qapi-types.py   | 286 ++--
 scripts/qapi.py |  10 +-
 tests/qapi-schema/qapi-schema-test.json |   4 +-
 4 files changed, 164 insertions(+), 187 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index d960dc0..c9e21fc 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -545,22 +545,6 @@ Example:
 $ cat qapi-generated/example-qapi-types.c
 [Uninteresting stuff omitted...]
 
-void qapi_free_UserDefOneList(UserDefOneList *obj)
-{
-QapiDeallocVisitor *md;
-Visitor *v;
-
-if (!obj) {
-return;
-}
-
-md = qapi_dealloc_visitor_new();
-v = qapi_dealloc_get_visitor(md);
-visit_type_UserDefOneList(v, &obj, NULL, NULL);
-qapi_dealloc_visitor_cleanup(md);
-}
-
-
 void qapi_free_UserDefOne(UserDefOne *obj)
 {
 QapiDeallocVisitor *md;
@@ -575,6 +559,21 @@ Example:
 visit_type_UserDefOne(v, &obj, NULL, NULL);
 qapi_dealloc_visitor_cleanup(md);
 }
+
+void qapi_free_UserDefOneList(UserDefOneList *obj)
+{
+QapiDeallocVisitor *md;
+Visitor *v;
+
+if (!obj) {
+return;
+}
+
+md = qapi_dealloc_visitor_new();
+v = qapi_dealloc_get_visitor(md);
+visit_type_UserDefOneList(v, &obj, NULL, NULL);
+qapi_dealloc_visitor_cleanup(md);
+}
 $ cat qapi-generated/example-qapi-types.h
 [Uninteresting stuff omitted...]
 
@@ -585,24 +584,24 @@ Example:
 
 typedef struct UserDefOne UserDefOne;
 
-typedef struct UserDefOneList {
+typedef struct UserDefOneList UserDefOneList;
+
+struct UserDefOne {
+int64_t integer;
+char *string;
+};
+
+void qapi_free_UserDefOne(UserDefOne *obj);
+
+struct UserDefOneList {
 union {
 UserDefOne *value;
 uint64_t padding;
 };
 struct UserDefOneList *next;
-} UserDefOneList;
-
-
-[Functions on built-in types omitted...]
-
-struct UserDefOne {
-int64_t integer;
-char *string;
 };
 
 void qapi_free_UserDefOneList(UserDefOneList *obj);
-void qapi_free_UserDefOne(UserDefOne *obj);
 
 #endif
 
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 23e9505..d78f529 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -2,88 +2,67 @@
 # QAPI types generator
 #
 # Copyright IBM, Corp. 2011
+# Copyright (c) 2013-2015 Red Hat Inc.
 #
 # Authors:
 #  Anthony Liguori 
+#  Markus Armbruster 
 #
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-from ordereddict import OrderedDict
 from qapi import *
 
-def generate_fwd_builtin(name):
-return mcgen('''
-
-typedef struct %(name)sList {
-union {
-%(type)s value;
-uint64_t padding;
-};
-struct %(name)sList *next;
-} %(name)sList;
-''',
- type=c_type(name),
- name=name)
-
-def generate_fwd_struct(name):
+def gen_fwd_object_or_array(name):
 return mcgen('''
 
 typedef struct %(name)s %(name)s;
-
-typedef struct 

[Qemu-devel] [PULL 13/26] qapi: Clean up after recent conversions to QAPISchemaVisitor

[Markus Armbruster]

Generate just 'FOO' instead of 'struct FOO' when possible.

Drop helper functions that are now unused.

Make pep8 and pylint reasonably happy.

Rename generate_FOO() functions to gen_FOO() for consistency.

Use more consistent and sensible variable names.

Consistently use c_ for mapping keys when their value is a C
identifier or type.

Simplify gen_enum() and gen_visit_union()

Consistently use single quotes for C text string literals.

Signed-off-by: Markus Armbruster 
Message-Id: <1442401589-24189-14-git-send-email-arm...@redhat.com>
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt   |   2 +-
 scripts/qapi-commands.py | 140 ++-
 scripts/qapi-event.py| 122 -
 scripts/qapi-types.py|  79 ++
 scripts/qapi-visit.py| 127 ++
 scripts/qapi.py  | 131 +---
 6 files changed, 274 insertions(+), 327 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 81e87d2..3eaff36 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -598,7 +598,7 @@ Example:
 UserDefOne *value;
 uint64_t padding;
 };
-struct UserDefOneList *next;
+UserDefOneList *next;
 };
 
 void qapi_free_UserDefOneList(UserDefOneList *obj);
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index cbff356..0501582 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -15,20 +15,22 @@
 from qapi import *
 import re
 
-def generate_command_decl(name, args, ret_type):
-arglist=""
-if args:
-for memb in args.members:
-argtype = memb.type.c_type(is_param=True)
+
+def gen_command_decl(name, arg_type, ret_type):
+argstr = ''
+if arg_type:
+for memb in arg_type.members:
 if memb.optional:
-arglist += "bool has_%s, " % c_name(memb.name)
-arglist += "%s %s, " % (argtype, c_name(memb.name))
+argstr += 'bool has_%s, ' % c_name(memb.name)
+argstr += '%s %s, ' % (memb.type.c_type(is_param=True),
+   c_name(memb.name))
 return mcgen('''
-%(ret_type)s qmp_%(name)s(%(args)sError **errp);
+%(c_type)s qmp_%(c_name)s(%(args)sError **errp);
 ''',
- ret_type=(ret_type and ret_type.c_type()) or 'void',
- name=c_name(name),
- args=arglist)
+ c_type=(ret_type and ret_type.c_type()) or 'void',
+ c_name=c_name(name),
+ args=argstr)
+
 
 def gen_err_check(err):
 if not err:
@@ -40,37 +42,42 @@ if (%(err)s) {
 ''',
  err=err)
 
-def gen_sync_call(name, args, ret_type):
-ret = ""
-arglist=""
-retval=""
-if ret_type:
-retval = "retval = "
-if args:
-for memb in args.members:
+
+def gen_call(name, arg_type, ret_type):
+ret = ''
+
+argstr = ''
+if arg_type:
+for memb in arg_type.members:
 if memb.optional:
-arglist += "has_%s, " % c_name(memb.name)
-arglist += "%s, " % c_name(memb.name)
+argstr += 'has_%s, ' % c_name(memb.name)
+argstr += '%s, ' % c_name(memb.name)
+
+lhs = ''
+if ret_type:
+lhs = 'retval = '
+
 push_indent()
 ret = mcgen('''
-%(retval)sqmp_%(name)s(%(args)s&local_err);
+%(lhs)sqmp_%(c_name)s(%(args)s&local_err);
 ''',
-name=c_name(name), args=arglist, retval=retval)
+c_name=c_name(name), args=argstr, lhs=lhs)
 if ret_type:
 ret += gen_err_check('local_err')
 ret += mcgen('''
 
 qmp_marshal_output_%(c_name)s(retval, ret, &local_err);
 ''',
-c_name=c_name(name))
+ c_name=c_name(name))
 pop_indent()
 return ret
 
-def gen_visitor_input_containers_decl(args):
-ret = ""
+
+def gen_visitor_input_containers_decl(arg_type):
+ret = ''
 
 push_indent()
-if args:
+if arg_type:
 ret += mcgen('''
 QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
 QapiDeallocVisitor *md;
@@ -80,17 +87,18 @@ Visitor *v;
 
 return ret
 
-def gen_visitor_input_vars_decl(args):
-ret = ""
+
+def gen_visitor_input_vars_decl(arg_type):
+ret = ''
 push_indent()
 
-if args:
-for memb in args.members:
+if arg_type:
+for memb in arg_type.members:
 if memb.optional:
 ret += mcgen('''
-bool has_%(argname)s = false;
+bool has_%(c_name)s = false;
 ''',
- argname=c_name(memb.name))
+ c_name=c_name(memb.name))
 ret += mcgen('''
 %(c_type)s %(c_name)s = %(c_null)s;
 ''',
@@ -101,19 +109,20 @@ bool has_%(argname)s = false;
 pop_indent()
 return ret
 
-def gen

[Qemu-devel] [PULL 18/26] qapi-commands: De-duplicate output marshaling functions

[Markus Armbruster]

gen_marshal_output() uses its parameter name only for name of the
generated function.  Name it after the type being marshaled instead of
its caller, and drop duplicates.

Saves 7 copies of qmp_marshal_output_int() in qemu-ga, and one copy of
qmp_marshal_output_str() in qemu-system-*.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-19-git-send-email-arm...@redhat.com>
---
 docs/qapi-code-gen.txt   |  4 ++--
 scripts/qapi-commands.py | 17 ++---
 2 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 50da1d0..0884b58 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -721,7 +721,7 @@ Example:
 $ cat qapi-generated/example-qmp-marshal.c
 [Uninteresting stuff omitted...]
 
-static void qmp_marshal_output_my_command(UserDefOne *ret_in, QObject 
**ret_out, Error **errp)
+static void qmp_marshal_output_UserDefOne(UserDefOne *ret_in, QObject 
**ret_out, Error **errp)
 {
 Error *local_err = NULL;
 QmpOutputVisitor *mo = qmp_output_visitor_new();
@@ -764,7 +764,7 @@ Example:
 goto out;
 }
 
-qmp_marshal_output_my_command(retval, ret, &local_err);
+qmp_marshal_output_UserDefOne(retval, ret, &local_err);
 
 out:
 error_propagate(errp, local_err);
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 833768e..810a897 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -62,7 +62,7 @@ def gen_call(name, arg_type, ret_type):
 
 qmp_marshal_output_%(c_name)s(retval, ret, &local_err);
 ''',
- c_name=c_name(name))
+ c_name=ret_type.c_name())
 pop_indent()
 return ret
 
@@ -166,10 +166,10 @@ qapi_dealloc_visitor_cleanup(md);
 return ret
 
 
-def gen_marshal_output(name, ret_type):
+def gen_marshal_output(ret_type):
 return mcgen('''
 
-static void qmp_marshal_output_%(c_cmd_name)s(%(c_type)s ret_in, QObject 
**ret_out, Error **errp)
+static void qmp_marshal_output_%(c_name)s(%(c_type)s ret_in, QObject 
**ret_out, Error **errp)
 {
 Error *local_err = NULL;
 QmpOutputVisitor *mo = qmp_output_visitor_new();
@@ -192,8 +192,7 @@ out:
 qapi_dealloc_visitor_cleanup(md);
 }
 ''',
- c_type=ret_type.c_type(), c_cmd_name=c_name(name),
- c_name=ret_type.c_name())
+ c_type=ret_type.c_type(), c_name=ret_type.c_name())
 
 
 def gen_marshal_proto(name):
@@ -272,24 +271,28 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
 self.decl = None
 self.defn = None
 self._regy = None
+self._visited_ret_types = None
 
 def visit_begin(self, schema):
 self.decl = ''
 self.defn = ''
 self._regy = ''
+self._visited_ret_types = set()
 
 def visit_end(self):
 if not middle_mode:
 self.defn += gen_registry(self._regy)
 self._regy = None
+self._visited_ret_types = None
 
 def visit_command(self, name, info, arg_type, ret_type,
   gen, success_response):
 if not gen:
 return
 self.decl += gen_command_decl(name, arg_type, ret_type)
-if ret_type:
-self.defn += gen_marshal_output(name, ret_type)
+if ret_type and ret_type not in self._visited_ret_types:
+self._visited_ret_types.add(ret_type)
+self.defn += gen_marshal_output(ret_type)
 if middle_mode:
 self.decl += gen_marshal_decl(name)
 self.defn += gen_marshal(name, arg_type, ret_type)
-- 
2.4.3

[Qemu-devel] [PULL 17/26] qapi: De-duplicate parameter list generation

[Markus Armbruster]

Generated qapi-event.[ch] lose line breaks.  No change otherwise.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-18-git-send-email-arm...@redhat.com>
---
 scripts/qapi-commands.py | 11 ++-
 scripts/qapi-event.py| 18 +++---
 scripts/qapi.py  | 16 
 3 files changed, 21 insertions(+), 24 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index c68659a..833768e 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -17,19 +17,12 @@ import re
 
 
 def gen_command_decl(name, arg_type, ret_type):
-argstr = ''
-if arg_type:
-for memb in arg_type.members:
-if memb.optional:
-argstr += 'bool has_%s, ' % c_name(memb.name)
-argstr += '%s %s, ' % (memb.type.c_type(is_param=True),
-   c_name(memb.name))
 return mcgen('''
-%(c_type)s qmp_%(c_name)s(%(args)sError **errp);
+%(c_type)s qmp_%(c_name)s(%(params)s);
 ''',
  c_type=(ret_type and ret_type.c_type()) or 'void',
  c_name=c_name(name),
- args=argstr)
+ params=gen_params(arg_type, 'Error **errp'))
 
 
 def gen_err_check(err):
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 5873a05..d15fad9 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -15,21 +15,9 @@ from qapi import *
 
 
 def gen_event_send_proto(name, arg_type):
-api_name = "void qapi_event_send_%s(" % c_name(name).lower()
-l = len(api_name)
-
-if arg_type:
-for m in arg_type.members:
-if m.optional:
-api_name += "bool has_%s,\n" % c_name(m.name)
-api_name += "".ljust(l)
-
-api_name += "%s %s,\n" % (m.type.c_type(is_param=True),
-  c_name(m.name))
-api_name += "".ljust(l)
-
-api_name += "Error **errp)"
-return api_name
+return 'void qapi_event_send_%(c_name)s(%(param)s)' % {
+'c_name': c_name(name.lower()),
+'param': gen_params(arg_type, 'Error **errp')}
 
 
 def gen_event_send_decl(name, arg_type):
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 7ac72f6..6b6f1ae 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1469,6 +1469,22 @@ extern const char *const %(c_name)s_lookup[];
  c_name=c_name(name))
 return ret
 
+def gen_params(arg_type, extra):
+if not arg_type:
+return extra
+assert not arg_type.variants
+ret = ''
+sep = ''
+for memb in arg_type.members:
+ret += sep
+sep = ', '
+if memb.optional:
+ret += 'bool has_%s, ' % c_name(memb.name)
+ret += '%s %s' % (memb.type.c_type(is_param=True), c_name(memb.name))
+if extra:
+ret += sep + extra
+return ret
+
 #
 # Common command line parsing
 #
-- 
2.4.3

[Qemu-devel] [PULL 16/26] qapi: Rename qmp_marshal_input_FOO() to qmp_marshal_FOO()

[Markus Armbruster]

These functions marshal both input and output.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-17-git-send-email-arm...@redhat.com>
---
 docs/qapi-code-gen.txt|   4 +-
 docs/writing-qmp-commands.txt |   8 +-
 monitor.c |   2 +-
 qmp-commands.hx   | 242 +-
 qmp.c |   6 +-
 scripts/qapi-commands.py  |   4 +-
 6 files changed, 133 insertions(+), 133 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 3eaff36..50da1d0 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -744,7 +744,7 @@ Example:
 qapi_dealloc_visitor_cleanup(md);
 }
 
-static void qmp_marshal_input_my_command(QDict *args, QObject **ret, Error 
**errp)
+static void qmp_marshal_my_command(QDict *args, QObject **ret, Error 
**errp)
 {
 Error *local_err = NULL;
 UserDefOne *retval;
@@ -777,7 +777,7 @@ Example:
 
 static void qmp_init_marshal(void)
 {
-qmp_register_command("my-command", qmp_marshal_input_my_command, 
QCO_NO_OPTIONS);
+qmp_register_command("my-command", qmp_marshal_my_command, 
QCO_NO_OPTIONS);
 }
 
 qapi_init(qmp_init_marshal);
diff --git a/docs/writing-qmp-commands.txt b/docs/writing-qmp-commands.txt
index f7693ca..8647cac 100644
--- a/docs/writing-qmp-commands.txt
+++ b/docs/writing-qmp-commands.txt
@@ -127,7 +127,7 @@ following at the bottom:
 {
 .name   = "hello-world",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_hello_world,
+.mhandler.cmd_new = qmp_marshal_hello_world,
 },
 
 You're done. Now build qemu, run it as suggested in the "Testing" section,
@@ -179,7 +179,7 @@ The last step is to update the qmp-commands.hx file:
 {
 .name   = "hello-world",
 .args_type  = "message:s?",
-.mhandler.cmd_new = qmp_marshal_input_hello_world,
+.mhandler.cmd_new = qmp_marshal_hello_world,
 },
 
 Notice that the "args_type" member got our "message" argument. The character
@@ -461,7 +461,7 @@ The last step is to add the correspoding entry in the 
qmp-commands.hx file:
 {
 .name   = "query-alarm-clock",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_query_alarm_clock,
+.mhandler.cmd_new = qmp_marshal_query_alarm_clock,
 },
 
 Time to test the new command. Build qemu, run it as described in the "Testing"
@@ -607,7 +607,7 @@ To test this you have to add the corresponding 
qmp-commands.hx entry:
 {
 .name   = "query-alarm-methods",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_query_alarm_methods,
+.mhandler.cmd_new = qmp_marshal_query_alarm_methods,
 },
 
 Now Build qemu, run it as explained in the "Testing" section and try our new
diff --git a/monitor.c b/monitor.c
index 1f43263..dde1f22 100644
--- a/monitor.c
+++ b/monitor.c
@@ -3912,7 +3912,7 @@ static QObject *get_qmp_greeting(void)
 {
 QObject *ver = NULL;
 
-qmp_marshal_input_query_version(NULL, &ver, NULL);
+qmp_marshal_query_version(NULL, &ver, NULL);
 return qobject_from_jsonf("{'QMP':{'version': %p,'capabilities': 
[]}}",ver);
 }
 
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 9848fd8..4640a3d 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -63,7 +63,7 @@ EQMP
 {
 .name   = "quit",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_quit,
+.mhandler.cmd_new = qmp_marshal_quit,
 },
 
 SQMP
@@ -84,7 +84,7 @@ EQMP
 {
 .name   = "eject",
 .args_type  = "force:-f,device:B",
-.mhandler.cmd_new = qmp_marshal_input_eject,
+.mhandler.cmd_new = qmp_marshal_eject,
 },
 
 SQMP
@@ -110,7 +110,7 @@ EQMP
 {
 .name   = "change",
 .args_type  = "device:B,target:F,arg:s?",
-.mhandler.cmd_new = qmp_marshal_input_change,
+.mhandler.cmd_new = qmp_marshal_change,
 },
 
 SQMP
@@ -146,7 +146,7 @@ EQMP
 {
 .name   = "screendump",
 .args_type  = "filename:F",
-.mhandler.cmd_new = qmp_marshal_input_screendump,
+.mhandler.cmd_new = qmp_marshal_screendump,
 },
 
 SQMP
@@ -169,7 +169,7 @@ EQMP
 {
 .name   = "stop",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_stop,
+.mhandler.cmd_new = qmp_marshal_stop,
 },
 
 SQMP
@@ -190,7 +190,7 @@ EQMP
 {
 .name   = "cont",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_cont,
+.mhandler.cmd_new = qmp_marshal_cont,
 },
 
 SQMP
@@ -211,7 +211,7 @@ EQMP
 {
 .name   = "system_wakeup",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_system_wakeup,
+.mhandler.cmd_new = qmp_marshal_system_wakeup,
 },
 
 SQMP
@@ -

[Qemu-devel] [PULL 22/26] qom: Don't use 'gen': false for qom-get, qom-set, object-add

[Markus Armbruster]

With the previous commit, the generated marshalers just work, and save
us a bit of handwritten code.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-23-git-send-email-arm...@redhat.com>
---
 include/monitor/monitor.h |  3 ---
 qapi-schema.json  |  9 +++--
 qmp-commands.hx   |  6 +++---
 qmp.c | 21 +++--
 scripts/qapi.py   |  1 +
 5 files changed, 14 insertions(+), 26 deletions(-)

diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 9aff47e..bc6cb6d 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -42,9 +42,6 @@ void monitor_read_command(Monitor *mon, int show_prompt);
 int monitor_read_password(Monitor *mon, ReadLineFunc *readline_func,
   void *opaque);
 
-void qmp_qom_set(QDict *qdict, QObject **ret, Error **errp);
-void qmp_qom_get(QDict *qdict, QObject **ret, Error **errp);
-void qmp_object_add(QDict *qdict, QObject **ret, Error **errp);
 void object_add(const char *type, const char *id, const QDict *qdict,
 Visitor *v, Error **errp);
 
diff --git a/qapi-schema.json b/qapi-schema.json
index 2bada60..756ed9e 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1698,8 +1698,7 @@
 ##
 { 'command': 'qom-get',
   'data': { 'path': 'str', 'property': 'str' },
-  'returns': '**',
-  'gen': false }
+  'returns': 'any' }
 
 ##
 # @qom-set:
@@ -1716,8 +1715,7 @@
 # Since: 1.2
 ##
 { 'command': 'qom-set',
-  'data': { 'path': 'str', 'property': 'str', 'value': '**' },
-  'gen': false }
+  'data': { 'path': 'str', 'property': 'str', 'value': 'any' } }
 
 ##
 # @set_password:
@@ -2127,8 +2125,7 @@
 # Since: 2.0
 ##
 { 'command': 'object-add',
-  'data': {'qom-type': 'str', 'id': 'str', '*props': '**'},
-  'gen': false }
+  'data': {'qom-type': 'str', 'id': 'str', '*props': 'any'} }
 
 ##
 # @object-del:
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 4640a3d..7c74e20 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -953,7 +953,7 @@ EQMP
 {
 .name   = "object-add",
 .args_type  = "qom-type:s,id:s,props:q?",
-.mhandler.cmd_new = qmp_object_add,
+.mhandler.cmd_new = qmp_marshal_object_add,
 },
 
 SQMP
@@ -3575,13 +3575,13 @@ EQMP
 {
 .name   = "qom-set",
.args_type  = "path:s,property:s,value:q",
-   .mhandler.cmd_new = qmp_qom_set,
+.mhandler.cmd_new = qmp_marshal_qom_set,
 },
 
 {
 .name   = "qom-get",
.args_type  = "path:s,property:s",
-   .mhandler.cmd_new = qmp_qom_get,
+.mhandler.cmd_new = qmp_marshal_qom_get,
 },
 
 {
diff --git a/qmp.c b/qmp.c
index 3feae9f..057a7cb 100644
--- a/qmp.c
+++ b/qmp.c
@@ -234,12 +234,9 @@ ObjectPropertyInfoList *qmp_qom_list(const char *path, 
Error **errp)
 return props;
 }
 
-/* FIXME: teach qapi about how to pass through Visitors */
-void qmp_qom_set(QDict *qdict, QObject **ret, Error **errp)
+void qmp_qom_set(const char *path, const char *property, QObject *value,
+ Error **errp)
 {
-const char *path = qdict_get_str(qdict, "path");
-const char *property = qdict_get_str(qdict, "property");
-QObject *value = qdict_get(qdict, "value");
 Object *obj;
 
 obj = object_resolve_path(path, NULL);
@@ -252,20 +249,18 @@ void qmp_qom_set(QDict *qdict, QObject **ret, Error 
**errp)
 object_property_set_qobject(obj, value, property, errp);
 }
 
-void qmp_qom_get(QDict *qdict, QObject **ret, Error **errp)
+QObject *qmp_qom_get(const char *path, const char *property, Error **errp)
 {
-const char *path = qdict_get_str(qdict, "path");
-const char *property = qdict_get_str(qdict, "property");
 Object *obj;
 
 obj = object_resolve_path(path, NULL);
 if (!obj) {
 error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
   "Device '%s' not found", path);
-return;
+return NULL;
 }
 
-*ret = object_property_get_qobject(obj, property, errp);
+return object_property_get_qobject(obj, property, errp);
 }
 
 void qmp_set_password(const char *protocol, const char *password,
@@ -661,11 +656,9 @@ out:
 object_unref(obj);
 }
 
-void qmp_object_add(QDict *qdict, QObject **ret, Error **errp)
+void qmp_object_add(const char *type, const char *id,
+bool has_props, QObject *props, Error **errp)
 {
-const char *type = qdict_get_str(qdict, "qom-type");
-const char *id = qdict_get_str(qdict, "id");
-QObject *props = qdict_get(qdict, "props");
 const QDict *pdict = NULL;
 QmpInputVisitor *qiv;
 
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 6a21bd6..8808ae6 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -40,6 +40,7 @@ builtin_types = {
 returns_whitelist = [
 # From QMP:
 'human-monitor-command',
+'qom-get',
 'query-migrate-cache-size',
 'query-tpm-models',
 'query-tpm-types'

[Qemu-devel] [PULL 19/26] qapi: Improve built-in type documentation

[Markus Armbruster]

Clarify how they map to JSON.  Add how they map to C.  Fix the
reference to StringInputVisitor.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-20-git-send-email-arm...@redhat.com>
---
 docs/qapi-code-gen.txt | 29 ++---
 1 file changed, 18 insertions(+), 11 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 0884b58..5c2c278 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -140,17 +140,24 @@ must have a value that forms a struct name.
 
 === Built-in Types ===
 
-The following types are built-in to the parser:
-  'str' - arbitrary UTF-8 string
-  'int' - 64-bit signed integer (although the C code may place further
-  restrictions on acceptable range)
-  'number' - floating point number
-  'bool' - JSON value of true or false
-  'int8', 'int16', 'int32', 'int64' - like 'int', but enforce maximum
-  bit size
-  'uint8', 'uint16', 'uint32', 'uint64' - unsigned counterparts
-  'size' - like 'uint64', but allows scaled suffix from command line
-   visitor
+The following types are predefined, and map to C as follows:
+
+  SchemaC  JSON
+  str   char * any JSON string, UTF-8
+  numberdouble any JSON number
+  int   int64_ta JSON number without fractional part
+   that fits into the C integer type
+  int8  int8_t likewise
+  int16 int16_tlikewise
+  int32 int32_tlikewise
+  int64 int64_tlikewise
+  uint8 uint8_tlikewise
+  uint16uint16_t   likewise
+  uint32uint32_t   likewise
+  uint64uint64_t   likewise
+  size  uint64_t   like uint64_t, except StringInputVisitor
+   accepts size suffixes
+  bool  bool   JSON true or false
 
 
 === Includes ===
-- 
2.4.3

[Qemu-devel] [PULL 24/26] qapi: Pseudo-type '**' is now unused, drop it

[Markus Armbruster]

'gen': false needs to stay for now, because netdev_add is still using
it.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-25-git-send-email-arm...@redhat.com>
---
 docs/qapi-code-gen.txt| 18 ++
 scripts/qapi.py   | 20 
 tests/Makefile|  2 +-
 tests/qapi-schema/type-bypass-no-gen.err  |  1 -
 tests/qapi-schema/type-bypass-no-gen.exit |  1 -
 tests/qapi-schema/type-bypass-no-gen.json |  2 --
 tests/qapi-schema/type-bypass-no-gen.out  |  0
 tests/qapi-schema/type-bypass.err |  0
 tests/qapi-schema/type-bypass.exit|  1 -
 tests/qapi-schema/type-bypass.json|  2 --
 tests/qapi-schema/type-bypass.out |  4 
 11 files changed, 11 insertions(+), 40 deletions(-)
 delete mode 100644 tests/qapi-schema/type-bypass-no-gen.err
 delete mode 100644 tests/qapi-schema/type-bypass-no-gen.exit
 delete mode 100644 tests/qapi-schema/type-bypass-no-gen.json
 delete mode 100644 tests/qapi-schema/type-bypass-no-gen.out
 delete mode 100644 tests/qapi-schema/type-bypass.err
 delete mode 100644 tests/qapi-schema/type-bypass.exit
 delete mode 100644 tests/qapi-schema/type-bypass.json
 delete mode 100644 tests/qapi-schema/type-bypass.out

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 4e331a0..b917962 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -111,10 +111,7 @@ and field names within a type, should be all lower case 
with words
 separated by a hyphen.  However, some existing older commands and
 complex types use underscore; when extending such expressions,
 consistency is preferred over blindly avoiding underscore.  Event
-names should be ALL_CAPS with words separated by underscore.  The
-special string '**' appears for some commands that manually perform
-their own type checking rather than relying on the type-safe code
-produced by the qapi code generators.
+names should be ALL_CAPS with words separated by underscore.
 
 Any name (command, event, type, field, or enum value) beginning with
 "x-" is marked experimental, and may be withdrawn or changed
@@ -461,14 +458,11 @@ which would validate this Client JSON Protocol 
transaction:
  <= { "return": [ { "value": "one" }, { } ] }
 
 In rare cases, QAPI cannot express a type-safe representation of a
-corresponding Client JSON Protocol command.  In these cases, if the
-command expression includes the key 'gen' with boolean value false,
-then the 'data' or 'returns' member that intends to bypass generated
-type-safety and do its own manual validation should use an inline
-dictionary definition, with a value of '**' rather than a valid type
-name for the keys that the generated code will not validate.  Please
-try to avoid adding new commands that rely on this, and instead use
-type-safe unions.  For an example of bypass usage:
+corresponding Client JSON Protocol command.  You then have to suppress
+generation of a marshalling function by including a key 'gen' with
+boolean value false, and instead write your own function.  Please try
+to avoid adding new commands that rely on this, and instead use
+type-safe unions.  For an example of this usage:
 
  { 'command': 'netdev_add',
'data': {'type': 'str', 'id': 'str'},
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 8808ae6..f407297 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -428,15 +428,12 @@ def is_enum(name):
 
 def check_type(expr_info, source, value, allow_array = False,
allow_dict = False, allow_optional = False,
-   allow_star = False, allow_metas = []):
+   allow_metas = []):
 global all_names
 
 if value is None:
 return
 
-if allow_star and value == '**':
-return
-
 # Check if array type for value is okay
 if isinstance(value, list):
 if not allow_array:
@@ -450,10 +447,6 @@ def check_type(expr_info, source, value, allow_array = 
False,
 
 # Check if type name for value is okay
 if isinstance(value, str):
-if value == '**':
-raise QAPIExprError(expr_info,
-"%s uses '**' but did not request 'gen':false"
-% source)
 if not value in all_names:
 raise QAPIExprError(expr_info,
 "%s uses unknown type '%s'"
@@ -479,7 +472,7 @@ def check_type(expr_info, source, value, allow_array = 
False,
 # Todo: allow dictionaries to represent default values of
 # an optional argument.
 check_type(expr_info, "Member '%s' of %s" % (key, source), arg,
-   allow_array=True, allow_star=allow_star,
+   allow_array=True,
allow_metas=['built-in', 'union', 'alternate', 'struct',
 'enum'])
 
@@ -499,18 +492,16 @@ def check_member_clash(expr_info, base_name, d

[Qemu-devel] [PULL 07/26] qapi-visit: Convert to QAPISchemaVisitor, fixing bugs

[Markus Armbruster]

Fixes flat unions to visit the base's base members (the previous
commit merely added them to the struct).  Same test case.

Patch's effect on visit_type_UserDefFlatUnion():

 static void visit_type_UserDefFlatUnion_fields(Visitor *m, 
UserDefFlatUnion **obj, Error **errp)
 {
 Error *err = NULL;

+visit_type_int(m, &(*obj)->integer, "integer", &err);
+if (err) {
+goto out;
+}
 visit_type_str(m, &(*obj)->string, "string", &err);
 if (err) {
 goto out;

Test cases updated for the bug fix.

Fixes alternates to generate a visitor for their implicit enumeration
type.  None of them are currently used, obviously.  Example:
block-core.json's BlockdevRef now generates
visit_type_BlockdevRefKind().

Code is generated in a different order now, and therefore has got a
few new forward declarations.  Doesn't matter.

The guard QAPI_VISIT_BUILTIN_VISITOR_DECL is renamed to
QAPI_VISIT_BUILTIN.

The previous commit's two ugly special cases exist here, too.  Mark
both TODO.

Signed-off-by: Markus Armbruster 
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 scripts/qapi-visit.py   | 270 +---
 tests/qapi-schema/qapi-schema-test.json |   3 -
 tests/test-qmp-input-strict.c   |   4 +-
 tests/test-qmp-input-visitor.c  |   4 +-
 4 files changed, 117 insertions(+), 164 deletions(-)

diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index c493964..8e6e8ca 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -12,7 +12,6 @@
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-from ordereddict import OrderedDict
 from qapi import *
 import re
 
@@ -24,13 +23,13 @@ def generate_visit_implicit_struct(type):
 return ''
 implicit_structs_seen.add(type)
 ret = ''
-if type not in struct_fields_seen:
+if type.name not in struct_fields_seen:
 # Need a forward declaration
 ret += mcgen('''
 
 static void visit_type_%(c_type)s_fields(Visitor *m, %(c_type)s **obj, Error 
**errp);
 ''',
- c_type=type_name(type))
+ c_type=type.c_name())
 
 ret += mcgen('''
 
@@ -46,7 +45,7 @@ static void visit_type_implicit_%(c_type)s(Visitor *m, 
%(c_type)s **obj, Error *
 error_propagate(errp, err);
 }
 ''',
- c_type=type_name(type))
+ c_type=type.c_name())
 return ret
 
 def generate_visit_struct_fields(name, members, base = None):
@@ -74,24 +73,24 @@ if (err) {
 goto out;
 }
 ''',
- type=type_name(base), c_name=c_name('base'))
+ type=base.c_name(), c_name=c_name('base'))
 
-for argname, argentry, optional in parse_args(members):
-if optional:
+for memb in members:
+if memb.optional:
 ret += mcgen('''
 visit_optional(m, &(*obj)->has_%(c_name)s, "%(name)s", &err);
 if (!err && (*obj)->has_%(c_name)s) {
 ''',
- c_name=c_name(argname), name=argname)
+ c_name=c_name(memb.name), name=memb.name)
 push_indent()
 
 ret += mcgen('''
 visit_type_%(type)s(m, &(*obj)->%(c_name)s, "%(name)s", &err);
 ''',
- type=type_name(argentry), c_name=c_name(argname),
- name=argname)
+ type=memb.type.c_name(), c_name=c_name(memb.name),
+ name=memb.name)
 
-if optional:
+if memb.optional:
 pop_indent()
 ret += mcgen('''
 }
@@ -136,12 +135,7 @@ def generate_visit_struct_body(name):
 
 return ret
 
-def generate_visit_struct(expr):
-
-name = expr['struct']
-members = expr['data']
-base = expr.get('base')
-
+def gen_visit_struct(name, base, members):
 ret = generate_visit_struct_fields(name, members, base)
 
 ret += mcgen('''
@@ -158,10 +152,10 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, 
const char *name, Error **e
 ''')
 return ret
 
-def generate_visit_list(name):
+def gen_visit_list(name, element_type):
 return mcgen('''
 
-void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, 
Error **errp)
+void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error 
**errp)
 {
 Error *err = NULL;
 GenericList *i, **prev;
@@ -174,8 +168,8 @@ void visit_type_%(name)sList(Visitor *m, %(name)sList 
**obj, const char *name, E
 for (prev = (GenericList **)obj;
  !err && (i = visit_next_list(m, prev, &err)) != NULL;
  prev = &i) {
-%(name)sList *native_i = (%(name)sList *)i;
-visit_type_%(name)s(m, &native_i->value, NULL, &err);
+%(name)s *native_i = (%(name)s *)i;
+visit_type_%(c_elt_type)s(m, &native_i->value, NULL, &err);
 }
 
 error_propagate(errp, err);
@@ -185,7 +179,8 @@ out:
 error_propagate(errp, err);
 }
 ''',
-  

[Qemu-devel] [PULL 21/26] qapi: Introduce a first class 'any' type

[Markus Armbruster]

It's first class, because unlike '**', it actually works, i.e. doesn't
require 'gen': false.

'**' will go away next.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
---
 docs/qapi-code-gen.txt |  1 +
 include/qapi/visitor-impl.h|  2 +
 include/qapi/visitor.h |  1 +
 qapi/qapi-dealloc-visitor.c|  9 
 qapi/qapi-visit-core.c |  6 +++
 qapi/qmp-input-visitor.c   | 11 +
 qapi/qmp-output-visitor.c  |  9 
 scripts/qapi-types.py  |  1 +
 scripts/qapi.py|  9 ++--
 tests/Makefile |  3 +-
 tests/qapi-schema/args-any.err |  1 +
 .../{flat-union-base-star.exit => args-any.exit}   |  0
 tests/qapi-schema/args-any.json|  2 +
 .../{flat-union-base-star.out => args-any.out} |  0
 tests/qapi-schema/flat-union-base-any.err  |  1 +
 tests/qapi-schema/flat-union-base-any.exit |  1 +
 ...ion-base-star.json => flat-union-base-any.json} |  2 +-
 tests/qapi-schema/flat-union-base-any.out  |  0
 tests/qapi-schema/flat-union-base-star.err |  1 -
 tests/qapi-schema/qapi-schema-test.json|  5 +-
 tests/qapi-schema/qapi-schema-test.out |  9 +++-
 tests/qapi-schema/type-bypass.out  |  4 +-
 tests/test-qmp-commands.c  |  5 ++
 tests/test-qmp-input-visitor.c | 45 ++
 tests/test-qmp-output-visitor.c| 53 ++
 25 files changed, 171 insertions(+), 10 deletions(-)
 create mode 100644 tests/qapi-schema/args-any.err
 rename tests/qapi-schema/{flat-union-base-star.exit => args-any.exit} (100%)
 create mode 100644 tests/qapi-schema/args-any.json
 rename tests/qapi-schema/{flat-union-base-star.out => args-any.out} (100%)
 create mode 100644 tests/qapi-schema/flat-union-base-any.err
 create mode 100644 tests/qapi-schema/flat-union-base-any.exit
 rename tests/qapi-schema/{flat-union-base-star.json => 
flat-union-base-any.json} (95%)
 create mode 100644 tests/qapi-schema/flat-union-base-any.out
 delete mode 100644 tests/qapi-schema/flat-union-base-star.err

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 5c2c278..70fdae7 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -158,6 +158,7 @@ The following types are predefined, and map to C as follows:
   size  uint64_t   like uint64_t, except StringInputVisitor
accepts size suffixes
   bool  bool   JSON true or false
+  any   QObject *  any JSON value
 
 
 === Includes ===
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index f4a2f74..8c0ba57 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -40,6 +40,8 @@ struct Visitor
 void (*type_str)(Visitor *v, char **obj, const char *name, Error **errp);
 void (*type_number)(Visitor *v, double *obj, const char *name,
 Error **errp);
+void (*type_any)(Visitor *v, QObject **obj, const char *name,
+ Error **errp);
 
 /* May be NULL */
 void (*optional)(Visitor *v, bool *present, const char *name,
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 00ba104..cfc19a6 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -58,6 +58,7 @@ void visit_type_size(Visitor *v, uint64_t *obj, const char 
*name, Error **errp);
 void visit_type_bool(Visitor *v, bool *obj, const char *name, Error **errp);
 void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp);
 void visit_type_number(Visitor *v, double *obj, const char *name, Error 
**errp);
+void visit_type_any(Visitor *v, QObject **obj, const char *name, Error **errp);
 bool visit_start_union(Visitor *v, bool data_present, Error **errp);
 void visit_end_union(Visitor *v, bool data_present, Error **errp);
 
diff --git a/qapi/qapi-dealloc-visitor.c b/qapi/qapi-dealloc-visitor.c
index d7f92c5..737deab 100644
--- a/qapi/qapi-dealloc-visitor.c
+++ b/qapi/qapi-dealloc-visitor.c
@@ -151,6 +151,14 @@ static void qapi_dealloc_type_number(Visitor *v, double 
*obj, const char *name,
 {
 }
 
+static void qapi_dealloc_type_anything(Visitor *v, QObject **obj,
+   const char *name, Error **errp)
+{
+if (obj) {
+qobject_decref(*obj);
+}
+}
+
 static void qapi_dealloc_type_size(Visitor *v, uint64_t *obj, const char *name,
Error **errp)
 {
@@ -216,6 +224,7 @@ QapiDeallocVisitor *qapi_dealloc_visitor_new(void)
 v->visitor.type_bool = qapi_dealloc_type_bool;
 v->visitor.type_str = qapi_dealloc_type_str;
 v->visitor.type_number = qapi_dealloc_type_number;
+v->visitor.type_

Re: [Qemu-devel] [PATCH 2/6] hw/s390x/css: Use type_init() instead of machine_init()

[Christian Borntraeger]

Am 19.09.2015 um 17:36 schrieb Andreas Färber:
> Here, machine_init() is being misused to initialize global variables.
> Being all local, it's safe to use the slightly earlier type_init().
> 
> This prepares for dropping machine_init().
> 
> Signed-off-by: Andreas Färber 

Acked-by: Christian Borntraeger 

I think this should go via your tree to coordinate with machine_init removal.



> ---
>  hw/s390x/css.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/s390x/css.c b/hw/s390x/css.c
> index c033612..d164fa8 100644
> --- a/hw/s390x/css.c
> +++ b/hw/s390x/css.c
> @@ -1509,7 +1509,7 @@ static void css_init(void)
>  channel_subsys->chnmon_active = false;
>  QTAILQ_INIT(&channel_subsys->io_adapters);
>  }
> -machine_init(css_init);
> +type_init(css_init)
> 
>  void css_reset_sch(SubchDev *sch)
>  {
> 

Re: [Qemu-devel] [PATCH] virtio-9p: migrate virtio subsections

[Greg Kurz]

On Sat, 19 Sep 2015 15:34:02 +0530
"Aneesh Kumar K.V"  wrote:

> Greg Kurz  writes:
> 
> > In a cross-endian setup, the virtio-9p device has state in @device_endian. 
> > It
> > must be migrated.
> >
> > Signed-off-by: Greg Kurz 
> 
> With 9p mounted, we don't support qemu migration. We have
> migration blocker added in v9fs_attach. 
> 

Ok I should have been more precise... If you unmount the 9p share, then
migrate, then try to remount, you end up with a hung mount command because
@device_endian has the wrong value...

> > ---
> >  hw/9pfs/virtio-9p-device.c |   11 +++
> >  1 file changed, 11 insertions(+)
> >
> > diff --git a/hw/9pfs/virtio-9p-device.c b/hw/9pfs/virtio-9p-device.c
> > index 93a407c45926..e3abcfaffb2a 100644
> > --- a/hw/9pfs/virtio-9p-device.c
> > +++ b/hw/9pfs/virtio-9p-device.c
> > @@ -43,6 +43,16 @@ static void virtio_9p_get_config(VirtIODevice *vdev, 
> > uint8_t *config)
> >  g_free(cfg);
> >  }
> >
> > +static void virtio_9p_save(QEMUFile *f, void *opaque)
> > +{
> > +virtio_save(VIRTIO_DEVICE(opaque), f);
> > +}
> > +
> > +static int virtio_9p_load(QEMUFile *f, void *opaque, int version_id)
> > +{
> > +return virtio_load(VIRTIO_DEVICE(opaque), f, version_id);
> > +}
> > +
> >  static void virtio_9p_device_realize(DeviceState *dev, Error **errp)
> >  {
> >  VirtIODevice *vdev = VIRTIO_DEVICE(dev);
> > @@ -130,6 +140,7 @@ static void virtio_9p_device_realize(DeviceState *dev, 
> > Error **errp)
> >  }
> >  v9fs_path_free(&path);
> >
> > +register_savevm(dev, "virtio-9p", -1, 1, virtio_9p_save, 
> > virtio_9p_load, s);
> >  return;
> >  out:
> >  g_free(s->ctx.fs_root);

Re: [Qemu-devel] [PATCH v4] ppc/spapr: Implement H_RANDOM hypercall in QEMU

[Thomas Huth]

On 21/09/15 10:01, Greg Kurz wrote:
> On Mon, 21 Sep 2015 12:10:00 +1000
> David Gibson  wrote:
> 
>> On Fri, Sep 18, 2015 at 11:05:52AM +0200, Greg Kurz wrote:
>>> On Thu, 17 Sep 2015 10:49:41 +0200
>>> Thomas Huth  wrote:
>>>
 The PAPR interface defines a hypercall to pass high-quality
 hardware generated random numbers to guests. Recent kernels can
 already provide this hypercall to the guest if the right hardware
 random number generator is available. But in case the user wants
 to use another source like EGD, or QEMU is running with an older
 kernel, we should also have this call in QEMU, so that guests that
 do not support virtio-rng yet can get good random numbers, too.

 This patch now adds a new pseudo-device to QEMU that either
 directly provides this hypercall to the guest or is able to
 enable the in-kernel hypercall if available.
...
>>> It is a good thing that the user can choose between in-kernel and backend,
>>> and this patch does the work.
>>>
>>> This being said, I am not sure about the use case where a user has a hwrng
>>> capable platform and wants to run guests without any hwrng support at all is
>>> an appropriate default behavior... I guess we will find more users that want
>>> in-kernel being the default if it is available.
>>>
>>> The patch below modifies yours to do just this: the pseudo-device is only
>>> created if hwrng is present and not already created.
>>
>> I have mixed feelings about this.  On the one hand, I agree that it
>> would be nice to allow H_RANDOM support by default.  On the other hand
>> the patch below leaves no way to turn it off for testing purposes.  It
>> also adds another place where the guest hardware depends on the host
>> configuration, which adds to the already substantial mess of ensuring
>> that source and destination hardware configuration matches for
>> migration.
> 
> Yeah, describing the guest hw is really essential for migration... this
> is best addressed at the libvirt level with a full XML description of
> the machine... but FWIW if we are talking about running pseries on a
> POWER8 or newer host, I am not aware about "hwrng-less" boards... but
> I am probably missing something :)

Maybe it would be at least ok to enable it by default as long as
"-nodefaults" has not been specified as command line option?

> Back to Thomas' patch, it does the job and brings H_RANDOM, which is
> currently missing.
> 
> Acked-by: Greg Kurz 
> 
> I could test both use-kvm and backend flavors (including migration).
> 
> Tested-by: Greg Kurz 

Thanks!

 Thomas




signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 5/7] qdev: Protect device-list-properties against broken devices

[David Hildenbrand]

> Am 18.09.2015 um 14:00 schrieb Markus Armbruster:
> > Several devices don't survive object_unref(object_new(T)): they crash
> > or hang during cleanup, or they leave dangling pointers behind.
> > 
> > This breaks at least device-list-properties, because
> > qmp_device_list_properties() needs to create a device to find its
> > properties.  Broken in commit f4eb32b "qmp: show QOM properties in
> > device-list-properties", v2.1.  Example reproducer:
> > 
> > $ qemu-system-aarch64 -nodefaults -display none -machine none -S -qmp 
> > stdio
> > {"QMP": {"version": {"qemu": {"micro": 50, "minor": 4, "major": 2}, 
> > "package": ""}, "capabilities": []}}
> > { "execute": "qmp_capabilities" }
> > {"return": {}}
> > { "execute": "device-list-properties", "arguments": { "typename": 
> > "pxa2xx-pcmcia" } }
> > qemu-system-aarch64: /home/armbru/work/qemu/memory.c:1307: 
> > memory_region_finalize: Assertion `((&mr->subregions)->tqh_first == ((void 
> > *)0))' failed.
> > Aborted (core dumped)
> > [Exit 134 (SIGABRT)]
> > 
> > Unfortunately, I can't fix the problems in these devices right now.
> > Instead, add DeviceClass member cannot_even_create_with_object_new_yet
> > to mark them:
> > 
> > * Crash or hang during cleanup (didn't debug them, so I can't say
> >   why): "pxa2xx-pcmcia", "realview_pci", "versatile_pci",
> >   "s390-sclp-event-facility", "sclp"
> 
> Ack for the sclp things. Theses devices are created by the machine and
> sclp creates the event-facility, so not having a way to query properties
> for these devices is better than a hang.
> 
> David, can you have a look on why these devices fail as outlined?
> 

The problem was that the quiesce and cpu hotplug sclp event devices had no
parent (onoly a parent bus). So when the bus (inside the event facility) was
removed, it looped forever trying remove/unparent it's "bus childs" (which had
no parent).

sclp (parent=machine)
-> even-facility (parent=sclp)
-> bus (parent=event-facility)
  -> quiesce (parent=null)
  -> cpu hotplug (parent=null)

Maybe that helps others struggling with the same symptoms.


Just a quick comment on the introspection:

I don't think it is a good idea to expose properties that way. Temporarily
creating devices for the sake of querying property names sounds very wrong to
me, because certain devices require certain knowledge about how and when they
can be created.

Feels a little like hacking an interface to a java program, which allows to
create any object of a special kind dynamically (constructor arguments?),
reading some member variable (names) via reflections and then throwing that
object away. Which sounds very wrong to me.

Wonder if it wouldn't make more sense to query only the static properties of a
device. I mean if the dynamic properties are dynamic by definition (and can
change during runtime). So looking up the static properties via the typename
feels more KIS-style to me. Of course, the relevant properties would have to be
defined statically then, which shouldn't be a problem.

Dynamic properties that are actually created could depend on almost
everything in the system - why fake something that we don't know.

David

Re: [Qemu-devel] [PATCH v4] ppc/spapr: Implement H_RANDOM hypercall in QEMU

[Greg Kurz]

On Mon, 21 Sep 2015 10:26:52 +0200
Thomas Huth  wrote:

> On 21/09/15 10:01, Greg Kurz wrote:
> > On Mon, 21 Sep 2015 12:10:00 +1000
> > David Gibson  wrote:
> > 
> >> On Fri, Sep 18, 2015 at 11:05:52AM +0200, Greg Kurz wrote:
> >>> On Thu, 17 Sep 2015 10:49:41 +0200
> >>> Thomas Huth  wrote:
> >>>
>  The PAPR interface defines a hypercall to pass high-quality
>  hardware generated random numbers to guests. Recent kernels can
>  already provide this hypercall to the guest if the right hardware
>  random number generator is available. But in case the user wants
>  to use another source like EGD, or QEMU is running with an older
>  kernel, we should also have this call in QEMU, so that guests that
>  do not support virtio-rng yet can get good random numbers, too.
> 
>  This patch now adds a new pseudo-device to QEMU that either
>  directly provides this hypercall to the guest or is able to
>  enable the in-kernel hypercall if available.
> ...
> >>> It is a good thing that the user can choose between in-kernel and backend,
> >>> and this patch does the work.
> >>>
> >>> This being said, I am not sure about the use case where a user has a hwrng
> >>> capable platform and wants to run guests without any hwrng support at all 
> >>> is
> >>> an appropriate default behavior... I guess we will find more users that 
> >>> want
> >>> in-kernel being the default if it is available.
> >>>
> >>> The patch below modifies yours to do just this: the pseudo-device is only
> >>> created if hwrng is present and not already created.
> >>
> >> I have mixed feelings about this.  On the one hand, I agree that it
> >> would be nice to allow H_RANDOM support by default.  On the other hand
> >> the patch below leaves no way to turn it off for testing purposes.  It
> >> also adds another place where the guest hardware depends on the host
> >> configuration, which adds to the already substantial mess of ensuring
> >> that source and destination hardware configuration matches for
> >> migration.
> > 
> > Yeah, describing the guest hw is really essential for migration... this
> > is best addressed at the libvirt level with a full XML description of
> > the machine... but FWIW if we are talking about running pseries on a
> > POWER8 or newer host, I am not aware about "hwrng-less" boards... but
> > I am probably missing something :)
> 
> Maybe it would be at least ok to enable it by default as long as
> "-nodefaults" has not been specified as command line option?
> 

It makes a lot of sense indeed. I guess David should take your patch
as it is now and the default behavior could be a follow up.

> > Back to Thomas' patch, it does the job and brings H_RANDOM, which is
> > currently missing.
> > 
> > Acked-by: Greg Kurz 
> > 
> > I could test both use-kvm and backend flavors (including migration).
> > 
> > Tested-by: Greg Kurz 
> 
> Thanks!
> 
>  Thomas
> 
> 



pgpMbH_Bk2AwG.pgp
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v4 07/22] vhost: alloc shareable log

[Michael S. Tsirkin]

On Sat, Sep 19, 2015 at 12:11:58PM +0200, marcandre.lur...@redhat.com wrote:
> From: Marc-André Lureau 
> 
> If the backend is of type VHOST_BACKEND_TYPE_USER, allocate
> shareable memory. Next patch will only allocate when the backend
> has the required feature.
> 
> Note: vhost_log_get() can use a global "vhost_log" that can be shared by
> several vhost devices. We may want instead a common shareable log and a
> common non-shareable one.
> 
> Signed-off-by: Marc-André Lureau 

Well you do at least need to count the number of times the log is
shared.  Otherwise, if you share the log, then unshare it, you are left
with a shared one.


> ---
>  hw/virtio/vhost.c | 38 +++---
>  include/hw/virtio/vhost.h |  3 ++-
>  2 files changed, 33 insertions(+), 8 deletions(-)
> 
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index a08c36b..cd3af16 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -18,6 +18,7 @@
>  #include "qemu/atomic.h"
>  #include "qemu/range.h"
>  #include "qemu/error-report.h"
> +#include "qemu/memfd.h"
>  #include 
>  #include "exec/address-spaces.h"
>  #include "hw/virtio/virtio-bus.h"
> @@ -286,20 +287,34 @@ static uint64_t vhost_get_log_size(struct vhost_dev 
> *dev)
>  }
>  return log_size;
>  }
> -static struct vhost_log *vhost_log_alloc(uint64_t size)
> +
> +static struct vhost_log *vhost_log_alloc(uint64_t size, bool share)
>  {
> -struct vhost_log *log = g_malloc0(sizeof *log + size * 
> sizeof(*(log->log)));
> +struct vhost_log *log;
> +uint64_t logsize = size * sizeof(*(log->log));
> +int fd = -1;
> +
> +log = g_new0(struct vhost_log, 1);
> +if (share) {
> +log->log = qemu_memfd_alloc("vhost-log", logsize,
> +F_SEAL_GROW|F_SEAL_SHRINK|F_SEAL_SEAL, 
> &fd);
> +memset(log->log, 0, logsize);
> +} else {
> +log->log = g_malloc0(logsize);
> +}
>  
>  log->size = size;
>  log->refcnt = 1;
> +log->fd = fd;
>  
>  return log;
>  }
>  
> -static struct vhost_log *vhost_log_get(uint64_t size)
> +static struct vhost_log *vhost_log_get(uint64_t size, bool share)
>  {
> -if (!vhost_log || vhost_log->size != size) {
> -vhost_log = vhost_log_alloc(size);
> +if (!vhost_log || vhost_log->size != size ||
> +(share && vhost_log->fd == -1)) {
> +vhost_log = vhost_log_alloc(size, share);
>  } else {
>  ++vhost_log->refcnt;
>  }
> @@ -324,13 +339,21 @@ static void vhost_log_put(struct vhost_dev *dev, bool 
> sync)
>  if (vhost_log == log) {
>  vhost_log = NULL;
>  }
> +
> +if (log->fd == -1) {
> +g_free(log->log);
> +} else {
> +qemu_memfd_free(log->log, log->size * sizeof(*(log->log)),
> +log->fd);
> +}
>  g_free(log);
>  }
>  }
>  
>  static inline void vhost_dev_log_resize(struct vhost_dev* dev, uint64_t size)
>  {
> -struct vhost_log *log = vhost_log_get(size);
> +bool share = dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER;
> +struct vhost_log *log = vhost_log_get(size, share);
>  uint64_t log_base = (uintptr_t)log->log;
>  int r;
>  
> @@ -1136,9 +1159,10 @@ int vhost_dev_start(struct vhost_dev *hdev, 
> VirtIODevice *vdev)
>  
>  if (hdev->log_enabled) {
>  uint64_t log_base;
> +bool share = hdev->vhost_ops->backend_type == 
> VHOST_BACKEND_TYPE_USER;
>  
>  hdev->log_size = vhost_get_log_size(hdev);
> -hdev->log = vhost_log_get(hdev->log_size);
> +hdev->log = vhost_log_get(hdev->log_size, share);
>  log_base = (uintptr_t)hdev->log->log;
>  r = hdev->vhost_ops->vhost_call(hdev, VHOST_SET_LOG_BASE,
>  hdev->log_size ? &log_base : NULL);
> diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
> index 6467c73..ab1dcac 100644
> --- a/include/hw/virtio/vhost.h
> +++ b/include/hw/virtio/vhost.h
> @@ -31,7 +31,8 @@ typedef unsigned long vhost_log_chunk_t;
>  struct vhost_log {
>  unsigned long long size;
>  int refcnt;
> -vhost_log_chunk_t log[0];
> +int fd;
> +vhost_log_chunk_t *log;
>  };
>  
>  struct vhost_memory;
> -- 
> 2.4.3

Re: [Qemu-devel] [PATCH v4 12/22] vhost: only use shared log if in use by backend

[Michael S. Tsirkin]

On Sat, Sep 19, 2015 at 12:12:03PM +0200, marcandre.lur...@redhat.com wrote:
> From: Marc-André Lureau 
> 
> Do not allocate a shared log if the backend doesn't support it.
> 
> Signed-off-by: Marc-André Lureau 

If not there, we probably want to block live migration, correct?

> ---
>  hw/virtio/vhost-user.c|  9 +
>  hw/virtio/vhost.c | 15 ++-
>  include/hw/virtio/vhost-backend.h |  4 
>  3 files changed, 23 insertions(+), 5 deletions(-)
> 
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index 3cb5559..d12cd66 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -508,10 +508,19 @@ static int vhost_user_cleanup(struct vhost_dev *dev)
>  return 0;
>  }
>  
> +static bool vhost_user_want_shm_log(struct vhost_dev *dev)
> +{
> +assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER);
> +
> +return virtio_has_feature(dev->protocol_features,
> +  VHOST_USER_PROTOCOL_F_LOG_SHMFD);
> +}
> +
>  const VhostOps user_ops = {
>  .backend_type = VHOST_BACKEND_TYPE_USER,
>  .vhost_backend_init = vhost_user_init,
>  .vhost_backend_cleanup = vhost_user_cleanup,
> +.vhost_backend_want_shm_log = vhost_user_want_shm_log,
>  
>  .vhost_net_set_backend = vhost_net_set_backend,
>  .vhost_scsi_set_endpoint = vhost_scsi_set_endpoint,
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index 0fa7b69..d5070fb 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -350,10 +350,15 @@ static void vhost_log_put(struct vhost_dev *dev, bool 
> sync)
>  }
>  }
>  
> -static inline void vhost_dev_log_resize(struct vhost_dev* dev, uint64_t size)
> +static bool vhost_dev_log_is_shared(struct vhost_dev *dev)
>  {
> -bool share = dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER;
> -struct vhost_log *log = vhost_log_get(size, share);
> +return dev->vhost_ops->vhost_backend_want_shm_log &&
> +   dev->vhost_ops->vhost_backend_want_shm_log(dev);
> +}
> +
> +static inline void vhost_dev_log_resize(struct vhost_dev *dev, uint64_t size)
> +{
> +struct vhost_log *log = vhost_log_get(size, 
> vhost_dev_log_is_shared(dev));
>  uint64_t log_base = (uintptr_t)log->log;
>  int r;
>  
> @@ -1161,10 +1166,10 @@ int vhost_dev_start(struct vhost_dev *hdev, 
> VirtIODevice *vdev)
>  
>  if (hdev->log_enabled) {
>  uint64_t log_base;
> -bool share = hdev->vhost_ops->backend_type == 
> VHOST_BACKEND_TYPE_USER;
>  
>  hdev->log_size = vhost_get_log_size(hdev);
> -hdev->log = vhost_log_get(hdev->log_size, share);
> +hdev->log = vhost_log_get(hdev->log_size,
> +  vhost_dev_log_is_shared(hdev));
>  log_base = (uintptr_t)hdev->log->log;
>  r = hdev->vhost_ops->vhost_set_log_base(hdev,
>  hdev->log_size ? log_base : 
> 0,
> diff --git a/include/hw/virtio/vhost-backend.h 
> b/include/hw/virtio/vhost-backend.h
> index a87f0b3..44bf200 100644
> --- a/include/hw/virtio/vhost-backend.h
> +++ b/include/hw/virtio/vhost-backend.h
> @@ -11,6 +11,8 @@
>  #ifndef VHOST_BACKEND_H_
>  #define VHOST_BACKEND_H_
>  
> +#include 
> +
>  typedef enum VhostBackendType {
>  VHOST_BACKEND_TYPE_NONE = 0,
>  VHOST_BACKEND_TYPE_KERNEL = 1,
> @@ -28,6 +30,7 @@ struct vhost_scsi_target;
>  
>  typedef int (*vhost_backend_init)(struct vhost_dev *dev, void *opaque);
>  typedef int (*vhost_backend_cleanup)(struct vhost_dev *dev);
> +typedef bool (*vhost_backend_want_shm_log)(struct vhost_dev *dev);
>  
>  typedef int (*vhost_net_set_backend_op)(struct vhost_dev *dev,
>  struct vhost_vring_file *file);
> @@ -67,6 +70,7 @@ typedef struct VhostOps {
>  
>  vhost_backend_init vhost_backend_init;
>  vhost_backend_cleanup vhost_backend_cleanup;
> +vhost_backend_want_shm_log vhost_backend_want_shm_log;
>  
>  vhost_net_set_backend_op vhost_net_set_backend;
>  vhost_scsi_set_endpoint_op vhost_scsi_set_endpoint;
> -- 
> 2.4.3

Re: [Qemu-devel] [PATCH PULL v3 08/11] crypto: add sanity checking of TLS x509 credentials

[Daniel P. Berrange]

On Sat, Sep 19, 2015 at 09:47:01PM -0700, Peter Crosthwaite wrote:
> On Tue, Sep 15, 2015 at 7:36 AM, Daniel P. Berrange  
> wrote:
> > If the administrator incorrectly sets up their x509 certificates,
> > the errors seen at runtime during connection attempts are very
> > obscure and difficult to diagnose. This has been a particular
> > problem for people using openssl to generate their certificates
> > instead of the gnutls certtool, because the openssl tools don't
> > turn on the various x509 extensions that gnutls expects to be
> > present by default.
> >
> > This change thus adds support in the TLS credentials object to
> > sanity check the certificates when QEMU first loads them. This
> > gives the administrator immediate feedback for the majority of
> > common configuration mistakes, reducing the pain involved in
> > setting up TLS. The code is derived from equivalent code that
> > has been part of libvirt's TLS support and has been seen to be
> > valuable in assisting admins.
> >
> > It is possible to disable the sanity checking, however, via
> > the new 'sanity-check' property on the tls-creds object type,
> > with a value of 'no'.
> >
> > Unit tests are included in this change to verify the correctness
> > of the sanity checking code in all the key scenarios it is
> > intended to cope with. As part of the test suite, the pkix_asn1_tab.c
> > from gnutls is imported. This file is intentionally copied from the
> > (long since obsolete) gnutls 1.6.3 source tree, since that version
> > was still under GPLv2+, rather than the GPLv3+ of gnutls >= 2.0.
> >
> > Signed-off-by: Daniel P. Berrange 
> > ---
> >  configure|   22 +
> >  crypto/tlscredsx509.c|  546 +++
> >  include/crypto/tlscredsx509.h|1 +
> >  tests/.gitignore |3 +
> >  tests/Makefile   |5 +
> >  tests/crypto-tls-x509-helpers.c  |  485 +
> >  tests/crypto-tls-x509-helpers.h  |  133 +
> >  tests/pkix_asn1_tab.c| 1104 
> > ++
> >  tests/test-crypto-tlscredsx509.c |  731 +
> >  trace-events |5 +
> >  10 files changed, 3035 insertions(+)
> >  create mode 100644 tests/crypto-tls-x509-helpers.c
> >  create mode 100644 tests/crypto-tls-x509-helpers.h
> >  create mode 100644 tests/pkix_asn1_tab.c
> >  create mode 100644 tests/test-crypto-tlscredsx509.c
> >
> > diff --git a/configure b/configure
> > index d7c24cd..bdd302c 100755
> > --- a/configure
> > +++ b/configure
> > @@ -416,6 +416,9 @@ if test "$debug_info" = "yes"; then
> >  LDFLAGS="-g $LDFLAGS"
> >  fi
> >
> > +test_cflags=""
> > +test_libs=""
> > +
> >  # make source path absolute
> >  source_path=`cd "$source_path"; pwd`
> >
> > @@ -2249,6 +2252,19 @@ if test "$gnutls_nettle" != "no"; then
> >  fi
> >  fi
> >
> > +##
> > +# libtasn1 - only for the TLS creds/session test suite
> > +
> > +tasn1=yes
> > +if $pkg_config --exists "libtasn1"; then
> > +tasn1_cflags=`$pkg_config --cflags libtasn1`
> > +tasn1_libs=`$pkg_config --libs libtasn1`
> > +test_cflags="$test_cflags $tasn1_cflags"
> > +test_libs="$test_libs $tasn1_libs"
> > +else
> > +tasn1=no
> > +fi
> > +
> >
> >  ##
> >  # VTE probe
> > @@ -4574,6 +4590,7 @@ echo "GNUTLS support$gnutls"
> >  echo "GNUTLS hash   $gnutls_hash"
> >  echo "GNUTLS gcrypt $gnutls_gcrypt"
> >  echo "GNUTLS nettle $gnutls_nettle ${gnutls_nettle+($nettle_version)}"
> > +echo "libtasn1  $tasn1"
> >  echo "VTE support   $vte"
> >  echo "curses support$curses"
> >  echo "curl support  $curl"
> > @@ -4945,6 +4962,9 @@ if test "$gnutls_nettle" = "yes" ; then
> >echo "CONFIG_GNUTLS_NETTLE=y" >> $config_host_mak
> >echo "CONFIG_NETTLE_VERSION_MAJOR=${nettle_version%%.*}" >> 
> > $config_host_mak
> >  fi
> > +if test "$tasn1" = "yes" ; then
> > +  echo "CONFIG_TASN1=y" >> $config_host_mak
> > +fi
> >  if test "$vte" = "yes" ; then
> >echo "CONFIG_VTE=y" >> $config_host_mak
> >echo "VTE_CFLAGS=$vte_cflags" >> $config_host_mak
> > @@ -5268,6 +5288,8 @@ echo "EXESUF=$EXESUF" >> $config_host_mak
> >  echo "DSOSUF=$DSOSUF" >> $config_host_mak
> >  echo "LDFLAGS_SHARED=$LDFLAGS_SHARED" >> $config_host_mak
> >  echo "LIBS_QGA+=$libs_qga" >> $config_host_mak
> > +echo "TEST_LIBS=$test_libs" >> $config_host_mak
> > +echo "TEST_CFLAGS=$test_cflags" >> $config_host_mak
> 
> I am not too sure exactly why yet, but this breaks the build for me
> when using pixman submodule with --enable-werror configure:

[snip]

I have pixman pre-installed so didn't notice this. I'll investigate
and report back...


Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- htt

Re: [Qemu-devel] [PATCH V4 0/2] Move sdhci.h to include/hw/sd

[Sai Pavan Boddu]

Ping

Thanks,
Sai Pavan

> -Original Message-
> From: Sai Pavan Boddu [mailto:sai.pavan.bo...@xilinx.com]
> Sent: Monday, September 14, 2015 2:57 PM
> To: qemu-devel@nongnu.org; stefa...@gmail.com;
> crosthwaitepe...@gmail.com; peter.mayd...@linaro.org
> Cc: Alistair Francis; Edgar Iglesias; Sai Pavan Boddu
> Subject: [PATCH V4 0/2] Move sdhci.h to include/hw/sd
> 
> Move sdhci.h splitting it into common and internal.
> Create a new directory for sd in include/hw/.
> Correct paths of sd.h in at every instance of #include.
> 
> Sai Pavan Boddu (2):
>   sd.h: Move sd.h to include/hw/sd/
>   sdhci: Split sdhci.h for public and internal device usage
> 
>  hw/sd/milkymist-memcard.c|  2 +-
>  hw/sd/omap_mmc.c |  2 +-
>  hw/sd/pl181.c|  2 +-
>  hw/sd/pxa2xx_mmci.c  |  2 +-
>  hw/sd/sd.c   |  2 +-
>  hw/sd/sdhci.c|  1 -
>  hw/sd/sdhci.h| 67 +---
>  hw/sd/ssi-sd.c   |  2 +-
>  include/hw/{ => sd}/sd.h |  0
>  include/hw/sd/sdhci-common.h | 92
> 
>  10 files changed, 100 insertions(+), 72 deletions(-)
>  rename include/hw/{ => sd}/sd.h (100%)
>  create mode 100644 include/hw/sd/sdhci-common.h
> 
> --
> 1.9.1

Re: [Qemu-devel] [PATCH v4 11/22] vhost-user: send log shm fd along with log_base

[Michael S. Tsirkin]

On Sat, Sep 19, 2015 at 12:12:02PM +0200, marcandre.lur...@redhat.com wrote:
> From: Marc-André Lureau 
> 
> Send the shm for the dirty pages logging if the backend supports
> VHOST_USER_PROTOCOL_F_LOG_SHMFD.
> 
> Signed-off-by: Marc-André Lureau 
> ---
>  hw/virtio/vhost-backend.c |  3 ++-
>  hw/virtio/vhost-user.c| 17 ++---
>  hw/virtio/vhost.c |  5 +++--
>  include/hw/virtio/vhost-backend.h |  4 +++-
>  4 files changed, 22 insertions(+), 7 deletions(-)
> 
> diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c
> index bf2d1d4..3f2d6ea 100644
> --- a/hw/virtio/vhost-backend.c
> +++ b/hw/virtio/vhost-backend.c
> @@ -66,7 +66,8 @@ static int vhost_scsi_get_abi_version(struct vhost_dev 
> *dev, int *version)
>  return vhost_kernel_call(dev, VHOST_SCSI_GET_ABI_VERSION, version);
>  }
>  
> -static int vhost_set_log_base(struct vhost_dev *dev, uint64_t base)
> +static int vhost_set_log_base(struct vhost_dev *dev, uint64_t base,
> +  struct vhost_log *log)
>  {
>  return vhost_kernel_call(dev, VHOST_SET_LOG_BASE, &base);
>  }
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index d3334f5..3cb5559 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -26,7 +26,9 @@
>  #define VHOST_MEMORY_MAX_NREGIONS8
>  
>  #define VHOST_USER_F_PROTOCOL_FEATURES 30
> -#define VHOST_USER_PROTOCOL_FEATURE_MASK 0x0ULL
> +
> +#define VHOST_USER_PROTOCOL_FEATURE_MASK 0x1ULL
> +#define VHOST_USER_PROTOCOL_F_LOG_SHMFD 0
>  
>  typedef enum VhostUserRequest {
>  VHOST_USER_NONE = 0,
> @@ -182,8 +184,11 @@ static int vhost_scsi_get_abi_version(struct vhost_dev 
> *dev, int *version)
>  return -1;
>  }
>  
> -static int vhost_set_log_base(struct vhost_dev *dev, uint64_t base)
> +static int vhost_set_log_base(struct vhost_dev *dev, uint64_t base,
> +  struct vhost_log *log)
>  {
> +int fds[VHOST_MEMORY_MAX_NREGIONS];
> +size_t fd_num = 0;
>  VhostUserMsg msg = {
>  .request = VHOST_USER_SET_LOG_BASE,
>  .flags = VHOST_USER_VERSION,
> @@ -191,7 +196,13 @@ static int vhost_set_log_base(struct vhost_dev *dev, 
> uint64_t base)
>  .size = sizeof(m.u64),
>  };
>  
> -vhost_user_write(dev, &msg, NULL, 0);
> +if (virtio_has_feature(dev->protocol_features,
> +   VHOST_USER_PROTOCOL_F_LOG_SHMFD) &&
> +log->fd != -1) {
> +fds[fd_num++] = log->fd;
> +}
> +
> +vhost_user_write(dev, &msg, fds, fd_num);
>  
>  return 0;
>  }

Doesn't this one have to get a response?
Otherwise the other side might still be using the old
buffer after we've updated this one.

Probably need a protocol feature, but I guess we can reuse
VHOST_USER_PROTOCOL_F_LOG_SHMFD - IIUC logging doesn't
work without this anyway ...


> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index 83ef164..0fa7b69 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -359,7 +359,7 @@ static inline void vhost_dev_log_resize(struct vhost_dev* 
> dev, uint64_t size)
>  
>  /* inform backend of log switching, this must be done before
> releasing the current log, to ensure no logging is lost */
> -r = dev->vhost_ops->vhost_set_log_base(dev, log_base);
> +r = dev->vhost_ops->vhost_set_log_base(dev, log_base, log);
>  assert(r >= 0);
>  vhost_log_put(dev, true);
>  dev->log = log;
> @@ -1167,7 +1167,8 @@ int vhost_dev_start(struct vhost_dev *hdev, 
> VirtIODevice *vdev)
>  hdev->log = vhost_log_get(hdev->log_size, share);
>  log_base = (uintptr_t)hdev->log->log;
>  r = hdev->vhost_ops->vhost_set_log_base(hdev,
> -hdev->log_size ? log_base : 
> 0);
> +hdev->log_size ? log_base : 
> 0,
> +hdev->log);
>  if (r < 0) {
>  r = -errno;
>  goto fail_log;
> diff --git a/include/hw/virtio/vhost-backend.h 
> b/include/hw/virtio/vhost-backend.h
> index 42cfb87..a87f0b3 100644
> --- a/include/hw/virtio/vhost-backend.h
> +++ b/include/hw/virtio/vhost-backend.h
> @@ -20,6 +20,7 @@ typedef enum VhostBackendType {
>  
>  struct vhost_dev;
>  struct vhost_memory;
> +struct vhost_log;
>  struct vhost_vring_file;
>  struct vhost_vring_state;
>  struct vhost_vring_addr;
> @@ -36,7 +37,8 @@ typedef int (*vhost_scsi_clear_endpoint_op)(struct 
> vhost_dev *dev,
>  struct vhost_scsi_target 
> *target);
>  typedef int (*vhost_scsi_get_abi_version_op)(struct vhost_dev *dev,
>   int *version);
> -typedef int (*vhost_set_log_base_op)(struct vhost_dev *dev, uint64_t base);
> +typedef int (*vhost_set_log_base_op)(struct vhost_dev *dev, uint64_t base,
> + struct vhost_log *log);
>  typedef int (*vhost

Re: [Qemu-devel] [PATCH v4 09/22] vhost: use a function for each call

[Michael S. Tsirkin]

On Mon, Sep 21, 2015 at 09:33:04AM +0200, Thibaut Collet wrote:
> > +static int vhost_reset_owner(struct vhost_dev *dev)
> > +{
> > +VhostUserMsg msg = {
> > +.request = VHOST_USER_RESET_OWNER,
> > +.flags = VHOST_USER_VERSION,
> > +};
> > +
> > +vhost_user_write(dev, &msg, NULL, 0);
> > +
> >  return 0;
> >  }
> >
> 
> If you define a specific function for each messages it can be
> interesting to add function for messages to get/set protocol features
> (VHOST_USER_GET_PROTOCOL_FEATURES and
> VHOST_USER_SET_PROTOCOL_FEATURES) and modify the vhost_user_init
> function to call these specific functions (this function can be
> already modified to use the specific function for
> VHOST_USER_GET_FEATURES message to avoid code duplication)

I'm not sure what this would buy us ... can be a patch on top.

-- 
MST

Re: [Qemu-devel] [PATCH v4 09/22] vhost: use a function for each call

[Michael S. Tsirkin]

On Sat, Sep 19, 2015 at 12:12:00PM +0200, marcandre.lur...@redhat.com wrote:
> From: Marc-André Lureau 
> 
> Replace the generic vhost_call() by specific functions for each
> function call to help with type safety and changing arguments.
> 
> Signed-off-by: Marc-André Lureau 

OK but I would rather not make logging feature depend on the
refactoring.
How about moving each call over from vhost_call by a separate patch?
Start with vhost_set_log_base as the 1st one, this way I can apply
logging support while we still review the larger refactorings.

> ---
>  hw/net/vhost_net.c|  12 +-
>  hw/scsi/vhost-scsi.c  |   7 +-
>  hw/virtio/vhost-backend.c | 140 +++--
>  hw/virtio/vhost-user.c| 402 
> ++
>  hw/virtio/vhost.c |  34 ++--
>  include/hw/virtio/vhost-backend.h |  59 +-
>  6 files changed, 484 insertions(+), 170 deletions(-)
> 
> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
> index 9d32d76..d116fb3 100644
> --- a/hw/net/vhost_net.c
> +++ b/hw/net/vhost_net.c
> @@ -243,8 +243,7 @@ static int vhost_net_start_one(struct vhost_net *net,
>  file.fd = net->backend;
>  for (file.index = 0; file.index < net->dev.nvqs; ++file.index) {
>  const VhostOps *vhost_ops = net->dev.vhost_ops;
> -r = vhost_ops->vhost_call(&net->dev, VHOST_NET_SET_BACKEND,
> -  &file);
> +r = vhost_ops->vhost_net_set_backend(&net->dev, &file);
>  if (r < 0) {
>  r = -errno;
>  goto fail;
> @@ -257,8 +256,7 @@ fail:
>  if (net->nc->info->type == NET_CLIENT_OPTIONS_KIND_TAP) {
>  while (file.index-- > 0) {
>  const VhostOps *vhost_ops = net->dev.vhost_ops;
> -int r = vhost_ops->vhost_call(&net->dev, VHOST_NET_SET_BACKEND,
> -  &file);
> +int r = vhost_ops->vhost_net_set_backend(&net->dev, &file);
>  assert(r >= 0);
>  }
>  }
> @@ -280,15 +278,13 @@ static void vhost_net_stop_one(struct vhost_net *net,
>  if (net->nc->info->type == NET_CLIENT_OPTIONS_KIND_TAP) {
>  for (file.index = 0; file.index < net->dev.nvqs; ++file.index) {
>  const VhostOps *vhost_ops = net->dev.vhost_ops;
> -int r = vhost_ops->vhost_call(&net->dev, VHOST_NET_SET_BACKEND,
> -  &file);
> +int r = vhost_ops->vhost_net_set_backend(&net->dev, &file);
>  assert(r >= 0);
>  }
>  } else if (net->nc->info->type == NET_CLIENT_OPTIONS_KIND_VHOST_USER) {
>  for (file.index = 0; file.index < net->dev.nvqs; ++file.index) {
>  const VhostOps *vhost_ops = net->dev.vhost_ops;
> -int r = vhost_ops->vhost_call(&net->dev, VHOST_RESET_OWNER,
> -  NULL);
> +int r = vhost_ops->vhost_reset_owner(&net->dev);
>  assert(r >= 0);
>  }
>  }
> diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c
> index bac9ddb..a0034ab 100644
> --- a/hw/scsi/vhost-scsi.c
> +++ b/hw/scsi/vhost-scsi.c
> @@ -45,7 +45,7 @@ static int vhost_scsi_set_endpoint(VHostSCSI *s)
>  
>  memset(&backend, 0, sizeof(backend));
>  pstrcpy(backend.vhost_wwpn, sizeof(backend.vhost_wwpn), vs->conf.wwpn);
> -ret = vhost_ops->vhost_call(&s->dev, VHOST_SCSI_SET_ENDPOINT, &backend);
> +ret = vhost_ops->vhost_scsi_set_endpoint(&s->dev, &backend);
>  if (ret < 0) {
>  return -errno;
>  }
> @@ -60,7 +60,7 @@ static void vhost_scsi_clear_endpoint(VHostSCSI *s)
>  
>  memset(&backend, 0, sizeof(backend));
>  pstrcpy(backend.vhost_wwpn, sizeof(backend.vhost_wwpn), vs->conf.wwpn);
> -vhost_ops->vhost_call(&s->dev, VHOST_SCSI_CLEAR_ENDPOINT, &backend);
> +vhost_ops->vhost_scsi_clear_endpoint(&s->dev, &backend);
>  }
>  
>  static int vhost_scsi_start(VHostSCSI *s)
> @@ -76,8 +76,7 @@ static int vhost_scsi_start(VHostSCSI *s)
>  return -ENOSYS;
>  }
>  
> -ret = vhost_ops->vhost_call(&s->dev,
> -VHOST_SCSI_GET_ABI_VERSION, &abi_version);
> +ret = vhost_ops->vhost_scsi_get_abi_version(&s->dev, &abi_version);
>  if (ret < 0) {
>  return -errno;
>  }
> diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c
> index 4d68a27..bf2d1d4 100644
> --- a/hw/virtio/vhost-backend.c
> +++ b/hw/virtio/vhost-backend.c
> @@ -11,19 +11,10 @@
>  #include "hw/virtio/vhost.h"
>  #include "hw/virtio/vhost-backend.h"
>  #include "qemu/error-report.h"
> +#include "linux/vhost.h"
>  
>  #include 
>  
> -static int vhost_kernel_call(struct vhost_dev *dev, unsigned long int 
> request,
> - void *arg)
> -{
> -int fd = (uintptr_t) dev->opaque;
> -
> -assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_KERNEL);
> -
> -return ioctl(fd,

[Qemu-devel] [PULL 0/1] spice: surface switch fast path requires same format too.

[Gerd Hoffmann]

  Hi,

Spice patch queue, carrying a single bugfix.

please pull,
  Gerd

The following changes since commit 18640989a9f5e4d2e84b566c52ff1fccfa0dbf4a:

  Merge remote-tracking branch 'remotes/afaerber/tags/qom-devices-for-peter' 
into staging (2015-09-19 15:59:52 +0100)

are available in the git repository at:


  git://anongit.freedesktop.org/spice/qemu tags/pull-spice-20150921-1

for you to fetch changes up to b2af43cc379e1d4c30d92af257bedebf0e3f618a:

  spice: surface switch fast path requires same format too. (2015-09-21 
09:52:07 +0200)


spice: surface switch fast path requires same format too.


Gerd Hoffmann (1):
  spice: surface switch fast path requires same format too.

 ui/spice-display.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

[Qemu-devel] [PULL 1/1] spice: surface switch fast path requires same format too.

[Gerd Hoffmann]

Commit "555e72f spice: rework mirror allocation, add no-resize fast path"
adds a fast path for surface switches which does't go through the full
primary surface destroy and re-recreation in case the new surface is
identical to the old one (page-flip).  It checks the size only though,
but the format must be identical too.  This patch adds the format check.

Commit "0002a51 ui/spice: Support shared surface for most pixman
formats" increases the chance to actually trigger this.

https://bugzilla.redhat.com/show_bug.cgi?id=1247479

Signed-off-by: Gerd Hoffmann 
---
 ui/spice-display.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/ui/spice-display.c b/ui/spice-display.c
index 77c8cba..0489131 100644
--- a/ui/spice-display.c
+++ b/ui/spice-display.c
@@ -408,7 +408,8 @@ void qemu_spice_display_switch(SimpleSpiceDisplay *ssd,
 
 if (surface && ssd->surface &&
 surface_width(surface) == pixman_image_get_width(ssd->surface) &&
-surface_height(surface) == pixman_image_get_height(ssd->surface)) {
+surface_height(surface) == pixman_image_get_height(ssd->surface) &&
+surface_format(surface) == pixman_image_get_format(ssd->surface)) {
 /* no-resize fast path: just swap backing store */
 dprint(1, "%s/%d: fast (%dx%d)\n", __func__, ssd->qxl.id,
surface_width(surface), surface_height(surface));
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH] hmp-commands-info.hx: fix drops hmp "info pci"

[Paolo Bonzini]

On 21/09/2015 07:58, Markus Armbruster wrote:
> Pavel Butsykin  writes:
> 
>> On 18.09.2015 19:26, Eric Blake wrote:
>>> On 09/18/2015 10:06 AM, Pavel Butsykin wrote:
 The hmp command "info pci" accidentally lost when moving from monitor.c

 Signed-off-by: Pavel Butsykin 
 ---
   hmp-commands-info.hx | 14 ++
   1 file changed, 14 insertions(+)
>>> Paolo's variant has a better commit message:
>>> https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg04893.html
>>>
>> Ok. I just had to respond to the situation)
> 
> Appreciated!  It's much better to have two fixes for a regression
> quickly than one late :)

Are you pulling one of the two?

Paolo

Re: [Qemu-devel] [Qemu-stable] Recent patches for 2.4

[Paolo Bonzini]

On 21/09/2015 09:40, Peter Lieven wrote:
>>>
>>>
>>> My backport (Linux only and without tests etc.) of the original fix
>>> is here:
>>> https://github.com/plieven/qemu/commit/0ddcdc62a85f705017df16421d769e82b70f9b37
>>>
>> Could you be missing edec47c?
> 
> Indeed this one fixed the deadlock with -S in cmdline.
> Are there other important fixes from the series?
> 
> I currently have these two:
>  main-loop: fix qemu_notify_event for aio_notify optimization
>  AioContext: fix broken ctx->dispatching optimization

The full list is

e4efd8a488d0a68b0af34d8ee245463df7c8bdf4 qemu-timer: initialize 
"timers_done_ev" to set
12d69ac03b45156356b240424623719f15d8143e tests: remove irrelevant assertions 
from test-aio
6493c975af75be5b8d9ade954239bdf5492b7911 aio-win32: reorganize polling loop
eabc977973103527bbb8fed69c91cfaa6691f8ab AioContext: fix broken 
ctx->dispatching optimization
21a03d17f2edb1e63f7137d97ba355cc6f19d79f AioContext: fix broken placement of 
event_notifier_test_and_clear
05e514b1d4d5bd4209e2c8bbc76ff05c85a235f3 AioContext: optimize clearing the 
EventNotifier
edec47cfef96209987cb7922286cb384916aae02 main-loop: fix qemu_notify_event for 
aio_notify optimization
fed105e2756dde98efa5e80baca02ae516dd1e51 virtio-blk-dataplane: delete bottom 
half before the AioContext is freed
a076972a4d36381d610a854f0c336507650a1d34 AioContext: avoid leaking BHs on 
cleanup
ca96ac44dcd290566090b2435bc828fded356ad9 AioContext: force event loop iteration 
using BH

Paolo

[Qemu-devel] Ping: [PATCH v2 0/2] PCI-e device multi-function hot-add support

[Cao jin]

On 09/16/2015 10:02 AM, Cao jin wrote:

Support PCI-e device hot-add multi-function via device_add, just ensure
add the function 0 is added last. While allow user to roll back in the
middle via device_del, in case user regret.

According to Alex`s comment, v2 has following update:
1. patch 1/2 remove if() which seems breaks the assumptions elsewhere
2. patch 2/2 change device_del process: qemu fake the device detach condition,
make OS trigger the action

Cao jin (2):
   PCI-e device multi-function hot-add support
   PCI-e device multi-function hot-add support

  hw/pci/pcie.c | 36 +++-
  1 file changed, 27 insertions(+), 9 deletions(-)



--
Yours Sincerely,

Cao Jin

Re: [Qemu-devel] [Qemu-stable] Recent patches for 2.4

[Peter Lieven]

Am 21.09.2015 um 11:41 schrieb Paolo Bonzini:


On 21/09/2015 09:40, Peter Lieven wrote:


My backport (Linux only and without tests etc.) of the original fix
is here:
https://github.com/plieven/qemu/commit/0ddcdc62a85f705017df16421d769e82b70f9b37


Could you be missing edec47c?

Indeed this one fixed the deadlock with -S in cmdline.
Are there other important fixes from the series?

I currently have these two:
  main-loop: fix qemu_notify_event for aio_notify optimization
  AioContext: fix broken ctx->dispatching optimization

The full list is

e4efd8a488d0a68b0af34d8ee245463df7c8bdf4 qemu-timer: initialize 
"timers_done_ev" to set
12d69ac03b45156356b240424623719f15d8143e tests: remove irrelevant assertions 
from test-aio
6493c975af75be5b8d9ade954239bdf5492b7911 aio-win32: reorganize polling loop
eabc977973103527bbb8fed69c91cfaa6691f8ab AioContext: fix broken 
ctx->dispatching optimization
21a03d17f2edb1e63f7137d97ba355cc6f19d79f AioContext: fix broken placement of 
event_notifier_test_and_clear
05e514b1d4d5bd4209e2c8bbc76ff05c85a235f3 AioContext: optimize clearing the 
EventNotifier
edec47cfef96209987cb7922286cb384916aae02 main-loop: fix qemu_notify_event for 
aio_notify optimization
fed105e2756dde98efa5e80baca02ae516dd1e51 virtio-blk-dataplane: delete bottom 
half before the AioContext is freed
a076972a4d36381d610a854f0c336507650a1d34 AioContext: avoid leaking BHs on 
cleanup
ca96ac44dcd290566090b2435bc828fded356ad9 AioContext: force event loop iteration 
using BH

Paolo


thanks, I will have a look at them. If I understand correctly I should consider 
backporting all?

Peter

Re: [Qemu-devel] [PATCH] spapr: generate DT node names

[Thomas Huth]

On 19/09/15 00:06, Laurent Vivier wrote:
> When DT node names for PCI devices are generated by SLOF,
> they are generated according to the type of the device
> (for instance, ethernet for virtio-net-pci device).
> 
> Node name for hotplugged devices is generated by QEMU.
> This patch adds the mechanic to QEMU to create the node
> name according to the device type too.
> 
> The data structure has been roughly copied from OpenBIOS/OpenHackware,
> node names from SLOF.

Note that many names are mandated by the "PCI Bus Binding to:
IEEE Std 1275-1994" specification, e.g. available here:

 http://www.o3one.org/hwdocs/openfirmware/pci_supplement_2_1.pdf

See "Table 1: Generic Names based upon Class Code".

AFAIK the SLOF names are based on that spec, so most names should
hopefully be fine already.

[...]
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index a2feb4c..ca3a0ba 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -944,6 +944,487 @@ static void populate_resource_props(PCIDevice *d, 
> ResourceProps *rp)
>  rp->assigned_len = assigned_idx * sizeof(ResourceFields);
[...]
> +
> +typedef struct pci_class_t pci_class_t;
> +typedef struct pci_subclass_t pci_subclass_t;
> +typedef struct pci_iface_t pci_iface_t;
> +
> +struct pci_iface_t {
> +uint8_t iface;
> +const char *name;
> +};
> +
> +struct pci_subclass_t {
> +uint8_t subclass;
> +const char *name;
> +const pci_iface_t *iface;
> +};
> +
> +struct pci_class_t {
> +const char *name;
> +const pci_subclass_t *subc;
> +};

According to the QEMU CODING_STYLEs, "Structured
type names are in CamelCase", and AFAIK not using a "_t" suffix.

> +static const pci_subclass_t undef_subclass[] = {
> +{ 0x01, "display", NULL },
> +{ 0xFF, NULL, NULL, NULL },
> +};
> +
> +static const pci_subclass_t mass_subclass[] = {
> +{ PCI_SUBCLASS_STORAGE_SCSI, "scsi", NULL },
> +{ PCI_SUBCLASS_STORAGE_IDE, "ide", NULL },
> +{ PCI_SUBCLASS_STORAGE_FLOPPY, "fdc", NULL },
> +{ PCI_SUBCLASS_STORAGE_IPI, "ipi", NULL },
> +{ PCI_SUBCLASS_STORAGE_RAID, "raid", NULL },
> +{ PCI_SUBCLASS_STORAGE_ATA, "ata", NULL },
> +{ PCI_SUBCLASS_STORAGE_SATA, "sata", NULL },
> +{ PCI_SUBCLASS_STORAGE_SAS, "sas", NULL },
> +{ PCI_SUBCLASS_STORAGE_OTHER, "mass-storage", NULL },
> +{ 0xFF, NULL, NULL },
> +};
> +
> +static const pci_subclass_t net_subclass[] = {
> +{ PCI_SUBCLASS_NETWORK_ETHERNET, "ethernet", NULL },
> +{ PCI_SUBCLASS_NETWORK_TOKEN_RING, "token-ring", NULL },
> +{ PCI_SUBCLASS_NETWORK_FDDI, "fddi", NULL },
> +{ PCI_SUBCLASS_NETWORK_ATM, "atm", NULL },
> +{ PCI_SUBCLASS_NETWORK_ISDN, "isdn", NULL },
> +{ PCI_SUBCLASS_NETWORK_WORDFIP, "worldfip", NULL },
> +{ PCI_SUBCLASS_NETWORK_PICMG214, "picmg", NULL },
> +{ PCI_SUBCLASS_NETWORK_OTHER, "network", NULL },
> +{ 0xFF, NULL, NULL },
> +};
> +
> +static const struct pci_iface_t vga_iface[] = {
> +{ 0x00, "VGA", },
> +{ 0x01, "8514-compatible", },
> +{ 0xFF, NULL, },
> +};
> +
> +static const pci_subclass_t displ_subclass[] = {
> +{ PCI_SUBCLASS_DISPLAY_VGA, "vga", vga_iface },
> +{ PCI_SUBCLASS_DISPLAY_XGA, "xga", NULL },
> +{ PCI_SUBCLASS_DISPLAY_3D, "3d-controller", NULL },
> +{ PCI_SUBCLASS_DISPLAY_OTHER, "misc-display-controller", NULL },
> +{ 0xFF, NULL, NULL },
> +};

I somewhat dislike such long device node names like
"misc-display-controller" (since I sometimes have to type in them
manually at the SLOF prompt) ... and the PCI bus binding spec says that
the default name should be "display" here instead, so simply use
"display" instead of "misc-display-controller" ?

> +static const pci_subclass_t media_subclass[] = {
> +{ PCI_SUBCLASS_MULTIMEDIA_VIDEO, "video", NULL },
> +{ PCI_SUBCLASS_MULTIMEDIA_AUDIO, "sound", NULL },
> +{ PCI_SUBCLASS_MULTIMEDIA_PHONE, "telephony", NULL },
> +{ PCI_SUBCLASS_MULTIMEDIA_OTHER, "misc-multimedia-device", NULL },
> +{ 0xFF, NULL, NULL },
> +};

s/misc-multimedia-device/multimedia/ ?

"misc-" prefix and "-device" suffix IMHO do not make too much sense for
a device tree node.

> +static const pci_subclass_t mem_subclass[] = {
> +{ PCI_SUBCLASS_MEMORY_RAM, "memory", NULL },
> +{ PCI_SUBCLASS_MEMORY_FLASH, "flash", NULL },
> +{ 0xFF, NULL, NULL },
> +};
> +
> +
> +static const pci_subclass_t bridg_subclass[] = {
> +{ PCI_SUBCLASS_BRIDGE_HOST, "host", NULL },
> +{ PCI_SUBCLASS_BRIDGE_ISA, "isa", NULL },
> +{ PCI_SUBCLASS_BRIDGE_EISA, "eisa", NULL },
> +{ PCI_SUBCLASS_BRIDGE_MC, "mca", NULL },
> +{ PCI_SUBCLASS_BRIDGE_PCI, "pci", NULL },
> +{ PCI_SUBCLASS_BRIDGE_PCMCIA, "pcmcia", NULL },
> +{ PCI_SUBCLASS_BRIDGE_NUBUS, "nubus", NULL },
> +{ PCI_SUBCLASS_BRIDGE_CARDBUS, "cardbus", NULL },
> +{ PCI_SUBCLASS_BRIDGE_RACEWAY, "raceway", NULL },
> +{ PCI_SUBCLASS_BRIDGE_PCI_SEMITP, "semi-transparent-pci", NULL },
> +{ PCI_SUBCLASS_BRIDGE_IB_PCI, "infiniband", NULL },
> +

Re: [Qemu-devel] RFC: virtio-peer shared memory based peer communication device

[Jan Kiszka]

On 2015-09-18 23:11, Paolo Bonzini wrote:
> On 18/09/2015 18:29, Claudio Fontana wrote:
>>
>> this is a first RFC for virtio-peer 0.1, which is still very much a work in 
>> progress:
>>
>> https://github.com/hw-claudio/virtio-peer/wiki
>>
>> It is also available as PDF there, but the text is reproduced here for 
>> commenting:
>>
>> Peer shared memory communication device (virtio-peer)
> 
> Apart from the windows idea, how does virtio-peer compare to virtio-rpmsg?

rpmsg is a very specialized thing. It targets single AMP cores, assuming
that those have full access to the main memory. And it is also a
centralized approach where all message go through the main Linux
instance. I suspect we could cover that use case as well with generic
inter-vm shared memory device, but I didn't think about all details yet.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

[Qemu-devel] [PATCH v3 0/9] hmp command IO- and Local APIC dump state

[Denis V. Lunev]

Added the hmp command to query IO- and Local APIC registers state,
it can be very useful to identify problems related to the emulation devices.

(qemu) info lapic
dumping local APIC state for CPU 0

LVT0 0x00010700 active-hi edge  masked  ExtINT (vec 0)
LVT1 0x0400 active-hi edge  NMI
LVTPC0x0001 active-hi edge  masked  Fixed  (vec 0)
LVTERR   0x00fe active-hi edge  Fixed  (vec 254)
LVTTHMR  0x0001 active-hi edge  masked  Fixed  (vec 0)
LVTT 0x00ef active-hi edge one-shot Fixed  (vec 239)
TimerDCR=0x3 (divide by 16) initial_count = 62278
SPIV 0x01ff APIC enabled, focus=off, spurious vec 255
ICR  0x00fd physical edge de-assert no
ICR2 0x0001 cpu 1 (X2APIC ID)
ESR  0x
ISR
IRR  239

APR 0x00 TPR 0x00 DFR 0x0f LDR 0x00 PPR 0x00

(qemu) info ioapic
ioapic id=0x00 sel=0x2e (redir[15])
pin 0  0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 1  0x0031 dest=0 vec=49  active-hi edge fixed  physical
pin 2  0x0030 dest=0 vec=48  active-hi edge fixed  physical
pin 3  0x0033 dest=0 vec=51  active-hi edge fixed  physical
pin 4  0x00010034 dest=0 vec=52  active-hi edge  masked fixed  physical
pin 5  0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 6  0x0036 dest=0 vec=54  active-hi edge fixed  physical
pin 7  0x0037 dest=0 vec=55  active-hi edge fixed  physical
pin 8  0x0038 dest=0 vec=56  active-hi edge fixed  physical
pin 9  0x00018041 dest=0 vec=65  active-hi level masked fixed  physical
pin 10 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 11 0x03008081 dest=3 vec=129 active-hi levelfixed  physical
pin 12 0x003c dest=0 vec=60  active-hi edge fixed  physical
pin 13 0x003d dest=0 vec=61  active-hi edge fixed  physical
pin 14 0x003e dest=0 vec=62  active-hi edge fixed  physical
pin 15 0x003f dest=0 vec=63  active-hi edge fixed  physical
pin 16 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 17 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 18 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 19 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 20 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 21 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 22 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 23 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
IRR (none)

Changes from v1:
 - implementation of hmp commands moved to the target-i386 part
 - the cpu_dump_apic_local_state interface moved to the target-i386 part

Changes from v2:
 - lapic and ioapic format dump are fixed for more human readable view

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 

Pavel Butsykin (9):
  apic_internal.h: make some apic_get_* functions externally visible
  apic_internal.h: rename ESR_ILLEGAL_ADDRESS to
APIC_ESR_ILLEGAL_ADDRESS
  apic_internal.h: added more constants
  apic_internal.h: fix formatting and drop unused consts
  monitor: make monitor_fprintf and mon_get_cpu externally visible
  hmp: added local apic dump state
  ioapic_internal.h: added more constants
  hmp: added io apic dump state
  hmp: implemented io apic dump state for TCG

 disas.c   |  10 --
 hmp-commands-info.hx  |  32 +++
 hw/i386/kvm/ioapic.c  |  10 ++
 hw/intc/apic.c|  22 +
 hw/intc/ioapic.c  |  12 +++
 hw/intc/ioapic_common.c   |  37 
 include/hw/i386/apic_internal.h   |  96 ---
 include/hw/i386/ioapic_internal.h |   7 ++
 include/hw/i386/pc.h  |   5 +
 include/monitor/hmp-target.h  |   3 +
 include/monitor/monitor.h |   1 +
 monitor.c |   5 +-
 target-i386/cpu.h |   3 +
 target-i386/helper.c  | 190 ++
 target-i386/monitor.c |  17 
 15 files changed, 407 insertions(+), 43 deletions(-)

-- 
2.1.4

[Qemu-devel] [PATCH 9/9] hmp: implemented io apic dump state for TCG

[Denis V. Lunev]

From: Pavel Butsykin 

Added support emulator for the hmp command "info ioapic"

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 hw/intc/ioapic.c  | 12 
 include/hw/i386/pc.h  |  1 +
 target-i386/monitor.c |  2 ++
 3 files changed, 15 insertions(+)

diff --git a/hw/intc/ioapic.c b/hw/intc/ioapic.c
index b527932..8c3aeae 100644
--- a/hw/intc/ioapic.c
+++ b/hw/intc/ioapic.c
@@ -20,6 +20,7 @@
  * License along with this library; if not, see .
  */
 
+#include "monitor/monitor.h"
 #include "hw/hw.h"
 #include "hw/i386/pc.h"
 #include "hw/i386/ioapic.h"
@@ -137,6 +138,17 @@ void ioapic_eoi_broadcast(int vector)
 }
 }
 
+void ioapic_dump_state(Monitor *mon, const QDict *qdict)
+{
+int i;
+
+for (i = 0; i < MAX_IOAPICS; i++) {
+if (ioapics[i] != 0) {
+ioapic_print_redtbl(mon, ioapics[i]);
+}
+}
+}
+
 static uint64_t
 ioapic_mem_read(void *opaque, hwaddr addr, unsigned int size)
 {
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index 539cf64..7c9f3a5 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -126,6 +126,7 @@ void hmp_info_irq(Monitor *mon, const QDict *qdict);
 /* ioapic.c */
 
 void kvm_ioapic_dump_state(Monitor *mon, const QDict *qdict);
+void ioapic_dump_state(Monitor *mon, const QDict *qdict);
 
 /* Global System Interrupts */
 
diff --git a/target-i386/monitor.c b/target-i386/monitor.c
index 829fff9..aac6b1b 100644
--- a/target-i386/monitor.c
+++ b/target-i386/monitor.c
@@ -505,5 +505,7 @@ void hmp_info_io_apic(Monitor *mon, const QDict *qdict)
 {
 if (kvm_irqchip_in_kernel()) {
 kvm_ioapic_dump_state(mon, qdict);
+} else {
+ioapic_dump_state(mon, qdict);
 }
 }
-- 
2.1.4

[Qemu-devel] [PATCH 5/9] monitor: make monitor_fprintf and mon_get_cpu externally visible

[Denis V. Lunev]

From: Pavel Butsykin 

monitor_fprintf and mon_get_cpu will be used in the target-specific monitor,
so it is advisable to make it external.

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 disas.c  | 10 --
 include/monitor/hmp-target.h |  1 +
 include/monitor/monitor.h|  1 +
 monitor.c|  5 ++---
 4 files changed, 4 insertions(+), 13 deletions(-)

diff --git a/disas.c b/disas.c
index 0ae70c2..45878fa 100644
--- a/disas.c
+++ b/disas.c
@@ -392,16 +392,6 @@ monitor_read_memory (bfd_vma memaddr, bfd_byte *myaddr, 
int length,
 return 0;
 }
 
-static int GCC_FMT_ATTR(2, 3)
-monitor_fprintf(FILE *stream, const char *fmt, ...)
-{
-va_list ap;
-va_start(ap, fmt);
-monitor_vprintf((Monitor *)stream, fmt, ap);
-va_end(ap);
-return 0;
-}
-
 /* Disassembler for the monitor.
See target_disas for a description of flags. */
 void monitor_disas(Monitor *mon, CPUState *cpu,
diff --git a/include/monitor/hmp-target.h b/include/monitor/hmp-target.h
index 611541d..c64f523 100644
--- a/include/monitor/hmp-target.h
+++ b/include/monitor/hmp-target.h
@@ -37,6 +37,7 @@ struct MonitorDef {
 const MonitorDef *target_monitor_defs(void);
 
 CPUArchState *mon_get_cpu_env(void);
+CPUState *mon_get_cpu(void);
 
 void hmp_info_mem(Monitor *mon, const QDict *qdict);
 void hmp_info_tlb(Monitor *mon, const QDict *qdict);
diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 9aff47e..f95a384 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -34,6 +34,7 @@ int monitor_fd_param(Monitor *mon, const char *fdname, Error 
**errp);
 void monitor_vprintf(Monitor *mon, const char *fmt, va_list ap)
 GCC_FMT_ATTR(2, 0);
 void monitor_printf(Monitor *mon, const char *fmt, ...) GCC_FMT_ATTR(2, 3);
+int monitor_fprintf(FILE *stream, const char *fmt, ...) GCC_FMT_ATTR(2, 3);
 void monitor_flush(Monitor *mon);
 int monitor_set_cpu(int cpu_index);
 int monitor_get_cpu_index(void);
diff --git a/monitor.c b/monitor.c
index 1f43263..d303579 100644
--- a/monitor.c
+++ b/monitor.c
@@ -371,8 +371,7 @@ void monitor_printf(Monitor *mon, const char *fmt, ...)
 va_end(ap);
 }
 
-static int GCC_FMT_ATTR(2, 3) monitor_fprintf(FILE *stream,
-  const char *fmt, ...)
+int monitor_fprintf(FILE *stream, const char *fmt, ...)
 {
 va_list ap;
 va_start(ap, fmt);
@@ -941,7 +940,7 @@ int monitor_set_cpu(int cpu_index)
 return 0;
 }
 
-static CPUState *mon_get_cpu(void)
+CPUState *mon_get_cpu(void)
 {
 if (!cur_mon->mon_cpu) {
 monitor_set_cpu(0);
-- 
2.1.4

[Qemu-devel] [PATCH 3/9] apic_internal.h: added more constants

[Denis V. Lunev]

From: Pavel Butsykin 

These constants are needed for optimal access to
bit fields local apic registers without magic numbers.

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 include/hw/i386/apic_internal.h | 58 +
 1 file changed, 58 insertions(+)

diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index 188131c..a1db16e 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -50,14 +50,72 @@
 #define APIC_TRIGGER_EDGE   0
 #define APIC_TRIGGER_LEVEL  1
 
+#define APIC_VECTOR_MASK0xff
+#define APIC_DCR_MASK   0xf
+
+#define APIC_LVT_TIMER_SHIFT17
+#define APIC_LVT_MASKED_SHIFT   16
+#define APIC_LVT_LEVEL_TRIGGER_SHIFT15
+#define APIC_LVT_REMOTE_IRR_SHIFT   14
+#define APIC_LVT_INT_POLARITY_SHIFT 13
+#define APIC_LVT_DELIV_STS_SHIFT12
+#define APIC_LVT_DELIV_MOD_SHIFT8
+
+#define APIC_LVT_TIMER_TSCDEADLINE  (2 << APIC_LVT_TIMER_SHIFT)
 #define APIC_LVT_TIMER_PERIODIC (1<<17)
 #define APIC_LVT_MASKED (1<<16)
 #define APIC_LVT_LEVEL_TRIGGER  (1<<15)
 #define APIC_LVT_REMOTE_IRR (1<<14)
 #define APIC_INPUT_POLARITY (1<<13)
 #define APIC_SEND_PENDING   (1<<12)
+#define APIC_LVT_INT_POLARITY   (1 << APIC_LVT_INT_POLARITY_SHIFT)
+#define APIC_LVT_DELIV_STS  (1 << APIC_LVT_DELIV_STS_SHIFT)
+#define APIC_LVT_DELIV_MOD  (7 << APIC_LVT_DELIV_MOD_SHIFT)
+
+#define APIC_ESR_ILL_ADDRESS_SHIFT  7
+#define APIC_ESR_RECV_ILL_VECT_SHIFT6
+#define APIC_ESR_SEND_ILL_VECT_SHIFT5
+#define APIC_ESR_RECV_ACCEPT_SHIFT  3
+#define APIC_ESR_SEND_ACCEPT_SHIFT  2
+#define APIC_ESR_RECV_CHECK_SUM_SHIFT   1
 
 #define APIC_ESR_ILLEGAL_ADDRESS(1 << 7)
+#define APIC_ESR_RECV_ILLEGAL_VECT  (1 << APIC_ESR_RECV_ILL_VECT_SHIFT)
+#define APIC_ESR_SEND_ILLEGAL_VECT  (1 << APIC_ESR_SEND_ILL_VECT_SHIFT)
+#define APIC_ESR_RECV_ACCEPT(1 << APIC_ESR_RECV_ACCEPT_SHIFT)
+#define APIC_ESR_SEND_ACCEPT(1 << APIC_ESR_SEND_ACCEPT_SHIFT)
+#define APIC_ESR_RECV_CHECK_SUM (1 << APIC_ESR_RECV_CHECK_SUM_SHIFT)
+#define APIC_ESR_SEND_CHECK_SUM 1
+
+#define APIC_ICR_DEST_SHIFT 24
+#define APIC_ICR_DEST_SHORT_SHIFT   18
+#define APIC_ICR_TRIGGER_MOD_SHIFT  15
+#define APIC_ICR_LEVEL_SHIFT14
+#define APIC_ICR_DELIV_STS_SHIFT12
+#define APIC_ICR_DEST_MOD_SHIFT 11
+#define APIC_ICR_DELIV_MOD_SHIFT8
+
+#define APIC_ICR_DEST_SHORT (3 << APIC_ICR_DEST_SHORT_SHIFT)
+#define APIC_ICR_TRIGGER_MOD(1 << APIC_ICR_TRIGGER_MOD_SHIFT)
+#define APIC_ICR_LEVEL  (1 << APIC_ICR_LEVEL_SHIFT)
+#define APIC_ICR_DELIV_STS  (1 << APIC_ICR_DELIV_STS_SHIFT)
+#define APIC_ICR_DEST_MOD   (1 << APIC_ICR_DEST_MOD_SHIFT)
+#define APIC_ICR_DELIV_MOD  (7 << APIC_ICR_DELIV_MOD_SHIFT)
+
+#define APIC_PR_CLASS_SHIFT 4
+#define APIC_PR_SUB_CLASS   0xf
+
+#define APIC_LOGDEST_XAPIC_SHIFT4
+#define APIC_LOGDEST_XAPIC_ID   0xf
+
+#define APIC_LOGDEST_X2APIC_SHIFT   16
+#define APIC_LOGDEST_X2APIC_ID  0x
+
+#define APIC_SPURIO_FOCUS_SHIFT 9
+#define APIC_SPURIO_ENABLED_SHIFT   8
+
+#define APIC_SPURIO_FOCUS   (1 << APIC_SPURIO_FOCUS_SHIFT)
+#define APIC_SPURIO_ENABLED (1 << APIC_SPURIO_ENABLED_SHIFT)
 
 #define APIC_SV_DIRECTED_IO (1<<12)
 #define APIC_SV_ENABLE  (1<<8)
-- 
2.1.4

[Qemu-devel] [PATCH 4/9] apic_internal.h: fix formatting and drop unused consts

[Denis V. Lunev]

From: Pavel Butsykin 

Fix formatting of local apic definitions and drop unused constant
APIC_INPUT_POLARITY, APIC_SEND_PENDING. Magic numbers in shifts are
replaced with constants defined just above.

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 include/hw/i386/apic_internal.h | 16 +++-
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index a1db16e..7813396 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -62,12 +62,10 @@
 #define APIC_LVT_DELIV_MOD_SHIFT8
 
 #define APIC_LVT_TIMER_TSCDEADLINE  (2 << APIC_LVT_TIMER_SHIFT)
-#define APIC_LVT_TIMER_PERIODIC (1<<17)
-#define APIC_LVT_MASKED (1<<16)
-#define APIC_LVT_LEVEL_TRIGGER  (1<<15)
-#define APIC_LVT_REMOTE_IRR (1<<14)
-#define APIC_INPUT_POLARITY (1<<13)
-#define APIC_SEND_PENDING   (1<<12)
+#define APIC_LVT_TIMER_PERIODIC (1 << APIC_LVT_TIMER_SHIFT)
+#define APIC_LVT_MASKED (1 << APIC_LVT_MASKED_SHIFT)
+#define APIC_LVT_LEVEL_TRIGGER  (1 << APIC_LVT_LEVEL_TRIGGER_SHIFT)
+#define APIC_LVT_REMOTE_IRR (1 << APIC_LVT_REMOTE_IRR_SHIFT)
 #define APIC_LVT_INT_POLARITY   (1 << APIC_LVT_INT_POLARITY_SHIFT)
 #define APIC_LVT_DELIV_STS  (1 << APIC_LVT_DELIV_STS_SHIFT)
 #define APIC_LVT_DELIV_MOD  (7 << APIC_LVT_DELIV_MOD_SHIFT)
@@ -79,7 +77,7 @@
 #define APIC_ESR_SEND_ACCEPT_SHIFT  2
 #define APIC_ESR_RECV_CHECK_SUM_SHIFT   1
 
-#define APIC_ESR_ILLEGAL_ADDRESS(1 << 7)
+#define APIC_ESR_ILLEGAL_ADDRESS(1 << APIC_ESR_ILL_ADDRESS_SHIFT)
 #define APIC_ESR_RECV_ILLEGAL_VECT  (1 << APIC_ESR_RECV_ILL_VECT_SHIFT)
 #define APIC_ESR_SEND_ILLEGAL_VECT  (1 << APIC_ESR_SEND_ILL_VECT_SHIFT)
 #define APIC_ESR_RECV_ACCEPT(1 << APIC_ESR_RECV_ACCEPT_SHIFT)
@@ -117,8 +115,8 @@
 #define APIC_SPURIO_FOCUS   (1 << APIC_SPURIO_FOCUS_SHIFT)
 #define APIC_SPURIO_ENABLED (1 << APIC_SPURIO_ENABLED_SHIFT)
 
-#define APIC_SV_DIRECTED_IO (1<<12)
-#define APIC_SV_ENABLE  (1<<8)
+#define APIC_SV_DIRECTED_IO (1 << 12)
+#define APIC_SV_ENABLE  (1 << 8)
 
 #define VAPIC_ENABLE_BIT0
 #define VAPIC_ENABLE_MASK   (1 << VAPIC_ENABLE_BIT)
-- 
2.1.4

[Qemu-devel] [PATCH 1/9] apic_internal.h: make some apic_get_* functions externally visible

[Denis V. Lunev]

From: Pavel Butsykin 

Move apic_get_bit(), apic_set_bit() to apic_internal.h, make the
apic_get_ppr symbol external. It's necessary to work with isr, tmr,
irr and ppr outside hw/intc/apic.c

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 hw/intc/apic.c  | 18 +-
 include/hw/i386/apic_internal.h | 18 ++
 2 files changed, 19 insertions(+), 17 deletions(-)

diff --git a/hw/intc/apic.c b/hw/intc/apic.c
index 77b639c..52ac8b2 100644
--- a/hw/intc/apic.c
+++ b/hw/intc/apic.c
@@ -51,14 +51,6 @@ static int apic_ffs_bit(uint32_t value)
 return ctz32(value);
 }
 
-static inline void apic_set_bit(uint32_t *tab, int index)
-{
-int i, mask;
-i = index >> 5;
-mask = 1 << (index & 0x1f);
-tab[i] |= mask;
-}
-
 static inline void apic_reset_bit(uint32_t *tab, int index)
 {
 int i, mask;
@@ -67,14 +59,6 @@ static inline void apic_reset_bit(uint32_t *tab, int index)
 tab[i] &= ~mask;
 }
 
-static inline int apic_get_bit(uint32_t *tab, int index)
-{
-int i, mask;
-i = index >> 5;
-mask = 1 << (index & 0x1f);
-return !!(tab[i] & mask);
-}
-
 /* return -1 if no bit is set */
 static int get_highest_priority_int(uint32_t *tab)
 {
@@ -318,7 +302,7 @@ static uint8_t apic_get_tpr(APICCommonState *s)
 return s->tpr >> 4;
 }
 
-static int apic_get_ppr(APICCommonState *s)
+int apic_get_ppr(APICCommonState *s)
 {
 int tpr, isrv, ppr;
 
diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index 26632ac..5a213bc 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -147,4 +147,22 @@ void apic_enable_vapic(DeviceState *d, hwaddr paddr);
 void vapic_report_tpr_access(DeviceState *dev, CPUState *cpu, target_ulong ip,
  TPRAccess access);
 
+int apic_get_ppr(APICCommonState *s);
+
+static inline void apic_set_bit(uint32_t *tab, int index)
+{
+int i, mask;
+i = index >> 5;
+mask = 1 << (index & 0x1f);
+tab[i] |= mask;
+}
+
+static inline int apic_get_bit(uint32_t *tab, int index)
+{
+int i, mask;
+i = index >> 5;
+mask = 1 << (index & 0x1f);
+return !!(tab[i] & mask);
+}
+
 #endif /* !QEMU_APIC_INTERNAL_H */
-- 
2.1.4

[Qemu-devel] [PATCH 2/9] apic_internal.h: rename ESR_ILLEGAL_ADDRESS to APIC_ESR_ILLEGAL_ADDRESS

[Denis V. Lunev]

From: Pavel Butsykin 

Added prefix APIC_ for determining the constant of a particular subsystem,
improve the overall readability and match other constant names.

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 hw/intc/apic.c  | 4 ++--
 include/hw/i386/apic_internal.h | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/hw/intc/apic.c b/hw/intc/apic.c
index 52ac8b2..0a840b8 100644
--- a/hw/intc/apic.c
+++ b/hw/intc/apic.c
@@ -723,7 +723,7 @@ static uint32_t apic_mem_readl(void *opaque, hwaddr addr)
 val = s->divide_conf;
 break;
 default:
-s->esr |= ESR_ILLEGAL_ADDRESS;
+s->esr |= APIC_ESR_ILLEGAL_ADDRESS;
 val = 0;
 break;
 }
@@ -836,7 +836,7 @@ static void apic_mem_writel(void *opaque, hwaddr addr, 
uint32_t val)
 }
 break;
 default:
-s->esr |= ESR_ILLEGAL_ADDRESS;
+s->esr |= APIC_ESR_ILLEGAL_ADDRESS;
 break;
 }
 }
diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index 5a213bc..188131c 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -57,7 +57,7 @@
 #define APIC_INPUT_POLARITY (1<<13)
 #define APIC_SEND_PENDING   (1<<12)
 
-#define ESR_ILLEGAL_ADDRESS (1 << 7)
+#define APIC_ESR_ILLEGAL_ADDRESS(1 << 7)
 
 #define APIC_SV_DIRECTED_IO (1<<12)
 #define APIC_SV_ENABLE  (1<<8)
-- 
2.1.4

[Qemu-devel] [PATCH 8/9] hmp: added io apic dump state

[Denis V. Lunev]

From: Pavel Butsykin 

Added the hmp command to query io apic state, may be usefull after guest
crashes to understand IRQ routing in guest.

Implementation is only for kvm here. The dump will look like
(qemu) info ioapic
ioapic id=0x00 sel=0x2e (redir[15])
pin 0  0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 1  0x0031 dest=0 vec=49  active-hi edge fixed  physical
...
pin 23 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
IRR (none)

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 hmp-commands-info.hx  | 16 
 hw/i386/kvm/ioapic.c  | 10 ++
 hw/intc/ioapic_common.c   | 37 +
 include/hw/i386/ioapic_internal.h |  2 ++
 include/hw/i386/pc.h  |  4 
 include/monitor/hmp-target.h  |  1 +
 target-i386/monitor.c |  9 +
 7 files changed, 79 insertions(+)

diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index dba7839..b346fd1 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -128,6 +128,22 @@ STEXI
 Show local APIC state
 ETEXI
 
+#if defined(TARGET_I386)
+{
+.name   = "ioapic",
+.args_type  = "",
+.params = "",
+.help   = "show io apic state",
+.mhandler.cmd = hmp_info_io_apic,
+},
+#endif
+
+STEXI
+@item info ioapic
+@findex ioapic
+Show io APIC state
+ETEXI
+
 {
 .name   = "cpus",
 .args_type  = "",
diff --git a/hw/i386/kvm/ioapic.c b/hw/i386/kvm/ioapic.c
index d2a6c4c..b7390ca 100644
--- a/hw/i386/kvm/ioapic.c
+++ b/hw/i386/kvm/ioapic.c
@@ -10,6 +10,7 @@
  * See the COPYING file in the top-level directory.
  */
 
+#include "monitor/monitor.h"
 #include "hw/i386/pc.h"
 #include "hw/i386/ioapic_internal.h"
 #include "hw/i386/apic_internal.h"
@@ -110,6 +111,15 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
 }
 }
 
+void kvm_ioapic_dump_state(Monitor *mon, const QDict *qdict)
+{
+IOAPICCommonState s;
+
+kvm_ioapic_get(&s);
+
+ioapic_print_redtbl(mon, &s);
+}
+
 static void kvm_ioapic_reset(DeviceState *dev)
 {
 IOAPICCommonState *s = IOAPIC_COMMON(dev);
diff --git a/hw/intc/ioapic_common.c b/hw/intc/ioapic_common.c
index 8b7d118..762cfc9 100644
--- a/hw/intc/ioapic_common.c
+++ b/hw/intc/ioapic_common.c
@@ -19,6 +19,7 @@
  * License along with this library; if not, see .
  */
 
+#include "monitor/monitor.h"
 #include "hw/i386/ioapic.h"
 #include "hw/i386/ioapic_internal.h"
 #include "hw/sysbus.h"
@@ -31,6 +32,42 @@
  */
 int ioapic_no;
 
+void ioapic_print_redtbl(Monitor *mon, IOAPICCommonState *s)
+{
+static const char *delm_str[] = {
+"fixed", "lowest", "SMI", "...", "NMI", "INIT", "...", "extINT"};
+int i;
+
+monitor_printf(mon, "ioapic id=0x%02x sel=0x%02x", s->id, s->ioregsel);
+if (s->ioregsel) {
+monitor_printf(mon, " (redir[%u])\n",
+   (s->ioregsel - IOAPIC_REG_REDTBL_BASE) >> 1);
+} else {
+monitor_printf(mon, "\n");
+}
+for (i = 0; i < IOAPIC_NUM_PINS; i++) {
+uint64_t entry = s->ioredtbl[i];
+uint32_t delm = (uint32_t)((entry & IOAPIC_LVT_DELIV_MODE) >>
+   IOAPIC_LVT_DELIV_MODE_SHIFT);
+monitor_printf(mon, "pin %-2u 0x%016jx dest=%jx vec=%-3lu "
+   "%s %-5s %-6s %-6s %s\n",
+   i, entry,
+   (entry >> IOAPIC_LVT_DEST_SHIFT) &
+(entry & IOAPIC_LVT_DEST_MODE ? 0xff : 0xf),
+   entry & IOAPIC_VECTOR_MASK,
+   entry & IOAPIC_LVT_POLARITY ? "active-lo" : "active-hi",
+   entry & IOAPIC_LVT_TRIGGER_MODE ? "level" : "edge",
+   entry & IOAPIC_LVT_MASKED ? "masked" : "",
+   delm_str[delm],
+   entry & IOAPIC_LVT_DEST_MODE ? "logical" : "physical");
+}
+if (s->irr == 0) {
+monitor_printf(mon, "IRR (none)\n");
+} else {
+monitor_printf(mon, "IRR %08x\n", s->irr);
+}
+}
+
 void ioapic_reset_common(DeviceState *dev)
 {
 IOAPICCommonState *s = IOAPIC_COMMON(dev);
diff --git a/include/hw/i386/ioapic_internal.h 
b/include/hw/i386/ioapic_internal.h
index 4f7764e..797ed47 100644
--- a/include/hw/i386/ioapic_internal.h
+++ b/include/hw/i386/ioapic_internal.h
@@ -105,4 +105,6 @@ struct IOAPICCommonState {
 
 void ioapic_reset_common(DeviceState *dev);
 
+void ioapic_print_redtbl(Monitor *mon, IOAPICCommonState *s);
+
 #endif /* !QEMU_IOAPIC_INTERNAL_H */
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index 3e002c9..539cf64 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -123,6 +123,10 @@ int pic_get_output(DeviceState *d);
 void hmp_info_pic(Monitor *mon, const QDict *qdict);
 void hmp_info_irq(Monitor *mon, cons

[Qemu-devel] [PATCH 7/9] ioapic_internal.h: added more constants

[Denis V. Lunev]

From: Pavel Butsykin 

Added the masks for easy  access to fields of the redirection table entry

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 include/hw/i386/ioapic_internal.h | 5 +
 1 file changed, 5 insertions(+)

diff --git a/include/hw/i386/ioapic_internal.h 
b/include/hw/i386/ioapic_internal.h
index 3be3352..4f7764e 100644
--- a/include/hw/i386/ioapic_internal.h
+++ b/include/hw/i386/ioapic_internal.h
@@ -40,7 +40,12 @@
 #define IOAPIC_LVT_DELIV_MODE_SHIFT 8
 
 #define IOAPIC_LVT_MASKED   (1 << IOAPIC_LVT_MASKED_SHIFT)
+#define IOAPIC_LVT_TRIGGER_MODE (1 << IOAPIC_LVT_TRIGGER_MODE_SHIFT)
 #define IOAPIC_LVT_REMOTE_IRR   (1 << IOAPIC_LVT_REMOTE_IRR_SHIFT)
+#define IOAPIC_LVT_POLARITY (1 << IOAPIC_LVT_POLARITY_SHIFT)
+#define IOAPIC_LVT_DELIV_STATUS (1 << IOAPIC_LVT_DELIV_STATUS_SHIFT)
+#define IOAPIC_LVT_DEST_MODE(1 << IOAPIC_LVT_DEST_MODE_SHIFT)
+#define IOAPIC_LVT_DELIV_MODE   (7 << IOAPIC_LVT_DELIV_MODE_SHIFT)
 
 #define IOAPIC_TRIGGER_EDGE 0
 #define IOAPIC_TRIGGER_LEVEL1
-- 
2.1.4

[Qemu-devel] [PATCH 6/9] hmp: added local apic dump state

[Denis V. Lunev]

From: Pavel Butsykin 

Added the hmp command to query local apic registers state, may be
usefull after guest crashes to understand IRQ routing in guest.

(qemu) info lapic
dumping local APIC state for CPU 0

LVT0 0x00010700 active-hi edge  masked  ExtINT (vec 0)
LVT1 0x0400 active-hi edge  NMI
LVTPC0x0001 active-hi edge  masked  Fixed  (vec 0)
LVTERR   0x00fe active-hi edge  Fixed  (vec 254)
LVTTHMR  0x0001 active-hi edge  masked  Fixed  (vec 0)
LVTT 0x00ef active-hi edge one-shot Fixed  (vec 239)
TimerDCR=0x3 (divide by 16) initial_count = 62278
SPIV 0x01ff APIC enabled, focus=off, spurious vec 255
ICR  0x00fd physical edge de-assert no
ICR2 0x0001 cpu 1 (X2APIC ID)
ESR  0x
ISR
IRR  239

APR 0x00 TPR 0x00 DFR 0x0f LDR 0x00 PPR 0x00

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 hmp-commands-info.hx |  16 
 include/monitor/hmp-target.h |   1 +
 target-i386/cpu.h|   3 +
 target-i386/helper.c | 190 +++
 target-i386/monitor.c|   6 ++
 5 files changed, 216 insertions(+)

diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index 9f5a158..dba7839 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -112,6 +112,22 @@ STEXI
 Show the cpu registers.
 ETEXI
 
+#if defined(TARGET_I386)
+{
+.name   = "lapic",
+.args_type  = "",
+.params = "",
+.help   = "show local apic state",
+.mhandler.cmd = hmp_info_local_apic,
+},
+#endif
+
+STEXI
+@item info lapic
+@findex lapic
+Show local APIC state
+ETEXI
+
 {
 .name   = "cpus",
 .args_type  = "",
diff --git a/include/monitor/hmp-target.h b/include/monitor/hmp-target.h
index c64f523..be50577 100644
--- a/include/monitor/hmp-target.h
+++ b/include/monitor/hmp-target.h
@@ -42,5 +42,6 @@ CPUState *mon_get_cpu(void);
 void hmp_info_mem(Monitor *mon, const QDict *qdict);
 void hmp_info_tlb(Monitor *mon, const QDict *qdict);
 void hmp_mce(Monitor *mon, const QDict *qdict);
+void hmp_info_local_apic(Monitor *mon, const QDict *qdict);
 
 #endif /* MONITOR_COMMON */
diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index 5231e8c..527eb99 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -1353,4 +1353,7 @@ void enable_compat_apic_id_mode(void);
 #define APIC_DEFAULT_ADDRESS 0xfee0
 #define APIC_SPACE_SIZE  0x10
 
+void x86_cpu_dump_local_apic_state(CPUState *cs, FILE *f,
+   fprintf_function cpu_fprintf, int flags);
+
 #endif /* CPU_I386_H */
diff --git a/target-i386/helper.c b/target-i386/helper.c
index 5480a96..ff9a0f2 100644
--- a/target-i386/helper.c
+++ b/target-i386/helper.c
@@ -23,6 +23,7 @@
 #ifndef CONFIG_USER_ONLY
 #include "sysemu/sysemu.h"
 #include "monitor/monitor.h"
+#include "hw/i386/apic_internal.h"
 #endif
 
 static void cpu_x86_version(CPUX86State *env, int *family, int *model)
@@ -177,6 +178,195 @@ done:
 cpu_fprintf(f, "\n");
 }
 
+#ifndef CONFIG_USER_ONLY
+
+/* ARRAY_SIZE check is not required because
+ * DeliveryMode(dm) has a size of 3 bit.
+ */
+static inline const char *dm2str(uint32_t dm)
+{
+static const char *str[] = {
+"Fixed",
+"...",
+"SMI",
+"...",
+"NMI",
+"INIT",
+"...",
+"ExtINT"
+};
+return str[dm];
+}
+
+static void dump_apic_lvt(FILE *f, fprintf_function cpu_fprintf,
+  const char *name, uint32_t lvt, bool is_timer)
+{
+uint32_t dm = (lvt & APIC_LVT_DELIV_MOD) >> APIC_LVT_DELIV_MOD_SHIFT;
+cpu_fprintf(f,
+"%s\t 0x%08x %s %-5s %-6s %-7s %-12s %-6s",
+name, lvt,
+lvt & APIC_LVT_INT_POLARITY ? "active-lo" : "active-hi",
+lvt & APIC_LVT_LEVEL_TRIGGER ? "level" : "edge",
+lvt & APIC_LVT_MASKED ? "masked" : "",
+lvt & APIC_LVT_DELIV_STS ? "pending" : "",
+!is_timer ?
+"" : lvt & APIC_LVT_TIMER_PERIODIC ?
+"periodic" : lvt & APIC_LVT_TIMER_TSCDEADLINE ?
+"tsc-deadline" : "one-shot",
+dm2str(dm));
+if (dm != APIC_DM_NMI) {
+cpu_fprintf(f, " (vec %u)\n", lvt & APIC_VECTOR_MASK);
+} else {
+cpu_fprintf(f, "\n");
+}
+}
+
+/* ARRAY_SIZE check is not required because
+ * destination shorthand has a size of 2 bit.
+ */
+static inline const char *shorthand2str(uint32_t shorthand)
+{
+const char *str[] = {
+"no", "self", "all-self", "all"
+};
+return str[shorthand];
+}
+
+static inline uint8_t divider_conf(uint32_t divide_conf)
+{
+uint8_t divide_val = ((divide_conf & 0x8) >> 1) | (divid

Re: [Qemu-devel] [FIRST-PATCH] updated the negation STREQ to STRNEQ

[Michal Privoznik]

[You've got the wrong list. This is a libvirt patch and as such it
should have been sent to the libvir-l...@redhat.com]

On 19.09.2015 16:59, Lasya wrote:
> Hi,
> This is my first patch in QEMU. I have solved a bite-sized bug. I have 
> updated the !STREQ with STRNEQ in the following files. Thanks.
> 
> ---
>  src/bhyve/bhyve_driver.c  |  2 +-
>  src/conf/network_conf.c   |  4 ++--
>  src/conf/nwfilter_conf.c  |  2 +-
>  src/conf/nwfilter_params.c|  2 +-
>  src/conf/storage_conf.c   |  2 +-
>  src/lxc/lxc_fuse.c|  4 ++--
>  src/openvz/openvz_driver.c|  2 +-
>  src/qemu/qemu_command.c   |  6 +++---
>  src/qemu/qemu_domain.c|  2 +-
>  src/qemu/qemu_hotplug.c   |  2 +-
>  src/security/security_manager.c   |  2 +-
>  src/security/security_selinux.c   | 12 ++--
>  src/storage/storage_backend_logical.c |  2 +-
>  src/util/virfile.c|  2 +-
>  src/util/virsystemd.c |  2 +-
>  src/vz/vz_driver.c|  2 +-
>  src/vz/vz_sdk.c   |  2 +-
>  src/xen/xend_internal.c   |  2 +-
>  src/xenconfig/xen_sxpr.c  |  2 +-
>  tests/commandtest.c   | 10 +-
>  tests/securityselinuxlabeltest.c  |  2 +-
>  tests/virauthconfigtest.c |  2 +-
>  tests/virbitmaptest.c | 12 ++--
>  tests/vircgrouptest.c |  2 +-
>  tests/virkeyfiletest.c|  8 
>  tests/virnetsockettest.c  |  2 +-
>  tests/virtypedparamtest.c |  2 +-
>  tests/viruritest.c| 16 
>  28 files changed, 56 insertions(+), 56 deletions(-)

Okay, you've got all the occurrences. However, the syntax-check rule is
missing. You may want to look at cfg.mk for some examples. syntax-check
is our make target that checks the code if it complies with our coding
standards. After this patch we want a rule that forbids !STREQ or !STRNEQ.

Otherwise looking very good. Looking forward to v2.

Michal

Re: [Qemu-devel] [FIRST-PATCH] updated the negation STREQ to STRNEQ

[Michal Privoznik]

On 21.09.2015 13:05, Michal Privoznik wrote:
> [You've got the wrong list. This is a libvirt patch and as such it
> should have been sent to the libvir-l...@redhat.com]
> 
> On 19.09.2015 16:59, Lasya wrote:
>> Hi,
>> This is my first patch in QEMU. I have solved a bite-sized bug. I have 
>> updated the !STREQ with STRNEQ in the following files. Thanks.

Oh, and the commit message is rather unhelpful. I mean, we tend to
explain in there why is the commit needed, what does it fix, and so on.
If you look at git history you'll get the idea.

>>
>> ---
>>  src/bhyve/bhyve_driver.c  |  2 +-
>>  src/conf/network_conf.c   |  4 ++--
>>  src/conf/nwfilter_conf.c  |  2 +-
>>  src/conf/nwfilter_params.c|  2 +-
>>  src/conf/storage_conf.c   |  2 +-
>>  src/lxc/lxc_fuse.c|  4 ++--
>>  src/openvz/openvz_driver.c|  2 +-
>>  src/qemu/qemu_command.c   |  6 +++---
>>  src/qemu/qemu_domain.c|  2 +-
>>  src/qemu/qemu_hotplug.c   |  2 +-
>>  src/security/security_manager.c   |  2 +-
>>  src/security/security_selinux.c   | 12 ++--
>>  src/storage/storage_backend_logical.c |  2 +-
>>  src/util/virfile.c|  2 +-
>>  src/util/virsystemd.c |  2 +-
>>  src/vz/vz_driver.c|  2 +-
>>  src/vz/vz_sdk.c   |  2 +-
>>  src/xen/xend_internal.c   |  2 +-
>>  src/xenconfig/xen_sxpr.c  |  2 +-
>>  tests/commandtest.c   | 10 +-
>>  tests/securityselinuxlabeltest.c  |  2 +-
>>  tests/virauthconfigtest.c |  2 +-
>>  tests/virbitmaptest.c | 12 ++--
>>  tests/vircgrouptest.c |  2 +-
>>  tests/virkeyfiletest.c|  8 
>>  tests/virnetsockettest.c  |  2 +-
>>  tests/virtypedparamtest.c |  2 +-
>>  tests/viruritest.c| 16 
>>  28 files changed, 56 insertions(+), 56 deletions(-)
> 
> Okay, you've got all the occurrences. However, the syntax-check rule is
> missing. You may want to look at cfg.mk for some examples. syntax-check
> is our make target that checks the code if it complies with our coding
> standards. After this patch we want a rule that forbids !STREQ or !STRNEQ.
> 
> Otherwise looking very good. Looking forward to v2.

Michal

[Qemu-devel] [PATCH v3 1/1] block/gluster: add support for multiple gluster backup volfile servers

[Prasanna Kumar Kalever]

This patch adds a way to specify multiple backup volfile servers to the gluster
block backend of QEMU with tcp|rdma transport types and their port numbers.

Problem:

Currenly VM Image on gluster volume is specified like this:

file=gluster[+tcp]://server1[:port]/testvol/a.img

Assuming we have have three servers in trustred pool with replica 3 volume
in action and unfortunately server1 (mentioned in the command above) went down
for some reason, since the volume is replica 3 we now have other 2 servers
active from which we can boot the VM.

But currently there is no mechanism to pass the other 2 gluster server
addresses to qemu.

Solution:

New way of specifying VM Image on gluster volume with backup volfile servers:
(We still support old syntax to maintain backward compatibility)

Basic command line syntax looks like:

Pattern I:
 -drive driver=gluster,
volname=testvol,file.image-path=/path/a.raw,
backup-volfile-servers.0.server=1.2.3.4,
   [backup-volfile-servers.0.port=24007,]
   [backup-volfile-servers.0.transport=tcp,]
backup-volfile-servers.1.server=5.6.7.8,
   [backup-volfile-servers.1.port=24008,]
   [backup-volfile-servers.1.transport=rdma,] ...

Pattern II:
 'json:{"driver":"qcow2","file":{"driver":"gluster",
   "volname":"testvol","image-path":"/path/a.qcow2",
   "backup-volfile-servers":[{tuple0},{tuple1}, ...{tupleN}]}}'

   driver   => 'gluster' (protocol name)
   volname  => name of gluster volume where our VM image resides
   image-path   => is the absolute path of image in gluster volume

  {tuple}   => {"server":"1.2.3.4"[,"port":"24007","transport":"tcp"]}

   server   => server address (hostname/ipv4/ipv6 addresses)
   port => port number on which glusterd is listening. (default 24007)
   tranport => transport type used to connect to gluster management daemon,
it can be tcp|rdma (default 'tcp')

Examples:
1.
 -drive driver=qcow2,file.driver=gluster,
file.volname=testvol,file.image-path=/path/a.qcow2,
file.backup-volfile-servers.0.server=1.2.3.4,
file.backup-volfile-servers.0.port=24007,
file.backup-volfile-servers.0.transport=tcp,
file.backup-volfile-servers.1.server=5.6.7.8,
file.backup-volfile-servers.1.port=24008,
file.backup-volfile-servers.1.transport=rdma
2.
 'json:{"driver":"qcow2","file":{"driver":"gluster","volname":"testvol",
 "image-path":"/path/a.qcow2","backup-volfile-servers":
 [{"server":"1.2.3.4","port":"24007","transport":"tcp"},
  {"server":"4.5.6.7","port":"24008","transport":"rdma"},] } }'

This patch gives a mechanism to provide all the server addresses which are in
replica set, so in case server1 is down VM can still boot from any of the
active servers.

This is equivalent to the backup-volfile-servers option supported by
mount.glusterfs (FUSE way of mounting gluster volume)

This patch depends on a recent fix in libgfapi raised as part of this work:
http://review.gluster.org/#/c/12114/ (not merged yet)

Credits: Sincere thanks to Kevin Wolf  and
"Deepak C Shetty"  for inputs and all their support

v1:
multiple server addresses but common port number and transport type
pattern: URI syntax with query (?) delimitor
syntax:
file=gluster[+transport-type]://server1:24007/testvol/a.img\
 ?backup-volfile-servers=server2&backup-volfile-servers=server3

v2:
multiple server addresses each have their own port number, but all use
common transport type
pattern: URI syntax  with query (?) delimiter
syntax:
file=gluster[+transport-type]://[server[:port]]/testvol/a.img\
 [?backup-volfile-servers=server1[:port]\
  &backup-volfile-servers=server2[:port]]

v3:
multiple server addresses each have their own port number and transport type
pattern: changed to json
syntax:
'json:{"driver":"qcow2","file":{"driver":"gluster","volname":"testvol",
   "image-path":"/path/a.qcow2","backup-volfile-servers":
 [{"server":"1.2.3.4","port":"24007","transport":"tcp"},
  {"server":"4.5.6.7","port":"24008","transport":"rdma"},] } }'

Signed-off-by: Prasanna Kumar Kalever 
---
 block/gluster.c  | 412 +--
 qapi/block-core.json |  40 -
 2 files changed, 405 insertions(+), 47 deletions(-)

diff --git a/block/gluster.c b/block/gluster.c
index 1eb3a8c..48ae62e 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -11,6 +11,17 @@
 #include "block/block_int.h"
 #include "qemu/uri.h"
 
+#define MAX_PORTNUM_SIZE  6   /* max port num 65535 */
+
+#define GLUSTER_OPT_FILENAME  "filename"
+#define GLUSTER_OPT_VOLNAME   "volname"
+#define GLUSTER_OPT_IMAGE_PATH"image-path"
+#define GLUSTER_OPT_SERVER"server"
+#define GLUSTER_OPT_PORT  "port"
+#define GLUSTER_OPT_TRANSPORT "transport"
+#define GLUSTER_OPT_READ_PATTERN  "backup-volfile-servers."
+
+
 typedef struct GlusterAIOCB {
 int64_t size

[Qemu-devel] [PATCH] pc: memhotplug: rise minimum DIMM addr/size alignment to 128Mb

[Igor Mammedov]

it's attempt to workaround virtio bug reported earlier:
http://lists.nongnu.org/archive/html/qemu-devel/2015-08/msg00522.html
where virtio can't handle buffer that crosses border
between 2 DIMM's (i.e. 2 MemoryRegions).

Testing showed that virtio doesn't hit above bug
with 128Mb DIMM's granularity. Also linux memory
hotplug can handle hotplugged memory starting with
128Mb memory sections so lets rise minimum size limit
to 128Mb and align starting DIMM address on 128Mb.

It's certainly not the fix but it reduces risk of
crashing VM till virtio is fixed.
It also could be improved in guest's virtio side if it
would align buffers on 128Mb border and limit max  buffer
size to the same value.

Signed-off-by: Igor Mammedov 
---
Based on PCI tree as it has patches that add
2.5 machine type.
---
 hw/i386/pc.c |  8 +---
 hw/i386/pc_piix.c| 12 ++--
 hw/i386/pc_q35.c | 12 ++--
 include/hw/i386/pc.h |  5 ++---
 4 files changed, 27 insertions(+), 10 deletions(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index b5107f7..ddb6710 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1645,8 +1645,9 @@ static void pc_dimm_plug(HotplugHandler *hotplug_dev,
 MemoryRegion *mr = ddc->get_memory_region(dimm);
 uint64_t align = TARGET_PAGE_SIZE;
 
-if (memory_region_get_alignment(mr) && pcms->enforce_aligned_dimm) {
-align = memory_region_get_alignment(mr);
+if (pcms->enforce_aligned_dimm) {
+align = MAX(memory_region_get_alignment(mr),
+pcms->enforce_aligned_dimm);
 }
 
 if (!pcms->acpi_dev) {
@@ -1936,7 +1937,8 @@ static void pc_machine_initfn(Object *obj)
 "Enable vmport (pc & q35)",
 &error_abort);
 
-pcms->enforce_aligned_dimm = true;
+/* align DIMM starting address/size by 128Mb */
+pcms->enforce_aligned_dimm = 1ULL << 27;
 object_property_add_bool(obj, PC_MACHINE_ENFORCE_ALIGNED_DIMM,
  pc_machine_get_aligned_dimm,
  NULL, &error_abort);
diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index caa4edc..7671905 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -301,9 +301,17 @@ static void pc_init1(MachineState *machine,
 }
 }
 
+static void pc_compat_2_4(MachineState *machine)
+{
+PCMachineState *pcms = PC_MACHINE(machine);
+
+pcms->enforce_aligned_dimm = TARGET_PAGE_SIZE;
+}
+
 static void pc_compat_2_3(MachineState *machine)
 {
 PCMachineState *pcms = PC_MACHINE(machine);
+pc_compat_2_4(machine);
 savevm_skip_section_footers();
 if (kvm_enabled()) {
 pcms->smm = ON_OFF_AUTO_OFF;
@@ -326,7 +334,7 @@ static void pc_compat_2_1(MachineState *machine)
 pc_compat_2_2(machine);
 smbios_uuid_encoded = false;
 x86_cpu_compat_kvm_no_autodisable(FEAT_8000_0001_ECX, CPUID_EXT3_SVM);
-pcms->enforce_aligned_dimm = false;
+pcms->enforce_aligned_dimm = 0;
 }
 
 static void pc_compat_2_0(MachineState *machine)
@@ -485,7 +493,7 @@ static void pc_i440fx_2_4_machine_options(MachineClass *m)
 SET_MACHINE_COMPAT(m, PC_COMPAT_2_4);
 }
 
-DEFINE_I440FX_MACHINE(v2_4, "pc-i440fx-2.4", NULL,
+DEFINE_I440FX_MACHINE(v2_4, "pc-i440fx-2.4", pc_compat_2_3,
   pc_i440fx_2_4_machine_options)
 
 
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 506b6bf..72b479f 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -284,9 +284,17 @@ static void pc_q35_init(MachineState *machine)
 }
 }
 
+static void pc_compat_2_4(MachineState *machine)
+{
+PCMachineState *pcms = PC_MACHINE(machine);
+
+pcms->enforce_aligned_dimm = TARGET_PAGE_SIZE;
+}
+
 static void pc_compat_2_3(MachineState *machine)
 {
 PCMachineState *pcms = PC_MACHINE(machine);
+pc_compat_2_4(machine);
 savevm_skip_section_footers();
 if (kvm_enabled()) {
 pcms->smm = ON_OFF_AUTO_OFF;
@@ -307,7 +315,7 @@ static void pc_compat_2_1(MachineState *machine)
 PCMachineState *pcms = PC_MACHINE(machine);
 
 pc_compat_2_2(machine);
-pcms->enforce_aligned_dimm = false;
+pcms->enforce_aligned_dimm = 0;
 smbios_uuid_encoded = false;
 x86_cpu_compat_kvm_no_autodisable(FEAT_8000_0001_ECX, CPUID_EXT3_SVM);
 }
@@ -388,7 +396,7 @@ static void pc_q35_2_4_machine_options(MachineClass *m)
 SET_MACHINE_COMPAT(m, PC_COMPAT_2_4);
 }
 
-DEFINE_Q35_MACHINE(v2_4, "pc-q35-2.4", NULL,
+DEFINE_Q35_MACHINE(v2_4, "pc-q35-2.4", pc_compat_2_4,
pc_q35_2_4_machine_options);
 
 
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index 6896328..fdcf0ec 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -23,8 +23,7 @@
 /**
  * PCMachineState:
  * @acpi_dev: link to ACPI PM device that performs ACPI hotplug handling
- * @enforce_aligned_dimm: check that DIMM's address/size is aligned by
- *backend's alignment value if provided
+ * @enforce_aligned_dimm: minimal DIMM's address/size alignment

Re: [Qemu-devel] [PATCH 0/8] target-i386: Implement debug extensions

[Paolo Bonzini]

On 15/09/2015 20:45, Richard Henderson wrote:
> Best guess, since I can't find any code that actually uses them.
> Linux actively turns them off at boot...

I've sent a kvm-unit-tests patch to test debug extensions.  It shows
that debug extensions work, but the following needs to be squashed in
patch 4:

diff --git a/target-i386/bpt_helper.c b/target-i386/bpt_helper.c
index c258598..b24e446 100644
--- a/target-i386/bpt_helper.c
+++ b/target-i386/bpt_helper.c
@@ -134,14 +134,14 @@ void cpu_x86_update_dr7(CPUX86State *env, uint32_t 
new_dr7)
 int mod = ((old_dr7 | old_dr7 * 2) ^ (new_dr7 | new_dr7 * 2)) & 0xff;
 
 for (i = 0; i < DR7_MAX_BP; i++) {
-if (mod & (2 << i * 2)) {
-/* We know that register i has changed enable state;
-   recheck what that state should be and apply.  */
-if (hw_breakpoint_enabled(new_dr7, i)) {
-iobpt |= hw_breakpoint_insert(env, i);
-} else {
-hw_breakpoint_remove(env, i);
-}
+if ((mod & (2 << i * 2)) && !hw_breakpoint_enabled(new_dr7, i)) {
+hw_breakpoint_remove(env, i);
+}
+}
+env->dr[7] = new_dr7 | DR7_FIXED_1;
+for (i = 0; i < DR7_MAX_BP; i++) {
+if (mod & (2 << i * 2) && hw_breakpoint_enabled(new_dr7, i)) {
+iobpt |= hw_breakpoint_insert(env, i);
 } else if (hw_breakpoint_type(new_dr7, i) == DR7_TYPE_IO_RW
&& hw_breakpoint_enabled(new_dr7, i)) {
 iobpt |= HF_IOBPT_MASK;

Otherwise, hw_breakpoint_insert doesn't work because it expects to
see an updated env->dr[7].

There are a couple other issues that the tests expose, but they are
not regressions so I will send patches later.

Paolo

Re: [Qemu-devel] RFC: virtio-peer shared memory based peer communication device

[Michael S. Tsirkin]

On Fri, Sep 18, 2015 at 06:29:27PM +0200, Claudio Fontana wrote:
> Hello,
> 
> this is a first RFC for virtio-peer 0.1, which is still very much a work in 
> progress:
> 
> https://github.com/hw-claudio/virtio-peer/wiki
> 
> It is also available as PDF there, but the text is reproduced here for 
> commenting:
> 
> Peer shared memory communication device (virtio-peer)
> 
> General Overview
> 
> (I recommend looking at the PDF for some clarifying pictures)
> 
> The Virtio Peer shared memory communication device (virtio-peer) is a
> virtual device which allows high performance low latency guest to
> guest communication. It uses a new queue extension feature tentatively
> called VIRTIO_F_WINDOW which indicates that descriptor tables,
> available and used rings and Queue Data reside in physical memory
> ranges called Windows, each identified with an unique identifier
> called WindowID.

So if I had to summarize the difference from regular virtio,
I'd say the main one is that this uses window id + offset
instead of the physical address.


My question is - why do it?

All windows are in memory space, are they not?

How about guest using full physical addresses,
and hypervisor sending the window physical address
to VM2?

VM2 can uses that to find both window id and offset.


This way at least VM1 can use regular virtio without changes.

-- 
MST

Re: [Qemu-devel] RFC: virtio-peer shared memory based peer communication device

[Paolo Bonzini]

On 21/09/2015 12:47, Jan Kiszka wrote:
>> > Apart from the windows idea, how does virtio-peer compare to virtio-rpmsg?
> rpmsg is a very specialized thing. It targets single AMP cores, assuming
> that those have full access to the main memory.

Yes, this is why I did say "apart from the windows idea".

> And it is also a
> centralized approach where all message go through the main Linux
> instance. I suspect we could cover that use case as well with generic
> inter-vm shared memory device, but I didn't think about all details yet.

The virtqueue handling seems very similar between the two.  However, the
messages for rpmsg however have a small header (struct rpmsg_hdr in
include/linux/rpmsg.h) and there is a weird feature bit VIRTIO_RPMSG_F_NS.

So I guess virtio-rpmsg and virtio-peer are about as similar as
virtio-serial and virtio-peer.

Paolo

Re: [Qemu-devel] [Qemu-stable] Recent patches for 2.4

[Paolo Bonzini]

On 21/09/2015 11:52, Peter Lieven wrote:
>>
>> e4efd8a488d0a68b0af34d8ee245463df7c8bdf4 qemu-timer: initialize 
>> "timers_done_ev" to set
>> 12d69ac03b45156356b240424623719f15d8143e tests: remove irrelevant assertions 
>> from test-aio
>> 6493c975af75be5b8d9ade954239bdf5492b7911 aio-win32: reorganize polling loop
>> eabc977973103527bbb8fed69c91cfaa6691f8ab AioContext: fix broken 
>> ctx->dispatching optimization
>> 21a03d17f2edb1e63f7137d97ba355cc6f19d79f AioContext: fix broken placement of 
>> event_notifier_test_and_clear
>> 05e514b1d4d5bd4209e2c8bbc76ff05c85a235f3 AioContext: optimize clearing the 
>> EventNotifier
>> edec47cfef96209987cb7922286cb384916aae02 main-loop: fix qemu_notify_event 
>> for aio_notify optimization
>> fed105e2756dde98efa5e80baca02ae516dd1e51 virtio-blk-dataplane: delete bottom 
>> half before the AioContext is freed
>> a076972a4d36381d610a854f0c336507650a1d34 AioContext: avoid leaking BHs on 
>> cleanup
>> ca96ac44dcd290566090b2435bc828fded356ad9 AioContext: force event loop 
>> iteration using BH
>>
>> Paolo
> 
> thanks, I will have a look at them. If I understand correctly I should
> consider backporting all?

It's the safest bet, yes.

Paolo

Re: [Qemu-devel] [PATCH] pc: memhotplug: rise minimum DIMM addr/size alignment to 128Mb

[Paolo Bonzini]

On 21/09/2015 13:50, Igor Mammedov wrote:
> it's attempt to workaround virtio bug reported earlier:
> http://lists.nongnu.org/archive/html/qemu-devel/2015-08/msg00522.html
> where virtio can't handle buffer that crosses border
> between 2 DIMM's (i.e. 2 MemoryRegions).
> 
> Testing showed that virtio doesn't hit above bug
> with 128Mb DIMM's granularity. Also linux memory
> hotplug can handle hotplugged memory starting with
> 128Mb memory sections so lets rise minimum size limit
> to 128Mb and align starting DIMM address on 128Mb.
> 
> It's certainly not the fix but it reduces risk of
> crashing VM till virtio is fixed.
> It also could be improved in guest's virtio side if it
> would align buffers on 128Mb border and limit max  buffer
> size to the same value.
> 
> Signed-off-by: Igor Mammedov 

This seems to be easily handled at a level above QEMU---and the fix
would be available to older machine types as well.  This patch would
also make it quite a bit harder to test the real fix with QEMU. It is
not alone a reason to NACK it but should also be kept in mind.

Aligning to 4K makes some sense, since 4K is the page size, but
enforcing an arbitrary alignment above 4K is policy that does not belong
in QEMU.

To some extend, enforcing natural alignment would be okay as a
workaround for the virtio bug as well.  It would also make it easier to
ensure that hotplugged hugetlbfs-backed memory can use hugepages in the
guest.  Does it make sense to you?

Paolo

> ---
> Based on PCI tree as it has patches that add
> 2.5 machine type.
> ---
>  hw/i386/pc.c |  8 +---
>  hw/i386/pc_piix.c| 12 ++--
>  hw/i386/pc_q35.c | 12 ++--
>  include/hw/i386/pc.h |  5 ++---
>  4 files changed, 27 insertions(+), 10 deletions(-)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index b5107f7..ddb6710 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -1645,8 +1645,9 @@ static void pc_dimm_plug(HotplugHandler *hotplug_dev,
>  MemoryRegion *mr = ddc->get_memory_region(dimm);
>  uint64_t align = TARGET_PAGE_SIZE;
>  
> -if (memory_region_get_alignment(mr) && pcms->enforce_aligned_dimm) {
> -align = memory_region_get_alignment(mr);
> +if (pcms->enforce_aligned_dimm) {
> +align = MAX(memory_region_get_alignment(mr),
> +pcms->enforce_aligned_dimm);
>  }
>  
>  if (!pcms->acpi_dev) {
> @@ -1936,7 +1937,8 @@ static void pc_machine_initfn(Object *obj)
>  "Enable vmport (pc & q35)",
>  &error_abort);
>  
> -pcms->enforce_aligned_dimm = true;
> +/* align DIMM starting address/size by 128Mb */
> +pcms->enforce_aligned_dimm = 1ULL << 27;
>  object_property_add_bool(obj, PC_MACHINE_ENFORCE_ALIGNED_DIMM,
>   pc_machine_get_aligned_dimm,
>   NULL, &error_abort);
> diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
> index caa4edc..7671905 100644
> --- a/hw/i386/pc_piix.c
> +++ b/hw/i386/pc_piix.c
> @@ -301,9 +301,17 @@ static void pc_init1(MachineState *machine,
>  }
>  }
>  
> +static void pc_compat_2_4(MachineState *machine)
> +{
> +PCMachineState *pcms = PC_MACHINE(machine);
> +
> +pcms->enforce_aligned_dimm = TARGET_PAGE_SIZE;
> +}
> +
>  static void pc_compat_2_3(MachineState *machine)
>  {
>  PCMachineState *pcms = PC_MACHINE(machine);
> +pc_compat_2_4(machine);
>  savevm_skip_section_footers();
>  if (kvm_enabled()) {
>  pcms->smm = ON_OFF_AUTO_OFF;
> @@ -326,7 +334,7 @@ static void pc_compat_2_1(MachineState *machine)
>  pc_compat_2_2(machine);
>  smbios_uuid_encoded = false;
>  x86_cpu_compat_kvm_no_autodisable(FEAT_8000_0001_ECX, CPUID_EXT3_SVM);
> -pcms->enforce_aligned_dimm = false;
> +pcms->enforce_aligned_dimm = 0;
>  }
>  
>  static void pc_compat_2_0(MachineState *machine)
> @@ -485,7 +493,7 @@ static void pc_i440fx_2_4_machine_options(MachineClass *m)
>  SET_MACHINE_COMPAT(m, PC_COMPAT_2_4);
>  }
>  
> -DEFINE_I440FX_MACHINE(v2_4, "pc-i440fx-2.4", NULL,
> +DEFINE_I440FX_MACHINE(v2_4, "pc-i440fx-2.4", pc_compat_2_3,
>pc_i440fx_2_4_machine_options)
>  
>  
> diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
> index 506b6bf..72b479f 100644
> --- a/hw/i386/pc_q35.c
> +++ b/hw/i386/pc_q35.c
> @@ -284,9 +284,17 @@ static void pc_q35_init(MachineState *machine)
>  }
>  }
>  
> +static void pc_compat_2_4(MachineState *machine)
> +{
> +PCMachineState *pcms = PC_MACHINE(machine);
> +
> +pcms->enforce_aligned_dimm = TARGET_PAGE_SIZE;
> +}
> +
>  static void pc_compat_2_3(MachineState *machine)
>  {
>  PCMachineState *pcms = PC_MACHINE(machine);
> +pc_compat_2_4(machine);
>  savevm_skip_section_footers();
>  if (kvm_enabled()) {
>  pcms->smm = ON_OFF_AUTO_OFF;
> @@ -307,7 +315,7 @@ static void pc_compat_2_1(MachineState *machine)
>  PCMachineState *pcms = PC_MACHINE(machine);
>  
>  

[Qemu-devel] [PATCH 3/5] ide: add support for cancelable read requests

[Peter Lieven]

this patch adds a new aio readv compatible function which copies
all data through a bounce buffer. The benefit is that these requests
can be flagged as canceled to avoid guest memory corruption when
a canceled request is completed by the backend at a later stage.

If an IDE protocol wants to use this function it has to pipe
all read requests through ide_readv_cancelable and it may then
enable requests_cancelable in the IDEState.

If this state is enable we can avoid the blocking blk_drain_all
in case of a BMDMA reset.

Currently only read operations are cancelable thus we can only
use this logic for read-only devices.

Signed-off-by: Peter Lieven 
---
 hw/ide/core.c | 54 ++
 hw/ide/internal.h | 16 
 hw/ide/pci.c  | 42 --
 3 files changed, 98 insertions(+), 14 deletions(-)

diff --git a/hw/ide/core.c b/hw/ide/core.c
index 317406d..24547ce 100644
--- a/hw/ide/core.c
+++ b/hw/ide/core.c
@@ -561,6 +561,59 @@ static bool ide_sect_range_ok(IDEState *s,
 return true;
 }
 
+static void ide_readv_cancelable_cb(void *opaque, int ret)
+{
+IDECancelableRequest *req = opaque;
+if (!req->canceled) {
+if (!ret) {
+qemu_iovec_from_buf(req->org_qiov, 0, req->buf, 
req->org_qiov->size);
+}
+req->org_cb(req->org_opaque, ret);
+}
+QLIST_REMOVE(req, list);
+qemu_vfree(req->buf);
+qemu_iovec_destroy(&req->qiov);
+g_free(req);
+}
+
+#define MAX_CANCELABLE_REQS 16
+
+BlockAIOCB *ide_readv_cancelable(IDEState *s, int64_t sector_num,
+ QEMUIOVector *iov, int nb_sectors,
+ BlockCompletionFunc *cb, void *opaque)
+{
+BlockAIOCB *aioreq;
+IDECancelableRequest *req;
+int c = 0;
+
+QLIST_FOREACH(req, &s->cancelable_requests, list) {
+c++;
+}
+if (c > MAX_CANCELABLE_REQS) {
+return NULL;
+}
+
+req = g_new0(IDECancelableRequest, 1);
+qemu_iovec_init(&req->qiov, 1);
+req->buf = qemu_blockalign(blk_bs(s->blk), iov->size);
+qemu_iovec_add(&req->qiov, req->buf, iov->size);
+req->org_qiov = iov;
+req->org_cb = cb;
+req->org_opaque = opaque;
+
+aioreq = blk_aio_readv(s->blk, sector_num, &req->qiov, nb_sectors,
+   ide_readv_cancelable_cb, req);
+if (aioreq == NULL) {
+qemu_vfree(req->buf);
+qemu_iovec_destroy(&req->qiov);
+g_free(req);
+} else {
+QLIST_INSERT_HEAD(&s->cancelable_requests, req, list);
+}
+
+return aioreq;
+}
+
 static void ide_sector_read(IDEState *s);
 
 static void ide_sector_read_cb(void *opaque, int ret)
@@ -805,6 +858,7 @@ void ide_start_dma(IDEState *s, BlockCompletionFunc *cb)
 s->bus->retry_unit = s->unit;
 s->bus->retry_sector_num = ide_get_sector(s);
 s->bus->retry_nsector = s->nsector;
+s->bus->s = s;
 if (s->bus->dma->ops->start_dma) {
 s->bus->dma->ops->start_dma(s->bus->dma, s, cb);
 }
diff --git a/hw/ide/internal.h b/hw/ide/internal.h
index 05e93ff..ad188c2 100644
--- a/hw/ide/internal.h
+++ b/hw/ide/internal.h
@@ -343,6 +343,16 @@ enum ide_dma_cmd {
 #define ide_cmd_is_read(s) \
((s)->dma_cmd == IDE_DMA_READ)
 
+typedef struct IDECancelableRequest {
+QLIST_ENTRY(IDECancelableRequest) list;
+QEMUIOVector qiov;
+uint8_t *buf;
+QEMUIOVector *org_qiov;
+BlockCompletionFunc *org_cb;
+void *org_opaque;
+bool canceled;
+} IDECancelableRequest;
+
 /* NOTE: IDEState represents in fact one drive */
 struct IDEState {
 IDEBus *bus;
@@ -396,6 +406,8 @@ struct IDEState {
 BlockAIOCB *pio_aiocb;
 struct iovec iov;
 QEMUIOVector qiov;
+QLIST_HEAD(, IDECancelableRequest) cancelable_requests;
+bool requests_cancelable;
 /* ATA DMA state */
 int32_t io_buffer_offset;
 int32_t io_buffer_size;
@@ -468,6 +480,7 @@ struct IDEBus {
 uint8_t retry_unit;
 int64_t retry_sector_num;
 uint32_t retry_nsector;
+IDEState *s;
 };
 
 #define TYPE_IDE_DEVICE "ide-device"
@@ -572,6 +585,9 @@ void ide_set_inactive(IDEState *s, bool more);
 BlockAIOCB *ide_issue_trim(BlockBackend *blk,
 int64_t sector_num, QEMUIOVector *qiov, int nb_sectors,
 BlockCompletionFunc *cb, void *opaque);
+BlockAIOCB *ide_readv_cancelable(IDEState *s, int64_t sector_num,
+ QEMUIOVector *iov, int nb_sectors,
+ BlockCompletionFunc *cb, void *opaque);
 
 /* hw/ide/atapi.c */
 void ide_atapi_cmd(IDEState *s);
diff --git a/hw/ide/pci.c b/hw/ide/pci.c
index d31ff88..5587183 100644
--- a/hw/ide/pci.c
+++ b/hw/ide/pci.c
@@ -240,21 +240,35 @@ void bmdma_cmd_writeb(BMDMAState *bm, uint32_t val)
 /* Ignore writes to SSBM if it keeps the old value */
 if ((val & BM_CMD_START) != (bm->cmd & BM_CMD_START)) {
 if (!(val & BM_CMD_START)) {
-/*
- * We can't cancel Sc

[Qemu-devel] [PATCH 4/5] ide/atapi: enable cancelable requests

[Peter Lieven]

Signed-off-by: Peter Lieven 
---
 hw/ide/atapi.c | 4 ++--
 hw/ide/core.c  | 1 +
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/hw/ide/atapi.c b/hw/ide/atapi.c
index b209ed9..ab45495 100644
--- a/hw/ide/atapi.c
+++ b/hw/ide/atapi.c
@@ -141,7 +141,7 @@ static int cd_read_sector(IDEState *s, int lba, void *buf, 
int sector_size)
 s->iov.iov_len = 4 * BDRV_SECTOR_SIZE;
 qemu_iovec_init_external(&s->qiov, &s->iov, 1);
 
-if (blk_aio_readv(s->blk, (int64_t)lba << 2, &s->qiov, 4,
+if (ide_readv_cancelable(s, (int64_t)lba << 2, &s->qiov, 4,
   cd_read_sector_cb, s) == NULL) {
 return -EIO;
 }
@@ -368,7 +368,7 @@ static void ide_atapi_cmd_read_dma_cb(void *opaque, int ret)
 s->bus->dma->iov.iov_len = n * 4 * 512;
 qemu_iovec_init_external(&s->bus->dma->qiov, &s->bus->dma->iov, 1);
 
-s->bus->dma->aiocb = blk_aio_readv(s->blk, (int64_t)s->lba << 2,
+s->bus->dma->aiocb = ide_readv_cancelable(s, (int64_t)s->lba << 2,
&s->bus->dma->qiov, n * 4,
ide_atapi_cmd_read_dma_cb, s);
 if (s->bus->dma->aiocb == NULL) {
diff --git a/hw/ide/core.c b/hw/ide/core.c
index 24547ce..5c7a346 100644
--- a/hw/ide/core.c
+++ b/hw/ide/core.c
@@ -2330,6 +2330,7 @@ int ide_init_drive(IDEState *s, BlockBackend *blk, 
IDEDriveKind kind,
 if (kind == IDE_CD) {
 blk_set_dev_ops(blk, &ide_cd_block_ops, s);
 blk_set_guest_block_size(blk, 2048);
+s->requests_cancelable = true;
 } else {
 if (!blk_is_inserted(s->blk)) {
 error_report("Device needs media, but drive is empty");
-- 
1.9.1

Re: [Qemu-devel] RFC: virtio-peer shared memory based peer communication device

[Jan Kiszka]

On 2015-09-21 14:13, Michael S. Tsirkin wrote:
> On Fri, Sep 18, 2015 at 06:29:27PM +0200, Claudio Fontana wrote:
>> Hello,
>>
>> this is a first RFC for virtio-peer 0.1, which is still very much a work in 
>> progress:
>>
>> https://github.com/hw-claudio/virtio-peer/wiki
>>
>> It is also available as PDF there, but the text is reproduced here for 
>> commenting:
>>
>> Peer shared memory communication device (virtio-peer)
>>
>> General Overview
>>
>> (I recommend looking at the PDF for some clarifying pictures)
>>
>> The Virtio Peer shared memory communication device (virtio-peer) is a
>> virtual device which allows high performance low latency guest to
>> guest communication. It uses a new queue extension feature tentatively
>> called VIRTIO_F_WINDOW which indicates that descriptor tables,
>> available and used rings and Queue Data reside in physical memory
>> ranges called Windows, each identified with an unique identifier
>> called WindowID.
> 
> So if I had to summarize the difference from regular virtio,
> I'd say the main one is that this uses window id + offset
> instead of the physical address.
> 
> 
> My question is - why do it?
> 
> All windows are in memory space, are they not?
> 
> How about guest using full physical addresses,
> and hypervisor sending the window physical address
> to VM2?
> 
> VM2 can uses that to find both window id and offset.
> 
> 
> This way at least VM1 can use regular virtio without changes.

What would be the value of having different drivers in VM1 and VM2,
specifically if both run Linux?

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

[Qemu-devel] [PATCH 2/5] ide/atapi: blk_aio_readv may return NULL

[Peter Lieven]

Signed-off-by: Peter Lieven 
---
 hw/ide/atapi.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/hw/ide/atapi.c b/hw/ide/atapi.c
index 9257e1c..b209ed9 100644
--- a/hw/ide/atapi.c
+++ b/hw/ide/atapi.c
@@ -371,6 +371,10 @@ static void ide_atapi_cmd_read_dma_cb(void *opaque, int 
ret)
 s->bus->dma->aiocb = blk_aio_readv(s->blk, (int64_t)s->lba << 2,
&s->bus->dma->qiov, n * 4,
ide_atapi_cmd_read_dma_cb, s);
+if (s->bus->dma->aiocb == NULL) {
+ide_atapi_io_error(s, -EIO);
+goto eot;
+}
 return;
 
 eot:
-- 
1.9.1

[Qemu-devel] [PATCH 1/5] ide/atapi: make PIO read requests async

[Peter Lieven]

PIO read requests on the ATAPI interface used to be sync blk requests.
This has to siginificant drawbacks. First the main loop hangs util an
I/O request is completed and secondly if the I/O request does not
complete (e.g. due to an unresponsive storage) Qemu hangs completely.

Signed-off-by: Peter Lieven 
---
 hw/ide/atapi.c | 69 --
 1 file changed, 43 insertions(+), 26 deletions(-)

diff --git a/hw/ide/atapi.c b/hw/ide/atapi.c
index 747f466..9257e1c 100644
--- a/hw/ide/atapi.c
+++ b/hw/ide/atapi.c
@@ -105,31 +105,51 @@ static void cd_data_to_raw(uint8_t *buf, int lba)
 memset(buf, 0, 288);
 }
 
-static int cd_read_sector(IDEState *s, int lba, uint8_t *buf, int sector_size)
+static void cd_read_sector_cb(void *opaque, int ret)
 {
-int ret;
+IDEState *s = opaque;
 
-switch(sector_size) {
-case 2048:
-block_acct_start(blk_get_stats(s->blk), &s->acct,
- 4 * BDRV_SECTOR_SIZE, BLOCK_ACCT_READ);
-ret = blk_read(s->blk, (int64_t)lba << 2, buf, 4);
-block_acct_done(blk_get_stats(s->blk), &s->acct);
-break;
-case 2352:
-block_acct_start(blk_get_stats(s->blk), &s->acct,
- 4 * BDRV_SECTOR_SIZE, BLOCK_ACCT_READ);
-ret = blk_read(s->blk, (int64_t)lba << 2, buf + 16, 4);
-block_acct_done(blk_get_stats(s->blk), &s->acct);
-if (ret < 0)
-return ret;
-cd_data_to_raw(buf, lba);
-break;
-default:
-ret = -EIO;
-break;
+block_acct_done(blk_get_stats(s->blk), &s->acct);
+
+if (ret < 0) {
+ide_atapi_io_error(s, ret);
+return;
+}
+
+if (s->cd_sector_size == 2352) {
+cd_data_to_raw(s->io_buffer, s->lba);
 }
-return ret;
+
+s->lba++;
+s->io_buffer_index = 0;
+s->status &= ~BUSY_STAT;
+
+ide_atapi_cmd_reply_end(s);
+}
+
+static int cd_read_sector(IDEState *s, int lba, void *buf, int sector_size)
+{
+if (sector_size != 2048 && sector_size != 2352) {
+return -EINVAL;
+}
+
+s->iov.iov_base = buf;
+if (sector_size == 2352) {
+buf += 4;
+}
+
+s->iov.iov_len = 4 * BDRV_SECTOR_SIZE;
+qemu_iovec_init_external(&s->qiov, &s->iov, 1);
+
+if (blk_aio_readv(s->blk, (int64_t)lba << 2, &s->qiov, 4,
+  cd_read_sector_cb, s) == NULL) {
+return -EIO;
+}
+
+block_acct_start(blk_get_stats(s->blk), &s->acct,
+ 4 * BDRV_SECTOR_SIZE, BLOCK_ACCT_READ);
+s->status |= BUSY_STAT;
+return 0;
 }
 
 void ide_atapi_cmd_ok(IDEState *s)
@@ -190,10 +210,8 @@ void ide_atapi_cmd_reply_end(IDEState *s)
 ret = cd_read_sector(s, s->lba, s->io_buffer, s->cd_sector_size);
 if (ret < 0) {
 ide_atapi_io_error(s, ret);
-return;
 }
-s->lba++;
-s->io_buffer_index = 0;
+return;
 }
 if (s->elementary_transfer_size > 0) {
 /* there are some data left to transmit in this elementary
@@ -275,7 +293,6 @@ static void ide_atapi_cmd_read_pio(IDEState *s, int lba, 
int nb_sectors,
 s->io_buffer_index = sector_size;
 s->cd_sector_size = sector_size;
 
-s->status = READY_STAT | SEEK_STAT;
 ide_atapi_cmd_reply_end(s);
 }
 
-- 
1.9.1

Re: [Qemu-devel] [PATCH] pc: memhotplug: rise minimum DIMM addr/size alignment to 128Mb

[Igor Mammedov]

On Mon, 21 Sep 2015 14:22:23 +0200
Paolo Bonzini  wrote:

> 
> 
> On 21/09/2015 13:50, Igor Mammedov wrote:
> > it's attempt to workaround virtio bug reported earlier:
> > http://lists.nongnu.org/archive/html/qemu-devel/2015-08/msg00522.html
> > where virtio can't handle buffer that crosses border
> > between 2 DIMM's (i.e. 2 MemoryRegions).
> > 
> > Testing showed that virtio doesn't hit above bug
> > with 128Mb DIMM's granularity. Also linux memory
> > hotplug can handle hotplugged memory starting with
> > 128Mb memory sections so lets rise minimum size limit
> > to 128Mb and align starting DIMM address on 128Mb.
> > 
> > It's certainly not the fix but it reduces risk of
> > crashing VM till virtio is fixed.
> > It also could be improved in guest's virtio side if it
> > would align buffers on 128Mb border and limit max  buffer
> > size to the same value.
> > 
> > Signed-off-by: Igor Mammedov 
> 
> This seems to be easily handled at a level above QEMU---and the fix
> would be available to older machine types as well.  This patch would
> also make it quite a bit harder to test the real fix with QEMU. It is
> not alone a reason to NACK it but should also be kept in mind.
> 
> Aligning to 4K makes some sense, since 4K is the page size, but
> enforcing an arbitrary alignment above 4K is policy that does not belong
> in QEMU.
>
> To some extend, enforcing natural alignment would be okay as a
> workaround for the virtio bug as well.  It would also make it easier to
> ensure that hotplugged hugetlbfs-backed memory can use hugepages in the
> guest.  Does it make sense to you?
in current machine types we already enforce backend-s address/size alignment,
which is file's page size for hugetlbfs-backed memory and 2Mb for RAM backend.

So I guess we could try to apply workaround to virtio on guest side,
aligning and limiting max buffer size to 2Mb, it should work for 'old'
machine types as well.

> 
> Paolo
> 
> > ---
> > Based on PCI tree as it has patches that add
> > 2.5 machine type.
> > ---
> >  hw/i386/pc.c |  8 +---
> >  hw/i386/pc_piix.c| 12 ++--
> >  hw/i386/pc_q35.c | 12 ++--
> >  include/hw/i386/pc.h |  5 ++---
> >  4 files changed, 27 insertions(+), 10 deletions(-)
> > 
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index b5107f7..ddb6710 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -1645,8 +1645,9 @@ static void pc_dimm_plug(HotplugHandler *hotplug_dev,
> >  MemoryRegion *mr = ddc->get_memory_region(dimm);
> >  uint64_t align = TARGET_PAGE_SIZE;
> >  
> > -if (memory_region_get_alignment(mr) && pcms->enforce_aligned_dimm) {
> > -align = memory_region_get_alignment(mr);
> > +if (pcms->enforce_aligned_dimm) {
> > +align = MAX(memory_region_get_alignment(mr),
> > +pcms->enforce_aligned_dimm);
> >  }
> >  
> >  if (!pcms->acpi_dev) {
> > @@ -1936,7 +1937,8 @@ static void pc_machine_initfn(Object *obj)
> >  "Enable vmport (pc & q35)",
> >  &error_abort);
> >  
> > -pcms->enforce_aligned_dimm = true;
> > +/* align DIMM starting address/size by 128Mb */
> > +pcms->enforce_aligned_dimm = 1ULL << 27;
> >  object_property_add_bool(obj, PC_MACHINE_ENFORCE_ALIGNED_DIMM,
> >   pc_machine_get_aligned_dimm,
> >   NULL, &error_abort);
> > diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
> > index caa4edc..7671905 100644
> > --- a/hw/i386/pc_piix.c
> > +++ b/hw/i386/pc_piix.c
> > @@ -301,9 +301,17 @@ static void pc_init1(MachineState *machine,
> >  }
> >  }
> >  
> > +static void pc_compat_2_4(MachineState *machine)
> > +{
> > +PCMachineState *pcms = PC_MACHINE(machine);
> > +
> > +pcms->enforce_aligned_dimm = TARGET_PAGE_SIZE;
> > +}
> > +
> >  static void pc_compat_2_3(MachineState *machine)
> >  {
> >  PCMachineState *pcms = PC_MACHINE(machine);
> > +pc_compat_2_4(machine);
> >  savevm_skip_section_footers();
> >  if (kvm_enabled()) {
> >  pcms->smm = ON_OFF_AUTO_OFF;
> > @@ -326,7 +334,7 @@ static void pc_compat_2_1(MachineState *machine)
> >  pc_compat_2_2(machine);
> >  smbios_uuid_encoded = false;
> >  x86_cpu_compat_kvm_no_autodisable(FEAT_8000_0001_ECX, CPUID_EXT3_SVM);
> > -pcms->enforce_aligned_dimm = false;
> > +pcms->enforce_aligned_dimm = 0;
> >  }
> >  
> >  static void pc_compat_2_0(MachineState *machine)
> > @@ -485,7 +493,7 @@ static void pc_i440fx_2_4_machine_options(MachineClass 
> > *m)
> >  SET_MACHINE_COMPAT(m, PC_COMPAT_2_4);
> >  }
> >  
> > -DEFINE_I440FX_MACHINE(v2_4, "pc-i440fx-2.4", NULL,
> > +DEFINE_I440FX_MACHINE(v2_4, "pc-i440fx-2.4", pc_compat_2_3,
> >pc_i440fx_2_4_machine_options)
> >  
> >  
> > diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
> > index 506b6bf..72b479f 100644
> > --- a/hw/i386/pc_q35.c
> > +++ b/hw/i386/pc_q35.c
> > @@ -284,9 +284,17

[Qemu-devel] [PATCH 0/5] ide: avoid main-loop hang on CDROM/NFS failure

[Peter Lieven]

This series aims at avoiding a hanging main-loop if a vserver has a
CDROM image mounted from a NFS share and that NFS share goes down.
Typical situation is that users mount an CDROM ISO to install something
and then forget to eject that CDROM afterwards.
As a consequence this mounted CD is able to bring down the
whole vserver if the backend NFS share is unreachable. This is bad
especially if the CDROM itself is not needed anymore at this point.

This series aims at fixing 3 blocking I/O operations that would
hang if the NFS server is unavailable:
 - ATAPI PIO read requests used sync calls to blk_read, convert
   them to an async variant.
 - If a busmaster DMA request is cancelled all requests are drained.
   Convert the drain to an async request canceling.
 - query-block in the HMP or QMP hangs because it indirectly calls
   bdrv_get_allocated_file_size.

Note that Patch 5 is only included for completeness.

Peter

Peter Lieven (5):
  ide/atapi: make PIO read requests async
  ide/atapi: blk_aio_readv may return NULL
  ide: add support for cancelable read requests
  ide/atapi: enable cancelable requests
  block/nfs: cache allocated filesize for read-only files

 block/nfs.c   | 36 ++
 hw/ide/atapi.c| 75 +++
 hw/ide/core.c | 55 
 hw/ide/internal.h | 16 
 hw/ide/pci.c  | 42 ---
 5 files changed, 183 insertions(+), 41 deletions(-)

-- 
1.9.1

Re: [Qemu-devel] [PATCH] pc: memhotplug: rise minimum DIMM addr/size alignment to 128Mb

[Paolo Bonzini]

On 21/09/2015 15:05, Igor Mammedov wrote:
>> > To some extend, enforcing natural alignment would be okay as a
>> > workaround for the virtio bug as well.  It would also make it easier to
>> > ensure that hotplugged hugetlbfs-backed memory can use hugepages in the
>> > guest.  Does it make sense to you?
> in current machine types we already enforce backend-s address/size alignment,
> which is file's page size for hugetlbfs-backed memory and 2Mb for RAM backend.

Right, but it's not enough if the guest's physical address is not
aligned to 2Mb/1Gb too.  This is why we changed i440FX and q35 to have
only 3 and 2 gigabytes of low memory (down from 3.5 and 2 IIRC).

> So I guess we could try to apply workaround to virtio on guest side,
> aligning and limiting max buffer size to 2Mb, it should work for 'old'
> machine types as well.

That would make sense and it would be complementary to natural alignment
of DIMMs in the host.  This would give:

hostguest
old old fails
old new works (virtio workaround)
new old works (natural alignment)
new new works (choose your favorite workaround)

Paolo

Re: [Qemu-devel] [PATCH] linux-user: fix cmsg conversion in case of multiple headers

[Peter Maydell]

On 20 September 2015 at 22:34, Jonathan Neuschäfer
 wrote:
> On Fri, Sep 04, 2015 at 01:48:39PM +0100, Peter Maydell wrote:
>> On 3 September 2015 at 06:27, Jonathan Neuschäfer  
>> wrote:
>> > Currently, __target_cmsg_nxthdr compares a pointer derived from
>> > target_cmsg against the msg_control field of target_msgh (through
>> > subtraction).  This failed for me when emulating i386 code under x86_64,
>> > because pointers in the host address space and pointers in the guest
>> > address space were not the same.  This patch passes the initial value of
>> > target_cmsg into __target_cmsg_nxthdr.
>> >
>> > I found and fixed two more related bugs:
>> > - __target_cmsg_nxthdr now returns the new cmsg pointer instead of the
>> >   old one.
>> > - tgt_space (in host_to_target_cmsg) doesn't count "sizeof (struct
>> >   target_cmsghdr)" twice anymore.
>> >
>> > Signed-off-by: Jonathan Neuschäfer 
>>
>> Reviewed-by: Peter Maydell 
>
> Ping.
>
> What's the status of this patch?

It's waiting for Riku to wake up and put it into a linux-user
pull request.

thanks
-- MPM

[Qemu-devel] [PATCH 5/5] block/nfs: cache allocated filesize for read-only files

[Peter Lieven]

If the file is readonly its not expected to grow so
save the blocking call to nfs_fstat_async and use
the value saved at connection time. Also important
the monitor (and thus the main loop) will not hang
if block device info is queried and the NFS share
is unresponsive.

Signed-off-by: Peter Lieven 
Reviewed-by: Max Reitz 
Reviewed-by: Jeff Cody 
---
 block/nfs.c | 36 
 1 file changed, 36 insertions(+)

diff --git a/block/nfs.c b/block/nfs.c
index c026ff6..5ffd19f 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -43,6 +43,7 @@ typedef struct NFSClient {
 int events;
 bool has_zero_init;
 AioContext *aio_context;
+blkcnt_t st_blocks;
 } NFSClient;
 
 typedef struct NFSRPC {
@@ -374,6 +375,7 @@ static int64_t nfs_client_open(NFSClient *client, const 
char *filename,
 }
 
 ret = DIV_ROUND_UP(st.st_size, BDRV_SECTOR_SIZE);
+client->st_blocks = st.st_blocks;
 client->has_zero_init = S_ISREG(st.st_mode);
 goto out;
 fail:
@@ -464,6 +466,11 @@ static int64_t 
nfs_get_allocated_file_size(BlockDriverState *bs)
 NFSRPC task = {0};
 struct stat st;
 
+if (bdrv_is_read_only(bs) &&
+!(bs->open_flags & BDRV_O_NOCACHE)) {
+return client->st_blocks * 512;
+}
+
 task.st = &st;
 if (nfs_fstat_async(client->context, client->fh, nfs_co_generic_cb,
 &task) != 0) {
@@ -484,6 +491,34 @@ static int nfs_file_truncate(BlockDriverState *bs, int64_t 
offset)
 return nfs_ftruncate(client->context, client->fh, offset);
 }
 
+/* Note that this will not re-establish a connection with the NFS server
+ * - it is effectively a NOP.  */
+static int nfs_reopen_prepare(BDRVReopenState *state,
+  BlockReopenQueue *queue, Error **errp)
+{
+NFSClient *client = state->bs->opaque;
+struct stat st;
+int ret = 0;
+
+if (state->flags & BDRV_O_RDWR && bdrv_is_read_only(state->bs)) {
+error_setg(errp, "Cannot open a read-only mount as read-write");
+return -EACCES;
+}
+
+/* Update cache for read-only reopens */
+if (!(state->flags & BDRV_O_RDWR)) {
+ret = nfs_fstat(client->context, client->fh, &st);
+if (ret < 0) {
+error_setg(errp, "Failed to fstat file: %s",
+   nfs_get_error(client->context));
+return ret;
+}
+client->st_blocks = st.st_blocks;
+}
+
+return 0;
+}
+
 static BlockDriver bdrv_nfs = {
 .format_name= "nfs",
 .protocol_name  = "nfs",
@@ -499,6 +534,7 @@ static BlockDriver bdrv_nfs = {
 .bdrv_file_open = nfs_file_open,
 .bdrv_close = nfs_file_close,
 .bdrv_create= nfs_file_create,
+.bdrv_reopen_prepare= nfs_reopen_prepare,
 
 .bdrv_co_readv  = nfs_co_readv,
 .bdrv_co_writev = nfs_co_writev,
-- 
1.9.1

Re: [Qemu-devel] [PATCH v2 0/2] Dynamic module support for block drivers

[Marc Marí]

Ping!

Anyone has more comments for the next version?

Thanks
Marc

>On Tue,  8 Sep 2015 15:53:20 +0200
>Marc Marí  wrote:
>
> The current module infrastructure has been improved to enable dynamic
> module loading.
> 
> This reduces the load time for very simple guests. For the following
> configuration (very loaded)
> 
> ./configure --enable-sdl --enable-gtk --enable-vte --enable-curses \
> --enable-vnc --enable-vnc-{jpeg,tls,sasl,png} --enable-virtfs \
> --enable-brlapi --enable-curl --enable-fdt --enable-bluez \
> --enable-kvm --enable-rdma --enable-uuid --enable-vde \
> --enable-linux-aio --enable-cap-ng --enable-attr
> --enable-vhost-net \ --enable-vhost-scsi --enable-spice --enable-rbd
> --enable-libiscsi \ --enable-smartcard-nss --enable-guest-agent
> --enable-libusb \ --enable-usb-redir --enable-lzo --enable-snappy
> --enable-bzip2 \ --enable-seccomp --enable-coroutine-pool
> --enable-glusterfs \ --enable-tpm --enable-libssh2 --enable-vhdx
> --enable-numa \ --enable-tcmalloc --target-list=x86_64-softmmu
> 
> With modules disabled, there are 142 libraries loaded at startup.
> Time is the following:
>  LD time: 0.065 seconds
>  QEMU time: 0.02 seconds
>  Total time: 0.085 seconds
> 
> With this patch series and modules enabled, there are 128 libraries
> loaded at startup. Time is the following:
>  LD time: 0.02 seconds
>  QEMU time: 0.02 seconds
>  Total time: 0.04 seconds
> 
> Where LD time is the time between the program startup and the jump to
> main, and QEMU time is the time between the start of main and the
> first kvm_entry.
> 
> These results are just with a few block drivers, that were already a
> module. Adding more modules (block or not block) should be easy, and
> will reduce the load time even more.
> 
> Marc Marí (2):
>   Add dynamic module loading for block drivers
>   Add dynamic generation of module_block.h
> 
>  .gitignore  |   1 +
>  Makefile|  10 ++-
>  block.c |  70 +
>  configure   |   2 +-
>  include/qemu/module.h   |   3 +
>  scripts/modules/module_block.py | 134
> 
> util/module.c   |  38  7 files changed,
> 227 insertions(+), 31 deletions(-) create mode 100755
> scripts/modules/module_block.py
> 

Re: [Qemu-devel] [PATCH] pc: memhotplug: rise minimum DIMM addr/size alignment to 128Mb

[Igor Mammedov]

On Mon, 21 Sep 2015 15:13:17 +0200
Paolo Bonzini  wrote:

> 
> 
> On 21/09/2015 15:05, Igor Mammedov wrote:
> >> > To some extend, enforcing natural alignment would be okay as a
> >> > workaround for the virtio bug as well.  It would also make it easier to
> >> > ensure that hotplugged hugetlbfs-backed memory can use hugepages in the
> >> > guest.  Does it make sense to you?
> > in current machine types we already enforce backend-s address/size 
> > alignment,
> > which is file's page size for hugetlbfs-backed memory and 2Mb for RAM 
> > backend.
> 
> Right, but it's not enough if the guest's physical address is not
> aligned to 2Mb/1Gb too.  This is why we changed i440FX and q35 to have
> only 3 and 2 gigabytes of low memory (down from 3.5 and 2 IIRC).
DIMM's GPA is aligned to backend's alignment since 2.2,
it should be aligned 2Mb/1Gb depending on what hugetlbfs file is used
so that already works as expected

> 
> > So I guess we could try to apply workaround to virtio on guest side,
> > aligning and limiting max buffer size to 2Mb, it should work for 'old'
> > machine types as well.
> 
> That would make sense and it would be complementary to natural alignment
> of DIMMs in the host.  This would give:
> 
>   hostguest
>   old old fails
>   old new works (virtio workaround)
>   new old works (natural alignment)
+ not sure if it would work,
  I've though that virtio refactoring, that drops requirement
  for buffer to be inside of only one MemoryRegion, would 
  touch virtio in QEMU and on guest side too.

>   new new works (choose your favorite workaround)
> 
> Paolo

Re: [Qemu-devel] [PATCH] pc: memhotplug: rise minimum DIMM addr/size alignment to 128Mb

[Paolo Bonzini]

On 21/09/2015 15:32, Igor Mammedov wrote:
> On Mon, 21 Sep 2015 15:13:17 +0200
> Paolo Bonzini  wrote:
> 
>>
>>
>> On 21/09/2015 15:05, Igor Mammedov wrote:
> To some extend, enforcing natural alignment would be okay as a
> workaround for the virtio bug as well.  It would also make it easier to
> ensure that hotplugged hugetlbfs-backed memory can use hugepages in the
> guest.  Does it make sense to you?
>>> in current machine types we already enforce backend-s address/size 
>>> alignment,
>>> which is file's page size for hugetlbfs-backed memory and 2Mb for RAM 
>>> backend.
>>
>> Right, but it's not enough if the guest's physical address is not
>> aligned to 2Mb/1Gb too.  This is why we changed i440FX and q35 to have
>> only 3 and 2 gigabytes of low memory (down from 3.5 and 2 IIRC).
> 
> DIMM's GPA is aligned to backend's alignment since 2.2,
> it should be aligned 2Mb/1Gb depending on what hugetlbfs file is used
> so that already works as expected

Oh, ok.  This is what pcms->enforce_aligned_dimm does when true, and
this is also what I was missing.  Great!

>>> So I guess we could try to apply workaround to virtio on guest side,
>>> aligning and limiting max buffer size to 2Mb, it should work for 'old'
>>> machine types as well.
>>
>> That would make sense and it would be complementary to natural alignment
>> of DIMMs in the host.  This would give:
>>
>>  hostguest
>>  old old fails
>>  old new works (virtio workaround)
>>  new old works (natural alignment)
> + not sure if it would work,
>   I've though that virtio refactoring, that drops requirement
>   for buffer to be inside of only one MemoryRegion, would 
>   touch virtio in QEMU and on guest side too.

Right, it would theoretically not be enough.  However, I think it would
work in practice because hot-plugged DIMMs should be bigger than 128 MiB.

Paolo

Re: [Qemu-devel] [PATCH v2 0/2] Dynamic module support for block drivers

[Denis V. Lunev]

On 09/08/2015 04:53 PM, Marc Marí wrote:

The current module infrastructure has been improved to enable dynamic module
loading.

This reduces the load time for very simple guests. For the following
configuration (very loaded)

./configure --enable-sdl --enable-gtk --enable-vte --enable-curses \
 --enable-vnc --enable-vnc-{jpeg,tls,sasl,png} --enable-virtfs \
 --enable-brlapi --enable-curl --enable-fdt --enable-bluez \
 --enable-kvm --enable-rdma --enable-uuid --enable-vde \
 --enable-linux-aio --enable-cap-ng --enable-attr --enable-vhost-net \
 --enable-vhost-scsi --enable-spice --enable-rbd --enable-libiscsi \
 --enable-smartcard-nss --enable-guest-agent --enable-libusb \
 --enable-usb-redir --enable-lzo --enable-snappy --enable-bzip2 \
 --enable-seccomp --enable-coroutine-pool --enable-glusterfs \
 --enable-tpm --enable-libssh2 --enable-vhdx --enable-numa \
 --enable-tcmalloc --target-list=x86_64-softmmu

With modules disabled, there are 142 libraries loaded at startup. Time is
the following:
  LD time: 0.065 seconds
  QEMU time: 0.02 seconds
  Total time: 0.085 seconds

With this patch series and modules enabled, there are 128 libraries loaded
at startup. Time is the following:
  LD time: 0.02 seconds
  QEMU time: 0.02 seconds
  Total time: 0.04 seconds

Where LD time is the time between the program startup and the jump to main,
and QEMU time is the time between the start of main and the first kvm_entry.

These results are just with a few block drivers, that were already a module.
Adding more modules (block or not block) should be easy, and will reduce
the load time even more.

Marc Marí (2):
   Add dynamic module loading for block drivers
   Add dynamic generation of module_block.h

  .gitignore  |   1 +
  Makefile|  10 ++-
  block.c |  70 +
  configure   |   2 +-
  include/qemu/module.h   |   3 +
  scripts/modules/module_block.py | 134 
  util/module.c   |  38 
  7 files changed, 227 insertions(+), 31 deletions(-)
  create mode 100755 scripts/modules/module_block.py



From my point of view the design looks a bit complex.
The approach should be quite similar to one used
in Linux kernel.

If the block driver is configured as a module, block_init
should register proper hooks and create generic lists.
C code parsing does not look like a good approach
to me.

If block_init is a bad name, we could use something like
module_init.

Re: [Qemu-devel] [PATCH v2 1/2] Add dynamic module loading for block drivers

[Denis V. Lunev]

On 09/08/2015 04:53 PM, Marc Marí wrote:

Extend the current module interface to allow for block drivers to be loaded
dynamically on request.

The only block drivers that can be converted into modules are the drivers
that don't perform any init operation except for registering themselves. This
is why libiscsi has been disabled as a module.

All the necessary module information is located in a new structure found in
include/qemu/module_block.h

Signed-off-by: Marc Marí 
---
  block.c | 70 +++
  configure   |  2 +-
  include/qemu/module.h   |  3 ++
  include/qemu/module_block.h | 90 +
  util/module.c   | 38 ++-
  5 files changed, 175 insertions(+), 28 deletions(-)
  create mode 100644 include/qemu/module_block.h

diff --git a/block.c b/block.c
index 090923c..814429a 100644
--- a/block.c
+++ b/block.c
@@ -27,6 +27,7 @@
  #include "block/block_int.h"
  #include "block/blockjob.h"
  #include "qemu/error-report.h"
+#include "qemu/module_block.h"
  #include "qemu/module.h"
  #include "qapi/qmp/qerror.h"
  #include "qapi/qmp/qjson.h"
@@ -277,11 +278,30 @@ void bdrv_add_close_notifier(BlockDriverState *bs, 
Notifier *notify)
  BlockDriver *bdrv_find_format(const char *format_name)
  {
  BlockDriver *drv1;
+size_t i;
+
  QLIST_FOREACH(drv1, &bdrv_drivers, list) {
  if (!strcmp(drv1->format_name, format_name)) {
  return drv1;
  }
  }
+
+for (i = 0; i < ARRAY_SIZE(block_driver_modules); ++i) {
+if (!strcmp(block_driver_modules[i].format_name, format_name)) {
+block_module_load_one(block_driver_modules[i].library_name);
+/* Copying code is not nice, but this way the current discovery is
+ * not modified. Calling recursively could fail if the library
+ * has been deleted.
+ */
+QLIST_FOREACH(drv1, &bdrv_drivers, list) {
+if (!strcmp(drv1->format_name, format_name)) {
+return drv1;
+}
+}
+}
+}
+
+
  return NULL;
  }
  
@@ -484,8 +504,15 @@ int get_tmp_filename(char *filename, int size)

  static BlockDriver *find_hdev_driver(const char *filename)
  {
  int score_max = 0, score;
+size_t i;
  BlockDriver *drv = NULL, *d;
  
+for (i = 0; i < ARRAY_SIZE(block_driver_modules); ++i) {

+if (block_driver_modules[i].has_probe_device) {
+block_module_load_one(block_driver_modules[i].library_name);
+}
+}
+
  QLIST_FOREACH(d, &bdrv_drivers, list) {
  if (d->bdrv_probe_device) {
  score = d->bdrv_probe_device(filename);
@@ -507,6 +534,7 @@ BlockDriver *bdrv_find_protocol(const char *filename,
  char protocol[128];
  int len;
  const char *p;
+size_t i;
  
  /* TODO Drivers without bdrv_file_open must be specified explicitly */
  
@@ -533,6 +561,7 @@ BlockDriver *bdrv_find_protocol(const char *filename,

  len = sizeof(protocol) - 1;
  memcpy(protocol, filename, len);
  protocol[len] = '\0';
+
  QLIST_FOREACH(drv1, &bdrv_drivers, list) {
  if (drv1->protocol_name &&
  !strcmp(drv1->protocol_name, protocol)) {
@@ -540,6 +569,23 @@ BlockDriver *bdrv_find_protocol(const char *filename,
  }
  }
  
+for (i = 0; i < ARRAY_SIZE(block_driver_modules); ++i) {

+if (block_driver_modules[i].protocol_name &&
+!strcmp(block_driver_modules[i].protocol_name, protocol)) {
+block_module_load_one(block_driver_modules[i].library_name);
+/* Copying code is not nice, but this way the current discovery is
+ * not modified. Calling recursively could fail if the library
+ * has been deleted.
+ */
+QLIST_FOREACH(drv1, &bdrv_drivers, list) {
+if (drv1->protocol_name &&
+!strcmp(drv1->protocol_name, protocol)) {
+return drv1;
+}
+}
+}
+}
+
  error_setg(errp, "Unknown protocol '%s'", protocol);
  return NULL;
  }
@@ -562,8 +608,15 @@ BlockDriver *bdrv_probe_all(const uint8_t *buf, int 
buf_size,
  const char *filename)
  {
  int score_max = 0, score;
+size_t i;
  BlockDriver *drv = NULL, *d;
  
+for (i = 0; i < ARRAY_SIZE(block_driver_modules); ++i) {

+if (block_driver_modules[i].has_probe) {
+block_module_load_one(block_driver_modules[i].library_name);
+}
+}
+
  QLIST_FOREACH(d, &bdrv_drivers, list) {
  if (d->bdrv_probe) {
  score = d->bdrv_probe(buf, buf_size, filename);
@@ -2784,6 +2837,7 @@ void bdrv_iterate_format(void (*it)(void *opaque, const 
char *name),
  BlockDriver *drv;
  int count = 0;
  int i;
+size_t n;
  const char **formats = NULL;
  

Re: [Qemu-devel] [PATCH v3 0/9] hmp command IO- and Local APIC dump state

[Paolo Bonzini]

On 21/09/2015 12:57, Denis V. Lunev wrote:
> Added the hmp command to query IO- and Local APIC registers state,
> it can be very useful to identify problems related to the emulation devices.
> 
> (qemu) info lapic
> dumping local APIC state for CPU 0
> 
> LVT0 0x00010700 active-hi edge  masked  ExtINT (vec 0)
> LVT1 0x0400 active-hi edge  NMI
> LVTPC0x0001 active-hi edge  masked  Fixed  (vec 0)
> LVTERR   0x00fe active-hi edge  Fixed  (vec 
> 254)
> LVTTHMR  0x0001 active-hi edge  masked  Fixed  (vec 0)
> LVTT 0x00ef active-hi edge one-shot Fixed  (vec 
> 239)
> TimerDCR=0x3 (divide by 16) initial_count = 62278
> SPIV 0x01ff APIC enabled, focus=off, spurious vec 255
> ICR  0x00fd physical edge de-assert no

s/no/no-shorthand/ or similar.

> ICR2 0x0001 cpu 1 (X2APIC ID)
> ESR  0x
> ISR

Adding a "(none)" here would be nice.

> IRR  239
> 
> APR 0x00 TPR 0x00 DFR 0x0f LDR 0x00 PPR 0x00
> 
> (qemu) info ioapic
> ioapic id=0x00 sel=0x2e (redir[15])
> pin 0  0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 1  0x0031 dest=0 vec=49  active-hi edge fixed  
> physical
> pin 2  0x0030 dest=0 vec=48  active-hi edge fixed  
> physical
> pin 3  0x0033 dest=0 vec=51  active-hi edge fixed  
> physical
> pin 4  0x00010034 dest=0 vec=52  active-hi edge  masked fixed  
> physical
> pin 5  0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 6  0x0036 dest=0 vec=54  active-hi edge fixed  
> physical
> pin 7  0x0037 dest=0 vec=55  active-hi edge fixed  
> physical
> pin 8  0x0038 dest=0 vec=56  active-hi edge fixed  
> physical
> pin 9  0x00018041 dest=0 vec=65  active-hi level masked fixed  
> physical
> pin 10 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 11 0x03008081 dest=3 vec=129 active-hi levelfixed  
> physical
> pin 12 0x003c dest=0 vec=60  active-hi edge fixed  
> physical
> pin 13 0x003d dest=0 vec=61  active-hi edge fixed  
> physical
> pin 14 0x003e dest=0 vec=62  active-hi edge fixed  
> physical
> pin 15 0x003f dest=0 vec=63  active-hi edge fixed  
> physical
> pin 16 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 17 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 18 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 19 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 20 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 21 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 22 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 23 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> IRR (none)

I would print IRR as a bitmap here, like ISR/IRR in the local APIC dump.
 Similarly, it would be nice to print the "Remote IRR" bits of the
redirection table (printing them as a bitmap with the same format).

> 
> Changes from v1:
>  - implementation of hmp commands moved to the target-i386 part
>  - the cpu_dump_apic_local_state interface moved to the target-i386 part
> 
> Changes from v2:
>  - lapic and ioapic format dump are fixed for more human readable view
> 
> Signed-off-by: Pavel Butsykin 
> Signed-off-by: Denis V. Lunev 
> CC: Paolo Bonzini 
> CC: Andreas Färber 

I would like a second look at the HMP code.

Paolo

> Pavel Butsykin (9):
>   apic_internal.h: make some apic_get_* functions externally visible
>   apic_internal.h: rename ESR_ILLEGAL_ADDRESS to
> APIC_ESR_ILLEGAL_ADDRESS
>   apic_internal.h: added more constants
>   apic_internal.h: fix formatting and drop unused consts
>   monitor: make monitor_fprintf and mon_get_cpu externally visible
>   hmp: added local apic dump state
>   ioapic_internal.h: added more constants
>   hmp: added io apic dump state
>   hmp: implemented io apic dump state for TCG
> 
>  disas.c   |  10 --
>  hmp-commands-info.hx  |  32 +++
>  hw/i386/kvm/ioapic.c  |  10 ++
>  hw/intc/apic.c|  22 +
>  hw/intc/ioapic.c  |  12 +++
>  hw/intc/ioapic_common.c   |  37 
>  include/hw/i386/apic_internal.h   |  96 ---
>  include/hw/i386/ioapic_internal.h |   7 ++
>  include/hw/i386/pc.h  |   5 +
>  include/monitor/hmp-target.h  |   3 +
>  include/monitor/monitor.h |   1 +
>  monitor.c |   5 +-
>  target-i386/cpu.h |   3 

Re: [Qemu-devel] [PATCH v4 07/22] vhost: alloc shareable log

[Marc-André Lureau]

Hi

- Original Message -
> On Sat, Sep 19, 2015 at 12:11:58PM +0200, marcandre.lur...@redhat.com wrote:
> > From: Marc-André Lureau 
> > 
> > If the backend is of type VHOST_BACKEND_TYPE_USER, allocate
> > shareable memory. Next patch will only allocate when the backend
> > has the required feature.
> > 
> > Note: vhost_log_get() can use a global "vhost_log" that can be shared by
> > several vhost devices. We may want instead a common shareable log and a
> > common non-shareable one.
> > 
> > Signed-off-by: Marc-André Lureau 
> 
> Well you do at least need to count the number of times the log is
> shared.  Otherwise, if you share the log, then unshare it, you are left
> with a shared one.

Do you mean that if the device is removed (or stopped, so that the log is 
unref) and a shm log is no longer needed, it should replace the log of other 
devices with a non-shm log? In this case, wouldn't it make more sense to have a 
log shm per device, because replacing other devices log do not seem simple 
without more tracking of devices sharing the log.

Can this be considered a future enhancement?
 
> 
> > ---
> >  hw/virtio/vhost.c | 38 +++---
> >  include/hw/virtio/vhost.h |  3 ++-
> >  2 files changed, 33 insertions(+), 8 deletions(-)
> > 
> > diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> > index a08c36b..cd3af16 100644
> > --- a/hw/virtio/vhost.c
> > +++ b/hw/virtio/vhost.c
> > @@ -18,6 +18,7 @@
> >  #include "qemu/atomic.h"
> >  #include "qemu/range.h"
> >  #include "qemu/error-report.h"
> > +#include "qemu/memfd.h"
> >  #include 
> >  #include "exec/address-spaces.h"
> >  #include "hw/virtio/virtio-bus.h"
> > @@ -286,20 +287,34 @@ static uint64_t vhost_get_log_size(struct vhost_dev
> > *dev)
> >  }
> >  return log_size;
> >  }
> > -static struct vhost_log *vhost_log_alloc(uint64_t size)
> > +
> > +static struct vhost_log *vhost_log_alloc(uint64_t size, bool share)
> >  {
> > -struct vhost_log *log = g_malloc0(sizeof *log + size *
> > sizeof(*(log->log)));
> > +struct vhost_log *log;
> > +uint64_t logsize = size * sizeof(*(log->log));
> > +int fd = -1;
> > +
> > +log = g_new0(struct vhost_log, 1);
> > +if (share) {
> > +log->log = qemu_memfd_alloc("vhost-log", logsize,
> > +F_SEAL_GROW|F_SEAL_SHRINK|F_SEAL_SEAL,
> > &fd);
> > +memset(log->log, 0, logsize);
> > +} else {
> > +log->log = g_malloc0(logsize);
> > +}
> >  
> >  log->size = size;
> >  log->refcnt = 1;
> > +log->fd = fd;
> >  
> >  return log;
> >  }
> >  
> > -static struct vhost_log *vhost_log_get(uint64_t size)
> > +static struct vhost_log *vhost_log_get(uint64_t size, bool share)
> >  {
> > -if (!vhost_log || vhost_log->size != size) {
> > -vhost_log = vhost_log_alloc(size);
> > +if (!vhost_log || vhost_log->size != size ||
> > +(share && vhost_log->fd == -1)) {
> > +vhost_log = vhost_log_alloc(size, share);
> >  } else {
> >  ++vhost_log->refcnt;
> >  }
> > @@ -324,13 +339,21 @@ static void vhost_log_put(struct vhost_dev *dev, bool
> > sync)
> >  if (vhost_log == log) {
> >  vhost_log = NULL;
> >  }
> > +
> > +if (log->fd == -1) {
> > +g_free(log->log);
> > +} else {
> > +qemu_memfd_free(log->log, log->size * sizeof(*(log->log)),
> > +log->fd);
> > +}
> >  g_free(log);
> >  }
> >  }
> >  
> >  static inline void vhost_dev_log_resize(struct vhost_dev* dev, uint64_t
> >  size)
> >  {
> > -struct vhost_log *log = vhost_log_get(size);
> > +bool share = dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER;
> > +struct vhost_log *log = vhost_log_get(size, share);
> >  uint64_t log_base = (uintptr_t)log->log;
> >  int r;
> >  
> > @@ -1136,9 +1159,10 @@ int vhost_dev_start(struct vhost_dev *hdev,
> > VirtIODevice *vdev)
> >  
> >  if (hdev->log_enabled) {
> >  uint64_t log_base;
> > +bool share = hdev->vhost_ops->backend_type ==
> > VHOST_BACKEND_TYPE_USER;
> >  
> >  hdev->log_size = vhost_get_log_size(hdev);
> > -hdev->log = vhost_log_get(hdev->log_size);
> > +hdev->log = vhost_log_get(hdev->log_size, share);
> >  log_base = (uintptr_t)hdev->log->log;
> >  r = hdev->vhost_ops->vhost_call(hdev, VHOST_SET_LOG_BASE,
> >  hdev->log_size ? &log_base :
> >  NULL);
> > diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
> > index 6467c73..ab1dcac 100644
> > --- a/include/hw/virtio/vhost.h
> > +++ b/include/hw/virtio/vhost.h
> > @@ -31,7 +31,8 @@ typedef unsigned long vhost_log_chunk_t;
> >  struct vhost_log {
> >  unsigned long long size;
> >  int refcnt;
> > -vhost_log_chunk_t log[0];
> > +int fd;
> > +vhost_log_chunk_t *log;
> >  };

Re: [Qemu-devel] [PATCH v4 12/22] vhost: only use shared log if in use by backend

[Marc-André Lureau]

Hi

- Original Message -
> On Sat, Sep 19, 2015 at 12:12:03PM +0200, marcandre.lur...@redhat.com wrote:
> > From: Marc-André Lureau 
> > 
> > Do not allocate a shared log if the backend doesn't support it.
> > 
> > Signed-off-by: Marc-André Lureau 
> 
> If not there, we probably want to block live migration, correct?

That make sense to me, I'll add that.

> > ---
> >  hw/virtio/vhost-user.c|  9 +
> >  hw/virtio/vhost.c | 15 ++-
> >  include/hw/virtio/vhost-backend.h |  4 
> >  3 files changed, 23 insertions(+), 5 deletions(-)
> > 
> > diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> > index 3cb5559..d12cd66 100644
> > --- a/hw/virtio/vhost-user.c
> > +++ b/hw/virtio/vhost-user.c
> > @@ -508,10 +508,19 @@ static int vhost_user_cleanup(struct vhost_dev *dev)
> >  return 0;
> >  }
> >  
> > +static bool vhost_user_want_shm_log(struct vhost_dev *dev)
> > +{
> > +assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER);
> > +
> > +return virtio_has_feature(dev->protocol_features,
> > +  VHOST_USER_PROTOCOL_F_LOG_SHMFD);
> > +}
> > +
> >  const VhostOps user_ops = {
> >  .backend_type = VHOST_BACKEND_TYPE_USER,
> >  .vhost_backend_init = vhost_user_init,
> >  .vhost_backend_cleanup = vhost_user_cleanup,
> > +.vhost_backend_want_shm_log = vhost_user_want_shm_log,
> >  
> >  .vhost_net_set_backend = vhost_net_set_backend,
> >  .vhost_scsi_set_endpoint = vhost_scsi_set_endpoint,
> > diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> > index 0fa7b69..d5070fb 100644
> > --- a/hw/virtio/vhost.c
> > +++ b/hw/virtio/vhost.c
> > @@ -350,10 +350,15 @@ static void vhost_log_put(struct vhost_dev *dev, bool
> > sync)
> >  }
> >  }
> >  
> > -static inline void vhost_dev_log_resize(struct vhost_dev* dev, uint64_t
> > size)
> > +static bool vhost_dev_log_is_shared(struct vhost_dev *dev)
> >  {
> > -bool share = dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER;
> > -struct vhost_log *log = vhost_log_get(size, share);
> > +return dev->vhost_ops->vhost_backend_want_shm_log &&
> > +   dev->vhost_ops->vhost_backend_want_shm_log(dev);
> > +}
> > +
> > +static inline void vhost_dev_log_resize(struct vhost_dev *dev, uint64_t
> > size)
> > +{
> > +struct vhost_log *log = vhost_log_get(size,
> > vhost_dev_log_is_shared(dev));
> >  uint64_t log_base = (uintptr_t)log->log;
> >  int r;
> >  
> > @@ -1161,10 +1166,10 @@ int vhost_dev_start(struct vhost_dev *hdev,
> > VirtIODevice *vdev)
> >  
> >  if (hdev->log_enabled) {
> >  uint64_t log_base;
> > -bool share = hdev->vhost_ops->backend_type ==
> > VHOST_BACKEND_TYPE_USER;
> >  
> >  hdev->log_size = vhost_get_log_size(hdev);
> > -hdev->log = vhost_log_get(hdev->log_size, share);
> > +hdev->log = vhost_log_get(hdev->log_size,
> > +  vhost_dev_log_is_shared(hdev));
> >  log_base = (uintptr_t)hdev->log->log;
> >  r = hdev->vhost_ops->vhost_set_log_base(hdev,
> >  hdev->log_size ? log_base
> >  : 0,
> > diff --git a/include/hw/virtio/vhost-backend.h
> > b/include/hw/virtio/vhost-backend.h
> > index a87f0b3..44bf200 100644
> > --- a/include/hw/virtio/vhost-backend.h
> > +++ b/include/hw/virtio/vhost-backend.h
> > @@ -11,6 +11,8 @@
> >  #ifndef VHOST_BACKEND_H_
> >  #define VHOST_BACKEND_H_
> >  
> > +#include 
> > +
> >  typedef enum VhostBackendType {
> >  VHOST_BACKEND_TYPE_NONE = 0,
> >  VHOST_BACKEND_TYPE_KERNEL = 1,
> > @@ -28,6 +30,7 @@ struct vhost_scsi_target;
> >  
> >  typedef int (*vhost_backend_init)(struct vhost_dev *dev, void *opaque);
> >  typedef int (*vhost_backend_cleanup)(struct vhost_dev *dev);
> > +typedef bool (*vhost_backend_want_shm_log)(struct vhost_dev *dev);
> >  
> >  typedef int (*vhost_net_set_backend_op)(struct vhost_dev *dev,
> >  struct vhost_vring_file *file);
> > @@ -67,6 +70,7 @@ typedef struct VhostOps {
> >  
> >  vhost_backend_init vhost_backend_init;
> >  vhost_backend_cleanup vhost_backend_cleanup;
> > +vhost_backend_want_shm_log vhost_backend_want_shm_log;
> >  
> >  vhost_net_set_backend_op vhost_net_set_backend;
> >  vhost_scsi_set_endpoint_op vhost_scsi_set_endpoint;
> > --
> > 2.4.3
> 

Re: [Qemu-devel] [PATCH v4 11/22] vhost-user: send log shm fd along with log_base

[Marc-André Lureau]

- Original Message -
> On Sat, Sep 19, 2015 at 12:12:02PM +0200, marcandre.lur...@redhat.com wrote:
> > From: Marc-André Lureau 
> > 
> > Send the shm for the dirty pages logging if the backend supports
> > VHOST_USER_PROTOCOL_F_LOG_SHMFD.
> > 
> > Signed-off-by: Marc-André Lureau 
> > ---
> >  hw/virtio/vhost-backend.c |  3 ++-
> >  hw/virtio/vhost-user.c| 17 ++---
> >  hw/virtio/vhost.c |  5 +++--
> >  include/hw/virtio/vhost-backend.h |  4 +++-
> >  4 files changed, 22 insertions(+), 7 deletions(-)
> > 
> > diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c
> > index bf2d1d4..3f2d6ea 100644
> > --- a/hw/virtio/vhost-backend.c
> > +++ b/hw/virtio/vhost-backend.c
> > @@ -66,7 +66,8 @@ static int vhost_scsi_get_abi_version(struct vhost_dev
> > *dev, int *version)
> >  return vhost_kernel_call(dev, VHOST_SCSI_GET_ABI_VERSION, version);
> >  }
> >  
> > -static int vhost_set_log_base(struct vhost_dev *dev, uint64_t base)
> > +static int vhost_set_log_base(struct vhost_dev *dev, uint64_t base,
> > +  struct vhost_log *log)
> >  {
> >  return vhost_kernel_call(dev, VHOST_SET_LOG_BASE, &base);
> >  }
> > diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> > index d3334f5..3cb5559 100644
> > --- a/hw/virtio/vhost-user.c
> > +++ b/hw/virtio/vhost-user.c
> > @@ -26,7 +26,9 @@
> >  #define VHOST_MEMORY_MAX_NREGIONS8
> >  
> >  #define VHOST_USER_F_PROTOCOL_FEATURES 30
> > -#define VHOST_USER_PROTOCOL_FEATURE_MASK 0x0ULL
> > +
> > +#define VHOST_USER_PROTOCOL_FEATURE_MASK 0x1ULL
> > +#define VHOST_USER_PROTOCOL_F_LOG_SHMFD 0
> >  
> >  typedef enum VhostUserRequest {
> >  VHOST_USER_NONE = 0,
> > @@ -182,8 +184,11 @@ static int vhost_scsi_get_abi_version(struct vhost_dev
> > *dev, int *version)
> >  return -1;
> >  }
> >  
> > -static int vhost_set_log_base(struct vhost_dev *dev, uint64_t base)
> > +static int vhost_set_log_base(struct vhost_dev *dev, uint64_t base,
> > +  struct vhost_log *log)
> >  {
> > +int fds[VHOST_MEMORY_MAX_NREGIONS];
> > +size_t fd_num = 0;
> >  VhostUserMsg msg = {
> >  .request = VHOST_USER_SET_LOG_BASE,
> >  .flags = VHOST_USER_VERSION,
> > @@ -191,7 +196,13 @@ static int vhost_set_log_base(struct vhost_dev *dev,
> > uint64_t base)
> >  .size = sizeof(m.u64),
> >  };
> >  
> > -vhost_user_write(dev, &msg, NULL, 0);
> > +if (virtio_has_feature(dev->protocol_features,
> > +   VHOST_USER_PROTOCOL_F_LOG_SHMFD) &&
> > +log->fd != -1) {
> > +fds[fd_num++] = log->fd;
> > +}
> > +
> > +vhost_user_write(dev, &msg, fds, fd_num);
> >  
> >  return 0;
> >  }
> 
> Doesn't this one have to get a response?
> Otherwise the other side might still be using the old
> buffer after we've updated this one.

That make sense, though I wonder about all other SET commands in the protocol 
then. Isn't there other commands that may change the fd or memory region over 
time?

> Probably need a protocol feature, but I guess we can reuse
> VHOST_USER_PROTOCOL_F_LOG_SHMFD - IIUC logging doesn't
> work without this anyway ...

ok

> 
> 
> > diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> > index 83ef164..0fa7b69 100644
> > --- a/hw/virtio/vhost.c
> > +++ b/hw/virtio/vhost.c
> > @@ -359,7 +359,7 @@ static inline void vhost_dev_log_resize(struct
> > vhost_dev* dev, uint64_t size)
> >  
> >  /* inform backend of log switching, this must be done before
> > releasing the current log, to ensure no logging is lost */
> > -r = dev->vhost_ops->vhost_set_log_base(dev, log_base);
> > +r = dev->vhost_ops->vhost_set_log_base(dev, log_base, log);
> >  assert(r >= 0);
> >  vhost_log_put(dev, true);
> >  dev->log = log;
> > @@ -1167,7 +1167,8 @@ int vhost_dev_start(struct vhost_dev *hdev,
> > VirtIODevice *vdev)
> >  hdev->log = vhost_log_get(hdev->log_size, share);
> >  log_base = (uintptr_t)hdev->log->log;
> >  r = hdev->vhost_ops->vhost_set_log_base(hdev,
> > -hdev->log_size ? log_base
> > : 0);
> > +hdev->log_size ? log_base
> > : 0,
> > +hdev->log);
> >  if (r < 0) {
> >  r = -errno;
> >  goto fail_log;
> > diff --git a/include/hw/virtio/vhost-backend.h
> > b/include/hw/virtio/vhost-backend.h
> > index 42cfb87..a87f0b3 100644
> > --- a/include/hw/virtio/vhost-backend.h
> > +++ b/include/hw/virtio/vhost-backend.h
> > @@ -20,6 +20,7 @@ typedef enum VhostBackendType {
> >  
> >  struct vhost_dev;
> >  struct vhost_memory;
> > +struct vhost_log;
> >  struct vhost_vring_file;
> >  struct vhost_vring_state;
> >  struct vhost_vring_addr;
> > @@ -36,7 +37,8 @@ typedef int (*vhost_scsi_clear_endpoint_op)(struct
> > vhost_dev *dev,
> > 

Re: [Qemu-devel] [PATCH 0/8] target-i386: Implement debug extensions

[Eduardo Habkost]

On Mon, Sep 21, 2015 at 02:05:52PM +0200, Paolo Bonzini wrote:
> On 15/09/2015 20:45, Richard Henderson wrote:
> > Best guess, since I can't find any code that actually uses them.
> > Linux actively turns them off at boot...
> 
> I've sent a kvm-unit-tests patch to test debug extensions.  It shows
> that debug extensions work, but the following needs to be squashed in
> patch 4:
> 
> diff --git a/target-i386/bpt_helper.c b/target-i386/bpt_helper.c
> index c258598..b24e446 100644
> --- a/target-i386/bpt_helper.c
> +++ b/target-i386/bpt_helper.c
> @@ -134,14 +134,14 @@ void cpu_x86_update_dr7(CPUX86State *env, uint32_t 
> new_dr7)
>  int mod = ((old_dr7 | old_dr7 * 2) ^ (new_dr7 | new_dr7 * 2)) & 0xff;
>  
>  for (i = 0; i < DR7_MAX_BP; i++) {
> -if (mod & (2 << i * 2)) {
> -/* We know that register i has changed enable state;
> -   recheck what that state should be and apply.  */
> -if (hw_breakpoint_enabled(new_dr7, i)) {
> -iobpt |= hw_breakpoint_insert(env, i);
> -} else {
> -hw_breakpoint_remove(env, i);
> -}
> +if ((mod & (2 << i * 2)) && !hw_breakpoint_enabled(new_dr7, i)) {
> +hw_breakpoint_remove(env, i);
> +}
> +}
> +env->dr[7] = new_dr7 | DR7_FIXED_1;
> +for (i = 0; i < DR7_MAX_BP; i++) {
> +if (mod & (2 << i * 2) && hw_breakpoint_enabled(new_dr7, i)) {
> +iobpt |= hw_breakpoint_insert(env, i);
>  } else if (hw_breakpoint_type(new_dr7, i) == DR7_TYPE_IO_RW
> && hw_breakpoint_enabled(new_dr7, i)) {
>  iobpt |= HF_IOBPT_MASK;

Thanks! If that's the only change needed in the series, I can squash it
when applying. I just need your Signed-off-by line.

-- 
Eduardo

Re: [Qemu-devel] [PATCH v4 09/22] vhost: use a function for each call

[Marc-André Lureau]

- Original Message -
> On Sat, Sep 19, 2015 at 12:12:00PM +0200, marcandre.lur...@redhat.com wrote:
> > From: Marc-André Lureau 
> > 
> > Replace the generic vhost_call() by specific functions for each
> > function call to help with type safety and changing arguments.
> > 
> > Signed-off-by: Marc-André Lureau 
> 
> OK but I would rather not make logging feature depend on the
> refactoring.

The refactoring is straightforward, but I can split this patch.

> How about moving each call over from vhost_call by a separate patch?
> Start with vhost_set_log_base as the 1st one, this way I can apply
> logging support while we still review the larger refactorings.
> 

ok

> > ---
> >  hw/net/vhost_net.c|  12 +-
> >  hw/scsi/vhost-scsi.c  |   7 +-
> >  hw/virtio/vhost-backend.c | 140 +++--
> >  hw/virtio/vhost-user.c| 402
> >  ++
> >  hw/virtio/vhost.c |  34 ++--
> >  include/hw/virtio/vhost-backend.h |  59 +-
> >  6 files changed, 484 insertions(+), 170 deletions(-)
> > 
> > diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
> > index 9d32d76..d116fb3 100644
> > --- a/hw/net/vhost_net.c
> > +++ b/hw/net/vhost_net.c
> > @@ -243,8 +243,7 @@ static int vhost_net_start_one(struct vhost_net *net,
> >  file.fd = net->backend;
> >  for (file.index = 0; file.index < net->dev.nvqs; ++file.index) {
> >  const VhostOps *vhost_ops = net->dev.vhost_ops;
> > -r = vhost_ops->vhost_call(&net->dev, VHOST_NET_SET_BACKEND,
> > -  &file);
> > +r = vhost_ops->vhost_net_set_backend(&net->dev, &file);
> >  if (r < 0) {
> >  r = -errno;
> >  goto fail;
> > @@ -257,8 +256,7 @@ fail:
> >  if (net->nc->info->type == NET_CLIENT_OPTIONS_KIND_TAP) {
> >  while (file.index-- > 0) {
> >  const VhostOps *vhost_ops = net->dev.vhost_ops;
> > -int r = vhost_ops->vhost_call(&net->dev,
> > VHOST_NET_SET_BACKEND,
> > -  &file);
> > +int r = vhost_ops->vhost_net_set_backend(&net->dev, &file);
> >  assert(r >= 0);
> >  }
> >  }
> > @@ -280,15 +278,13 @@ static void vhost_net_stop_one(struct vhost_net *net,
> >  if (net->nc->info->type == NET_CLIENT_OPTIONS_KIND_TAP) {
> >  for (file.index = 0; file.index < net->dev.nvqs; ++file.index) {
> >  const VhostOps *vhost_ops = net->dev.vhost_ops;
> > -int r = vhost_ops->vhost_call(&net->dev,
> > VHOST_NET_SET_BACKEND,
> > -  &file);
> > +int r = vhost_ops->vhost_net_set_backend(&net->dev, &file);
> >  assert(r >= 0);
> >  }
> >  } else if (net->nc->info->type == NET_CLIENT_OPTIONS_KIND_VHOST_USER)
> >  {
> >  for (file.index = 0; file.index < net->dev.nvqs; ++file.index) {
> >  const VhostOps *vhost_ops = net->dev.vhost_ops;
> > -int r = vhost_ops->vhost_call(&net->dev, VHOST_RESET_OWNER,
> > -  NULL);
> > +int r = vhost_ops->vhost_reset_owner(&net->dev);
> >  assert(r >= 0);
> >  }
> >  }
> > diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c
> > index bac9ddb..a0034ab 100644
> > --- a/hw/scsi/vhost-scsi.c
> > +++ b/hw/scsi/vhost-scsi.c
> > @@ -45,7 +45,7 @@ static int vhost_scsi_set_endpoint(VHostSCSI *s)
> >  
> >  memset(&backend, 0, sizeof(backend));
> >  pstrcpy(backend.vhost_wwpn, sizeof(backend.vhost_wwpn),
> >  vs->conf.wwpn);
> > -ret = vhost_ops->vhost_call(&s->dev, VHOST_SCSI_SET_ENDPOINT,
> > &backend);
> > +ret = vhost_ops->vhost_scsi_set_endpoint(&s->dev, &backend);
> >  if (ret < 0) {
> >  return -errno;
> >  }
> > @@ -60,7 +60,7 @@ static void vhost_scsi_clear_endpoint(VHostSCSI *s)
> >  
> >  memset(&backend, 0, sizeof(backend));
> >  pstrcpy(backend.vhost_wwpn, sizeof(backend.vhost_wwpn),
> >  vs->conf.wwpn);
> > -vhost_ops->vhost_call(&s->dev, VHOST_SCSI_CLEAR_ENDPOINT, &backend);
> > +vhost_ops->vhost_scsi_clear_endpoint(&s->dev, &backend);
> >  }
> >  
> >  static int vhost_scsi_start(VHostSCSI *s)
> > @@ -76,8 +76,7 @@ static int vhost_scsi_start(VHostSCSI *s)
> >  return -ENOSYS;
> >  }
> >  
> > -ret = vhost_ops->vhost_call(&s->dev,
> > -VHOST_SCSI_GET_ABI_VERSION, &abi_version);
> > +ret = vhost_ops->vhost_scsi_get_abi_version(&s->dev, &abi_version);
> >  if (ret < 0) {
> >  return -errno;
> >  }
> > diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c
> > index 4d68a27..bf2d1d4 100644
> > --- a/hw/virtio/vhost-backend.c
> > +++ b/hw/virtio/vhost-backend.c
> > @@ -11,19 +11,10 @@
> >  #include "hw/virtio/vhost.h"
> >  #include "hw/virtio/vhost-backend.h"
> >  #inclu

[Qemu-devel] [PATCH] vmxnet3: Support reading IMR registers on bar0

[Shmulik Ladkani]

Instead of asserting, return the actual IMR register value.
This is aligned with what's returned on ESXi.

Signed-off-by: Shmulik Ladkani 
Tested-by: Dana Rubin 

---
 hw/net/vmxnet3.c | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/hw/net/vmxnet3.c b/hw/net/vmxnet3.c
index 04159c8..97f19dc 100644
--- a/hw/net/vmxnet3.c
+++ b/hw/net/vmxnet3.c
@@ -1165,9 +1165,13 @@ vmxnet3_io_bar0_write(void *opaque, hwaddr addr,
 static uint64_t
 vmxnet3_io_bar0_read(void *opaque, hwaddr addr, unsigned size)
 {
+VMXNET3State *s = opaque;
+
 if (VMW_IS_MULTIREG_ADDR(addr, VMXNET3_REG_IMR,
 VMXNET3_MAX_INTRS, VMXNET3_REG_ALIGN)) {
-g_assert_not_reached();
+int l = VMW_MULTIREG_IDX_BY_ADDR(addr, VMXNET3_REG_IMR,
+ VMXNET3_REG_ALIGN);
+return s->interrupt_states[l].is_masked;
 }
 
 VMW_CBPRN("BAR0 unknown read [%" PRIx64 "], size %d", addr, size);
-- 
1.9.1

Re: [Qemu-devel] [PATCH 0/8] target-i386: Implement debug extensions

[Paolo Bonzini]

On 21/09/2015 16:05, Eduardo Habkost wrote:
> On Mon, Sep 21, 2015 at 02:05:52PM +0200, Paolo Bonzini wrote:
>> On 15/09/2015 20:45, Richard Henderson wrote:
>>> Best guess, since I can't find any code that actually uses them.
>>> Linux actively turns them off at boot...
>>
>> I've sent a kvm-unit-tests patch to test debug extensions.  It shows
>> that debug extensions work, but the following needs to be squashed in
>> patch 4:
>>
>> diff --git a/target-i386/bpt_helper.c b/target-i386/bpt_helper.c
>> index c258598..b24e446 100644
>> --- a/target-i386/bpt_helper.c
>> +++ b/target-i386/bpt_helper.c
>> @@ -134,14 +134,14 @@ void cpu_x86_update_dr7(CPUX86State *env, uint32_t 
>> new_dr7)
>>  int mod = ((old_dr7 | old_dr7 * 2) ^ (new_dr7 | new_dr7 * 2)) & 
>> 0xff;
>>  
>>  for (i = 0; i < DR7_MAX_BP; i++) {
>> -if (mod & (2 << i * 2)) {
>> -/* We know that register i has changed enable state;
>> -   recheck what that state should be and apply.  */
>> -if (hw_breakpoint_enabled(new_dr7, i)) {
>> -iobpt |= hw_breakpoint_insert(env, i);
>> -} else {
>> -hw_breakpoint_remove(env, i);
>> -}
>> +if ((mod & (2 << i * 2)) && !hw_breakpoint_enabled(new_dr7, i)) 
>> {
>> +hw_breakpoint_remove(env, i);
>> +}
>> +}
>> +env->dr[7] = new_dr7 | DR7_FIXED_1;
>> +for (i = 0; i < DR7_MAX_BP; i++) {
>> +if (mod & (2 << i * 2) && hw_breakpoint_enabled(new_dr7, i)) {
>> +iobpt |= hw_breakpoint_insert(env, i);
>>  } else if (hw_breakpoint_type(new_dr7, i) == DR7_TYPE_IO_RW
>> && hw_breakpoint_enabled(new_dr7, i)) {
>>  iobpt |= HF_IOBPT_MASK;
> 
> Thanks! If that's the only change needed in the series, I can squash it
> when applying. I just need your Signed-off-by line.

Signed-off-by: Paolo Bonzini 

:)

but perhaps wait a little for Richard to chime in.

Paolo

Re: [Qemu-devel] [PATCH] hmp-commands-info.hx: fix drops hmp "info pci"

[Markus Armbruster]

Paolo Bonzini  writes:

> On 21/09/2015 07:58, Markus Armbruster wrote:
>> Pavel Butsykin  writes:
>> 
>>> On 18.09.2015 19:26, Eric Blake wrote:
 On 09/18/2015 10:06 AM, Pavel Butsykin wrote:
> The hmp command "info pci" accidentally lost when moving from monitor.c
>
> Signed-off-by: Pavel Butsykin 
> ---
>   hmp-commands-info.hx | 14 ++
>   1 file changed, 14 insertions(+)
 Paolo's variant has a better commit message:
 https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg04893.html

>>> Ok. I just had to respond to the situation)
>> 
>> Appreciated!  It's much better to have two fixes for a regression
>> quickly than one late :)
>
> Are you pulling one of the two?

Can do.  I'll reply to the one I apply to my monitor-next branch.

Re: [Qemu-devel] [PATCH v2 4/4] target-i386: Call cpu_exec_init() on realize

[Eduardo Habkost]

On Mon, Sep 21, 2015 at 11:11:47AM +0530, Bharata B Rao wrote:
> On Sat, Sep 19, 2015 at 1:08 AM, Eduardo Habkost  wrote:
> > QOM instance_init functions are not supposed to have any side-effects,
> > as new objects may be created at any moment for querying property
> > information (see qmp_device_list_properties()).
> >
> > Calling cpu_exec_init() also affects QEMU's ability to handle errors
> > during CPU creation, as some actions done by cpu_exec_init() can't be
> > reverted.
> >
> > Move cpu_exec_init() call to realize so a simple object_new() won't
> > trigger it, and so that it is called after some basic validation of CPU
> > parameters.
> 
> Since you are moving cpu_exec_init() to realize, does it make sense to
> define unrealize and call cpu_exec_exit() from it ?

It does make sense. But it needs to be done more carefully because
currently cpu_exec_exit() is likely to make QEMU crash, and calling it
from unrealize would make the crash triggerable using a QMP qom-set
command.

-- 
Eduardo

Re: [Qemu-devel] [PATCH V4 2/2] sdhci: Split sdhci.h for public and internal device usage

[Peter Crosthwaite]

On Mon, Sep 14, 2015 at 2:26 AM, Sai Pavan Boddu
 wrote:
> Split sdhci.h into sdhci-common.h(pubilc Version in include/) and
> sdhci.h(internal version in hw/sd) base on register declarations and
> object declaration.
>
> Signed-off-by: Sai Pavan Boddu 
> ---
> Changes for V4:
> Remain the name of internal version of sdchi.h as same. And change
> Re-Adding qemu-common.h header.
> the one which is in includes/ to sdhci-common.h
> Changes for V2:
> Create new area in includes for sd. And move sdhci.h to same.
> Changes for V3:
> Split the headers to public and common.
> ---
>  hw/sd/sdhci.c|  1 -
>  hw/sd/sdhci.h| 67 +---
>  include/hw/sd/sdhci-common.h | 92 
> 

I think the split is good (and needed for SDHCI SoCification) but the
names are inconsistent. The public header should trump the private
when it comes to who gets the name "sdhci.h". So we should have
include/hw/sd/sdhci.h with the SoC embeddable and
hw/sd/sdhci-internal.h for the implementation private content. Check
target-arm/cpu-internal.h vs target-arm/cpu.h for a similar concept.

Regards,
Peter

>  3 files changed, 94 insertions(+), 66 deletions(-)
>  create mode 100644 include/hw/sd/sdhci-common.h
>
> diff --git a/hw/sd/sdhci.c b/hw/sd/sdhci.c
> index e63367b..4860f41 100644
> --- a/hw/sd/sdhci.c
> +++ b/hw/sd/sdhci.c
> @@ -28,7 +28,6 @@
>  #include "sysemu/dma.h"
>  #include "qemu/timer.h"
>  #include "qemu/bitops.h"
> -
>  #include "sdhci.h"
>
>  /* host controller debug messages */
> diff --git a/hw/sd/sdhci.h b/hw/sd/sdhci.h
> index a45593f..667c60c 100644
> --- a/hw/sd/sdhci.h
> +++ b/hw/sd/sdhci.h
> @@ -21,14 +21,10 @@
>   * You should have received a copy of the GNU General Public License along
>   * with this program; if not, see .
>   */
> -
>  #ifndef SDHCI_H
>  #define SDHCI_H
>
> -#include "qemu-common.h"
> -#include "hw/pci/pci.h"
> -#include "hw/sysbus.h"
> -#include "hw/sd/sd.h"
> +#include "hw/sd/sdhci-common.h"
>
>  /* R/W SDMA System Address register 0x0 */
>  #define SDHC_SYSAD 0x00
> @@ -231,65 +227,6 @@ enum {
>  sdhc_gap_write  = 2   /* SDHC stopped at block gap during write 
> operation */
>  };
>
> -/* SD/MMC host controller state */
> -typedef struct SDHCIState {
> -union {
> -PCIDevice pcidev;
> -SysBusDevice busdev;
> -};
> -SDState *card;
> -MemoryRegion iomem;
> -
> -QEMUTimer *insert_timer;   /* timer for 'changing' sd card. */
> -QEMUTimer *transfer_timer;
> -qemu_irq eject_cb;
> -qemu_irq ro_cb;
> -qemu_irq irq;
> -
> -uint32_t sdmasysad;/* SDMA System Address register */
> -uint16_t blksize;  /* Host DMA Buff Boundary and Transfer BlkSize 
> Reg */
> -uint16_t blkcnt;   /* Blocks count for current transfer */
> -uint32_t argument; /* Command Argument Register */
> -uint16_t trnmod;   /* Transfer Mode Setting Register */
> -uint16_t cmdreg;   /* Command Register */
> -uint32_t rspreg[4];/* Response Registers 0-3 */
> -uint32_t prnsts;   /* Present State Register */
> -uint8_t  hostctl;  /* Host Control Register */
> -uint8_t  pwrcon;   /* Power control Register */
> -uint8_t  blkgap;   /* Block Gap Control Register */
> -uint8_t  wakcon;   /* WakeUp Control Register */
> -uint16_t clkcon;   /* Clock control Register */
> -uint8_t  timeoutcon;   /* Timeout Control Register */
> -uint8_t  admaerr;  /* ADMA Error Status Register */
> -uint16_t norintsts;/* Normal Interrupt Status Register */
> -uint16_t errintsts;/* Error Interrupt Status Register */
> -uint16_t norintstsen;  /* Normal Interrupt Status Enable Register */
> -uint16_t errintstsen;  /* Error Interrupt Status Enable Register */
> -uint16_t norintsigen;  /* Normal Interrupt Signal Enable Register */
> -uint16_t errintsigen;  /* Error Interrupt Signal Enable Register */
> -uint16_t acmd12errsts; /* Auto CMD12 error status register */
> -uint64_t admasysaddr;  /* ADMA System Address Register */
> -
> -uint32_t capareg;  /* Capabilities Register */
> -uint32_t maxcurr;  /* Maximum Current Capabilities Register */
> -uint8_t  *fifo_buffer; /* SD host i/o FIFO buffer */
> -uint32_t buf_maxsz;
> -uint16_t data_count;   /* current element in FIFO buffer */
> -uint8_t  stopped_state;/* Current SDHC state */
> -/* Buffer Data Port Register - virtual access point to R and W buffers */
> -/* Software Reset Register - always reads as 0 */
> -/* Force Event Auto CMD12 Error Interrupt Reg - write only */
> -/* Force Event Error Interrupt Register- write only */
> -/* RO Host Controller Version Register always reads as 0x2401 */
> -} SDHCIState;
> -
>  extern const VMStateDescription sdhci_vmstate;
>
> -#define TYPE_PCI_SDHCI "sdhci-pci

Re: [Qemu-devel] [PATCH v2 0/2] Dynamic module support for block drivers

[Marc Marí]

On Mon, 21 Sep 2015 16:43:27 +0300
"Denis V. Lunev"  wrote:

> On 09/08/2015 04:53 PM, Marc Marí wrote:
> > The current module infrastructure has been improved to enable
> > dynamic module loading.
> >
> > This reduces the load time for very simple guests. For the following
> > configuration (very loaded)
> >
> > ./configure --enable-sdl --enable-gtk --enable-vte --enable-curses \
> >  --enable-vnc --enable-vnc-{jpeg,tls,sasl,png} --enable-virtfs \
> >  --enable-brlapi --enable-curl --enable-fdt --enable-bluez \
> >  --enable-kvm --enable-rdma --enable-uuid --enable-vde \
> >  --enable-linux-aio --enable-cap-ng --enable-attr
> > --enable-vhost-net \ --enable-vhost-scsi --enable-spice
> > --enable-rbd --enable-libiscsi \ --enable-smartcard-nss
> > --enable-guest-agent --enable-libusb \ --enable-usb-redir
> > --enable-lzo --enable-snappy --enable-bzip2 \ --enable-seccomp
> > --enable-coroutine-pool --enable-glusterfs \ --enable-tpm
> > --enable-libssh2 --enable-vhdx --enable-numa \ --enable-tcmalloc
> > --target-list=x86_64-softmmu
> >
> > With modules disabled, there are 142 libraries loaded at startup.
> > Time is the following:
> >   LD time: 0.065 seconds
> >   QEMU time: 0.02 seconds
> >   Total time: 0.085 seconds
> >
> > With this patch series and modules enabled, there are 128 libraries
> > loaded at startup. Time is the following:
> >   LD time: 0.02 seconds
> >   QEMU time: 0.02 seconds
> >   Total time: 0.04 seconds
> >
> > Where LD time is the time between the program startup and the jump
> > to main, and QEMU time is the time between the start of main and
> > the first kvm_entry.
> >
> > These results are just with a few block drivers, that were already
> > a module. Adding more modules (block or not block) should be easy,
> > and will reduce the load time even more.
> >
> > Marc Marí (2):
> >Add dynamic module loading for block drivers
> >Add dynamic generation of module_block.h
> >
> >   .gitignore  |   1 +
> >   Makefile|  10 ++-
> >   block.c |  70 +
> >   configure   |   2 +-
> >   include/qemu/module.h   |   3 +
> >   scripts/modules/module_block.py | 134
> > 
> > util/module.c   |  38  7 files changed,
> > 227 insertions(+), 31 deletions(-) create mode 100755
> > scripts/modules/module_block.py
> >
> 
>  From my point of view the design looks a bit complex.
> The approach should be quite similar to one used
> in Linux kernel.
> 
> If the block driver is configured as a module, block_init
> should register proper hooks and create generic lists.
> C code parsing does not look like a good approach
> to me.
> 
> If block_init is a bad name, we could use something like
> module_init.
> 

I think applying the kind of modules of the Linux kernel here would
mean reworking an important part of the drivers that want to be
converted into modules.

If that's the case, I think it's better to have a less good solution
that is still good enough, and doesn't mean reworking half of the QEMU
codebase. Correct me if I'm wrong.

But of course, this is not the only valid approach.

Thanks
Marc

Re: [Qemu-devel] [PATCH v2 0/2] Dynamic module support for block drivers

[Denis V. Lunev]

On 09/21/2015 05:44 PM, Marc Marí wrote:

On Mon, 21 Sep 2015 16:43:27 +0300
"Denis V. Lunev"  wrote:


On 09/08/2015 04:53 PM, Marc Marí wrote:

The current module infrastructure has been improved to enable
dynamic module loading.

This reduces the load time for very simple guests. For the following
configuration (very loaded)

./configure --enable-sdl --enable-gtk --enable-vte --enable-curses \
  --enable-vnc --enable-vnc-{jpeg,tls,sasl,png} --enable-virtfs \
  --enable-brlapi --enable-curl --enable-fdt --enable-bluez \
  --enable-kvm --enable-rdma --enable-uuid --enable-vde \
  --enable-linux-aio --enable-cap-ng --enable-attr
--enable-vhost-net \ --enable-vhost-scsi --enable-spice
--enable-rbd --enable-libiscsi \ --enable-smartcard-nss
--enable-guest-agent --enable-libusb \ --enable-usb-redir
--enable-lzo --enable-snappy --enable-bzip2 \ --enable-seccomp
--enable-coroutine-pool --enable-glusterfs \ --enable-tpm
--enable-libssh2 --enable-vhdx --enable-numa \ --enable-tcmalloc
--target-list=x86_64-softmmu

With modules disabled, there are 142 libraries loaded at startup.
Time is the following:
   LD time: 0.065 seconds
   QEMU time: 0.02 seconds
   Total time: 0.085 seconds

With this patch series and modules enabled, there are 128 libraries
loaded at startup. Time is the following:
   LD time: 0.02 seconds
   QEMU time: 0.02 seconds
   Total time: 0.04 seconds

Where LD time is the time between the program startup and the jump
to main, and QEMU time is the time between the start of main and
the first kvm_entry.

These results are just with a few block drivers, that were already
a module. Adding more modules (block or not block) should be easy,
and will reduce the load time even more.

Marc Marí (2):
Add dynamic module loading for block drivers
Add dynamic generation of module_block.h

   .gitignore  |   1 +
   Makefile|  10 ++-
   block.c |  70 +
   configure   |   2 +-
   include/qemu/module.h   |   3 +
   scripts/modules/module_block.py | 134

util/module.c   |  38  7 files changed,
227 insertions(+), 31 deletions(-) create mode 100755
scripts/modules/module_block.py


  From my point of view the design looks a bit complex.
The approach should be quite similar to one used
in Linux kernel.

If the block driver is configured as a module, block_init
should register proper hooks and create generic lists.
C code parsing does not look like a good approach
to me.

If block_init is a bad name, we could use something like
module_init.


I think applying the kind of modules of the Linux kernel here would
mean reworking an important part of the drivers that want to be
converted into modules.

If that's the case, I think it's better to have a less good solution
that is still good enough, and doesn't mean reworking half of the QEMU
codebase. Correct me if I'm wrong.

But of course, this is not the only valid approach.

Thanks
Marc

Frankly speaking this does not look to me like a very big deal
and a huge rework. Build system should create 2 object files
for a modularized driver:
- one with driver stub which has required properties aka has_probe, name etc
  and a bit meaning that the driver is not present
- one with driver code
and from my point of view changes will not be big with this
approach.

Anyway, this is my opinion about this stuff. That is all, I not
against your change, I would like to note that there is a better
way which will allow further improvements.

Den