Re: [Qemu-devel] [PATCH 3/7] device-introspect-test: New, covering device introspection

[Markus Armbruster]

Eric Blake  writes:

> On 09/18/2015 06:00 AM, Markus Armbruster wrote:
>> The test doesn't check the output makes any sense, only that QEMU
>
> Reads slightly better as:
> s/check/check that/

Okay.

>> survives.  Useful since we've had an astounding number of crash bugs
>> around there.
>> 
>> In fact, we have a bunch of them right now: several devices crash or
>> hang, and all CPUs leave a dangling pointer behind.  Blacklist them in
>> the test.  The next commits will fix.
>> 
>> Signed-off-by: Markus Armbruster 
>> ---
>>  tests/Makefile |  11 ++-
>>  tests/device-introspect-test.c | 149 
>> +
>>  2 files changed, 157 insertions(+), 3 deletions(-)
>>  create mode 100644 tests/device-introspect-test.c
>> 
>
>> @@ -478,8 +483,8 @@ $(patsubst %, check-qtest-%, $(QTEST_TARGETS)): 
>> check-qtest-%: $(check-qtest-y)
>>  $(call quiet-command,QTEST_QEMU_BINARY=$*-softmmu/qemu-system-$* \
>>  QTEST_QEMU_IMG=qemu-img$(EXESUF) \
>>  MALLOC_PERTURB_=$${MALLOC_PERTURB_:-$$((RANDOM % 255 + 1))} \
>> -gtester $(GTESTER_OPTIONS) -m=$(SPEED) 
>> $(check-qtest-$*-y),"GTESTER $@")
>> -$(if $(CONFIG_GCOV),@for f in $(gcov-files-$*-y); do \
>> +gtester $(GTESTER_OPTIONS) -m=$(SPEED) $(check-qtest-generic-y) 
>> $(check-qtest-$*-y),"GTESTER $@")
>
> Long line; worth adding a \ line-wrap?

I'll look into it.

>> +++ b/tests/device-introspect-test.c
>> @@ -0,0 +1,149 @@
>
>> +/*
>> + * Covers QMP device-list-properties and HMP device_add help.  We
>> + * currently don't check their output makes sense, only that QEMU
>
> again, might be better as:
> s/check/check that/

Okay.

>> +static void test_one_device(const char *type)
>> +{
>> +QDict *resp;
>> +char *help;
>> +
>> +/* FIXME device-list-properties crashes for abstract device, skip  */
>> +if (strcmp(type, "device")) {
>> +resp = qmp("{'execute': 'device-list-properties',"
>> +   " 'arguments': {'typename': %s}}",
>> +   type);
>> +QDECREF(resp);
>> +}
>> +
>> +help = hmp("device_add \"%s,help\"", type);
>> +g_free(help);
>
> The comment mentions a skip, and the commit message header mentioned a
> blacklist, but I'm not seeing a skip here. Am I missing something?

The conditional skips part of the test when type is "device", because
that part crashes.

>> +
>> +static bool blacklisted(const char *type)
>> +{
>> +static const char *blacklist[] = {
>> +/* crash in object_unref(): */
>> +"pxa2xx-pcmcia",
>> +/* hang in object_unref(): */
>> +"realview_pci", "versatile_pci", "s390-sclp-event-facility", "sclp",
>> +/* create a CPU, thus use after free (see below): */
>> +"allwinner-a10", "digic", "fsl,imx25", "fsl,imx31", "xlnx,zynqmp",
>> +};
>
> Okay, here's the blacklist the commit message mentioned, but maybe that
> means the earlier comment is stale?

The commit message is slightly inaccurate: unlike the other devices,
"device" isn't blacklisted completely.  Good enough?

Re: [Qemu-devel] [PATCH 5/7] qdev: Protect device-list-properties against broken devices

[Markus Armbruster]

Eduardo Habkost  writes:

> On Fri, Sep 18, 2015 at 02:00:38PM +0200, Markus Armbruster wrote:
>> Several devices don't survive object_unref(object_new(T)): they crash
>> or hang during cleanup, or they leave dangling pointers behind.
>> 
>> This breaks at least device-list-properties, because
>> qmp_device_list_properties() needs to create a device to find its
>> properties.  Broken in commit f4eb32b "qmp: show QOM properties in
>> device-list-properties", v2.1.  Example reproducer:
>> 
>> $ qemu-system-aarch64 -nodefaults -display none -machine none -S -qmp 
>> stdio
>> {"QMP": {"version": {"qemu": {"micro": 50, "minor": 4, "major": 2}, 
>> "package": ""}, "capabilities": []}}
>> { "execute": "qmp_capabilities" }
>> {"return": {}}
>> { "execute": "device-list-properties", "arguments": { "typename": 
>> "pxa2xx-pcmcia" } }
>> qemu-system-aarch64: /home/armbru/work/qemu/memory.c:1307: 
>> memory_region_finalize: Assertion `((>subregions)->tqh_first == ((void 
>> *)0))' failed.
>> Aborted (core dumped)
>> [Exit 134 (SIGABRT)]
>> 
>> Unfortunately, I can't fix the problems in these devices right now.
>> Instead, add DeviceClass member cannot_even_create_with_object_new_yet
>> to mark them:
>> 
>> * Crash or hang during cleanup (didn't debug them, so I can't say
>>   why): "pxa2xx-pcmcia", "realview_pci", "versatile_pci",
>>   "s390-sclp-event-facility", "sclp"
>> 
>> * Dangling pointers: all CPUs, plus "allwinner-a10", "digic",
>>   "fsl,imx25", "fsl,imx31", "xlnx,zynqmp", because they create CPUs
>
> That's isn't true for all CPU classes, only the ones that (incorrectly)
> call cpu_exec_init() on instance_init instead of realize. I believe at
> least TYPE_POWERPC_CPU is safe already.

Okay, I'll try to mark only the ones that actually screw up.

Thanks!

Re: [Qemu-devel] [PATCH 2/7] libqtest: New hmp() & friends

[Markus Armbruster]

Eric Blake  writes:

> On 09/18/2015 06:00 AM, Markus Armbruster wrote:
>> New convenience function hmp() to facilitate use of
>> human-monitor-command in tests.  Use it to simplify its existing uses.
>> 
>> To blend into existing libqtest code, also add qtest_hmpv() and
>> qtest_hmp().  That, and the egregiously verbose GTK-Doc comment format
>> make this patch look bigger than it is.
>> 
>> Signed-off-by: Markus Armbruster 
>> ---
>>  tests/drive_del-test.c | 22 ++
>>  tests/ide-test.c   |  8 ++--
>>  tests/libqtest.c   | 35 +++
>>  tests/libqtest.h   | 33 +
>>  4 files changed, 76 insertions(+), 22 deletions(-)
>> 
>
>
>> @@ -774,6 +801,14 @@ void qmp_discard_response(const char *fmt, ...)
>>  qtest_qmpv_discard_response(global_qtest, fmt, ap);
>>  va_end(ap);
>>  }
>> +char *hmp(const char *fmt, ...)
>> +{
>> +va_list ap;
>> +
>> +va_start(ap, fmt);
>> +return qtest_hmpv(global_qtest, fmt, ap);
>> +va_end(ap);
>
> Umm, that isn't quite what you meant :)

D'oh!

> With the dead code after return fixed,
>
> Reviewed-by: Eric Blake 

Thanks!

Re: [Qemu-devel] [PATCH v4] ppc/spapr: Implement H_RANDOM hypercall in QEMU

[Thomas Huth]

On 21/09/15 04:10, David Gibson wrote:
> On Fri, Sep 18, 2015 at 11:05:52AM +0200, Greg Kurz wrote:
>> On Thu, 17 Sep 2015 10:49:41 +0200
>> Thomas Huth  wrote:
>>
>>> The PAPR interface defines a hypercall to pass high-quality
>>> hardware generated random numbers to guests. Recent kernels can
>>> already provide this hypercall to the guest if the right hardware
>>> random number generator is available. But in case the user wants
>>> to use another source like EGD, or QEMU is running with an older
>>> kernel, we should also have this call in QEMU, so that guests that
>>> do not support virtio-rng yet can get good random numbers, too.
>>>
>>> This patch now adds a new pseudo-device to QEMU that either
>>> directly provides this hypercall to the guest or is able to
>>> enable the in-kernel hypercall if available. The in-kernel
>>> hypercall can be enabled with the use-kvm property, e.g.:
>>>
>>>  qemu-system-ppc64 -device spapr-rng,use-kvm=true
>>>
>>> For handling the hypercall in QEMU instead, a "RngBackend" is
>>> required since the hypercall should provide "good" random data
>>> instead of pseudo-random (like from a "simple" library function
>>> like rand() or g_random_int()). Since there are multiple RngBackends
>>> available, the user must select an appropriate back-end via the
>>> "rng" property of the device, e.g.:
>>>
>>>  qemu-system-ppc64 -object rng-random,filename=/dev/hwrng,id=gid0 \
>>>-device spapr-rng,rng=gid0 ...
>>>
>>> See http://wiki.qemu-project.org/Features-Done/VirtIORNG for
>>> other example of specifying RngBackends.
>>>
>>> Signed-off-by: Thomas Huth 
>>> ---
>>
>> It is a good thing that the user can choose between in-kernel and backend,
>> and this patch does the work.
>>
>> This being said, I am not sure about the use case where a user has a hwrng
>> capable platform and wants to run guests without any hwrng support at all is
>> an appropriate default behavior... I guess we will find more users that want
>> in-kernel being the default if it is available.
>>
>> The patch below modifies yours to do just this: the pseudo-device is only
>> created if hwrng is present and not already created.
> 
> I have mixed feelings about this.  On the one hand, I agree that it
> would be nice to allow H_RANDOM support by default.  On the other hand
> the patch below leaves no way to turn it off for testing purposes.  It
> also adds another place where the guest hardware depends on the host
> configuration, which adds to the already substantial mess of ensuring
> that source and destination hardware configuration matches for
> migration.

I thought about this question on the weekend and came to the same
conclusion. I think if we want to enable this by default, it likely
should rather be done at the libvirt level instead?

 Thomas





signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 5/7] qdev: Protect device-list-properties against broken devices

[Markus Armbruster]

Eric Blake  writes:

> On 09/18/2015 06:00 AM, Markus Armbruster wrote:
>> Several devices don't survive object_unref(object_new(T)): they crash
>> or hang during cleanup, or they leave dangling pointers behind.
>> 
>
>> Unfortunately, I can't fix the problems in these devices right now.
>> Instead, add DeviceClass member cannot_even_create_with_object_new_yet
>> to mark them:
>
> (intentionally) Ugly because it is a workaround, but then again, giving
> the attribute an ugly name will help call attention to the specific
> device owners to fix the mess.  I can live with that.

Named in Anthony's memory (see commit efec3dd) ;-P

>> This also protects -device FOO,help, which uses the same machinery
>> since commit ef52358 "qdev-monitor: include QOM properties in -device
>> FOO, help output", v2.2.  Example reproducer:
>> 
>> $ qemu-system-* -machine none -device pxa2xx-pcmcia,help
>> 
>> Before:
>> 
>> qemu-system-aarch64: .../memory.c:1307: memory_region_finalize:
>> Assertion `((>subregions)->tqh_first == ((void *)0))' failed.
>> 
>> After:
>> 
>> Can't list properties of device 'pxa2xx-pcmcia'
>
> Not ideal, but much better than a crash, so it gets my vote for
> inclusion as an incremental improvement.
>
>
>> +++ b/include/hw/qdev-core.h
>> @@ -114,6 +114,19 @@ typedef struct DeviceClass {
>>   * TODO remove once we're there
>>   */
>>  bool cannot_instantiate_with_device_add_yet;
>> +/*
>> + * Does this device model survive object_unref(object_new(TNAME))?
>> + * All device models should, and this flag shouldn't exist.  Some
>> + * devices crash in object_new(), some crash or hang in
>> + * object_unref().  Makes introspecting properties with
>> + * qmp_device_list_properties() dangerous.  Bad, because it's used
>> + * by -device FOO,help.  This flag serves to protect that code.
>> + * It should never be set without a comment explaining why it is
>> + * set.
>> + * TODO remove once we're there
>> + */
>> +bool cannot_even_create_with_object_new_yet;
>
> And a sufficiently verbose explanation for why the code is so ugly.
>
>> @@ -123,9 +97,6 @@ static void test_device_intro_concrete(void)
>>  type = qdict_get_try_str(qobject_to_qdict(qlist_entry_obj(entry)),
>>  "name");
>>  g_assert(type);
>> -if (blacklisted(type)) {
>> -continue;   /* FIXME broken device, skip */
>> -}
>
> The devices are still broken, but the testsuite no longer flags them as
> broken because it no longer crashes or hangs, and you intentionally
> wrote the test to treat any output (including a graceful error message)
> as successful use of the probe.  The ugly attribute is now our only
> documentation of the problems, but that is still something sufficient to
> hopefully get it fixed.
>
> Reviewed-by: Eric Blake 

Thanks!

Re: [Qemu-devel] [PATCH 5/7] qdev: Protect device-list-properties against broken devices

[Markus Armbruster]

Eduardo Habkost  writes:

> On Fri, Sep 18, 2015 at 08:42:54PM +0200, Thomas Huth wrote:
>> On 18/09/15 14:00, Markus Armbruster wrote:
>> > Several devices don't survive object_unref(object_new(T)): they crash
>> > or hang during cleanup, or they leave dangling pointers behind.
>> > 
>> > This breaks at least device-list-properties, because
>> > qmp_device_list_properties() needs to create a device to find its
>> > properties.  Broken in commit f4eb32b "qmp: show QOM properties in
>> > device-list-properties", v2.1.  Example reproducer:
>> > 
>> > $ qemu-system-aarch64 -nodefaults -display none -machine none -S -qmp 
>> > stdio
>> > {"QMP": {"version": {"qemu": {"micro": 50, "minor": 4, "major": 2}, 
>> > "package": ""}, "capabilities": []}}
>> > { "execute": "qmp_capabilities" }
>> > {"return": {}}
>> > { "execute": "device-list-properties", "arguments": { "typename": 
>> > "pxa2xx-pcmcia" } }
>> > qemu-system-aarch64: /home/armbru/work/qemu/memory.c:1307: 
>> > memory_region_finalize: Assertion `((>subregions)->tqh_first == ((void 
>> > *)0))' failed.
>> > Aborted (core dumped)
>> > [Exit 134 (SIGABRT)]
>> > 
>> > Unfortunately, I can't fix the problems in these devices right now.
>> > Instead, add DeviceClass member cannot_even_create_with_object_new_yet
>> > to mark them:
>> > 
>> > * Crash or hang during cleanup (didn't debug them, so I can't say
>> >   why): "pxa2xx-pcmcia", "realview_pci", "versatile_pci",
>> >   "s390-sclp-event-facility", "sclp"
>> > 
>> > * Dangling pointers: all CPUs, plus "allwinner-a10", "digic",
>> >   "fsl,imx25", "fsl,imx31", "xlnx,zynqmp", because they create CPUs
>> > 
>> > * Assert kvm_enabled(): "host-x86_64-cpu", host-i386-cpu",
>> >   "host-powerpc64-cpu", "host-embedded-powerpc-cpu",
>> >   "host-powerpc-cpu"
>> 
>> I just had a look at the powerpc code - you're likely talking about
>> the "assert(kvm_enabled());" in the kvmppc_host_cpu_initfn() in
>> target-ppc/kvm.c ? That should be fine, I think, because
>> kvm_ppc_register_host_cpu_type() is only done on ppc when KVM has been
>> enabled.

Easy to verify on a KVM-capable PPC host: try -device C,help with KVM on
and off, where C is the appropriate host CPU.

> It's true that currently the assert() will never trigger, but we will
> have to eventually move class registration to type_init if we want to
> make a generic query-cpu-definitions implementation work properly
> without depending on global machine accel configuration.

Good point.  These CPUs need a TODO marker.

> It won't hurt to set cannot_even_create_with_object_new_yet properly to
> reflect that the class isn't ready yet.

Marking cannot_even_create_with_object_new_yet() is the obvious TODO
marker.  It has an unnecessary side effect: we disable introspection for
these CPUs needlessly.  Tolerable?

Re: [Qemu-devel] [PATCH 7/7] tests: Simplify how qom-test is run

[Markus Armbruster]

Andreas Färber  writes:

> Am 18.09.2015 um 16:24 schrieb Markus Armbruster:
>> Andreas Färber  writes:
>>> Am 18.09.2015 um 14:00 schrieb Markus Armbruster:
 Add it to check-qtest-generic-y instead of check-qtest-$(target)-y for
 every target.

 Signed-off-by: Markus Armbruster 
 ---
  tests/Makefile | 5 +
  1 file changed, 1 insertion(+), 4 deletions(-)

 diff --git a/tests/Makefile b/tests/Makefile
 index 4559045..28c5f93 100644
 --- a/tests/Makefile
 +++ b/tests/Makefile
 @@ -219,10 +219,7 @@ gcov-files-ppc64-y += ppc64-softmmu/hw/ppc/spapr_pci.c
  check-qtest-microblazeel-y = $(check-qtest-microblaze-y)
  check-qtest-xtensaeb-y = $(check-qtest-xtensa-y)
  
 -# qom-test works for all sysemu architectures:
 -$(foreach target,$(SYSEMU_TARGET_LIST), \
 - $(if $(findstring tests/qom-test$(EXESUF),
 $(check-qtest-$(target)-y)),, \
 -  $(eval check-qtest-$(target)-y += tests/qom-test$(EXESUF
 +check-qtest-generic-y += tests/qom-test$(EXESUF)
>>>
>>> Does this -generic- have the same filtering code to avoid running the
>>> tests twice for x86_64, aarch64, ppc64, etc.? Please don't regress.
>> 
>> I'm dense today.  Can you explain the filtering code to me?
>
> For practical purpose,s x86_64 adds all tests from i386, that included
> qom-test then. If we now add it for x86_64 too, it got executed twice,
> which the above $(if ...) fixes by not adding it for x86_64 if it's
> already in. Just checking whether -generic- has equivalent filtering or
> other code somewhere else?

I'll double-check.  Thanks!

Re: [Qemu-devel] [Qemu-block] [PATCH] throttle: test that snapshots move the throttling configuration

[Alberto Garcia]

On Fri 18 Sep 2015 05:54:36 PM CEST, Max Reitz  wrote:

>>  create mode 100644 tests/qemu-iotests/096
>>  create mode 100644 tests/qemu-iotests/096.out
>
> Looks good, I'd just like to throw in that 096 is in use by my
> looks-dead-but-actually-is-not and
> only-waiting-for-the-blockbackend-and-media-series-to-get-merged series
> "block: Rework bdrv_close_all()":
> http://lists.nongnu.org/archive/html/qemu-block/2015-03/msg00048.html

Ah, I don't care about the number, if someone wants to change it when
applying my patch I have no problem with that.

Berto

Re: [Qemu-devel] [PATCH v5 14/38] block: Remove wr_highest_sector from BlockAcctStats

[Kevin Wolf]

Am 18.09.2015 um 18:59 hat Eric Blake geschrieben:
> On 09/18/2015 09:22 AM, Max Reitz wrote:
> > BlockAcctStats contains statistics about the data transferred from and
> > to the device; wr_highest_sector does not fit in with the rest.
> > 
> > Furthermore, those statistics are supposed to be specific for a certain
> > device and not necessarily for a BDS (see the comment above
> > bdrv_get_stats()); on the other hand, wr_highest_sector may be a rather
> > important information to know for each BDS. When BlockAcctStats is
> > finally removed from the BDS, we will want to keep wr_highest_sector in
> > the BDS.
> > 
> > Finally, wr_highest_sector is renamed to wr_highest_offset and given the
> > appropriate meaning. Externally, it is represented as an offset so there
> > is no point in doing something different internally. Its definition is
> > changed to match that in qapi/block-core.json which is "the offset after
> > the greatest byte written to". Doing so should not cause any harm since
> > if external programs tried to calculate the volume usage by
> > (wr_highest_offset + 512) / volume_size, after this patch they will just
> > assume the volume to be full slightly earlier than before.
> > 
> > Signed-off-by: Max Reitz 
> > Reviewed-by: Eric Blake 
> > Reviewed-by: Alberto Garcia 
> 
> Still holds, but...
> 
> > +++ b/qmp-commands.hx
> > @@ -2481,8 +2481,8 @@ Each json-object contain the following:
> >  - "wr_total_time_ns": total time spend on writes in nano-seconds 
> > (json-int)
> >  - "rd_total_time_ns": total time spend on reads in nano-seconds 
> > (json-int)
> >  - "flush_total_time_ns": total time spend on cache flushes in 
> > nano-seconds (json-int)
> > -- "wr_highest_offset": Highest offset of a sector written since the
> > -   BlockDriverState has been opened (json-int)
> > +- "wr_highest_offset": The offset after the greatest byte written to 
> > the
> > +   BlockDriverState since it has been opened 
> > (json-int)
> 
> ...someday, I'd really like to have this stat show as non-zero even when
> first opening the device (before writing to it). Right now, you have no
> clue how full a backing device is prior to starting a block-commit; you
> have to start writing to it to get a feel for its current usage.

With which value would it start? You don't want the file/device size
because for block devices that's more than is actually used yet.

What you really want to know is a number for the parent node, which
isn't readily available for qcow2 (you would have to scan the refcounts
in the last refcount block on startup) and doesn't really exist for most
other formats (very few of them can be used on block devices, most rely
on the file size).

Kevin


pgpJwnKFZPzag.pgp
Description: PGP signature

[Qemu-devel] [PULL 04/26] qapi: New QAPISchemaVisitor

[Markus Armbruster]

The visitor will help keeping the code generation code simple and
reasonably separated from QAPISchema details.

Signed-off-by: Markus Armbruster 
Message-Id: <1442401589-24189-5-git-send-email-arm...@redhat.com>
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 scripts/qapi.py | 64 +
 1 file changed, 64 insertions(+)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 401b87d..36e0702 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -772,6 +772,39 @@ class QAPISchemaEntity(object):
 def check(self, schema):
 pass
 
+def visit(self, visitor):
+pass
+
+
+class QAPISchemaVisitor(object):
+def visit_begin(self, schema):
+pass
+
+def visit_end(self):
+pass
+
+def visit_builtin_type(self, name, info, json_type):
+pass
+
+def visit_enum_type(self, name, info, values, prefix):
+pass
+
+def visit_array_type(self, name, info, element_type):
+pass
+
+def visit_object_type(self, name, info, base, members, variants):
+pass
+
+def visit_alternate_type(self, name, info, variants):
+pass
+
+def visit_command(self, name, info, arg_type, ret_type,
+  gen, success_response):
+pass
+
+def visit_event(self, name, info, arg_type):
+pass
+
 
 class QAPISchemaType(QAPISchemaEntity):
 def c_type(self, is_param=False):
@@ -818,6 +851,9 @@ class QAPISchemaBuiltinType(QAPISchemaType):
 def json_type(self):
 return self._json_type_name
 
+def visit(self, visitor):
+visitor.visit_builtin_type(self.name, self.info, self.json_type())
+
 
 class QAPISchemaEnumType(QAPISchemaType):
 def __init__(self, name, info, values, prefix):
@@ -841,6 +877,10 @@ class QAPISchemaEnumType(QAPISchemaType):
 def json_type(self):
 return 'string'
 
+def visit(self, visitor):
+visitor.visit_enum_type(self.name, self.info,
+self.values, self.prefix)
+
 
 class QAPISchemaArrayType(QAPISchemaType):
 def __init__(self, name, info, element_type):
@@ -856,6 +896,9 @@ class QAPISchemaArrayType(QAPISchemaType):
 def json_type(self):
 return 'array'
 
+def visit(self, visitor):
+visitor.visit_array_type(self.name, self.info, self.element_type)
+
 
 class QAPISchemaObjectType(QAPISchemaType):
 def __init__(self, name, info, base, local_members, variants):
@@ -904,6 +947,10 @@ class QAPISchemaObjectType(QAPISchemaType):
 def json_type(self):
 return 'object'
 
+def visit(self, visitor):
+visitor.visit_object_type(self.name, self.info,
+  self.base, self.local_members, self.variants)
+
 
 class QAPISchemaObjectTypeMember(object):
 def __init__(self, name, typ, optional):
@@ -971,6 +1018,9 @@ class QAPISchemaAlternateType(QAPISchemaType):
 def json_type(self):
 return 'value'
 
+def visit(self, visitor):
+visitor.visit_alternate_type(self.name, self.info, self.variants)
+
 
 class QAPISchemaCommand(QAPISchemaEntity):
 def __init__(self, name, info, arg_type, ret_type, gen, success_response):
@@ -993,6 +1043,11 @@ class QAPISchemaCommand(QAPISchemaEntity):
 self.ret_type = schema.lookup_type(self._ret_type_name)
 assert isinstance(self.ret_type, QAPISchemaType)
 
+def visit(self, visitor):
+visitor.visit_command(self.name, self.info,
+  self.arg_type, self.ret_type,
+  self.gen, self.success_response)
+
 
 class QAPISchemaEvent(QAPISchemaEntity):
 def __init__(self, name, info, arg_type):
@@ -1007,6 +1062,9 @@ class QAPISchemaEvent(QAPISchemaEntity):
 assert isinstance(self.arg_type, QAPISchemaObjectType)
 assert not self.arg_type.variants   # not implemented
 
+def visit(self, visitor):
+visitor.visit_event(self.name, self.info, self.arg_type)
+
 
 class QAPISchema(object):
 def __init__(self, fname):
@@ -1204,6 +1262,12 @@ class QAPISchema(object):
 for ent in self._entity_dict.values():
 ent.check(self)
 
+def visit(self, visitor):
+visitor.visit_begin(self)
+for name in sorted(self._entity_dict.keys()):
+self._entity_dict[name].visit(visitor)
+visitor.visit_end()
+
 
 #
 # Code generation helpers
-- 
2.4.3

[Qemu-devel] [PULL 01/26] qapi: Rename class QAPISchema to QAPISchemaParser

[Markus Armbruster]

I want to name a new class QAPISchema.

While there, make it a new-style class.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-2-git-send-email-arm...@redhat.com>
---
 scripts/qapi.py | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index c4423b7..2a0f465 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -103,7 +103,7 @@ class QAPIExprError(Exception):
 return error_path(self.info['parent']) + \
 "%s:%d: %s" % (self.info['file'], self.info['line'], self.msg)
 
-class QAPISchema:
+class QAPISchemaParser(object):
 
 def __init__(self, fp, previously_included = [], incl_info = None):
 abs_fname = os.path.abspath(fp.name)
@@ -149,8 +149,8 @@ class QAPISchema:
 except IOError, e:
 raise QAPIExprError(expr_info,
 '%s: %s' % (e.strerror, include))
-exprs_include = QAPISchema(fobj, previously_included,
-   expr_info)
+exprs_include = QAPISchemaParser(fobj, previously_included,
+ expr_info)
 self.exprs.extend(exprs_include.exprs)
 else:
 expr_elem = {'expr': expr,
@@ -755,7 +755,7 @@ def check_exprs(exprs):
 
 def parse_schema(fname):
 try:
-schema = QAPISchema(open(fname, "r"))
+schema = QAPISchemaParser(open(fname, "r"))
 return check_exprs(schema.exprs)
 except (QAPISchemaError, QAPIExprError), e:
 print >>sys.stderr, e
-- 
2.4.3

[Qemu-devel] [PULL 03/26] qapi: QAPISchema code generation helper methods

[Markus Armbruster]

New methods c_name(), c_type(), c_null(), json_type(),
alternate_qtype().

Signed-off-by: Markus Armbruster 
Message-Id: <1442401589-24189-4-git-send-email-arm...@redhat.com>
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 scripts/qapi.py | 94 -
 1 file changed, 87 insertions(+), 7 deletions(-)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index d0dfabe..401b87d 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -766,17 +766,57 @@ class QAPISchemaEntity(object):
 self.name = name
 self.info = info
 
+def c_name(self):
+return c_name(self.name)
+
 def check(self, schema):
 pass
 
 
 class QAPISchemaType(QAPISchemaEntity):
-pass
+def c_type(self, is_param=False):
+return c_name(self.name) + pointer_suffix
+
+def c_null(self):
+return 'NULL'
+
+def json_type(self):
+pass
+
+def alternate_qtype(self):
+json2qtype = {
+'string':  'QTYPE_QSTRING',
+'number':  'QTYPE_QFLOAT',
+'int': 'QTYPE_QINT',
+'boolean': 'QTYPE_QBOOL',
+'object':  'QTYPE_QDICT'
+}
+return json2qtype.get(self.json_type())
 
 
 class QAPISchemaBuiltinType(QAPISchemaType):
-def __init__(self, name):
+def __init__(self, name, json_type, c_type, c_null):
 QAPISchemaType.__init__(self, name, None)
+assert not c_type or isinstance(c_type, str)
+assert json_type in ('string', 'number', 'int', 'boolean', 'null',
+ 'value')
+self._json_type_name = json_type
+self._c_type_name = c_type
+self._c_null_val = c_null
+
+def c_name(self):
+return self.name
+
+def c_type(self, is_param=False):
+if is_param and self.name == 'str':
+return 'const ' + self._c_type_name
+return self._c_type_name
+
+def c_null(self):
+return self._c_null_val
+
+def json_type(self):
+return self._json_type_name
 
 
 class QAPISchemaEnumType(QAPISchemaType):
@@ -791,6 +831,16 @@ class QAPISchemaEnumType(QAPISchemaType):
 def check(self, schema):
 assert len(set(self.values)) == len(self.values)
 
+def c_type(self, is_param=False):
+return c_name(self.name)
+
+def c_null(self):
+return c_enum_const(self.name, (self.values + ['MAX'])[0],
+self.prefix)
+
+def json_type(self):
+return 'string'
+
 
 class QAPISchemaArrayType(QAPISchemaType):
 def __init__(self, name, info, element_type):
@@ -803,6 +853,9 @@ class QAPISchemaArrayType(QAPISchemaType):
 self.element_type = schema.lookup_type(self._element_type_name)
 assert self.element_type
 
+def json_type(self):
+return 'array'
+
 
 class QAPISchemaObjectType(QAPISchemaType):
 def __init__(self, name, info, base, local_members, variants):
@@ -840,6 +893,17 @@ class QAPISchemaObjectType(QAPISchemaType):
 self.variants.check(schema, members, seen)
 self.members = members
 
+def c_name(self):
+assert self.info
+return QAPISchemaType.c_name(self)
+
+def c_type(self, is_param=False):
+assert self.info
+return QAPISchemaType.c_type(self)
+
+def json_type(self):
+return 'object'
+
 
 class QAPISchemaObjectTypeMember(object):
 def __init__(self, name, typ, optional):
@@ -904,6 +968,9 @@ class QAPISchemaAlternateType(QAPISchemaType):
 def check(self, schema):
 self.variants.check(schema, [], {})
 
+def json_type(self):
+return 'value'
+
 
 class QAPISchemaCommand(QAPISchemaEntity):
 def __init__(self, name, info, arg_type, ret_type, gen, success_response):
@@ -969,15 +1036,28 @@ class QAPISchema(object):
 def lookup_type(self, name):
 return self.lookup_entity(name, QAPISchemaType)
 
-def _def_builtin_type(self, name):
-self._def_entity(QAPISchemaBuiltinType(name))
+def _def_builtin_type(self, name, json_type, c_type, c_null):
+self._def_entity(QAPISchemaBuiltinType(name, json_type,
+   c_type, c_null))
 if name != '**':
 self._make_array_type(name) # TODO really needed?
 
 def _def_predefineds(self):
-for t in ['str', 'number', 'int', 'int8', 'int16', 'int32', 'int64',
-  'uint8', 'uint16', 'uint32', 'uint64', 'size', 'bool', '**']:
-self._def_builtin_type(t)
+for t in [('str','string',  'char' + pointer_suffix, 'NULL'),
+  ('number', 'number',  'double',   '0'),
+  ('int','int', 'int64_t',  '0'),
+  ('int8',   'int', 'int8_t',   '0'),
+  ('int16',  'int', 'int16_t',  '0'),
+  ('int32',  'int', 'int32_t',  '0'),
+   

[Qemu-devel] [PULL 05/26] tests/qapi-schema: Convert test harness to QAPISchemaVisitor

[Markus Armbruster]

The old code prints the result of parsing (list of expression
dictionaries), and partial results of semantic analysis (list of enum
dictionaries, list of struct dictionaries).

The new code prints a trace of a schema visit, i.e. what the back-ends
are going to use.  Built-in and array types are omitted, because
they're boring.

Signed-off-by: Markus Armbruster 
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 tests/qapi-schema/alternate-good.out|  15 +-
 tests/qapi-schema/args-member-array.out |  13 +-
 tests/qapi-schema/comments.out  |   4 +-
 tests/qapi-schema/empty.out |   3 -
 tests/qapi-schema/enum-empty.out|   4 +-
 tests/qapi-schema/event-case.out|   4 +-
 tests/qapi-schema/flat-union-reverse-define.out |  21 ++-
 tests/qapi-schema/ident-with-escape.out |   7 +-
 tests/qapi-schema/include-relpath.out   |   4 +-
 tests/qapi-schema/include-repetition.out|   4 +-
 tests/qapi-schema/include-simple.out|   4 +-
 tests/qapi-schema/indented-expr.out |   7 +-
 tests/qapi-schema/qapi-schema-test.out  | 218 +---
 tests/qapi-schema/returns-int.out   |   5 +-
 tests/qapi-schema/test-qapi.py  |  47 -
 tests/qapi-schema/type-bypass.out   |   7 +-
 16 files changed, 249 insertions(+), 118 deletions(-)

diff --git a/tests/qapi-schema/alternate-good.out 
b/tests/qapi-schema/alternate-good.out
index 99848ee..3d765ff 100644
--- a/tests/qapi-schema/alternate-good.out
+++ b/tests/qapi-schema/alternate-good.out
@@ -1,6 +1,9 @@
-[OrderedDict([('struct', 'Data'), ('data', OrderedDict([('*number', 'int'), 
('*name', 'str')]))]),
- OrderedDict([('enum', 'Enum'), ('data', ['hello', 'world'])]),
- OrderedDict([('alternate', 'Alt'), ('data', OrderedDict([('value', 'int'), 
('string', 'Enum'), ('struct', 'Data')]))])]
-[{'enum_name': 'Enum', 'enum_values': ['hello', 'world']},
- {'enum_name': 'AltKind', 'enum_values': None}]
-[OrderedDict([('struct', 'Data'), ('data', OrderedDict([('*number', 'int'), 
('*name', 'str')]))])]
+alternate Alt
+case value: int
+case string: Enum
+case struct: Data
+enum AltKind ['value', 'string', 'struct']
+object Data
+member number: int optional=True
+member name: str optional=True
+enum Enum ['hello', 'world']
diff --git a/tests/qapi-schema/args-member-array.out 
b/tests/qapi-schema/args-member-array.out
index c39fa25..b67384c 100644
--- a/tests/qapi-schema/args-member-array.out
+++ b/tests/qapi-schema/args-member-array.out
@@ -1,5 +1,8 @@
-[OrderedDict([('enum', 'abc'), ('data', ['a', 'b', 'c'])]),
- OrderedDict([('struct', 'def'), ('data', OrderedDict([('array', ['abc'])]))]),
- OrderedDict([('command', 'okay'), ('data', OrderedDict([('member1', ['int']), 
('member2', ['def'])]))])]
-[{'enum_name': 'abc', 'enum_values': ['a', 'b', 'c']}]
-[OrderedDict([('struct', 'def'), ('data', OrderedDict([('array', ['abc'])]))])]
+object :obj-okay-arg
+member member1: intList optional=False
+member member2: defList optional=False
+enum abc ['a', 'b', 'c']
+object def
+member array: abcList optional=False
+command okay :obj-okay-arg -> None
+   gen=True success_response=True
diff --git a/tests/qapi-schema/comments.out b/tests/qapi-schema/comments.out
index 4ce3dcf..6161b90 100644
--- a/tests/qapi-schema/comments.out
+++ b/tests/qapi-schema/comments.out
@@ -1,3 +1 @@
-[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
-[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
-[]
+enum Status ['good', 'bad', 'ugly']
diff --git a/tests/qapi-schema/empty.out b/tests/qapi-schema/empty.out
index b7f89a4..e69de29 100644
--- a/tests/qapi-schema/empty.out
+++ b/tests/qapi-schema/empty.out
@@ -1,3 +0,0 @@
-[]
-[]
-[]
diff --git a/tests/qapi-schema/enum-empty.out b/tests/qapi-schema/enum-empty.out
index 3b75c16..e09b00f 100644
--- a/tests/qapi-schema/enum-empty.out
+++ b/tests/qapi-schema/enum-empty.out
@@ -1,3 +1 @@
-[OrderedDict([('enum', 'MyEnum'), ('data', [])])]
-[{'enum_name': 'MyEnum', 'enum_values': []}]
-[]
+enum MyEnum []
diff --git a/tests/qapi-schema/event-case.out b/tests/qapi-schema/event-case.out
index 3764bc7..b5ae4c2 100644
--- a/tests/qapi-schema/event-case.out
+++ b/tests/qapi-schema/event-case.out
@@ -1,3 +1 @@
-[OrderedDict([('event', 'oops')])]
-[]
-[]
+event oops None
diff --git a/tests/qapi-schema/flat-union-reverse-define.out 
b/tests/qapi-schema/flat-union-reverse-define.out
index 1ed7b8a..477fb31 100644
--- a/tests/qapi-schema/flat-union-reverse-define.out
+++ b/tests/qapi-schema/flat-union-reverse-define.out
@@ -1,9 +1,12 @@
-[OrderedDict([('union', 'TestUnion'), ('base', 'TestBase'), ('discriminator', 
'enum1'), ('data', OrderedDict([('value1', 'TestTypeA'), ('value2', 
'TestTypeB')]))]),
- OrderedDict([('struct', 'TestBase'), ('data', 

[Qemu-devel] [PULL 09/26] qapi: De-duplicate enum code generation

[Markus Armbruster]

Duplicated in commit 21cd70d.  Yes, we can't import qapi-types, but
that's no excuse.  Move the helpers from qapi-types.py to qapi.py, and
replace the duplicates in qapi-event.py.

The generated event enumeration type's lookup table becomes
const-correct (see commit 2e4450f), and uses explicit indexes instead
of relying on order (see commit 912ae9c).

Signed-off-by: Markus Armbruster 
Message-Id: <1442401589-24189-10-git-send-email-arm...@redhat.com>
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt |  9 ---
 scripts/qapi-event.py  | 67 +++---
 scripts/qapi-types.py  | 55 -
 scripts/qapi.py| 55 +
 4 files changed, 64 insertions(+), 122 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index c9e21fc..81e87d2 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -829,9 +829,9 @@ Example:
 QDECREF(qmp);
 }
 
-const char *example_QAPIEvent_lookup[] = {
-"MY_EVENT",
-NULL,
+const char *const example_QAPIEvent_lookup[] = {
+[EXAMPLE_QAPI_EVENT_MY_EVENT] = "MY_EVENT",
+[EXAMPLE_QAPI_EVENT_MAX] = NULL,
 };
 $ cat qapi-generated/example-qapi-event.h
 [Uninteresting stuff omitted...]
@@ -846,10 +846,11 @@ Example:
 
 void qapi_event_send_my_event(Error **errp);
 
-extern const char *example_QAPIEvent_lookup[];
 typedef enum example_QAPIEvent {
 EXAMPLE_QAPI_EVENT_MY_EVENT = 0,
 EXAMPLE_QAPI_EVENT_MAX = 1,
 } example_QAPIEvent;
 
+extern const char *const example_QAPIEvent_lookup[];
+
 #endif
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index aec2d32..aed45d6 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -153,63 +153,6 @@ def generate_event_implement(api_name, event_name, params):
 
 return ret
 
-
-# Following are the functions that generate an enum type for all defined
-# events, similar to qapi-types.py. Here we already have enum name and
-# values which were generated before and recorded in event_enum_*. It also
-# works around the issue that "import qapi-types" can't work.
-
-def generate_event_enum_decl(event_enum_name, event_enum_values):
-lookup_decl = mcgen('''
-
-extern const char *%(event_enum_name)s_lookup[];
-''',
-event_enum_name = event_enum_name)
-
-enum_decl = mcgen('''
-typedef enum %(event_enum_name)s {
-''',
-  event_enum_name = event_enum_name)
-
-# append automatically generated _MAX value
-enum_max_value = c_enum_const(event_enum_name, "MAX")
-enum_values = event_enum_values + [ enum_max_value ]
-
-i = 0
-for value in enum_values:
-enum_decl += mcgen('''
-%(value)s = %(i)d,
-''',
- value = value,
- i = i)
-i += 1
-
-enum_decl += mcgen('''
-} %(event_enum_name)s;
-''',
-   event_enum_name = event_enum_name)
-
-return lookup_decl + enum_decl
-
-def generate_event_enum_lookup(event_enum_name, event_enum_strings):
-ret = mcgen('''
-
-const char *%(event_enum_name)s_lookup[] = {
-''',
-event_enum_name = event_enum_name)
-
-for string in event_enum_strings:
-ret += mcgen('''
-"%(string)s",
-''',
- string = string)
-
-ret += mcgen('''
-NULL,
-};
-''')
-return ret
-
 (input_file, output_dir, do_c, do_h, prefix, dummy) = parse_command_line()
 
 c_comment = '''
@@ -266,8 +209,7 @@ fdecl.write(mcgen('''
 exprs = QAPISchema(input_file).get_exprs()
 
 event_enum_name = c_name(prefix + "QAPIEvent", protect=False)
-event_enum_values = []
-event_enum_strings = []
+event_names = []
 
 for expr in exprs:
 if expr.has_key('event'):
@@ -286,12 +228,11 @@ for expr in exprs:
 fdef.write(ret)
 
 # Record it, and generate enum later
-event_enum_values.append(event_enum_value)
-event_enum_strings.append(event_name)
+event_names.append(event_name)
 
-ret = generate_event_enum_decl(event_enum_name, event_enum_values)
+ret = generate_enum(event_enum_name, event_names)
 fdecl.write(ret)
-ret = generate_event_enum_lookup(event_enum_name, event_enum_strings)
+ret = generate_enum_lookup(event_enum_name, event_names)
 fdef.write(ret)
 
 close_output(fdef, fdecl)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index d78f529..76c82d9 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -80,61 +80,6 @@ struct %(name)s {
 
 return ret
 
-def generate_enum_lookup(name, values, prefix=None):
-ret = mcgen('''
-
-const char *const %(name)s_lookup[] = {
-''',
-name=c_name(name))
-for value in values:
-index = c_enum_const(name, value, prefix)
-ret += mcgen('''
-[%(index)s] = "%(value)s",
-''',
-   

[Qemu-devel] [PULL 13/26] qapi: Clean up after recent conversions to QAPISchemaVisitor

[Markus Armbruster]

Generate just 'FOO' instead of 'struct FOO' when possible.

Drop helper functions that are now unused.

Make pep8 and pylint reasonably happy.

Rename generate_FOO() functions to gen_FOO() for consistency.

Use more consistent and sensible variable names.

Consistently use c_ for mapping keys when their value is a C
identifier or type.

Simplify gen_enum() and gen_visit_union()

Consistently use single quotes for C text string literals.

Signed-off-by: Markus Armbruster 
Message-Id: <1442401589-24189-14-git-send-email-arm...@redhat.com>
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt   |   2 +-
 scripts/qapi-commands.py | 140 ++-
 scripts/qapi-event.py| 122 -
 scripts/qapi-types.py|  79 ++
 scripts/qapi-visit.py| 127 ++
 scripts/qapi.py  | 131 +---
 6 files changed, 274 insertions(+), 327 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 81e87d2..3eaff36 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -598,7 +598,7 @@ Example:
 UserDefOne *value;
 uint64_t padding;
 };
-struct UserDefOneList *next;
+UserDefOneList *next;
 };
 
 void qapi_free_UserDefOneList(UserDefOneList *obj);
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index cbff356..0501582 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -15,20 +15,22 @@
 from qapi import *
 import re
 
-def generate_command_decl(name, args, ret_type):
-arglist=""
-if args:
-for memb in args.members:
-argtype = memb.type.c_type(is_param=True)
+
+def gen_command_decl(name, arg_type, ret_type):
+argstr = ''
+if arg_type:
+for memb in arg_type.members:
 if memb.optional:
-arglist += "bool has_%s, " % c_name(memb.name)
-arglist += "%s %s, " % (argtype, c_name(memb.name))
+argstr += 'bool has_%s, ' % c_name(memb.name)
+argstr += '%s %s, ' % (memb.type.c_type(is_param=True),
+   c_name(memb.name))
 return mcgen('''
-%(ret_type)s qmp_%(name)s(%(args)sError **errp);
+%(c_type)s qmp_%(c_name)s(%(args)sError **errp);
 ''',
- ret_type=(ret_type and ret_type.c_type()) or 'void',
- name=c_name(name),
- args=arglist)
+ c_type=(ret_type and ret_type.c_type()) or 'void',
+ c_name=c_name(name),
+ args=argstr)
+
 
 def gen_err_check(err):
 if not err:
@@ -40,37 +42,42 @@ if (%(err)s) {
 ''',
  err=err)
 
-def gen_sync_call(name, args, ret_type):
-ret = ""
-arglist=""
-retval=""
-if ret_type:
-retval = "retval = "
-if args:
-for memb in args.members:
+
+def gen_call(name, arg_type, ret_type):
+ret = ''
+
+argstr = ''
+if arg_type:
+for memb in arg_type.members:
 if memb.optional:
-arglist += "has_%s, " % c_name(memb.name)
-arglist += "%s, " % c_name(memb.name)
+argstr += 'has_%s, ' % c_name(memb.name)
+argstr += '%s, ' % c_name(memb.name)
+
+lhs = ''
+if ret_type:
+lhs = 'retval = '
+
 push_indent()
 ret = mcgen('''
-%(retval)sqmp_%(name)s(%(args)s_err);
+%(lhs)sqmp_%(c_name)s(%(args)s_err);
 ''',
-name=c_name(name), args=arglist, retval=retval)
+c_name=c_name(name), args=argstr, lhs=lhs)
 if ret_type:
 ret += gen_err_check('local_err')
 ret += mcgen('''
 
 qmp_marshal_output_%(c_name)s(retval, ret, _err);
 ''',
-c_name=c_name(name))
+ c_name=c_name(name))
 pop_indent()
 return ret
 
-def gen_visitor_input_containers_decl(args):
-ret = ""
+
+def gen_visitor_input_containers_decl(arg_type):
+ret = ''
 
 push_indent()
-if args:
+if arg_type:
 ret += mcgen('''
 QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
 QapiDeallocVisitor *md;
@@ -80,17 +87,18 @@ Visitor *v;
 
 return ret
 
-def gen_visitor_input_vars_decl(args):
-ret = ""
+
+def gen_visitor_input_vars_decl(arg_type):
+ret = ''
 push_indent()
 
-if args:
-for memb in args.members:
+if arg_type:
+for memb in arg_type.members:
 if memb.optional:
 ret += mcgen('''
-bool has_%(argname)s = false;
+bool has_%(c_name)s = false;
 ''',
- argname=c_name(memb.name))
+ c_name=c_name(memb.name))
 ret += mcgen('''
 %(c_type)s %(c_name)s = %(c_null)s;
 ''',
@@ -101,19 +109,20 @@ bool has_%(argname)s = false;
   

[Qemu-devel] [PULL 18/26] qapi-commands: De-duplicate output marshaling functions

[Markus Armbruster]

gen_marshal_output() uses its parameter name only for name of the
generated function.  Name it after the type being marshaled instead of
its caller, and drop duplicates.

Saves 7 copies of qmp_marshal_output_int() in qemu-ga, and one copy of
qmp_marshal_output_str() in qemu-system-*.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-19-git-send-email-arm...@redhat.com>
---
 docs/qapi-code-gen.txt   |  4 ++--
 scripts/qapi-commands.py | 17 ++---
 2 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 50da1d0..0884b58 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -721,7 +721,7 @@ Example:
 $ cat qapi-generated/example-qmp-marshal.c
 [Uninteresting stuff omitted...]
 
-static void qmp_marshal_output_my_command(UserDefOne *ret_in, QObject 
**ret_out, Error **errp)
+static void qmp_marshal_output_UserDefOne(UserDefOne *ret_in, QObject 
**ret_out, Error **errp)
 {
 Error *local_err = NULL;
 QmpOutputVisitor *mo = qmp_output_visitor_new();
@@ -764,7 +764,7 @@ Example:
 goto out;
 }
 
-qmp_marshal_output_my_command(retval, ret, _err);
+qmp_marshal_output_UserDefOne(retval, ret, _err);
 
 out:
 error_propagate(errp, local_err);
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 833768e..810a897 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -62,7 +62,7 @@ def gen_call(name, arg_type, ret_type):
 
 qmp_marshal_output_%(c_name)s(retval, ret, _err);
 ''',
- c_name=c_name(name))
+ c_name=ret_type.c_name())
 pop_indent()
 return ret
 
@@ -166,10 +166,10 @@ qapi_dealloc_visitor_cleanup(md);
 return ret
 
 
-def gen_marshal_output(name, ret_type):
+def gen_marshal_output(ret_type):
 return mcgen('''
 
-static void qmp_marshal_output_%(c_cmd_name)s(%(c_type)s ret_in, QObject 
**ret_out, Error **errp)
+static void qmp_marshal_output_%(c_name)s(%(c_type)s ret_in, QObject 
**ret_out, Error **errp)
 {
 Error *local_err = NULL;
 QmpOutputVisitor *mo = qmp_output_visitor_new();
@@ -192,8 +192,7 @@ out:
 qapi_dealloc_visitor_cleanup(md);
 }
 ''',
- c_type=ret_type.c_type(), c_cmd_name=c_name(name),
- c_name=ret_type.c_name())
+ c_type=ret_type.c_type(), c_name=ret_type.c_name())
 
 
 def gen_marshal_proto(name):
@@ -272,24 +271,28 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
 self.decl = None
 self.defn = None
 self._regy = None
+self._visited_ret_types = None
 
 def visit_begin(self, schema):
 self.decl = ''
 self.defn = ''
 self._regy = ''
+self._visited_ret_types = set()
 
 def visit_end(self):
 if not middle_mode:
 self.defn += gen_registry(self._regy)
 self._regy = None
+self._visited_ret_types = None
 
 def visit_command(self, name, info, arg_type, ret_type,
   gen, success_response):
 if not gen:
 return
 self.decl += gen_command_decl(name, arg_type, ret_type)
-if ret_type:
-self.defn += gen_marshal_output(name, ret_type)
+if ret_type and ret_type not in self._visited_ret_types:
+self._visited_ret_types.add(ret_type)
+self.defn += gen_marshal_output(ret_type)
 if middle_mode:
 self.decl += gen_marshal_decl(name)
 self.defn += gen_marshal(name, arg_type, ret_type)
-- 
2.4.3

[Qemu-devel] [PULL 06/26] qapi-types: Convert to QAPISchemaVisitor, fixing flat unions

[Markus Armbruster]

Fixes flat unions to get the base's base members.  Test case is from
commit 2fc0043, in qapi-schema-test.json:

{ 'union': 'UserDefFlatUnion',
  'base': 'UserDefUnionBase',
  'discriminator': 'enum1',
  'data': { 'value1' : 'UserDefA',
'value2' : 'UserDefB',
'value3' : 'UserDefB' } }

{ 'struct': 'UserDefUnionBase',
  'base': 'UserDefZero',
  'data': { 'string': 'str', 'enum1': 'EnumOne' } }

{ 'struct': 'UserDefZero',
  'data': { 'integer': 'int' } }

Patch's effect on UserDefFlatUnion:

 struct UserDefFlatUnion {
 /* Members inherited from UserDefUnionBase: */
+int64_t integer;
 char *string;
 EnumOne enum1;
 /* Own members: */
 union { /* union tag is @enum1 */
 void *data;
 UserDefA *value1;
 UserDefB *value2;
 UserDefB *value3;
 };
 };

Flat union visitors remain broken.  They'll be fixed next.

Code is generated in a different order now, but that doesn't matter.

The two guards QAPI_TYPES_BUILTIN_STRUCT_DECL and
QAPI_TYPES_BUILTIN_CLEANUP_DECL are replaced by just
QAPI_TYPES_BUILTIN.

Two ugly special cases for simple unions now stand out like sore
thumbs:

1. The type tag is named 'type' everywhere, except in generated C,
   where it's 'kind'.

2. QAPISchema lowers simple unions to semantically equivalent flat
   unions.  However, the C generated for a simple unions differs from
   the C generated for its equivalent flat union, and we therefore
   need special code to preserve that pointless difference for now.

Mark both TODO.

Signed-off-by: Markus Armbruster 
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt  |  51 +++---
 scripts/qapi-types.py   | 286 ++--
 scripts/qapi.py |  10 +-
 tests/qapi-schema/qapi-schema-test.json |   4 +-
 4 files changed, 164 insertions(+), 187 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index d960dc0..c9e21fc 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -545,22 +545,6 @@ Example:
 $ cat qapi-generated/example-qapi-types.c
 [Uninteresting stuff omitted...]
 
-void qapi_free_UserDefOneList(UserDefOneList *obj)
-{
-QapiDeallocVisitor *md;
-Visitor *v;
-
-if (!obj) {
-return;
-}
-
-md = qapi_dealloc_visitor_new();
-v = qapi_dealloc_get_visitor(md);
-visit_type_UserDefOneList(v, , NULL, NULL);
-qapi_dealloc_visitor_cleanup(md);
-}
-
-
 void qapi_free_UserDefOne(UserDefOne *obj)
 {
 QapiDeallocVisitor *md;
@@ -575,6 +559,21 @@ Example:
 visit_type_UserDefOne(v, , NULL, NULL);
 qapi_dealloc_visitor_cleanup(md);
 }
+
+void qapi_free_UserDefOneList(UserDefOneList *obj)
+{
+QapiDeallocVisitor *md;
+Visitor *v;
+
+if (!obj) {
+return;
+}
+
+md = qapi_dealloc_visitor_new();
+v = qapi_dealloc_get_visitor(md);
+visit_type_UserDefOneList(v, , NULL, NULL);
+qapi_dealloc_visitor_cleanup(md);
+}
 $ cat qapi-generated/example-qapi-types.h
 [Uninteresting stuff omitted...]
 
@@ -585,24 +584,24 @@ Example:
 
 typedef struct UserDefOne UserDefOne;
 
-typedef struct UserDefOneList {
+typedef struct UserDefOneList UserDefOneList;
+
+struct UserDefOne {
+int64_t integer;
+char *string;
+};
+
+void qapi_free_UserDefOne(UserDefOne *obj);
+
+struct UserDefOneList {
 union {
 UserDefOne *value;
 uint64_t padding;
 };
 struct UserDefOneList *next;
-} UserDefOneList;
-
-
-[Functions on built-in types omitted...]
-
-struct UserDefOne {
-int64_t integer;
-char *string;
 };
 
 void qapi_free_UserDefOneList(UserDefOneList *obj);
-void qapi_free_UserDefOne(UserDefOne *obj);
 
 #endif
 
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 23e9505..d78f529 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -2,88 +2,67 @@
 # QAPI types generator
 #
 # Copyright IBM, Corp. 2011
+# Copyright (c) 2013-2015 Red Hat Inc.
 #
 # Authors:
 #  Anthony Liguori 
+#  Markus Armbruster 
 #
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-from ordereddict import OrderedDict
 from qapi import *
 
-def generate_fwd_builtin(name):
-return mcgen('''
-
-typedef struct %(name)sList {
-union {
-%(type)s value;
-uint64_t padding;
-};
-struct %(name)sList *next;
-} %(name)sList;
-''',
- type=c_type(name),
- name=name)
-
-def generate_fwd_struct(name):
+def 

[Qemu-devel] [PULL 07/26] qapi-visit: Convert to QAPISchemaVisitor, fixing bugs

[Markus Armbruster]

Fixes flat unions to visit the base's base members (the previous
commit merely added them to the struct).  Same test case.

Patch's effect on visit_type_UserDefFlatUnion():

 static void visit_type_UserDefFlatUnion_fields(Visitor *m, 
UserDefFlatUnion **obj, Error **errp)
 {
 Error *err = NULL;

+visit_type_int(m, &(*obj)->integer, "integer", );
+if (err) {
+goto out;
+}
 visit_type_str(m, &(*obj)->string, "string", );
 if (err) {
 goto out;

Test cases updated for the bug fix.

Fixes alternates to generate a visitor for their implicit enumeration
type.  None of them are currently used, obviously.  Example:
block-core.json's BlockdevRef now generates
visit_type_BlockdevRefKind().

Code is generated in a different order now, and therefore has got a
few new forward declarations.  Doesn't matter.

The guard QAPI_VISIT_BUILTIN_VISITOR_DECL is renamed to
QAPI_VISIT_BUILTIN.

The previous commit's two ugly special cases exist here, too.  Mark
both TODO.

Signed-off-by: Markus Armbruster 
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 scripts/qapi-visit.py   | 270 +---
 tests/qapi-schema/qapi-schema-test.json |   3 -
 tests/test-qmp-input-strict.c   |   4 +-
 tests/test-qmp-input-visitor.c  |   4 +-
 4 files changed, 117 insertions(+), 164 deletions(-)

diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index c493964..8e6e8ca 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -12,7 +12,6 @@
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-from ordereddict import OrderedDict
 from qapi import *
 import re
 
@@ -24,13 +23,13 @@ def generate_visit_implicit_struct(type):
 return ''
 implicit_structs_seen.add(type)
 ret = ''
-if type not in struct_fields_seen:
+if type.name not in struct_fields_seen:
 # Need a forward declaration
 ret += mcgen('''
 
 static void visit_type_%(c_type)s_fields(Visitor *m, %(c_type)s **obj, Error 
**errp);
 ''',
- c_type=type_name(type))
+ c_type=type.c_name())
 
 ret += mcgen('''
 
@@ -46,7 +45,7 @@ static void visit_type_implicit_%(c_type)s(Visitor *m, 
%(c_type)s **obj, Error *
 error_propagate(errp, err);
 }
 ''',
- c_type=type_name(type))
+ c_type=type.c_name())
 return ret
 
 def generate_visit_struct_fields(name, members, base = None):
@@ -74,24 +73,24 @@ if (err) {
 goto out;
 }
 ''',
- type=type_name(base), c_name=c_name('base'))
+ type=base.c_name(), c_name=c_name('base'))
 
-for argname, argentry, optional in parse_args(members):
-if optional:
+for memb in members:
+if memb.optional:
 ret += mcgen('''
 visit_optional(m, &(*obj)->has_%(c_name)s, "%(name)s", );
 if (!err && (*obj)->has_%(c_name)s) {
 ''',
- c_name=c_name(argname), name=argname)
+ c_name=c_name(memb.name), name=memb.name)
 push_indent()
 
 ret += mcgen('''
 visit_type_%(type)s(m, &(*obj)->%(c_name)s, "%(name)s", );
 ''',
- type=type_name(argentry), c_name=c_name(argname),
- name=argname)
+ type=memb.type.c_name(), c_name=c_name(memb.name),
+ name=memb.name)
 
-if optional:
+if memb.optional:
 pop_indent()
 ret += mcgen('''
 }
@@ -136,12 +135,7 @@ def generate_visit_struct_body(name):
 
 return ret
 
-def generate_visit_struct(expr):
-
-name = expr['struct']
-members = expr['data']
-base = expr.get('base')
-
+def gen_visit_struct(name, base, members):
 ret = generate_visit_struct_fields(name, members, base)
 
 ret += mcgen('''
@@ -158,10 +152,10 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, 
const char *name, Error **e
 ''')
 return ret
 
-def generate_visit_list(name):
+def gen_visit_list(name, element_type):
 return mcgen('''
 
-void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, 
Error **errp)
+void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error 
**errp)
 {
 Error *err = NULL;
 GenericList *i, **prev;
@@ -174,8 +168,8 @@ void visit_type_%(name)sList(Visitor *m, %(name)sList 
**obj, const char *name, E
 for (prev = (GenericList **)obj;
  !err && (i = visit_next_list(m, prev, )) != NULL;
  prev = ) {
-%(name)sList *native_i = (%(name)sList *)i;
-visit_type_%(name)s(m, _i->value, NULL, );
+%(name)s *native_i = (%(name)s *)i;
+visit_type_%(c_elt_type)s(m, _i->value, NULL, );
 }
 
 error_propagate(errp, err);
@@ -185,7 +179,8 @@ out:
 error_propagate(errp, err);
 

[Qemu-devel] [PULL 24/26] qapi: Pseudo-type '**' is now unused, drop it

[Markus Armbruster]

'gen': false needs to stay for now, because netdev_add is still using
it.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-25-git-send-email-arm...@redhat.com>
---
 docs/qapi-code-gen.txt| 18 ++
 scripts/qapi.py   | 20 
 tests/Makefile|  2 +-
 tests/qapi-schema/type-bypass-no-gen.err  |  1 -
 tests/qapi-schema/type-bypass-no-gen.exit |  1 -
 tests/qapi-schema/type-bypass-no-gen.json |  2 --
 tests/qapi-schema/type-bypass-no-gen.out  |  0
 tests/qapi-schema/type-bypass.err |  0
 tests/qapi-schema/type-bypass.exit|  1 -
 tests/qapi-schema/type-bypass.json|  2 --
 tests/qapi-schema/type-bypass.out |  4 
 11 files changed, 11 insertions(+), 40 deletions(-)
 delete mode 100644 tests/qapi-schema/type-bypass-no-gen.err
 delete mode 100644 tests/qapi-schema/type-bypass-no-gen.exit
 delete mode 100644 tests/qapi-schema/type-bypass-no-gen.json
 delete mode 100644 tests/qapi-schema/type-bypass-no-gen.out
 delete mode 100644 tests/qapi-schema/type-bypass.err
 delete mode 100644 tests/qapi-schema/type-bypass.exit
 delete mode 100644 tests/qapi-schema/type-bypass.json
 delete mode 100644 tests/qapi-schema/type-bypass.out

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 4e331a0..b917962 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -111,10 +111,7 @@ and field names within a type, should be all lower case 
with words
 separated by a hyphen.  However, some existing older commands and
 complex types use underscore; when extending such expressions,
 consistency is preferred over blindly avoiding underscore.  Event
-names should be ALL_CAPS with words separated by underscore.  The
-special string '**' appears for some commands that manually perform
-their own type checking rather than relying on the type-safe code
-produced by the qapi code generators.
+names should be ALL_CAPS with words separated by underscore.
 
 Any name (command, event, type, field, or enum value) beginning with
 "x-" is marked experimental, and may be withdrawn or changed
@@ -461,14 +458,11 @@ which would validate this Client JSON Protocol 
transaction:
  <= { "return": [ { "value": "one" }, { } ] }
 
 In rare cases, QAPI cannot express a type-safe representation of a
-corresponding Client JSON Protocol command.  In these cases, if the
-command expression includes the key 'gen' with boolean value false,
-then the 'data' or 'returns' member that intends to bypass generated
-type-safety and do its own manual validation should use an inline
-dictionary definition, with a value of '**' rather than a valid type
-name for the keys that the generated code will not validate.  Please
-try to avoid adding new commands that rely on this, and instead use
-type-safe unions.  For an example of bypass usage:
+corresponding Client JSON Protocol command.  You then have to suppress
+generation of a marshalling function by including a key 'gen' with
+boolean value false, and instead write your own function.  Please try
+to avoid adding new commands that rely on this, and instead use
+type-safe unions.  For an example of this usage:
 
  { 'command': 'netdev_add',
'data': {'type': 'str', 'id': 'str'},
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 8808ae6..f407297 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -428,15 +428,12 @@ def is_enum(name):
 
 def check_type(expr_info, source, value, allow_array = False,
allow_dict = False, allow_optional = False,
-   allow_star = False, allow_metas = []):
+   allow_metas = []):
 global all_names
 
 if value is None:
 return
 
-if allow_star and value == '**':
-return
-
 # Check if array type for value is okay
 if isinstance(value, list):
 if not allow_array:
@@ -450,10 +447,6 @@ def check_type(expr_info, source, value, allow_array = 
False,
 
 # Check if type name for value is okay
 if isinstance(value, str):
-if value == '**':
-raise QAPIExprError(expr_info,
-"%s uses '**' but did not request 'gen':false"
-% source)
 if not value in all_names:
 raise QAPIExprError(expr_info,
 "%s uses unknown type '%s'"
@@ -479,7 +472,7 @@ def check_type(expr_info, source, value, allow_array = 
False,
 # Todo: allow dictionaries to represent default values of
 # an optional argument.
 check_type(expr_info, "Member '%s' of %s" % (key, source), arg,
-   allow_array=True, allow_star=allow_star,
+   allow_array=True,
allow_metas=['built-in', 'union', 'alternate', 'struct',
 'enum'])
 
@@ 

Re: [Qemu-devel] [PATCH v18 00/21] Deterministic replay core

[Pavel Dovgaluk]

Hi!

Paolo, have you reviewed these patches?

Pavel Dovgalyuk

> -Original Message-
> From: Pavel Dovgalyuk [mailto:pavel.dovga...@ispras.ru]
> Sent: Thursday, September 17, 2015 7:23 PM
> To: qemu-devel@nongnu.org
> Cc: edgar.igles...@xilinx.com; peter.mayd...@linaro.org; 
> igor.rubi...@gmail.com;
> ebl...@redhat.com; mark.bur...@greensocs.com; r...@ispras.ru; 
> batuz...@ispras.ru;
> maria.klimushenk...@ispras.ru; pavel.dovga...@ispras.ru; pbonz...@redhat.com; 
> hi...@cert.org;
> alex.ben...@linaro.org; fred.kon...@greensocs.com
> Subject: [PATCH v18 00/21] Deterministic replay core
> 
> This set of patches is related to the reverse execution and deterministic
> replay of qemu execution. This implementation of deterministic replay can
> be used for deterministic debugging of guest code through gdb remote
> interface.
> 
> Core set of patches does not include support for reverse debugging commands
> of gdb, block devices' operations, USB replay support.
> 
> These patches include only core function of the replay,
> excluding the support for replaying serial, audio, network, and USB devices'
> operations. Reverse debugging and monitor commands were also excluded to
> be submitted later as separate patches.
> 
> Execution recording writes non-deterministic events log, which can be later
> used for replaying the execution anywhere and for unlimited number of times.
> It also supports checkpointing for faster rewinding during reverse debugging.
> Execution replaying reads the log and replays all non-deterministic events
> including external input, hardware clocks, and interrupts.
> 
> Full version of deterministic replay has the following features:
>  * Deterministically replays whole system execution and all contents of the 
> memory,
>state of the hadrware devices, clocks, and screen of the VM.
>  * Writes execution log into the file for latter replaying for multiple times
>on different machines.
>  * Supports i386, x86_64, ARM, PowerPC, and MIPS hardware platforms.
>  * Performs deterministic replay of all operations with keyboard and mouse
>input devices.
>  * Supports auto-checkpointing for convenient reverse debugging.
> 
> Usage of the record/replay core:
>  * First, record the execution, by adding the following string to the command 
> line:
>'-icount shift=7,rr=record,rrfile=replay.bin -net none'.
>Block devices' images are not actually changed in the recording mode,
>because all of the changes are written to the temporary overlay file.
>  * Then you can replay it for the multiple times by using another command
>line option: '-icount shift=7,rr=replay,rrfile=replay.bin -net none'
>  * '-net none' option should also be specified if network replay patches
>are not applied.
>  * Do not add any disk images to VM, because they are not supported by
>the core patches.
> 
> Papers with description of deterministic replay implementation:
> http://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html
> http://dl.acm.org/citation.cfm?id=2786805.2803179
> 
> Public repository with current version of the patches:
> https://github.com/Dovgalyuk/qemu/tree/rr-17
> 
> Modifications of qemu include:
>  * wrappers for clock and time functions to save their return values in the 
> log
>  * saving different asynchronous events (e.g. system shutdown) into the log
>  * synchronization of the threads from thread pool
>  * recording/replaying user input (mouse and keyboard)
>  * adding internal events for cpu and io synchronization
> 
> v18 changes:
>  * Patches were updated to match upstream version
>  * Added missed replay-user.c file
> 
> v17 changes:
>  * Removed useless stub functions (as suggested by Paolo Bonzini)
>  * Refined checkpoint-related code (as suggested by Paolo Bonzini)
>  * Improved icount processing (as suggested by Paolo Bonzini)
>  * Added checkpoint for suspend event (as suggested by Paolo Bonzini)
>  * Fixed linux-user configurations build
>  * Minor fixes
> 
> v16 changes:
>  * Several warnings were fixed
> 
> v15 changes:
>  * Tested record/replay with MIPS and PowerPC guests
>  * Published the patches on github
>  * Fixed replay mutex operation in icount mode
>  * Fixed timers processing in record/replay mode
> 
> v14 changes:
>  * Minor fixes
> 
> v13 changes:
>  * Introduced "ptimer trigger" event (as suggested by Paolo Bonzini)
> 
> v12 changes:
>  * Removed block patches from the core patches set.
> 
> v11 changes:
>  * Fixed instructions event processing.
>  * Added some mutex protection calls for replay.
>  * Fixed replaying read operations for qcow2.
>  * Fixed rtc reads on initializations stage.
>  * Eliminated some warnings in replay module.
>  * Fixed misprints in documentation for replay (as suggested by Eric Blake)
> 
> v10 changes:
>  * Fixed queue processing for bottom halves (as suggested by Paolo Bonzini)
>  * Rewritten several replay functions (as suggested by Paolo Bonzini)
>  * Some minor fixes.
> 
> v9 changes:
>  * 

[Qemu-devel] [PULL 12/26] qapi: Replace dirty is_c_ptr() by method c_null()

[Markus Armbruster]

is_c_ptr() looks whether the end of the C text for the type looks like
a pointer.  Works, but is fragile.

We now have a better tool: use QAPISchemaType method c_null().  The
initializers for non-pointers become prettier: 0, false or the
enumeration constant with the value 0 instead of {0}.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-13-git-send-email-arm...@redhat.com>
---
 scripts/qapi-commands.py | 16 +---
 scripts/qapi.py  |  3 ---
 2 files changed, 5 insertions(+), 14 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index a68517a..cbff356 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -91,18 +91,12 @@ def gen_visitor_input_vars_decl(args):
 bool has_%(argname)s = false;
 ''',
  argname=c_name(memb.name))
-if is_c_ptr(memb.type.c_type()):
-ret += mcgen('''
-%(argtype)s %(argname)s = NULL;
+ret += mcgen('''
+%(c_type)s %(c_name)s = %(c_null)s;
 ''',
- argname=c_name(memb.name),
- argtype=memb.type.c_type())
-else:
-ret += mcgen('''
-%(argtype)s %(argname)s = {0};
-''',
- argname=c_name(memb.name),
- argtype=memb.type.c_type())
+ c_name=c_name(memb.name),
+ c_type=memb.type.c_type(),
+ c_null=memb.type.c_null())
 
 pop_indent()
 return ret
diff --git a/scripts/qapi.py b/scripts/qapi.py
index ba32aac..0ffd02d 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1441,9 +1441,6 @@ def c_type(value, is_param=False):
 assert isinstance(value, str) and value != ""
 return c_name(value) + pointer_suffix
 
-def is_c_ptr(value):
-return value.endswith(pointer_suffix)
-
 def genindent(count):
 ret = ""
 for i in range(count):
-- 
2.4.3

[Qemu-devel] [PULL 08/26] qapi-commands: Convert to QAPISchemaVisitor

[Markus Armbruster]

Output unchanged apart from reordering and white-space.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Message-Id: <1442401589-24189-9-git-send-email-arm...@redhat.com>
Reviewed-by: Daniel P. Berrange 
---
 scripts/qapi-commands.py | 159 +++
 scripts/qapi.py  |   2 +-
 2 files changed, 91 insertions(+), 70 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 12bdc4c..a68517a 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -12,21 +12,22 @@
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-from ordereddict import OrderedDict
 from qapi import *
 import re
 
 def generate_command_decl(name, args, ret_type):
 arglist=""
-for argname, argtype, optional in parse_args(args):
-argtype = c_type(argtype, is_param=True)
-if optional:
-arglist += "bool has_%s, " % c_name(argname)
-arglist += "%s %s, " % (argtype, c_name(argname))
+if args:
+for memb in args.members:
+argtype = memb.type.c_type(is_param=True)
+if memb.optional:
+arglist += "bool has_%s, " % c_name(memb.name)
+arglist += "%s %s, " % (argtype, c_name(memb.name))
 return mcgen('''
 %(ret_type)s qmp_%(name)s(%(args)sError **errp);
 ''',
- ret_type=c_type(ret_type), name=c_name(name),
+ ret_type=(ret_type and ret_type.c_type()) or 'void',
+ name=c_name(name),
  args=arglist)
 
 def gen_err_check(err):
@@ -45,10 +46,11 @@ def gen_sync_call(name, args, ret_type):
 retval=""
 if ret_type:
 retval = "retval = "
-for argname, argtype, optional in parse_args(args):
-if optional:
-arglist += "has_%s, " % c_name(argname)
-arglist += "%s, " % (c_name(argname))
+if args:
+for memb in args.members:
+if memb.optional:
+arglist += "has_%s, " % c_name(memb.name)
+arglist += "%s, " % c_name(memb.name)
 push_indent()
 ret = mcgen('''
 %(retval)sqmp_%(name)s(%(args)s_err);
@@ -68,7 +70,7 @@ def gen_visitor_input_containers_decl(args):
 ret = ""
 
 push_indent()
-if len(args) > 0:
+if args:
 ret += mcgen('''
 QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
 QapiDeallocVisitor *md;
@@ -81,22 +83,26 @@ Visitor *v;
 def gen_visitor_input_vars_decl(args):
 ret = ""
 push_indent()
-for argname, argtype, optional in parse_args(args):
-if optional:
-ret += mcgen('''
+
+if args:
+for memb in args.members:
+if memb.optional:
+ret += mcgen('''
 bool has_%(argname)s = false;
 ''',
- argname=c_name(argname))
-if is_c_ptr(argtype):
-ret += mcgen('''
+ argname=c_name(memb.name))
+if is_c_ptr(memb.type.c_type()):
+ret += mcgen('''
 %(argtype)s %(argname)s = NULL;
 ''',
- argname=c_name(argname), argtype=c_type(argtype))
-else:
-ret += mcgen('''
+ argname=c_name(memb.name),
+ argtype=memb.type.c_type())
+else:
+ret += mcgen('''
 %(argtype)s %(argname)s = {0};
 ''',
- argname=c_name(argname), argtype=c_type(argtype))
+ argname=c_name(memb.name),
+ argtype=memb.type.c_type())
 
 pop_indent()
 return ret
@@ -106,7 +112,7 @@ def gen_visitor_input_block(args, dealloc=False):
 errparg = '_err'
 errarg = 'local_err'
 
-if len(args) == 0:
+if not args:
 return ret
 
 push_indent()
@@ -124,25 +130,26 @@ v = qapi_dealloc_get_visitor(md);
 v = qmp_input_get_visitor(mi);
 ''')
 
-for argname, argtype, optional in parse_args(args):
-if optional:
+for memb in args.members:
+if memb.optional:
 ret += mcgen('''
 visit_optional(v, _%(c_name)s, "%(name)s", %(errp)s);
 ''',
- c_name=c_name(argname), name=argname, errp=errparg)
+ c_name=c_name(memb.name), name=memb.name,
+ errp=errparg)
 ret += gen_err_check(errarg)
 ret += mcgen('''
 if (has_%(c_name)s) {
 ''',
- c_name=c_name(argname))
+ c_name=c_name(memb.name))
 push_indent()
 ret += mcgen('''
 visit_type_%(visitor)s(v, &%(c_name)s, "%(name)s", %(errp)s);
 ''',
- c_name=c_name(argname), name=argname, argtype=argtype,
- visitor=type_name(argtype), errp=errparg)
+ c_name=c_name(memb.name), name=memb.name,
+

[Qemu-devel] [PULL 00/26] qapi: QMP introspection

[Markus Armbruster]

The following changes since commit 18640989a9f5e4d2e84b566c52ff1fccfa0dbf4a:

  Merge remote-tracking branch 'remotes/afaerber/tags/qom-devices-for-peter' 
into staging (2015-09-19 15:59:52 +0100)

are available in the git repository at:

  git://repo.or.cz/qemu/armbru.git tags/pull-qapi-2015-09-21

for you to fetch changes up to 1a9a507b2e3e90aa719c96b4c092e7fad7215f21:

  qapi-introspect: Hide type names (2015-09-21 09:56:49 +0200)


qapi: QMP introspection


Markus Armbruster (26):
  qapi: Rename class QAPISchema to QAPISchemaParser
  qapi: New QAPISchema intermediate reperesentation
  qapi: QAPISchema code generation helper methods
  qapi: New QAPISchemaVisitor
  tests/qapi-schema: Convert test harness to QAPISchemaVisitor
  qapi-types: Convert to QAPISchemaVisitor, fixing flat unions
  qapi-visit: Convert to QAPISchemaVisitor, fixing bugs
  qapi-commands: Convert to QAPISchemaVisitor
  qapi: De-duplicate enum code generation
  qapi-event: Eliminate global variable event_enum_value
  qapi-event: Convert to QAPISchemaVisitor, fixing data with base
  qapi: Replace dirty is_c_ptr() by method c_null()
  qapi: Clean up after recent conversions to QAPISchemaVisitor
  qapi-visit: Rearrange code a bit
  qapi-commands: Rearrange code
  qapi: Rename qmp_marshal_input_FOO() to qmp_marshal_FOO()
  qapi: De-duplicate parameter list generation
  qapi-commands: De-duplicate output marshaling functions
  qapi: Improve built-in type documentation
  qapi: Make output visitor return qnull() instead of NULL
  qapi: Introduce a first class 'any' type
  qom: Don't use 'gen': false for qom-get, qom-set, object-add
  qapi-schema: Fix up misleading specification of netdev_add
  qapi: Pseudo-type '**' is now unused, drop it
  qapi: New QMP command query-qmp-schema for QMP introspection
  qapi-introspect: Hide type names

 .gitignore |   1 +
 Makefile   |   9 +-
 Makefile.objs  |   4 +-
 docs/qapi-code-gen.txt | 355 +--
 docs/writing-qmp-commands.txt  |   8 +-
 include/monitor/monitor.h  |   3 -
 include/qapi/visitor-impl.h|   2 +
 include/qapi/visitor.h |   1 +
 monitor.c  |  18 +-
 qapi-schema.json   |  25 +-
 qapi/introspect.json   | 273 
 qapi/qapi-dealloc-visitor.c|   9 +
 qapi/qapi-visit-core.c |   6 +
 qapi/qmp-input-visitor.c   |  11 +
 qapi/qmp-output-visitor.c  |  17 +-
 qmp-commands.hx| 269 
 qmp.c  |  27 +-
 scripts/qapi-commands.py   | 286 +
 scripts/qapi-event.py  | 243 +++
 scripts/qapi-introspect.py | 213 +++
 scripts/qapi-types.py  | 375 +--
 scripts/qapi-visit.py  | 369 +--
 scripts/qapi.py| 702 ++---
 tests/.gitignore   |   1 +
 tests/Makefile |  15 +-
 tests/qapi-schema/alternate-good.out   |  16 +-
 tests/qapi-schema/args-any.err |   1 +
 .../{type-bypass-no-gen.exit => args-any.exit} |   0
 tests/qapi-schema/args-any.json|   2 +
 .../qapi-schema/{type-bypass.err => args-any.out}  |   0
 tests/qapi-schema/args-member-array.out|  14 +-
 tests/qapi-schema/comments.out |   5 +-
 tests/qapi-schema/empty.out|   4 +-
 tests/qapi-schema/enum-empty.out   |   5 +-
 tests/qapi-schema/event-case.out   |   5 +-
 tests/qapi-schema/flat-union-base-any.err  |   1 +
 ...ion-base-star.exit => flat-union-base-any.exit} |   0
 ...ion-base-star.json => flat-union-base-any.json} |   2 +-
 ...e-bypass-no-gen.out => flat-union-base-any.out} |   0
 tests/qapi-schema/flat-union-base-star.err |   1 -
 tests/qapi-schema/flat-union-base-star.out |   0
 tests/qapi-schema/flat-union-reverse-define.out|  22 +-
 tests/qapi-schema/ident-with-escape.out|   8 +-
 tests/qapi-schema/include-relpath.out  |   5 +-
 tests/qapi-schema/include-repetition.out   |   5 +-
 tests/qapi-schema/include-simple.out   |   5 +-
 tests/qapi-schema/indented-expr.out|   8 +-
 

[Qemu-devel] [PULL 20/26] qapi: Make output visitor return qnull() instead of NULL

[Markus Armbruster]

Before commit 1d10b44, it crashed.  Since then, it returns NULL, with
a FIXME comment.  The FIXME is valid: code that assumes QObject *
can't be null exists.  I'm not aware of a way to feed this problematic
return value to code that actually chokes on null in the current code,
but the next few commits will create one, failing "make check".

Commit 481b002 solved a very similar problem by introducing a special
null QObject.  Using this special null QObject is clearly the right
way to resolve this FIXME, so do that, and update the test
accordingly.

However, the patch isn't quite right: it messes up the reference
counting.  After about SIZE_MAX visits, the reference counter
overflows, failing the assertion in qnull_destroy_obj().  Because
that's many orders of magnitude more visits of nulls than we expect,
we take this patch despite its flaws, to get the QMP introspection
stuff in without further delay.  We'll want to fix it for real before
the release.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-21-git-send-email-arm...@redhat.com>
---
 qapi/qmp-output-visitor.c   | 8 ++--
 tests/test-qmp-output-visitor.c | 3 ++-
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/qapi/qmp-output-visitor.c b/qapi/qmp-output-visitor.c
index efc19d5..8dce087 100644
--- a/qapi/qmp-output-visitor.c
+++ b/qapi/qmp-output-visitor.c
@@ -66,9 +66,13 @@ static QObject *qmp_output_first(QmpOutputVisitor *qov)
 {
 QStackEntry *e = QTAILQ_LAST(>stack, QStack);
 
-/* FIXME - find a better way to deal with NULL values */
+/*
+ * FIXME Wrong, because qmp_output_get_qobject() will increment
+ * the refcnt *again*.  We need to think through how visitors
+ * handle null.
+ */
 if (!e) {
-return NULL;
+return qnull();
 }
 
 return e->value;
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 338ada0..a48ae72 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -485,7 +485,8 @@ static void test_visitor_out_empty(TestOutputVisitorData 
*data,
 QObject *arg;
 
 arg = qmp_output_get_qobject(data->qov);
-g_assert(!arg);
+g_assert(qobject_type(arg) == QTYPE_QNULL);
+qobject_decref(arg);
 }
 
 static void init_native_list(UserDefNativeListUnion *cvalue)
-- 
2.4.3

[Qemu-devel] [PULL 23/26] qapi-schema: Fix up misleading specification of netdev_add

[Markus Armbruster]

It doesn't take a 'props' argument, let alone one in the format
"NAME=VALUE,..."

The bogus arguments specification doesn't matter due to 'gen': false.
Clean it up to be incomplete rather than wrong, and document the
incompleteness.

While there, improve netdev_add usage example in the manual: add a
device option to show how it's done.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-24-git-send-email-arm...@redhat.com>
---
 docs/qapi-code-gen.txt |  2 +-
 qapi-schema.json   | 13 +++--
 qmp-commands.hx|  4 +++-
 3 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 70fdae7..4e331a0 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -471,7 +471,7 @@ try to avoid adding new commands that rely on this, and 
instead use
 type-safe unions.  For an example of bypass usage:
 
  { 'command': 'netdev_add',
-   'data': {'type': 'str', 'id': 'str', '*props': '**'},
+   'data': {'type': 'str', 'id': 'str'},
'gen': false }
 
 Normally, the QAPI schema is used to describe synchronous exchanges,
diff --git a/qapi-schema.json b/qapi-schema.json
index 756ed9e..3ff9fec 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2079,11 +2079,12 @@
 #
 # @id: the name of the new network backend
 #
-# @props: #optional a list of properties to be passed to the backend in
-# the format 'name=value', like 'ifname=tap0,script=no'
+# Additional arguments depend on the type.
 #
-# Notes: The semantics of @props is not well defined.  Future commands will be
-#introduced that provide stronger typing for backend creation.
+# TODO This command effectively bypasses QAPI completely due to its
+# "additional arguments" business.  It shouldn't have been added to
+# the schema in this form.  It should be qapified properly, or
+# replaced by a properly qapified command.
 #
 # Since: 0.14.0
 #
@@ -2091,8 +2092,8 @@
 #  If @type is not a valid network backend, DeviceNotFound
 ##
 { 'command': 'netdev_add',
-  'data': {'type': 'str', 'id': 'str', '*props': '**'},
-  'gen': false }
+  'data': {'type': 'str', 'id': 'str'},
+  'gen': false }# so we can get the additional arguments
 
 ##
 # @netdev_del:
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 7c74e20..5a54406 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -917,7 +917,9 @@ Arguments:
 
 Example:
 
--> { "execute": "netdev_add", "arguments": { "type": "user", "id": "netdev1" } 
}
+-> { "execute": "netdev_add",
+ "arguments": { "type": "user", "id": "netdev1",
+"dnssearch": "example.org" } }
 <- { "return": {} }
 
 Note: The supported device options are the same ones supported by the '-netdev'
-- 
2.4.3

[Qemu-devel] [PULL 25/26] qapi: New QMP command query-qmp-schema for QMP introspection

[Markus Armbruster]

qapi/introspect.json defines the introspection schema.  It's designed
for QMP introspection, but should do for similar uses, such as QGA.

The introspection schema does not reflect all the rules and
restrictions that apply to QAPI schemata.  A valid QAPI schema has an
introspection value conforming to the introspection schema, but the
converse is not true.

Introspection lowers away a number of schema details, and makes
implicit things explicit:

* The built-in types are declared with their JSON type.

  All integer types are mapped to 'int', because how many bits we use
  internally is an implementation detail.  It could be pressed into
  external interface service as very approximate range information,
  but that's a bad idea.  If we need range information, we better do
  it properly.

* Implicit type definitions are made explicit, and given
  auto-generated names:

  - Array types, named by appending "List" to the name of their
element type, like in generated C.

  - The enumeration types implicitly defined by simple union types,
named by appending "Kind" to the name of their simple union type,
like in generated C.

  - Types that don't occur in generated C.  Their names start with ':'
so they don't clash with the user's names.

* All type references are by name.

* The struct and union types are generalized into an object type.

* Base types are flattened.

* Commands take a single argument and return a single result.

  Dictionary argument or list result is an implicit type definition.

  The empty object type is used when a command takes no arguments or
  produces no results.

  The argument is always of object type, but the introspection schema
  doesn't reflect that.

  The 'gen': false directive is omitted as implementation detail.

  The 'success-response' directive is omitted as well for now, even
  though it's not an implementation detail, because it's not used by
  QMP.

* Events carry a single data value.

  Implicit type definition and empty object type use, just like for
  commands.

  The value is of object type, but the introspection schema doesn't
  reflect that.

* Types not used by commands or events are omitted.

  Indirect use counts as use.

* Optional members have a default, which can only be null right now

  Instead of a mandatory "optional" flag, we have an optional default.
  No default means mandatory, default null means optional without
  default value.  Non-null is available for optional with default
  (possible future extension).

* Clients should *not* look up types by name, because type names are
  not ABI.  Look up the command or event you're interested in, then
  follow the references.

  TODO Should we hide the type names to eliminate the temptation?

New generator scripts/qapi-introspect.py computes an introspection
value for its input, and generates a C variable holding it.

It can generate awfully long lines.  Marked TODO.

A new test-qmp-input-visitor test case feeds its result for both
tests/qapi-schema/qapi-schema-test.json and qapi-schema.json to a
QmpInputVisitor to verify it actually conforms to the schema.

New QMP command query-qmp-schema takes its return value from that
variable.  Its reply is some 85KiBytes for me right now.

If this turns out to be too much, we have a couple of options:

* We can use shorter names in the JSON.  Not the QMP style.

* Optionally return the sub-schema for commands and events given as
  arguments.

  Right now qmp_query_schema() sends the string literal computed by
  qmp-introspect.py.  To compute sub-schema at run time, we'd have to
  duplicate parts of qapi-introspect.py in C.  Unattractive.

* Let clients cache the output of query-qmp-schema.

  It changes only on QEMU upgrades, i.e. rarely.  Provide a command
  query-qmp-schema-hash.  Clients can have a cache indexed by hash,
  and re-query the schema only when they don't have it cached.  Even
  simpler: put the hash in the QMP greeting.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 .gitignore  |   1 +
 Makefile|   9 +-
 Makefile.objs   |   4 +-
 docs/qapi-code-gen.txt  | 237 +++-
 monitor.c   |  16 ++
 qapi-schema.json|   3 +
 qapi/introspect.json| 277 
 qmp-commands.hx |  17 ++
 scripts/qapi-introspect.py  | 184 
 scripts/qapi.py |  13 +-
 tests/.gitignore|   1 +
 tests/Makefile  |  10 +-
 tests/qapi-schema/alternate-good.out|   1 +
 tests/qapi-schema/args-member-array.out |   1 +
 tests/qapi-schema/comments.out  |   1 +
 

[Qemu-devel] [PULL 02/26] qapi: New QAPISchema intermediate reperesentation

[Markus Armbruster]

The QAPI code generators work with a syntax tree (nested dictionaries)
plus a few symbol tables (also dictionaries) on the side.

They have clearly outgrown these simple data structures.  There's lots
of rummaging around in dictionaries, and information is recomputed on
the fly.  For the work I'm going to do, I want more clearly defined
and more convenient interfaces.

Going forward, I also want less coupling between the back-ends and the
syntax tree, to make messing with the syntax easier.

Create a bunch of classes to represent QAPI schemata.

Have the QAPISchema initializer call the parser, then walk the syntax
tree to create the new internal representation, and finally perform
semantic analysis.

Shortcut: the semantic analysis still relies on existing check_exprs()
to do the actual semantic checking.  All this code needs to move into
the classes.  Mark as TODO.

Simple unions are lowered to flat unions.  Flat unions and structs are
represented as a more general object type.

Catching name collisions in generated code would be nice.  Mark as
TODO.

We generate array types eagerly, even though most of them aren't used.
Mark as TODO.

Nothing uses the new intermediate representation just yet, thus no
change to generated files.

Signed-off-by: Markus Armbruster 
Reviewed-by: Daniel P. Berrange 
Reviewed-by: Eric Blake 
---
 scripts/qapi-commands.py   |   2 +-
 scripts/qapi-event.py  |   2 +-
 scripts/qapi-types.py  |   2 +-
 scripts/qapi-visit.py  |   2 +-
 scripts/qapi.py| 382 -
 tests/qapi-schema/test-qapi.py |   2 +-
 6 files changed, 378 insertions(+), 14 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 890ce5d..12bdc4c 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -272,7 +272,7 @@ for o, a in opts:
 if o in ("-m", "--middle"):
 middle_mode = True
 
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
 commands = filter(lambda expr: expr.has_key('command'), exprs)
 commands = filter(lambda expr: not expr.has_key('gen'), commands)
 
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 7f238df..aec2d32 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -263,7 +263,7 @@ fdecl.write(mcgen('''
 ''',
   prefix=prefix))
 
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
 
 event_enum_name = c_name(prefix + "QAPIEvent", protect=False)
 event_enum_values = []
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index a8453d1..23e9505 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -336,7 +336,7 @@ fdecl.write(mcgen('''
 #include 
 '''))
 
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
 
 fdecl.write(guardstart("QAPI_TYPES_BUILTIN_STRUCT_DECL"))
 for typename in builtin_types.keys():
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 3cd662b..c493964 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -446,7 +446,7 @@ fdecl.write(mcgen('''
 ''',
   prefix=prefix))
 
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
 
 # to avoid header dependency hell, we always generate declarations
 # for built-in types in our header files and simply guard them
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 2a0f465..d0dfabe 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -302,6 +302,8 @@ class QAPISchemaParser(object):
 
 #
 # Semantic analysis of schema expressions
+# TODO fold into QAPISchema
+# TODO catching name collisions in generated code would be nice
 #
 
 def find_base_fields(base):
@@ -751,15 +753,377 @@ def check_exprs(exprs):
 else:
 assert False, 'unexpected meta type'
 
-return map(lambda expr_elem: expr_elem['expr'], exprs)
-
-def parse_schema(fname):
-try:
-schema = QAPISchemaParser(open(fname, "r"))
-return check_exprs(schema.exprs)
-except (QAPISchemaError, QAPIExprError), e:
-print >>sys.stderr, e
-exit(1)
+return exprs
+
+
+#
+# Schema compiler frontend
+#
+
+class QAPISchemaEntity(object):
+def __init__(self, name, info):
+assert isinstance(name, str)
+self.name = name
+self.info = info
+
+def check(self, schema):
+pass
+
+
+class QAPISchemaType(QAPISchemaEntity):
+pass
+
+
+class QAPISchemaBuiltinType(QAPISchemaType):
+def __init__(self, name):
+QAPISchemaType.__init__(self, name, None)
+
+
+class QAPISchemaEnumType(QAPISchemaType):
+def __init__(self, name, info, values, prefix):
+QAPISchemaType.__init__(self, name, info)
+for v in values:
+assert isinstance(v, str)
+assert prefix is None or isinstance(prefix, str)
+self.values = values
+self.prefix = prefix
+
+def check(self, 

[Qemu-devel] [PULL 15/26] qapi-commands: Rearrange code

[Markus Armbruster]

Rename gen_marshal_input() to gen_marshal(), because the generated
function marshals both arguments and results.

Rename gen_visitor_input_containers_decl() to gen_marshal_vars(), and
move the other variable declarations there, too.

Rename gen_visitor_input_block() to gen_marshal_input_visit(), and
rearrange its code slightly.

Rename gen_marshal_input_decl() to gen_marshal_proto(), because the
result isn't a full declaration, unlike gen_command_decl()'s.

New gen_marshal_decl() actually returns a full declaration.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-16-git-send-email-arm...@redhat.com>
---
 scripts/qapi-commands.py | 87 ++--
 1 file changed, 39 insertions(+), 48 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 0501582..6a52c88 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -59,6 +59,7 @@ def gen_call(name, arg_type, ret_type):
 
 push_indent()
 ret = mcgen('''
+
 %(lhs)sqmp_%(c_name)s(%(args)s_err);
 ''',
 c_name=c_name(name), args=argstr, lhs=lhs)
@@ -73,26 +74,26 @@ qmp_marshal_output_%(c_name)s(retval, ret, _err);
 return ret
 
 
-def gen_visitor_input_containers_decl(arg_type):
-ret = ''
+def gen_marshal_vars(arg_type, ret_type):
+ret = mcgen('''
+Error *local_err = NULL;
+''')
 
 push_indent()
+
+if ret_type:
+ret += mcgen('''
+%(c_type)s retval;
+''',
+ c_type=ret_type.c_type())
+
 if arg_type:
 ret += mcgen('''
 QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
 QapiDeallocVisitor *md;
 Visitor *v;
 ''')
-pop_indent()
 
-return ret
-
-
-def gen_visitor_input_vars_decl(arg_type):
-ret = ''
-push_indent()
-
-if arg_type:
 for memb in arg_type.members:
 if memb.optional:
 ret += mcgen('''
@@ -105,15 +106,19 @@ bool has_%(c_name)s = false;
  c_name=c_name(memb.name),
  c_type=memb.type.c_type(),
  c_null=memb.type.c_null())
+ret += '\n'
+else:
+ret += mcgen('''
+
+(void)args;
+''')
 
 pop_indent()
 return ret
 
 
-def gen_visitor_input_block(arg_type, dealloc=False):
+def gen_marshal_input_visit(arg_type, dealloc=False):
 ret = ''
-errparg = '_err'
-errarg = 'local_err'
 
 if not arg_type:
 return ret
@@ -129,6 +134,8 @@ md = qapi_dealloc_visitor_new();
 v = qapi_dealloc_get_visitor(md);
 ''')
 else:
+errparg = '_err'
+errarg = 'local_err'
 ret += mcgen('''
 v = qmp_input_get_visitor(mi);
 ''')
@@ -167,10 +174,7 @@ qapi_dealloc_visitor_cleanup(md);
 
 
 def gen_marshal_output(name, ret_type):
-if not ret_type:
-return ''
-
-ret = mcgen('''
+return mcgen('''
 
 static void qmp_marshal_output_%(c_cmd_name)s(%(c_type)s ret_in, QObject 
**ret_out, Error **errp)
 {
@@ -195,47 +199,34 @@ out:
 qapi_dealloc_visitor_cleanup(md);
 }
 ''',
-c_type=ret_type.c_type(), c_cmd_name=c_name(name),
-c_name=ret_type.c_name())
+ c_type=ret_type.c_type(), c_cmd_name=c_name(name),
+ c_name=ret_type.c_name())
 
-return ret
 
-
-def gen_marshal_input_decl(name):
+def gen_marshal_proto(name):
 ret = 'void qmp_marshal_input_%s(QDict *args, QObject **ret, Error 
**errp)' % c_name(name)
 if not middle_mode:
 ret = 'static ' + ret
 return ret
 
 
-def gen_marshal_input(name, arg_type, ret_type):
-hdr = gen_marshal_input_decl(name)
+def gen_marshal_decl(name):
+return mcgen('''
+%(proto)s;
+''',
+ proto=gen_marshal_proto(name))
 
+
+def gen_marshal(name, arg_type, ret_type):
 ret = mcgen('''
 
-%(header)s
+%(proto)s
 {
-Error *local_err = NULL;
 ''',
-header=hdr)
-
-if ret_type:
-ret += mcgen('''
-%(c_type)s retval;
-''',
- c_type=ret_type.c_type())
-
-if arg_type:
-ret += gen_visitor_input_containers_decl(arg_type)
-ret += gen_visitor_input_vars_decl(arg_type) + '\n'
-ret += gen_visitor_input_block(arg_type) + '\n'
-else:
-ret += mcgen('''
-
-(void)args;
-
-''')
+proto=gen_marshal_proto(name))
 
+ret += gen_marshal_vars(arg_type, ret_type)
+ret += gen_marshal_input_visit(arg_type)
 ret += gen_call(name, arg_type, ret_type)
 
 if re.search('^ *goto out;', ret, re.MULTILINE):
@@ -246,7 +237,7 @@ out:
 ret += mcgen('''
 error_propagate(errp, local_err);
 ''')
-ret += gen_visitor_input_block(arg_type, dealloc=True)
+ret += gen_marshal_input_visit(arg_type, dealloc=True)
 ret += mcgen('''
 }
 ''')
@@ -307,8 +298,8 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
 if ret_type:
   

[Qemu-devel] [PULL 16/26] qapi: Rename qmp_marshal_input_FOO() to qmp_marshal_FOO()

[Markus Armbruster]

These functions marshal both input and output.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-17-git-send-email-arm...@redhat.com>
---
 docs/qapi-code-gen.txt|   4 +-
 docs/writing-qmp-commands.txt |   8 +-
 monitor.c |   2 +-
 qmp-commands.hx   | 242 +-
 qmp.c |   6 +-
 scripts/qapi-commands.py  |   4 +-
 6 files changed, 133 insertions(+), 133 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 3eaff36..50da1d0 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -744,7 +744,7 @@ Example:
 qapi_dealloc_visitor_cleanup(md);
 }
 
-static void qmp_marshal_input_my_command(QDict *args, QObject **ret, Error 
**errp)
+static void qmp_marshal_my_command(QDict *args, QObject **ret, Error 
**errp)
 {
 Error *local_err = NULL;
 UserDefOne *retval;
@@ -777,7 +777,7 @@ Example:
 
 static void qmp_init_marshal(void)
 {
-qmp_register_command("my-command", qmp_marshal_input_my_command, 
QCO_NO_OPTIONS);
+qmp_register_command("my-command", qmp_marshal_my_command, 
QCO_NO_OPTIONS);
 }
 
 qapi_init(qmp_init_marshal);
diff --git a/docs/writing-qmp-commands.txt b/docs/writing-qmp-commands.txt
index f7693ca..8647cac 100644
--- a/docs/writing-qmp-commands.txt
+++ b/docs/writing-qmp-commands.txt
@@ -127,7 +127,7 @@ following at the bottom:
 {
 .name   = "hello-world",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_hello_world,
+.mhandler.cmd_new = qmp_marshal_hello_world,
 },
 
 You're done. Now build qemu, run it as suggested in the "Testing" section,
@@ -179,7 +179,7 @@ The last step is to update the qmp-commands.hx file:
 {
 .name   = "hello-world",
 .args_type  = "message:s?",
-.mhandler.cmd_new = qmp_marshal_input_hello_world,
+.mhandler.cmd_new = qmp_marshal_hello_world,
 },
 
 Notice that the "args_type" member got our "message" argument. The character
@@ -461,7 +461,7 @@ The last step is to add the correspoding entry in the 
qmp-commands.hx file:
 {
 .name   = "query-alarm-clock",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_query_alarm_clock,
+.mhandler.cmd_new = qmp_marshal_query_alarm_clock,
 },
 
 Time to test the new command. Build qemu, run it as described in the "Testing"
@@ -607,7 +607,7 @@ To test this you have to add the corresponding 
qmp-commands.hx entry:
 {
 .name   = "query-alarm-methods",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_query_alarm_methods,
+.mhandler.cmd_new = qmp_marshal_query_alarm_methods,
 },
 
 Now Build qemu, run it as explained in the "Testing" section and try our new
diff --git a/monitor.c b/monitor.c
index 1f43263..dde1f22 100644
--- a/monitor.c
+++ b/monitor.c
@@ -3912,7 +3912,7 @@ static QObject *get_qmp_greeting(void)
 {
 QObject *ver = NULL;
 
-qmp_marshal_input_query_version(NULL, , NULL);
+qmp_marshal_query_version(NULL, , NULL);
 return qobject_from_jsonf("{'QMP':{'version': %p,'capabilities': 
[]}}",ver);
 }
 
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 9848fd8..4640a3d 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -63,7 +63,7 @@ EQMP
 {
 .name   = "quit",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_quit,
+.mhandler.cmd_new = qmp_marshal_quit,
 },
 
 SQMP
@@ -84,7 +84,7 @@ EQMP
 {
 .name   = "eject",
 .args_type  = "force:-f,device:B",
-.mhandler.cmd_new = qmp_marshal_input_eject,
+.mhandler.cmd_new = qmp_marshal_eject,
 },
 
 SQMP
@@ -110,7 +110,7 @@ EQMP
 {
 .name   = "change",
 .args_type  = "device:B,target:F,arg:s?",
-.mhandler.cmd_new = qmp_marshal_input_change,
+.mhandler.cmd_new = qmp_marshal_change,
 },
 
 SQMP
@@ -146,7 +146,7 @@ EQMP
 {
 .name   = "screendump",
 .args_type  = "filename:F",
-.mhandler.cmd_new = qmp_marshal_input_screendump,
+.mhandler.cmd_new = qmp_marshal_screendump,
 },
 
 SQMP
@@ -169,7 +169,7 @@ EQMP
 {
 .name   = "stop",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_stop,
+.mhandler.cmd_new = qmp_marshal_stop,
 },
 
 SQMP
@@ -190,7 +190,7 @@ EQMP
 {
 .name   = "cont",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_cont,
+.mhandler.cmd_new = qmp_marshal_cont,
 },
 
 SQMP
@@ -211,7 +211,7 @@ EQMP
 {
 .name   = "system_wakeup",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_system_wakeup,
+

[Qemu-devel] [PULL 22/26] qom: Don't use 'gen': false for qom-get, qom-set, object-add

[Markus Armbruster]

With the previous commit, the generated marshalers just work, and save
us a bit of handwritten code.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-23-git-send-email-arm...@redhat.com>
---
 include/monitor/monitor.h |  3 ---
 qapi-schema.json  |  9 +++--
 qmp-commands.hx   |  6 +++---
 qmp.c | 21 +++--
 scripts/qapi.py   |  1 +
 5 files changed, 14 insertions(+), 26 deletions(-)

diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 9aff47e..bc6cb6d 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -42,9 +42,6 @@ void monitor_read_command(Monitor *mon, int show_prompt);
 int monitor_read_password(Monitor *mon, ReadLineFunc *readline_func,
   void *opaque);
 
-void qmp_qom_set(QDict *qdict, QObject **ret, Error **errp);
-void qmp_qom_get(QDict *qdict, QObject **ret, Error **errp);
-void qmp_object_add(QDict *qdict, QObject **ret, Error **errp);
 void object_add(const char *type, const char *id, const QDict *qdict,
 Visitor *v, Error **errp);
 
diff --git a/qapi-schema.json b/qapi-schema.json
index 2bada60..756ed9e 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1698,8 +1698,7 @@
 ##
 { 'command': 'qom-get',
   'data': { 'path': 'str', 'property': 'str' },
-  'returns': '**',
-  'gen': false }
+  'returns': 'any' }
 
 ##
 # @qom-set:
@@ -1716,8 +1715,7 @@
 # Since: 1.2
 ##
 { 'command': 'qom-set',
-  'data': { 'path': 'str', 'property': 'str', 'value': '**' },
-  'gen': false }
+  'data': { 'path': 'str', 'property': 'str', 'value': 'any' } }
 
 ##
 # @set_password:
@@ -2127,8 +2125,7 @@
 # Since: 2.0
 ##
 { 'command': 'object-add',
-  'data': {'qom-type': 'str', 'id': 'str', '*props': '**'},
-  'gen': false }
+  'data': {'qom-type': 'str', 'id': 'str', '*props': 'any'} }
 
 ##
 # @object-del:
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 4640a3d..7c74e20 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -953,7 +953,7 @@ EQMP
 {
 .name   = "object-add",
 .args_type  = "qom-type:s,id:s,props:q?",
-.mhandler.cmd_new = qmp_object_add,
+.mhandler.cmd_new = qmp_marshal_object_add,
 },
 
 SQMP
@@ -3575,13 +3575,13 @@ EQMP
 {
 .name   = "qom-set",
.args_type  = "path:s,property:s,value:q",
-   .mhandler.cmd_new = qmp_qom_set,
+.mhandler.cmd_new = qmp_marshal_qom_set,
 },
 
 {
 .name   = "qom-get",
.args_type  = "path:s,property:s",
-   .mhandler.cmd_new = qmp_qom_get,
+.mhandler.cmd_new = qmp_marshal_qom_get,
 },
 
 {
diff --git a/qmp.c b/qmp.c
index 3feae9f..057a7cb 100644
--- a/qmp.c
+++ b/qmp.c
@@ -234,12 +234,9 @@ ObjectPropertyInfoList *qmp_qom_list(const char *path, 
Error **errp)
 return props;
 }
 
-/* FIXME: teach qapi about how to pass through Visitors */
-void qmp_qom_set(QDict *qdict, QObject **ret, Error **errp)
+void qmp_qom_set(const char *path, const char *property, QObject *value,
+ Error **errp)
 {
-const char *path = qdict_get_str(qdict, "path");
-const char *property = qdict_get_str(qdict, "property");
-QObject *value = qdict_get(qdict, "value");
 Object *obj;
 
 obj = object_resolve_path(path, NULL);
@@ -252,20 +249,18 @@ void qmp_qom_set(QDict *qdict, QObject **ret, Error 
**errp)
 object_property_set_qobject(obj, value, property, errp);
 }
 
-void qmp_qom_get(QDict *qdict, QObject **ret, Error **errp)
+QObject *qmp_qom_get(const char *path, const char *property, Error **errp)
 {
-const char *path = qdict_get_str(qdict, "path");
-const char *property = qdict_get_str(qdict, "property");
 Object *obj;
 
 obj = object_resolve_path(path, NULL);
 if (!obj) {
 error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
   "Device '%s' not found", path);
-return;
+return NULL;
 }
 
-*ret = object_property_get_qobject(obj, property, errp);
+return object_property_get_qobject(obj, property, errp);
 }
 
 void qmp_set_password(const char *protocol, const char *password,
@@ -661,11 +656,9 @@ out:
 object_unref(obj);
 }
 
-void qmp_object_add(QDict *qdict, QObject **ret, Error **errp)
+void qmp_object_add(const char *type, const char *id,
+bool has_props, QObject *props, Error **errp)
 {
-const char *type = qdict_get_str(qdict, "qom-type");
-const char *id = qdict_get_str(qdict, "id");
-QObject *props = qdict_get(qdict, "props");
 const QDict *pdict = NULL;
 QmpInputVisitor *qiv;
 
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 6a21bd6..8808ae6 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -40,6 +40,7 @@ builtin_types = {
 returns_whitelist = [
 # From QMP:
 'human-monitor-command',
+'qom-get',
 

Re: [Qemu-devel] [PATCH v4 09/22] vhost: use a function for each call

[Thibaut Collet]

On Sat, Sep 19, 2015 at 12:12 PM,   wrote:
> From: Marc-André Lureau 
>
> Replace the generic vhost_call() by specific functions for each
> function call to help with type safety and changing arguments.
>
> Signed-off-by: Marc-André Lureau 
> ---
>  hw/net/vhost_net.c|  12 +-
>  hw/scsi/vhost-scsi.c  |   7 +-
>  hw/virtio/vhost-backend.c | 140 +++--
>  hw/virtio/vhost-user.c| 402 
> ++
>  hw/virtio/vhost.c |  34 ++--
>  include/hw/virtio/vhost-backend.h |  59 +-
>  6 files changed, 484 insertions(+), 170 deletions(-)
>
> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
> index 9d32d76..d116fb3 100644
> --- a/hw/net/vhost_net.c
> +++ b/hw/net/vhost_net.c
> @@ -243,8 +243,7 @@ static int vhost_net_start_one(struct vhost_net *net,
>  file.fd = net->backend;
>  for (file.index = 0; file.index < net->dev.nvqs; ++file.index) {
>  const VhostOps *vhost_ops = net->dev.vhost_ops;
> -r = vhost_ops->vhost_call(>dev, VHOST_NET_SET_BACKEND,
> -  );
> +r = vhost_ops->vhost_net_set_backend(>dev, );
>  if (r < 0) {
>  r = -errno;
>  goto fail;
> @@ -257,8 +256,7 @@ fail:
>  if (net->nc->info->type == NET_CLIENT_OPTIONS_KIND_TAP) {
>  while (file.index-- > 0) {
>  const VhostOps *vhost_ops = net->dev.vhost_ops;
> -int r = vhost_ops->vhost_call(>dev, VHOST_NET_SET_BACKEND,
> -  );
> +int r = vhost_ops->vhost_net_set_backend(>dev, );
>  assert(r >= 0);
>  }
>  }
> @@ -280,15 +278,13 @@ static void vhost_net_stop_one(struct vhost_net *net,
>  if (net->nc->info->type == NET_CLIENT_OPTIONS_KIND_TAP) {
>  for (file.index = 0; file.index < net->dev.nvqs; ++file.index) {
>  const VhostOps *vhost_ops = net->dev.vhost_ops;
> -int r = vhost_ops->vhost_call(>dev, VHOST_NET_SET_BACKEND,
> -  );
> +int r = vhost_ops->vhost_net_set_backend(>dev, );
>  assert(r >= 0);
>  }
>  } else if (net->nc->info->type == NET_CLIENT_OPTIONS_KIND_VHOST_USER) {
>  for (file.index = 0; file.index < net->dev.nvqs; ++file.index) {
>  const VhostOps *vhost_ops = net->dev.vhost_ops;
> -int r = vhost_ops->vhost_call(>dev, VHOST_RESET_OWNER,
> -  NULL);
> +int r = vhost_ops->vhost_reset_owner(>dev);
>  assert(r >= 0);
>  }
>  }
> diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c
> index bac9ddb..a0034ab 100644
> --- a/hw/scsi/vhost-scsi.c
> +++ b/hw/scsi/vhost-scsi.c
> @@ -45,7 +45,7 @@ static int vhost_scsi_set_endpoint(VHostSCSI *s)
>
>  memset(, 0, sizeof(backend));
>  pstrcpy(backend.vhost_wwpn, sizeof(backend.vhost_wwpn), vs->conf.wwpn);
> -ret = vhost_ops->vhost_call(>dev, VHOST_SCSI_SET_ENDPOINT, );
> +ret = vhost_ops->vhost_scsi_set_endpoint(>dev, );
>  if (ret < 0) {
>  return -errno;
>  }
> @@ -60,7 +60,7 @@ static void vhost_scsi_clear_endpoint(VHostSCSI *s)
>
>  memset(, 0, sizeof(backend));
>  pstrcpy(backend.vhost_wwpn, sizeof(backend.vhost_wwpn), vs->conf.wwpn);
> -vhost_ops->vhost_call(>dev, VHOST_SCSI_CLEAR_ENDPOINT, );
> +vhost_ops->vhost_scsi_clear_endpoint(>dev, );
>  }
>
>  static int vhost_scsi_start(VHostSCSI *s)
> @@ -76,8 +76,7 @@ static int vhost_scsi_start(VHostSCSI *s)
>  return -ENOSYS;
>  }
>
> -ret = vhost_ops->vhost_call(>dev,
> -VHOST_SCSI_GET_ABI_VERSION, _version);
> +ret = vhost_ops->vhost_scsi_get_abi_version(>dev, _version);
>  if (ret < 0) {
>  return -errno;
>  }
> diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c
> index 4d68a27..bf2d1d4 100644
> --- a/hw/virtio/vhost-backend.c
> +++ b/hw/virtio/vhost-backend.c
> @@ -11,19 +11,10 @@
>  #include "hw/virtio/vhost.h"
>  #include "hw/virtio/vhost-backend.h"
>  #include "qemu/error-report.h"
> +#include "linux/vhost.h"
>
>  #include 
>
> -static int vhost_kernel_call(struct vhost_dev *dev, unsigned long int 
> request,
> - void *arg)
> -{
> -int fd = (uintptr_t) dev->opaque;
> -
> -assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_KERNEL);
> -
> -return ioctl(fd, request, arg);
> -}
> -
>  static int vhost_kernel_init(struct vhost_dev *dev, void *opaque)
>  {
>  assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_KERNEL);
> @@ -42,11 +33,136 @@ static int vhost_kernel_cleanup(struct vhost_dev *dev)
>  return close(fd);
>  }
>
> +static int vhost_kernel_call(struct vhost_dev *dev,
> + unsigned 

Re: [Qemu-devel] [Qemu-stable] Recent patches for 2.4

[Peter Lieven]

Am 07.09.2015 um 21:14 schrieb Paolo Bonzini:


On 07/09/2015 21:11, Peter Lieven wrote:

It helps, but I have a small issue when my backport of the patch
is applied.

I launch qemu witch a cmdline like this to probe for enforcable CPU types.

echo quit | qemu -enable-kvm -monitor stdio -nodefaults -nographic -cpu 
Haswell,enforce -S

Qemu hangs when entering the quit command. If I omit the -S it works. And 
without the
fix it also works with -S.

My backport (Linux only and without tests etc.) of the original fix is here:
https://github.com/plieven/qemu/commit/0ddcdc62a85f705017df16421d769e82b70f9b37

Could you be missing edec47c?


Indeed this one fixed the deadlock with -S in cmdline.
Are there other important fixes from the series?

I currently have these two:
 main-loop: fix qemu_notify_event for aio_notify optimization
 AioContext: fix broken ctx->dispatching optimization

Thanks,
Peter

Re: [Qemu-devel] [PATCH v4] ppc/spapr: Implement H_RANDOM hypercall in QEMU

[Greg Kurz]

On Mon, 21 Sep 2015 12:10:00 +1000
David Gibson  wrote:

> On Fri, Sep 18, 2015 at 11:05:52AM +0200, Greg Kurz wrote:
> > On Thu, 17 Sep 2015 10:49:41 +0200
> > Thomas Huth  wrote:
> > 
> > > The PAPR interface defines a hypercall to pass high-quality
> > > hardware generated random numbers to guests. Recent kernels can
> > > already provide this hypercall to the guest if the right hardware
> > > random number generator is available. But in case the user wants
> > > to use another source like EGD, or QEMU is running with an older
> > > kernel, we should also have this call in QEMU, so that guests that
> > > do not support virtio-rng yet can get good random numbers, too.
> > > 
> > > This patch now adds a new pseudo-device to QEMU that either
> > > directly provides this hypercall to the guest or is able to
> > > enable the in-kernel hypercall if available. The in-kernel
> > > hypercall can be enabled with the use-kvm property, e.g.:
> > > 
> > >  qemu-system-ppc64 -device spapr-rng,use-kvm=true
> > > 
> > > For handling the hypercall in QEMU instead, a "RngBackend" is
> > > required since the hypercall should provide "good" random data
> > > instead of pseudo-random (like from a "simple" library function
> > > like rand() or g_random_int()). Since there are multiple RngBackends
> > > available, the user must select an appropriate back-end via the
> > > "rng" property of the device, e.g.:
> > > 
> > >  qemu-system-ppc64 -object rng-random,filename=/dev/hwrng,id=gid0 \
> > >-device spapr-rng,rng=gid0 ...
> > > 
> > > See http://wiki.qemu-project.org/Features-Done/VirtIORNG for
> > > other example of specifying RngBackends.
> > > 
> > > Signed-off-by: Thomas Huth 
> > > ---
> > 
> > It is a good thing that the user can choose between in-kernel and backend,
> > and this patch does the work.
> > 
> > This being said, I am not sure about the use case where a user has a hwrng
> > capable platform and wants to run guests without any hwrng support at all is
> > an appropriate default behavior... I guess we will find more users that want
> > in-kernel being the default if it is available.
> > 
> > The patch below modifies yours to do just this: the pseudo-device is only
> > created if hwrng is present and not already created.
> 
> I have mixed feelings about this.  On the one hand, I agree that it
> would be nice to allow H_RANDOM support by default.  On the other hand
> the patch below leaves no way to turn it off for testing purposes.  It

This could be handled with a new spapr property, say 'use-hwrng', defaulting
to true.

> also adds another place where the guest hardware depends on the host
> configuration, which adds to the already substantial mess of ensuring
> that source and destination hardware configuration matches for
> migration.
> 

Yeah, describing the guest hw is really essential for migration... this
is best addressed at the libvirt level with a full XML description of
the machine... but FWIW if we are talking about running pseries on a
POWER8 or newer host, I am not aware about "hwrng-less" boards... but
I am probably missing something :)

Back to Thomas' patch, it does the job and brings H_RANDOM, which is
currently missing.

Acked-by: Greg Kurz 

I could test both use-kvm and backend flavors (including migration).

Tested-by: Greg Kurz 

Thanks.

--
Greg


pgp6suFnxthfL.pgp
Description: OpenPGP digital signature

[Qemu-devel] [PULL 11/26] qapi-event: Convert to QAPISchemaVisitor, fixing data with base

[Markus Armbruster]

Fixes events whose data is struct with base to include the struct's
base members.  Test case is qapi-schema-test.json's event
__org.qemu_x-command:

{ 'event': '__ORG.QEMU_X-EVENT', 'data': '__org.qemu_x-Struct' }

{ 'struct': '__org.qemu_x-Struct', 'base': '__org.qemu_x-Base',
  'data': { '__org.qemu_x-member2': 'str' } }

{ 'struct': '__org.qemu_x-Base',
  'data': { '__org.qemu_x-member1': '__org.qemu_x-Enum' } }

Patch's effect on generated qapi_event_send___org_qemu_x_event():

-void qapi_event_send___org_qemu_x_event(const char *__org_qemu_x_member2,
+void qapi_event_send___org_qemu_x_event(__org_qemu_x_Enum 
__org_qemu_x_member1,
+const char *__org_qemu_x_member2,
 Error **errp)
 {
 QDict *qmp;
@@ -224,6 +225,10 @@ void qapi_event_send___org_qemu_x_event(
 goto clean;
 }

+visit_type___org_qemu_x_Enum(v, &__org_qemu_x_member1, 
"__org.qemu_x-member1", _err);
+if (local_err) {
+goto clean;
+}
 visit_type_str(v, (char **)&__org_qemu_x_member2, 
"__org.qemu_x-member2", _err);
 if (local_err) {
 goto clean;

Code is generated in a different order now, but that doesn't matter.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
---
 scripts/qapi-event.py   | 89 +
 tests/qapi-schema/qapi-schema-test.json |  3 --
 2 files changed, 47 insertions(+), 45 deletions(-)

diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 537da17..d19acda 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -2,14 +2,15 @@
 # QAPI event generator
 #
 # Copyright (c) 2014 Wenchao Xia
+# Copyright (c) 2015 Red Hat Inc.
 #
 # Authors:
 #  Wenchao Xia 
+#  Markus Armbruster 
 #
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-from ordereddict import OrderedDict
 from qapi import *
 
 def _generate_event_api_name(event_name, params):
@@ -17,13 +18,13 @@ def _generate_event_api_name(event_name, params):
 l = len(api_name)
 
 if params:
-for argname, argentry, optional in parse_args(params):
-if optional:
-api_name += "bool has_%s,\n" % c_name(argname)
+for m in params.members:
+if m.optional:
+api_name += "bool has_%s,\n" % c_name(m.name)
 api_name += "".ljust(l)
 
-api_name += "%s %s,\n" % (c_type(argentry, is_param=True),
-  c_name(argname))
+api_name += "%s %s,\n" % (m.type.c_type(is_param=True),
+  c_name(m.name))
 api_name += "".ljust(l)
 
 api_name += "Error **errp)"
@@ -51,7 +52,7 @@ def generate_event_implement(api_name, event_name, params):
 """,
 api_name = api_name)
 
-if params:
+if params and params.members:
 ret += mcgen("""
 QmpOutputVisitor *qov;
 Visitor *v;
@@ -72,7 +73,7 @@ def generate_event_implement(api_name, event_name, params):
  event_name = event_name)
 
 # step 3: visit the params if params != None
-if params:
+if params and params.members:
 ret += mcgen("""
 qov = qmp_output_visitor_new();
 g_assert(qov);
@@ -89,15 +90,15 @@ def generate_event_implement(api_name, event_name, params):
 """,
 event_name = event_name)
 
-for argname, argentry, optional in parse_args(params):
-if optional:
+for memb in params.members:
+if memb.optional:
 ret += mcgen("""
 if (has_%(var)s) {
 """,
- var = c_name(argname))
+ var=c_name(memb.name))
 push_indent()
 
-if argentry == "str":
+if memb.type.name == "str":
 var_type = "(char **)"
 else:
 var_type = ""
@@ -109,11 +110,11 @@ def generate_event_implement(api_name, event_name, 
params):
 }
 """,
  var_type = var_type,
- var = c_name(argname),
- type = type_name(argentry),
- name = argname)
+ var=c_name(memb.name),
+ type=memb.type.c_name(),
+ name=memb.name)
 
-if optional:
+if memb.optional:
 pop_indent()
 ret += mcgen("""
 }
@@ -140,7 +141,7 @@ def generate_event_implement(api_name, event_name, params):
  event_enum_value = c_enum_const(event_enum_name, event_name))
 
 # step 5: clean up
-if params:
+if params and 

[Qemu-devel] [PULL 26/26] qapi-introspect: Hide type names

[Markus Armbruster]

To eliminate the temptation for clients to look up types by name
(which are not ABI), replace all type names by meaningless strings.

Reduces output of query-schema by 13 out of 85KiB.

As a debugging aid, provide option -u to suppress the hiding.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Message-Id: <1442401589-24189-27-git-send-email-arm...@redhat.com>
---
 docs/qapi-code-gen.txt | 36 +---
 qapi/introspect.json   | 12 
 scripts/qapi-introspect.py | 41 +++--
 3 files changed, 56 insertions(+), 33 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index f321d4b..b1c8361 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -530,13 +530,16 @@ additional variant members depending on the value of 
meta-type.
 Each SchemaInfo object describes a wire ABI entity of a certain
 meta-type: a command, event or one of several kinds of type.
 
-SchemaInfo for entities defined in the QAPI schema have the same name
-as in the schema.  This is the case for all commands and events, and
-most types.
+SchemaInfo for commands and events have the same name as in the QAPI
+schema.
 
 Command and event names are part of the wire ABI, but type names are
-not.  Therefore, looking up a type by its name in the QAPI schema is
-wrong.  Look up the command or event, then follow references by name.
+not.  Therefore, the SchemaInfo for types have auto-generated
+meaningless names.  For readability, the examples in this section use
+meaningful type names instead.
+
+To examine a type, start with a command or event using it, then follow
+references by name.
 
 QAPI schema definitions not reachable that way are omitted.
 
@@ -567,8 +570,7 @@ object type without members.  The event may not have a data 
member on
 the wire then.
 
 Each command or event defined with dictionary-valued 'data' in the
-QAPI schema implicitly defines an object type called ":obj-NAME-arg",
-where NAME is the command or event's name.
+QAPI schema implicitly defines an object type.
 
 Example: the SchemaInfo for EVENT_C from section Events
 
@@ -623,12 +625,9 @@ Note that base types are "flattened": its members are 
included in the
 
 A simple union implicitly defines an enumeration type for its implicit
 discriminator (called "type" on the wire, see section Union types).
-Such a type's name is made by appending "Kind" to the simple union's
-name.
 
 A simple union implicitly defines an object type for each of its
-variants.  The type's name is ":obj-NAME-wrapper", where NAME is the
-name of the name of the variant's type.
+variants.
 
 Example: the SchemaInfo for simple union BlockdevOptions from section
 Union types
@@ -659,8 +658,7 @@ Example: the SchemaInfo for BlockRef from section Alternate 
types
 
 The SchemaInfo for an array type has meta-type "array", and variant
 member "element-type", which names the array's element type.  Array
-types are implicitly defined.  An array type's name is made by
-appending "List" to its element type's name.
+types are implicitly defined.
 
 Example: the SchemaInfo for ['str']
 
@@ -1067,13 +1065,13 @@ Example:
 [Uninteresting stuff omitted...]
 
 const char example_qmp_schema_json[] = "["
-"{\"arg-type\": \":empty\", \"meta-type\": \"event\", \"name\": 
\"MY_EVENT\"}, "
+"{\"arg-type\": \"0\", \"meta-type\": \"event\", \"name\": 
\"MY_EVENT\"}, "
+"{\"arg-type\": \"1\", \"meta-type\": \"command\", \"name\": 
\"my-command\", \"ret-type\": \"2\"}, "
+"{\"members\": [], \"meta-type\": \"object\", \"name\": \"0\"}, "
+"{\"members\": [{\"name\": \"arg1\", \"type\": \"2\"}], \"meta-type\": 
\"object\", \"name\": \"1\"}, "
+"{\"members\": [{\"name\": \"integer\", \"type\": \"int\"}, {\"name\": 
\"string\", \"type\": \"str\"}], \"meta-type\": \"object\", \"name\": \"2\"}, "
 "{\"json-type\": \"int\", \"meta-type\": \"builtin\", \"name\": 
\"int\"}, "
-"{\"json-type\": \"string\", \"meta-type\": \"builtin\", \"name\": 
\"str\"}, "
-"{\"members\": [], \"meta-type\": \"object\", \"name\": \":empty\"}, "
-"{\"members\": [{\"name\": \"arg1\", \"type\": \"UserDefOne\"}], 
\"meta-type\": \"object\", \"name\": \":obj-my-command-arg\"}, "
-"{\"members\": [{\"name\": \"integer\", \"type\": \"int\"}, {\"name\": 
\"string\", \"type\": \"str\"}], \"meta-type\": \"object\", \"name\": 
\"UserDefOne\"}, "
-"{\"arg-type\": \":obj-my-command-arg\", \"meta-type\": \"command\", 
\"name\": \"my-command\", \"ret-type\": \"UserDefOne\"}]";
+"{\"json-type\": \"string\", \"meta-type\": \"builtin\", \"name\": 
\"str\"}]";
 $ cat qapi-generated/example-qmp-introspect.h
 [Uninteresting stuff omitted...]
 
diff --git a/qapi/introspect.json b/qapi/introspect.json
index 9c8ad53..cc50dc6 100644
--- a/qapi/introspect.json
+++ b/qapi/introspect.json
@@ -75,17 +75,13 @@
 # 

[Qemu-devel] [PULL 14/26] qapi-visit: Rearrange code a bit

[Markus Armbruster]

Move gen_visit_decl() to a better place.  Inline
generate_visit_struct_body().

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-15-git-send-email-arm...@redhat.com>
---
 scripts/qapi-visit.py | 50 --
 1 file changed, 20 insertions(+), 30 deletions(-)

diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index fae70e2..97343cf 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -19,6 +19,16 @@ implicit_structs_seen = set()
 struct_fields_seen = set()
 
 
+def gen_visit_decl(name, scalar=False):
+c_type = c_name(name) + ' *'
+if not scalar:
+c_type += '*'
+return mcgen('''
+void visit_type_%(c_name)s(Visitor *m, %(c_type)sobj, const char *name, Error 
**errp);
+''',
+ c_name=c_name(name), c_type=c_type)
+
+
 def gen_visit_implicit_struct(typ):
 if typ in implicit_structs_seen:
 return ''
@@ -117,12 +127,17 @@ out:
 return ret
 
 
-def gen_visit_struct_body(name):
+def gen_visit_struct(name, base, members):
+ret = gen_visit_struct_fields(name, base, members)
+
 # FIXME: if *obj is NULL on entry, and visit_start_struct() assigns to
 # *obj, but then visit_type_FOO_fields() fails, we should clean up *obj
 # rather than leaving it non-NULL. As currently written, the caller must
 # call qapi_free_FOO() to avoid a memory leak of the partial FOO.
-ret = mcgen('''
+ret += mcgen('''
+
+void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, 
Error **errp)
+{
 Error *err = NULL;
 
 visit_start_struct(m, (void **)obj, "%(name)s", name, sizeof(%(c_name)s), 
);
@@ -133,25 +148,10 @@ def gen_visit_struct_body(name):
 visit_end_struct(m, );
 }
 error_propagate(errp, err);
-''',
-name=name, c_name=c_name(name))
-return ret
-
-
-def gen_visit_struct(name, base, members):
-ret = gen_visit_struct_fields(name, base, members)
-ret += mcgen('''
-
-void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, 
Error **errp)
-{
-''',
- c_name=c_name(name))
-
-ret += gen_visit_struct_body(name)
-
-ret += mcgen('''
 }
-''')
+''',
+ name=name, c_name=c_name(name))
+
 return ret
 
 
@@ -342,16 +342,6 @@ out:
 return ret
 
 
-def gen_visit_decl(name, scalar=False):
-c_type = c_name(name) + ' *'
-if not scalar:
-c_type += '*'
-return mcgen('''
-void visit_type_%(c_name)s(Visitor *m, %(c_type)sobj, const char *name, Error 
**errp);
-''',
- c_name=c_name(name), c_type=c_type)
-
-
 class QAPISchemaGenVisitVisitor(QAPISchemaVisitor):
 def __init__(self):
 self.decl = None
-- 
2.4.3

[Qemu-devel] [PULL 10/26] qapi-event: Eliminate global variable event_enum_value

[Markus Armbruster]

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-11-git-send-email-arm...@redhat.com>
---
 scripts/qapi-event.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index aed45d6..537da17 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -137,7 +137,7 @@ def generate_event_implement(api_name, event_name, params):
 emit(%(event_enum_value)s, qmp, _err);
 
 """,
- event_enum_value = event_enum_value)
+ event_enum_value = c_enum_const(event_enum_name, event_name))
 
 # step 5: clean up
 if params:
@@ -223,7 +223,6 @@ for expr in exprs:
 fdecl.write(ret)
 
 # We need an enum value per event
-event_enum_value = c_enum_const(event_enum_name, event_name)
 ret = generate_event_implement(api_name, event_name, params)
 fdef.write(ret)
 
-- 
2.4.3

[Qemu-devel] [PULL 17/26] qapi: De-duplicate parameter list generation

[Markus Armbruster]

Generated qapi-event.[ch] lose line breaks.  No change otherwise.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-18-git-send-email-arm...@redhat.com>
---
 scripts/qapi-commands.py | 11 ++-
 scripts/qapi-event.py| 18 +++---
 scripts/qapi.py  | 16 
 3 files changed, 21 insertions(+), 24 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index c68659a..833768e 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -17,19 +17,12 @@ import re
 
 
 def gen_command_decl(name, arg_type, ret_type):
-argstr = ''
-if arg_type:
-for memb in arg_type.members:
-if memb.optional:
-argstr += 'bool has_%s, ' % c_name(memb.name)
-argstr += '%s %s, ' % (memb.type.c_type(is_param=True),
-   c_name(memb.name))
 return mcgen('''
-%(c_type)s qmp_%(c_name)s(%(args)sError **errp);
+%(c_type)s qmp_%(c_name)s(%(params)s);
 ''',
  c_type=(ret_type and ret_type.c_type()) or 'void',
  c_name=c_name(name),
- args=argstr)
+ params=gen_params(arg_type, 'Error **errp'))
 
 
 def gen_err_check(err):
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 5873a05..d15fad9 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -15,21 +15,9 @@ from qapi import *
 
 
 def gen_event_send_proto(name, arg_type):
-api_name = "void qapi_event_send_%s(" % c_name(name).lower()
-l = len(api_name)
-
-if arg_type:
-for m in arg_type.members:
-if m.optional:
-api_name += "bool has_%s,\n" % c_name(m.name)
-api_name += "".ljust(l)
-
-api_name += "%s %s,\n" % (m.type.c_type(is_param=True),
-  c_name(m.name))
-api_name += "".ljust(l)
-
-api_name += "Error **errp)"
-return api_name
+return 'void qapi_event_send_%(c_name)s(%(param)s)' % {
+'c_name': c_name(name.lower()),
+'param': gen_params(arg_type, 'Error **errp')}
 
 
 def gen_event_send_decl(name, arg_type):
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 7ac72f6..6b6f1ae 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1469,6 +1469,22 @@ extern const char *const %(c_name)s_lookup[];
  c_name=c_name(name))
 return ret
 
+def gen_params(arg_type, extra):
+if not arg_type:
+return extra
+assert not arg_type.variants
+ret = ''
+sep = ''
+for memb in arg_type.members:
+ret += sep
+sep = ', '
+if memb.optional:
+ret += 'bool has_%s, ' % c_name(memb.name)
+ret += '%s %s' % (memb.type.c_type(is_param=True), c_name(memb.name))
+if extra:
+ret += sep + extra
+return ret
+
 #
 # Common command line parsing
 #
-- 
2.4.3

[Qemu-devel] [PULL 19/26] qapi: Improve built-in type documentation

[Markus Armbruster]

Clarify how they map to JSON.  Add how they map to C.  Fix the
reference to StringInputVisitor.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
Message-Id: <1442401589-24189-20-git-send-email-arm...@redhat.com>
---
 docs/qapi-code-gen.txt | 29 ++---
 1 file changed, 18 insertions(+), 11 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 0884b58..5c2c278 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -140,17 +140,24 @@ must have a value that forms a struct name.
 
 === Built-in Types ===
 
-The following types are built-in to the parser:
-  'str' - arbitrary UTF-8 string
-  'int' - 64-bit signed integer (although the C code may place further
-  restrictions on acceptable range)
-  'number' - floating point number
-  'bool' - JSON value of true or false
-  'int8', 'int16', 'int32', 'int64' - like 'int', but enforce maximum
-  bit size
-  'uint8', 'uint16', 'uint32', 'uint64' - unsigned counterparts
-  'size' - like 'uint64', but allows scaled suffix from command line
-   visitor
+The following types are predefined, and map to C as follows:
+
+  SchemaC  JSON
+  str   char * any JSON string, UTF-8
+  numberdouble any JSON number
+  int   int64_ta JSON number without fractional part
+   that fits into the C integer type
+  int8  int8_t likewise
+  int16 int16_tlikewise
+  int32 int32_tlikewise
+  int64 int64_tlikewise
+  uint8 uint8_tlikewise
+  uint16uint16_t   likewise
+  uint32uint32_t   likewise
+  uint64uint64_t   likewise
+  size  uint64_t   like uint64_t, except StringInputVisitor
+   accepts size suffixes
+  bool  bool   JSON true or false
 
 
 === Includes ===
-- 
2.4.3

[Qemu-devel] [PULL 21/26] qapi: Introduce a first class 'any' type

[Markus Armbruster]

It's first class, because unlike '**', it actually works, i.e. doesn't
require 'gen': false.

'**' will go away next.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
---
 docs/qapi-code-gen.txt |  1 +
 include/qapi/visitor-impl.h|  2 +
 include/qapi/visitor.h |  1 +
 qapi/qapi-dealloc-visitor.c|  9 
 qapi/qapi-visit-core.c |  6 +++
 qapi/qmp-input-visitor.c   | 11 +
 qapi/qmp-output-visitor.c  |  9 
 scripts/qapi-types.py  |  1 +
 scripts/qapi.py|  9 ++--
 tests/Makefile |  3 +-
 tests/qapi-schema/args-any.err |  1 +
 .../{flat-union-base-star.exit => args-any.exit}   |  0
 tests/qapi-schema/args-any.json|  2 +
 .../{flat-union-base-star.out => args-any.out} |  0
 tests/qapi-schema/flat-union-base-any.err  |  1 +
 tests/qapi-schema/flat-union-base-any.exit |  1 +
 ...ion-base-star.json => flat-union-base-any.json} |  2 +-
 tests/qapi-schema/flat-union-base-any.out  |  0
 tests/qapi-schema/flat-union-base-star.err |  1 -
 tests/qapi-schema/qapi-schema-test.json|  5 +-
 tests/qapi-schema/qapi-schema-test.out |  9 +++-
 tests/qapi-schema/type-bypass.out  |  4 +-
 tests/test-qmp-commands.c  |  5 ++
 tests/test-qmp-input-visitor.c | 45 ++
 tests/test-qmp-output-visitor.c| 53 ++
 25 files changed, 171 insertions(+), 10 deletions(-)
 create mode 100644 tests/qapi-schema/args-any.err
 rename tests/qapi-schema/{flat-union-base-star.exit => args-any.exit} (100%)
 create mode 100644 tests/qapi-schema/args-any.json
 rename tests/qapi-schema/{flat-union-base-star.out => args-any.out} (100%)
 create mode 100644 tests/qapi-schema/flat-union-base-any.err
 create mode 100644 tests/qapi-schema/flat-union-base-any.exit
 rename tests/qapi-schema/{flat-union-base-star.json => 
flat-union-base-any.json} (95%)
 create mode 100644 tests/qapi-schema/flat-union-base-any.out
 delete mode 100644 tests/qapi-schema/flat-union-base-star.err

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 5c2c278..70fdae7 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -158,6 +158,7 @@ The following types are predefined, and map to C as follows:
   size  uint64_t   like uint64_t, except StringInputVisitor
accepts size suffixes
   bool  bool   JSON true or false
+  any   QObject *  any JSON value
 
 
 === Includes ===
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index f4a2f74..8c0ba57 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -40,6 +40,8 @@ struct Visitor
 void (*type_str)(Visitor *v, char **obj, const char *name, Error **errp);
 void (*type_number)(Visitor *v, double *obj, const char *name,
 Error **errp);
+void (*type_any)(Visitor *v, QObject **obj, const char *name,
+ Error **errp);
 
 /* May be NULL */
 void (*optional)(Visitor *v, bool *present, const char *name,
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 00ba104..cfc19a6 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -58,6 +58,7 @@ void visit_type_size(Visitor *v, uint64_t *obj, const char 
*name, Error **errp);
 void visit_type_bool(Visitor *v, bool *obj, const char *name, Error **errp);
 void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp);
 void visit_type_number(Visitor *v, double *obj, const char *name, Error 
**errp);
+void visit_type_any(Visitor *v, QObject **obj, const char *name, Error **errp);
 bool visit_start_union(Visitor *v, bool data_present, Error **errp);
 void visit_end_union(Visitor *v, bool data_present, Error **errp);
 
diff --git a/qapi/qapi-dealloc-visitor.c b/qapi/qapi-dealloc-visitor.c
index d7f92c5..737deab 100644
--- a/qapi/qapi-dealloc-visitor.c
+++ b/qapi/qapi-dealloc-visitor.c
@@ -151,6 +151,14 @@ static void qapi_dealloc_type_number(Visitor *v, double 
*obj, const char *name,
 {
 }
 
+static void qapi_dealloc_type_anything(Visitor *v, QObject **obj,
+   const char *name, Error **errp)
+{
+if (obj) {
+qobject_decref(*obj);
+}
+}
+
 static void qapi_dealloc_type_size(Visitor *v, uint64_t *obj, const char *name,
Error **errp)
 {
@@ -216,6 +224,7 @@ QapiDeallocVisitor *qapi_dealloc_visitor_new(void)
 v->visitor.type_bool = qapi_dealloc_type_bool;
 v->visitor.type_str = qapi_dealloc_type_str;
 

Re: [Qemu-devel] [PATCH 2/6] hw/s390x/css: Use type_init() instead of machine_init()

[Christian Borntraeger]

Am 19.09.2015 um 17:36 schrieb Andreas Färber:
> Here, machine_init() is being misused to initialize global variables.
> Being all local, it's safe to use the slightly earlier type_init().
> 
> This prepares for dropping machine_init().
> 
> Signed-off-by: Andreas Färber 

Acked-by: Christian Borntraeger 

I think this should go via your tree to coordinate with machine_init removal.



> ---
>  hw/s390x/css.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/s390x/css.c b/hw/s390x/css.c
> index c033612..d164fa8 100644
> --- a/hw/s390x/css.c
> +++ b/hw/s390x/css.c
> @@ -1509,7 +1509,7 @@ static void css_init(void)
>  channel_subsys->chnmon_active = false;
>  QTAILQ_INIT(_subsys->io_adapters);
>  }
> -machine_init(css_init);
> +type_init(css_init)
> 
>  void css_reset_sch(SubchDev *sch)
>  {
> 

Re: [Qemu-devel] [PATCH] target-tilegx: Implement v*add and v*sub instructions

[Chen Gang]

On 2015年09月19日 10:34, Richard Henderson wrote:
> 
> There's a trick for this that's more efficient for 4 or more elements
> per vector (i.e. good for v2 and v1, but not v4):
> 
>a + b = (a & 0x7f7f7f7f) + (b & 0x7f7f7f7f)) ^ ((a ^ b) & 0x80808080)
> 
>a - b = (a | 0x80808080) - (b & 0x7f7f7f7f)) ^ ((a ^ ~b) & 0x80808080)
> 

For me, we need use "(a ^ b) & 0x80..." instead of "(a ^ ~b) & 0x80...".


Thanks.
-- 
Chen Gang

Open, share, and attitude like air, water, and life which God blessed

Re: [Qemu-devel] [PATCH v6 06/14] blockjob: Add .commit and .abort block job actions

[John Snow]

On 09/15/2015 02:11 AM, Fam Zheng wrote:
> Reviewed-by: Max Reitz 
> Signed-off-by: Fam Zheng 
> ---
>  include/block/blockjob.h | 18 ++
>  1 file changed, 18 insertions(+)
> 
> diff --git a/include/block/blockjob.h b/include/block/blockjob.h
> index 3e7ad21..a7b497c 100644
> --- a/include/block/blockjob.h
> +++ b/include/block/blockjob.h
> @@ -50,6 +50,24 @@ typedef struct BlockJobDriver {
>   * manually.
>   */
>  void (*complete)(BlockJob *job, Error **errp);
> +
> +/**
> + * If the callback is not NULL, it will be invoked when all the jobs
> + * belonging to the same transaction complete; or upon this job's
> + * completion if it is not in a transaction. Skipped if NULL.
> + *
> + * Exactly one of .commit() and .abort() will be called for each job.
> + */
> +void (*commit)(BlockJob *job);
> +

I find this phrasing strange, but maybe it's just me. "Exactly one of
commit and abort will be called for each job" implies [to me] that it'd
be possible to call commit for one, but abort for different jobs [in a
transaction] -- but clearly we don't mean that. It is the "for each job"
that implies an iteration over a collection to me.

Just above we say "[commit] will be invoked when all the jobs belonging
to the same transaction are complete" which itself implies either all
jobs will be committed or all jobs will be aborted.

Maybe:

"All jobs will complete with a call to either .commit() or .abort() but
never both."

But I might be being too bikesheddy.

> +/**
> + * If the callback is not NULL, it will be invoked when any job in the
> + * same transaction fails; or upon this job's failure (due to error or
> + * cancellation) if it is not in a transaction. Skipped if NULL.
> + *
> + * Exactly one of .commit() and .abort() will be called for each job.
> + */
> +void (*abort)(BlockJob *job);
>  } BlockJobDriver;
>  
>  /**
> 

I'm probably just too picky.

Reviewed-by: John Snow 

[Qemu-devel] [PATCH v2] target-tilegx: Implement v1multu instruction

[gang . chen . 5i5j]

From: Chen Gang 

Only according to v1shrs implementation.

Signed-off-by: Chen Gang 
Reviewed-by: Richard Henderson 
---
 target-tilegx/helper.h  |  2 ++
 target-tilegx/simd_helper.c | 13 +
 target-tilegx/translate.c   |  4 
 3 files changed, 19 insertions(+)

diff --git a/target-tilegx/helper.h b/target-tilegx/helper.h
index 6d98f3a..90e558d 100644
--- a/target-tilegx/helper.h
+++ b/target-tilegx/helper.h
@@ -16,3 +16,5 @@ DEF_HELPER_FLAGS_2(v1add, TCG_CALL_NO_RWG_SE, i64, i64, i64)
 DEF_HELPER_FLAGS_2(v1sub, TCG_CALL_NO_RWG_SE, i64, i64, i64)
 DEF_HELPER_FLAGS_2(v2add, TCG_CALL_NO_RWG_SE, i64, i64, i64)
 DEF_HELPER_FLAGS_2(v2sub, TCG_CALL_NO_RWG_SE, i64, i64, i64)
+
+DEF_HELPER_FLAGS_2(v1multu, TCG_CALL_NO_RWG_SE, i64, i64, i64)
diff --git a/target-tilegx/simd_helper.c b/target-tilegx/simd_helper.c
index 00265fe..b98573e 100644
--- a/target-tilegx/simd_helper.c
+++ b/target-tilegx/simd_helper.c
@@ -107,3 +107,16 @@ uint64_t helper_v2shrs(uint64_t a, uint64_t b)
 }
 return r;
 }
+
+uint64_t helper_v1multu(uint64_t a, uint64_t b)
+{
+uint64_t r = 0;
+int i;
+
+for (i = 0; i < 64; i += 8) {
+uint64_t ae = (uint8_t)(a >> i);
+uint64_t be = (uint8_t)(b >> i);
+r |= ((ae * be) & 0xff) << i;
+}
+return r;
+}
diff --git a/target-tilegx/translate.c b/target-tilegx/translate.c
index 297de5c..a9fc4ce 100644
--- a/target-tilegx/translate.c
+++ b/target-tilegx/translate.c
@@ -1113,7 +1113,11 @@ static TileExcp gen_rrr_opcode(DisasContext *dc, 
unsigned opext,
 case OE_RRR(V1MINU, 0, X1):
 case OE_RRR(V1MNZ, 0, X0):
 case OE_RRR(V1MNZ, 0, X1):
+return TILEGX_EXCP_OPCODE_UNIMPLEMENTED;
 case OE_RRR(V1MULTU, 0, X0):
+gen_helper_v1multu(tdest, tsrca, tsrcb);
+mnemonic = "v1multu";
+break;
 case OE_RRR(V1MULUS, 0, X0):
 case OE_RRR(V1MULU, 0, X0):
 case OE_RRR(V1MZ, 0, X0):
-- 
1.9.3

Re: [Qemu-devel] [PATCH v2] target-tilegx: Implement v1multu instruction

[Chen Gang]

The related code are changed, so this patch has to be reconstructed.

Thanks

On 9/22/15 06:26, gang.chen.5...@gmail.com wrote:
> From: Chen Gang 
>
> Only according to v1shrs implementation.
>
> Signed-off-by: Chen Gang 
> Reviewed-by: Richard Henderson 
> ---
> target-tilegx/helper.h | 2 ++
> target-tilegx/simd_helper.c | 13 +
> target-tilegx/translate.c | 4 
> 3 files changed, 19 insertions(+)
>
> diff --git a/target-tilegx/helper.h b/target-tilegx/helper.h
> index 6d98f3a..90e558d 100644
> --- a/target-tilegx/helper.h
> +++ b/target-tilegx/helper.h
> @@ -16,3 +16,5 @@ DEF_HELPER_FLAGS_2(v1add, TCG_CALL_NO_RWG_SE, i64, i64, i64)
> DEF_HELPER_FLAGS_2(v1sub, TCG_CALL_NO_RWG_SE, i64, i64, i64)
> DEF_HELPER_FLAGS_2(v2add, TCG_CALL_NO_RWG_SE, i64, i64, i64)
> DEF_HELPER_FLAGS_2(v2sub, TCG_CALL_NO_RWG_SE, i64, i64, i64)
> +
> +DEF_HELPER_FLAGS_2(v1multu, TCG_CALL_NO_RWG_SE, i64, i64, i64)
> diff --git a/target-tilegx/simd_helper.c b/target-tilegx/simd_helper.c
> index 00265fe..b98573e 100644
> --- a/target-tilegx/simd_helper.c
> +++ b/target-tilegx/simd_helper.c
> @@ -107,3 +107,16 @@ uint64_t helper_v2shrs(uint64_t a, uint64_t b)
> }
> return r;
> }
> +
> +uint64_t helper_v1multu(uint64_t a, uint64_t b)
> +{
> + uint64_t r = 0;
> + int i;
> +
> + for (i = 0; i < 64; i += 8) {
> + uint64_t ae = (uint8_t)(a>> i);
> + uint64_t be = (uint8_t)(b>> i);
> + r |= ((ae * be) & 0xff) << i;
> + }
> + return r;
> +}
> diff --git a/target-tilegx/translate.c b/target-tilegx/translate.c
> index 297de5c..a9fc4ce 100644
> --- a/target-tilegx/translate.c
> +++ b/target-tilegx/translate.c
> @@ -1113,7 +1113,11 @@ static TileExcp gen_rrr_opcode(DisasContext *dc, 
> unsigned opext,
> case OE_RRR(V1MINU, 0, X1):
> case OE_RRR(V1MNZ, 0, X0):
> case OE_RRR(V1MNZ, 0, X1):
> + return TILEGX_EXCP_OPCODE_UNIMPLEMENTED;
> case OE_RRR(V1MULTU, 0, X0):
> + gen_helper_v1multu(tdest, tsrca, tsrcb);
> + mnemonic = "v1multu";
> + break;
> case OE_RRR(V1MULUS, 0, X0):
> case OE_RRR(V1MULU, 0, X0):
> case OE_RRR(V1MZ, 0, X0):
>

--
Chen Gang (陈刚)

Open, share, and attitude like air, water, and life which God blessed
  

Re: [Qemu-devel] [Bug 893208] Re: qemu on ARM hosts can't boot i386 image

[Marina Kovalevna]

Thanks for looking into it, Laszlo. I've already tried dosbox and had
no idea qemu was impractical.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/893208

Title:
  qemu on ARM hosts can't boot i386 image

Status in QEMU:
  New
Status in Linaro QEMU:
  New

Bug description:
  If you apply some workarounds for bug 870990, bug 883133 and bug
  883136 QEMU still cannot boot the i386
  debian_squeeze_i386_standard.qcow2 image from
  http://people.debian.org/~aurel32/qemu/i386/ -- grub starts to boot
  but something causes the system to reset just before display of the
  blue-background grub menu, so we go round in a loop forever. This
  image boots OK on i386 hosted qemu so this indicates some kind of ARM-
  host specific bug.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/893208/+subscriptions

Re: [Qemu-devel] [PATCH v6 09/14] block: Add block job transactions

[Fam Zheng]

On Mon, 09/21 19:23, John Snow wrote:
> >  void block_job_completed(BlockJob *job, int ret)
> >  {
> >  BlockDriverState *bs = job->bs;
> > @@ -98,8 +191,13 @@ void block_job_completed(BlockJob *job, int ret)
> >  assert(!job->completed);
> >  job->completed = true;
> >  job->ret = ret;
> > -job->cb(job->opaque, ret);
> > -block_job_unref(job);
> > +if (!job->txn) {
> > +block_job_completed_single(job);
> > +} else if (ret < 0 || block_job_is_cancelled(job)) {
> > +block_job_completed_txn_abort(job);
> > +} else {
> 
> over here it just appears we assert that ret is simply greater than or
> equal to zero.

Yeah, I don't think we have any possible positive ret value.

Fam

[Qemu-devel] [PATCH v7 06/14] blockjob: Add .commit and .abort block job actions

[Fam Zheng]

Reviewed-by: Max Reitz 
Reviewed-by: John Snow 
Signed-off-by: Fam Zheng 
---
 include/block/blockjob.h | 20 
 1 file changed, 20 insertions(+)

diff --git a/include/block/blockjob.h b/include/block/blockjob.h
index 3e7ad21..aee39a4 100644
--- a/include/block/blockjob.h
+++ b/include/block/blockjob.h
@@ -50,6 +50,26 @@ typedef struct BlockJobDriver {
  * manually.
  */
 void (*complete)(BlockJob *job, Error **errp);
+
+/**
+ * If the callback is not NULL, it will be invoked when all the jobs
+ * belonging to the same transaction complete; or upon this job's
+ * completion if it is not in a transaction. Skipped if NULL.
+ *
+ * All jobs will complete with a call to either .commit() or .abort() but
+ * never both.
+ */
+void (*commit)(BlockJob *job);
+
+/**
+ * If the callback is not NULL, it will be invoked when any job in the
+ * same transaction fails; or upon this job's failure (due to error or
+ * cancellation) if it is not in a transaction. Skipped if NULL.
+ *
+ * All jobs will complete with a call to either .commit() or .abort() but
+ * never both.
+ */
+void (*abort)(BlockJob *job);
 } BlockJobDriver;
 
 /**
-- 
2.4.3

[Qemu-devel] [RFC PATCH v0 1/2] spapr: Allocate HTAB from machine init

[Bharata B Rao]

Allocate HTAB from ppc_spapr_init() so that we can abort the guest
if requested HTAB size is't allocated by the host. However retain the
htab reset call in spapr_reset_htab() so that HTAB gets reset (and
not allocated) during machine reset.

Signed-off-by: Bharata B Rao 
---
 hw/ppc/spapr.c | 13 -
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 7f4f196..4692122 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -979,7 +979,7 @@ static void emulate_spapr_hypercall(PowerPCCPU *cpu)
 #define CLEAN_HPTE(_hpte)  ((*(uint64_t *)(_hpte)) &= 
tswap64(~HPTE64_V_HPTE_DIRTY))
 #define DIRTY_HPTE(_hpte)  ((*(uint64_t *)(_hpte)) |= 
tswap64(HPTE64_V_HPTE_DIRTY))
 
-static void spapr_reset_htab(sPAPRMachineState *spapr)
+static void spapr_alloc_htab(sPAPRMachineState *spapr)
 {
 long shift;
 int index;
@@ -1012,6 +1012,16 @@ static void spapr_reset_htab(sPAPRMachineState *spapr)
 DIRTY_HPTE(HPTE(spapr->htab, index));
 }
 }
+}
+
+static void spapr_reset_htab(sPAPRMachineState *spapr)
+{
+/*
+ * We have already allocated the hash page table, this call will
+ * not again allocate but only result in clearing of hash page
+ * table entries.
+ */
+kvmppc_reset_htab(spapr->htab_shift);
 
 /* Update the RMA size if necessary */
 if (spapr->vrma_adjust) {
@@ -1709,6 +1719,7 @@ static void ppc_spapr_init(MachineState *machine)
 }
 spapr->htab_shift++;
 }
+spapr_alloc_htab(spapr);
 
 /* Set up Interrupt Controller before we create the VCPUs */
 spapr->icp = xics_system_init(machine,
-- 
2.1.0

[Qemu-devel] [RFC PATCH v0 2/2] spapr: Abort when HTAB of requested size isn't allocated

[Bharata B Rao]

Terminate the guest when HTAB of requested size isn't allocated by
the host.

When memory hotplug is attempted on a guest that has booted with
less than requested HTAB size, the guest kernel will not be able
to gracefully fail the hotplug request. This patch will ensure that
we never end up in a situation where memory hotplug fails due to
less than requested HTAB size.

Signed-off-by: Bharata B Rao 
---
 hw/ppc/spapr.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 4692122..66446af 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -992,6 +992,10 @@ static void spapr_alloc_htab(sPAPRMachineState *spapr)
 
 if (shift > 0) {
 /* Kernel handles htab, we don't need to allocate one */
+if (shift != spapr->htab_shift) {
+error_setg(_abort, "Failed to allocated requested HTAB 
size");
+}
+
 spapr->htab_shift = shift;
 kvmppc_kern_htab = true;
 
-- 
2.1.0

Re: [Qemu-devel] [PATCH v2] target-tilegx: Implement v*add and v*sub instructions

[Richard Henderson]

On 09/21/2015 05:45 PM, Chen Gang wrote:

Oh, v1sub and v2sub look incorrect, need use ^b instead of ^~b.


They're incorrect, but it's not ~b.  Look back and the equation I gave you.


r~

[Qemu-devel] [PATCH v7 04/14] backup: Extract dirty bitmap handling as a separate function

[Fam Zheng]

This will be reused by the coming new transactional completion code.

Signed-off-by: Fam Zheng 
Reviewed-by: John Snow 
Reviewed-by: Max Reitz 
---
 block/backup.c | 27 +--
 1 file changed, 17 insertions(+), 10 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index 965654d..609b199 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -211,6 +211,22 @@ static void backup_iostatus_reset(BlockJob *job)
 bdrv_iostatus_reset(s->target);
 }
 
+static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret)
+{
+BdrvDirtyBitmap *bm;
+BlockDriverState *bs = job->common.bs;
+
+if (ret < 0 || block_job_is_cancelled(>common)) {
+/* Merge the successor back into the parent, delete nothing. */
+bm = bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL);
+assert(bm);
+} else {
+/* Everything is fine, delete this bitmap and install the backup. */
+bm = bdrv_dirty_bitmap_abdicate(bs, job->sync_bitmap, NULL);
+assert(bm);
+}
+}
+
 static const BlockJobDriver backup_job_driver = {
 .instance_size  = sizeof(BackupBlockJob),
 .job_type   = BLOCK_JOB_TYPE_BACKUP,
@@ -430,16 +446,7 @@ static void coroutine_fn backup_run(void *opaque)
 qemu_co_rwlock_unlock(>flush_rwlock);
 
 if (job->sync_bitmap) {
-BdrvDirtyBitmap *bm;
-if (ret < 0 || block_job_is_cancelled(>common)) {
-/* Merge the successor back into the parent, delete nothing. */
-bm = bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL);
-assert(bm);
-} else {
-/* Everything is fine, delete this bitmap and install the backup. 
*/
-bm = bdrv_dirty_bitmap_abdicate(bs, job->sync_bitmap, NULL);
-assert(bm);
-}
+backup_cleanup_sync_bitmap(job, ret);
 }
 hbitmap_free(job->bitmap);
 
-- 
2.4.3

[Qemu-devel] [RFC PATCH v0 0/2] spapr: Abort when HTAB size requirement can't be met

[Bharata B Rao]

HTAB size is a factor of maximum memory size that is specified by maxmem=
command line option. In cases where there is shortage of host memory, host
will not be able to allocate contiguous memory for guest HTAB and will
instead allocate a smaller HTAB. This usually is not a problem but when
user starts hotplugging memory to the guest, we can run out of HTAB entries
and hence memory hotplug fails. This failure should have been handled
gracefully by the guest kernel, but currently it leads to guest kernel OOPS.
This will eventually get fixed when the handling of memory hotplug is
completely moved to kernel for PowerKVM.

Prevent such kernel failure by refusing to boot the guest when requested
HTAB size can't be allocated. However HTAB allocation happens in the
reset path from where it is too late to abort. Hence this patchset
moves the HTAB allocation to machine init and aborts if HTAB size
requirement isn't met.

Two previous related attempts to address this problem:

1. http://lists.nongnu.org/archive/html/qemu-devel/2015-07/msg03325.html
   Aborts the guest from the reset path - not desirable.
2. https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg02602.html
   Silently disables memory hotplug - not desirable.

This patchset applies against David Gibson's spapr-next.

Bharata B Rao (2):
  spapr: Allocate HTAB from machine init
  spapr: Abort when HTAB of requested size isn't allocated

 hw/ppc/spapr.c | 17 -
 1 file changed, 16 insertions(+), 1 deletion(-)

-- 
2.1.0

Re: [Qemu-devel] [PULL 00/26] qapi: QMP introspection

[Peter Maydell]

On 21 September 2015 at 01:03, Markus Armbruster  wrote:
> The following changes since commit 18640989a9f5e4d2e84b566c52ff1fccfa0dbf4a:
>
>   Merge remote-tracking branch 'remotes/afaerber/tags/qom-devices-for-peter' 
> into staging (2015-09-19 15:59:52 +0100)
>
> are available in the git repository at:
>
>   git://repo.or.cz/qemu/armbru.git tags/pull-qapi-2015-09-21
>
> for you to fetch changes up to 1a9a507b2e3e90aa719c96b4c092e7fad7215f21:
>
>   qapi-introspect: Hide type names (2015-09-21 09:56:49 +0200)
>
> 
> qapi: QMP introspection
>
> 
> Markus Armbruster (26):
>   qapi: Rename class QAPISchema to QAPISchemaParser
>   qapi: New QAPISchema intermediate reperesentation
>   qapi: QAPISchema code generation helper methods
>   qapi: New QAPISchemaVisitor
>   tests/qapi-schema: Convert test harness to QAPISchemaVisitor
>   qapi-types: Convert to QAPISchemaVisitor, fixing flat unions
>   qapi-visit: Convert to QAPISchemaVisitor, fixing bugs
>   qapi-commands: Convert to QAPISchemaVisitor
>   qapi: De-duplicate enum code generation
>   qapi-event: Eliminate global variable event_enum_value
>   qapi-event: Convert to QAPISchemaVisitor, fixing data with base
>   qapi: Replace dirty is_c_ptr() by method c_null()
>   qapi: Clean up after recent conversions to QAPISchemaVisitor
>   qapi-visit: Rearrange code a bit
>   qapi-commands: Rearrange code
>   qapi: Rename qmp_marshal_input_FOO() to qmp_marshal_FOO()
>   qapi: De-duplicate parameter list generation
>   qapi-commands: De-duplicate output marshaling functions
>   qapi: Improve built-in type documentation
>   qapi: Make output visitor return qnull() instead of NULL
>   qapi: Introduce a first class 'any' type
>   qom: Don't use 'gen': false for qom-get, qom-set, object-add
>   qapi-schema: Fix up misleading specification of netdev_add
>   qapi: Pseudo-type '**' is now unused, drop it
>   qapi: New QMP command query-qmp-schema for QMP introspection
>   qapi-introspect: Hide type names

Applied, thanks.

-- PMM

[Qemu-devel] [PATCH 3/5] ide: add support for cancelable read requests

[Peter Lieven]

this patch adds a new aio readv compatible function which copies
all data through a bounce buffer. The benefit is that these requests
can be flagged as canceled to avoid guest memory corruption when
a canceled request is completed by the backend at a later stage.

If an IDE protocol wants to use this function it has to pipe
all read requests through ide_readv_cancelable and it may then
enable requests_cancelable in the IDEState.

If this state is enable we can avoid the blocking blk_drain_all
in case of a BMDMA reset.

Currently only read operations are cancelable thus we can only
use this logic for read-only devices.

Signed-off-by: Peter Lieven 
---
 hw/ide/core.c | 54 ++
 hw/ide/internal.h | 16 
 hw/ide/pci.c  | 42 --
 3 files changed, 98 insertions(+), 14 deletions(-)

diff --git a/hw/ide/core.c b/hw/ide/core.c
index 317406d..24547ce 100644
--- a/hw/ide/core.c
+++ b/hw/ide/core.c
@@ -561,6 +561,59 @@ static bool ide_sect_range_ok(IDEState *s,
 return true;
 }
 
+static void ide_readv_cancelable_cb(void *opaque, int ret)
+{
+IDECancelableRequest *req = opaque;
+if (!req->canceled) {
+if (!ret) {
+qemu_iovec_from_buf(req->org_qiov, 0, req->buf, 
req->org_qiov->size);
+}
+req->org_cb(req->org_opaque, ret);
+}
+QLIST_REMOVE(req, list);
+qemu_vfree(req->buf);
+qemu_iovec_destroy(>qiov);
+g_free(req);
+}
+
+#define MAX_CANCELABLE_REQS 16
+
+BlockAIOCB *ide_readv_cancelable(IDEState *s, int64_t sector_num,
+ QEMUIOVector *iov, int nb_sectors,
+ BlockCompletionFunc *cb, void *opaque)
+{
+BlockAIOCB *aioreq;
+IDECancelableRequest *req;
+int c = 0;
+
+QLIST_FOREACH(req, >cancelable_requests, list) {
+c++;
+}
+if (c > MAX_CANCELABLE_REQS) {
+return NULL;
+}
+
+req = g_new0(IDECancelableRequest, 1);
+qemu_iovec_init(>qiov, 1);
+req->buf = qemu_blockalign(blk_bs(s->blk), iov->size);
+qemu_iovec_add(>qiov, req->buf, iov->size);
+req->org_qiov = iov;
+req->org_cb = cb;
+req->org_opaque = opaque;
+
+aioreq = blk_aio_readv(s->blk, sector_num, >qiov, nb_sectors,
+   ide_readv_cancelable_cb, req);
+if (aioreq == NULL) {
+qemu_vfree(req->buf);
+qemu_iovec_destroy(>qiov);
+g_free(req);
+} else {
+QLIST_INSERT_HEAD(>cancelable_requests, req, list);
+}
+
+return aioreq;
+}
+
 static void ide_sector_read(IDEState *s);
 
 static void ide_sector_read_cb(void *opaque, int ret)
@@ -805,6 +858,7 @@ void ide_start_dma(IDEState *s, BlockCompletionFunc *cb)
 s->bus->retry_unit = s->unit;
 s->bus->retry_sector_num = ide_get_sector(s);
 s->bus->retry_nsector = s->nsector;
+s->bus->s = s;
 if (s->bus->dma->ops->start_dma) {
 s->bus->dma->ops->start_dma(s->bus->dma, s, cb);
 }
diff --git a/hw/ide/internal.h b/hw/ide/internal.h
index 05e93ff..ad188c2 100644
--- a/hw/ide/internal.h
+++ b/hw/ide/internal.h
@@ -343,6 +343,16 @@ enum ide_dma_cmd {
 #define ide_cmd_is_read(s) \
((s)->dma_cmd == IDE_DMA_READ)
 
+typedef struct IDECancelableRequest {
+QLIST_ENTRY(IDECancelableRequest) list;
+QEMUIOVector qiov;
+uint8_t *buf;
+QEMUIOVector *org_qiov;
+BlockCompletionFunc *org_cb;
+void *org_opaque;
+bool canceled;
+} IDECancelableRequest;
+
 /* NOTE: IDEState represents in fact one drive */
 struct IDEState {
 IDEBus *bus;
@@ -396,6 +406,8 @@ struct IDEState {
 BlockAIOCB *pio_aiocb;
 struct iovec iov;
 QEMUIOVector qiov;
+QLIST_HEAD(, IDECancelableRequest) cancelable_requests;
+bool requests_cancelable;
 /* ATA DMA state */
 int32_t io_buffer_offset;
 int32_t io_buffer_size;
@@ -468,6 +480,7 @@ struct IDEBus {
 uint8_t retry_unit;
 int64_t retry_sector_num;
 uint32_t retry_nsector;
+IDEState *s;
 };
 
 #define TYPE_IDE_DEVICE "ide-device"
@@ -572,6 +585,9 @@ void ide_set_inactive(IDEState *s, bool more);
 BlockAIOCB *ide_issue_trim(BlockBackend *blk,
 int64_t sector_num, QEMUIOVector *qiov, int nb_sectors,
 BlockCompletionFunc *cb, void *opaque);
+BlockAIOCB *ide_readv_cancelable(IDEState *s, int64_t sector_num,
+ QEMUIOVector *iov, int nb_sectors,
+ BlockCompletionFunc *cb, void *opaque);
 
 /* hw/ide/atapi.c */
 void ide_atapi_cmd(IDEState *s);
diff --git a/hw/ide/pci.c b/hw/ide/pci.c
index d31ff88..5587183 100644
--- a/hw/ide/pci.c
+++ b/hw/ide/pci.c
@@ -240,21 +240,35 @@ void bmdma_cmd_writeb(BMDMAState *bm, uint32_t val)
 /* Ignore writes to SSBM if it keeps the old value */
 if ((val & BM_CMD_START) != (bm->cmd & BM_CMD_START)) {
 if (!(val & BM_CMD_START)) {
-/*
- * We can't cancel Scatter Gather DMA 

[Qemu-devel] [PATCH 4/5] ide/atapi: enable cancelable requests

[Peter Lieven]

Signed-off-by: Peter Lieven 
---
 hw/ide/atapi.c | 4 ++--
 hw/ide/core.c  | 1 +
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/hw/ide/atapi.c b/hw/ide/atapi.c
index b209ed9..ab45495 100644
--- a/hw/ide/atapi.c
+++ b/hw/ide/atapi.c
@@ -141,7 +141,7 @@ static int cd_read_sector(IDEState *s, int lba, void *buf, 
int sector_size)
 s->iov.iov_len = 4 * BDRV_SECTOR_SIZE;
 qemu_iovec_init_external(>qiov, >iov, 1);
 
-if (blk_aio_readv(s->blk, (int64_t)lba << 2, >qiov, 4,
+if (ide_readv_cancelable(s, (int64_t)lba << 2, >qiov, 4,
   cd_read_sector_cb, s) == NULL) {
 return -EIO;
 }
@@ -368,7 +368,7 @@ static void ide_atapi_cmd_read_dma_cb(void *opaque, int ret)
 s->bus->dma->iov.iov_len = n * 4 * 512;
 qemu_iovec_init_external(>bus->dma->qiov, >bus->dma->iov, 1);
 
-s->bus->dma->aiocb = blk_aio_readv(s->blk, (int64_t)s->lba << 2,
+s->bus->dma->aiocb = ide_readv_cancelable(s, (int64_t)s->lba << 2,
>bus->dma->qiov, n * 4,
ide_atapi_cmd_read_dma_cb, s);
 if (s->bus->dma->aiocb == NULL) {
diff --git a/hw/ide/core.c b/hw/ide/core.c
index 24547ce..5c7a346 100644
--- a/hw/ide/core.c
+++ b/hw/ide/core.c
@@ -2330,6 +2330,7 @@ int ide_init_drive(IDEState *s, BlockBackend *blk, 
IDEDriveKind kind,
 if (kind == IDE_CD) {
 blk_set_dev_ops(blk, _cd_block_ops, s);
 blk_set_guest_block_size(blk, 2048);
+s->requests_cancelable = true;
 } else {
 if (!blk_is_inserted(s->blk)) {
 error_report("Device needs media, but drive is empty");
-- 
1.9.1

[Qemu-devel] [PATCH 2/5] ide/atapi: blk_aio_readv may return NULL

[Peter Lieven]

Signed-off-by: Peter Lieven 
---
 hw/ide/atapi.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/hw/ide/atapi.c b/hw/ide/atapi.c
index 9257e1c..b209ed9 100644
--- a/hw/ide/atapi.c
+++ b/hw/ide/atapi.c
@@ -371,6 +371,10 @@ static void ide_atapi_cmd_read_dma_cb(void *opaque, int 
ret)
 s->bus->dma->aiocb = blk_aio_readv(s->blk, (int64_t)s->lba << 2,
>bus->dma->qiov, n * 4,
ide_atapi_cmd_read_dma_cb, s);
+if (s->bus->dma->aiocb == NULL) {
+ide_atapi_io_error(s, -EIO);
+goto eot;
+}
 return;
 
 eot:
-- 
1.9.1

Re: [Qemu-devel] RFC: virtio-peer shared memory based peer communication device

[Jan Kiszka]

On 2015-09-18 23:11, Paolo Bonzini wrote:
> On 18/09/2015 18:29, Claudio Fontana wrote:
>>
>> this is a first RFC for virtio-peer 0.1, which is still very much a work in 
>> progress:
>>
>> https://github.com/hw-claudio/virtio-peer/wiki
>>
>> It is also available as PDF there, but the text is reproduced here for 
>> commenting:
>>
>> Peer shared memory communication device (virtio-peer)
> 
> Apart from the windows idea, how does virtio-peer compare to virtio-rpmsg?

rpmsg is a very specialized thing. It targets single AMP cores, assuming
that those have full access to the main memory. And it is also a
centralized approach where all message go through the main Linux
instance. I suspect we could cover that use case as well with generic
inter-vm shared memory device, but I didn't think about all details yet.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

[Qemu-devel] [PATCH 4/9] apic_internal.h: fix formatting and drop unused consts

[Denis V. Lunev]

From: Pavel Butsykin 

Fix formatting of local apic definitions and drop unused constant
APIC_INPUT_POLARITY, APIC_SEND_PENDING. Magic numbers in shifts are
replaced with constants defined just above.

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 include/hw/i386/apic_internal.h | 16 +++-
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index a1db16e..7813396 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -62,12 +62,10 @@
 #define APIC_LVT_DELIV_MOD_SHIFT8
 
 #define APIC_LVT_TIMER_TSCDEADLINE  (2 << APIC_LVT_TIMER_SHIFT)
-#define APIC_LVT_TIMER_PERIODIC (1<<17)
-#define APIC_LVT_MASKED (1<<16)
-#define APIC_LVT_LEVEL_TRIGGER  (1<<15)
-#define APIC_LVT_REMOTE_IRR (1<<14)
-#define APIC_INPUT_POLARITY (1<<13)
-#define APIC_SEND_PENDING   (1<<12)
+#define APIC_LVT_TIMER_PERIODIC (1 << APIC_LVT_TIMER_SHIFT)
+#define APIC_LVT_MASKED (1 << APIC_LVT_MASKED_SHIFT)
+#define APIC_LVT_LEVEL_TRIGGER  (1 << APIC_LVT_LEVEL_TRIGGER_SHIFT)
+#define APIC_LVT_REMOTE_IRR (1 << APIC_LVT_REMOTE_IRR_SHIFT)
 #define APIC_LVT_INT_POLARITY   (1 << APIC_LVT_INT_POLARITY_SHIFT)
 #define APIC_LVT_DELIV_STS  (1 << APIC_LVT_DELIV_STS_SHIFT)
 #define APIC_LVT_DELIV_MOD  (7 << APIC_LVT_DELIV_MOD_SHIFT)
@@ -79,7 +77,7 @@
 #define APIC_ESR_SEND_ACCEPT_SHIFT  2
 #define APIC_ESR_RECV_CHECK_SUM_SHIFT   1
 
-#define APIC_ESR_ILLEGAL_ADDRESS(1 << 7)
+#define APIC_ESR_ILLEGAL_ADDRESS(1 << APIC_ESR_ILL_ADDRESS_SHIFT)
 #define APIC_ESR_RECV_ILLEGAL_VECT  (1 << APIC_ESR_RECV_ILL_VECT_SHIFT)
 #define APIC_ESR_SEND_ILLEGAL_VECT  (1 << APIC_ESR_SEND_ILL_VECT_SHIFT)
 #define APIC_ESR_RECV_ACCEPT(1 << APIC_ESR_RECV_ACCEPT_SHIFT)
@@ -117,8 +115,8 @@
 #define APIC_SPURIO_FOCUS   (1 << APIC_SPURIO_FOCUS_SHIFT)
 #define APIC_SPURIO_ENABLED (1 << APIC_SPURIO_ENABLED_SHIFT)
 
-#define APIC_SV_DIRECTED_IO (1<<12)
-#define APIC_SV_ENABLE  (1<<8)
+#define APIC_SV_DIRECTED_IO (1 << 12)
+#define APIC_SV_ENABLE  (1 << 8)
 
 #define VAPIC_ENABLE_BIT0
 #define VAPIC_ENABLE_MASK   (1 << VAPIC_ENABLE_BIT)
-- 
2.1.4

[Qemu-devel] [PATCH 1/9] apic_internal.h: make some apic_get_* functions externally visible

[Denis V. Lunev]

From: Pavel Butsykin 

Move apic_get_bit(), apic_set_bit() to apic_internal.h, make the
apic_get_ppr symbol external. It's necessary to work with isr, tmr,
irr and ppr outside hw/intc/apic.c

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 hw/intc/apic.c  | 18 +-
 include/hw/i386/apic_internal.h | 18 ++
 2 files changed, 19 insertions(+), 17 deletions(-)

diff --git a/hw/intc/apic.c b/hw/intc/apic.c
index 77b639c..52ac8b2 100644
--- a/hw/intc/apic.c
+++ b/hw/intc/apic.c
@@ -51,14 +51,6 @@ static int apic_ffs_bit(uint32_t value)
 return ctz32(value);
 }
 
-static inline void apic_set_bit(uint32_t *tab, int index)
-{
-int i, mask;
-i = index >> 5;
-mask = 1 << (index & 0x1f);
-tab[i] |= mask;
-}
-
 static inline void apic_reset_bit(uint32_t *tab, int index)
 {
 int i, mask;
@@ -67,14 +59,6 @@ static inline void apic_reset_bit(uint32_t *tab, int index)
 tab[i] &= ~mask;
 }
 
-static inline int apic_get_bit(uint32_t *tab, int index)
-{
-int i, mask;
-i = index >> 5;
-mask = 1 << (index & 0x1f);
-return !!(tab[i] & mask);
-}
-
 /* return -1 if no bit is set */
 static int get_highest_priority_int(uint32_t *tab)
 {
@@ -318,7 +302,7 @@ static uint8_t apic_get_tpr(APICCommonState *s)
 return s->tpr >> 4;
 }
 
-static int apic_get_ppr(APICCommonState *s)
+int apic_get_ppr(APICCommonState *s)
 {
 int tpr, isrv, ppr;
 
diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index 26632ac..5a213bc 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -147,4 +147,22 @@ void apic_enable_vapic(DeviceState *d, hwaddr paddr);
 void vapic_report_tpr_access(DeviceState *dev, CPUState *cpu, target_ulong ip,
  TPRAccess access);
 
+int apic_get_ppr(APICCommonState *s);
+
+static inline void apic_set_bit(uint32_t *tab, int index)
+{
+int i, mask;
+i = index >> 5;
+mask = 1 << (index & 0x1f);
+tab[i] |= mask;
+}
+
+static inline int apic_get_bit(uint32_t *tab, int index)
+{
+int i, mask;
+i = index >> 5;
+mask = 1 << (index & 0x1f);
+return !!(tab[i] & mask);
+}
+
 #endif /* !QEMU_APIC_INTERNAL_H */
-- 
2.1.4

[Qemu-devel] [PATCH 2/9] apic_internal.h: rename ESR_ILLEGAL_ADDRESS to APIC_ESR_ILLEGAL_ADDRESS

[Denis V. Lunev]

From: Pavel Butsykin 

Added prefix APIC_ for determining the constant of a particular subsystem,
improve the overall readability and match other constant names.

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 hw/intc/apic.c  | 4 ++--
 include/hw/i386/apic_internal.h | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/hw/intc/apic.c b/hw/intc/apic.c
index 52ac8b2..0a840b8 100644
--- a/hw/intc/apic.c
+++ b/hw/intc/apic.c
@@ -723,7 +723,7 @@ static uint32_t apic_mem_readl(void *opaque, hwaddr addr)
 val = s->divide_conf;
 break;
 default:
-s->esr |= ESR_ILLEGAL_ADDRESS;
+s->esr |= APIC_ESR_ILLEGAL_ADDRESS;
 val = 0;
 break;
 }
@@ -836,7 +836,7 @@ static void apic_mem_writel(void *opaque, hwaddr addr, 
uint32_t val)
 }
 break;
 default:
-s->esr |= ESR_ILLEGAL_ADDRESS;
+s->esr |= APIC_ESR_ILLEGAL_ADDRESS;
 break;
 }
 }
diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index 5a213bc..188131c 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -57,7 +57,7 @@
 #define APIC_INPUT_POLARITY (1<<13)
 #define APIC_SEND_PENDING   (1<<12)
 
-#define ESR_ILLEGAL_ADDRESS (1 << 7)
+#define APIC_ESR_ILLEGAL_ADDRESS(1 << 7)
 
 #define APIC_SV_DIRECTED_IO (1<<12)
 #define APIC_SV_ENABLE  (1<<8)
-- 
2.1.4

Re: [Qemu-devel] [PATCH] linux-user: fix cmsg conversion in case of multiple headers

[Peter Maydell]

On 20 September 2015 at 22:34, Jonathan Neuschäfer
 wrote:
> On Fri, Sep 04, 2015 at 01:48:39PM +0100, Peter Maydell wrote:
>> On 3 September 2015 at 06:27, Jonathan Neuschäfer  
>> wrote:
>> > Currently, __target_cmsg_nxthdr compares a pointer derived from
>> > target_cmsg against the msg_control field of target_msgh (through
>> > subtraction).  This failed for me when emulating i386 code under x86_64,
>> > because pointers in the host address space and pointers in the guest
>> > address space were not the same.  This patch passes the initial value of
>> > target_cmsg into __target_cmsg_nxthdr.
>> >
>> > I found and fixed two more related bugs:
>> > - __target_cmsg_nxthdr now returns the new cmsg pointer instead of the
>> >   old one.
>> > - tgt_space (in host_to_target_cmsg) doesn't count "sizeof (struct
>> >   target_cmsghdr)" twice anymore.
>> >
>> > Signed-off-by: Jonathan Neuschäfer 
>>
>> Reviewed-by: Peter Maydell 
>
> Ping.
>
> What's the status of this patch?

It's waiting for Riku to wake up and put it into a linux-user
pull request.

thanks
-- MPM

[Qemu-devel] [PATCH v3 1/1] block/gluster: add support for multiple gluster backup volfile servers

[Prasanna Kumar Kalever]

This patch adds a way to specify multiple backup volfile servers to the gluster
block backend of QEMU with tcp|rdma transport types and their port numbers.

Problem:

Currenly VM Image on gluster volume is specified like this:

file=gluster[+tcp]://server1[:port]/testvol/a.img

Assuming we have have three servers in trustred pool with replica 3 volume
in action and unfortunately server1 (mentioned in the command above) went down
for some reason, since the volume is replica 3 we now have other 2 servers
active from which we can boot the VM.

But currently there is no mechanism to pass the other 2 gluster server
addresses to qemu.

Solution:

New way of specifying VM Image on gluster volume with backup volfile servers:
(We still support old syntax to maintain backward compatibility)

Basic command line syntax looks like:

Pattern I:
 -drive driver=gluster,
volname=testvol,file.image-path=/path/a.raw,
backup-volfile-servers.0.server=1.2.3.4,
   [backup-volfile-servers.0.port=24007,]
   [backup-volfile-servers.0.transport=tcp,]
backup-volfile-servers.1.server=5.6.7.8,
   [backup-volfile-servers.1.port=24008,]
   [backup-volfile-servers.1.transport=rdma,] ...

Pattern II:
 'json:{"driver":"qcow2","file":{"driver":"gluster",
   "volname":"testvol","image-path":"/path/a.qcow2",
   "backup-volfile-servers":[{tuple0},{tuple1}, ...{tupleN}]}}'

   driver   => 'gluster' (protocol name)
   volname  => name of gluster volume where our VM image resides
   image-path   => is the absolute path of image in gluster volume

  {tuple}   => {"server":"1.2.3.4"[,"port":"24007","transport":"tcp"]}

   server   => server address (hostname/ipv4/ipv6 addresses)
   port => port number on which glusterd is listening. (default 24007)
   tranport => transport type used to connect to gluster management daemon,
it can be tcp|rdma (default 'tcp')

Examples:
1.
 -drive driver=qcow2,file.driver=gluster,
file.volname=testvol,file.image-path=/path/a.qcow2,
file.backup-volfile-servers.0.server=1.2.3.4,
file.backup-volfile-servers.0.port=24007,
file.backup-volfile-servers.0.transport=tcp,
file.backup-volfile-servers.1.server=5.6.7.8,
file.backup-volfile-servers.1.port=24008,
file.backup-volfile-servers.1.transport=rdma
2.
 'json:{"driver":"qcow2","file":{"driver":"gluster","volname":"testvol",
 "image-path":"/path/a.qcow2","backup-volfile-servers":
 [{"server":"1.2.3.4","port":"24007","transport":"tcp"},
  {"server":"4.5.6.7","port":"24008","transport":"rdma"},] } }'

This patch gives a mechanism to provide all the server addresses which are in
replica set, so in case server1 is down VM can still boot from any of the
active servers.

This is equivalent to the backup-volfile-servers option supported by
mount.glusterfs (FUSE way of mounting gluster volume)

This patch depends on a recent fix in libgfapi raised as part of this work:
http://review.gluster.org/#/c/12114/ (not merged yet)

Credits: Sincere thanks to Kevin Wolf  and
"Deepak C Shetty"  for inputs and all their support

v1:
multiple server addresses but common port number and transport type
pattern: URI syntax with query (?) delimitor
syntax:
file=gluster[+transport-type]://server1:24007/testvol/a.img\
 ?backup-volfile-servers=server2=server3

v2:
multiple server addresses each have their own port number, but all use
common transport type
pattern: URI syntax  with query (?) delimiter
syntax:
file=gluster[+transport-type]://[server[:port]]/testvol/a.img\
 [?backup-volfile-servers=server1[:port]\
  =server2[:port]]

v3:
multiple server addresses each have their own port number and transport type
pattern: changed to json
syntax:
'json:{"driver":"qcow2","file":{"driver":"gluster","volname":"testvol",
   "image-path":"/path/a.qcow2","backup-volfile-servers":
 [{"server":"1.2.3.4","port":"24007","transport":"tcp"},
  {"server":"4.5.6.7","port":"24008","transport":"rdma"},] } }'

Signed-off-by: Prasanna Kumar Kalever 
---
 block/gluster.c  | 412 +--
 qapi/block-core.json |  40 -
 2 files changed, 405 insertions(+), 47 deletions(-)

diff --git a/block/gluster.c b/block/gluster.c
index 1eb3a8c..48ae62e 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -11,6 +11,17 @@
 #include "block/block_int.h"
 #include "qemu/uri.h"
 
+#define MAX_PORTNUM_SIZE  6   /* max port num 65535 */
+
+#define GLUSTER_OPT_FILENAME  "filename"
+#define GLUSTER_OPT_VOLNAME   "volname"
+#define GLUSTER_OPT_IMAGE_PATH"image-path"
+#define GLUSTER_OPT_SERVER"server"
+#define GLUSTER_OPT_PORT  "port"
+#define GLUSTER_OPT_TRANSPORT "transport"
+#define GLUSTER_OPT_READ_PATTERN  "backup-volfile-servers."
+
+
 typedef struct 

Re: [Qemu-devel] RFC: virtio-peer shared memory based peer communication device

[Michael S. Tsirkin]

On Fri, Sep 18, 2015 at 06:29:27PM +0200, Claudio Fontana wrote:
> Hello,
> 
> this is a first RFC for virtio-peer 0.1, which is still very much a work in 
> progress:
> 
> https://github.com/hw-claudio/virtio-peer/wiki
> 
> It is also available as PDF there, but the text is reproduced here for 
> commenting:
> 
> Peer shared memory communication device (virtio-peer)
> 
> General Overview
> 
> (I recommend looking at the PDF for some clarifying pictures)
> 
> The Virtio Peer shared memory communication device (virtio-peer) is a
> virtual device which allows high performance low latency guest to
> guest communication. It uses a new queue extension feature tentatively
> called VIRTIO_F_WINDOW which indicates that descriptor tables,
> available and used rings and Queue Data reside in physical memory
> ranges called Windows, each identified with an unique identifier
> called WindowID.

So if I had to summarize the difference from regular virtio,
I'd say the main one is that this uses window id + offset
instead of the physical address.


My question is - why do it?

All windows are in memory space, are they not?

How about guest using full physical addresses,
and hypervisor sending the window physical address
to VM2?

VM2 can uses that to find both window id and offset.


This way at least VM1 can use regular virtio without changes.

-- 
MST

Re: [Qemu-devel] [PATCH v3 0/9] hmp command IO- and Local APIC dump state

[Paolo Bonzini]

On 21/09/2015 12:57, Denis V. Lunev wrote:
> Added the hmp command to query IO- and Local APIC registers state,
> it can be very useful to identify problems related to the emulation devices.
> 
> (qemu) info lapic
> dumping local APIC state for CPU 0
> 
> LVT0 0x00010700 active-hi edge  masked  ExtINT (vec 0)
> LVT1 0x0400 active-hi edge  NMI
> LVTPC0x0001 active-hi edge  masked  Fixed  (vec 0)
> LVTERR   0x00fe active-hi edge  Fixed  (vec 
> 254)
> LVTTHMR  0x0001 active-hi edge  masked  Fixed  (vec 0)
> LVTT 0x00ef active-hi edge one-shot Fixed  (vec 
> 239)
> TimerDCR=0x3 (divide by 16) initial_count = 62278
> SPIV 0x01ff APIC enabled, focus=off, spurious vec 255
> ICR  0x00fd physical edge de-assert no

s/no/no-shorthand/ or similar.

> ICR2 0x0001 cpu 1 (X2APIC ID)
> ESR  0x
> ISR

Adding a "(none)" here would be nice.

> IRR  239
> 
> APR 0x00 TPR 0x00 DFR 0x0f LDR 0x00 PPR 0x00
> 
> (qemu) info ioapic
> ioapic id=0x00 sel=0x2e (redir[15])
> pin 0  0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 1  0x0031 dest=0 vec=49  active-hi edge fixed  
> physical
> pin 2  0x0030 dest=0 vec=48  active-hi edge fixed  
> physical
> pin 3  0x0033 dest=0 vec=51  active-hi edge fixed  
> physical
> pin 4  0x00010034 dest=0 vec=52  active-hi edge  masked fixed  
> physical
> pin 5  0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 6  0x0036 dest=0 vec=54  active-hi edge fixed  
> physical
> pin 7  0x0037 dest=0 vec=55  active-hi edge fixed  
> physical
> pin 8  0x0038 dest=0 vec=56  active-hi edge fixed  
> physical
> pin 9  0x00018041 dest=0 vec=65  active-hi level masked fixed  
> physical
> pin 10 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 11 0x03008081 dest=3 vec=129 active-hi levelfixed  
> physical
> pin 12 0x003c dest=0 vec=60  active-hi edge fixed  
> physical
> pin 13 0x003d dest=0 vec=61  active-hi edge fixed  
> physical
> pin 14 0x003e dest=0 vec=62  active-hi edge fixed  
> physical
> pin 15 0x003f dest=0 vec=63  active-hi edge fixed  
> physical
> pin 16 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 17 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 18 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 19 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 20 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 21 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 22 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> pin 23 0x0001 dest=0 vec=0   active-hi edge  masked fixed  
> physical
> IRR (none)

I would print IRR as a bitmap here, like ISR/IRR in the local APIC dump.
 Similarly, it would be nice to print the "Remote IRR" bits of the
redirection table (printing them as a bitmap with the same format).

> 
> Changes from v1:
>  - implementation of hmp commands moved to the target-i386 part
>  - the cpu_dump_apic_local_state interface moved to the target-i386 part
> 
> Changes from v2:
>  - lapic and ioapic format dump are fixed for more human readable view
> 
> Signed-off-by: Pavel Butsykin 
> Signed-off-by: Denis V. Lunev 
> CC: Paolo Bonzini 
> CC: Andreas Färber 

I would like a second look at the HMP code.

Paolo

> Pavel Butsykin (9):
>   apic_internal.h: make some apic_get_* functions externally visible
>   apic_internal.h: rename ESR_ILLEGAL_ADDRESS to
> APIC_ESR_ILLEGAL_ADDRESS
>   apic_internal.h: added more constants
>   apic_internal.h: fix formatting and drop unused consts
>   monitor: make monitor_fprintf and mon_get_cpu externally visible
>   hmp: added local apic dump state
>   ioapic_internal.h: added more constants
>   hmp: added io apic dump state
>   hmp: implemented io apic dump state for TCG
> 
>  disas.c   |  10 --
>  hmp-commands-info.hx  |  32 +++
>  hw/i386/kvm/ioapic.c  |  10 ++
>  hw/intc/apic.c|  22 +
>  hw/intc/ioapic.c  |  12 +++
>  hw/intc/ioapic_common.c   |  37 
>  include/hw/i386/apic_internal.h   |  96 ---
>  include/hw/i386/ioapic_internal.h |   7 ++
>  include/hw/i386/pc.h  |   5 +
>  include/monitor/hmp-target.h  |   3 +
>  include/monitor/monitor.h |   1 +
>  

[Qemu-devel] [PATCH 3/9] apic_internal.h: added more constants

[Denis V. Lunev]

From: Pavel Butsykin 

These constants are needed for optimal access to
bit fields local apic registers without magic numbers.

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 include/hw/i386/apic_internal.h | 58 +
 1 file changed, 58 insertions(+)

diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index 188131c..a1db16e 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -50,14 +50,72 @@
 #define APIC_TRIGGER_EDGE   0
 #define APIC_TRIGGER_LEVEL  1
 
+#define APIC_VECTOR_MASK0xff
+#define APIC_DCR_MASK   0xf
+
+#define APIC_LVT_TIMER_SHIFT17
+#define APIC_LVT_MASKED_SHIFT   16
+#define APIC_LVT_LEVEL_TRIGGER_SHIFT15
+#define APIC_LVT_REMOTE_IRR_SHIFT   14
+#define APIC_LVT_INT_POLARITY_SHIFT 13
+#define APIC_LVT_DELIV_STS_SHIFT12
+#define APIC_LVT_DELIV_MOD_SHIFT8
+
+#define APIC_LVT_TIMER_TSCDEADLINE  (2 << APIC_LVT_TIMER_SHIFT)
 #define APIC_LVT_TIMER_PERIODIC (1<<17)
 #define APIC_LVT_MASKED (1<<16)
 #define APIC_LVT_LEVEL_TRIGGER  (1<<15)
 #define APIC_LVT_REMOTE_IRR (1<<14)
 #define APIC_INPUT_POLARITY (1<<13)
 #define APIC_SEND_PENDING   (1<<12)
+#define APIC_LVT_INT_POLARITY   (1 << APIC_LVT_INT_POLARITY_SHIFT)
+#define APIC_LVT_DELIV_STS  (1 << APIC_LVT_DELIV_STS_SHIFT)
+#define APIC_LVT_DELIV_MOD  (7 << APIC_LVT_DELIV_MOD_SHIFT)
+
+#define APIC_ESR_ILL_ADDRESS_SHIFT  7
+#define APIC_ESR_RECV_ILL_VECT_SHIFT6
+#define APIC_ESR_SEND_ILL_VECT_SHIFT5
+#define APIC_ESR_RECV_ACCEPT_SHIFT  3
+#define APIC_ESR_SEND_ACCEPT_SHIFT  2
+#define APIC_ESR_RECV_CHECK_SUM_SHIFT   1
 
 #define APIC_ESR_ILLEGAL_ADDRESS(1 << 7)
+#define APIC_ESR_RECV_ILLEGAL_VECT  (1 << APIC_ESR_RECV_ILL_VECT_SHIFT)
+#define APIC_ESR_SEND_ILLEGAL_VECT  (1 << APIC_ESR_SEND_ILL_VECT_SHIFT)
+#define APIC_ESR_RECV_ACCEPT(1 << APIC_ESR_RECV_ACCEPT_SHIFT)
+#define APIC_ESR_SEND_ACCEPT(1 << APIC_ESR_SEND_ACCEPT_SHIFT)
+#define APIC_ESR_RECV_CHECK_SUM (1 << APIC_ESR_RECV_CHECK_SUM_SHIFT)
+#define APIC_ESR_SEND_CHECK_SUM 1
+
+#define APIC_ICR_DEST_SHIFT 24
+#define APIC_ICR_DEST_SHORT_SHIFT   18
+#define APIC_ICR_TRIGGER_MOD_SHIFT  15
+#define APIC_ICR_LEVEL_SHIFT14
+#define APIC_ICR_DELIV_STS_SHIFT12
+#define APIC_ICR_DEST_MOD_SHIFT 11
+#define APIC_ICR_DELIV_MOD_SHIFT8
+
+#define APIC_ICR_DEST_SHORT (3 << APIC_ICR_DEST_SHORT_SHIFT)
+#define APIC_ICR_TRIGGER_MOD(1 << APIC_ICR_TRIGGER_MOD_SHIFT)
+#define APIC_ICR_LEVEL  (1 << APIC_ICR_LEVEL_SHIFT)
+#define APIC_ICR_DELIV_STS  (1 << APIC_ICR_DELIV_STS_SHIFT)
+#define APIC_ICR_DEST_MOD   (1 << APIC_ICR_DEST_MOD_SHIFT)
+#define APIC_ICR_DELIV_MOD  (7 << APIC_ICR_DELIV_MOD_SHIFT)
+
+#define APIC_PR_CLASS_SHIFT 4
+#define APIC_PR_SUB_CLASS   0xf
+
+#define APIC_LOGDEST_XAPIC_SHIFT4
+#define APIC_LOGDEST_XAPIC_ID   0xf
+
+#define APIC_LOGDEST_X2APIC_SHIFT   16
+#define APIC_LOGDEST_X2APIC_ID  0x
+
+#define APIC_SPURIO_FOCUS_SHIFT 9
+#define APIC_SPURIO_ENABLED_SHIFT   8
+
+#define APIC_SPURIO_FOCUS   (1 << APIC_SPURIO_FOCUS_SHIFT)
+#define APIC_SPURIO_ENABLED (1 << APIC_SPURIO_ENABLED_SHIFT)
 
 #define APIC_SV_DIRECTED_IO (1<<12)
 #define APIC_SV_ENABLE  (1<<8)
-- 
2.1.4

[Qemu-devel] [PATCH v3 0/9] hmp command IO- and Local APIC dump state

[Denis V. Lunev]

Added the hmp command to query IO- and Local APIC registers state,
it can be very useful to identify problems related to the emulation devices.

(qemu) info lapic
dumping local APIC state for CPU 0

LVT0 0x00010700 active-hi edge  masked  ExtINT (vec 0)
LVT1 0x0400 active-hi edge  NMI
LVTPC0x0001 active-hi edge  masked  Fixed  (vec 0)
LVTERR   0x00fe active-hi edge  Fixed  (vec 254)
LVTTHMR  0x0001 active-hi edge  masked  Fixed  (vec 0)
LVTT 0x00ef active-hi edge one-shot Fixed  (vec 239)
TimerDCR=0x3 (divide by 16) initial_count = 62278
SPIV 0x01ff APIC enabled, focus=off, spurious vec 255
ICR  0x00fd physical edge de-assert no
ICR2 0x0001 cpu 1 (X2APIC ID)
ESR  0x
ISR
IRR  239

APR 0x00 TPR 0x00 DFR 0x0f LDR 0x00 PPR 0x00

(qemu) info ioapic
ioapic id=0x00 sel=0x2e (redir[15])
pin 0  0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 1  0x0031 dest=0 vec=49  active-hi edge fixed  physical
pin 2  0x0030 dest=0 vec=48  active-hi edge fixed  physical
pin 3  0x0033 dest=0 vec=51  active-hi edge fixed  physical
pin 4  0x00010034 dest=0 vec=52  active-hi edge  masked fixed  physical
pin 5  0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 6  0x0036 dest=0 vec=54  active-hi edge fixed  physical
pin 7  0x0037 dest=0 vec=55  active-hi edge fixed  physical
pin 8  0x0038 dest=0 vec=56  active-hi edge fixed  physical
pin 9  0x00018041 dest=0 vec=65  active-hi level masked fixed  physical
pin 10 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 11 0x03008081 dest=3 vec=129 active-hi levelfixed  physical
pin 12 0x003c dest=0 vec=60  active-hi edge fixed  physical
pin 13 0x003d dest=0 vec=61  active-hi edge fixed  physical
pin 14 0x003e dest=0 vec=62  active-hi edge fixed  physical
pin 15 0x003f dest=0 vec=63  active-hi edge fixed  physical
pin 16 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 17 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 18 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 19 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 20 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 21 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 22 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 23 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
IRR (none)

Changes from v1:
 - implementation of hmp commands moved to the target-i386 part
 - the cpu_dump_apic_local_state interface moved to the target-i386 part

Changes from v2:
 - lapic and ioapic format dump are fixed for more human readable view

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 

Pavel Butsykin (9):
  apic_internal.h: make some apic_get_* functions externally visible
  apic_internal.h: rename ESR_ILLEGAL_ADDRESS to
APIC_ESR_ILLEGAL_ADDRESS
  apic_internal.h: added more constants
  apic_internal.h: fix formatting and drop unused consts
  monitor: make monitor_fprintf and mon_get_cpu externally visible
  hmp: added local apic dump state
  ioapic_internal.h: added more constants
  hmp: added io apic dump state
  hmp: implemented io apic dump state for TCG

 disas.c   |  10 --
 hmp-commands-info.hx  |  32 +++
 hw/i386/kvm/ioapic.c  |  10 ++
 hw/intc/apic.c|  22 +
 hw/intc/ioapic.c  |  12 +++
 hw/intc/ioapic_common.c   |  37 
 include/hw/i386/apic_internal.h   |  96 ---
 include/hw/i386/ioapic_internal.h |   7 ++
 include/hw/i386/pc.h  |   5 +
 include/monitor/hmp-target.h  |   3 +
 include/monitor/monitor.h |   1 +
 monitor.c |   5 +-
 target-i386/cpu.h |   3 +
 target-i386/helper.c  | 190 ++
 target-i386/monitor.c |  17 
 15 files changed, 407 insertions(+), 43 deletions(-)

-- 
2.1.4

[Qemu-devel] [PATCH 9/9] hmp: implemented io apic dump state for TCG

[Denis V. Lunev]

From: Pavel Butsykin 

Added support emulator for the hmp command "info ioapic"

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 hw/intc/ioapic.c  | 12 
 include/hw/i386/pc.h  |  1 +
 target-i386/monitor.c |  2 ++
 3 files changed, 15 insertions(+)

diff --git a/hw/intc/ioapic.c b/hw/intc/ioapic.c
index b527932..8c3aeae 100644
--- a/hw/intc/ioapic.c
+++ b/hw/intc/ioapic.c
@@ -20,6 +20,7 @@
  * License along with this library; if not, see .
  */
 
+#include "monitor/monitor.h"
 #include "hw/hw.h"
 #include "hw/i386/pc.h"
 #include "hw/i386/ioapic.h"
@@ -137,6 +138,17 @@ void ioapic_eoi_broadcast(int vector)
 }
 }
 
+void ioapic_dump_state(Monitor *mon, const QDict *qdict)
+{
+int i;
+
+for (i = 0; i < MAX_IOAPICS; i++) {
+if (ioapics[i] != 0) {
+ioapic_print_redtbl(mon, ioapics[i]);
+}
+}
+}
+
 static uint64_t
 ioapic_mem_read(void *opaque, hwaddr addr, unsigned int size)
 {
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index 539cf64..7c9f3a5 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -126,6 +126,7 @@ void hmp_info_irq(Monitor *mon, const QDict *qdict);
 /* ioapic.c */
 
 void kvm_ioapic_dump_state(Monitor *mon, const QDict *qdict);
+void ioapic_dump_state(Monitor *mon, const QDict *qdict);
 
 /* Global System Interrupts */
 
diff --git a/target-i386/monitor.c b/target-i386/monitor.c
index 829fff9..aac6b1b 100644
--- a/target-i386/monitor.c
+++ b/target-i386/monitor.c
@@ -505,5 +505,7 @@ void hmp_info_io_apic(Monitor *mon, const QDict *qdict)
 {
 if (kvm_irqchip_in_kernel()) {
 kvm_ioapic_dump_state(mon, qdict);
+} else {
+ioapic_dump_state(mon, qdict);
 }
 }
-- 
2.1.4

[Qemu-devel] [PATCH 5/9] monitor: make monitor_fprintf and mon_get_cpu externally visible

[Denis V. Lunev]

From: Pavel Butsykin 

monitor_fprintf and mon_get_cpu will be used in the target-specific monitor,
so it is advisable to make it external.

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 disas.c  | 10 --
 include/monitor/hmp-target.h |  1 +
 include/monitor/monitor.h|  1 +
 monitor.c|  5 ++---
 4 files changed, 4 insertions(+), 13 deletions(-)

diff --git a/disas.c b/disas.c
index 0ae70c2..45878fa 100644
--- a/disas.c
+++ b/disas.c
@@ -392,16 +392,6 @@ monitor_read_memory (bfd_vma memaddr, bfd_byte *myaddr, 
int length,
 return 0;
 }
 
-static int GCC_FMT_ATTR(2, 3)
-monitor_fprintf(FILE *stream, const char *fmt, ...)
-{
-va_list ap;
-va_start(ap, fmt);
-monitor_vprintf((Monitor *)stream, fmt, ap);
-va_end(ap);
-return 0;
-}
-
 /* Disassembler for the monitor.
See target_disas for a description of flags. */
 void monitor_disas(Monitor *mon, CPUState *cpu,
diff --git a/include/monitor/hmp-target.h b/include/monitor/hmp-target.h
index 611541d..c64f523 100644
--- a/include/monitor/hmp-target.h
+++ b/include/monitor/hmp-target.h
@@ -37,6 +37,7 @@ struct MonitorDef {
 const MonitorDef *target_monitor_defs(void);
 
 CPUArchState *mon_get_cpu_env(void);
+CPUState *mon_get_cpu(void);
 
 void hmp_info_mem(Monitor *mon, const QDict *qdict);
 void hmp_info_tlb(Monitor *mon, const QDict *qdict);
diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 9aff47e..f95a384 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -34,6 +34,7 @@ int monitor_fd_param(Monitor *mon, const char *fdname, Error 
**errp);
 void monitor_vprintf(Monitor *mon, const char *fmt, va_list ap)
 GCC_FMT_ATTR(2, 0);
 void monitor_printf(Monitor *mon, const char *fmt, ...) GCC_FMT_ATTR(2, 3);
+int monitor_fprintf(FILE *stream, const char *fmt, ...) GCC_FMT_ATTR(2, 3);
 void monitor_flush(Monitor *mon);
 int monitor_set_cpu(int cpu_index);
 int monitor_get_cpu_index(void);
diff --git a/monitor.c b/monitor.c
index 1f43263..d303579 100644
--- a/monitor.c
+++ b/monitor.c
@@ -371,8 +371,7 @@ void monitor_printf(Monitor *mon, const char *fmt, ...)
 va_end(ap);
 }
 
-static int GCC_FMT_ATTR(2, 3) monitor_fprintf(FILE *stream,
-  const char *fmt, ...)
+int monitor_fprintf(FILE *stream, const char *fmt, ...)
 {
 va_list ap;
 va_start(ap, fmt);
@@ -941,7 +940,7 @@ int monitor_set_cpu(int cpu_index)
 return 0;
 }
 
-static CPUState *mon_get_cpu(void)
+CPUState *mon_get_cpu(void)
 {
 if (!cur_mon->mon_cpu) {
 monitor_set_cpu(0);
-- 
2.1.4

Re: [Qemu-devel] [FIRST-PATCH] updated the negation STREQ to STRNEQ

[Michal Privoznik]

[You've got the wrong list. This is a libvirt patch and as such it
should have been sent to the libvir-l...@redhat.com]

On 19.09.2015 16:59, Lasya wrote:
> Hi,
> This is my first patch in QEMU. I have solved a bite-sized bug. I have 
> updated the !STREQ with STRNEQ in the following files. Thanks.
> 
> ---
>  src/bhyve/bhyve_driver.c  |  2 +-
>  src/conf/network_conf.c   |  4 ++--
>  src/conf/nwfilter_conf.c  |  2 +-
>  src/conf/nwfilter_params.c|  2 +-
>  src/conf/storage_conf.c   |  2 +-
>  src/lxc/lxc_fuse.c|  4 ++--
>  src/openvz/openvz_driver.c|  2 +-
>  src/qemu/qemu_command.c   |  6 +++---
>  src/qemu/qemu_domain.c|  2 +-
>  src/qemu/qemu_hotplug.c   |  2 +-
>  src/security/security_manager.c   |  2 +-
>  src/security/security_selinux.c   | 12 ++--
>  src/storage/storage_backend_logical.c |  2 +-
>  src/util/virfile.c|  2 +-
>  src/util/virsystemd.c |  2 +-
>  src/vz/vz_driver.c|  2 +-
>  src/vz/vz_sdk.c   |  2 +-
>  src/xen/xend_internal.c   |  2 +-
>  src/xenconfig/xen_sxpr.c  |  2 +-
>  tests/commandtest.c   | 10 +-
>  tests/securityselinuxlabeltest.c  |  2 +-
>  tests/virauthconfigtest.c |  2 +-
>  tests/virbitmaptest.c | 12 ++--
>  tests/vircgrouptest.c |  2 +-
>  tests/virkeyfiletest.c|  8 
>  tests/virnetsockettest.c  |  2 +-
>  tests/virtypedparamtest.c |  2 +-
>  tests/viruritest.c| 16 
>  28 files changed, 56 insertions(+), 56 deletions(-)

Okay, you've got all the occurrences. However, the syntax-check rule is
missing. You may want to look at cfg.mk for some examples. syntax-check
is our make target that checks the code if it complies with our coding
standards. After this patch we want a rule that forbids !STREQ or !STRNEQ.

Otherwise looking very good. Looking forward to v2.

Michal

Re: [Qemu-devel] [PATCH 0/8] target-i386: Implement debug extensions

[Paolo Bonzini]

On 15/09/2015 20:45, Richard Henderson wrote:
> Best guess, since I can't find any code that actually uses them.
> Linux actively turns them off at boot...

I've sent a kvm-unit-tests patch to test debug extensions.  It shows
that debug extensions work, but the following needs to be squashed in
patch 4:

diff --git a/target-i386/bpt_helper.c b/target-i386/bpt_helper.c
index c258598..b24e446 100644
--- a/target-i386/bpt_helper.c
+++ b/target-i386/bpt_helper.c
@@ -134,14 +134,14 @@ void cpu_x86_update_dr7(CPUX86State *env, uint32_t 
new_dr7)
 int mod = ((old_dr7 | old_dr7 * 2) ^ (new_dr7 | new_dr7 * 2)) & 0xff;
 
 for (i = 0; i < DR7_MAX_BP; i++) {
-if (mod & (2 << i * 2)) {
-/* We know that register i has changed enable state;
-   recheck what that state should be and apply.  */
-if (hw_breakpoint_enabled(new_dr7, i)) {
-iobpt |= hw_breakpoint_insert(env, i);
-} else {
-hw_breakpoint_remove(env, i);
-}
+if ((mod & (2 << i * 2)) && !hw_breakpoint_enabled(new_dr7, i)) {
+hw_breakpoint_remove(env, i);
+}
+}
+env->dr[7] = new_dr7 | DR7_FIXED_1;
+for (i = 0; i < DR7_MAX_BP; i++) {
+if (mod & (2 << i * 2) && hw_breakpoint_enabled(new_dr7, i)) {
+iobpt |= hw_breakpoint_insert(env, i);
 } else if (hw_breakpoint_type(new_dr7, i) == DR7_TYPE_IO_RW
&& hw_breakpoint_enabled(new_dr7, i)) {
 iobpt |= HF_IOBPT_MASK;

Otherwise, hw_breakpoint_insert doesn't work because it expects to
see an updated env->dr[7].

There are a couple other issues that the tests expose, but they are
not regressions so I will send patches later.

Paolo

[Qemu-devel] [PATCH 1/5] ide/atapi: make PIO read requests async

[Peter Lieven]

PIO read requests on the ATAPI interface used to be sync blk requests.
This has to siginificant drawbacks. First the main loop hangs util an
I/O request is completed and secondly if the I/O request does not
complete (e.g. due to an unresponsive storage) Qemu hangs completely.

Signed-off-by: Peter Lieven 
---
 hw/ide/atapi.c | 69 --
 1 file changed, 43 insertions(+), 26 deletions(-)

diff --git a/hw/ide/atapi.c b/hw/ide/atapi.c
index 747f466..9257e1c 100644
--- a/hw/ide/atapi.c
+++ b/hw/ide/atapi.c
@@ -105,31 +105,51 @@ static void cd_data_to_raw(uint8_t *buf, int lba)
 memset(buf, 0, 288);
 }
 
-static int cd_read_sector(IDEState *s, int lba, uint8_t *buf, int sector_size)
+static void cd_read_sector_cb(void *opaque, int ret)
 {
-int ret;
+IDEState *s = opaque;
 
-switch(sector_size) {
-case 2048:
-block_acct_start(blk_get_stats(s->blk), >acct,
- 4 * BDRV_SECTOR_SIZE, BLOCK_ACCT_READ);
-ret = blk_read(s->blk, (int64_t)lba << 2, buf, 4);
-block_acct_done(blk_get_stats(s->blk), >acct);
-break;
-case 2352:
-block_acct_start(blk_get_stats(s->blk), >acct,
- 4 * BDRV_SECTOR_SIZE, BLOCK_ACCT_READ);
-ret = blk_read(s->blk, (int64_t)lba << 2, buf + 16, 4);
-block_acct_done(blk_get_stats(s->blk), >acct);
-if (ret < 0)
-return ret;
-cd_data_to_raw(buf, lba);
-break;
-default:
-ret = -EIO;
-break;
+block_acct_done(blk_get_stats(s->blk), >acct);
+
+if (ret < 0) {
+ide_atapi_io_error(s, ret);
+return;
+}
+
+if (s->cd_sector_size == 2352) {
+cd_data_to_raw(s->io_buffer, s->lba);
 }
-return ret;
+
+s->lba++;
+s->io_buffer_index = 0;
+s->status &= ~BUSY_STAT;
+
+ide_atapi_cmd_reply_end(s);
+}
+
+static int cd_read_sector(IDEState *s, int lba, void *buf, int sector_size)
+{
+if (sector_size != 2048 && sector_size != 2352) {
+return -EINVAL;
+}
+
+s->iov.iov_base = buf;
+if (sector_size == 2352) {
+buf += 4;
+}
+
+s->iov.iov_len = 4 * BDRV_SECTOR_SIZE;
+qemu_iovec_init_external(>qiov, >iov, 1);
+
+if (blk_aio_readv(s->blk, (int64_t)lba << 2, >qiov, 4,
+  cd_read_sector_cb, s) == NULL) {
+return -EIO;
+}
+
+block_acct_start(blk_get_stats(s->blk), >acct,
+ 4 * BDRV_SECTOR_SIZE, BLOCK_ACCT_READ);
+s->status |= BUSY_STAT;
+return 0;
 }
 
 void ide_atapi_cmd_ok(IDEState *s)
@@ -190,10 +210,8 @@ void ide_atapi_cmd_reply_end(IDEState *s)
 ret = cd_read_sector(s, s->lba, s->io_buffer, s->cd_sector_size);
 if (ret < 0) {
 ide_atapi_io_error(s, ret);
-return;
 }
-s->lba++;
-s->io_buffer_index = 0;
+return;
 }
 if (s->elementary_transfer_size > 0) {
 /* there are some data left to transmit in this elementary
@@ -275,7 +293,6 @@ static void ide_atapi_cmd_read_pio(IDEState *s, int lba, 
int nb_sectors,
 s->io_buffer_index = sector_size;
 s->cd_sector_size = sector_size;
 
-s->status = READY_STAT | SEEK_STAT;
 ide_atapi_cmd_reply_end(s);
 }
 
-- 
1.9.1

[Qemu-devel] [PATCH 6/9] hmp: added local apic dump state

[Denis V. Lunev]

From: Pavel Butsykin 

Added the hmp command to query local apic registers state, may be
usefull after guest crashes to understand IRQ routing in guest.

(qemu) info lapic
dumping local APIC state for CPU 0

LVT0 0x00010700 active-hi edge  masked  ExtINT (vec 0)
LVT1 0x0400 active-hi edge  NMI
LVTPC0x0001 active-hi edge  masked  Fixed  (vec 0)
LVTERR   0x00fe active-hi edge  Fixed  (vec 254)
LVTTHMR  0x0001 active-hi edge  masked  Fixed  (vec 0)
LVTT 0x00ef active-hi edge one-shot Fixed  (vec 239)
TimerDCR=0x3 (divide by 16) initial_count = 62278
SPIV 0x01ff APIC enabled, focus=off, spurious vec 255
ICR  0x00fd physical edge de-assert no
ICR2 0x0001 cpu 1 (X2APIC ID)
ESR  0x
ISR
IRR  239

APR 0x00 TPR 0x00 DFR 0x0f LDR 0x00 PPR 0x00

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 hmp-commands-info.hx |  16 
 include/monitor/hmp-target.h |   1 +
 target-i386/cpu.h|   3 +
 target-i386/helper.c | 190 +++
 target-i386/monitor.c|   6 ++
 5 files changed, 216 insertions(+)

diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index 9f5a158..dba7839 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -112,6 +112,22 @@ STEXI
 Show the cpu registers.
 ETEXI
 
+#if defined(TARGET_I386)
+{
+.name   = "lapic",
+.args_type  = "",
+.params = "",
+.help   = "show local apic state",
+.mhandler.cmd = hmp_info_local_apic,
+},
+#endif
+
+STEXI
+@item info lapic
+@findex lapic
+Show local APIC state
+ETEXI
+
 {
 .name   = "cpus",
 .args_type  = "",
diff --git a/include/monitor/hmp-target.h b/include/monitor/hmp-target.h
index c64f523..be50577 100644
--- a/include/monitor/hmp-target.h
+++ b/include/monitor/hmp-target.h
@@ -42,5 +42,6 @@ CPUState *mon_get_cpu(void);
 void hmp_info_mem(Monitor *mon, const QDict *qdict);
 void hmp_info_tlb(Monitor *mon, const QDict *qdict);
 void hmp_mce(Monitor *mon, const QDict *qdict);
+void hmp_info_local_apic(Monitor *mon, const QDict *qdict);
 
 #endif /* MONITOR_COMMON */
diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index 5231e8c..527eb99 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -1353,4 +1353,7 @@ void enable_compat_apic_id_mode(void);
 #define APIC_DEFAULT_ADDRESS 0xfee0
 #define APIC_SPACE_SIZE  0x10
 
+void x86_cpu_dump_local_apic_state(CPUState *cs, FILE *f,
+   fprintf_function cpu_fprintf, int flags);
+
 #endif /* CPU_I386_H */
diff --git a/target-i386/helper.c b/target-i386/helper.c
index 5480a96..ff9a0f2 100644
--- a/target-i386/helper.c
+++ b/target-i386/helper.c
@@ -23,6 +23,7 @@
 #ifndef CONFIG_USER_ONLY
 #include "sysemu/sysemu.h"
 #include "monitor/monitor.h"
+#include "hw/i386/apic_internal.h"
 #endif
 
 static void cpu_x86_version(CPUX86State *env, int *family, int *model)
@@ -177,6 +178,195 @@ done:
 cpu_fprintf(f, "\n");
 }
 
+#ifndef CONFIG_USER_ONLY
+
+/* ARRAY_SIZE check is not required because
+ * DeliveryMode(dm) has a size of 3 bit.
+ */
+static inline const char *dm2str(uint32_t dm)
+{
+static const char *str[] = {
+"Fixed",
+"...",
+"SMI",
+"...",
+"NMI",
+"INIT",
+"...",
+"ExtINT"
+};
+return str[dm];
+}
+
+static void dump_apic_lvt(FILE *f, fprintf_function cpu_fprintf,
+  const char *name, uint32_t lvt, bool is_timer)
+{
+uint32_t dm = (lvt & APIC_LVT_DELIV_MOD) >> APIC_LVT_DELIV_MOD_SHIFT;
+cpu_fprintf(f,
+"%s\t 0x%08x %s %-5s %-6s %-7s %-12s %-6s",
+name, lvt,
+lvt & APIC_LVT_INT_POLARITY ? "active-lo" : "active-hi",
+lvt & APIC_LVT_LEVEL_TRIGGER ? "level" : "edge",
+lvt & APIC_LVT_MASKED ? "masked" : "",
+lvt & APIC_LVT_DELIV_STS ? "pending" : "",
+!is_timer ?
+"" : lvt & APIC_LVT_TIMER_PERIODIC ?
+"periodic" : lvt & APIC_LVT_TIMER_TSCDEADLINE ?
+"tsc-deadline" : "one-shot",
+dm2str(dm));
+if (dm != APIC_DM_NMI) {
+cpu_fprintf(f, " (vec %u)\n", lvt & APIC_VECTOR_MASK);
+} else {
+cpu_fprintf(f, "\n");
+}
+}
+
+/* ARRAY_SIZE check is not required because
+ * destination shorthand has a size of 2 bit.
+ */
+static inline const char *shorthand2str(uint32_t shorthand)
+{
+const char *str[] = {
+"no", "self", "all-self", "all"
+};
+return str[shorthand];
+}
+
+static inline 

[Qemu-devel] [PATCH 8/9] hmp: added io apic dump state

[Denis V. Lunev]

From: Pavel Butsykin 

Added the hmp command to query io apic state, may be usefull after guest
crashes to understand IRQ routing in guest.

Implementation is only for kvm here. The dump will look like
(qemu) info ioapic
ioapic id=0x00 sel=0x2e (redir[15])
pin 0  0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
pin 1  0x0031 dest=0 vec=49  active-hi edge fixed  physical
...
pin 23 0x0001 dest=0 vec=0   active-hi edge  masked fixed  physical
IRR (none)

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 hmp-commands-info.hx  | 16 
 hw/i386/kvm/ioapic.c  | 10 ++
 hw/intc/ioapic_common.c   | 37 +
 include/hw/i386/ioapic_internal.h |  2 ++
 include/hw/i386/pc.h  |  4 
 include/monitor/hmp-target.h  |  1 +
 target-i386/monitor.c |  9 +
 7 files changed, 79 insertions(+)

diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index dba7839..b346fd1 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -128,6 +128,22 @@ STEXI
 Show local APIC state
 ETEXI
 
+#if defined(TARGET_I386)
+{
+.name   = "ioapic",
+.args_type  = "",
+.params = "",
+.help   = "show io apic state",
+.mhandler.cmd = hmp_info_io_apic,
+},
+#endif
+
+STEXI
+@item info ioapic
+@findex ioapic
+Show io APIC state
+ETEXI
+
 {
 .name   = "cpus",
 .args_type  = "",
diff --git a/hw/i386/kvm/ioapic.c b/hw/i386/kvm/ioapic.c
index d2a6c4c..b7390ca 100644
--- a/hw/i386/kvm/ioapic.c
+++ b/hw/i386/kvm/ioapic.c
@@ -10,6 +10,7 @@
  * See the COPYING file in the top-level directory.
  */
 
+#include "monitor/monitor.h"
 #include "hw/i386/pc.h"
 #include "hw/i386/ioapic_internal.h"
 #include "hw/i386/apic_internal.h"
@@ -110,6 +111,15 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
 }
 }
 
+void kvm_ioapic_dump_state(Monitor *mon, const QDict *qdict)
+{
+IOAPICCommonState s;
+
+kvm_ioapic_get();
+
+ioapic_print_redtbl(mon, );
+}
+
 static void kvm_ioapic_reset(DeviceState *dev)
 {
 IOAPICCommonState *s = IOAPIC_COMMON(dev);
diff --git a/hw/intc/ioapic_common.c b/hw/intc/ioapic_common.c
index 8b7d118..762cfc9 100644
--- a/hw/intc/ioapic_common.c
+++ b/hw/intc/ioapic_common.c
@@ -19,6 +19,7 @@
  * License along with this library; if not, see .
  */
 
+#include "monitor/monitor.h"
 #include "hw/i386/ioapic.h"
 #include "hw/i386/ioapic_internal.h"
 #include "hw/sysbus.h"
@@ -31,6 +32,42 @@
  */
 int ioapic_no;
 
+void ioapic_print_redtbl(Monitor *mon, IOAPICCommonState *s)
+{
+static const char *delm_str[] = {
+"fixed", "lowest", "SMI", "...", "NMI", "INIT", "...", "extINT"};
+int i;
+
+monitor_printf(mon, "ioapic id=0x%02x sel=0x%02x", s->id, s->ioregsel);
+if (s->ioregsel) {
+monitor_printf(mon, " (redir[%u])\n",
+   (s->ioregsel - IOAPIC_REG_REDTBL_BASE) >> 1);
+} else {
+monitor_printf(mon, "\n");
+}
+for (i = 0; i < IOAPIC_NUM_PINS; i++) {
+uint64_t entry = s->ioredtbl[i];
+uint32_t delm = (uint32_t)((entry & IOAPIC_LVT_DELIV_MODE) >>
+   IOAPIC_LVT_DELIV_MODE_SHIFT);
+monitor_printf(mon, "pin %-2u 0x%016jx dest=%jx vec=%-3lu "
+   "%s %-5s %-6s %-6s %s\n",
+   i, entry,
+   (entry >> IOAPIC_LVT_DEST_SHIFT) &
+(entry & IOAPIC_LVT_DEST_MODE ? 0xff : 0xf),
+   entry & IOAPIC_VECTOR_MASK,
+   entry & IOAPIC_LVT_POLARITY ? "active-lo" : "active-hi",
+   entry & IOAPIC_LVT_TRIGGER_MODE ? "level" : "edge",
+   entry & IOAPIC_LVT_MASKED ? "masked" : "",
+   delm_str[delm],
+   entry & IOAPIC_LVT_DEST_MODE ? "logical" : "physical");
+}
+if (s->irr == 0) {
+monitor_printf(mon, "IRR (none)\n");
+} else {
+monitor_printf(mon, "IRR %08x\n", s->irr);
+}
+}
+
 void ioapic_reset_common(DeviceState *dev)
 {
 IOAPICCommonState *s = IOAPIC_COMMON(dev);
diff --git a/include/hw/i386/ioapic_internal.h 
b/include/hw/i386/ioapic_internal.h
index 4f7764e..797ed47 100644
--- a/include/hw/i386/ioapic_internal.h
+++ b/include/hw/i386/ioapic_internal.h
@@ -105,4 +105,6 @@ struct IOAPICCommonState {
 
 void ioapic_reset_common(DeviceState *dev);
 
+void ioapic_print_redtbl(Monitor *mon, IOAPICCommonState *s);
+
 #endif /* !QEMU_IOAPIC_INTERNAL_H */
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index 3e002c9..539cf64 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -123,6 +123,10 @@ int 

[Qemu-devel] [PATCH 7/9] ioapic_internal.h: added more constants

[Denis V. Lunev]

From: Pavel Butsykin 

Added the masks for easy  access to fields of the redirection table entry

Signed-off-by: Pavel Butsykin 
Signed-off-by: Denis V. Lunev 
CC: Paolo Bonzini 
CC: Andreas Färber 
---
 include/hw/i386/ioapic_internal.h | 5 +
 1 file changed, 5 insertions(+)

diff --git a/include/hw/i386/ioapic_internal.h 
b/include/hw/i386/ioapic_internal.h
index 3be3352..4f7764e 100644
--- a/include/hw/i386/ioapic_internal.h
+++ b/include/hw/i386/ioapic_internal.h
@@ -40,7 +40,12 @@
 #define IOAPIC_LVT_DELIV_MODE_SHIFT 8
 
 #define IOAPIC_LVT_MASKED   (1 << IOAPIC_LVT_MASKED_SHIFT)
+#define IOAPIC_LVT_TRIGGER_MODE (1 << IOAPIC_LVT_TRIGGER_MODE_SHIFT)
 #define IOAPIC_LVT_REMOTE_IRR   (1 << IOAPIC_LVT_REMOTE_IRR_SHIFT)
+#define IOAPIC_LVT_POLARITY (1 << IOAPIC_LVT_POLARITY_SHIFT)
+#define IOAPIC_LVT_DELIV_STATUS (1 << IOAPIC_LVT_DELIV_STATUS_SHIFT)
+#define IOAPIC_LVT_DEST_MODE(1 << IOAPIC_LVT_DEST_MODE_SHIFT)
+#define IOAPIC_LVT_DELIV_MODE   (7 << IOAPIC_LVT_DELIV_MODE_SHIFT)
 
 #define IOAPIC_TRIGGER_EDGE 0
 #define IOAPIC_TRIGGER_LEVEL1
-- 
2.1.4

Re: [Qemu-devel] [FIRST-PATCH] updated the negation STREQ to STRNEQ

[Michal Privoznik]

On 21.09.2015 13:05, Michal Privoznik wrote:
> [You've got the wrong list. This is a libvirt patch and as such it
> should have been sent to the libvir-l...@redhat.com]
> 
> On 19.09.2015 16:59, Lasya wrote:
>> Hi,
>> This is my first patch in QEMU. I have solved a bite-sized bug. I have 
>> updated the !STREQ with STRNEQ in the following files. Thanks.

Oh, and the commit message is rather unhelpful. I mean, we tend to
explain in there why is the commit needed, what does it fix, and so on.
If you look at git history you'll get the idea.

>>
>> ---
>>  src/bhyve/bhyve_driver.c  |  2 +-
>>  src/conf/network_conf.c   |  4 ++--
>>  src/conf/nwfilter_conf.c  |  2 +-
>>  src/conf/nwfilter_params.c|  2 +-
>>  src/conf/storage_conf.c   |  2 +-
>>  src/lxc/lxc_fuse.c|  4 ++--
>>  src/openvz/openvz_driver.c|  2 +-
>>  src/qemu/qemu_command.c   |  6 +++---
>>  src/qemu/qemu_domain.c|  2 +-
>>  src/qemu/qemu_hotplug.c   |  2 +-
>>  src/security/security_manager.c   |  2 +-
>>  src/security/security_selinux.c   | 12 ++--
>>  src/storage/storage_backend_logical.c |  2 +-
>>  src/util/virfile.c|  2 +-
>>  src/util/virsystemd.c |  2 +-
>>  src/vz/vz_driver.c|  2 +-
>>  src/vz/vz_sdk.c   |  2 +-
>>  src/xen/xend_internal.c   |  2 +-
>>  src/xenconfig/xen_sxpr.c  |  2 +-
>>  tests/commandtest.c   | 10 +-
>>  tests/securityselinuxlabeltest.c  |  2 +-
>>  tests/virauthconfigtest.c |  2 +-
>>  tests/virbitmaptest.c | 12 ++--
>>  tests/vircgrouptest.c |  2 +-
>>  tests/virkeyfiletest.c|  8 
>>  tests/virnetsockettest.c  |  2 +-
>>  tests/virtypedparamtest.c |  2 +-
>>  tests/viruritest.c| 16 
>>  28 files changed, 56 insertions(+), 56 deletions(-)
> 
> Okay, you've got all the occurrences. However, the syntax-check rule is
> missing. You may want to look at cfg.mk for some examples. syntax-check
> is our make target that checks the code if it complies with our coding
> standards. After this patch we want a rule that forbids !STREQ or !STRNEQ.
> 
> Otherwise looking very good. Looking forward to v2.

Michal

Re: [Qemu-devel] RFC: virtio-peer shared memory based peer communication device

[Paolo Bonzini]

On 21/09/2015 12:47, Jan Kiszka wrote:
>> > Apart from the windows idea, how does virtio-peer compare to virtio-rpmsg?
> rpmsg is a very specialized thing. It targets single AMP cores, assuming
> that those have full access to the main memory.

Yes, this is why I did say "apart from the windows idea".

> And it is also a
> centralized approach where all message go through the main Linux
> instance. I suspect we could cover that use case as well with generic
> inter-vm shared memory device, but I didn't think about all details yet.

The virtqueue handling seems very similar between the two.  However, the
messages for rpmsg however have a small header (struct rpmsg_hdr in
include/linux/rpmsg.h) and there is a weird feature bit VIRTIO_RPMSG_F_NS.

So I guess virtio-rpmsg and virtio-peer are about as similar as
virtio-serial and virtio-peer.

Paolo

Re: [Qemu-devel] [PATCH] pc: memhotplug: rise minimum DIMM addr/size alignment to 128Mb

[Igor Mammedov]

On Mon, 21 Sep 2015 14:22:23 +0200
Paolo Bonzini  wrote:

> 
> 
> On 21/09/2015 13:50, Igor Mammedov wrote:
> > it's attempt to workaround virtio bug reported earlier:
> > http://lists.nongnu.org/archive/html/qemu-devel/2015-08/msg00522.html
> > where virtio can't handle buffer that crosses border
> > between 2 DIMM's (i.e. 2 MemoryRegions).
> > 
> > Testing showed that virtio doesn't hit above bug
> > with 128Mb DIMM's granularity. Also linux memory
> > hotplug can handle hotplugged memory starting with
> > 128Mb memory sections so lets rise minimum size limit
> > to 128Mb and align starting DIMM address on 128Mb.
> > 
> > It's certainly not the fix but it reduces risk of
> > crashing VM till virtio is fixed.
> > It also could be improved in guest's virtio side if it
> > would align buffers on 128Mb border and limit max  buffer
> > size to the same value.
> > 
> > Signed-off-by: Igor Mammedov 
> 
> This seems to be easily handled at a level above QEMU---and the fix
> would be available to older machine types as well.  This patch would
> also make it quite a bit harder to test the real fix with QEMU. It is
> not alone a reason to NACK it but should also be kept in mind.
> 
> Aligning to 4K makes some sense, since 4K is the page size, but
> enforcing an arbitrary alignment above 4K is policy that does not belong
> in QEMU.
>
> To some extend, enforcing natural alignment would be okay as a
> workaround for the virtio bug as well.  It would also make it easier to
> ensure that hotplugged hugetlbfs-backed memory can use hugepages in the
> guest.  Does it make sense to you?
in current machine types we already enforce backend-s address/size alignment,
which is file's page size for hugetlbfs-backed memory and 2Mb for RAM backend.

So I guess we could try to apply workaround to virtio on guest side,
aligning and limiting max buffer size to 2Mb, it should work for 'old'
machine types as well.

> 
> Paolo
> 
> > ---
> > Based on PCI tree as it has patches that add
> > 2.5 machine type.
> > ---
> >  hw/i386/pc.c |  8 +---
> >  hw/i386/pc_piix.c| 12 ++--
> >  hw/i386/pc_q35.c | 12 ++--
> >  include/hw/i386/pc.h |  5 ++---
> >  4 files changed, 27 insertions(+), 10 deletions(-)
> > 
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index b5107f7..ddb6710 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -1645,8 +1645,9 @@ static void pc_dimm_plug(HotplugHandler *hotplug_dev,
> >  MemoryRegion *mr = ddc->get_memory_region(dimm);
> >  uint64_t align = TARGET_PAGE_SIZE;
> >  
> > -if (memory_region_get_alignment(mr) && pcms->enforce_aligned_dimm) {
> > -align = memory_region_get_alignment(mr);
> > +if (pcms->enforce_aligned_dimm) {
> > +align = MAX(memory_region_get_alignment(mr),
> > +pcms->enforce_aligned_dimm);
> >  }
> >  
> >  if (!pcms->acpi_dev) {
> > @@ -1936,7 +1937,8 @@ static void pc_machine_initfn(Object *obj)
> >  "Enable vmport (pc & q35)",
> >  _abort);
> >  
> > -pcms->enforce_aligned_dimm = true;
> > +/* align DIMM starting address/size by 128Mb */
> > +pcms->enforce_aligned_dimm = 1ULL << 27;
> >  object_property_add_bool(obj, PC_MACHINE_ENFORCE_ALIGNED_DIMM,
> >   pc_machine_get_aligned_dimm,
> >   NULL, _abort);
> > diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
> > index caa4edc..7671905 100644
> > --- a/hw/i386/pc_piix.c
> > +++ b/hw/i386/pc_piix.c
> > @@ -301,9 +301,17 @@ static void pc_init1(MachineState *machine,
> >  }
> >  }
> >  
> > +static void pc_compat_2_4(MachineState *machine)
> > +{
> > +PCMachineState *pcms = PC_MACHINE(machine);
> > +
> > +pcms->enforce_aligned_dimm = TARGET_PAGE_SIZE;
> > +}
> > +
> >  static void pc_compat_2_3(MachineState *machine)
> >  {
> >  PCMachineState *pcms = PC_MACHINE(machine);
> > +pc_compat_2_4(machine);
> >  savevm_skip_section_footers();
> >  if (kvm_enabled()) {
> >  pcms->smm = ON_OFF_AUTO_OFF;
> > @@ -326,7 +334,7 @@ static void pc_compat_2_1(MachineState *machine)
> >  pc_compat_2_2(machine);
> >  smbios_uuid_encoded = false;
> >  x86_cpu_compat_kvm_no_autodisable(FEAT_8000_0001_ECX, CPUID_EXT3_SVM);
> > -pcms->enforce_aligned_dimm = false;
> > +pcms->enforce_aligned_dimm = 0;
> >  }
> >  
> >  static void pc_compat_2_0(MachineState *machine)
> > @@ -485,7 +493,7 @@ static void pc_i440fx_2_4_machine_options(MachineClass 
> > *m)
> >  SET_MACHINE_COMPAT(m, PC_COMPAT_2_4);
> >  }
> >  
> > -DEFINE_I440FX_MACHINE(v2_4, "pc-i440fx-2.4", NULL,
> > +DEFINE_I440FX_MACHINE(v2_4, "pc-i440fx-2.4", pc_compat_2_3,
> >pc_i440fx_2_4_machine_options)
> >  
> >  
> > diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
> > index 506b6bf..72b479f 100644
> > --- a/hw/i386/pc_q35.c
> > +++ 

Re: [Qemu-devel] [PATCH] pc: memhotplug: rise minimum DIMM addr/size alignment to 128Mb

[Paolo Bonzini]

On 21/09/2015 15:05, Igor Mammedov wrote:
>> > To some extend, enforcing natural alignment would be okay as a
>> > workaround for the virtio bug as well.  It would also make it easier to
>> > ensure that hotplugged hugetlbfs-backed memory can use hugepages in the
>> > guest.  Does it make sense to you?
> in current machine types we already enforce backend-s address/size alignment,
> which is file's page size for hugetlbfs-backed memory and 2Mb for RAM backend.

Right, but it's not enough if the guest's physical address is not
aligned to 2Mb/1Gb too.  This is why we changed i440FX and q35 to have
only 3 and 2 gigabytes of low memory (down from 3.5 and 2 IIRC).

> So I guess we could try to apply workaround to virtio on guest side,
> aligning and limiting max buffer size to 2Mb, it should work for 'old'
> machine types as well.

That would make sense and it would be complementary to natural alignment
of DIMMs in the host.  This would give:

hostguest
old old fails
old new works (virtio workaround)
new old works (natural alignment)
new new works (choose your favorite workaround)

Paolo

[Qemu-devel] [PATCH 5/5] block/nfs: cache allocated filesize for read-only files

[Peter Lieven]

If the file is readonly its not expected to grow so
save the blocking call to nfs_fstat_async and use
the value saved at connection time. Also important
the monitor (and thus the main loop) will not hang
if block device info is queried and the NFS share
is unresponsive.

Signed-off-by: Peter Lieven 
Reviewed-by: Max Reitz 
Reviewed-by: Jeff Cody 
---
 block/nfs.c | 36 
 1 file changed, 36 insertions(+)

diff --git a/block/nfs.c b/block/nfs.c
index c026ff6..5ffd19f 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -43,6 +43,7 @@ typedef struct NFSClient {
 int events;
 bool has_zero_init;
 AioContext *aio_context;
+blkcnt_t st_blocks;
 } NFSClient;
 
 typedef struct NFSRPC {
@@ -374,6 +375,7 @@ static int64_t nfs_client_open(NFSClient *client, const 
char *filename,
 }
 
 ret = DIV_ROUND_UP(st.st_size, BDRV_SECTOR_SIZE);
+client->st_blocks = st.st_blocks;
 client->has_zero_init = S_ISREG(st.st_mode);
 goto out;
 fail:
@@ -464,6 +466,11 @@ static int64_t 
nfs_get_allocated_file_size(BlockDriverState *bs)
 NFSRPC task = {0};
 struct stat st;
 
+if (bdrv_is_read_only(bs) &&
+!(bs->open_flags & BDRV_O_NOCACHE)) {
+return client->st_blocks * 512;
+}
+
 task.st = 
 if (nfs_fstat_async(client->context, client->fh, nfs_co_generic_cb,
 ) != 0) {
@@ -484,6 +491,34 @@ static int nfs_file_truncate(BlockDriverState *bs, int64_t 
offset)
 return nfs_ftruncate(client->context, client->fh, offset);
 }
 
+/* Note that this will not re-establish a connection with the NFS server
+ * - it is effectively a NOP.  */
+static int nfs_reopen_prepare(BDRVReopenState *state,
+  BlockReopenQueue *queue, Error **errp)
+{
+NFSClient *client = state->bs->opaque;
+struct stat st;
+int ret = 0;
+
+if (state->flags & BDRV_O_RDWR && bdrv_is_read_only(state->bs)) {
+error_setg(errp, "Cannot open a read-only mount as read-write");
+return -EACCES;
+}
+
+/* Update cache for read-only reopens */
+if (!(state->flags & BDRV_O_RDWR)) {
+ret = nfs_fstat(client->context, client->fh, );
+if (ret < 0) {
+error_setg(errp, "Failed to fstat file: %s",
+   nfs_get_error(client->context));
+return ret;
+}
+client->st_blocks = st.st_blocks;
+}
+
+return 0;
+}
+
 static BlockDriver bdrv_nfs = {
 .format_name= "nfs",
 .protocol_name  = "nfs",
@@ -499,6 +534,7 @@ static BlockDriver bdrv_nfs = {
 .bdrv_file_open = nfs_file_open,
 .bdrv_close = nfs_file_close,
 .bdrv_create= nfs_file_create,
+.bdrv_reopen_prepare= nfs_reopen_prepare,
 
 .bdrv_co_readv  = nfs_co_readv,
 .bdrv_co_writev = nfs_co_writev,
-- 
1.9.1

Re: [Qemu-devel] [PATCH v2 0/2] Dynamic module support for block drivers

[Marc Marí]

Ping!

Anyone has more comments for the next version?

Thanks
Marc

>On Tue,  8 Sep 2015 15:53:20 +0200
>Marc Marí  wrote:
>
> The current module infrastructure has been improved to enable dynamic
> module loading.
> 
> This reduces the load time for very simple guests. For the following
> configuration (very loaded)
> 
> ./configure --enable-sdl --enable-gtk --enable-vte --enable-curses \
> --enable-vnc --enable-vnc-{jpeg,tls,sasl,png} --enable-virtfs \
> --enable-brlapi --enable-curl --enable-fdt --enable-bluez \
> --enable-kvm --enable-rdma --enable-uuid --enable-vde \
> --enable-linux-aio --enable-cap-ng --enable-attr
> --enable-vhost-net \ --enable-vhost-scsi --enable-spice --enable-rbd
> --enable-libiscsi \ --enable-smartcard-nss --enable-guest-agent
> --enable-libusb \ --enable-usb-redir --enable-lzo --enable-snappy
> --enable-bzip2 \ --enable-seccomp --enable-coroutine-pool
> --enable-glusterfs \ --enable-tpm --enable-libssh2 --enable-vhdx
> --enable-numa \ --enable-tcmalloc --target-list=x86_64-softmmu
> 
> With modules disabled, there are 142 libraries loaded at startup.
> Time is the following:
>  LD time: 0.065 seconds
>  QEMU time: 0.02 seconds
>  Total time: 0.085 seconds
> 
> With this patch series and modules enabled, there are 128 libraries
> loaded at startup. Time is the following:
>  LD time: 0.02 seconds
>  QEMU time: 0.02 seconds
>  Total time: 0.04 seconds
> 
> Where LD time is the time between the program startup and the jump to
> main, and QEMU time is the time between the start of main and the
> first kvm_entry.
> 
> These results are just with a few block drivers, that were already a
> module. Adding more modules (block or not block) should be easy, and
> will reduce the load time even more.
> 
> Marc Marí (2):
>   Add dynamic module loading for block drivers
>   Add dynamic generation of module_block.h
> 
>  .gitignore  |   1 +
>  Makefile|  10 ++-
>  block.c |  70 +
>  configure   |   2 +-
>  include/qemu/module.h   |   3 +
>  scripts/modules/module_block.py | 134
> 
> util/module.c   |  38  7 files changed,
> 227 insertions(+), 31 deletions(-) create mode 100755
> scripts/modules/module_block.py
> 

Re: [Qemu-devel] [PATCH] pc: memhotplug: rise minimum DIMM addr/size alignment to 128Mb

[Igor Mammedov]

On Mon, 21 Sep 2015 15:13:17 +0200
Paolo Bonzini  wrote:

> 
> 
> On 21/09/2015 15:05, Igor Mammedov wrote:
> >> > To some extend, enforcing natural alignment would be okay as a
> >> > workaround for the virtio bug as well.  It would also make it easier to
> >> > ensure that hotplugged hugetlbfs-backed memory can use hugepages in the
> >> > guest.  Does it make sense to you?
> > in current machine types we already enforce backend-s address/size 
> > alignment,
> > which is file's page size for hugetlbfs-backed memory and 2Mb for RAM 
> > backend.
> 
> Right, but it's not enough if the guest's physical address is not
> aligned to 2Mb/1Gb too.  This is why we changed i440FX and q35 to have
> only 3 and 2 gigabytes of low memory (down from 3.5 and 2 IIRC).
DIMM's GPA is aligned to backend's alignment since 2.2,
it should be aligned 2Mb/1Gb depending on what hugetlbfs file is used
so that already works as expected

> 
> > So I guess we could try to apply workaround to virtio on guest side,
> > aligning and limiting max buffer size to 2Mb, it should work for 'old'
> > machine types as well.
> 
> That would make sense and it would be complementary to natural alignment
> of DIMMs in the host.  This would give:
> 
>   hostguest
>   old old fails
>   old new works (virtio workaround)
>   new old works (natural alignment)
+ not sure if it would work,
  I've though that virtio refactoring, that drops requirement
  for buffer to be inside of only one MemoryRegion, would 
  touch virtio in QEMU and on guest side too.

>   new new works (choose your favorite workaround)
> 
> Paolo

Re: [Qemu-devel] [PATCH] pc: memhotplug: rise minimum DIMM addr/size alignment to 128Mb

[Paolo Bonzini]

On 21/09/2015 15:32, Igor Mammedov wrote:
> On Mon, 21 Sep 2015 15:13:17 +0200
> Paolo Bonzini  wrote:
> 
>>
>>
>> On 21/09/2015 15:05, Igor Mammedov wrote:
> To some extend, enforcing natural alignment would be okay as a
> workaround for the virtio bug as well.  It would also make it easier to
> ensure that hotplugged hugetlbfs-backed memory can use hugepages in the
> guest.  Does it make sense to you?
>>> in current machine types we already enforce backend-s address/size 
>>> alignment,
>>> which is file's page size for hugetlbfs-backed memory and 2Mb for RAM 
>>> backend.
>>
>> Right, but it's not enough if the guest's physical address is not
>> aligned to 2Mb/1Gb too.  This is why we changed i440FX and q35 to have
>> only 3 and 2 gigabytes of low memory (down from 3.5 and 2 IIRC).
> 
> DIMM's GPA is aligned to backend's alignment since 2.2,
> it should be aligned 2Mb/1Gb depending on what hugetlbfs file is used
> so that already works as expected

Oh, ok.  This is what pcms->enforce_aligned_dimm does when true, and
this is also what I was missing.  Great!

>>> So I guess we could try to apply workaround to virtio on guest side,
>>> aligning and limiting max buffer size to 2Mb, it should work for 'old'
>>> machine types as well.
>>
>> That would make sense and it would be complementary to natural alignment
>> of DIMMs in the host.  This would give:
>>
>>  hostguest
>>  old old fails
>>  old new works (virtio workaround)
>>  new old works (natural alignment)
> + not sure if it would work,
>   I've though that virtio refactoring, that drops requirement
>   for buffer to be inside of only one MemoryRegion, would 
>   touch virtio in QEMU and on guest side too.

Right, it would theoretically not be enough.  However, I think it would
work in practice because hot-plugged DIMMs should be bigger than 128 MiB.

Paolo

Re: [Qemu-devel] [PATCH v2 0/2] Dynamic module support for block drivers

[Denis V. Lunev]

On 09/08/2015 04:53 PM, Marc Marí wrote:

The current module infrastructure has been improved to enable dynamic module
loading.

This reduces the load time for very simple guests. For the following
configuration (very loaded)

./configure --enable-sdl --enable-gtk --enable-vte --enable-curses \
 --enable-vnc --enable-vnc-{jpeg,tls,sasl,png} --enable-virtfs \
 --enable-brlapi --enable-curl --enable-fdt --enable-bluez \
 --enable-kvm --enable-rdma --enable-uuid --enable-vde \
 --enable-linux-aio --enable-cap-ng --enable-attr --enable-vhost-net \
 --enable-vhost-scsi --enable-spice --enable-rbd --enable-libiscsi \
 --enable-smartcard-nss --enable-guest-agent --enable-libusb \
 --enable-usb-redir --enable-lzo --enable-snappy --enable-bzip2 \
 --enable-seccomp --enable-coroutine-pool --enable-glusterfs \
 --enable-tpm --enable-libssh2 --enable-vhdx --enable-numa \
 --enable-tcmalloc --target-list=x86_64-softmmu

With modules disabled, there are 142 libraries loaded at startup. Time is
the following:
  LD time: 0.065 seconds
  QEMU time: 0.02 seconds
  Total time: 0.085 seconds

With this patch series and modules enabled, there are 128 libraries loaded
at startup. Time is the following:
  LD time: 0.02 seconds
  QEMU time: 0.02 seconds
  Total time: 0.04 seconds

Where LD time is the time between the program startup and the jump to main,
and QEMU time is the time between the start of main and the first kvm_entry.

These results are just with a few block drivers, that were already a module.
Adding more modules (block or not block) should be easy, and will reduce
the load time even more.

Marc Marí (2):
   Add dynamic module loading for block drivers
   Add dynamic generation of module_block.h

  .gitignore  |   1 +
  Makefile|  10 ++-
  block.c |  70 +
  configure   |   2 +-
  include/qemu/module.h   |   3 +
  scripts/modules/module_block.py | 134 
  util/module.c   |  38 
  7 files changed, 227 insertions(+), 31 deletions(-)
  create mode 100755 scripts/modules/module_block.py



From my point of view the design looks a bit complex.
The approach should be quite similar to one used
in Linux kernel.

If the block driver is configured as a module, block_init
should register proper hooks and create generic lists.
C code parsing does not look like a good approach
to me.

If block_init is a bad name, we could use something like
module_init.

Re: [Qemu-devel] [PATCH] pc: memhotplug: rise minimum DIMM addr/size alignment to 128Mb

[Paolo Bonzini]

On 21/09/2015 13:50, Igor Mammedov wrote:
> it's attempt to workaround virtio bug reported earlier:
> http://lists.nongnu.org/archive/html/qemu-devel/2015-08/msg00522.html
> where virtio can't handle buffer that crosses border
> between 2 DIMM's (i.e. 2 MemoryRegions).
> 
> Testing showed that virtio doesn't hit above bug
> with 128Mb DIMM's granularity. Also linux memory
> hotplug can handle hotplugged memory starting with
> 128Mb memory sections so lets rise minimum size limit
> to 128Mb and align starting DIMM address on 128Mb.
> 
> It's certainly not the fix but it reduces risk of
> crashing VM till virtio is fixed.
> It also could be improved in guest's virtio side if it
> would align buffers on 128Mb border and limit max  buffer
> size to the same value.
> 
> Signed-off-by: Igor Mammedov 

This seems to be easily handled at a level above QEMU---and the fix
would be available to older machine types as well.  This patch would
also make it quite a bit harder to test the real fix with QEMU. It is
not alone a reason to NACK it but should also be kept in mind.

Aligning to 4K makes some sense, since 4K is the page size, but
enforcing an arbitrary alignment above 4K is policy that does not belong
in QEMU.

To some extend, enforcing natural alignment would be okay as a
workaround for the virtio bug as well.  It would also make it easier to
ensure that hotplugged hugetlbfs-backed memory can use hugepages in the
guest.  Does it make sense to you?

Paolo

> ---
> Based on PCI tree as it has patches that add
> 2.5 machine type.
> ---
>  hw/i386/pc.c |  8 +---
>  hw/i386/pc_piix.c| 12 ++--
>  hw/i386/pc_q35.c | 12 ++--
>  include/hw/i386/pc.h |  5 ++---
>  4 files changed, 27 insertions(+), 10 deletions(-)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index b5107f7..ddb6710 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -1645,8 +1645,9 @@ static void pc_dimm_plug(HotplugHandler *hotplug_dev,
>  MemoryRegion *mr = ddc->get_memory_region(dimm);
>  uint64_t align = TARGET_PAGE_SIZE;
>  
> -if (memory_region_get_alignment(mr) && pcms->enforce_aligned_dimm) {
> -align = memory_region_get_alignment(mr);
> +if (pcms->enforce_aligned_dimm) {
> +align = MAX(memory_region_get_alignment(mr),
> +pcms->enforce_aligned_dimm);
>  }
>  
>  if (!pcms->acpi_dev) {
> @@ -1936,7 +1937,8 @@ static void pc_machine_initfn(Object *obj)
>  "Enable vmport (pc & q35)",
>  _abort);
>  
> -pcms->enforce_aligned_dimm = true;
> +/* align DIMM starting address/size by 128Mb */
> +pcms->enforce_aligned_dimm = 1ULL << 27;
>  object_property_add_bool(obj, PC_MACHINE_ENFORCE_ALIGNED_DIMM,
>   pc_machine_get_aligned_dimm,
>   NULL, _abort);
> diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
> index caa4edc..7671905 100644
> --- a/hw/i386/pc_piix.c
> +++ b/hw/i386/pc_piix.c
> @@ -301,9 +301,17 @@ static void pc_init1(MachineState *machine,
>  }
>  }
>  
> +static void pc_compat_2_4(MachineState *machine)
> +{
> +PCMachineState *pcms = PC_MACHINE(machine);
> +
> +pcms->enforce_aligned_dimm = TARGET_PAGE_SIZE;
> +}
> +
>  static void pc_compat_2_3(MachineState *machine)
>  {
>  PCMachineState *pcms = PC_MACHINE(machine);
> +pc_compat_2_4(machine);
>  savevm_skip_section_footers();
>  if (kvm_enabled()) {
>  pcms->smm = ON_OFF_AUTO_OFF;
> @@ -326,7 +334,7 @@ static void pc_compat_2_1(MachineState *machine)
>  pc_compat_2_2(machine);
>  smbios_uuid_encoded = false;
>  x86_cpu_compat_kvm_no_autodisable(FEAT_8000_0001_ECX, CPUID_EXT3_SVM);
> -pcms->enforce_aligned_dimm = false;
> +pcms->enforce_aligned_dimm = 0;
>  }
>  
>  static void pc_compat_2_0(MachineState *machine)
> @@ -485,7 +493,7 @@ static void pc_i440fx_2_4_machine_options(MachineClass *m)
>  SET_MACHINE_COMPAT(m, PC_COMPAT_2_4);
>  }
>  
> -DEFINE_I440FX_MACHINE(v2_4, "pc-i440fx-2.4", NULL,
> +DEFINE_I440FX_MACHINE(v2_4, "pc-i440fx-2.4", pc_compat_2_3,
>pc_i440fx_2_4_machine_options)
>  
>  
> diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
> index 506b6bf..72b479f 100644
> --- a/hw/i386/pc_q35.c
> +++ b/hw/i386/pc_q35.c
> @@ -284,9 +284,17 @@ static void pc_q35_init(MachineState *machine)
>  }
>  }
>  
> +static void pc_compat_2_4(MachineState *machine)
> +{
> +PCMachineState *pcms = PC_MACHINE(machine);
> +
> +pcms->enforce_aligned_dimm = TARGET_PAGE_SIZE;
> +}
> +
>  static void pc_compat_2_3(MachineState *machine)
>  {
>  PCMachineState *pcms = PC_MACHINE(machine);
> +pc_compat_2_4(machine);
>  savevm_skip_section_footers();
>  if (kvm_enabled()) {
>  pcms->smm = ON_OFF_AUTO_OFF;
> @@ -307,7 +315,7 @@ static void pc_compat_2_1(MachineState *machine)
>  PCMachineState *pcms = PC_MACHINE(machine);
> 

Re: [Qemu-devel] RFC: virtio-peer shared memory based peer communication device

[Jan Kiszka]

On 2015-09-21 14:13, Michael S. Tsirkin wrote:
> On Fri, Sep 18, 2015 at 06:29:27PM +0200, Claudio Fontana wrote:
>> Hello,
>>
>> this is a first RFC for virtio-peer 0.1, which is still very much a work in 
>> progress:
>>
>> https://github.com/hw-claudio/virtio-peer/wiki
>>
>> It is also available as PDF there, but the text is reproduced here for 
>> commenting:
>>
>> Peer shared memory communication device (virtio-peer)
>>
>> General Overview
>>
>> (I recommend looking at the PDF for some clarifying pictures)
>>
>> The Virtio Peer shared memory communication device (virtio-peer) is a
>> virtual device which allows high performance low latency guest to
>> guest communication. It uses a new queue extension feature tentatively
>> called VIRTIO_F_WINDOW which indicates that descriptor tables,
>> available and used rings and Queue Data reside in physical memory
>> ranges called Windows, each identified with an unique identifier
>> called WindowID.
> 
> So if I had to summarize the difference from regular virtio,
> I'd say the main one is that this uses window id + offset
> instead of the physical address.
> 
> 
> My question is - why do it?
> 
> All windows are in memory space, are they not?
> 
> How about guest using full physical addresses,
> and hypervisor sending the window physical address
> to VM2?
> 
> VM2 can uses that to find both window id and offset.
> 
> 
> This way at least VM1 can use regular virtio without changes.

What would be the value of having different drivers in VM1 and VM2,
specifically if both run Linux?

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

[Qemu-devel] [PATCH 0/5] ide: avoid main-loop hang on CDROM/NFS failure

[Peter Lieven]

This series aims at avoiding a hanging main-loop if a vserver has a
CDROM image mounted from a NFS share and that NFS share goes down.
Typical situation is that users mount an CDROM ISO to install something
and then forget to eject that CDROM afterwards.
As a consequence this mounted CD is able to bring down the
whole vserver if the backend NFS share is unreachable. This is bad
especially if the CDROM itself is not needed anymore at this point.

This series aims at fixing 3 blocking I/O operations that would
hang if the NFS server is unavailable:
 - ATAPI PIO read requests used sync calls to blk_read, convert
   them to an async variant.
 - If a busmaster DMA request is cancelled all requests are drained.
   Convert the drain to an async request canceling.
 - query-block in the HMP or QMP hangs because it indirectly calls
   bdrv_get_allocated_file_size.

Note that Patch 5 is only included for completeness.

Peter

Peter Lieven (5):
  ide/atapi: make PIO read requests async
  ide/atapi: blk_aio_readv may return NULL
  ide: add support for cancelable read requests
  ide/atapi: enable cancelable requests
  block/nfs: cache allocated filesize for read-only files

 block/nfs.c   | 36 ++
 hw/ide/atapi.c| 75 +++
 hw/ide/core.c | 55 
 hw/ide/internal.h | 16 
 hw/ide/pci.c  | 42 ---
 5 files changed, 183 insertions(+), 41 deletions(-)

-- 
1.9.1

Re: [Qemu-devel] [PATCH v2 1/2] Add dynamic module loading for block drivers

[Denis V. Lunev]

On 09/08/2015 04:53 PM, Marc Marí wrote:

Extend the current module interface to allow for block drivers to be loaded
dynamically on request.

The only block drivers that can be converted into modules are the drivers
that don't perform any init operation except for registering themselves. This
is why libiscsi has been disabled as a module.

All the necessary module information is located in a new structure found in
include/qemu/module_block.h

Signed-off-by: Marc Marí 
---
  block.c | 70 +++
  configure   |  2 +-
  include/qemu/module.h   |  3 ++
  include/qemu/module_block.h | 90 +
  util/module.c   | 38 ++-
  5 files changed, 175 insertions(+), 28 deletions(-)
  create mode 100644 include/qemu/module_block.h

diff --git a/block.c b/block.c
index 090923c..814429a 100644
--- a/block.c
+++ b/block.c
@@ -27,6 +27,7 @@
  #include "block/block_int.h"
  #include "block/blockjob.h"
  #include "qemu/error-report.h"
+#include "qemu/module_block.h"
  #include "qemu/module.h"
  #include "qapi/qmp/qerror.h"
  #include "qapi/qmp/qjson.h"
@@ -277,11 +278,30 @@ void bdrv_add_close_notifier(BlockDriverState *bs, 
Notifier *notify)
  BlockDriver *bdrv_find_format(const char *format_name)
  {
  BlockDriver *drv1;
+size_t i;
+
  QLIST_FOREACH(drv1, _drivers, list) {
  if (!strcmp(drv1->format_name, format_name)) {
  return drv1;
  }
  }
+
+for (i = 0; i < ARRAY_SIZE(block_driver_modules); ++i) {
+if (!strcmp(block_driver_modules[i].format_name, format_name)) {
+block_module_load_one(block_driver_modules[i].library_name);
+/* Copying code is not nice, but this way the current discovery is
+ * not modified. Calling recursively could fail if the library
+ * has been deleted.
+ */
+QLIST_FOREACH(drv1, _drivers, list) {
+if (!strcmp(drv1->format_name, format_name)) {
+return drv1;
+}
+}
+}
+}
+
+
  return NULL;
  }
  
@@ -484,8 +504,15 @@ int get_tmp_filename(char *filename, int size)

  static BlockDriver *find_hdev_driver(const char *filename)
  {
  int score_max = 0, score;
+size_t i;
  BlockDriver *drv = NULL, *d;
  
+for (i = 0; i < ARRAY_SIZE(block_driver_modules); ++i) {

+if (block_driver_modules[i].has_probe_device) {
+block_module_load_one(block_driver_modules[i].library_name);
+}
+}
+
  QLIST_FOREACH(d, _drivers, list) {
  if (d->bdrv_probe_device) {
  score = d->bdrv_probe_device(filename);
@@ -507,6 +534,7 @@ BlockDriver *bdrv_find_protocol(const char *filename,
  char protocol[128];
  int len;
  const char *p;
+size_t i;
  
  /* TODO Drivers without bdrv_file_open must be specified explicitly */
  
@@ -533,6 +561,7 @@ BlockDriver *bdrv_find_protocol(const char *filename,

  len = sizeof(protocol) - 1;
  memcpy(protocol, filename, len);
  protocol[len] = '\0';
+
  QLIST_FOREACH(drv1, _drivers, list) {
  if (drv1->protocol_name &&
  !strcmp(drv1->protocol_name, protocol)) {
@@ -540,6 +569,23 @@ BlockDriver *bdrv_find_protocol(const char *filename,
  }
  }
  
+for (i = 0; i < ARRAY_SIZE(block_driver_modules); ++i) {

+if (block_driver_modules[i].protocol_name &&
+!strcmp(block_driver_modules[i].protocol_name, protocol)) {
+block_module_load_one(block_driver_modules[i].library_name);
+/* Copying code is not nice, but this way the current discovery is
+ * not modified. Calling recursively could fail if the library
+ * has been deleted.
+ */
+QLIST_FOREACH(drv1, _drivers, list) {
+if (drv1->protocol_name &&
+!strcmp(drv1->protocol_name, protocol)) {
+return drv1;
+}
+}
+}
+}
+
  error_setg(errp, "Unknown protocol '%s'", protocol);
  return NULL;
  }
@@ -562,8 +608,15 @@ BlockDriver *bdrv_probe_all(const uint8_t *buf, int 
buf_size,
  const char *filename)
  {
  int score_max = 0, score;
+size_t i;
  BlockDriver *drv = NULL, *d;
  
+for (i = 0; i < ARRAY_SIZE(block_driver_modules); ++i) {

+if (block_driver_modules[i].has_probe) {
+block_module_load_one(block_driver_modules[i].library_name);
+}
+}
+
  QLIST_FOREACH(d, _drivers, list) {
  if (d->bdrv_probe) {
  score = d->bdrv_probe(buf, buf_size, filename);
@@ -2784,6 +2837,7 @@ void bdrv_iterate_format(void (*it)(void *opaque, const 
char *name),
  BlockDriver *drv;
  int count = 0;
  int i;
+size_t n;
  const char **formats = NULL;
  
  

Re: [Qemu-devel] [PATCH v6 06/14] blockjob: Add .commit and .abort block job actions

[Fam Zheng]

On Mon, 09/21 18:29, John Snow wrote:
> 
> 
> On 09/15/2015 02:11 AM, Fam Zheng wrote:
> > Reviewed-by: Max Reitz 
> > Signed-off-by: Fam Zheng 
> > ---
> >  include/block/blockjob.h | 18 ++
> >  1 file changed, 18 insertions(+)
> > 
> > diff --git a/include/block/blockjob.h b/include/block/blockjob.h
> > index 3e7ad21..a7b497c 100644
> > --- a/include/block/blockjob.h
> > +++ b/include/block/blockjob.h
> > @@ -50,6 +50,24 @@ typedef struct BlockJobDriver {
> >   * manually.
> >   */
> >  void (*complete)(BlockJob *job, Error **errp);
> > +
> > +/**
> > + * If the callback is not NULL, it will be invoked when all the jobs
> > + * belonging to the same transaction complete; or upon this job's
> > + * completion if it is not in a transaction. Skipped if NULL.
> > + *
> > + * Exactly one of .commit() and .abort() will be called for each job.
> > + */
> > +void (*commit)(BlockJob *job);
> > +
> 
> I find this phrasing strange, but maybe it's just me. "Exactly one of
> commit and abort will be called for each job" implies [to me] that it'd
> be possible to call commit for one, but abort for different jobs [in a
> transaction] -- but clearly we don't mean that. It is the "for each job"
> that implies an iteration over a collection to me.
> 
> Just above we say "[commit] will be invoked when all the jobs belonging
> to the same transaction are complete" which itself implies either all
> jobs will be committed or all jobs will be aborted.
> 
> Maybe:
> 
> "All jobs will complete with a call to either .commit() or .abort() but
> never both."
> 
> But I might be being too bikesheddy.
> 
> > +/**
> > + * If the callback is not NULL, it will be invoked when any job in the
> > + * same transaction fails; or upon this job's failure (due to error or
> > + * cancellation) if it is not in a transaction. Skipped if NULL.
> > + *
> > + * Exactly one of .commit() and .abort() will be called for each job.
> > + */
> > +void (*abort)(BlockJob *job);
> >  } BlockJobDriver;
> >  
> >  /**
> > 
> 
> I'm probably just too picky.
> 
> Reviewed-by: John Snow 
> 

No problem, It makese sense, I'll use your words :)

Thanks.

Fam

[Qemu-devel] [PATCH v7 14/14] tests: add BlockJobTxn unit test

[Fam Zheng]

From: Stefan Hajnoczi 

The BlockJobTxn unit test verifies that both single jobs and pairs of
jobs behave as a transaction group.  Either all jobs complete
successfully or the group is cancelled.

Signed-off-by: Stefan Hajnoczi 
Signed-off-by: Fam Zheng 
Reviewed-by: Max Reitz 
---
 tests/Makefile|   3 +
 tests/test-blockjob-txn.c | 244 ++
 2 files changed, 247 insertions(+)
 create mode 100644 tests/test-blockjob-txn.c

diff --git a/tests/Makefile b/tests/Makefile
index 4063639..e887a61 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -47,6 +47,8 @@ check-unit-y += tests/test-thread-pool$(EXESUF)
 gcov-files-test-thread-pool-y = thread-pool.c
 gcov-files-test-hbitmap-y = util/hbitmap.c
 check-unit-y += tests/test-hbitmap$(EXESUF)
+gcov-files-test-hbitmap-y = blockjob.c
+check-unit-y += tests/test-blockjob-txn$(EXESUF)
 check-unit-y += tests/test-x86-cpuid$(EXESUF)
 # all code tested by test-x86-cpuid is inside topology.h
 gcov-files-test-x86-cpuid-y =
@@ -307,6 +309,7 @@ tests/test-coroutine$(EXESUF): tests/test-coroutine.o 
$(test-block-obj-y)
 tests/test-aio$(EXESUF): tests/test-aio.o $(test-block-obj-y)
 tests/test-rfifolock$(EXESUF): tests/test-rfifolock.o $(test-util-obj-y)
 tests/test-throttle$(EXESUF): tests/test-throttle.o $(test-block-obj-y)
+tests/test-blockjob-txn$(EXESUF): tests/test-blockjob-txn.o 
$(test-block-obj-y) $(test-util-obj-y)
 tests/test-thread-pool$(EXESUF): tests/test-thread-pool.o $(test-block-obj-y)
 tests/test-iov$(EXESUF): tests/test-iov.o $(test-util-obj-y)
 tests/test-hbitmap$(EXESUF): tests/test-hbitmap.o $(test-util-obj-y)
diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c
new file mode 100644
index 000..000f70e
--- /dev/null
+++ b/tests/test-blockjob-txn.c
@@ -0,0 +1,244 @@
+/*
+ * Blockjob transactions tests
+ *
+ * Copyright Red Hat, Inc. 2015
+ *
+ * Authors:
+ *  Stefan Hajnoczi
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ */
+
+#include 
+#include "qapi/error.h"
+#include "qemu/main-loop.h"
+#include "block/blockjob.h"
+
+typedef struct {
+BlockJob common;
+unsigned int iterations;
+bool use_timer;
+int rc;
+int *result;
+} TestBlockJob;
+
+static const BlockJobDriver test_block_job_driver = {
+.instance_size = sizeof(TestBlockJob),
+};
+
+static void test_block_job_complete(BlockJob *job, void *opaque)
+{
+BlockDriverState *bs = job->bs;
+int rc = (intptr_t)opaque;
+
+if (block_job_is_cancelled(job)) {
+rc = -ECANCELED;
+}
+
+block_job_completed(job, rc);
+bdrv_unref(bs);
+}
+
+static void coroutine_fn test_block_job_run(void *opaque)
+{
+TestBlockJob *s = opaque;
+BlockJob *job = >common;
+
+while (s->iterations--) {
+if (s->use_timer) {
+block_job_sleep_ns(job, QEMU_CLOCK_REALTIME, 0);
+} else {
+block_job_yield(job);
+}
+
+if (block_job_is_cancelled(job)) {
+break;
+}
+}
+
+block_job_defer_to_main_loop(job, test_block_job_complete,
+ (void *)(intptr_t)s->rc);
+}
+
+typedef struct {
+TestBlockJob *job;
+int *result;
+} TestBlockJobCBData;
+
+static void test_block_job_cb(void *opaque, int ret)
+{
+TestBlockJobCBData *data = opaque;
+if (!ret && block_job_is_cancelled(>job->common)) {
+ret = -ECANCELED;
+}
+*data->result = ret;
+g_free(data);
+}
+
+/* Create a block job that completes with a given return code after a given
+ * number of event loop iterations.  The return code is stored in the given
+ * result pointer.
+ *
+ * The event loop iterations can either be handled automatically with a 0 delay
+ * timer, or they can be stepped manually by entering the coroutine.
+ */
+static BlockJob *test_block_job_start(unsigned int iterations,
+  bool use_timer,
+  int rc, int *result)
+{
+BlockDriverState *bs;
+TestBlockJob *s;
+TestBlockJobCBData *data;
+
+data = g_new0(TestBlockJobCBData, 1);
+bs = bdrv_new();
+s = block_job_create(_block_job_driver, bs, 0, test_block_job_cb,
+ data, _abort);
+s->iterations = iterations;
+s->use_timer = use_timer;
+s->rc = rc;
+s->result = result;
+s->common.co = qemu_coroutine_create(test_block_job_run);
+data->job = s;
+data->result = result;
+qemu_coroutine_enter(s->common.co, s);
+return >common;
+}
+
+static void test_single_job(int expected)
+{
+BlockJob *job;
+BlockJobTxn *txn;
+int result = -EINPROGRESS;
+
+txn = block_job_txn_new();
+job = test_block_job_start(1, true, expected, );
+block_job_txn_add_job(txn, job);
+
+if (expected == -ECANCELED) {
+  

[Qemu-devel] [PATCH v7 10/14] blockdev: make BlockJobTxn available to qmp 'transaction'

[Fam Zheng]

From: Stefan Hajnoczi 

Provide a BlockJobTxn to actions executed in a qmp 'transaction'
command.  This allows actions to make their block jobs either complete
as a group or fail/cancel together.

The next patch adds the first user.

Signed-off-by: Stefan Hajnoczi 
Reviewed-by: Fam Zheng 
Reviewed-by: Max Reitz 
Signed-off-by: Fam Zheng 
---
 blockdev.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/blockdev.c b/blockdev.c
index 796bc64..ed50cb4 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1285,6 +1285,7 @@ typedef struct BlkActionOps {
 struct BlkActionState {
 TransactionAction *action;
 const BlkActionOps *ops;
+BlockJobTxn *block_job_txn;
 QSIMPLEQ_ENTRY(BlkActionState) entry;
 };
 
@@ -1883,12 +1884,15 @@ static const BlkActionOps actions[] = {
 void qmp_transaction(TransactionActionList *dev_list, Error **errp)
 {
 TransactionActionList *dev_entry = dev_list;
+BlockJobTxn *block_job_txn;
 BlkActionState *state, *next;
 Error *local_err = NULL;
 
 QSIMPLEQ_HEAD(snap_bdrv_states, BlkActionState) snap_bdrv_states;
 QSIMPLEQ_INIT(_bdrv_states);
 
+block_job_txn = block_job_txn_new();
+
 /* drain all i/o before any operations */
 bdrv_drain_all();
 
@@ -1908,6 +1912,7 @@ void qmp_transaction(TransactionActionList *dev_list, 
Error **errp)
 state = g_malloc0(ops->instance_size);
 state->ops = ops;
 state->action = dev_info;
+state->block_job_txn = block_job_txn;
 QSIMPLEQ_INSERT_TAIL(_bdrv_states, state, entry);
 
 state->ops->prepare(state, _err);
@@ -1940,6 +1945,7 @@ exit:
 }
 g_free(state);
 }
+block_job_txn_unref(block_job_txn);
 }
 
 
-- 
2.4.3

[Qemu-devel] [PATCH v5 46/46] qapi: Allow anonymous base for flat union

[Eric Blake]

Rather than requiring all flat unions to explicitly create
a separate base struct, we want to allow the qapi schema
to specify the common fields via an inline dictionary. This
is similar to how commands can specify inline types for the
arguments.

Now that the feature is legal, we can drop the former
flat-union-bad-base negative test, and instead change the
positive tests in qapi-schema-test to use it.

Signed-off-by: Eric Blake 
---
 docs/qapi-code-gen.txt | 28 ++--
 scripts/qapi-commands.py   |  2 +-
 scripts/qapi-event.py  |  2 +-
 scripts/qapi-types.py  |  2 +-
 scripts/qapi-visit.py  | 15 +++
 scripts/qapi.py| 20 +---
 tests/Makefile |  1 -
 tests/qapi-schema/flat-union-bad-base.err  |  1 -
 tests/qapi-schema/flat-union-bad-base.exit |  1 -
 tests/qapi-schema/flat-union-bad-base.json | 13 -
 tests/qapi-schema/flat-union-bad-base.out  |  0
 tests/qapi-schema/qapi-schema-test.json|  2 +-
 tests/qapi-schema/qapi-schema-test.out |  5 -
 13 files changed, 46 insertions(+), 46 deletions(-)
 delete mode 100644 tests/qapi-schema/flat-union-bad-base.err
 delete mode 100644 tests/qapi-schema/flat-union-bad-base.exit
 delete mode 100644 tests/qapi-schema/flat-union-bad-base.json
 delete mode 100644 tests/qapi-schema/flat-union-bad-base.out

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 49e3586..b14dc98 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -278,7 +278,7 @@ better than open-coding the field to be type 'str'.
 === Union types ===

 Usage: { 'union': STRING, 'data': DICT }
-or:{ 'union': STRING, 'data': DICT, 'base': STRUCT-NAME,
+or:{ 'union': STRING, 'data': DICT, 'base': STRUCT-NAME-OR-DICT,
  'discriminator': ENUM-MEMBER-OF-BASE }

 Union types are used to let the user choose between several different
@@ -314,13 +314,16 @@ an implicit C enum 'NameKind' is created, corresponding 
to the union
 the union can be named 'max', as this would collide with the implicit
 enum.  The value for each branch can be of any type.

-A flat union definition specifies a struct as its base, and
-avoids nesting on the wire.  All branches of the union must be
-complex types, and the top-level fields of the union dictionary on
-the wire will be combination of fields from both the base type and the
-appropriate branch type (when merging two dictionaries, there must be
-no keys in common).  The 'discriminator' field must be the name of an
-enum-typed member of the base struct.
+A flat union definition avoids nesting on the wire, and specifies a
+set of common fields that occur in all variants of the union.  The
+'base' key must specifiy either a type name (the type must be a
+struct, not a union), or a dictionary representing an anonymous type.
+All branches of the union must be complex types, and the top-level
+fields of the union dictionary on the wire will be combination of
+fields from both the base type and the appropriate branch type (when
+merging two dictionaries, there must be no keys in common).  The
+'discriminator' field must be the name of an enum-typed member of the
+base struct.

 The following example enhances the above simple union example by
 adding a common field 'readonly', renaming the discriminator to
@@ -328,10 +331,8 @@ something more applicable, and reducing the number of {} 
required on
 the wire:

  { 'enum': 'BlockdevDriver', 'data': [ 'file', 'qcow2' ] }
- { 'struct': 'BlockdevCommonOptions',
-   'data': { 'driver': 'BlockdevDriver', 'readonly': 'bool' } }
  { 'union': 'BlockdevOptions',
-   'base': 'BlockdevCommonOptions',
+   'base': { 'driver': 'BlockdevDriver', 'readonly': 'bool' },
'discriminator': 'driver',
'data': { 'file': 'FileOptions',
  'qcow2': 'Qcow2Options' } }
@@ -349,7 +350,7 @@ code generator can ensure that branches exist for all 
values of the
 enum (although the order of the keys need not match the declaration of
 the enum).  In the resulting generated C data types, a flat union is
 represented as a struct with the base member fields included directly,
-and then a union of structures for each branch of the struct.
+and then a union of pointers to structures for each branch of the struct.

 A simple union can always be re-written as a flat union where the base
 class has a single member named 'type', and where each branch of the
@@ -360,10 +361,9 @@ union has a struct with a single member named 'data'.  
That is,
 is identical on the wire to:

  { 'enum': 'Enum', 'data': ['one', 'two'] }
- { 'struct': 'Base', 'data': { 'type': 'Enum' } }
  { 'struct': 'Branch1', 'data': { 'data': 'str' } }
  { 'struct': 'Branch2', 'data': { 'data': 'int' } }
- { 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
+ { 'union': 'Flat': 'base': { 'type': 'Enum' }, 

[Qemu-devel] [PATCH v5 39/46] qapi: Plumb in 'box' to qapi generator lower levels

[Eric Blake]

A future patch will add support for passing a qapi union
type as the 'data' of a command.  But to do that, the user
function for implementing the command, as called by the
generated marshal command, must take the corresponding C
struct as a single boxed pointer, rather than a breakdown
into one parameter per member.  This patch adds the
internal plubming of a 'box' flag associated with each
command and event.  For this patch, no behavior changes,
other than the testsuite outputting the value of the new
flag (always False for now).

Signed-off-by: Eric Blake 
---
 scripts/qapi-commands.py| 63 +++--
 scripts/qapi-event.py   | 34 ++
 scripts/qapi-introspect.py  |  4 +--
 scripts/qapi.py | 51 +++---
 tests/qapi-schema/args-member-array.out |  2 +-
 tests/qapi-schema/event-case.out|  1 +
 tests/qapi-schema/ident-with-escape.out |  2 +-
 tests/qapi-schema/indented-expr.out |  4 +--
 tests/qapi-schema/qapi-schema-test.out  | 17 +
 tests/qapi-schema/returns-int.out   |  2 +-
 tests/qapi-schema/test-qapi.py  |  8 +++--
 11 files changed, 111 insertions(+), 77 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 55287b1..d003e3c 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -16,24 +16,27 @@ from qapi import *
 import re


-def gen_command_decl(name, arg_type, ret_type):
+def gen_command_decl(name, arg_type, box, ret_type):
 return mcgen('''
 %(c_type)s qmp_%(c_name)s(%(params)s);
 ''',
  c_type=(ret_type and ret_type.c_type()) or 'void',
  c_name=c_name(name),
- params=gen_params(arg_type, 'Error **errp'))
+ params=gen_params(arg_type, box, 'Error **errp'))


-def gen_call(name, arg_type, ret_type):
+def gen_call(name, arg_type, box, ret_type):
 ret = ''

 argstr = ''
-if arg_type:
-for memb in arg_type.members:
-if memb.optional:
-argstr += 'has_%s, ' % c_name(memb.name)
-argstr += '%s, ' % c_name(memb.name)
+if box:
+assert False# not implemented
+else:
+if arg_type:
+for memb in arg_type.members:
+if memb.optional:
+argstr += 'has_%s, ' % c_name(memb.name)
+argstr += '%s, ' % c_name(memb.name)

 lhs = ''
 if ret_type:
@@ -56,7 +59,7 @@ qmp_marshal_output_%(c_name)s(retval, ret, );
 return ret


-def gen_marshal_vars(arg_type, ret_type):
+def gen_marshal_vars(arg_type, box, ret_type):
 ret = mcgen('''
 Error *err = NULL;
 ''')
@@ -76,18 +79,21 @@ QapiDeallocVisitor *md;
 Visitor *v;
 ''')

-for memb in arg_type.members:
-if memb.optional:
-ret += mcgen('''
+if box:
+assert False# not implemented
+else:
+for memb in arg_type.members:
+if memb.optional:
+ret += mcgen('''
 bool has_%(c_name)s = false;
 ''',
- c_name=c_name(memb.name))
-ret += mcgen('''
+ c_name=c_name(memb.name))
+ret += mcgen('''
 %(c_type)s %(c_name)s = %(c_null)s;
 ''',
- c_name=c_name(memb.name),
- c_type=memb.type.c_type(),
- c_null=memb.type.c_null())
+ c_name=c_name(memb.name),
+ c_type=memb.type.c_type(),
+ c_null=memb.type.c_null())
 ret += '\n'
 else:
 ret += mcgen('''
@@ -99,7 +105,7 @@ bool has_%(c_name)s = false;
 return ret


-def gen_marshal_input_visit(arg_type, dealloc=False):
+def gen_marshal_input_visit(arg_type, box, dealloc=False):
 ret = ''

 if not arg_type:
@@ -120,7 +126,10 @@ v = qapi_dealloc_get_visitor(md);
 v = qmp_input_get_visitor(mi);
 ''')

-ret += gen_visit_fields(arg_type.members, '', False, errarg)
+if box:
+assert False# not implemented
+else:
+ret += gen_visit_fields(arg_type.members, '', False, errarg)

 if dealloc:
 ret += mcgen('''
@@ -173,7 +182,7 @@ def gen_marshal_decl(name):
  proto=gen_marshal_proto(name))


-def gen_marshal(name, arg_type, ret_type):
+def gen_marshal(name, arg_type, box, ret_type):
 ret = mcgen('''

 %(proto)s
@@ -181,9 +190,9 @@ def gen_marshal(name, arg_type, ret_type):
 ''',
 proto=gen_marshal_proto(name))

-ret += gen_marshal_vars(arg_type, ret_type)
-ret += gen_marshal_input_visit(arg_type)
-ret += gen_call(name, arg_type, ret_type)
+ret += gen_marshal_vars(arg_type, box, ret_type)
+ret += gen_marshal_input_visit(arg_type, box)
+ret += gen_call(name, arg_type, box, ret_type)

 if re.search('^ *goto out;', ret, 

Re: [Qemu-devel] [PATCH v6 05/14] blockjob: Introduce reference count

[John Snow]

On 09/15/2015 02:11 AM, Fam Zheng wrote:
> So that block_job_complete_sync can be simplified.
> 
> Reviewed-by: Max Reitz 
> Signed-off-by: Fam Zheng 
> ---
>  block/mirror.c   |  2 +-
>  blockjob.c   | 22 ++
>  include/block/blockjob.h | 18 +++---
>  3 files changed, 30 insertions(+), 12 deletions(-)
> 
> diff --git a/block/mirror.c b/block/mirror.c
> index a258926..3472ad4 100644
> --- a/block/mirror.c
> +++ b/block/mirror.c
> @@ -736,7 +736,7 @@ static void mirror_start_job(BlockDriverState *bs, 
> BlockDriverState *target,
>  s->dirty_bitmap = bdrv_create_dirty_bitmap(bs, granularity, NULL, errp);
>  if (!s->dirty_bitmap) {
>  g_free(s->replaces);
> -block_job_release(bs);
> +block_job_unref(>common);
>  return;
>  }
>  bdrv_set_enable_write_cache(s->target, true);
> diff --git a/blockjob.c b/blockjob.c
> index 62bb906..ec12887 100644
> --- a/blockjob.c
> +++ b/blockjob.c
> @@ -58,6 +58,7 @@ void *block_job_create(const BlockJobDriver *driver, 
> BlockDriverState *bs,
>  job->cb= cb;
>  job->opaque= opaque;
>  job->busy  = true;
> +job->refcnt= 1;
>  bs->job = job;
>  
>  /* Only set speed when necessary to avoid NotSupported error */
> @@ -66,7 +67,7 @@ void *block_job_create(const BlockJobDriver *driver, 
> BlockDriverState *bs,
>  
>  block_job_set_speed(job, speed, _err);
>  if (local_err) {
> -block_job_release(bs);
> +block_job_unref(job);
>  error_propagate(errp, local_err);
>  return NULL;
>  }
> @@ -74,14 +75,19 @@ void *block_job_create(const BlockJobDriver *driver, 
> BlockDriverState *bs,
>  return job;
>  }
>  
> -void block_job_release(BlockDriverState *bs)
> +void block_job_ref(BlockJob *job)
>  {
> -BlockJob *job = bs->job;
> +++job->refcnt;
> +}
>  
> -bs->job = NULL;
> -bdrv_op_unblock_all(bs, job->blocker);
> -error_free(job->blocker);
> -g_free(job);
> +void block_job_unref(BlockJob *job)
> +{
> +if (--job->refcnt == 0) {
> +job->bs->job = NULL;
> +bdrv_op_unblock_all(job->bs, job->blocker);
> +error_free(job->blocker);
> +g_free(job);
> +}
>  }
>  
>  void block_job_completed(BlockJob *job, int ret)
> @@ -90,7 +96,7 @@ void block_job_completed(BlockJob *job, int ret)
>  
>  assert(bs->job == job);
>  job->cb(job->opaque, ret);
> -block_job_release(bs);
> +block_job_unref(job);
>  }
>  
>  void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)
> diff --git a/include/block/blockjob.h b/include/block/blockjob.h
> index dd9d5e6..3e7ad21 100644
> --- a/include/block/blockjob.h
> +++ b/include/block/blockjob.h
> @@ -122,6 +122,9 @@ struct BlockJob {
>  
>  /** The opaque value that is passed to the completion function.  */
>  void *opaque;
> +
> +/** Reference count of the block job */
> +int refcnt;
>  };
>  
>  /**
> @@ -166,12 +169,21 @@ void block_job_sleep_ns(BlockJob *job, QEMUClockType 
> type, int64_t ns);
>  void block_job_yield(BlockJob *job);
>  
>  /**
> - * block_job_release:
> + * block_job_ref:
>   * @bs: The block device.
>   *
> - * Release job resources when an error occurred or job completed.
> + * Grab a reference to the block job. Should be paired with block_job_unref.
>   */
> -void block_job_release(BlockDriverState *bs);
> +void block_job_ref(BlockJob *job);
> +
> +/**
> + * block_job_unref:
> + * @bs: The block device.
> + *
> + * Release reference to the block job and release resources if it is the last
> + * reference.
> + */
> +void block_job_unref(BlockJob *job);
>  
>  /**
>   * block_job_completed:
> 

Reviewed-by: John Snow 

Re: [Qemu-devel] [PATCH v2 10/22] target-sh4: Add flags state to insn_start

[Aurelien Jarno]

On 2015-09-17 21:55, Richard Henderson wrote:
> Reviewed-by: Peter Maydell 
> Signed-off-by: Richard Henderson 
> ---
>  target-sh4/cpu.h   | 1 +
>  target-sh4/translate.c | 2 +-
>  2 files changed, 2 insertions(+), 1 deletion(-)

Reviewed-by: Aurelien Jarno 

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net

Re: [Qemu-devel] Compiling Qemu from Cygwin

[Mike Ladouceur]

Sorry, at this point, I've moved to compiling from Ubuntu as you suggested
earlier.

On Sun, Sep 20, 2015 at 9:19 PM, Mike Ladouceur 
wrote:

> Okay, i followed the steps from the reply to my email I noticed on the
> website, but never made its way to my inbox. Now, I'm stuck with the
> following errors:
>
> /home/mike/qemu/include/glib-compat.h: At top level:
> /home/mike/qemu/include/glib-compat.h:145:21: error: conflicting types for
> ‘g_cond_signal’
>  static inline void (g_cond_signal)(CompatGCond *cond)
>  ^
> In file included from
> /usr/x86_64-w64-mingw32/include/glib-2.0/glib/gasyncqueue.h:34:0,
>  from /usr/x86_64-w64-mingw32/include/glib-2.0/glib.h:34,
>  from qga/commands.c:13:
> /usr/x86_64-w64-mingw32/include/glib-2.0/glib/gthread.h:201:17: note:
> previous declaration of ‘g_cond_signal’ was here
>  voidg_cond_signal   (GCond  *cond);
>  ^
> In file included from /home/mike/qemu/include/qemu-common.h:25:0,
>  from ./qga/guest-agent-core.h:14,
>  from qga/commands.c:14:
> /home/mike/qemu/include/glib-compat.h: In function ‘g_cond_signal’:
> /home/mike/qemu/include/glib-compat.h:148:5: warning: passing argument 1
> of ‘g_cond_signal’ from incompatible pointer type [enabled by default]
>  g_cond_signal((GCond *) cond->once.retval);
>  ^
> /home/mike/qemu/include/glib-compat.h:145:21: note: expected ‘struct
> CompatGCond *’ but argument is of type ‘struct GCond *’
>  static inline void (g_cond_signal)(CompatGCond *cond)
>  ^
> /home/mike/qemu/include/glib-compat.h: At top level:
> /home/mike/qemu/include/glib-compat.h:154:24: error: static declaration of
> ‘g_thread_new’ follows non-static declaration
>  static inline GThread *g_thread_new(const char *name,
> ^
> In file included from
> /usr/x86_64-w64-mingw32/include/glib-2.0/glib/gasyncqueue.h:34:0,
>  from /usr/x86_64-w64-mingw32/include/glib-2.0/glib.h:34,
>  from qga/commands.c:13:
> /usr/x86_64-w64-mingw32/include/glib-2.0/glib/gthread.h:145:17: note:
> previous declaration of ‘g_thread_new’ was here
>  GThread *   g_thread_new(const gchar*name,
>  ^
> make: *** [qga/commands.o] Error 1
>
>
> On Fri, Sep 18, 2015 at 10:49 PM, Mike Ladouceur 
> wrote:
>
>> I tried from Ubuntu in Virtualbox and this is the error I now get and it
>> appears to be similar to the error from Cygwin. Configure seems to run
>> fine. Make fails. I'm officially lost.
>>
>> ./configure --cross-prefix=x86_64-w64-mingw32
>> (...)
>>   CCqga/commands-win32.o
>> qga/commands-win32.c: In function ‘qmp_guest_set_user_password’:
>> qga/commands-win32.c:1254:5: warning: passing argument 2 of
>> ‘g_base64_decode’ from incompatible pointer type [enabled by default]
>>  rawpasswddata = (char *)g_base64_decode(password, );
>>  ^
>> In file included from /usr/include/glib-2.0/glib.h:35:0,
>>  from qga/commands-win32.c:14:
>> /usr/include/glib-2.0/glib/gbase64.h:52:9: note: expected ‘gsize *’ but
>> argument is of type ‘size_t *’
>>  guchar *g_base64_decode (const gchar  *text,
>>  ^
>>   CCqga/channel-win32.o
>> qga/channel-win32.c: In function ‘ga_channel_create_watch’:
>> qga/channel-win32.c:199:24: warning: cast from pointer to integer of
>> different size [-Wpointer-to-int-cast]
>>  watch->pollfd.fd = (gintptr) c->rstate.ov.hEvent;
>> ^
>> qga/channel-win32.c: At top level:
>> qga/channel-win32.c:205:11: error: conflicting types for ‘ga_channel_read’
>>  GIOStatus ga_channel_read(GAChannel *c, char *buf, size_t size, gsize
>> *count)
>>^
>> In file included from qga/channel-win32.c:9:0:
>> ./qga/channel.h:30:11: note: previous declaration of ‘ga_channel_read’
>> was here
>>  GIOStatus ga_channel_read(GAChannel *c, gchar *buf, gsize size, gsize
>> *count);
>>^
>> qga/channel-win32.c:269:11: error: conflicting types for
>> ‘ga_channel_write_all’
>>  GIOStatus ga_channel_write_all(GAChannel *c, const char *buf, size_t
>> size)
>>^
>> In file included from qga/channel-win32.c:9:0:
>> ./qga/channel.h:31:11: note: previous declaration of
>> ‘ga_channel_write_all’ was here
>>  GIOStatus ga_channel_write_all(GAChannel *c, const gchar *buf, gsize
>> size);
>>^
>> qga/channel-win32.c: In function ‘ga_channel_open’:
>> qga/channel-win32.c:291:19: error: ‘MAXPATHLEN’ undeclared (first use in
>> this function)
>>  gchar newpath[MAXPATHLEN] = {0};
>>^
>> qga/channel-win32.c:291:19: note: each undeclared identifier is reported
>> only once for each function it appears in
>> qga/channel-win32.c:291:11: warning: unused variable ‘newpath’
>> [-Wunused-variable]
>>  gchar newpath[MAXPATHLEN] = {0};
>>^
>> make: *** 

[Qemu-devel] [PATCH v7 12/14] iotests: 124 - transactional failure test

[Fam Zheng]

From: John Snow 

Use a transaction to request an incremental backup across two drives.
Coerce one of the jobs to fail, and then re-run the transaction.

Verify that no bitmap data was lost due to the partial transaction
failure.

To support the 'transactional-cancel' QMP argument name it's necessary
for transaction_action() to convert underscores in Python argument names
to hyphens for QMP argument names.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
Signed-off-by: Stefan Hajnoczi 
Signed-off-by: Fam Zheng 
---
 tests/qemu-iotests/124 | 130 -
 tests/qemu-iotests/124.out |   4 +-
 2 files changed, 130 insertions(+), 4 deletions(-)

diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124
index 9c1977e..33d00ef 100644
--- a/tests/qemu-iotests/124
+++ b/tests/qemu-iotests/124
@@ -39,7 +39,7 @@ def try_remove(img):
 def transaction_action(action, **kwargs):
 return {
 'type': action,
-'data': kwargs
+'data': dict((k.replace('_', '-'), v) for k, v in kwargs.iteritems())
 }
 
 
@@ -139,9 +139,12 @@ class TestIncrementalBackup(iotests.QMPTestCase):
 def do_qmp_backup(self, error='Input/output error', **kwargs):
 res = self.vm.qmp('drive-backup', **kwargs)
 self.assert_qmp(res, 'return', {})
+return self.wait_qmp_backup(kwargs['device'], error)
 
+
+def wait_qmp_backup(self, device, error='Input/output error'):
 event = self.vm.event_wait(name="BLOCK_JOB_COMPLETED",
-   match={'data': {'device': 
kwargs['device']}})
+   match={'data': {'device': device}})
 self.assertNotEqual(event, None)
 
 try:
@@ -156,6 +159,12 @@ class TestIncrementalBackup(iotests.QMPTestCase):
 return False
 
 
+def wait_qmp_backup_cancelled(self, device):
+event = self.vm.event_wait(name='BLOCK_JOB_CANCELLED',
+   match={'data': {'device': device}})
+self.assertNotEqual(event, None)
+
+
 def create_anchor_backup(self, drive=None):
 if drive is None:
 drive = self.drives[-1]
@@ -375,6 +384,123 @@ class TestIncrementalBackup(iotests.QMPTestCase):
 self.check_backups()
 
 
+def test_transaction_failure(self):
+'''Test: Verify backups made from a transaction that partially fails.
+
+Add a second drive with its own unique pattern, and add a bitmap to 
each
+drive. Use blkdebug to interfere with the backup on just one drive and
+attempt to create a coherent incremental backup across both drives.
+
+verify a failure in one but not both, then delete the failed stubs and
+re-run the same transaction.
+
+verify that both incrementals are created successfully.
+'''
+
+# Create a second drive, with pattern:
+drive1 = self.add_node('drive1')
+self.img_create(drive1['file'], drive1['fmt'])
+io_write_patterns(drive1['file'], (('0x14', 0, 512),
+   ('0x5d', '1M', '32k'),
+   ('0xcd', '32M', '124k')))
+
+# Create a blkdebug interface to this img as 'drive1'
+result = self.vm.qmp('blockdev-add', options={
+'id': drive1['id'],
+'driver': drive1['fmt'],
+'file': {
+'driver': 'blkdebug',
+'image': {
+'driver': 'file',
+'filename': drive1['file']
+},
+'set-state': [{
+'event': 'flush_to_disk',
+'state': 1,
+'new_state': 2
+}],
+'inject-error': [{
+'event': 'read_aio',
+'errno': 5,
+'state': 2,
+'immediately': False,
+'once': True
+}],
+}
+})
+self.assert_qmp(result, 'return', {})
+
+# Create bitmaps and full backups for both drives
+drive0 = self.drives[0]
+dr0bm0 = self.add_bitmap('bitmap0', drive0)
+dr1bm0 = self.add_bitmap('bitmap0', drive1)
+self.create_anchor_backup(drive0)
+self.create_anchor_backup(drive1)
+self.assert_no_active_block_jobs()
+self.assertFalse(self.vm.get_qmp_events(wait=False))
+
+# Emulate some writes
+self.hmp_io_writes(drive0['id'], (('0xab', 0, 512),
+  ('0xfe', '16M', '256k'),
+  ('0x64', '32736k', '64k')))
+self.hmp_io_writes(drive1['id'], (('0xba', 0, 512),
+  ('0xef', '16M', '256k'),
+  ('0x46', '32736k', '64k')))
+
+# Create 

[Qemu-devel] [PATCH v7 13/14] qmp-commands.hx: Update the supported 'transaction' operations

[Fam Zheng]

From: Kashyap Chamarthy 

Although the canonical source of reference for QMP commands is
qapi-schema.json, for consistency's sake, update qmp-commands.hx to
state the list of supported transactionable operations, namely:

drive-backup
blockdev-backup
blockdev-snapshot-internal-sync
abort
block-dirty-bitmap-add
block-dirty-bitmap-clear

Signed-off-by: Kashyap Chamarthy 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
Signed-off-by: John Snow 
Signed-off-by: Stefan Hajnoczi 
Signed-off-by: Fam Zheng 
---
 qmp-commands.hx | 21 -
 1 file changed, 16 insertions(+), 5 deletions(-)

diff --git a/qmp-commands.hx b/qmp-commands.hx
index 66f0300..365c874 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -1265,11 +1265,22 @@ SQMP
 transaction
 ---
 
-Atomically operate on one or more block devices.  The only supported operations
-for now are drive-backup, internal and external snapshotting.  A list of
-dictionaries is accepted, that contains the actions to be performed.
-If there is any failure performing any of the operations, all operations
-for the group are abandoned.
+Atomically operate on one or more block devices.  Operations that are
+currently supported:
+
+- drive-backup
+- blockdev-backup
+- blockdev-snapshot-sync
+- blockdev-snapshot-internal-sync
+- abort
+- block-dirty-bitmap-add
+- block-dirty-bitmap-clear
+
+Refer to the qemu/qapi-schema.json file for minimum required QEMU
+versions for these operations.  A list of dictionaries is accepted,
+that contains the actions to be performed.  If there is any failure
+performing any of the operations, all operations for the group are
+abandoned.
 
 For external snapshots, the dictionary contains the device, the file to use for
 the new snapshot, and the format.  The default format, if not specified, is
-- 
2.4.3

Re: [Qemu-devel] [PATCH v6 09/14] block: Add block job transactions

[John Snow]

On 09/15/2015 02:11 AM, Fam Zheng wrote:
> Sometimes block jobs must execute as a transaction group.  Finishing
> jobs wait until all other jobs are ready to complete successfully.
> Failure or cancellation of one job cancels the other jobs in the group.
> 
> Signed-off-by: Stefan Hajnoczi 
> [Rewrite the implementation which is now contained in block_job_completed.
> --Fam]
> Signed-off-by: Fam Zheng 
> ---
>  blockjob.c   | 135 
> ++-
>  include/block/block.h|   1 +
>  include/block/blockjob.h |  38 +
>  3 files changed, 172 insertions(+), 2 deletions(-)
> 
> diff --git a/blockjob.c b/blockjob.c
> index 36c18e0..91e8d3c 100644
> --- a/blockjob.c
> +++ b/blockjob.c
> @@ -36,6 +36,19 @@
>  #include "qemu/timer.h"
>  #include "qapi-event.h"
>  
> +/* Transactional group of block jobs */
> +struct BlockJobTxn {
> +
> +/* Is this txn being cancelled? */
> +bool aborting;
> +
> +/* List of jobs */
> +QLIST_HEAD(, BlockJob) jobs;
> +
> +/* Reference count */
> +int refcnt;
> +};
> +
>  void *block_job_create(const BlockJobDriver *driver, BlockDriverState *bs,
> int64_t speed, BlockCompletionFunc *cb,
> void *opaque, Error **errp)
> @@ -90,6 +103,86 @@ void block_job_unref(BlockJob *job)
>  }
>  }
>  
> +static void block_job_completed_single(BlockJob *job)
> +{
> +if (!job->ret) {
> +if (job->driver->commit) {
> +job->driver->commit(job);
> +}
> +} else {
> +if (job->driver->abort) {
> +job->driver->abort(job);
> +}
> +}
> +job->cb(job->opaque, job->ret);
> +if (job->txn) {
> +block_job_txn_unref(job->txn);
> +}
> +block_job_unref(job);
> +}
> +
> +static void block_job_completed_txn_abort(BlockJob *job)
> +{
> +AioContext *ctx;
> +BlockJobTxn *txn = job->txn;
> +BlockJob *other_job, *next;
> +
> +if (txn->aborting) {
> +/*
> + * We are cancelled by another job, which will handle everything.
> + */
> +return;
> +}
> +txn->aborting = true;
> +/* We are the first failed job. Cancel other jobs. */
> +QLIST_FOREACH(other_job, >jobs, txn_list) {
> +ctx = bdrv_get_aio_context(other_job->bs);
> +aio_context_acquire(ctx);
> +}
> +QLIST_FOREACH(other_job, >jobs, txn_list) {
> +if (other_job == job || other_job->completed) {
> +/* Other jobs are "effectively" cancelled by us, set the status 
> for
> + * them; this job, however, may or may not be cancelled, 
> depending
> + * on the caller, so leave it. */
> +if (other_job != job) {
> +other_job->cancelled = true;
> +}
> +continue;
> +}

This loop reads strangely to me due to this structure:

if (other_job == job || ...) {
  if (other_job != job) {
...
  }
}

why not:

foreach(...) {
  if (other_job == job) {
continue;
  }

  if (other_job->completed) {
other_job->cancelled = true;
continue;
  }

  /* job is neither ours, nor completed */
  block_job_cancel_sync(other_job);
  assert(other_job->completed);
}

> +block_job_cancel_sync(other_job);
> +assert(other_job->completed);
> +}
> +QLIST_FOREACH_SAFE(other_job, >jobs, txn_list, next) {
> +ctx = bdrv_get_aio_context(other_job->bs);
> +block_job_completed_single(other_job);
> +aio_context_release(ctx);
> +}
> +}
> +
> +static void block_job_completed_txn_success(BlockJob *job)
> +{
> +AioContext *ctx;
> +BlockJobTxn *txn = job->txn;
> +BlockJob *other_job, *next;
> +/*
> + * Successful completion, see if there are other running jobs in this
> + * txn.
> + */
> +QLIST_FOREACH(other_job, >jobs, txn_list) {
> +if (!other_job->completed) {
> +return;
> +}
> +}
> +/* We are the last completed job, commit the transaction. */
> +QLIST_FOREACH_SAFE(other_job, >jobs, txn_list, next) {
> +ctx = bdrv_get_aio_context(other_job->bs);
> +aio_context_acquire(ctx);
> +assert(other_job->ret == 0);

Here we assert that all jobs have a retcode of zero to be in the success
callback, but ...

> +block_job_completed_single(other_job);
> +aio_context_release(ctx);
> +}
> +}
> +
>  void block_job_completed(BlockJob *job, int ret)
>  {
>  BlockDriverState *bs = job->bs;
> @@ -98,8 +191,13 @@ void block_job_completed(BlockJob *job, int ret)
>  assert(!job->completed);
>  job->completed = true;
>  job->ret = ret;
> -job->cb(job->opaque, ret);
> -block_job_unref(job);
> +if (!job->txn) {
> +block_job_completed_single(job);
> +} else if (ret < 0 || block_job_is_cancelled(job)) {
> +block_job_completed_txn_abort(job);
> +} else {

over here it just 

Re: [Qemu-devel] [PATCH v6 14/14] tests: add BlockJobTxn unit test

[Fam Zheng]

On Fri, 09/18 13:45, Max Reitz wrote:
> On 15.09.2015 08:11, Fam Zheng wrote:
> > From: Stefan Hajnoczi 
> > 
> > The BlockJobTxn unit test verifies that both single jobs and pairs of
> > jobs behave as a transaction group.  Either all jobs complete
> > successfully or the group is cancelled.
> > 
> > Signed-off-by: Stefan Hajnoczi 
> > Signed-off-by: Fam Zheng 
> > Reviewed-by: Max Reitz 
> > ---
> >  tests/Makefile|   3 +
> >  tests/test-blockjob-txn.c | 244 
> > ++
> >  2 files changed, 247 insertions(+)
> >  create mode 100644 tests/test-blockjob-txn.c
> > 
> > diff --git a/tests/Makefile b/tests/Makefile
> > index 34c6136..89ce2f3 100644
> > --- a/tests/Makefile
> > +++ b/tests/Makefile
> > @@ -45,6 +45,8 @@ check-unit-y += tests/test-thread-pool$(EXESUF)
> >  gcov-files-test-thread-pool-y = thread-pool.c
> >  gcov-files-test-hbitmap-y = util/hbitmap.c
> >  check-unit-y += tests/test-hbitmap$(EXESUF)
> > +gcov-files-test-hbitmap-y = blockjob.c
> > +check-unit-y += tests/test-blockjob-txn$(EXESUF)
> >  check-unit-y += tests/test-x86-cpuid$(EXESUF)
> >  # all code tested by test-x86-cpuid is inside topology.h
> >  gcov-files-test-x86-cpuid-y =
> > @@ -294,6 +296,7 @@ tests/test-coroutine$(EXESUF): tests/test-coroutine.o 
> > $(block-obj-y) libqemuutil
> >  tests/test-aio$(EXESUF): tests/test-aio.o $(block-obj-y) libqemuutil.a 
> > libqemustub.a
> >  tests/test-rfifolock$(EXESUF): tests/test-rfifolock.o libqemuutil.a 
> > libqemustub.a
> >  tests/test-throttle$(EXESUF): tests/test-throttle.o $(block-obj-y) 
> > libqemuutil.a libqemustub.a
> > +tests/test-blockjob-txn$(EXESUF): tests/test-blockjob-txn.o $(block-obj-y) 
> > libqemuutil.a libqemustub.a
> >  tests/test-thread-pool$(EXESUF): tests/test-thread-pool.o $(block-obj-y) 
> > libqemuutil.a libqemustub.a
> >  tests/test-iov$(EXESUF): tests/test-iov.o libqemuutil.a
> >  tests/test-hbitmap$(EXESUF): tests/test-hbitmap.o libqemuutil.a 
> > libqemustub.a
> 
> This needs an update due to b124533e. The context is obvious, and I
> guess for the new line itself it's
> s/\$(block-obj-y).*$/$(test-block-obj-y)/.
> 
> If that is indeed so, feel free to keep my R-b.

Yes, thanks for pointint out. Also changed

s/libqemuutil.a libqemustub.a/$(test-util-obj-y)/

with your R-b.

Fam