Re: [Qemu-devel] [PATCH v7 00/11] Add basic "detach" support for dump-guest-memory

[Peter Xu]

On Thu, Feb 18, 2016 at 08:26:44AM +0100, Markus Armbruster wrote:
> Peter Xu  writes:
> 
> > Changes from v7:
> > - patch 8: use s->dump_info.page_size not TARGET_PAGE_SIZE
> > - patch 10: change DUMP_STATUS_MAX to DUMP_STATUS__MAX (this is to
> >   fix compile error for rebasing to latest master branch, still do
> >   not know why we need this change from generating "_MAX" to
> >   "__MAX" for enum types...)
> 
> Don't ask yourself why something was done, ask git :)

Sorry for my laziness... But it seems that it luckily brings me
another "-S" git trick. Lots of things to learn. :-)

> 
> $ git-log -S__MAX
> [...]
> commit 7fb1cf1606c78c9d5b538f29176fd5a101726a9d
> Author: Eric Blake 
> Date:   Wed Nov 18 01:52:57 2015 -0700
> 
> --->qapi: Don't let implicit enum MAX member collide
> 
> Now that we guarantee the user doesn't have any enum values
> beginning with a single underscore, we can use that for our
> own purposes.  Renaming ENUM_MAX to ENUM__MAX makes it obvious
> that the sentinel is generated.
> 
> This patch was mostly generated by applying a temporary patch:
> 
> |diff --git a/scripts/qapi.py b/scripts/qapi.py
> |index e6d014b..b862ec9 100644
> |--- a/scripts/qapi.py
> |+++ b/scripts/qapi.py
> |@@ -1570,6 +1570,7 @@ const char *const %(c_name)s_lookup[] = {
> | max_index = c_enum_const(name, 'MAX', prefix)
> | ret += mcgen('''
> | [%(max_index)s] = NULL,
> |+// %(max_index)s
> | };
> | ''',
> |max_index=max_index)
> 
> then running:
> 
> $ cat qapi-{types,event}.c tests/test-qapi-types.c |
> sed -n 's,^// \(.*\)MAX,s|\1MAX|\1_MAX|g,p' > list
> $ git grep -l _MAX | xargs sed -i -f list
> 
> The only things not generated are the changes in scripts/qapi.py.
> 
> Rejecting enum members named 'MAX' is now useless, and will be dropped
> in the next patch.
> 
> Signed-off-by: Eric Blake 
> Message-Id: <1447836791-369-23-git-send-email-ebl...@redhat.com>
> Reviewed-by: Juan Quintela 
> [Rebased to current master, commit message tweaked]
> Signed-off-by: Markus Armbruster 
> 
> I guess this isn't perfectly clear without more context, so let me add
> some.
> 
> The implicit FOO_MAX could collide with the user's FOO_MAX for QAPI
> enumeration values 'max', or case variations thereof.  Commit cf39359
> added extra code to catch that.
> 
> Renaming the implict FOO_MAX to FOO__MAX removes the need for the
> special rule "you can't use 'max' as an enumeration value", because the
> general rule "names starting with '_' are reserved" suffices then.
> 
> Fewer rules is better.

Yah. The commit is reasonable and explanation is clear. Thanks!

Peter

> 
> [...]

Re: [Qemu-devel] [PATCH] net/filter-redirector:Add filter-redirector

[Zhang Chen]

On 02/18/2016 10:41 AM, Jason Wang wrote:


On 02/05/2016 02:50 PM, Zhang Chen wrote:

From: ZhangChen 

Filter-redirector is a netfilter plugin.
It gives qemu the ability to redirect net packet.
redirector can redirect filter's net packet to outdev.
and redirect indev's packet to filter.

  filter
+
|
|
   redirector   |
+-+
|   | |
|   | |
|   | |
   indev ++ +>  outdev
| |   |
| |   |
| |   |
+-+
  |
  |
  v
   filter


  v

change it to   filter  filter .. guest
It's may more clearly expressed.


usage:

-netdev tap,id=hn0
-chardev socket,id=s0,host=ip_primary,port=X,server,nowait
-chardev socket,id=s1,host=ip_primary,port=Y,server,nowait
-filter-redirector,id=r0,netdev=hn0,queue=tx/rx/all,indev=s0,outdev=s1

Signed-off-by: ZhangChen 
Signed-off-by: Wen Congyang 
---

Thanks a lot for the patch. Like mirror, let's design a unit-test for
this. And what's more, is there any chance to unify the codes? (At least
parts of the codes could be reused).


We can make filter-redirector based on filter-mirror.
if you want to use redirector ,you must open mirror before.
like this:

-netdev tap,id=hn0
-chardev socket,id=mirror0,host=ip_primary,port=X,server,nowait
-filter-mirror,id=m0,netdev=hn0,queue=tx/rx/all,redirector=on,outdev=mirror0
-filter-redirector,id=r0,netdev=hn0,queue=tx/rx/all,indev=s0

How about this?



  net/Makefile.objs   |   1 +
  net/filter-redirector.c | 245 
  qemu-options.hx |   6 ++
  vl.c|   3 +-
  4 files changed, 254 insertions(+), 1 deletion(-)
  create mode 100644 net/filter-redirector.c

diff --git a/net/Makefile.objs b/net/Makefile.objs
index 5fa2f97..f4290a5 100644
--- a/net/Makefile.objs
+++ b/net/Makefile.objs
@@ -15,3 +15,4 @@ common-obj-$(CONFIG_VDE) += vde.o
  common-obj-$(CONFIG_NETMAP) += netmap.o
  common-obj-y += filter.o
  common-obj-y += filter-buffer.o
+common-obj-y += filter-redirector.o
diff --git a/net/filter-redirector.c b/net/filter-redirector.c
new file mode 100644
index 000..364e463
--- /dev/null
+++ b/net/filter-redirector.c
@@ -0,0 +1,245 @@
+/*
+ * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
+ * Copyright (c) 2016 FUJITSU LIMITED
+ * Copyright (c) 2016 Intel Corporation
+ *
+ * Author: Zhang Chen 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.
+ */
+
+#include "net/filter.h"
+#include "net/net.h"
+#include "qemu-common.h"
+#include "qapi/qmp/qerror.h"
+#include "qapi-visit.h"
+#include "qom/object.h"
+#include "qemu/main-loop.h"
+#include "qemu/error-report.h"
+#include "trace.h"
+#include "sysemu/char.h"
+#include "qemu/iov.h"
+#include "qemu/sockets.h"
+
+#define FILTER_REDIRECTOR(obj) \
+OBJECT_CHECK(RedirectorState, (obj), TYPE_FILTER_REDIRECTOR)
+
+#define TYPE_FILTER_REDIRECTOR "filter-redirector"
+#define REDIRECT_HEADER_LEN sizeof(uint32_t)
+
+typedef struct RedirectorState {
+NetFilterState parent_obj;
+NetQueue *incoming_queue;/* guest normal net queue */

The comment looks unless and maybe even wrong when queue=rx?


We design redirector that indev's data always be passed to guest finally.
so, It's no relation between the queue=rx/tx/all. just related to indev 
= xxx.

we need incoming_queue to inject packet from indev.



+char *indev;
+char *outdev;
+CharDriverState *chr_in;
+CharDriverState *chr_out;
+} RedirectorState;
+
+static ssize_t filter_redirector_send(NetFilterState *nf,
+   const struct iovec *iov,
+   int iovcnt)
+{
+RedirectorState *s = FILTER_REDIRECTOR(nf);
+ssize_t ret = 0;
+ssize_t size = 0;
+uint32_t len =  0;
+char *buf;
+
+size = iov_size(iov, iovcnt);
+len = htonl(size);
+if (!size) {
+return 0;
+}
+
+buf = g_malloc0(size);
+iov_to_buf(iov, iovcnt, 0, buf, size);
+ret = qemu_chr_fe_write_all(s->chr_out, (uint8_t *), sizeof(len));
+if (ret < 0) {

Similar to mirror, need check the return value against sizeof(len). And
the code could be simplified with something like:

goto out;

...

out:
 g_free(buf);
 return ret;

And you can see another advantages of unifying the codes 

Re: [Qemu-devel] [PATCH] net: check packet payload length

[P J P]

+-- On Thu, 18 Feb 2016, P J P wrote --+
| +-- On Wed, 17 Feb 2016, Markus Armbruster wrote --+
| | Is calling this function with a partial IPv4 TCP/UDP packet legitimate? 
| | If partial packet is okay, what about a partial header?
| 
|   Partial? That shouldn't harm I guess.

  For partial TCP/UDP headers it does return with no checksum.

if (plen < csum_offset+2)
return;
 
--
Prasad J Pandit / Red Hat Product Security Team
47AF CE69 3A90 54AA 9045 1053 DD13 3D32 FE5B 041F

Re: [Qemu-devel] [RFC] QMP: add query-hotpluggable-cpus

[Markus Armbruster]

David Gibson  writes:

> On Tue, Feb 16, 2016 at 01:35:42PM +0100, Markus Armbruster wrote:
>> Igor Mammedov  writes:
>> 
>> > On Mon, 15 Feb 2016 20:43:41 +0100
>> > Markus Armbruster  wrote:
>> >
>> >> Igor Mammedov  writes:
>> >> 
>> >> > it will allow mgmt to query present and possible to hotplug CPUs
>> >> > it is required from a target platform that wish to support
>> >> > command to set board specific MachineClass.possible_cpus() hook,
>> >> > which will return a list of possible CPUs with options
>> >> > that would be needed for hotplugging possible CPUs.
>> >> >
>> >> > For RFC there are:
>> >> >'arch_id': 'int' - mandatory unique CPU number,
>> >> >   for x86 it's APIC ID for ARM it's MPIDR
>> >> >'type': 'str' - CPU object type for usage with device_add
>> >> >
>> >> > and a set of optional fields that would allows mgmt tools
>> >> > to know at what granularity and where a new CPU could be
>> >> > hotplugged;
>> >> > [node],[socket],[core],[thread]
>> >> > Hopefully that should cover needs for CPU hotplug porposes for
>> >> > magor targets and we can extend structure in future adding
>> >> > more fields if it will be needed.
>> >> >
>> >> > also for present CPUs there is a 'cpu_link' field which
>> >> > would allow mgmt inspect whatever object/abstraction
>> >> > the target platform considers as CPU object.
>> >> >
>> >> > For RFC purposes implements only for x86 target so far.  
>> >> 
>> >> Adding ad hoc queries as we go won't scale.  Could this be solved by a
>> >> generic introspection interface?
>> > Do you mean generic QOM introspection?
>> 
>> Possibly, but I don't want to prematurely limit the conversation to QOM
>> introspection.
>> 
>> > Using QOM we could have '/cpus' container and create QOM links
>> > for exiting (populated links) and possible (empty links) CPUs.
>> > However in that case link's name will need have a special format
>> > that will convey an information necessary for mgmt to hotplug
>> > a CPU object, at least:
>> >   - where: [node],[socket],[core],[thread] options
>> >   - optionally what CPU object to use with device_add command
>> 
>> Encoding information in names feels wrong.
>
> Yeah :(.
>
>> > Another approach to do QOM introspection would be to model hierarchy 
>> > of objects like node/socket/core..., That's what Andreas
>> > worked on. Only it still suffers the same issue as above
>> > wrt introspection and hotplug, One can pre-create empty
>> > [nodes][sockets[cores]] containers at startup but then
>> > leaf nodes that could be hotplugged would be a links anyway
>> > and then again we need to give them special formatted names
>> > (not well documented at that mgmt could make sense of).
>> > That hierarchy would need to become stable ABI once
>> > mgmt will start using it and QOM tree is quite unstable
>> > now for that. For some targets it involves creating dummy
>> > containers like node/socket/core for x86 where just modeling
>> > a thread is sufficient.
>> 
>> I acknowledge your concern regarding QOM tree stability.  We have QOM
>> introspection commands since 1.2.  They make the QOM tree part of the
>> external interface, but we've never spelled out which parts of it (if
>> any) are ABI.  Until we do, parts become de facto ABI by being used in
>> anger.  As a result, we don't know something's ABI until it breaks.
>> 
>> Andreas, do you have an opinion on proper use of QOM by external
>> software?
>> 
>> > The similar but a bit more abstract approach was suggested
>> > by David https://lists.gnu.org/archive/html/qemu-ppc/2016-02/msg0.html
>> 
>> Cc'ing him.
>
> Actually I was already on the thread via my upstream email.
>
>> If I understand the high-level idea correctly, David
>> proposes to have an abstract type cpu-package with generic properties.
>> Its concrete subtypes are composed of whatever components make up the
>> hot-pluggable unit.
>
> Yes, that's pretty much it.
>
>> Management software can then use the generic properties to deal with hot
>> plug without having to know about the concrete subtypes, at least to
>> some useful degree.
>
> That's the plan.
>
>> Similarly, the generic properties suffice for implementing generic
>> high-level interfaces like -smp.
>
> Here it gets a bit fuzzier.  The idea is that the abstract type would
> still make sense in a post -smp world allowing heterogenous setups.
> However the concrete subtypes used for the time being are likely to
> get their configuration from -smp, whether directly or indirectly.
>
> My preferred option was for the machine type to "push" the smp
> configuration into the package objects, rather than having them look
> at the global directly.  However, in working with Bharata on a draft
> implementation, I'm not actually sure how to do that.  So we might end
> up looking at the global from the (concrete) package at least for the
> time being.
>
> At least, that's how 

Re: [Qemu-devel] [PATCH COLO-Frame v14 37/40] COLO: enable buffer filters for PVM

[Hailiang Zhang]

Hi Jason,

On 2016/2/18 11:46, Hailiang Zhang wrote:

On 2016/2/18 11:31, Jason Wang wrote:



On 02/06/2016 05:28 PM, zhanghailiang wrote:

Enable all buffer filters that added by COLO while
go into COLO process, and disable them while exit COLO.

Signed-off-by: zhanghailiang 
Cc: Jason Wang 
Cc: Yang Hongyang 
---
v14:
- New patch
---
  migration/colo.c | 32 
  1 file changed, 32 insertions(+)

diff --git a/migration/colo.c b/migration/colo.c
index 235578b..86a7638 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -104,10 +104,26 @@ static void secondary_vm_do_failover(void)
  }
  }

+static void colo_set_filter_status(NetFilterState *nf, void *opaque,
+   Error **errp)
+{
+char colo_filter[128];
+char *name = object_get_canonical_path_component(OBJECT(nf));
+char *status = opaque;
+
+snprintf(colo_filter, sizeof(colo_filter), "%scolo", nf->netdev_id);
+if (strcmp(colo_filter, name)) {
+return;
+}


Checking by name is not elegant. As we've discussed last time, why not
let filter-buffer track all filters with zero interval in a linked list
and just export a helper to disable and enable them all? Things will be
greatly simplified with this, and there's even no need for patch 36.




Yes, i know what you mean, but we have to add another
'QTAILQ_ENTRY() entry' like member into struct NetFilterState
if we do like that, is it acceptable ?



Sorry for the hasty reply, ;)

We can use a list to store all the colo related buffer filters
without adding any member to struct NetFilterState.

The codes will be like:

struct COLOListNode{
void *opaque;
QLIST_ENTRY(COLOListNode) node;
};
static QLIST_HEAD(, COLOListNode) COLOBufferFilters =
   QLIST_HEAD_INITIALIZER(COLOBufferFilters);

void colo_add_buffer_filter()
{
struct COLOListNode *filternode;
...

filter = object_new_with_props();

filternode = g_new0(struct COLOListNode, 1);
filternode->opaque = NETFILTER(filter);
QLIST_INSERT_HEAD(, filternode, node);
}

And we can track all the colo releated filters directly by
visiting the *COLOBufferFilters* list.

Thanks,
Hailiang


.

Re: [Qemu-devel] [PATCH v7 00/11] Add basic "detach" support for dump-guest-memory

[Markus Armbruster]

Peter Xu  writes:

> Changes from v7:
> - patch 8: use s->dump_info.page_size not TARGET_PAGE_SIZE
> - patch 10: change DUMP_STATUS_MAX to DUMP_STATUS__MAX (this is to
>   fix compile error for rebasing to latest master branch, still do
>   not know why we need this change from generating "_MAX" to
>   "__MAX" for enum types...)

Don't ask yourself why something was done, ask git :)

$ git-log -S__MAX
[...]
commit 7fb1cf1606c78c9d5b538f29176fd5a101726a9d
Author: Eric Blake 
Date:   Wed Nov 18 01:52:57 2015 -0700

--->qapi: Don't let implicit enum MAX member collide

Now that we guarantee the user doesn't have any enum values
beginning with a single underscore, we can use that for our
own purposes.  Renaming ENUM_MAX to ENUM__MAX makes it obvious
that the sentinel is generated.

This patch was mostly generated by applying a temporary patch:

|diff --git a/scripts/qapi.py b/scripts/qapi.py
|index e6d014b..b862ec9 100644
|--- a/scripts/qapi.py
|+++ b/scripts/qapi.py
|@@ -1570,6 +1570,7 @@ const char *const %(c_name)s_lookup[] = {
| max_index = c_enum_const(name, 'MAX', prefix)
| ret += mcgen('''
| [%(max_index)s] = NULL,
|+// %(max_index)s
| };
| ''',
|max_index=max_index)

then running:

$ cat qapi-{types,event}.c tests/test-qapi-types.c |
sed -n 's,^// \(.*\)MAX,s|\1MAX|\1_MAX|g,p' > list
$ git grep -l _MAX | xargs sed -i -f list

The only things not generated are the changes in scripts/qapi.py.

Rejecting enum members named 'MAX' is now useless, and will be dropped
in the next patch.

Signed-off-by: Eric Blake 
Message-Id: <1447836791-369-23-git-send-email-ebl...@redhat.com>
Reviewed-by: Juan Quintela 
[Rebased to current master, commit message tweaked]
Signed-off-by: Markus Armbruster 

I guess this isn't perfectly clear without more context, so let me add
some.

The implicit FOO_MAX could collide with the user's FOO_MAX for QAPI
enumeration values 'max', or case variations thereof.  Commit cf39359
added extra code to catch that.

Renaming the implict FOO_MAX to FOO__MAX removes the need for the
special rule "you can't use 'max' as an enumeration value", because the
general rule "names starting with '_' are reserved" suffices then.

Fewer rules is better.

[...]

[Qemu-devel] [PATCH v11 15/15] qapi: Change visit_start_implicit_struct to visit_start_alternate

[Eric Blake]

After recent changes, the only remaining use of
visit_start_implicit_struct() is for allocating the space needed
when visiting an alternate.  Since the term 'implicit struct' is
hard to explain, rename the function to its current usage.  While
at it, we can merge the functionality of visit_get_next_type()
into the same function, making it more like visit_start_struct().

Generated code is now slightly smaller:

| {
| Error *err = NULL;
|
|-visit_start_implicit_struct(v, (void**) obj, sizeof(BlockdevRef), );
|+visit_start_alternate(v, name, (GenericAlternate **)obj, sizeof(**obj),
|+  true, );
| if (err) {
| goto out;
| }
|-visit_get_next_type(v, name, &(*obj)->type, true, );
|-if (err) {
|-goto out_obj;
|-}
| switch ((*obj)->type) {
| case QTYPE_QDICT:
| visit_start_struct(v, name, NULL, 0, );
...
| }
|-out_obj:
|-visit_end_implicit_struct(v);
|+visit_end_alternate(v);
| out:
| error_propagate(errp, err);
| }

Signed-off-by: Eric Blake 

---
v11: rebase to earlier changes
v10: new patch
---
 include/qapi/visitor.h  | 49 +++--
 include/qapi/visitor-impl.h | 17 
 scripts/qapi-visit.py   | 10 +++--
 qapi/qapi-visit-core.c  | 40 
 qapi/qapi-dealloc-visitor.c | 13 ++--
 qapi/qmp-input-visitor.c| 24 +-
 6 files changed, 82 insertions(+), 71 deletions(-)

diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index a6678b1..83cad74 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -27,17 +27,52 @@ typedef struct GenericList {
 char padding[];
 } GenericList;

+/* This struct is layout-compatible with all Alternate types
+ * created by the qapi generator. */
+typedef struct GenericAlternate {
+QType type;
+char padding[];
+} GenericAlternate;
+
 void visit_start_struct(Visitor *v, const char *name, void **obj,
 size_t size, Error **errp);
 void visit_end_struct(Visitor *v, Error **errp);
-void visit_start_implicit_struct(Visitor *v, void **obj, size_t size,
- Error **errp);
-void visit_end_implicit_struct(Visitor *v);

 void visit_start_list(Visitor *v, const char *name, Error **errp);
 GenericList *visit_next_list(Visitor *v, GenericList **list, size_t size);
 void visit_end_list(Visitor *v);

+/*
+ * Start the visit of an alternate @obj with the given @size.
+ *
+ * @name specifies the relationship to the containing struct (ignored
+ * for a top level visit, the name of the key if this alternate is
+ * part of an object, or NULL if this alternate is part of a list).
+ *
+ * @obj must not be NULL. Input visitors will allocate @obj and
+ * determine the qtype of the next thing to be visited, stored in
+ * (*@obj)->type.  Other visitors will leave @obj unchanged.
+ *
+ * If @promote_int, treat integers as QTYPE_FLOAT.
+ *
+ * If successful, this must be paired with visit_end_alternate(), even
+ * if visiting the contents of the alternate fails.
+ */
+void visit_start_alternate(Visitor *v, const char *name,
+   GenericAlternate **obj, size_t size,
+   bool promote_int, Error **errp);
+
+/*
+ * Finish visiting an alternate type.
+ *
+ * Must be called after a successful visit_start_alternate(), even if
+ * an error occurred in the meantime.
+ *
+ * TODO: Should all the visit_end_* interfaces take obj parameter, so
+ * that dealloc visitor need not track what was passed in visit_start?
+ */
+void visit_end_alternate(Visitor *v);
+
 /**
  * Check if an optional member @name of an object needs visiting.
  * For input visitors, set *@present according to whether the
@@ -46,14 +81,6 @@ void visit_end_list(Visitor *v);
  */
 bool visit_optional(Visitor *v, const char *name, bool *present);

-/**
- * Determine the qtype of the item @name in the current object visit.
- * For input visitors, set *@type to the correct qtype of a qapi
- * alternate type; for other visitors, leave *@type unchanged.
- * If @promote_int, treat integers as QTYPE_FLOAT.
- */
-void visit_get_next_type(Visitor *v, const char *name, QType *type,
- bool promote_int, Error **errp);
 void visit_type_enum(Visitor *v, const char *name, int *obj,
  const char *const strings[], Error **errp);
 void visit_type_int(Visitor *v, const char *name, int64_t *obj, Error **errp);
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index c4af3e0..6a1ddfb 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -22,22 +22,23 @@ struct Visitor
  size_t size, Error **errp);
 void (*end_struct)(Visitor *v, Error **errp);

-void (*start_implicit_struct)(Visitor *v, void **obj, size_t size,
-  Error **errp);
-/* May be NULL */
-  

[Qemu-devel] [PATCH v11 09/15] qapi: Adjust layout of FooList types

[Eric Blake]

By sticking the next pointer first, we don't need a union with
64-bit padding for smaller types.  On 32-bit platforms, this
can reduce the size of uint8List from 16 bytes (or 12, depending
on whether 64-bit ints can tolerate 4-byte alignment) down to 8.
It has no effect on 64-bit platforms (where alignment still
dictates a 16-byte struct); but fewer anonymous unions is still
a win in my book.

It requires visit_next_list() to gain a size parameter, to know
what size element to allocate; comparable to the size parameter
of visit_start_struct().

I debated about going one step further, to allow for fewer casts,
by doing:
typedef GenericList GenericList;
struct GenericList {
GenericList *next;
};
struct FooList {
GenericList base;
Foo *value;
};
so that you convert to 'GenericList *' by '>base', and
back by 'container_of(generic, GenericList, base)' (as opposed to
the existing '(GenericList *)foolist' and '(FooList *)generic').
But doing that would require hoisting the declaration of
GenericList prior to inclusion of qapi-types.h, rather than its
current spot in visitor.h; it also makes iteration a bit more
verbose through 'foolist->base.next' instead of 'foolist->next'.

Note that for lists of objects, the 'value' payload is still
hidden behind a boxed pointer.  Someday, it would be nice to do:

struct FooList {
FooList *next;
Foo value;
};

for one less level of malloc for each list element.  This patch
is a step in that direction (now that 'next' is no longer at a
fixed non-zero offset within the struct, we can store more than
just a pointer's-worth of data as the value payload), but the
actual conversion would be a task for another series, as it will
touch a lot of code.

Signed-off-by: Eric Blake 

---
v11: assert that size is sane, improve commit message
v10: hoist earlier in series
v9: no change
v8: rebase to earlier changes
v7: new patch; probably too invasive to be worth it, especially if
we can't prove that our current size for uint8List is a bottleneck
---
 include/qapi/visitor.h   | 13 ++---
 include/qapi/visitor-impl.h  |  2 +-
 scripts/qapi-types.py|  5 +
 scripts/qapi-visit.py|  2 +-
 qapi/qapi-visit-core.c   |  5 +++--
 qapi/opts-visitor.c  |  4 ++--
 qapi/qapi-dealloc-visitor.c  |  3 ++-
 qapi/qmp-input-visitor.c |  5 +++--
 qapi/qmp-output-visitor.c|  3 ++-
 qapi/string-input-visitor.c  |  4 ++--
 qapi/string-output-visitor.c |  2 +-
 11 files changed, 24 insertions(+), 24 deletions(-)

diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 5e581dc..8a2d5cc 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -19,13 +19,12 @@
 #include "qapi/error.h"
 #include 

-typedef struct GenericList
-{
-union {
-void *value;
-uint64_t padding;
-};
+/* This struct is layout-compatible with all other *List structs
+ * created by the qapi generator.  It is used as a typical
+ * singly-linked list. */
+typedef struct GenericList {
 struct GenericList *next;
+char padding[];
 } GenericList;

 void visit_start_struct(Visitor *v, const char *name, void **obj,
@@ -36,7 +35,7 @@ void visit_start_implicit_struct(Visitor *v, void **obj, 
size_t size,
 void visit_end_implicit_struct(Visitor *v);

 void visit_start_list(Visitor *v, const char *name, Error **errp);
-GenericList *visit_next_list(Visitor *v, GenericList **list);
+GenericList *visit_next_list(Visitor *v, GenericList **list, size_t size);
 void visit_end_list(Visitor *v);

 /**
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index ea252f8..7905a28 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -29,7 +29,7 @@ struct Visitor

 void (*start_list)(Visitor *v, const char *name, Error **errp);
 /* Must be set */
-GenericList *(*next_list)(Visitor *v, GenericList **list);
+GenericList *(*next_list)(Visitor *v, GenericList **list, size_t size);
 /* Must be set */
 void (*end_list)(Visitor *v);

diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 7b0dca8..83f230a 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -26,11 +26,8 @@ def gen_array(name, element_type):
 return mcgen('''

 struct %(c_name)s {
-union {
-%(c_type)s value;
-uint64_t padding;
-};
 %(c_name)s *next;
+%(c_type)s value;
 };
 ''',
  c_name=c_name(name), c_type=element_type.c_type())
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 231e424..48ebd2b 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -173,7 +173,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, 
%(c_name)s **obj, Error
 }

 for (prev = (GenericList **)obj;
- !err && (i = visit_next_list(v, prev)) != NULL;
+ !err && (i = visit_next_list(v, prev, sizeof(**obj))) != NULL;
  prev = ) {
 %(c_name)s *native_i = 

[Qemu-devel] [PATCH v11 12/15] qapi: Don't box struct branch of alternate

[Eric Blake]

There's no reason to do two malloc's for an alternate type visiting
a QAPI struct; let's just inline the struct directly as the C union
branch of the struct.

Surprisingly, no clients were actually using the struct member prior
to this patch outside of the testsuite; an earlier patch in the series
added some testsuite coverage to make the effect of this patch more
obvious.

In qapi.py, c_type() gains a new is_unboxed flag to control when we
are emitting a C struct unboxed within the context of an outer
struct (different from our other two modes of usage with no flags
for normal local variable declarations, and with is_param for adding
'const' in a parameter list).  I don't know if there is any more
pythonic way of collapsing the two flags into a single parameter,
as we never have a caller setting both flags at once.

Ultimately, we want to also unbox branches for QAPI unions, but as
that touches a lot more client code, it is better as separate
patches.  But since unions and alternates share gen_variants(), I
had to hack in a way to test if we are visiting an alternate type
for setting the is_unboxed flag: look for a non-object branch.
This works because alternates have at least two branches, with at
most one object branch, while unions have only object branches.
The hack will go away in a later patch.

The generated code difference to qapi-types.h is relatively small:

| struct BlockdevRef {
| QType type;
| union { /* union tag is @type */
| void *data;
|-BlockdevOptions *definition;
|+BlockdevOptions definition;
| char *reference;
| } u;
| };

meanwhile, in qapi-visit.h, we trade the previous visit_type_FOO(obj)
(which allocates a pointer, handles a layer of {} from the JSON stream,
and visits the fields) with an inline call to visit_type_FOO(NULL) (to
consume the {} without allocation) and a direct visit of the fields:

| switch ((*obj)->type) {
| case QTYPE_QDICT:
|-visit_type_BlockdevOptions(v, name, &(*obj)->u.definition, );
|+visit_start_struct(v, name, NULL, 0, );
|+if (err) {
|+break;
|+}
|+visit_type_BlockdevOptions_fields(v, &(*obj)->u.definition, );
|+error_propagate(errp, err);
|+err = NULL;
|+visit_end_struct(v, );
| break;
| case QTYPE_QSTRING:
| visit_type_str(v, name, &(*obj)->u.reference, );

The visit of non-object fields is unchanged.

Signed-off-by: Eric Blake 

---
v11: drop visit_type_alternate_FOO(), improve commit message,
fix and test bug regarding visits to a union, rework some of
the hacks to make them more obvious
v10: new patch
---
 scripts/qapi.py | 12 +++-
 scripts/qapi-types.py   | 10 +-
 scripts/qapi-visit.py   | 33 +++--
 tests/test-qmp-input-visitor.c  | 20 ++--
 tests/test-qmp-output-visitor.c | 11 +--
 5 files changed, 58 insertions(+), 28 deletions(-)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 17bf633..849 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -824,7 +824,7 @@ class QAPISchemaVisitor(object):


 class QAPISchemaType(QAPISchemaEntity):
-def c_type(self, is_param=False):
+def c_type(self, is_param=False, is_unboxed=False):
 return c_name(self.name) + pointer_suffix

 def c_null(self):
@@ -857,7 +857,7 @@ class QAPISchemaBuiltinType(QAPISchemaType):
 def c_name(self):
 return self.name

-def c_type(self, is_param=False):
+def c_type(self, is_param=False, is_unboxed=False):
 if is_param and self.name == 'str':
 return 'const ' + self._c_type_name
 return self._c_type_name
@@ -891,7 +891,7 @@ class QAPISchemaEnumType(QAPISchemaType):
 # See QAPISchema._make_implicit_enum_type()
 return self.name.endswith('Kind')

-def c_type(self, is_param=False):
+def c_type(self, is_param=False, is_unboxed=False):
 return c_name(self.name)

 def member_names(self):
@@ -987,9 +987,11 @@ class QAPISchemaObjectType(QAPISchemaType):
 assert not self.is_implicit()
 return QAPISchemaType.c_name(self)

-def c_type(self, is_param=False):
+def c_type(self, is_param=False, is_unboxed=False):
 assert not self.is_implicit()
-return QAPISchemaType.c_type(self)
+if is_unboxed:
+return c_name(self.name)
+return c_name(self.name) + pointer_suffix

 def json_type(self):
 return 'object'
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 6ea0ae6..4dabe91 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -116,6 +116,14 @@ static inline %(base)s *qapi_%(c_name)s_base(const 
%(c_name)s *obj)


 def gen_variants(variants):
+# HACK: Determine if this is an alternate (at least one variant
+# is not an object); unions have all branches as objects.
+unboxed = False
+for v in variants.variants:
+ 

[Qemu-devel] [PATCH v11 10/15] qapi: Emit structs used as variants in topological order

[Eric Blake]

Right now, we emit the branches of union types as a boxed pointer,
and it suffices to have a forward declaration of the type.  However,
a future patch will swap things to directly use the branch type,
instead of hiding it behind a pointer.  For this to work, the
compiler needs the full definition of the type, not just a forward
declaration, prior to the union that is including the branch type.
This patch just adds topological sorting to hoist all types
mentioned in a branch of a union to be fully declared before the
union itself.  The sort is always possible, because we do not
allow circular union types that include themselves as a direct
branch (it is, however, still possible to include a branch type
that itself has a pointer to the union, for a type that can
indirectly recursively nest itself - that remains safe, because
that the member of the branch type will remain a pointer, and the
QMP representation of such a type adds another {} for each recurring
layer of the union type).

Signed-off-by: Eric Blake 

---
v11: pep8 cleanup
v10: new patch
---
 scripts/qapi-types.py | 23 ---
 1 file changed, 20 insertions(+), 3 deletions(-)

diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 83f230a..6ea0ae6 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -2,7 +2,7 @@
 # QAPI types generator
 #
 # Copyright IBM, Corp. 2011
-# Copyright (c) 2013-2015 Red Hat Inc.
+# Copyright (c) 2013-2016 Red Hat Inc.
 #
 # Authors:
 #  Anthony Liguori 
@@ -14,6 +14,11 @@
 from qapi import *


+# variants must be emitted before their container; track what has already
+# been output
+objects_seen = set()
+
+
 def gen_fwd_object_or_array(name):
 return mcgen('''

@@ -49,11 +54,23 @@ def gen_struct_fields(members):


 def gen_object(name, base, members, variants):
-ret = mcgen('''
+if name in objects_seen:
+return ''
+objects_seen.add(name)
+
+ret = ''
+if variants:
+for v in variants.variants:
+if (isinstance(v.type, QAPISchemaObjectType) and
+not v.type.is_implicit()):
+ret += gen_object(v.type.name, v.type.base,
+  v.type.local_members, v.type.variants)
+
+ret += mcgen('''

 struct %(c_name)s {
 ''',
-c_name=c_name(name))
+ c_name=c_name(name))

 if base:
 ret += mcgen('''
-- 
2.5.0

[Qemu-devel] [PATCH v11 13/15] qapi: Don't box branches of flat unions

[Eric Blake]

There's no reason to do two malloc's for a flat union; let's just
inline the branch struct directly into the C union branch of the
flat union.

Surprisingly, fewer clients were actually using explicit references
to the branch types in comparison to the number of flat unions
thus modified.

This lets us reduce the hack in qapi-types:gen_variants() added in
the previous patch; we no longer need to distinguish between
alternates and flat unions.

The change to unboxed structs means that u.data (added in commit
cee2dedb) is now coincident with random fields of each branch of
the flat union, whereas beforehand it was only coincident with
pointers (since all branches of a flat union have to be objects).
Note that this was already the case for simple unions - but there
we got lucky.  Remember, visit_start_union() blindly returns true
for all visitors except for the dealloc visitor, where it returns
the value !!obj->u.data, and that this result then controls
whether to proceed with the visit to the variant.  Pre-patch,
this meant that flat unions were testing whether the boxed pointer
was still NULL, and thereby skipping visit_end_implicit_struct()
and avoiding a NULL dereference if the pointer had not been
allocated.  The same was true for simple unions where the current
branch had pointer type, except there we bypassed visit_type_FOO().
But for simple unions where the current branch had scalar type, the
contents of that scalar meant that the decision to call
visit_type_FOO() was data-dependent - the reason we got lucky there
is that visit_type_FOO() for all scalar types in the dealloc visitor
is a no-op (only the pointer variants had anything to free), so it
did not matter whether the dealloc visit was skipped.  But with this
patch, we would risk leaking memory if we could skip a call to
visit_type_FOO_fields() based solely on a data-dependent decision.

But notice: in the dealloc visitor, visit_type_FOO() already handles
a NULL obj - it was only the visit_type_implicit_FOO() that was
failing to check for NULL. And now that we have refactored things to
have the branch be part of the parent struct, we no longer have a
separate pointer that can be NULL in the first place.  So we can just
delete the call to visit_start_union() altogether, and blindly visit
the branch type; there is no change in behavior except to the dealloc
visitor, where we now unconditionally visit the branch, but where that
visit is now always safe (for a flat union, we can no longer
dereference NULL, and for a simple union, visit_type_FOO() was already
safely handling NULL on pointer types).

Unfortunately, simple unions are not as easy to switch to unboxed
layout; because we are special-casing the hidden implicit type with
a single 'data' member, we really DO need to keep calling another
layer of visit_start_struct(), with a second malloc; although there
are some cleanups planned for simple unions in later patches.

Note that after this patch, visit_start_union() is unused, and the
only remaining use of visit_start_implicit_struct() is for alternate
types; the next couple of patches will do further cleanups based on
that fact.

Signed-off-by: Eric Blake 

---
v11: rebase to earlier changes, save cleanups for later, fix bug
with visit_start_union now being actively wrong
v10: new patch
---
 scripts/qapi-types.py   | 13 +++--
 scripts/qapi-visit.py   |  7 ++-
 cpus.c  | 18 ++
 hmp.c   | 12 ++--
 tests/test-qmp-input-visitor.c  | 10 +-
 tests/test-qmp-output-visitor.c |  6 ++
 6 files changed, 24 insertions(+), 42 deletions(-)

diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 4dabe91..eac90d2 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -116,14 +116,6 @@ static inline %(base)s *qapi_%(c_name)s_base(const 
%(c_name)s *obj)


 def gen_variants(variants):
-# HACK: Determine if this is an alternate (at least one variant
-# is not an object); unions have all branches as objects.
-unboxed = False
-for v in variants.variants:
-if not isinstance(v.type, QAPISchemaObjectType):
-unboxed = True
-break
-
 # FIXME: What purpose does data serve, besides preventing a union that
 # has a branch named 'data'? We use it in qapi-visit.py to decide
 # whether to bypass the switch statement if visiting the discriminator
@@ -140,11 +132,12 @@ def gen_variants(variants):

 for var in variants.variants:
 # Ugly special case for simple union TODO get rid of it
-typ = var.simple_union_type() or var.type
+simple_union_type = var.simple_union_type()
+typ = simple_union_type or var.type
 ret += mcgen('''
 %(c_type)s %(c_name)s;
 ''',
- c_type=typ.c_type(is_unboxed=unboxed),
+ c_type=typ.c_type(is_unboxed=not simple_union_type),
  

[Qemu-devel] [PATCH v11 08/15] qapi-visit: Less indirection in visit_type_Foo_fields()

[Eric Blake]

We were passing 'Foo **obj' to the internal helper function, but
all uses within the helper were via reads of '*obj'.  Refactor
things to pass one less level of indirection, by having the
callers dereference before calling.

For an example of the generated code change:

|-static void visit_type_BalloonInfo_fields(Visitor *v, BalloonInfo **obj, 
Error **errp)
|+static void visit_type_BalloonInfo_fields(Visitor *v, BalloonInfo *obj, Error 
**errp)
| {
| Error *err = NULL;
|
|-visit_type_int(v, "actual", &(*obj)->actual, );
|+visit_type_int(v, "actual", >actual, );
| error_propagate(errp, err);
| }
|
|@@ -261,7 +261,7 @@ void visit_type_BalloonInfo(Visitor *v,
| if (!*obj) {
| goto out_obj;
| }
|-visit_type_BalloonInfo_fields(v, obj, );
|+visit_type_BalloonInfo_fields(v, *obj, );
| out_obj:

The refactoring will also make it easier to reuse the helpers in
a future patch when implicit structs are stored directly in the
parent struct rather than boxed through a pointer.

Signed-off-by: Eric Blake 

---
v11: rebase to include variants, drop skiplast handling
v10: new patch
---
 scripts/qapi-visit.py | 20 ++--
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 4650c5b..231e424 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -39,7 +39,7 @@ def gen_visit_fields_decl(typ):
 if typ.name not in struct_fields_seen:
 ret += mcgen('''

-static void visit_type_%(c_type)s_fields(Visitor *v, %(c_type)s **obj, Error 
**errp);
+static void visit_type_%(c_type)s_fields(Visitor *v, %(c_type)s *obj, Error 
**errp);
 ''',
  c_type=typ.c_name())
 struct_fields_seen.add(typ.name)
@@ -61,7 +61,7 @@ static void visit_type_implicit_%(c_type)s(Visitor *v, 
%(c_type)s **obj, Error *

 visit_start_implicit_struct(v, (void **)obj, sizeof(%(c_type)s), );
 if (!err) {
-visit_type_%(c_type)s_fields(v, obj, errp);
+visit_type_%(c_type)s_fields(v, *obj, errp);
 visit_end_implicit_struct(v);
 }
 error_propagate(errp, err);
@@ -85,7 +85,7 @@ def gen_visit_struct_fields(name, base, members, 
variants=None):
 struct_fields_seen.add(name)
 ret += mcgen('''

-static void visit_type_%(c_name)s_fields(Visitor *v, %(c_name)s **obj, Error 
**errp)
+static void visit_type_%(c_name)s_fields(Visitor *v, %(c_name)s *obj, Error 
**errp)
 {
 Error *err = NULL;

@@ -94,19 +94,19 @@ static void visit_type_%(c_name)s_fields(Visitor *v, 
%(c_name)s **obj, Error **e

 if base:
 ret += mcgen('''
-visit_type_%(c_type)s_fields(v, (%(c_type)s **)obj, );
+visit_type_%(c_type)s_fields(v, (%(c_type)s *)obj, );
 ''',
  c_type=base.c_name())
 ret += gen_err_check()

-ret += gen_visit_fields(members, prefix='(*obj)->')
+ret += gen_visit_fields(members, prefix='obj->')

 if variants:
 ret += mcgen('''
-if (!visit_start_union(v, !!(*obj)->u.data, ) || err) {
+if (!visit_start_union(v, !!obj->u.data, ) || err) {
 goto out;
 }
-switch ((*obj)->%(c_name)s) {
+switch (obj->%(c_name)s) {
 ''',
  c_name=c_name(variants.tag_member.name))

@@ -121,13 +121,13 @@ static void visit_type_%(c_name)s_fields(Visitor *v, 
%(c_name)s **obj, Error **e
variants.tag_member.type.prefix))
 if simple_union_type:
 ret += mcgen('''
-visit_type_%(c_type)s(v, "data", &(*obj)->u.%(c_name)s, );
+visit_type_%(c_type)s(v, "data", >u.%(c_name)s, );
 ''',
  c_type=simple_union_type.c_name(),
  c_name=c_name(var.name))
 else:
 ret += mcgen('''
-visit_type_implicit_%(c_type)s(v, &(*obj)->u.%(c_name)s, );
+visit_type_implicit_%(c_type)s(v, >u.%(c_name)s, );
 ''',
  c_type=var.type.c_name(),
  c_name=c_name(var.name))
@@ -270,7 +270,7 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, 
%(c_name)s **obj, Error
 if (!*obj) {
 goto out_obj;
 }
-visit_type_%(c_name)s_fields(v, obj, );
+visit_type_%(c_name)s_fields(v, *obj, );
 error_propagate(errp, err);
 err = NULL;
 out_obj:
-- 
2.5.0

[Qemu-devel] [PATCH v11 02/15] qapi: Forbid empty unions and useless alternates

[Eric Blake]

Empty unions serve no purpose, and while we compile with gcc
which permits them, strict C99 forbids them.  We happen to inject
a dummy 'void *data' member into the C unions that represent QAPI
unions and alternates, but we want to get rid of that member (it
pollutes the namespace for no good reason), which would leave us
with an empty union if the user didn't provide any branches.  While
empty structs make sense in QAPI, empty unions don't add any
expressiveness to the QMP language.  So prohibit them at parse
time.  Update the documentation and testsuite to match.

Note that the documentation already mentioned that alternates
should have "two or more JSON data types"; so this also fixes
the code to enforce that.  However, we have existing uses of a
union type with only one branch, so the 2-or-more strictness
is intentionally limited to alternates.

Signed-off-by: Eric Blake 

---
v11: improve commit message
v10: hoist into earlier series
[no v7, v8, or v9]
v6: rebase to earlier qapi.py cleanups
---
 scripts/qapi.py | 12 ++--
 docs/qapi-code-gen.txt  | 15 ---
 tests/qapi-schema/alternate-empty.err   |  1 +
 tests/qapi-schema/alternate-empty.exit  |  2 +-
 tests/qapi-schema/alternate-empty.json  |  2 +-
 tests/qapi-schema/alternate-empty.out   |  5 -
 tests/qapi-schema/flat-union-empty.err  |  1 +
 tests/qapi-schema/flat-union-empty.exit |  2 +-
 tests/qapi-schema/flat-union-empty.json |  2 +-
 tests/qapi-schema/flat-union-empty.out  |  9 -
 tests/qapi-schema/union-empty.err   |  1 +
 tests/qapi-schema/union-empty.exit  |  2 +-
 tests/qapi-schema/union-empty.json  |  2 +-
 tests/qapi-schema/union-empty.out   |  6 --
 14 files changed, 27 insertions(+), 35 deletions(-)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index f40dc9e..f97236f 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -590,7 +590,10 @@ def check_union(expr, expr_info):
 "Discriminator '%s' must be of enumeration "
 "type" % discriminator)

-# Check every branch
+# Check every branch; don't allow an empty union
+if len(members) == 0:
+raise QAPIExprError(expr_info,
+"Union '%s' cannot have empty 'data'" % name)
 for (key, value) in members.items():
 check_name(expr_info, "Member of union '%s'" % name, key)

@@ -613,7 +616,11 @@ def check_alternate(expr, expr_info):
 members = expr['data']
 types_seen = {}

-# Check every branch
+# Check every branch; require at least two branches
+if len(members) < 2:
+raise QAPIExprError(expr_info,
+"Alternate '%s' should have at least two branches "
+"in 'data'" % name)
 for (key, value) in members.items():
 check_name(expr_info, "Member of alternate '%s'" % name, key)

@@ -1059,6 +1066,7 @@ class QAPISchemaObjectTypeVariants(object):
 assert bool(tag_member) != bool(tag_name)
 assert (isinstance(tag_name, str) or
 isinstance(tag_member, QAPISchemaObjectTypeMember))
+assert len(variants) > 0
 for v in variants:
 assert isinstance(v, QAPISchemaObjectTypeVariant)
 self.tag_name = tag_name
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 128f074..999f3b9 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -187,11 +187,11 @@ prevent incomplete include files.

 Usage: { 'struct': STRING, 'data': DICT, '*base': STRUCT-NAME }

-A struct is a dictionary containing a single 'data' key whose
-value is a dictionary.  This corresponds to a struct in C or an Object
-in JSON. Each value of the 'data' dictionary must be the name of a
-type, or a one-element array containing a type name.  An example of a
-struct is:
+A struct is a dictionary containing a single 'data' key whose value is
+a dictionary; the dictionary may be empty.  This corresponds to a
+struct in C or an Object in JSON. Each value of the 'data' dictionary
+must be the name of a type, or a one-element array containing a type
+name.  An example of a struct is:

  { 'struct': 'MyType',
'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } }
@@ -288,9 +288,10 @@ or:{ 'union': STRING, 'data': DICT, 'base': 
STRUCT-NAME,

 Union types are used to let the user choose between several different
 variants for an object.  There are two flavors: simple (no
-discriminator or base), flat (both discriminator and base).  A union
+discriminator or base), and flat (both discriminator and base).  A union
 type is defined using a data dictionary as explained in the following
-paragraphs.
+paragraphs.  The data dictionary for either type of union must not
+be empty.

 A simple union type defines a mapping from automatic discriminator
 values to data types like in this example:
diff --git 

[Qemu-devel] [PATCH v11 11/15] qapi-visit: Use common idiom in gen_visit_fields_decl()

[Eric Blake]

We have several instances of methods that do an early exit if
output is not needed, then log that output is being generated,
and finally produce the output; see qapi-types.py:gen_object()
and qapi-visit.py:gen_visit_implicit_struct().  The odd man
out was gen_visit_fields_decl(); rearrange it to be more like
the others.  No semantic change or difference to generated code.

Signed-off-by: Eric Blake 

---
v11: split out from 11/13
---
 scripts/qapi-visit.py | 11 +--
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 48ebd2b..089f4c4 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -35,15 +35,14 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, 
%(c_type)sobj, Error **


 def gen_visit_fields_decl(typ):
-ret = ''
-if typ.name not in struct_fields_seen:
-ret += mcgen('''
+if typ.name in struct_fields_seen:
+return ''
+struct_fields_seen.add(typ.name)
+return mcgen('''

 static void visit_type_%(c_type)s_fields(Visitor *v, %(c_type)s *obj, Error 
**errp);
 ''',
- c_type=typ.c_name())
-struct_fields_seen.add(typ.name)
-return ret
+ c_type=typ.c_name())


 def gen_visit_implicit_struct(typ):
-- 
2.5.0

[Qemu-devel] [PATCH v11 06/15] qapi: Visit variants in visit_type_FOO_fields()

[Eric Blake]

We initially created the static visit_type_FOO_fields() helper
function for reuse of code - we have cases where the initial
setup for a visit has different allocation (depending on whether
the fields represent a stand-alone type or are embedded as part
of a larger type), but where the actual field visits are
identical once a pointer is available.

Up until the previous patch, visit_type_FOO_fields() was only
used for structs (no variants), so it was covering every field
for each type where it was emitted.

Meanwhile, the code for visiting unions looks like:

static visit_type_U_fields() {
visit base;
visit local_members;
}
visit_type_U() {
visit_start_struct();
visit_type_U_fields();
visit variants;
visit_end_struct();
}

which splits the fields of the union visit across two functions.
Move the code to visit variants to live inside visit_type_U_fields(),
while making it conditional on having variants so that all other
instances of the helper function remain unchanged.  This is also
a step closer towards unifying struct and union visits, and towards
allowing one union type to be the branch of another flat union.

The resulting diff to the generated code is a bit hard to read,
but it can be verified that it touches only union types, and that
the end result is the following general structure:

static visit_type_U_fields() {
visit base;
visit local_members;
visit variants;
}
visit_type_U() {
visit_start_struct();
visit_type_U_fields();
visit_end_struct();
}

Signed-off-by: Eric Blake 

---
v11: new patch
---
 scripts/qapi-visit.py | 104 +-
 1 file changed, 53 insertions(+), 51 deletions(-)

diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 1051710..6cea7d6 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -71,11 +71,16 @@ static void visit_type_implicit_%(c_type)s(Visitor *v, 
%(c_type)s **obj, Error *
 return ret


-def gen_visit_struct_fields(name, base, members):
+def gen_visit_struct_fields(name, base, members, variants=None):
 ret = ''

 if base:
 ret += gen_visit_fields_decl(base)
+if variants:
+for var in variants.variants:
+# Ugly special case for simple union TODO get rid of it
+if not var.simple_union_type():
+ret += gen_visit_implicit_struct(var.type)

 struct_fields_seen.add(name)
 ret += mcgen('''
@@ -96,8 +101,49 @@ static void visit_type_%(c_name)s_fields(Visitor *v, 
%(c_name)s **obj, Error **e

 ret += gen_visit_fields(members, prefix='(*obj)->')

-# 'goto out' produced for base, and by gen_visit_fields() for each member
-if base or members:
+if variants:
+ret += mcgen('''
+if (!visit_start_union(v, !!(*obj)->u.data, ) || err) {
+goto out;
+}
+switch ((*obj)->%(c_name)s) {
+''',
+ c_name=c_name(variants.tag_member.name))
+
+for var in variants.variants:
+# TODO ugly special case for simple union
+simple_union_type = var.simple_union_type()
+ret += mcgen('''
+case %(case)s:
+''',
+ case=c_enum_const(variants.tag_member.type.name,
+   var.name,
+   variants.tag_member.type.prefix))
+if simple_union_type:
+ret += mcgen('''
+visit_type_%(c_type)s(v, "data", &(*obj)->u.%(c_name)s, );
+''',
+ c_type=simple_union_type.c_name(),
+ c_name=c_name(var.name))
+else:
+ret += mcgen('''
+visit_type_implicit_%(c_type)s(v, &(*obj)->u.%(c_name)s, );
+''',
+ c_type=var.type.c_name(),
+ c_name=c_name(var.name))
+ret += mcgen('''
+break;
+''')
+
+ret += mcgen('''
+default:
+abort();
+}
+''')
+
+# 'goto out' produced for base, by gen_visit_fields() for each member,
+# and if variants were present
+if base or members or variants:
 ret += mcgen('''

 out:
@@ -239,12 +285,7 @@ out:


 def gen_visit_union(name, base, members, variants):
-ret = gen_visit_struct_fields(name, base, members)
-
-for var in variants.variants:
-# Ugly special case for simple union TODO get rid of it
-if not var.simple_union_type():
-ret += gen_visit_implicit_struct(var.type)
+ret = gen_visit_struct_fields(name, base, members, variants)

 ret += mcgen('''

@@ -260,54 +301,15 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, 
%(c_name)s **obj, Error
 goto out_obj;
 }
 visit_type_%(c_name)s_fields(v, obj, );
-''',
- c_name=c_name(name))
-ret += gen_err_check(label='out_obj')
-ret += mcgen('''
-if (!visit_start_union(v, !!(*obj)->u.data, ) || err) {
-goto out_obj;

[Qemu-devel] [PATCH v11 03/15] qapi: Forbid 'any' inside an alternate

[Eric Blake]

The whole point of an alternate is to allow some type-safety while
still accepting more than one JSON type.  Meanwhile, the 'any'
type exists to bypass type-safety altogether.  The two are
incompatible: you can't accept every type, and still tell which
branch of the alternate to use for the parse; fix this to give a
sane error instead of a Python stack trace.

Signed-off-by: Eric Blake 

---
v11: new patch
---
 scripts/qapi.py  | 5 -
 tests/Makefile   | 1 +
 tests/qapi-schema/alternate-any.err  | 1 +
 tests/qapi-schema/alternate-any.exit | 1 +
 tests/qapi-schema/alternate-any.json | 4 
 tests/qapi-schema/alternate-any.out  | 0
 6 files changed, 11 insertions(+), 1 deletion(-)
 create mode 100644 tests/qapi-schema/alternate-any.err
 create mode 100644 tests/qapi-schema/alternate-any.exit
 create mode 100644 tests/qapi-schema/alternate-any.json
 create mode 100644 tests/qapi-schema/alternate-any.out

diff --git a/scripts/qapi.py b/scripts/qapi.py
index f97236f..17bf633 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -629,7 +629,10 @@ def check_alternate(expr, expr_info):
value,
allow_metas=['built-in', 'union', 'struct', 'enum'])
 qtype = find_alternate_member_qtype(value)
-assert qtype
+if not qtype:
+raise QAPIExprError(expr_info,
+"Alternate '%s' member '%s' cannot use "
+"type '%s'" % (name, key, value))
 if qtype in types_seen:
 raise QAPIExprError(expr_info,
 "Alternate '%s' member '%s' can't "
diff --git a/tests/Makefile b/tests/Makefile
index c1c605f..7c66d16 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -241,6 +241,7 @@ check-qtest-xtensaeb-y = $(check-qtest-xtensa-y)

 check-qtest-generic-y += tests/qom-test$(EXESUF)

+qapi-schema += alternate-any.json
 qapi-schema += alternate-array.json
 qapi-schema += alternate-base.json
 qapi-schema += alternate-clash.json
diff --git a/tests/qapi-schema/alternate-any.err 
b/tests/qapi-schema/alternate-any.err
new file mode 100644
index 000..aaa0154
--- /dev/null
+++ b/tests/qapi-schema/alternate-any.err
@@ -0,0 +1 @@
+tests/qapi-schema/alternate-any.json:2: Alternate 'Alt' member 'one' cannot 
use type 'any'
diff --git a/tests/qapi-schema/alternate-any.exit 
b/tests/qapi-schema/alternate-any.exit
new file mode 100644
index 000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/alternate-any.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/alternate-any.json 
b/tests/qapi-schema/alternate-any.json
new file mode 100644
index 000..e47a73a
--- /dev/null
+++ b/tests/qapi-schema/alternate-any.json
@@ -0,0 +1,4 @@
+# we do not allow the 'any' type as an alternate branch
+{ 'alternate': 'Alt',
+  'data': { 'one': 'any',
+'two': 'int' } }
diff --git a/tests/qapi-schema/alternate-any.out 
b/tests/qapi-schema/alternate-any.out
new file mode 100644
index 000..e69de29
-- 
2.5.0

[Qemu-devel] [PATCH v11 07/15] qapi-visit: Unify struct and union visit

[Eric Blake]

From: Markus Armbruster 

gen_visit_union() is now just like gen_visit_struct().  Rename
it to gen_visit_object(), use it for structs, and drop
gen_visit_struct().  Output is unchanged.

Signed-off-by: Markus Armbruster 
Message-Id: <1453902888-20457-4-git-send-email-arm...@redhat.com>
[split out variant handling, rebase to earlier changes]
Signed-off-by: Eric Blake 

---
v11: rebase on top of moved variant code
v10: new patch, replacing 31/37
---
 scripts/qapi-visit.py | 45 ++---
 1 file changed, 6 insertions(+), 39 deletions(-)

diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 6cea7d6..4650c5b 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -155,40 +155,6 @@ out:
 return ret


-def gen_visit_struct(name, base, members):
-ret = gen_visit_struct_fields(name, base, members)
-
-# FIXME: if *obj is NULL on entry, and visit_start_struct() assigns to
-# *obj, but then visit_type_FOO_fields() fails, we should clean up *obj
-# rather than leaving it non-NULL. As currently written, the caller must
-# call qapi_free_FOO() to avoid a memory leak of the partial FOO.
-ret += mcgen('''
-
-void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, 
Error **errp)
-{
-Error *err = NULL;
-
-visit_start_struct(v, name, (void **)obj, sizeof(%(c_name)s), );
-if (err) {
-goto out;
-}
-if (!*obj) {
-goto out_obj;
-}
-visit_type_%(c_name)s_fields(v, obj, );
-error_propagate(errp, err);
-err = NULL;
-out_obj:
-visit_end_struct(v, );
-out:
-error_propagate(errp, err);
-}
-''',
- c_name=c_name(name))
-
-return ret
-
-
 def gen_visit_list(name, element_type):
 # FIXME: if *obj is NULL on entry, and the first visit_next_list()
 # assigns to *obj, while a later one fails, we should clean up *obj
@@ -284,9 +250,13 @@ out:
 return ret


-def gen_visit_union(name, base, members, variants):
+def gen_visit_object(name, base, members, variants):
 ret = gen_visit_struct_fields(name, base, members, variants)

+# FIXME: if *obj is NULL on entry, and visit_start_struct() assigns to
+# *obj, but then visit_type_FOO_fields() fails, we should clean up *obj
+# rather than leaving it non-NULL. As currently written, the caller must
+# call qapi_free_FOO() to avoid a memory leak of the partial FOO.
 ret += mcgen('''

 void visit_type_%(c_name)s(Visitor *v, const char *name, %(c_name)s **obj, 
Error **errp)
@@ -363,10 +333,7 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor):

 def visit_object_type(self, name, info, base, members, variants):
 self.decl += gen_visit_decl(name)
-if variants:
-self.defn += gen_visit_union(name, base, members, variants)
-else:
-self.defn += gen_visit_struct(name, base, members)
+self.defn += gen_visit_object(name, base, members, variants)

 def visit_alternate_type(self, name, info, variants):
 self.decl += gen_visit_decl(name)
-- 
2.5.0

[Qemu-devel] [PATCH v11 04/15] qapi: Add tests of complex objects within alternate

[Eric Blake]

Upcoming patches will adjust how we visit an object branch of an
alternate; but we were completely lacking testsuite coverage.
Rectify this, so that the future patches will be able to highlight
the changes and still prove that we avoided regressions.

In particular, the use of a flat union UserDefFlatUnion rather
than a simple struct UserDefA as the branch will give us coverage
of an object with variants.  And visiting an alternate as both
the top level and as a nested member gives confidence in correct
memory allocation handling, especially if the test is run under
valgrind.

Signed-off-by: Eric Blake 

---
v11: new patch
---
 tests/test-qmp-input-visitor.c  | 37 -
 tests/test-qmp-output-visitor.c | 26 ++-
 tests/qapi-schema/qapi-schema-test.json |  4 +++-
 tests/qapi-schema/qapi-schema-test.out  |  4 +++-
 4 files changed, 67 insertions(+), 4 deletions(-)

diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index c72cdad..ef836d5 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -1,7 +1,7 @@
 /*
  * QMP Input Visitor unit-tests.
  *
- * Copyright (C) 2011, 2015 Red Hat Inc.
+ * Copyright (C) 2011-2016 Red Hat Inc.
  *
  * Authors:
  *  Luiz Capitulino 
@@ -309,6 +309,7 @@ static void test_visitor_in_alternate(TestInputVisitorData 
*data,
 Visitor *v;
 Error *err = NULL;
 UserDefAlternate *tmp;
+WrapAlternate *wrap;

 v = visitor_input_test_init(data, "42");
 visit_type_UserDefAlternate(v, NULL, , _abort);
@@ -322,10 +323,44 @@ static void 
test_visitor_in_alternate(TestInputVisitorData *data,
 g_assert_cmpstr(tmp->u.s, ==, "string");
 qapi_free_UserDefAlternate(tmp);

+v = visitor_input_test_init(data, "{'integer':1, 'string':'str', "
+"'enum1':'value1', 'boolean':true}");
+visit_type_UserDefAlternate(v, NULL, , _abort);
+g_assert_cmpint(tmp->type, ==, QTYPE_QDICT);
+g_assert_cmpint(tmp->u.udfu->integer, ==, 1);
+g_assert_cmpstr(tmp->u.udfu->string, ==, "str");
+g_assert_cmpint(tmp->u.udfu->enum1, ==, ENUM_ONE_VALUE1);
+g_assert_cmpint(tmp->u.udfu->u.value1->boolean, ==, true);
+g_assert_cmpint(tmp->u.udfu->u.value1->has_a_b, ==, false);
+qapi_free_UserDefAlternate(tmp);
+
 v = visitor_input_test_init(data, "false");
 visit_type_UserDefAlternate(v, NULL, , );
 error_free_or_abort();
 qapi_free_UserDefAlternate(tmp);
+
+v = visitor_input_test_init(data, "{ 'alt': 42 }");
+visit_type_WrapAlternate(v, NULL, , _abort);
+g_assert_cmpint(wrap->alt->type, ==, QTYPE_QINT);
+g_assert_cmpint(wrap->alt->u.i, ==, 42);
+qapi_free_WrapAlternate(wrap);
+
+v = visitor_input_test_init(data, "{ 'alt': 'string' }");
+visit_type_WrapAlternate(v, NULL, , _abort);
+g_assert_cmpint(wrap->alt->type, ==, QTYPE_QSTRING);
+g_assert_cmpstr(wrap->alt->u.s, ==, "string");
+qapi_free_WrapAlternate(wrap);
+
+v = visitor_input_test_init(data, "{ 'alt': {'integer':1, 'string':'str', "
+"'enum1':'value1', 'boolean':true} }");
+visit_type_WrapAlternate(v, NULL, , _abort);
+g_assert_cmpint(wrap->alt->type, ==, QTYPE_QDICT);
+g_assert_cmpint(wrap->alt->u.udfu->integer, ==, 1);
+g_assert_cmpstr(wrap->alt->u.udfu->string, ==, "str");
+g_assert_cmpint(wrap->alt->u.udfu->enum1, ==, ENUM_ONE_VALUE1);
+g_assert_cmpint(wrap->alt->u.udfu->u.value1->boolean, ==, true);
+g_assert_cmpint(wrap->alt->u.udfu->u.value1->has_a_b, ==, false);
+qapi_free_WrapAlternate(wrap);
 }

 static void test_visitor_in_alternate_number(TestInputVisitorData *data,
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 965f298..2b0f7e9 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -1,7 +1,7 @@
 /*
  * QMP Output Visitor unit-tests.
  *
- * Copyright (C) 2011, 2015 Red Hat Inc.
+ * Copyright (C) 2011-2016 Red Hat Inc.
  *
  * Authors:
  *  Luiz Capitulino 
@@ -427,6 +427,7 @@ static void 
test_visitor_out_alternate(TestOutputVisitorData *data,
 {
 QObject *arg;
 UserDefAlternate *tmp;
+QDict *qdict;

 tmp = g_new0(UserDefAlternate, 1);
 tmp->type = QTYPE_QINT;
@@ -453,6 +454,29 @@ static void 
test_visitor_out_alternate(TestOutputVisitorData *data,

 qapi_free_UserDefAlternate(tmp);
 qobject_decref(arg);
+
+tmp = g_new0(UserDefAlternate, 1);
+tmp->type = QTYPE_QDICT;
+tmp->u.udfu = g_new0(UserDefFlatUnion, 1);
+tmp->u.udfu->integer = 1;
+tmp->u.udfu->string = g_strdup("str");
+tmp->u.udfu->enum1 = ENUM_ONE_VALUE1;
+tmp->u.udfu->u.value1 = g_new0(UserDefA, 1);
+tmp->u.udfu->u.value1->boolean = true;
+
+visit_type_UserDefAlternate(data->ov, NULL, , _abort);
+arg = qmp_output_get_qobject(data->qov);
+
+

[Qemu-devel] [PATCH v11 14/15] qapi: Delete visit_start_union(), gen_visit_implicit_struct()

[Eric Blake]

Delete code rendered dead in the previous patch.  As explained
there, we no longer have any need to use visit_start_struct(),
and we no longer have any more code trying to create or use
visit_type_implicit_FOO().

Signed-off-by: Eric Blake 

---
v11: retitle, merge multiple cleanups, avoid bisection bug by hoisting
visit_start_union() hunk to patch where it matters
v10: retitle, hoist earlier in series, rebase, drop R-b
v9: no change
v8: rebase to 'name' motion
v7: rebase to earlier context changes, simplify 'obj && !*obj'
condition based on contract
v6: rebase due to deferring 7/46, and gen_err_check() improvements;
rewrite gen_visit_implicit_struct() more like other patterns
---
 include/qapi/visitor.h  |  1 -
 include/qapi/visitor-impl.h |  2 --
 scripts/qapi-visit.py   | 29 -
 qapi/qapi-visit-core.c  |  8 
 qapi/qapi-dealloc-visitor.c | 26 --
 5 files changed, 66 deletions(-)

diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 8a2d5cc..a6678b1 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -79,6 +79,5 @@ void visit_type_str(Visitor *v, const char *name, char **obj, 
Error **errp);
 void visit_type_number(Visitor *v, const char *name, double *obj,
Error **errp);
 void visit_type_any(Visitor *v, const char *name, QObject **obj, Error **errp);
-bool visit_start_union(Visitor *v, bool data_present, Error **errp);

 #endif
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 7905a28..c4af3e0 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -58,8 +58,6 @@ struct Visitor

 /* May be NULL; most useful for input visitors. */
 void (*optional)(Visitor *v, const char *name, bool *present);
-
-bool (*start_union)(Visitor *v, bool data_present, Error **errp);
 };

 void input_type_enum(Visitor *v, const char *name, int *obj,
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 367c459..d2c2dc0 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -15,10 +15,6 @@
 from qapi import *
 import re

-# visit_type_FOO_implicit() is emitted as needed; track if it has already
-# been output.
-implicit_structs_seen = set()
-
 # visit_type_FOO_fields() is always emitted; track if a forward declaration
 # or implementation has already been output.
 struct_fields_seen = set()
@@ -45,31 +41,6 @@ static void visit_type_%(c_type)s_fields(Visitor *v, 
%(c_type)s *obj, Error **er
  c_type=typ.c_name())


-def gen_visit_implicit_struct(typ):
-if typ in implicit_structs_seen:
-return ''
-implicit_structs_seen.add(typ)
-
-ret = gen_visit_fields_decl(typ)
-
-ret += mcgen('''
-
-static void visit_type_implicit_%(c_type)s(Visitor *v, %(c_type)s **obj, Error 
**errp)
-{
-Error *err = NULL;
-
-visit_start_implicit_struct(v, (void **)obj, sizeof(%(c_type)s), );
-if (!err) {
-visit_type_%(c_type)s_fields(v, *obj, errp);
-visit_end_implicit_struct(v);
-}
-error_propagate(errp, err);
-}
-''',
- c_type=typ.c_name())
-return ret
-
-
 def gen_visit_struct_fields(name, base, members, variants=None):
 ret = ''

diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index b4a0f21..f7b9980 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -61,14 +61,6 @@ void visit_end_list(Visitor *v)
 v->end_list(v);
 }

-bool visit_start_union(Visitor *v, bool data_present, Error **errp)
-{
-if (v->start_union) {
-return v->start_union(v, data_present, errp);
-}
-return true;
-}
-
 bool visit_optional(Visitor *v, const char *name, bool *present)
 {
 if (v->optional) {
diff --git a/qapi/qapi-dealloc-visitor.c b/qapi/qapi-dealloc-visitor.c
index 6667e8c..4eae555 100644
--- a/qapi/qapi-dealloc-visitor.c
+++ b/qapi/qapi-dealloc-visitor.c
@@ -169,31 +169,6 @@ static void qapi_dealloc_type_enum(Visitor *v, const char 
*name, int *obj,
 {
 }

-/* If there's no data present, the dealloc visitor has nothing to free.
- * Thus, indicate to visitor code that the subsequent union fields can
- * be skipped. This is not an error condition, since the cleanup of the
- * rest of an object can continue unhindered, so leave errp unset in
- * these cases.
- *
- * NOTE: In cases where we're attempting to deallocate an object that
- * may have missing fields, the field indicating the union type may
- * be missing. In such a case, it's possible we don't have enough
- * information to differentiate data_present == false from a case where
- * data *is* present but happens to be a scalar with a value of 0.
- * This is okay, since in the case of the dealloc visitor there's no
- * work that needs to done in either situation.
- *
- * The current inability in QAPI code to more thoroughly verify a union
- * type in such cases will likely need to be addressed if we wish to
- * implement this interface for other 

[Qemu-devel] [PATCH v11 05/15] qapi-visit: Simplify how we visit common union members

[Eric Blake]

From: Markus Armbruster 

For a simple union SU, gen_visit_union() generates a visit of its
single tag member, like this:

visit_type_SUKind(v, "type", &(*obj)->type, );

For a flat union FU with base B, it generates a visit of its base
fields:

visit_type_B_fields(v, (B **)obj, );

Instead, we can simply visit the common members using the same fields
visit function we use for structs, generated with
gen_visit_struct_fields().  This function visits the base if any, then
the local members.

For a simple union SU, visit_type_SU_fields() contains exactly the old
tag member visit, because there is no base, and the tag member is the
only member.  For instance, the code generated for qapi-schema.json's
KeyValue changes like this:

+static void visit_type_KeyValue_fields(Visitor *v, KeyValue **obj, Error 
**errp)
+{
+Error *err = NULL;
+
+visit_type_KeyValueKind(v, "type", &(*obj)->type, );
+if (err) {
+goto out;
+}
+
+out:
+error_propagate(errp, err);
+}
+
 void visit_type_KeyValue(Visitor *v, const char *name, KeyValue **obj, 
Error **errp)
 {
 Error *err = NULL;
@@ -4863,7 +4911,7 @@ void visit_type_KeyValue(Visitor *v, con
 if (!*obj) {
 goto out_obj;
 }
-visit_type_KeyValueKind(v, "type", &(*obj)->type, );
+visit_type_KeyValue_fields(v, obj, );
 if (err) {
 goto out_obj;
 }

For a flat union FU, visit_type_FU_fields() contains exactly the old
base fields visit, because there is a base, but no members.  For
instance, the code generated for qapi-schema.json's CpuInfo changes
like this:

 static void visit_type_CpuInfoBase_fields(Visitor *v, CpuInfoBase **obj, 
Error **errp);

+static void visit_type_CpuInfo_fields(Visitor *v, CpuInfo **obj, Error 
**errp)
+{
+Error *err = NULL;
+
+visit_type_CpuInfoBase_fields(v, (CpuInfoBase **)obj, );
+if (err) {
+goto out;
+}
+
+out:
+error_propagate(errp, err);
+}
+
 static void visit_type_CpuInfoX86_fields(Visitor *v, CpuInfoX86 **obj, 
Error **errp)
...
@@ -3485,7 +3509,7 @@ void visit_type_CpuInfo(Visitor *v, cons
 if (!*obj) {
 goto out_obj;
 }
-visit_type_CpuInfoBase_fields(v, (CpuInfoBase **)obj, );
+visit_type_CpuInfo_fields(v, obj, );
 if (err) {
 goto out_obj;
 }

As you see, the generated code grows a bit, but in practice, it's lost
in the noise: qapi-schema.json's qapi-visit.c gains roughly 1%.

This simplification became possible with commit 441cbac "qapi-visit:
Convert to QAPISchemaVisitor, fixing bugs".  It's a step towards
unifying gen_struct() and gen_union().

Signed-off-by: Markus Armbruster 
Message-Id: <1453902888-20457-2-git-send-email-arm...@redhat.com>
[rebase, improve commit message example]
Signed-off-by: Eric Blake 

---
v11: update commit message now that goto cleanup is removed from series
v10: new patch, but effectively split out from 31/37
---
 scripts/qapi-visit.py | 27 ---
 1 file changed, 4 insertions(+), 23 deletions(-)

diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 2bdb5a1..1051710 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -238,11 +238,8 @@ out:
 return ret


-def gen_visit_union(name, base, variants):
-ret = ''
-
-if base:
-ret += gen_visit_fields_decl(base)
+def gen_visit_union(name, base, members, variants):
+ret = gen_visit_struct_fields(name, base, members)

 for var in variants.variants:
 # Ugly special case for simple union TODO get rid of it
@@ -262,21 +259,9 @@ void visit_type_%(c_name)s(Visitor *v, const char *name, 
%(c_name)s **obj, Error
 if (!*obj) {
 goto out_obj;
 }
+visit_type_%(c_name)s_fields(v, obj, );
 ''',
  c_name=c_name(name))
-
-if base:
-ret += mcgen('''
-visit_type_%(c_name)s_fields(v, (%(c_name)s **)obj, );
-''',
- c_name=base.c_name())
-else:
-ret += mcgen('''
-visit_type_%(c_type)s(v, "%(name)s", &(*obj)->%(c_name)s, );
-''',
- c_type=variants.tag_member.type.c_name(),
- c_name=c_name(variants.tag_member.name),
- name=variants.tag_member.name)
 ret += gen_err_check(label='out_obj')
 ret += mcgen('''
 if (!visit_start_union(v, !!(*obj)->u.data, ) || err) {
@@ -377,11 +362,7 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor):
 def visit_object_type(self, name, info, base, members, variants):
 self.decl += gen_visit_decl(name)
 if variants:
-if members:
-# Members other than variants.tag_member not implemented
-assert len(members) == 1
-assert members[0] == variants.tag_member
- 

[Qemu-devel] [PATCH v11 01/15] qapi: Simplify excess input reporting in input visitors

[Eric Blake]

When reporting that an unvisited member remains at the end of an
input visit for a struct, we were using g_hash_table_find()
coupled with a callback function that always returns true, to
locate an arbitrary member of the hash table.  But if all we
need is an arbitrary entry, we can get that from a single-use
iterator, without needing a tautological callback function.

Technically, our cast of &(GQueue *) to (void **) is not strict
C (while void * must be able to hold all other pointers, nothing
says a void ** has to be the same width or representation as a
GQueue **).  The kosher way to write it would be the verbose:

void *tmp;
GQueue *any;
if (g_hash_table_iter_next(, NULL, )) {
any = tmp;

But our code base (not to mention glib itself) already has other
cases of assuming that ALL pointers have the same width and
representation, where a compiler would have to go out of its way
to mis-compile our borderline behavior.

Suggested-by: Markus Armbruster 
Signed-off-by: Eric Blake 
Reviewed-by: Marc-André Lureau 

---
v11: no change
v10: enhance commit message
v9: no change
v8: rebase to earlier changes
v7: retitle, rebase to earlier context changes
v6: new patch, based on comments on RFC against v5 7/46
---
 qapi/opts-visitor.c  | 12 +++-
 qapi/qmp-input-visitor.c | 14 +-
 2 files changed, 8 insertions(+), 18 deletions(-)

diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
index d54f75b..ae5b955 100644
--- a/qapi/opts-visitor.c
+++ b/qapi/opts-visitor.c
@@ -157,17 +157,11 @@ opts_start_struct(Visitor *v, const char *name, void 
**obj,
 }


-static gboolean
-ghr_true(gpointer ign_key, gpointer ign_value, gpointer ign_user_data)
-{
-return TRUE;
-}
-
-
 static void
 opts_end_struct(Visitor *v, Error **errp)
 {
 OptsVisitor *ov = to_ov(v);
+GHashTableIter iter;
 GQueue *any;

 if (--ov->depth > 0) {
@@ -175,8 +169,8 @@ opts_end_struct(Visitor *v, Error **errp)
 }

 /* we should have processed all (distinct) QemuOpt instances */
-any = g_hash_table_find(ov->unprocessed_opts, _true, NULL);
-if (any) {
+g_hash_table_iter_init(, ov->unprocessed_opts);
+if (g_hash_table_iter_next(, NULL, (void **))) {
 const QemuOpt *first;

 first = g_queue_peek_head(any);
diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index 362a1a3..2f48b95 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -90,12 +90,6 @@ static void qmp_input_push(QmpInputVisitor *qiv, QObject 
*obj, Error **errp)
 qiv->nb_stack++;
 }

-/** Only for qmp_input_pop. */
-static gboolean always_true(gpointer key, gpointer val, gpointer user_pkey)
-{
-*(const char **)user_pkey = (const char *)key;
-return TRUE;
-}

 static void qmp_input_pop(QmpInputVisitor *qiv, Error **errp)
 {
@@ -104,9 +98,11 @@ static void qmp_input_pop(QmpInputVisitor *qiv, Error 
**errp)
 if (qiv->strict) {
 GHashTable * const top_ht = qiv->stack[qiv->nb_stack - 1].h;
 if (top_ht) {
-if (g_hash_table_size(top_ht)) {
-const char *key;
-g_hash_table_find(top_ht, always_true, );
+GHashTableIter iter;
+const char *key;
+
+g_hash_table_iter_init(, top_ht);
+if (g_hash_table_iter_next(, (void **), NULL)) {
 error_setg(errp, QERR_QMP_EXTRA_MEMBER, key);
 }
 g_hash_table_unref(top_ht);
-- 
2.5.0

[Qemu-devel] [PATCH v11 00/15] prune some QAPI visitor cruft (was qapi cleanups subset E)

[Eric Blake]

I'm still working on my documentation patches for QAPI visitors
(https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg03504.html),
but am finding it easier to nuke bad code up front than to document
that it is bad only to later nuke it. So this pulls bits and pieces
of other patches that Markus I have previously posted, along with
some new glue, to get rid of some of the worst of the cruft.

v10 was here:
https://lists.gnu.org/archive/html/qemu-devel/2016-02/msg03282.html

Since then, I've folded in fixes for Markus' review comments,
including rearranging some hunks and retitling some patches,
and dropping the attempt to minimize generated gotos. The biggest
changes are moving where variants get visited (during
visit_type_FOO_fields() rather than visit_type_FOO()), dropping
visit_type_alternate_FOO() (open-coding it during
gen_visit_alternate()), and fixing bugs (properly handle
visit_start_union() so that there are no bisection points where
we are handling 'void *data' incorrectly).

001/15:[] [--] 'qapi: Simplify excess input reporting in input visitors'
002/15:[] [--] 'qapi: Forbid empty unions and useless alternates'
003/15:[down] 'qapi: Forbid 'any' inside an alternate'
004/15:[down] 'qapi: Add tests of complex objects within alternate'
005/15:[] [--] 'qapi-visit: Simplify how we visit common union members'
006/15:[down] 'qapi: Visit variants in visit_type_FOO_fields()'
007/15:[0051] [FC] 'qapi-visit: Unify struct and union visit'
008/15:[0012] [FC] 'qapi-visit: Less indirection in visit_type_Foo_fields()'
009/15:[0002] [FC] 'qapi: Adjust layout of FooList types'
010/15:[0004] [FC] 'qapi: Emit structs used as variants in topological order'
011/15:[down] 'qapi-visit: Use common idiom in gen_visit_fields_decl()'
012/15:[0148] [FC] 'qapi: Don't box struct branch of alternate'
013/15:[0074] [FC] 'qapi: Don't box branches of flat unions'
014/15:[down] 'qapi: Delete visit_start_union(), gen_visit_implicit_struct()'
015/15:[0001] [FC] 'qapi: Change visit_start_implicit_struct to 
visit_start_alternate'

Eric Blake (13):
  qapi: Simplify excess input reporting in input visitors
  qapi: Forbid empty unions and useless alternates
  qapi: Forbid 'any' inside an alternate
  qapi: Add tests of complex objects within alternate
  qapi: Visit variants in visit_type_FOO_fields()
  qapi-visit: Less indirection in visit_type_Foo_fields()
  qapi: Adjust layout of FooList types
  qapi: Emit structs used as variants in topological order
  qapi-visit: Use common idiom in gen_visit_fields_decl()
  qapi: Don't box struct branch of alternate
  qapi: Don't box branches of flat unions
  qapi: Delete visit_start_union(), gen_visit_implicit_struct()
  qapi: Change visit_start_implicit_struct to visit_start_alternate

Markus Armbruster (2):
  qapi-visit: Simplify how we visit common union members
  qapi-visit: Unify struct and union visit

 include/qapi/visitor.h  |  63 +---
 include/qapi/visitor-impl.h |  21 ++-
 scripts/qapi.py |  29 +++-
 scripts/qapi-types.py   |  33 +++--
 scripts/qapi-visit.py   | 254 
 qapi/qapi-visit-core.c  |  45 ++
 cpus.c  |  18 +--
 hmp.c   |  12 +-
 qapi/opts-visitor.c |  16 +-
 qapi/qapi-dealloc-visitor.c |  42 +-
 qapi/qmp-input-visitor.c|  43 +++---
 qapi/qmp-output-visitor.c   |   3 +-
 qapi/string-input-visitor.c |   4 +-
 qapi/string-output-visitor.c|   2 +-
 tests/test-qmp-input-visitor.c  |  39 -
 tests/test-qmp-output-visitor.c |  27 +++-
 docs/qapi-code-gen.txt  |  15 +-
 tests/Makefile  |   1 +
 tests/qapi-schema/alternate-any.err |   1 +
 tests/qapi-schema/alternate-any.exit|   1 +
 tests/qapi-schema/alternate-any.json|   4 +
 tests/qapi-schema/alternate-any.out |   0
 tests/qapi-schema/alternate-empty.err   |   1 +
 tests/qapi-schema/alternate-empty.exit  |   2 +-
 tests/qapi-schema/alternate-empty.json  |   2 +-
 tests/qapi-schema/alternate-empty.out   |   5 -
 tests/qapi-schema/flat-union-empty.err  |   1 +
 tests/qapi-schema/flat-union-empty.exit |   2 +-
 tests/qapi-schema/flat-union-empty.json |   2 +-
 tests/qapi-schema/flat-union-empty.out  |   9 --
 tests/qapi-schema/qapi-schema-test.json |   4 +-
 tests/qapi-schema/qapi-schema-test.out  |   4 +-
 tests/qapi-schema/union-empty.err   |   1 +
 tests/qapi-schema/union-empty.exit  |   2 +-
 tests/qapi-schema/union-empty.json  |   2 +-
 tests/qapi-schema/union-empty.out   |   6 -
 36 files changed, 347 insertions(+), 369 deletions(-)
 create mode 100644 tests/qapi-schema/alternate-any.err
 create mode 100644 tests/qapi-schema/alternate-any.exit
 create mode 100644 tests/qapi-schema/alternate-any.json
 create mode 100644 

Re: [Qemu-devel] [RFC PATCH v4] fw/pci: Add support for mapping Intel IGD via QEMU

[Tian, Kevin]

> From: Alex Williamson [mailto:alex.william...@redhat.com]
> Sent: Wednesday, February 17, 2016 9:50 PM
> 
> On Wed, 17 Feb 2016 11:09:49 +
> "Tian, Kevin"  wrote:
> 
> > > From: Alex Williamson
> > > Sent: Wednesday, February 17, 2016 5:39 AM
> > >
> > > QEMU provides two fw_cfg files to support IGD.  The first holds the
> > > OpRegion data which holds the Video BIOS Table (VBT).  This needs to
> > > be copied into reserved memory and the address stored in the ASL
> > > Storage register of the device at 0xFC offset in PCI config space.
> > > The OpRegion is generally 8KB.  This file is named "etc/igd-opregion".
> > >
> > > The second file tells us the required size of the stolen memory space
> > > for the device.  This is a dummy file, it has no backing so we only
> > > allocate the space without copying anything into it.  This space
> > > requires 1MB alignment and is generally either 1MB or 2MB, depending
> > > on the hardware config.  If the user has opted in QEMU to expose
> > > additional stolen memory beyond the GTT (GGMS), the GMS may add an
> > > additional 32MB to 512MB.  The base address of the reserved memory
> > > allocated for this is written back to the Base Data of Stolen Memory
> > > register (BDSM) at PCI config offset 0x5C on the device.  This file is
> > > named "etc/igd-bdsm".
> >
> > What would happen if guest tries to access this range while there is
> > no actual memory behind? Isn't it more clear to hide stolen memory
> > at all instead of reporting a dummy range?
> 
> It's a fw_cfg file, it's not exposed to the guest, the purpose is to
> convey the size of stolen memory, which the BIOS then allocates as
> reserved memory and writes back to the BDSM register.  It would be more
> clean to ignore stolen memory, but in cases where we need the vBIOS,
> such as laptops, where my LCD panel won't turn on without it, we don't
> have that luxury.  The vBIOS programs the device to use stolen memory,
> at least 1MB, I assume for GTT purposes, and makes use of that for VESA
> modes.  So, we need the vBIOS to support laptop panels, the vBIOS needs
> stolen memory for GTT space, therefore we need to provide some stolen
> memory.

Thanks for explaining the rationale. It's a bit surprise to me but anyway
since you do observe the effect. Allen might help confirm whether your
assumption makes sense. :-)

> 
> This support is enabled in QEMU by using a vBIOS, I assume vGPUs won't
> expose a ROM BAR and therefore won't enable this.  Additionally for
> BDW/SKL+ (gen8+) GPUs, if the user specifies rombar=0 then all of this
> is disabled, including the hostbridge and ISA bridge manipulation in
> order to support UPT mode.  Thanks,
> 

Correct. vGPU doesn't use a host vBIOS. We just leverage existing
Seabios for early booting.

Thanks
Kevin

Re: [Qemu-devel] [PATCH v2 3/3] migration: allow to suppress configuration section submission

[Amit Shah]

On (Thu) 18 Feb 2016 [13:50:25], David Gibson wrote:
> On Wed, Feb 17, 2016 at 05:06:43PM +0100, Greg Kurz wrote:
> > Since the addition of the configuration section in QEMU 2.4, it is 
> > impossible
> > to migrate a pseries-2.3 machine back to QEMU 2.3.
> > 
> > This patch makes it possible thanks to a new machine property which allows 
> > to
> > disable configuration section submission.
> > 
> > To disable submission, just add:
> > 
> > -machine suppress-config-section=on
> > 
> > Alternatively, if the target QEMU version isn't known at startup, this can
> > be done later from the QEMU monitor with:
> > 
> > qom-set /machine suppress-config-section on
> > 
> > This property won't be automatically set for pseries-2.3 because it would
> > then break backward migration to QEMU 2.4. If automatic behaviour is needed,
> > it is up to the tooling to handle this.
> 
> As noted elsewhere, I'd actually be ok with enabling it for
> pseries-2.3.  Basically we have to chose whether to work against qemu
> 2.3 or 2.4 out of the box, we can't have both, and it sounds like qemu
> 2.3 is more widely deployed than qemu 2.4.
> 
> > 
> > Signed-off-by: Greg Kurz 
> 
> Reviewed-by: David Gibson 
> 
> I'm not sure whose tree these need to go in via.

I'd like to see Juan's ack, and I'm fine if you take it via your tree.



Amit

Re: [Qemu-devel] [PATCH v7 00/11] Add basic "detach" support for dump-guest-memory

[Peter Xu]

The subject of cover letter is incorrect. It should be:

[PATCH v8 00/11] Add basic "detach" support for dump-guest-memory

rather than:

[PATCH v7 00/11] Add basic "detach" support for dump-guest-memory

Subjects of patches are with correct number, which is v8.

Sorry for the mistake.

Peter

On Thu, Feb 18, 2016 at 01:16:45PM +0800, Peter Xu wrote:
> Changes from v7:
> - patch 8: use s->dump_info.page_size not TARGET_PAGE_SIZE
> - patch 10: change DUMP_STATUS_MAX to DUMP_STATUS__MAX (this is to
>   fix compile error for rebasing to latest master branch, still do
>   not know why we need this change from generating "_MAX" to
>   "__MAX" for enum types...)
> 
> Changes from v6:
> - patch 9: fix leak of local_err due to patch switch.. [Fam]
> - patch 10: assert "result" before use [Fam]
> - patch 11: add Fam's reviewed-by.
> 
> For older patch, please refers to v6 series:
> 
> https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg01299.html
> 
> Peter Xu (11):
>   dump-guest-memory: cleanup: removing dump_{error|cleanup}().
>   dump-guest-memory: add "detach" flag for QMP/HMP interfaces.
>   dump-guest-memory: using static DumpState, add DumpStatus
>   dump-guest-memory: add dump_in_progress() helper function
>   dump-guest-memory: introduce dump_process() helper function.
>   dump-guest-memory: disable dump when in INMIGRATE state
>   dump-guest-memory: add "detach" support
>   DumpState: adding total_size and written_size fields
>   Dump: add qmp command "query-dump"
>   Dump: add hmp command "info dump"
>   dump-guest-memory: add qmp event DUMP_COMPLETED
> 
>  docs/qmp-events.txt |  18 
>  dump.c  | 215 
> ++--
>  hmp-commands-info.hx|  14 +++
>  hmp-commands.hx |   5 +-
>  hmp.c   |  26 -
>  hmp.h   |   1 +
>  include/qemu-common.h   |   4 +
>  include/sysemu/dump.h   |  15 +++
>  include/sysemu/memory_mapping.h |   4 +
>  memory_mapping.c|   3 +
>  qapi-schema.json|  56 ++-
>  qapi/event.json |  16 +++
>  qmp-commands.hx |  31 +-
>  qmp.c   |  14 +++
>  14 files changed, 359 insertions(+), 63 deletions(-)
> 
> -- 
> 2.4.3
> 

[Qemu-devel] [PATCH v8 10/11] Dump: add hmp command "info dump"

[Peter Xu]

It will calculate percentage of finished work from completed and
total.

Signed-off-by: Peter Xu 
---
 hmp-commands-info.hx | 14 ++
 hmp.c| 17 +
 hmp.h|  1 +
 3 files changed, 32 insertions(+)

diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index 9b71351..52539c3 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -786,6 +786,20 @@ STEXI
 Display the value of a storage key (s390 only)
 ETEXI
 
+{
+.name   = "dump",
+.args_type  = "",
+.params = "",
+.help   = "Display the latest dump status",
+.mhandler.cmd = hmp_info_dump,
+},
+
+STEXI
+@item info dump
+@findex dump
+Display the latest dump status.
+ETEXI
+
 STEXI
 @end table
 ETEXI
diff --git a/hmp.c b/hmp.c
index 7f65b32..d0d0557 100644
--- a/hmp.c
+++ b/hmp.c
@@ -2351,3 +2351,20 @@ void hmp_rocker_of_dpa_groups(Monitor *mon, const QDict 
*qdict)
 
 qapi_free_RockerOfDpaGroupList(list);
 }
+
+void hmp_info_dump(Monitor *mon, const QDict *qdict)
+{
+DumpQueryResult *result = qmp_query_dump(NULL);
+
+assert(result && result->status < DUMP_STATUS__MAX);
+monitor_printf(mon, "Status: %s\n", DumpStatus_lookup[result->status]);
+
+if (result->status == DUMP_STATUS_ACTIVE) {
+float percent = 0;
+assert(result->total != 0);
+percent = 100.0 * result->completed / result->total;
+monitor_printf(mon, "Finished: %.2f %%\n", percent);
+}
+
+qapi_free_DumpQueryResult(result);
+}
diff --git a/hmp.h b/hmp.h
index a8c5b5a..093d65f 100644
--- a/hmp.h
+++ b/hmp.h
@@ -131,5 +131,6 @@ void hmp_rocker(Monitor *mon, const QDict *qdict);
 void hmp_rocker_ports(Monitor *mon, const QDict *qdict);
 void hmp_rocker_of_dpa_flows(Monitor *mon, const QDict *qdict);
 void hmp_rocker_of_dpa_groups(Monitor *mon, const QDict *qdict);
+void hmp_info_dump(Monitor *mon, const QDict *qdict);
 
 #endif
-- 
2.4.3

[Qemu-devel] [PATCH v8 09/11] Dump: add qmp command "query-dump"

[Peter Xu]

When dump-guest-memory is requested with detach flag, after its
return, user could query its status using "query-dump" command (with
no argument). The result contains:

- status: current dump status
- completed: bytes written in the latest dump
- total: bytes to write in the latest dump

>From completed and total, we could know how much work
finished by calculating:

  100.0 * completed / total (%)

Reviewed-by:   Fam Zheng 
Signed-off-by: Peter Xu 
---
 dump.c   | 23 +++
 qapi-schema.json | 32 +++-
 qmp-commands.hx  | 27 ++-
 3 files changed, 76 insertions(+), 6 deletions(-)

diff --git a/dump.c b/dump.c
index cd4ae5e..9dc4946 100644
--- a/dump.c
+++ b/dump.c
@@ -1456,7 +1456,7 @@ static void dump_state_prepare(DumpState *s)
 bool dump_in_progress(void)
 {
 DumpState *state = _state_global;
-return (state->status == DUMP_STATUS_ACTIVE);
+return (atomic_read(>status) == DUMP_STATUS_ACTIVE);
 }
 
 /* calculate total size of memory to be dumped (taking filter into
@@ -1669,9 +1669,12 @@ static void dump_process(DumpState *s, Error **errp)
 create_vmcore(s, _err);
 }
 
-s->status = (local_err ? DUMP_STATUS_FAILED : DUMP_STATUS_COMPLETED);
-error_propagate(errp, local_err);
+/* make sure status is written after written_size updates */
+smp_wmb();
+atomic_set(>status,
+   (local_err ? DUMP_STATUS_FAILED : DUMP_STATUS_COMPLETED));
 
+error_propagate(errp, local_err);
 dump_cleanup(s);
 }
 
@@ -1689,6 +1692,18 @@ static void *dump_thread(void *data)
 return NULL;
 }
 
+DumpQueryResult *qmp_query_dump(Error **errp)
+{
+DumpQueryResult *result = g_new(DumpQueryResult, 1);
+DumpState *state = _state_global;
+result->status = atomic_read(>status);
+/* make sure we are reading status and written_size in order */
+smp_rmb();
+result->completed = state->written_size;
+result->total = state->total_size;
+return result;
+}
+
 void qmp_dump_guest_memory(bool paging, const char *file,
bool has_detach, bool detach,
bool has_begin, int64_t begin, bool has_length,
@@ -1779,7 +1794,7 @@ void qmp_dump_guest_memory(bool paging, const char *file,
   begin, length, _err);
 if (local_err) {
 error_propagate(errp, local_err);
-s->status = DUMP_STATUS_FAILED;
+atomic_set(>status, DUMP_STATUS_FAILED);
 return;
 }
 
diff --git a/qapi-schema.json b/qapi-schema.json
index ccd30c8..7b8f2a1 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2196,7 +2196,8 @@
 #   is the fd's name.
 #
 # @detach: #optional if true, QMP will return immediately rather than
-#  waiting for the dump to finish. (since 2.6).
+#  waiting for the dump to finish. The user can track progress
+#  using "query-dump". (since 2.6).
 #
 # @begin: #optional if specified, the starting physical address.
 #
@@ -2237,6 +2238,35 @@
   'data': [ 'none', 'active', 'completed', 'failed' ] }
 
 ##
+# @DumpQueryResult
+#
+# The result format for 'query-dump'.
+#
+# @status: enum of @DumpStatus, which shows current dump status
+#
+# @completed: bytes written in latest dump (uncompressed)
+#
+# @total: total bytes to be written in latest dump (uncompressed)
+#
+# Since 2.6
+##
+{ 'struct': 'DumpQueryResult',
+  'data': { 'status': 'DumpStatus',
+'completed': 'int',
+'total': 'int' } }
+
+##
+# @query-dump
+#
+# Query latest dump status.
+#
+# Returns: A @DumpStatus object showing the dump status.
+#
+# Since: 2.6
+##
+{ 'command': 'query-dump', 'returns': 'DumpQueryResult' }
+
+##
 # @DumpGuestMemoryCapability:
 #
 # A list of the available formats for dump-guest-memory
diff --git a/qmp-commands.hx b/qmp-commands.hx
index d30b1eb..daaafe5 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -856,7 +856,8 @@ Arguments:
 - "protocol": destination file(started with "file:") or destination file
   descriptor (started with "fd:") (json-string)
 - "detach": if specified, command will return immediately, without waiting
-for the dump to finish (json-bool)
+for the dump to finish. The user can track progress using
+"query-dump". (json-bool)
 - "begin": the starting physical address. It's optional, and should be 
specified
with length together (json-int)
 - "length": the memory size, in bytes. It's optional, and should be specified
@@ -896,6 +897,30 @@ Example:
 
 EQMP
 
+{
+.name   = "query-dump",
+.args_type  = "",
+.params = "",
+.help   = "query background dump status",
+.mhandler.cmd_new = qmp_marshal_query_dump,
+},
+
+SQMP
+query-dump
+--
+
+Query background dump status.
+
+Arguments: None.
+
+Example:
+
+-> { "execute": "query-dump" }
+<- { "return": { "status": "active", 

[Qemu-devel] [PATCH v8 07/11] dump-guest-memory: add "detach" support

[Peter Xu]

If "detach" is provided, one thread is created to do the dump work,
while main thread will return immediately. For each GuestPhysBlock,
adding one more field "mr" to points to MemoryRegion that it
belongs, also ref the mr before use.

Signed-off-by: Peter Xu 
Reviewed-by: Fam Zheng 
---
 dump.c  | 27 ++-
 include/sysemu/dump.h   |  1 +
 include/sysemu/memory_mapping.h |  4 
 memory_mapping.c|  3 +++
 4 files changed, 34 insertions(+), 1 deletion(-)

diff --git a/dump.c b/dump.c
index 923e3a5..9210a72 100644
--- a/dump.c
+++ b/dump.c
@@ -1643,6 +1643,20 @@ static void dump_process(DumpState *s, Error **errp)
 dump_cleanup(s);
 }
 
+static void *dump_thread(void *data)
+{
+Error *err = NULL;
+DumpState *s = (DumpState *)data;
+
+dump_process(s, );
+
+if (err) {
+/* TODO: notify user the error */
+error_free(err);
+}
+return NULL;
+}
+
 void qmp_dump_guest_memory(bool paging, const char *file,
bool has_detach, bool detach,
bool has_begin, int64_t begin, bool has_length,
@@ -1653,6 +1667,7 @@ void qmp_dump_guest_memory(bool paging, const char *file,
 int fd = -1;
 DumpState *s;
 Error *local_err = NULL;
+bool detach_p = false;
 
 if (runstate_check(RUN_STATE_INMIGRATE)) {
 error_setg(errp, "Dump not allowed during incoming migration.");
@@ -1684,6 +1699,9 @@ void qmp_dump_guest_memory(bool paging, const char *file,
 error_setg(errp, QERR_MISSING_PARAMETER, "begin");
 return;
 }
+if (has_detach) {
+detach_p = detach;
+}
 
 /* check whether lzo/snappy is supported */
 #ifndef CONFIG_LZO
@@ -1733,7 +1751,14 @@ void qmp_dump_guest_memory(bool paging, const char *file,
 return;
 }
 
-dump_process(s, errp);
+if (detach_p) {
+/* detached dump */
+qemu_thread_create(>dump_thread, "dump_thread", dump_thread,
+   s, QEMU_THREAD_DETACHED);
+} else {
+/* sync dump */
+dump_process(s, errp);
+}
 }
 
 DumpGuestMemoryCapability *qmp_query_dump_guest_memory_capability(Error **errp)
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index 1da3ddb..06393c3 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -181,6 +181,7 @@ typedef struct DumpState {
 
 bool has_format;  /* whether format is provided */
 DumpGuestMemoryFormat format; /* valid only if has_format == true */
+QemuThread dump_thread;   /* thread for detached dump */
 } DumpState;
 
 uint16_t cpu_to_dump16(DumpState *s, uint16_t val);
diff --git a/include/sysemu/memory_mapping.h b/include/sysemu/memory_mapping.h
index a75d59a..d46d879 100644
--- a/include/sysemu/memory_mapping.h
+++ b/include/sysemu/memory_mapping.h
@@ -16,6 +16,7 @@
 
 #include "qemu/queue.h"
 #include "qemu/typedefs.h"
+#include "exec/memory.h"
 
 typedef struct GuestPhysBlock {
 /* visible to guest, reflects PCI hole, etc */
@@ -27,6 +28,9 @@ typedef struct GuestPhysBlock {
 /* points into host memory */
 uint8_t *host_addr;
 
+/* points to the MemoryRegion that this block belongs to */
+MemoryRegion *mr;
+
 QTAILQ_ENTRY(GuestPhysBlock) next;
 } GuestPhysBlock;
 
diff --git a/memory_mapping.c b/memory_mapping.c
index 04db3ac..c8855de 100644
--- a/memory_mapping.c
+++ b/memory_mapping.c
@@ -178,6 +178,7 @@ void guest_phys_blocks_free(GuestPhysBlockList *list)
 
 QTAILQ_FOREACH_SAFE(p, >head, next, q) {
 QTAILQ_REMOVE(>head, p, next);
+memory_region_unref(p->mr);
 g_free(p);
 }
 list->num = 0;
@@ -241,6 +242,8 @@ static void guest_phys_blocks_region_add(MemoryListener 
*listener,
 block->target_start = target_start;
 block->target_end   = target_end;
 block->host_addr= host_addr;
+block->mr   = section->mr;
+memory_region_ref(section->mr);
 
 QTAILQ_INSERT_TAIL(>list->head, block, next);
 ++g->list->num;
-- 
2.4.3

[Qemu-devel] [PATCH v8 06/11] dump-guest-memory: disable dump when in INMIGRATE state

[Peter Xu]

Signed-off-by: Peter Xu 
Reviewed-by: Fam Zheng 
---
 dump.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/dump.c b/dump.c
index fed84a6..923e3a5 100644
--- a/dump.c
+++ b/dump.c
@@ -1654,6 +1654,11 @@ void qmp_dump_guest_memory(bool paging, const char *file,
 DumpState *s;
 Error *local_err = NULL;
 
+if (runstate_check(RUN_STATE_INMIGRATE)) {
+error_setg(errp, "Dump not allowed during incoming migration.");
+return;
+}
+
 /* if there is a dump in background, we should wait until the dump
  * finished */
 if (dump_in_progress()) {
-- 
2.4.3

[Qemu-devel] [PATCH v8 11/11] dump-guest-memory: add qmp event DUMP_COMPLETED

[Peter Xu]

One new QMP event DUMP_COMPLETED is added. When a dump finishes, one
DUMP_COMPLETED event will occur to notify the user.

Signed-off-by: Peter Xu 
Reviewed-by:   Fam Zheng 
---
 docs/qmp-events.txt | 18 ++
 dump.c  | 18 --
 qapi/event.json | 16 
 3 files changed, 46 insertions(+), 6 deletions(-)

diff --git a/docs/qmp-events.txt b/docs/qmp-events.txt
index 52eb7e2..4e3eb9e 100644
--- a/docs/qmp-events.txt
+++ b/docs/qmp-events.txt
@@ -220,6 +220,24 @@ Data:
   },
   "timestamp": { "seconds": 1265044230, "microseconds": 450486 } }
 
+DUMP_COMPLETED
+--
+
+Emitted when the guest has finished one memory dump.
+
+Data:
+
+- "result": DumpQueryResult type described in qapi-schema.json
+- "error": Error message when dump failed. This is only a
+  human-readable string provided when dump failed. It should not be
+  parsed in any way (json-string, optional)
+
+Example:
+
+{ "event": "DUMP_COMPLETED",
+  "data": {"result": {"total": 1090650112, "status": "completed",
+  "completed": 1090650112} } }
+
 GUEST_PANICKED
 --
 
diff --git a/dump.c b/dump.c
index 9dc4946..81d2d4f 100644
--- a/dump.c
+++ b/dump.c
@@ -25,6 +25,7 @@
 #include "sysemu/cpus.h"
 #include "qapi/qmp/qerror.h"
 #include "qmp-commands.h"
+#include "qapi-event.h"
 
 #include 
 #ifdef CONFIG_LZO
@@ -1662,6 +1663,7 @@ cleanup:
 static void dump_process(DumpState *s, Error **errp)
 {
 Error *local_err = NULL;
+DumpQueryResult *result = NULL;
 
 if (s->has_format && s->format != DUMP_GUEST_MEMORY_FORMAT_ELF) {
 create_kdump_vmcore(s, _err);
@@ -1674,6 +1676,15 @@ static void dump_process(DumpState *s, Error **errp)
 atomic_set(>status,
(local_err ? DUMP_STATUS_FAILED : DUMP_STATUS_COMPLETED));
 
+/* send DUMP_COMPLETED message (unconditionally) */
+result = qmp_query_dump(NULL);
+/* should never fail */
+assert(result);
+qapi_event_send_dump_completed(result, !!local_err, (local_err ? \
+   error_get_pretty(local_err) : NULL),
+   _abort);
+qapi_free_DumpQueryResult(result);
+
 error_propagate(errp, local_err);
 dump_cleanup(s);
 }
@@ -1682,13 +1693,8 @@ static void *dump_thread(void *data)
 {
 Error *err = NULL;
 DumpState *s = (DumpState *)data;
-
 dump_process(s, );
-
-if (err) {
-/* TODO: notify user the error */
-error_free(err);
-}
+error_free(err);
 return NULL;
 }
 
diff --git a/qapi/event.json b/qapi/event.json
index 390fd45..1a45a6c 100644
--- a/qapi/event.json
+++ b/qapi/event.json
@@ -369,3 +369,19 @@
 ##
 { 'event': 'MEM_UNPLUG_ERROR',
   'data': { 'device': 'str', 'msg': 'str' } }
+
+##
+# @DUMP_COMPLETED
+#
+# Emitted when background dump has completed
+#
+# @result: DumpQueryResult type described in qapi-schema.json.
+#
+# @error: #optional human-readable error string that provides
+# hint on why dump failed. Only presents on failure. The
+# user should not try to interpret the error string.
+#
+# Since: 2.6
+##
+{ 'event': 'DUMP_COMPLETED' ,
+  'data': { 'result': 'DumpQueryResult', '*error': 'str' } }
-- 
2.4.3

[Qemu-devel] [PATCH v8 05/11] dump-guest-memory: introduce dump_process() helper function.

[Peter Xu]

No functional change. Cleanup only.

Signed-off-by: Peter Xu 
Reviewed-by: Fam Zheng 
---
 dump.c| 31 +--
 include/sysemu/dump.h |  3 +++
 2 files changed, 24 insertions(+), 10 deletions(-)

diff --git a/dump.c b/dump.c
index 158d6ea..fed84a6 100644
--- a/dump.c
+++ b/dump.c
@@ -1465,6 +1465,9 @@ static void dump_init(DumpState *s, int fd, bool 
has_format,
 Error *err = NULL;
 int ret;
 
+s->has_format = has_format;
+s->format = format;
+
 /* kdump-compressed is conflict with paging and filter */
 if (has_format && format != DUMP_GUEST_MEMORY_FORMAT_ELF) {
 assert(!paging && !has_filter);
@@ -1623,6 +1626,23 @@ cleanup:
 dump_cleanup(s);
 }
 
+/* this operation might be time consuming. */
+static void dump_process(DumpState *s, Error **errp)
+{
+Error *local_err = NULL;
+
+if (s->has_format && s->format != DUMP_GUEST_MEMORY_FORMAT_ELF) {
+create_kdump_vmcore(s, _err);
+} else {
+create_vmcore(s, _err);
+}
+
+s->status = (local_err ? DUMP_STATUS_FAILED : DUMP_STATUS_COMPLETED);
+error_propagate(errp, local_err);
+
+dump_cleanup(s);
+}
+
 void qmp_dump_guest_memory(bool paging, const char *file,
bool has_detach, bool detach,
bool has_begin, int64_t begin, bool has_length,
@@ -1708,16 +1728,7 @@ void qmp_dump_guest_memory(bool paging, const char *file,
 return;
 }
 
-if (has_format && format != DUMP_GUEST_MEMORY_FORMAT_ELF) {
-create_kdump_vmcore(s, _err);
-} else {
-create_vmcore(s, _err);
-}
-
-s->status = (local_err ? DUMP_STATUS_FAILED : DUMP_STATUS_COMPLETED);
-error_propagate(errp, local_err);
-
-dump_cleanup(s);
+dump_process(s, errp);
 }
 
 DumpGuestMemoryCapability *qmp_query_dump_guest_memory_capability(Error **errp)
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index 21fc02d..1da3ddb 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -178,6 +178,9 @@ typedef struct DumpState {
 size_t num_dumpable;/* number of page that can be dumped */
 uint32_t flag_compress; /* indicate the compression format */
 DumpStatus status;  /* current dump status */
+
+bool has_format;  /* whether format is provided */
+DumpGuestMemoryFormat format; /* valid only if has_format == true */
 } DumpState;
 
 uint16_t cpu_to_dump16(DumpState *s, uint16_t val);
-- 
2.4.3

[Qemu-devel] [PATCH v8 02/11] dump-guest-memory: add "detach" flag for QMP/HMP interfaces.

[Peter Xu]

This patch only adds the interfaces, but does not implement them.
"detach" parameter is made optional, to make sure that all the old
dump-guest-memory requests will still be able to work.

Signed-off-by: Peter Xu 
Reviewed-by: Fam Zheng 
---
 dump.c   | 5 +++--
 hmp-commands.hx  | 5 +++--
 hmp.c| 9 +++--
 qapi-schema.json | 8 ++--
 qmp-commands.hx  | 6 --
 5 files changed, 23 insertions(+), 10 deletions(-)

diff --git a/dump.c b/dump.c
index 769c5f9..c4a62d9 100644
--- a/dump.c
+++ b/dump.c
@@ -1609,8 +1609,9 @@ cleanup:
 dump_cleanup(s);
 }
 
-void qmp_dump_guest_memory(bool paging, const char *file, bool has_begin,
-   int64_t begin, bool has_length,
+void qmp_dump_guest_memory(bool paging, const char *file,
+   bool has_detach, bool detach,
+   bool has_begin, int64_t begin, bool has_length,
int64_t length, bool has_format,
DumpGuestMemoryFormat format, Error **errp)
 {
diff --git a/hmp-commands.hx b/hmp-commands.hx
index bb52e4d..664d794 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1056,10 +1056,11 @@ ETEXI
 
 {
 .name   = "dump-guest-memory",
-.args_type  = 
"paging:-p,zlib:-z,lzo:-l,snappy:-s,filename:F,begin:i?,length:i?",
-.params = "[-p] [-z|-l|-s] filename [begin length]",
+.args_type  = 
"paging:-p,detach:-d,zlib:-z,lzo:-l,snappy:-s,filename:F,begin:i?,length:i?",
+.params = "[-p] [-d] [-z|-l|-s] filename [begin length]",
 .help   = "dump guest memory into file 'filename'.\n\t\t\t"
   "-p: do paging to get guest's memory mapping.\n\t\t\t"
+  "-d: return immediately (do not wait for 
completion).\n\t\t\t"
   "-z: dump in kdump-compressed format, with zlib 
compression.\n\t\t\t"
   "-l: dump in kdump-compressed format, with lzo 
compression.\n\t\t\t"
   "-s: dump in kdump-compressed format, with snappy 
compression.\n\t\t\t"
diff --git a/hmp.c b/hmp.c
index 996cb91..7f65b32 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1587,8 +1587,10 @@ void hmp_dump_guest_memory(Monitor *mon, const QDict 
*qdict)
 const char *file = qdict_get_str(qdict, "filename");
 bool has_begin = qdict_haskey(qdict, "begin");
 bool has_length = qdict_haskey(qdict, "length");
+bool has_detach = qdict_haskey(qdict, "detach");
 int64_t begin = 0;
 int64_t length = 0;
+bool detach = false;
 enum DumpGuestMemoryFormat dump_format = DUMP_GUEST_MEMORY_FORMAT_ELF;
 char *prot;
 
@@ -1616,11 +1618,14 @@ void hmp_dump_guest_memory(Monitor *mon, const QDict 
*qdict)
 if (has_length) {
 length = qdict_get_int(qdict, "length");
 }
+if (has_detach) {
+detach = qdict_get_bool(qdict, "detach");
+}
 
 prot = g_strconcat("file:", file, NULL);
 
-qmp_dump_guest_memory(paging, prot, has_begin, begin, has_length, length,
-  true, dump_format, );
+qmp_dump_guest_memory(paging, prot, true, detach, has_begin, begin,
+  has_length, length, true, dump_format, );
 hmp_handle_error(mon, );
 g_free(prot);
 }
diff --git a/qapi-schema.json b/qapi-schema.json
index 8d04897..caff580 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2195,6 +2195,9 @@
 #2. fd: the protocol starts with "fd:", and the following string
 #   is the fd's name.
 #
+# @detach: #optional if true, QMP will return immediately rather than
+#  waiting for the dump to finish. (since 2.6).
+#
 # @begin: #optional if specified, the starting physical address.
 #
 # @length: #optional if specified, the memory size, in bytes. If you don't
@@ -2211,8 +2214,9 @@
 # Since: 1.2
 ##
 { 'command': 'dump-guest-memory',
-  'data': { 'paging': 'bool', 'protocol': 'str', '*begin': 'int',
-'*length': 'int', '*format': 'DumpGuestMemoryFormat' } }
+  'data': { 'paging': 'bool', 'protocol': 'str', '*detach': 'bool',
+'*begin': 'int', '*length': 'int',
+'*format': 'DumpGuestMemoryFormat'} }
 
 ##
 # @DumpGuestMemoryCapability:
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 9fb0d78..d30b1eb 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -838,8 +838,8 @@ EQMP
 
 {
 .name   = "dump-guest-memory",
-.args_type  = "paging:b,protocol:s,begin:i?,end:i?,format:s?",
-.params = "-p protocol [begin] [length] [format]",
+.args_type  = 
"paging:b,protocol:s,detach:b?,begin:i?,end:i?,format:s?",
+.params = "-p protocol [-d] [begin] [length] [format]",
 .help   = "dump guest memory to file",
 .mhandler.cmd_new = qmp_marshal_dump_guest_memory,
 },
@@ -855,6 +855,8 @@ Arguments:
 - "paging": do paging to get guest's memory mapping (json-bool)
 - "protocol": 

[Qemu-devel] [PATCH v8 08/11] DumpState: adding total_size and written_size fields

[Peter Xu]

Here, total_size is the size in bytes to be dumped (raw data, which
means before compression), while written_size are bytes handled (raw
size too).

Signed-off-by: Peter Xu 
---
 dump.c| 32 
 include/sysemu/dump.h |  9 +
 2 files changed, 41 insertions(+)

diff --git a/dump.c b/dump.c
index 9210a72..cd4ae5e 100644
--- a/dump.c
+++ b/dump.c
@@ -331,6 +331,8 @@ static void write_data(DumpState *s, void *buf, int length, 
Error **errp)
 ret = fd_write_vmcore(buf, length, s);
 if (ret < 0) {
 error_setg(errp, "dump: failed to save memory");
+} else {
+s->written_size += length;
 }
 }
 
@@ -1324,6 +1326,7 @@ static void write_dump_pages(DumpState *s, Error **errp)
 goto out;
 }
 }
+s->written_size += s->dump_info.page_size;
 }
 
 ret = write_cache(_desc, NULL, 0, true);
@@ -1456,6 +1459,30 @@ bool dump_in_progress(void)
 return (state->status == DUMP_STATUS_ACTIVE);
 }
 
+/* calculate total size of memory to be dumped (taking filter into
+ * acoount.) */
+static int64_t dump_calculate_size(DumpState *s)
+{
+GuestPhysBlock *block;
+int64_t size = 0, total = 0, left = 0, right = 0;
+
+QTAILQ_FOREACH(block, >guest_phys_blocks.head, next) {
+if (s->has_filter) {
+/* calculate the overlapped region. */
+left = MAX(s->begin, block->target_start);
+right = MIN(s->begin + s->length, block->target_end);
+size = right - left;
+size = size > 0 ? size : 0;
+} else {
+/* count the whole region in */
+size = (block->target_end - block->target_start);
+}
+total += size;
+}
+
+return total;
+}
+
 static void dump_init(DumpState *s, int fd, bool has_format,
   DumpGuestMemoryFormat format, bool paging, bool 
has_filter,
   int64_t begin, int64_t length, Error **errp)
@@ -1467,6 +1494,7 @@ static void dump_init(DumpState *s, int fd, bool 
has_format,
 
 s->has_format = has_format;
 s->format = format;
+s->written_size = 0;
 
 /* kdump-compressed is conflict with paging and filter */
 if (has_format && format != DUMP_GUEST_MEMORY_FORMAT_ELF) {
@@ -1498,6 +1526,10 @@ static void dump_init(DumpState *s, int fd, bool 
has_format,
 
 guest_phys_blocks_init(>guest_phys_blocks);
 guest_phys_blocks_append(>guest_phys_blocks);
+s->total_size = dump_calculate_size(s);
+#ifdef DEBUG_DUMP_GUEST_MEMORY
+fprintf(stderr, "DUMP: total memory to dump: %lu\n", s->total_size);
+#endif
 
 s->start = get_start_block(s);
 if (s->start == -1) {
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index 06393c3..ef931be 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -182,6 +182,15 @@ typedef struct DumpState {
 bool has_format;  /* whether format is provided */
 DumpGuestMemoryFormat format; /* valid only if has_format == true */
 QemuThread dump_thread;   /* thread for detached dump */
+
+int64_t total_size;  /* total memory size (in bytes) to
+  * be dumped. When filter is
+  * enabled, this will only count
+  * those to be written. */
+int64_t written_size;/* written memory size (in bytes),
+  * this could be used to calculate
+  * how much work we have
+  * finished. */
 } DumpState;
 
 uint16_t cpu_to_dump16(DumpState *s, uint16_t val);
-- 
2.4.3

[Qemu-devel] [PATCH v8 04/11] dump-guest-memory: add dump_in_progress() helper function

[Peter Xu]

For now, it has no effect. It will be used in dump detach support.

Signed-off-by: Peter Xu 
Reviewed-by: Fam Zheng 
---
 dump.c| 13 +
 include/qemu-common.h |  4 
 qmp.c | 14 ++
 3 files changed, 31 insertions(+)

diff --git a/dump.c b/dump.c
index 434bc60..158d6ea 100644
--- a/dump.c
+++ b/dump.c
@@ -1450,6 +1450,12 @@ static void dump_state_prepare(DumpState *s)
 *s = (DumpState) { .status = DUMP_STATUS_ACTIVE };
 }
 
+bool dump_in_progress(void)
+{
+DumpState *state = _state_global;
+return (state->status == DUMP_STATUS_ACTIVE);
+}
+
 static void dump_init(DumpState *s, int fd, bool has_format,
   DumpGuestMemoryFormat format, bool paging, bool 
has_filter,
   int64_t begin, int64_t length, Error **errp)
@@ -1628,6 +1634,13 @@ void qmp_dump_guest_memory(bool paging, const char *file,
 DumpState *s;
 Error *local_err = NULL;
 
+/* if there is a dump in background, we should wait until the dump
+ * finished */
+if (dump_in_progress()) {
+error_setg(errp, "There is a dump in process, please wait.");
+return;
+}
+
 /*
  * kdump-compressed format need the whole memory dumped, so paging or
  * filter is not supported here.
diff --git a/include/qemu-common.h b/include/qemu-common.h
index f557be7..59ab759 100644
--- a/include/qemu-common.h
+++ b/include/qemu-common.h
@@ -494,4 +494,8 @@ int parse_debug_env(const char *name, int max, int initial);
 const char *qemu_ether_ntoa(const MACAddr *mac);
 void page_size_init(void);
 
+/* returns non-zero if dump is in progress, otherwise zero is
+ * returned. */
+bool dump_in_progress(void);
+
 #endif
diff --git a/qmp.c b/qmp.c
index 9a93d5e..3f16a77 100644
--- a/qmp.c
+++ b/qmp.c
@@ -103,6 +103,13 @@ void qmp_quit(Error **errp)
 
 void qmp_stop(Error **errp)
 {
+/* if there is a dump in background, we should wait until the dump
+ * finished */
+if (dump_in_progress()) {
+error_setg(errp, "There is a dump in process, please wait.");
+return;
+}
+
 if (runstate_check(RUN_STATE_INMIGRATE)) {
 autostart = 0;
 } else {
@@ -175,6 +182,13 @@ void qmp_cont(Error **errp)
 BlockBackend *blk;
 BlockDriverState *bs;
 
+/* if there is a dump in background, we should wait until the dump
+ * finished */
+if (dump_in_progress()) {
+error_setg(errp, "There is a dump in process, please wait.");
+return;
+}
+
 if (runstate_needs_reset()) {
 error_setg(errp, "Resetting the Virtual Machine is required");
 return;
-- 
2.4.3

[Qemu-devel] [PATCH v8 03/11] dump-guest-memory: using static DumpState, add DumpStatus

[Peter Xu]

Instead of malloc/free each time for DumpState, make it
static. Added DumpStatus to show status for dump.

This is to be used for detached dump.

Signed-off-by: Peter Xu 
Reviewed-by: Fam Zheng 
---
 dump.c| 21 -
 include/sysemu/dump.h |  2 ++
 qapi-schema.json  | 18 ++
 3 files changed, 36 insertions(+), 5 deletions(-)

diff --git a/dump.c b/dump.c
index c4a62d9..434bc60 100644
--- a/dump.c
+++ b/dump.c
@@ -1442,6 +1442,14 @@ static void get_max_mapnr(DumpState *s)
 s->max_mapnr = dump_paddr_to_pfn(s, last_block->target_end);
 }
 
+static DumpState dump_state_global = { .status = DUMP_STATUS_NONE };
+
+static void dump_state_prepare(DumpState *s)
+{
+/* zero the struct, setting status to active */
+*s = (DumpState) { .status = DUMP_STATUS_ACTIVE };
+}
+
 static void dump_init(DumpState *s, int fd, bool has_format,
   DumpGuestMemoryFormat format, bool paging, bool 
has_filter,
   int64_t begin, int64_t length, Error **errp)
@@ -1676,24 +1684,27 @@ void qmp_dump_guest_memory(bool paging, const char 
*file,
 return;
 }
 
-s = g_malloc0(sizeof(DumpState));
+s = _state_global;
+dump_state_prepare(s);
 
 dump_init(s, fd, has_format, format, paging, has_begin,
   begin, length, _err);
 if (local_err) {
-g_free(s);
 error_propagate(errp, local_err);
+s->status = DUMP_STATUS_FAILED;
 return;
 }
 
 if (has_format && format != DUMP_GUEST_MEMORY_FORMAT_ELF) {
-create_kdump_vmcore(s, errp);
+create_kdump_vmcore(s, _err);
 } else {
-create_vmcore(s, errp);
+create_vmcore(s, _err);
 }
 
+s->status = (local_err ? DUMP_STATUS_FAILED : DUMP_STATUS_COMPLETED);
+error_propagate(errp, local_err);
+
 dump_cleanup(s);
-g_free(s);
 }
 
 DumpGuestMemoryCapability *qmp_query_dump_guest_memory_capability(Error **errp)
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index 2f04b24..21fc02d 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -38,6 +38,7 @@
 
 #include "sysemu/dump-arch.h"
 #include "sysemu/memory_mapping.h"
+#include "qapi-types.h"
 
 typedef struct QEMU_PACKED MakedumpfileHeader {
 char signature[16]; /* = "makedumpfile" */
@@ -176,6 +177,7 @@ typedef struct DumpState {
 off_t offset_page;  /* offset of page part in vmcore */
 size_t num_dumpable;/* number of page that can be dumped */
 uint32_t flag_compress; /* indicate the compression format */
+DumpStatus status;  /* current dump status */
 } DumpState;
 
 uint16_t cpu_to_dump16(DumpState *s, uint16_t val);
diff --git a/qapi-schema.json b/qapi-schema.json
index caff580..ccd30c8 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2219,6 +2219,24 @@
 '*format': 'DumpGuestMemoryFormat'} }
 
 ##
+# @DumpStatus
+#
+# Describe the status of a long-running background guest memory dump.
+#
+# @none: no dump-guest-memory has started yet.
+#
+# @active: there is one dump running in background.
+#
+# @completed: the last dump has finished successfully.
+#
+# @failed: the last dump has failed.
+#
+# Since 2.6
+##
+{ 'enum': 'DumpStatus',
+  'data': [ 'none', 'active', 'completed', 'failed' ] }
+
+##
 # @DumpGuestMemoryCapability:
 #
 # A list of the available formats for dump-guest-memory
-- 
2.4.3

[Qemu-devel] [PATCH v8 01/11] dump-guest-memory: cleanup: removing dump_{error|cleanup}().

[Peter Xu]

It might be a little bit confusing and error prone to do
dump_cleanup() in these two functions. A better way is to do
dump_cleanup() before dump finish, no matter whether dump has
succeeded or not.

Signed-off-by: Peter Xu 
Reviewed-by: Fam Zheng 
---
 dump.c | 78 +++---
 1 file changed, 32 insertions(+), 46 deletions(-)

diff --git a/dump.c b/dump.c
index 96e1fc1..769c5f9 100644
--- a/dump.c
+++ b/dump.c
@@ -82,12 +82,6 @@ static int dump_cleanup(DumpState *s)
 return 0;
 }
 
-static void dump_error(DumpState *s, const char *reason, Error **errp)
-{
-dump_cleanup(s);
-error_setg(errp, "%s", reason);
-}
-
 static int fd_write_vmcore(const void *buf, size_t size, void *opaque)
 {
 DumpState *s = opaque;
@@ -128,7 +122,7 @@ static void write_elf64_header(DumpState *s, Error **errp)
 
 ret = fd_write_vmcore(_header, sizeof(elf_header), s);
 if (ret < 0) {
-dump_error(s, "dump: failed to write elf header", errp);
+error_setg(errp, "dump: failed to write elf header");
 }
 }
 
@@ -159,7 +153,7 @@ static void write_elf32_header(DumpState *s, Error **errp)
 
 ret = fd_write_vmcore(_header, sizeof(elf_header), s);
 if (ret < 0) {
-dump_error(s, "dump: failed to write elf header", errp);
+error_setg(errp, "dump: failed to write elf header");
 }
 }
 
@@ -182,7 +176,7 @@ static void write_elf64_load(DumpState *s, MemoryMapping 
*memory_mapping,
 
 ret = fd_write_vmcore(, sizeof(Elf64_Phdr), s);
 if (ret < 0) {
-dump_error(s, "dump: failed to write program header table", errp);
+error_setg(errp, "dump: failed to write program header table");
 }
 }
 
@@ -205,7 +199,7 @@ static void write_elf32_load(DumpState *s, MemoryMapping 
*memory_mapping,
 
 ret = fd_write_vmcore(, sizeof(Elf32_Phdr), s);
 if (ret < 0) {
-dump_error(s, "dump: failed to write program header table", errp);
+error_setg(errp, "dump: failed to write program header table");
 }
 }
 
@@ -225,7 +219,7 @@ static void write_elf64_note(DumpState *s, Error **errp)
 
 ret = fd_write_vmcore(, sizeof(Elf64_Phdr), s);
 if (ret < 0) {
-dump_error(s, "dump: failed to write program header table", errp);
+error_setg(errp, "dump: failed to write program header table");
 }
 }
 
@@ -245,7 +239,7 @@ static void write_elf64_notes(WriteCoreDumpFunction f, 
DumpState *s,
 id = cpu_index(cpu);
 ret = cpu_write_elf64_note(f, cpu, id, s);
 if (ret < 0) {
-dump_error(s, "dump: failed to write elf notes", errp);
+error_setg(errp, "dump: failed to write elf notes");
 return;
 }
 }
@@ -253,7 +247,7 @@ static void write_elf64_notes(WriteCoreDumpFunction f, 
DumpState *s,
 CPU_FOREACH(cpu) {
 ret = cpu_write_elf64_qemunote(f, cpu, s);
 if (ret < 0) {
-dump_error(s, "dump: failed to write CPU status", errp);
+error_setg(errp, "dump: failed to write CPU status");
 return;
 }
 }
@@ -275,7 +269,7 @@ static void write_elf32_note(DumpState *s, Error **errp)
 
 ret = fd_write_vmcore(, sizeof(Elf32_Phdr), s);
 if (ret < 0) {
-dump_error(s, "dump: failed to write program header table", errp);
+error_setg(errp, "dump: failed to write program header table");
 }
 }
 
@@ -290,7 +284,7 @@ static void write_elf32_notes(WriteCoreDumpFunction f, 
DumpState *s,
 id = cpu_index(cpu);
 ret = cpu_write_elf32_note(f, cpu, id, s);
 if (ret < 0) {
-dump_error(s, "dump: failed to write elf notes", errp);
+error_setg(errp, "dump: failed to write elf notes");
 return;
 }
 }
@@ -298,7 +292,7 @@ static void write_elf32_notes(WriteCoreDumpFunction f, 
DumpState *s,
 CPU_FOREACH(cpu) {
 ret = cpu_write_elf32_qemunote(f, cpu, s);
 if (ret < 0) {
-dump_error(s, "dump: failed to write CPU status", errp);
+error_setg(errp, "dump: failed to write CPU status");
 return;
 }
 }
@@ -326,7 +320,7 @@ static void write_elf_section(DumpState *s, int type, Error 
**errp)
 
 ret = fd_write_vmcore(, shdr_size, s);
 if (ret < 0) {
-dump_error(s, "dump: failed to write section header table", errp);
+error_setg(errp, "dump: failed to write section header table");
 }
 }
 
@@ -336,7 +330,7 @@ static void write_data(DumpState *s, void *buf, int length, 
Error **errp)
 
 ret = fd_write_vmcore(buf, length, s);
 if (ret < 0) {
-dump_error(s, "dump: failed to save memory", errp);
+error_setg(errp, "dump: failed to save memory");
 }
 }
 
@@ -568,11 +562,6 @@ static void dump_begin(DumpState *s, Error **errp)
 }
 }
 
-static void dump_completed(DumpState *s)
-{
-dump_cleanup(s);
-}
-
 static int get_next_block(DumpState *s, 

Re: [Qemu-devel] [PATCH v2 0/2] move qcow2_invalidate_cache() out of coroutine context

[Amit Shah]

Can someone from the block team please give this a review?

Thanks,

On (Fri) 12 Feb 2016 [09:39:32], Denis V. Lunev wrote:
> There is a possibility to hit an assert in qcow2_get_specific_info that
> s->qcow_version is undefined. This happens when VM in starting from
> suspended state, i.e. it processes incoming migration, and in the same
> time 'info block' is called.
> 
> The problem is that qcow2_invalidate_cache() closes the image and
> memset()s BDRVQcowState in the middle.
> 
> This operation should not be performed in coroutine context.
> 
> Changes from v1:
> - fixed spelling. Eric, thank you for spell checking
> 
> Signed-off-by: Denis V. Lunev 
> CC: Paolo Bonzini 
> CC: Juan Quintela 
> CC: Amit Shah 
> 

Amit

[Qemu-devel] [PATCH v7 00/11] Add basic "detach" support for dump-guest-memory

[Peter Xu]

Changes from v7:
- patch 8: use s->dump_info.page_size not TARGET_PAGE_SIZE
- patch 10: change DUMP_STATUS_MAX to DUMP_STATUS__MAX (this is to
  fix compile error for rebasing to latest master branch, still do
  not know why we need this change from generating "_MAX" to
  "__MAX" for enum types...)

Changes from v6:
- patch 9: fix leak of local_err due to patch switch.. [Fam]
- patch 10: assert "result" before use [Fam]
- patch 11: add Fam's reviewed-by.

For older patch, please refers to v6 series:

https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg01299.html

Peter Xu (11):
  dump-guest-memory: cleanup: removing dump_{error|cleanup}().
  dump-guest-memory: add "detach" flag for QMP/HMP interfaces.
  dump-guest-memory: using static DumpState, add DumpStatus
  dump-guest-memory: add dump_in_progress() helper function
  dump-guest-memory: introduce dump_process() helper function.
  dump-guest-memory: disable dump when in INMIGRATE state
  dump-guest-memory: add "detach" support
  DumpState: adding total_size and written_size fields
  Dump: add qmp command "query-dump"
  Dump: add hmp command "info dump"
  dump-guest-memory: add qmp event DUMP_COMPLETED

 docs/qmp-events.txt |  18 
 dump.c  | 215 ++--
 hmp-commands-info.hx|  14 +++
 hmp-commands.hx |   5 +-
 hmp.c   |  26 -
 hmp.h   |   1 +
 include/qemu-common.h   |   4 +
 include/sysemu/dump.h   |  15 +++
 include/sysemu/memory_mapping.h |   4 +
 memory_mapping.c|   3 +
 qapi-schema.json|  56 ++-
 qapi/event.json |  16 +++
 qmp-commands.hx |  31 +-
 qmp.c   |  14 +++
 14 files changed, 359 insertions(+), 63 deletions(-)

-- 
2.4.3

Re: [Qemu-devel] [RFC PATCH 2/2] pci: add PCIIOMMUOps and PCIIOMMUIntRemapFunc

[Peter Xu]

On Wed, Feb 17, 2016 at 08:46:18PM +0100, Paolo Bonzini wrote:
> 
> 
> On 17/02/2016 11:25, Peter Xu wrote:
> > This patch extended the current PCI IOMMU functions into operation list,
> > one new op is added to do interrupt remapping.
> > 
> > Currently it is not working since int_remap is always NULL. It only
> > provide a interface to extend PCI MSI to support interrupt remapping in
> > the future.
> > 
> > One helper function pci_setup_iommu_ops() is introduced. We can use this
> > instead of the origin pci_setup_iommu() one to extend interrupt
> > remapping on specific platform.
> 
> For MSI, I think interrupt remapping can be done directly in the IOMMU
> MemoryRegion.  You can just overlay a new MemoryRegion on top of the
> IOMMU region where MSIs are sent (that's around 0xFEE0, I don't
> remember where exactly).  It will catch interrupts sent by the device,
> remap them and forward them to the right interrupt destination in the host.

Yes, it should be 0xfee0. I'd say this is a much better idea, so
that I can leverage current memory region codes and avoid touching
PCI at all.

If the work is in iommu part, I think I can send my next RFC with
basic IR together next time.

> 
> I'm not sure about INTX interrupts, but I think that the host kernel
> remaps them simply by virtualizing the IOAPIC's redirection table.

Yes, what I understand is that, IOAPIC is handling all INTX
interrupts. To remap these interrupts, we just need a translation
for the IOAPIC IRQ table entries before the interrupts are delivered
to APIC bus.

There will need some code change in ACPI too to enumerate the IOAPIC
device in DMAR region, so that we can declare that "this IOMMU owns
the default IOAPIC".

If so, I can call vtd_* function in ioapic_service() directly right?
IIUC IOAPIC should be intel-specific too?

Thanks!
Peter

> 
> Paolo

Re: [Qemu-devel] [RFC 1/1] nbd (specification): add NBD_CMD_WRITE_ZEROES command

[Denis V. Lunev]

On 02/17/2016 11:58 PM, Eric Blake wrote:

On 02/17/2016 11:10 AM, Denis V. Lunev wrote:

This patch proposes a new command to reduce the amount of data passed
through the wire when it is known that the data is all zeroes. This
functionality is generally useful for mirroring or backup operations.

Currently available NBD_CMD_TRIM command can not be used as the
specification explicitely says that "a client MUST NOT make any

s/explicitely/explicitly/


assumptions about the contents of the export affected by this
[NBD_CMD_TRIM] command, until overwriting it again with `NBD_CMD_WRITE`"

Particular use case could be the following:

QEMU project uses own implementation of NBD server to transfer data
in between different instances of QEMU. Typically we tranfer VM virtual

s/tranfer/transfer/


disks over this channel. VM virtual disks are sparse and thus the
efficiency of backup and mirroring operations could be improved a lot.

Signed-off-by: Denis V. Lunev 
---
  doc/proto.md | 7 +++
  1 file changed, 7 insertions(+)

diff --git a/doc/proto.md b/doc/proto.md
index 43065b7..c94751a 100644
--- a/doc/proto.md
+++ b/doc/proto.md
@@ -241,6 +241,8 @@ immediately after the global flags field in oldstyle 
negotiation:
schedule I/O accesses as for a rotational medium
  - bit 5, `NBD_FLAG_SEND_TRIM`; should be set to 1 if the server supports
`NBD_CMD_TRIM` commands
+- bit 6, `NBD_FLAG_SEND_WRITE_ZEROES`; should be set to 1 if the server
+  supports `NBD_CMD_WRITE_ZEROES` commands
  
  # Client flags
  
@@ -446,6 +448,11 @@ The following request types exist:

  about the contents of the export affected by this command, until
  overwriting it again with `NBD_CMD_WRITE`.
  
+* `NBD_CMD_WRITE_ZEROES` (6)

+
+A request to write zeroes. The command is functional equivalent of
+the NBD_WRITE_COMMAND but without payload sent through the channel.

This lets us push holes during writes.
from my point this allows client to apply his policy. For QCOW2 output 
target the
client could skip the block. For RAW file he could decide whether to use 
UNMAP

and produce sparse file or use fallocate.

  Do we have the converse
operation, that is, an easy way to query if a block of data will read as
all zeroes, and therefore the client can bypass reading that portion of
the disk (in other words, an equivalent to lseek(SEEK_HOLE/SEEK_DATA))?


exactly!

static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)
...
ret = bdrv_get_block_status_above(source, NULL, sector_num,  
<--- query block state

  nb_sectors, , );
if (ret < 0 || pnum < nb_sectors ||
(ret & BDRV_BLOCK_DATA && !(ret & BDRV_BLOCK_ZERO))) {
bdrv_aio_readv(source, sector_num, >qiov, nb_sectors,
   mirror_read_complete, op);
} else if (ret & BDRV_BLOCK_ZERO) {
bdrv_aio_write_zeroes(s->target, sector_num, op->nb_sectors,  
<-- skip read op if allowed

  s->unmap ? BDRV_REQ_MAY_UNMAP : 0,
  mirror_write_complete, op);
} else {
assert(!(ret & BDRV_BLOCK_DATA));
bdrv_aio_discard(s->target, sector_num, op->nb_sectors,
 mirror_write_complete, op);
}
return delay_ns;

Actually I have tried early at day begins to add .bdrv_co_write_zeroes
callback to NBD and it just works as expected. The problem is that
callback can not be written using NDB_SEND_TRIM to conform with the
NBD spec. But in QEMU -> QEMU communication it just works.

http://lists.nongnu.org/archive/html/qemu-devel/2016-02/msg03810.html

Den

Re: [Qemu-devel] [RFC PATCH 0/2] ARM: add QMP command to query GIC version

[Peter Xu]

On Mon, Feb 15, 2016 at 03:22:05PM +, Daniel P. Berrange wrote:
> On Mon, Feb 15, 2016 at 04:08:33PM +0100, Markus Armbruster wrote:
> > Peter Xu  writes:
> > 
> > > On Mon, Feb 15, 2016 at 10:52:01AM +0100, Markus Armbruster wrote:
> > >> Peter Xu  writes:
> > >> 
> > >> > For ARM platform, we still do not have any interface to query
> > >> > whether current QEMU/host support specific GIC version. This
> > >> > patchset is trying to add one QMP interface for that. By querying
> > >> > the GIC capability using the new interface, one should know exactly
> > >> > what GIC version(s) the platform will support. The capability bits
> > >> > will be decided by both QEMU and host kernel.
> > >> >
> > >> > The current patchset only provides interface for review. Its handler
> > >> > is a fake one which returns empty always.
> > >> >
> > >> > The command interface I am planning to add is something like this:
> > >> >
> > >> > -> { "execute": "query-gic-capability" }
> > >> > <- { "return": [ "gicv2", "gicv2-kvm", "gicv3-kvm" ] }
> > >> >
> > >> > Currently, all the possible supported GIC versions are:
> > >> >
> > >> > - gicv2:  GIC version 2 without kernel IRQ chip
> > >> > - gicv2-kvm:  GIC version 2 with kernel IRQ chip
> > >> > - gicv3:  GIC version 3 without kernel IRQ chip (not supported)
> > >> > - gicv3-kvm:  GIC version 3 with kernel IRQ chip
> > >> >
> > >> > Since "gicv3" is still not supported (to use GICv3, kernel irqchip
> > >> > support is required for now, which corresponds to "gicv3-kvm"),
> > >> > currently the maximum superset of the result should be:
> > >> >
> > >> > ["gicv2", "gicv2-kvm", "gicv3-kvm"]
> > >> >
> > >> > Please help review whether the interface suits our need, also please
> > >> > point out any error I have made.
> > >> 
> > >> Adding ad hoc queries as we go won't scale.  Is there really no generic
> > >> way to get this information, e.g. with qom-get?
> > >
> > > Haven't used "qom-get" before, but it seems to fetch one property
> > > for a specific object. If so, will it be strange to hide some
> > > capability bits into every GIC objects (though there is possibly
> > > one object)?
> > 
> > Pardon my ignorance...  what are these "GIC objects"?
> > 
> > What exactly is the "GIC type", and how would the result of
> > query-gic-capability be used?
> > 
> > > I agree that we should keep the interface as simple as
> > > possible. I see that there are already commands that works just like
> > > this one, which is to query some capabilities from QEMU, like:
> > >
> > > - query-dump-guest-memory-capability
> > > - query-migrate-capabilities
> > 
> > I'm not saying we must not add another ad hoc query.  I'm trying to
> > figure out whether existing generic infrastructure can serve, or be
> > adapted to serve.  Once we establish the answer is "no" or "badly", I'm
> > willing to consider the ad hoc query.
> 
> Looking at this from the POV of solving the generic problem, what we
> have here is an object with an integer property, for which only certain
> values are permitted based on what host kernel/hardware we're runing
> on.
> 
> So to solve this generically, we would need a way to provide information
> in QOM as to what permitted values are for any given property. This would
> make sense for at least bool, int and enum properties, since they can all
> potentially have scenarios where the possible range of values is greater
> than the currently permissible range of values.

Is this work on any of our todo list (or anyone has started the
prototyping)?

It seems reasonable to provide such a generic interface, rather than
adding a "query-gic-capability" for GIC versions only. The problem
is that, I am not sure how eagerly we are wanting this GIC
interface, and when will this framework be there in QOM.

Thanks.
Peter

> 
> Regards,
> Daniel
> -- 
> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org  -o- http://virt-manager.org :|
> |: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
> |: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

[Qemu-devel] [PULL 19/26] cuda: port FILE_SERVER_FLAG command to new framework

[David Gibson]

From: Hervé Poussineau 

This command tells if computer should automatically wake-up after a power loss.

Reviewed-by: David Gibson 
Signed-off-by: Hervé Poussineau 
Reviewed-by: Mark Cave-Ayland 
Signed-off-by: David Gibson 
---
 hw/misc/macio/cuda.c | 17 -
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/hw/misc/macio/cuda.c b/hw/misc/macio/cuda.c
index ca11fc8..4ddec34 100644
--- a/hw/misc/macio/cuda.c
+++ b/hw/misc/macio/cuda.c
@@ -626,12 +626,28 @@ static bool cuda_cmd_reset_system(CUDAState *s,
 return true;
 }
 
+static bool cuda_cmd_set_file_server_flag(CUDAState *s,
+  const uint8_t *in_data, int in_len,
+  uint8_t *out_data, int *out_len)
+{
+if (in_len != 1) {
+return false;
+}
+
+qemu_log_mask(LOG_UNIMP,
+  "CUDA: unimplemented command FILE_SERVER_FLAG %d\n",
+  in_data[0]);
+return true;
+}
+
 static const CudaCommand handlers[] = {
 { CUDA_AUTOPOLL, "AUTOPOLL", cuda_cmd_autopoll },
 { CUDA_SET_AUTO_RATE, "SET_AUTO_RATE",  cuda_cmd_set_autorate },
 { CUDA_SET_DEVICE_LIST, "SET_DEVICE_LIST", cuda_cmd_set_device_list },
 { CUDA_POWERDOWN, "POWERDOWN", cuda_cmd_powerdown },
 { CUDA_RESET_SYSTEM, "RESET_SYSTEM", cuda_cmd_reset_system },
+{ CUDA_FILE_SERVER_FLAG, "FILE_SERVER_FLAG",
+  cuda_cmd_set_file_server_flag },
 };
 
 static void cuda_receive_packet(CUDAState *s,
@@ -679,7 +695,6 @@ static void cuda_receive_packet(CUDAState *s,
 obuf[6] = ti;
 cuda_send_packet_to_host(s, obuf, 7);
 return;
-case CUDA_FILE_SERVER_FLAG:
 case CUDA_SET_POWER_MESSAGES:
 cuda_send_packet_to_host(s, obuf, 3);
 return;
-- 
2.5.0

[Qemu-devel] [PULL 15/26] cuda: port SET_AUTO_RATE command to new framework

[David Gibson]

From: Hervé Poussineau 

Also implement the command, by removing the hardcoded period of 20 ms/50 Hz
and replacing it by the one requested by user.
Update VMState version to store this new parameter.

Signed-off-by: Hervé Poussineau 
Reviewed-by: Mark Cave-Ayland 
Signed-off-by: David Gibson 
---
 hw/misc/macio/cuda.c | 36 ++--
 hw/ppc/mac.h |  1 +
 2 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/hw/misc/macio/cuda.c b/hw/misc/macio/cuda.c
index ea4237a..71fd97c 100644
--- a/hw/misc/macio/cuda.c
+++ b/hw/misc/macio/cuda.c
@@ -106,7 +106,6 @@
 #define CUDA_COMBINED_FORMAT_IIC   0x25
 
 #define CUDA_TIMER_FREQ (470 / 6)
-#define CUDA_ADB_POLL_FREQ 50
 
 /* CUDA returns time_t's offset from Jan 1, 1904, not 1970 */
 #define RTC_OFFSET  2082844800
@@ -532,7 +531,7 @@ static void cuda_adb_poll(void *opaque)
 }
 timer_mod(s->adb_poll_timer,
qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) +
-   (get_ticks_per_sec() / CUDA_ADB_POLL_FREQ));
+   (get_ticks_per_sec() / (1000 / s->autopoll_rate_ms)));
 }
 
 /* description of commands */
@@ -560,7 +559,7 @@ static bool cuda_cmd_autopoll(CUDAState *s,
 if (autopoll) {
 timer_mod(s->adb_poll_timer,
   qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) +
-  (get_ticks_per_sec() / CUDA_ADB_POLL_FREQ));
+  (get_ticks_per_sec() / (1000 / s->autopoll_rate_ms)));
 } else {
 timer_del(s->adb_poll_timer);
 }
@@ -568,8 +567,32 @@ static bool cuda_cmd_autopoll(CUDAState *s,
 return true;
 }
 
+static bool cuda_cmd_set_autorate(CUDAState *s,
+  const uint8_t *in_data, int in_len,
+  uint8_t *out_data, int *out_len)
+{
+if (in_len != 1) {
+return false;
+}
+
+/* we don't want a period of 0 ms */
+/* FIXME: check what real hardware does */
+if (in_data[0] == 0) {
+return false;
+}
+
+s->autopoll_rate_ms = in_data[0];
+if (s->autopoll) {
+timer_mod(s->adb_poll_timer,
+  qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) +
+  (get_ticks_per_sec() / (1000 / s->autopoll_rate_ms)));
+}
+return true;
+}
+
 static const CudaCommand handlers[] = {
 { CUDA_AUTOPOLL, "AUTOPOLL", cuda_cmd_autopoll },
+{ CUDA_SET_AUTO_RATE, "SET_AUTO_RATE",  cuda_cmd_set_autorate },
 };
 
 static void cuda_receive_packet(CUDAState *s,
@@ -619,7 +642,6 @@ static void cuda_receive_packet(CUDAState *s,
 return;
 case CUDA_FILE_SERVER_FLAG:
 case CUDA_SET_DEVICE_LIST:
-case CUDA_SET_AUTO_RATE:
 case CUDA_SET_POWER_MESSAGES:
 cuda_send_packet_to_host(s, obuf, 3);
 return;
@@ -756,8 +778,8 @@ static const VMStateDescription vmstate_cuda_timer = {
 
 static const VMStateDescription vmstate_cuda = {
 .name = "cuda",
-.version_id = 3,
-.minimum_version_id = 3,
+.version_id = 4,
+.minimum_version_id = 4,
 .fields = (VMStateField[]) {
 VMSTATE_UINT8(a, CUDAState),
 VMSTATE_UINT8(b, CUDAState),
@@ -775,6 +797,7 @@ static const VMStateDescription vmstate_cuda = {
 VMSTATE_INT32(data_in_index, CUDAState),
 VMSTATE_INT32(data_out_index, CUDAState),
 VMSTATE_UINT8(autopoll, CUDAState),
+VMSTATE_UINT8(autopoll_rate_ms, CUDAState),
 VMSTATE_BUFFER(data_in, CUDAState),
 VMSTATE_BUFFER(data_out, CUDAState),
 VMSTATE_UINT32(tick_offset, CUDAState),
@@ -828,6 +851,7 @@ static void cuda_realizefn(DeviceState *dev, Error **errp)
 s->tick_offset = (uint32_t)mktimegm() + RTC_OFFSET;
 
 s->adb_poll_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, cuda_adb_poll, s);
+s->autopoll_rate_ms = 20;
 }
 
 static void cuda_initfn(Object *obj)
diff --git a/hw/ppc/mac.h b/hw/ppc/mac.h
index ecf7792..887c8c1 100644
--- a/hw/ppc/mac.h
+++ b/hw/ppc/mac.h
@@ -111,6 +111,7 @@ typedef struct CUDAState {
 int data_out_index;
 
 qemu_irq irq;
+uint8_t autopoll_rate_ms;
 uint8_t autopoll;
 uint8_t data_in[128];
 uint8_t data_out[16];
-- 
2.5.0

[Qemu-devel] [PULL 22/26] cuda: port SET_TIME command to new framework

[David Gibson]

From: Hervé Poussineau 

Reviewed-by: David Gibson 
Signed-off-by: Hervé Poussineau 
Reviewed-by: Mark Cave-Ayland 
Signed-off-by: David Gibson 
---
 hw/misc/macio/cuda.c | 24 ++--
 1 file changed, 18 insertions(+), 6 deletions(-)

diff --git a/hw/misc/macio/cuda.c b/hw/misc/macio/cuda.c
index 8ee1414..611b414 100644
--- a/hw/misc/macio/cuda.c
+++ b/hw/misc/macio/cuda.c
@@ -674,6 +674,23 @@ static bool cuda_cmd_get_time(CUDAState *s,
 return true;
 }
 
+static bool cuda_cmd_set_time(CUDAState *s,
+  const uint8_t *in_data, int in_len,
+  uint8_t *out_data, int *out_len)
+{
+uint32_t ti;
+
+if (in_len != 4) {
+return false;
+}
+
+ti = (((uint32_t)in_data[1]) << 24) + (((uint32_t)in_data[2]) << 16)
+ + (((uint32_t)in_data[3]) << 8) + in_data[4];
+s->tick_offset = ti - (qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL)
+   / get_ticks_per_sec());
+return true;
+}
+
 static const CudaCommand handlers[] = {
 { CUDA_AUTOPOLL, "AUTOPOLL", cuda_cmd_autopoll },
 { CUDA_SET_AUTO_RATE, "SET_AUTO_RATE",  cuda_cmd_set_autorate },
@@ -685,6 +702,7 @@ static const CudaCommand handlers[] = {
 { CUDA_SET_POWER_MESSAGES, "SET_POWER_MESSAGES",
   cuda_cmd_set_power_message },
 { CUDA_GET_TIME, "GET_TIME", cuda_cmd_get_time },
+{ CUDA_SET_TIME, "SET_TIME", cuda_cmd_set_time },
 };
 
 static void cuda_receive_packet(CUDAState *s,
@@ -692,7 +710,6 @@ static void cuda_receive_packet(CUDAState *s,
 {
 uint8_t obuf[16] = { CUDA_PACKET, 0, data[0] };
 int i, out_len = 0;
-uint32_t ti;
 
 for (i = 0; i < ARRAY_SIZE(handlers); i++) {
 const CudaCommand *desc = [i];
@@ -719,11 +736,6 @@ static void cuda_receive_packet(CUDAState *s,
 case CUDA_GET_6805_ADDR:
 cuda_send_packet_to_host(s, obuf, 3);
 return;
-case CUDA_SET_TIME:
-ti = (((uint32_t)data[1]) << 24) + (((uint32_t)data[2]) << 16) + 
(((uint32_t)data[3]) << 8) + data[4];
-s->tick_offset = ti - (qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) / 
get_ticks_per_sec());
-cuda_send_packet_to_host(s, obuf, 3);
-return;
 case CUDA_COMBINED_FORMAT_IIC:
 obuf[0] = ERROR_PACKET;
 obuf[1] = 0x5;
-- 
2.5.0

[Qemu-devel] [PULL 20/26] cuda: port SET_POWER_MESSAGES command to new framework

[David Gibson]

From: Hervé Poussineau 

Reviewed-by: David Gibson 
Signed-off-by: Hervé Poussineau 
Reviewed-by: Mark Cave-Ayland 
Signed-off-by: David Gibson 
---
 hw/misc/macio/cuda.c | 19 ---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/hw/misc/macio/cuda.c b/hw/misc/macio/cuda.c
index 4ddec34..16f9ad6 100644
--- a/hw/misc/macio/cuda.c
+++ b/hw/misc/macio/cuda.c
@@ -640,6 +640,20 @@ static bool cuda_cmd_set_file_server_flag(CUDAState *s,
 return true;
 }
 
+static bool cuda_cmd_set_power_message(CUDAState *s,
+   const uint8_t *in_data, int in_len,
+   uint8_t *out_data, int *out_len)
+{
+if (in_len != 1) {
+return false;
+}
+
+qemu_log_mask(LOG_UNIMP,
+  "CUDA: unimplemented command SET_POWER_MESSAGE %d\n",
+  in_data[0]);
+return true;
+}
+
 static const CudaCommand handlers[] = {
 { CUDA_AUTOPOLL, "AUTOPOLL", cuda_cmd_autopoll },
 { CUDA_SET_AUTO_RATE, "SET_AUTO_RATE",  cuda_cmd_set_autorate },
@@ -648,6 +662,8 @@ static const CudaCommand handlers[] = {
 { CUDA_RESET_SYSTEM, "RESET_SYSTEM", cuda_cmd_reset_system },
 { CUDA_FILE_SERVER_FLAG, "FILE_SERVER_FLAG",
   cuda_cmd_set_file_server_flag },
+{ CUDA_SET_POWER_MESSAGES, "SET_POWER_MESSAGES",
+  cuda_cmd_set_power_message },
 };
 
 static void cuda_receive_packet(CUDAState *s,
@@ -695,9 +711,6 @@ static void cuda_receive_packet(CUDAState *s,
 obuf[6] = ti;
 cuda_send_packet_to_host(s, obuf, 7);
 return;
-case CUDA_SET_POWER_MESSAGES:
-cuda_send_packet_to_host(s, obuf, 3);
-return;
 case CUDA_COMBINED_FORMAT_IIC:
 obuf[0] = ERROR_PACKET;
 obuf[1] = 0x5;
-- 
2.5.0

[Qemu-devel] [PULL 26/26] hw/ppc/spapr: Halt CPU when powering off via RTAS call

[David Gibson]

From: Thomas Huth 

The LoPAPR specification defines the following for the RTAS
power-off call: "On successful operation, does not return".
However, the implementation in QEMU currently returns and runs
the guest CPU again for some more cycles. This caused some
trouble with the new ppc implementation of the kvm-unit-tests
recently. So let's make sure that the QEMU implementation
follows the spec, thus stop the CPU to make sure that the
RTAS call does not return to the guest anymore.

Signed-off-by: Thomas Huth 
Tested-by: Andrew Jones 
Signed-off-by: David Gibson 
---
 hw/ppc/spapr_rtas.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
index 07ad672..b7c5ebd 100644
--- a/hw/ppc/spapr_rtas.c
+++ b/hw/ppc/spapr_rtas.c
@@ -113,6 +113,7 @@ static void rtas_power_off(PowerPCCPU *cpu, 
sPAPRMachineState *spapr,
 return;
 }
 qemu_system_shutdown_request();
+cpu_stop_current();
 rtas_st(rets, 0, RTAS_OUT_SUCCESS);
 }
 
-- 
2.5.0

[Qemu-devel] [PULL 06/26] pseries: Move hash page table allocation to reset time

[David Gibson]

At the moment the size of the hash page table (HPT) is fixed based on the
maximum memory allowed to the guest.  As such, we allocate the table during
machine construction, and just clear it at reset.

However, we're planning to implement a PAPR extension allowing the hash
page table to be resized at runtime.  This will mean that on reset we want
to revert it to the default size.  It also means that when migrating, we
need to make sure the destination allocates an HPT of size matching the
host, since the guest could have changed it before the migration.

This patch replaces the spapr_alloc_htab() and spapr_reset_htab() functions
with a new spapr_reallocate_hpt() function.  This is called at reset and
inbound migration only, not during machine init any more.

Signed-off-by: David Gibson 
Reviewed-by: Alexey Kardashevskiy 
---
 hw/ppc/spapr.c | 130 +
 1 file changed, 56 insertions(+), 74 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 2a81e8f..907439a 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1063,81 +1063,55 @@ static int spapr_hpt_shift_for_ramsize(uint64_t ramsize)
 return shift;
 }
 
-static void spapr_alloc_htab(sPAPRMachineState *spapr)
-{
-long shift;
-int index;
-
-/* allocate hash page table.  For now we always make this 16mb,
- * later we should probably make it scale to the size of guest
- * RAM */
-
-shift = kvmppc_reset_htab(spapr->htab_shift);
-if (shift < 0) {
-/*
- * For HV KVM, host kernel will return -ENOMEM when requested
- * HTAB size can't be allocated.
- */
-error_setg(_abort, "Failed to allocate HTAB of requested size, 
try with smaller maxmem");
-} else if (shift > 0) {
-/*
- * Kernel handles htab, we don't need to allocate one
- *
- * Older kernels can fall back to lower HTAB shift values,
- * but we don't allow booting of such guests.
- */
-if (shift != spapr->htab_shift) {
-error_setg(_abort, "Failed to allocate HTAB of requested 
size, try with smaller maxmem");
+static void spapr_reallocate_hpt(sPAPRMachineState *spapr, int shift,
+ Error **errp)
+{
+long rc;
+
+/* Clean up any HPT info from a previous boot */
+g_free(spapr->htab);
+spapr->htab = NULL;
+spapr->htab_shift = 0;
+close_htab_fd(spapr);
+
+rc = kvmppc_reset_htab(shift);
+if (rc < 0) {
+/* kernel-side HPT needed, but couldn't allocate one */
+error_setg_errno(errp, errno,
+ "Failed to allocate KVM HPT of order %d (try smaller 
maxmem?)",
+ shift);
+/* This is almost certainly fatal, but if the caller really
+ * wants to carry on with shift == 0, it's welcome to try */
+} else if (rc > 0) {
+/* kernel-side HPT allocated */
+if (rc != shift) {
+error_setg(errp,
+   "Requested order %d HPT, but kernel allocated order %ld 
(try smaller maxmem?)",
+   shift, rc);
 }
 
 spapr->htab_shift = shift;
 kvmppc_kern_htab = true;
 } else {
-/* Allocate htab */
-spapr->htab = qemu_memalign(HTAB_SIZE(spapr), HTAB_SIZE(spapr));
-
-/* And clear it */
-memset(spapr->htab, 0, HTAB_SIZE(spapr));
-
-for (index = 0; index < HTAB_SIZE(spapr) / HASH_PTE_SIZE_64; index++) {
-DIRTY_HPTE(HPTE(spapr->htab, index));
-}
-}
-}
-
-/*
- * Clear HTAB entries during reset.
- *
- * If host kernel has allocated HTAB, KVM_PPC_ALLOCATE_HTAB ioctl is
- * used to clear HTAB. Otherwise QEMU-allocated HTAB is cleared manually.
- */
-static void spapr_reset_htab(sPAPRMachineState *spapr)
-{
-long shift;
-int index;
+/* kernel-side HPT not needed, allocate in userspace instead */
+size_t size = 1ULL << shift;
+int i;
 
-shift = kvmppc_reset_htab(spapr->htab_shift);
-if (shift < 0) {
-error_setg(_abort, "Failed to reset HTAB");
-} else if (shift > 0) {
-if (shift != spapr->htab_shift) {
-error_setg(_abort, "Requested HTAB allocation failed during 
reset");
+spapr->htab = qemu_memalign(size, size);
+if (!spapr->htab) {
+error_setg_errno(errp, errno,
+ "Could not allocate HPT of order %d", shift);
+return;
 }
 
-close_htab_fd(spapr);
-} else {
-memset(spapr->htab, 0, HTAB_SIZE(spapr));
+memset(spapr->htab, 0, size);
+spapr->htab_shift = shift;
+kvmppc_kern_htab = false;
 
-for (index = 0; index < HTAB_SIZE(spapr) / HASH_PTE_SIZE_64; index++) {
-DIRTY_HPTE(HPTE(spapr->htab, index));
+for (i = 0; i < size / HASH_PTE_SIZE_64; i++) {
+DIRTY_HPTE(HPTE(spapr->htab, i));
  

[Qemu-devel] [PULL 18/26] cuda: port RESET_SYSTEM command to new framework

[David Gibson]

From: Hervé Poussineau 

Reviewed-by: David Gibson 
Signed-off-by: Hervé Poussineau 
Reviewed-by: Mark Cave-Ayland 
Signed-off-by: David Gibson 
---
 hw/misc/macio/cuda.c | 17 +
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/hw/misc/macio/cuda.c b/hw/misc/macio/cuda.c
index 9948e18..ca11fc8 100644
--- a/hw/misc/macio/cuda.c
+++ b/hw/misc/macio/cuda.c
@@ -614,11 +614,24 @@ static bool cuda_cmd_powerdown(CUDAState *s,
 return true;
 }
 
+static bool cuda_cmd_reset_system(CUDAState *s,
+  const uint8_t *in_data, int in_len,
+  uint8_t *out_data, int *out_len)
+{
+if (in_len != 0) {
+return false;
+}
+
+qemu_system_reset_request();
+return true;
+}
+
 static const CudaCommand handlers[] = {
 { CUDA_AUTOPOLL, "AUTOPOLL", cuda_cmd_autopoll },
 { CUDA_SET_AUTO_RATE, "SET_AUTO_RATE",  cuda_cmd_set_autorate },
 { CUDA_SET_DEVICE_LIST, "SET_DEVICE_LIST", cuda_cmd_set_device_list },
 { CUDA_POWERDOWN, "POWERDOWN", cuda_cmd_powerdown },
+{ CUDA_RESET_SYSTEM, "RESET_SYSTEM", cuda_cmd_reset_system },
 };
 
 static void cuda_receive_packet(CUDAState *s,
@@ -670,10 +683,6 @@ static void cuda_receive_packet(CUDAState *s,
 case CUDA_SET_POWER_MESSAGES:
 cuda_send_packet_to_host(s, obuf, 3);
 return;
-case CUDA_RESET_SYSTEM:
-cuda_send_packet_to_host(s, obuf, 3);
-qemu_system_reset_request();
-return;
 case CUDA_COMBINED_FORMAT_IIC:
 obuf[0] = ERROR_PACKET;
 obuf[1] = 0x5;
-- 
2.5.0

[Qemu-devel] [PULL 21/26] cuda: port GET_TIME command to new framework

[David Gibson]

From: Hervé Poussineau 

Reviewed-by: David Gibson 
Signed-off-by: Hervé Poussineau 
Reviewed-by: Mark Cave-Ayland 
Signed-off-by: David Gibson 
---
 hw/misc/macio/cuda.c | 29 +
 1 file changed, 21 insertions(+), 8 deletions(-)

diff --git a/hw/misc/macio/cuda.c b/hw/misc/macio/cuda.c
index 16f9ad6..8ee1414 100644
--- a/hw/misc/macio/cuda.c
+++ b/hw/misc/macio/cuda.c
@@ -654,6 +654,26 @@ static bool cuda_cmd_set_power_message(CUDAState *s,
 return true;
 }
 
+static bool cuda_cmd_get_time(CUDAState *s,
+  const uint8_t *in_data, int in_len,
+  uint8_t *out_data, int *out_len)
+{
+uint32_t ti;
+
+if (in_len != 0) {
+return false;
+}
+
+ti = s->tick_offset + (qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL)
+   / get_ticks_per_sec());
+out_data[0] = ti >> 24;
+out_data[1] = ti >> 16;
+out_data[2] = ti >> 8;
+out_data[3] = ti;
+*out_len = 4;
+return true;
+}
+
 static const CudaCommand handlers[] = {
 { CUDA_AUTOPOLL, "AUTOPOLL", cuda_cmd_autopoll },
 { CUDA_SET_AUTO_RATE, "SET_AUTO_RATE",  cuda_cmd_set_autorate },
@@ -664,6 +684,7 @@ static const CudaCommand handlers[] = {
   cuda_cmd_set_file_server_flag },
 { CUDA_SET_POWER_MESSAGES, "SET_POWER_MESSAGES",
   cuda_cmd_set_power_message },
+{ CUDA_GET_TIME, "GET_TIME", cuda_cmd_get_time },
 };
 
 static void cuda_receive_packet(CUDAState *s,
@@ -703,14 +724,6 @@ static void cuda_receive_packet(CUDAState *s,
 s->tick_offset = ti - (qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) / 
get_ticks_per_sec());
 cuda_send_packet_to_host(s, obuf, 3);
 return;
-case CUDA_GET_TIME:
-ti = s->tick_offset + (qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) / 
get_ticks_per_sec());
-obuf[3] = ti >> 24;
-obuf[4] = ti >> 16;
-obuf[5] = ti >> 8;
-obuf[6] = ti;
-cuda_send_packet_to_host(s, obuf, 7);
-return;
 case CUDA_COMBINED_FORMAT_IIC:
 obuf[0] = ERROR_PACKET;
 obuf[1] = 0x5;
-- 
2.5.0

[Qemu-devel] [PULL 24/26] cuda: remove CUDA_GET_SET_IIC/CUDA_COMBINED_FORMAT_IIC commands

[David Gibson]

From: Hervé Poussineau 

We currently don't emulate the I2C bus provided by CUDA.

Signed-off-by: Hervé Poussineau 
Reviewed-by: Mark Cave-Ayland 
Signed-off-by: David Gibson 
---
 hw/misc/macio/cuda.c | 23 ---
 1 file changed, 23 deletions(-)

diff --git a/hw/misc/macio/cuda.c b/hw/misc/macio/cuda.c
index d65a36a..481abdb 100644
--- a/hw/misc/macio/cuda.c
+++ b/hw/misc/macio/cuda.c
@@ -732,29 +732,6 @@ static void cuda_receive_packet(CUDAState *s,
 }
 }
 
-switch(data[0]) {
-case CUDA_COMBINED_FORMAT_IIC:
-obuf[0] = ERROR_PACKET;
-obuf[1] = 0x5;
-obuf[2] = CUDA_PACKET;
-obuf[3] = data[0];
-cuda_send_packet_to_host(s, obuf, 4);
-return;
-case CUDA_GET_SET_IIC:
-if (len == 4) {
-cuda_send_packet_to_host(s, obuf, 3);
-} else {
-obuf[0] = ERROR_PACKET;
-obuf[1] = 0x2;
-obuf[2] = CUDA_PACKET;
-obuf[3] = data[0];
-cuda_send_packet_to_host(s, obuf, 4);
-}
-return;
-default:
-break;
-}
-
 qemu_log_mask(LOG_GUEST_ERROR, "CUDA: unknown command 0x%02x\n", data[0]);
 obuf[0] = ERROR_PACKET;
 obuf[1] = 0x2; /* unknown command */
-- 
2.5.0

[Qemu-devel] [PULL 16/26] cuda: port SET_DEVICE_LIST command to new framework

[David Gibson]

From: Hervé Poussineau 

Also implement the command, by taking device list mask into account
when polling ADB devices.

Signed-off-by: Hervé Poussineau 
Reviewed-by: Mark Cave-Ayland 
Signed-off-by: David Gibson 
---
 hw/input/adb.c | 18 ++
 hw/misc/macio/cuda.c   | 18 --
 hw/ppc/mac.h   |  1 +
 include/hw/input/adb.h |  2 +-
 4 files changed, 28 insertions(+), 11 deletions(-)

diff --git a/hw/input/adb.c b/hw/input/adb.c
index c384856..f0ad0d4 100644
--- a/hw/input/adb.c
+++ b/hw/input/adb.c
@@ -89,7 +89,7 @@ int adb_request(ADBBusState *s, uint8_t *obuf, const uint8_t 
*buf, int len)
 }
 
 /* XXX: move that to cuda ? */
-int adb_poll(ADBBusState *s, uint8_t *obuf)
+int adb_poll(ADBBusState *s, uint8_t *obuf, uint16_t poll_mask)
 {
 ADBDevice *d;
 int olen, i;
@@ -100,13 +100,15 @@ int adb_poll(ADBBusState *s, uint8_t *obuf)
 if (s->poll_index >= s->nb_devices)
 s->poll_index = 0;
 d = s->devices[s->poll_index];
-buf[0] = ADB_READREG | (d->devaddr << 4);
-olen = adb_request(s, obuf + 1, buf, 1);
-/* if there is data, we poll again the same device */
-if (olen > 0) {
-obuf[0] = buf[0];
-olen++;
-break;
+if ((1 << d->devaddr) & poll_mask) {
+buf[0] = ADB_READREG | (d->devaddr << 4);
+olen = adb_request(s, obuf + 1, buf, 1);
+/* if there is data, we poll again the same device */
+if (olen > 0) {
+obuf[0] = buf[0];
+olen++;
+break;
+}
 }
 s->poll_index++;
 }
diff --git a/hw/misc/macio/cuda.c b/hw/misc/macio/cuda.c
index 71fd97c..a2e31d0 100644
--- a/hw/misc/macio/cuda.c
+++ b/hw/misc/macio/cuda.c
@@ -523,7 +523,7 @@ static void cuda_adb_poll(void *opaque)
 uint8_t obuf[ADB_MAX_OUT_LEN + 2];
 int olen;
 
-olen = adb_poll(>adb_bus, obuf + 2);
+olen = adb_poll(>adb_bus, obuf + 2, s->adb_poll_mask);
 if (olen > 0) {
 obuf[0] = ADB_PACKET;
 obuf[1] = 0x40; /* polled data */
@@ -590,9 +590,22 @@ static bool cuda_cmd_set_autorate(CUDAState *s,
 return true;
 }
 
+static bool cuda_cmd_set_device_list(CUDAState *s,
+ const uint8_t *in_data, int in_len,
+ uint8_t *out_data, int *out_len)
+{
+if (in_len != 2) {
+return false;
+}
+
+s->adb_poll_mask = (((uint16_t)in_data[0]) << 8) | in_data[1];
+return true;
+}
+
 static const CudaCommand handlers[] = {
 { CUDA_AUTOPOLL, "AUTOPOLL", cuda_cmd_autopoll },
 { CUDA_SET_AUTO_RATE, "SET_AUTO_RATE",  cuda_cmd_set_autorate },
+{ CUDA_SET_DEVICE_LIST, "SET_DEVICE_LIST", cuda_cmd_set_device_list },
 };
 
 static void cuda_receive_packet(CUDAState *s,
@@ -641,7 +654,6 @@ static void cuda_receive_packet(CUDAState *s,
 cuda_send_packet_to_host(s, obuf, 7);
 return;
 case CUDA_FILE_SERVER_FLAG:
-case CUDA_SET_DEVICE_LIST:
 case CUDA_SET_POWER_MESSAGES:
 cuda_send_packet_to_host(s, obuf, 3);
 return;
@@ -798,6 +810,7 @@ static const VMStateDescription vmstate_cuda = {
 VMSTATE_INT32(data_out_index, CUDAState),
 VMSTATE_UINT8(autopoll, CUDAState),
 VMSTATE_UINT8(autopoll_rate_ms, CUDAState),
+VMSTATE_UINT16(adb_poll_mask, CUDAState),
 VMSTATE_BUFFER(data_in, CUDAState),
 VMSTATE_BUFFER(data_out, CUDAState),
 VMSTATE_UINT32(tick_offset, CUDAState),
@@ -852,6 +865,7 @@ static void cuda_realizefn(DeviceState *dev, Error **errp)
 
 s->adb_poll_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, cuda_adb_poll, s);
 s->autopoll_rate_ms = 20;
+s->adb_poll_mask = 0x;
 }
 
 static void cuda_initfn(Object *obj)
diff --git a/hw/ppc/mac.h b/hw/ppc/mac.h
index 887c8c1..5764b86 100644
--- a/hw/ppc/mac.h
+++ b/hw/ppc/mac.h
@@ -111,6 +111,7 @@ typedef struct CUDAState {
 int data_out_index;
 
 qemu_irq irq;
+uint16_t adb_poll_mask;
 uint8_t autopoll_rate_ms;
 uint8_t autopoll;
 uint8_t data_in[128];
diff --git a/include/hw/input/adb.h b/include/hw/input/adb.h
index bdfccd4..db51d03 100644
--- a/include/hw/input/adb.h
+++ b/include/hw/input/adb.h
@@ -79,7 +79,7 @@ struct ADBBusState {
 
 int adb_request(ADBBusState *s, uint8_t *buf_out,
 const uint8_t *buf, int len);
-int adb_poll(ADBBusState *s, uint8_t *buf_out);
+int adb_poll(ADBBusState *s, uint8_t *buf_out, uint16_t poll_mask);
 
 #define TYPE_ADB_KEYBOARD "adb-keyboard"
 #define TYPE_ADB_MOUSE "adb-mouse"
-- 
2.5.0

[Qemu-devel] [PULL 10/26] hw/ppc/spapr: Implement h_set_dabr

[David Gibson]

From: Thomas Huth 

According to LoPAPR, h_set_dabr should simply set DABRX to 3
(if the register is available), and load the parameter into DABR.
If DABRX is not available, the hypervisor has to check the
"Breakpoint Translation" bit of the DABR register first.

Signed-off-by: Thomas Huth 
Signed-off-by: David Gibson 
---
 hw/ppc/spapr_hcall.c | 25 -
 1 file changed, 20 insertions(+), 5 deletions(-)

diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c
index 63f41ec..0004ca5 100644
--- a/hw/ppc/spapr_hcall.c
+++ b/hw/ppc/spapr_hcall.c
@@ -38,6 +38,12 @@ static void set_spr(CPUState *cs, int spr, target_ulong 
value,
 run_on_cpu(cs, do_spr_sync, );
 }
 
+static bool has_spr(PowerPCCPU *cpu, int spr)
+{
+/* We can test whether the SPR is defined by checking for a valid name */
+return cpu->env.spr_cb[spr].name != NULL;
+}
+
 static inline bool valid_pte_index(CPUPPCState *env, target_ulong pte_index)
 {
 /*
@@ -344,8 +350,19 @@ static target_ulong h_set_sprg0(PowerPCCPU *cpu, 
sPAPRMachineState *spapr,
 static target_ulong h_set_dabr(PowerPCCPU *cpu, sPAPRMachineState *spapr,
target_ulong opcode, target_ulong *args)
 {
-/* FIXME: actually implement this */
-return H_HARDWARE;
+if (!has_spr(cpu, SPR_DABR)) {
+return H_HARDWARE;  /* DABR register not available */
+}
+cpu_synchronize_state(CPU(cpu));
+
+if (has_spr(cpu, SPR_DABRX)) {
+cpu->env.spr[SPR_DABRX] = 0x3;  /* Use Problem and Privileged state */
+} else if (!(args[0] & 0x4)) {  /* Breakpoint Translation set? */
+return H_RESERVED_DABR;
+}
+
+cpu->env.spr[SPR_DABR] = args[0];
+return H_SUCCESS;
 }
 
 #define FLAGS_REGISTER_VPA 0x2000ULL
@@ -999,15 +1016,13 @@ static void hypercall_register_types(void)
 /* hcall-bulk */
 spapr_register_hypercall(H_BULK_REMOVE, h_bulk_remove);
 
-/* hcall-dabr */
-spapr_register_hypercall(H_SET_DABR, h_set_dabr);
-
 /* hcall-splpar */
 spapr_register_hypercall(H_REGISTER_VPA, h_register_vpa);
 spapr_register_hypercall(H_CEDE, h_cede);
 
 /* processor register resource access h-calls */
 spapr_register_hypercall(H_SET_SPRG0, h_set_sprg0);
+spapr_register_hypercall(H_SET_DABR, h_set_dabr);
 spapr_register_hypercall(H_SET_MODE, h_set_mode);
 
 /* "debugger" hcalls (also used by SLOF). Note: We do -not- differenciate
-- 
2.5.0

[Qemu-devel] [PULL 14/26] cuda: port AUTOPOLL command to new framework

[David Gibson]

From: Hervé Poussineau 

Reviewed-by: David Gibson 
Signed-off-by: Hervé Poussineau 
Reviewed-by: Mark Cave-Ayland 
Signed-off-by: David Gibson 
---
 hw/misc/macio/cuda.c | 40 +---
 1 file changed, 25 insertions(+), 15 deletions(-)

diff --git a/hw/misc/macio/cuda.c b/hw/misc/macio/cuda.c
index 81e34e7..ea4237a 100644
--- a/hw/misc/macio/cuda.c
+++ b/hw/misc/macio/cuda.c
@@ -544,14 +544,38 @@ typedef struct CudaCommand {
 uint8_t *out_args, int *out_len);
 } CudaCommand;
 
+static bool cuda_cmd_autopoll(CUDAState *s,
+  const uint8_t *in_data, int in_len,
+  uint8_t *out_data, int *out_len)
+{
+int autopoll;
+
+if (in_len != 1) {
+return false;
+}
+
+autopoll = (in_data[0] != 0);
+if (autopoll != s->autopoll) {
+s->autopoll = autopoll;
+if (autopoll) {
+timer_mod(s->adb_poll_timer,
+  qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) +
+  (get_ticks_per_sec() / CUDA_ADB_POLL_FREQ));
+} else {
+timer_del(s->adb_poll_timer);
+}
+}
+return true;
+}
+
 static const CudaCommand handlers[] = {
+{ CUDA_AUTOPOLL, "AUTOPOLL", cuda_cmd_autopoll },
 };
 
 static void cuda_receive_packet(CUDAState *s,
 const uint8_t *data, int len)
 {
 uint8_t obuf[16] = { CUDA_PACKET, 0, data[0] };
-int autopoll;
 int i, out_len = 0;
 uint32_t ti;
 
@@ -577,20 +601,6 @@ static void cuda_receive_packet(CUDAState *s,
 }
 
 switch(data[0]) {
-case CUDA_AUTOPOLL:
-autopoll = (data[1] != 0);
-if (autopoll != s->autopoll) {
-s->autopoll = autopoll;
-if (autopoll) {
-timer_mod(s->adb_poll_timer,
-   qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) +
-   (get_ticks_per_sec() / CUDA_ADB_POLL_FREQ));
-} else {
-timer_del(s->adb_poll_timer);
-}
-}
-cuda_send_packet_to_host(s, obuf, 3);
-return;
 case CUDA_GET_6805_ADDR:
 cuda_send_packet_to_host(s, obuf, 3);
 return;
-- 
2.5.0

[Qemu-devel] [PULL 11/26] hw/ppc/spapr: Implement the h_set_xdabr hypercall

[David Gibson]

From: Thomas Huth 

The H_SET_XDABR hypercall is similar to H_SET_DABR, but also sets
the extended DABR (DABRX) register.

Signed-off-by: Thomas Huth 
Signed-off-by: David Gibson 
---
 hw/ppc/spapr_hcall.c | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c
index 0004ca5..6e9b6be 100644
--- a/hw/ppc/spapr_hcall.c
+++ b/hw/ppc/spapr_hcall.c
@@ -365,6 +365,27 @@ static target_ulong h_set_dabr(PowerPCCPU *cpu, 
sPAPRMachineState *spapr,
 return H_SUCCESS;
 }
 
+static target_ulong h_set_xdabr(PowerPCCPU *cpu, sPAPRMachineState *spapr,
+target_ulong opcode, target_ulong *args)
+{
+target_ulong dabrx = args[1];
+
+if (!has_spr(cpu, SPR_DABR) || !has_spr(cpu, SPR_DABRX)) {
+return H_HARDWARE;
+}
+
+if ((dabrx & ~0xfULL) != 0 || (dabrx & H_DABRX_HYPERVISOR) != 0
+|| (dabrx & (H_DABRX_KERNEL | H_DABRX_USER)) == 0) {
+return H_PARAMETER;
+}
+
+cpu_synchronize_state(CPU(cpu));
+cpu->env.spr[SPR_DABRX] = dabrx;
+cpu->env.spr[SPR_DABR] = args[0];
+
+return H_SUCCESS;
+}
+
 #define FLAGS_REGISTER_VPA 0x2000ULL
 #define FLAGS_REGISTER_DTL 0x4000ULL
 #define FLAGS_REGISTER_SLBSHADOW   0x6000ULL
@@ -1023,6 +1044,7 @@ static void hypercall_register_types(void)
 /* processor register resource access h-calls */
 spapr_register_hypercall(H_SET_SPRG0, h_set_sprg0);
 spapr_register_hypercall(H_SET_DABR, h_set_dabr);
+spapr_register_hypercall(H_SET_XDABR, h_set_xdabr);
 spapr_register_hypercall(H_SET_MODE, h_set_mode);
 
 /* "debugger" hcalls (also used by SLOF). Note: We do -not- differenciate
-- 
2.5.0

[Qemu-devel] [PULL 09/26] hw/ppc/spapr: Add h_set_sprg0 hypercall

[David Gibson]

From: Thomas Huth 

This is a very simple hypercall that only sets up the SPRG0
register for the guest (since writing to SPRG0 was only permitted
to the hypervisor in older versions of the PowerISA).

Signed-off-by: Thomas Huth 
Signed-off-by: David Gibson 
---
 hw/ppc/spapr_hcall.c | 15 +--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c
index 12f8c33..63f41ec 100644
--- a/hw/ppc/spapr_hcall.c
+++ b/hw/ppc/spapr_hcall.c
@@ -332,6 +332,15 @@ static target_ulong h_read(PowerPCCPU *cpu, 
sPAPRMachineState *spapr,
 return H_SUCCESS;
 }
 
+static target_ulong h_set_sprg0(PowerPCCPU *cpu, sPAPRMachineState *spapr,
+target_ulong opcode, target_ulong *args)
+{
+cpu_synchronize_state(CPU(cpu));
+cpu->env.spr[SPR_SPRG0] = args[0];
+
+return H_SUCCESS;
+}
+
 static target_ulong h_set_dabr(PowerPCCPU *cpu, sPAPRMachineState *spapr,
target_ulong opcode, target_ulong *args)
 {
@@ -997,6 +1006,10 @@ static void hypercall_register_types(void)
 spapr_register_hypercall(H_REGISTER_VPA, h_register_vpa);
 spapr_register_hypercall(H_CEDE, h_cede);
 
+/* processor register resource access h-calls */
+spapr_register_hypercall(H_SET_SPRG0, h_set_sprg0);
+spapr_register_hypercall(H_SET_MODE, h_set_mode);
+
 /* "debugger" hcalls (also used by SLOF). Note: We do -not- differenciate
  * here between the "CI" and the "CACHE" variants, they will use whatever
  * mapping attributes qemu is using. When using KVM, the kernel will
@@ -1013,8 +1026,6 @@ static void hypercall_register_types(void)
 /* qemu/KVM-PPC specific hcalls */
 spapr_register_hypercall(KVMPPC_H_RTAS, h_rtas);
 
-spapr_register_hypercall(H_SET_MODE, h_set_mode);
-
 /* ibm,client-architecture-support support */
 spapr_register_hypercall(KVMPPC_H_CAS, h_client_architecture_support);
 }
-- 
2.5.0

[Qemu-devel] [PULL 25/26] pseries: Include missing pseries-2.5 compat properties in pseries-2.4

[David Gibson]

Commit 4b23699 "pseries: Add pseries-2.6 machine type" added a new
SPAPR_COMPAT_2_5 macro in the usual way.  However, it didn't add this
macro to the existing SPAPR_COMPAT_2_4 macro so that pseries-2.4
inherits newer compatibility properties which are needed for 2.5 and
earlier.

This corrects the oversight.

Reported-by: Laszlo Ersek 
Signed-off-by: David Gibson 
Reviewed-by: Laszlo Ersek 
Reviewed-by: Laurent Vivier 
---
 hw/ppc/spapr.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 276d6b4..e214a34 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2391,6 +2391,7 @@ DEFINE_SPAPR_MACHINE(2_5, "2.5", false);
  * pseries-2.4
  */
 #define SPAPR_COMPAT_2_4 \
+SPAPR_COMPAT_2_5 \
 HW_COMPAT_2_4
 
 static void spapr_machine_2_4_instance_options(MachineState *machine)
-- 
2.5.0

[Qemu-devel] [PULL 23/26] cuda: remove GET_6805_ADDR command

[David Gibson]

From: Hervé Poussineau 

It doesn't seem to be used, and operating systems should accept a 'unknown 
command' answer.

Signed-off-by: Hervé Poussineau 
Reviewed-by: Mark Cave-Ayland 
Signed-off-by: David Gibson 
---
 hw/misc/macio/cuda.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/hw/misc/macio/cuda.c b/hw/misc/macio/cuda.c
index 611b414..d65a36a 100644
--- a/hw/misc/macio/cuda.c
+++ b/hw/misc/macio/cuda.c
@@ -733,9 +733,6 @@ static void cuda_receive_packet(CUDAState *s,
 }
 
 switch(data[0]) {
-case CUDA_GET_6805_ADDR:
-cuda_send_packet_to_host(s, obuf, 3);
-return;
 case CUDA_COMBINED_FORMAT_IIC:
 obuf[0] = ERROR_PACKET;
 obuf[1] = 0x5;
-- 
2.5.0

[Qemu-devel] [PULL 01/26] hw: fix some debug message format strings

[David Gibson]

From: Alyssa Milburn 

Signed-off-by: Alyssa Milburn 
Signed-off-by: David Gibson 
---
 hw/nvram/mac_nvram.c   | 6 --
 hw/pci-host/uninorth.c | 4 ++--
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/hw/nvram/mac_nvram.c b/hw/nvram/mac_nvram.c
index 564ef93..1671f46 100644
--- a/hw/nvram/mac_nvram.c
+++ b/hw/nvram/mac_nvram.c
@@ -49,7 +49,8 @@ static void macio_nvram_writeb(void *opaque, hwaddr addr,
 
 addr = (addr >> s->it_shift) & (s->size - 1);
 s->data[addr] = value;
-NVR_DPRINTF("writeb addr %04" PHYS_PRIx " val %" PRIx64 "\n", addr, value);
+NVR_DPRINTF("writeb addr %04" HWADDR_PRIx " val %" PRIx64 "\n",
+addr, value);
 }
 
 static uint64_t macio_nvram_readb(void *opaque, hwaddr addr,
@@ -60,7 +61,8 @@ static uint64_t macio_nvram_readb(void *opaque, hwaddr addr,
 
 addr = (addr >> s->it_shift) & (s->size - 1);
 value = s->data[addr];
-NVR_DPRINTF("readb addr %04x val %x\n", (int)addr, value);
+NVR_DPRINTF("readb addr %04" HWADDR_PRIx " val %" PRIx32 "\n",
+addr, value);
 
 return value;
 }
diff --git a/hw/pci-host/uninorth.c b/hw/pci-host/uninorth.c
index 40a2e3e..15b1054 100644
--- a/hw/pci-host/uninorth.c
+++ b/hw/pci-host/uninorth.c
@@ -120,7 +120,7 @@ static void unin_data_write(void *opaque, hwaddr addr,
 {
 UNINState *s = opaque;
 PCIHostState *phb = PCI_HOST_BRIDGE(s);
-UNIN_DPRINTF("write addr %" TARGET_FMT_plx " len %d val %"PRIx64"\n",
+UNIN_DPRINTF("write addr " TARGET_FMT_plx " len %d val %"PRIx64"\n",
  addr, len, val);
 pci_data_write(phb->bus,
unin_get_config_reg(phb->config_reg, addr),
@@ -137,7 +137,7 @@ static uint64_t unin_data_read(void *opaque, hwaddr addr,
 val = pci_data_read(phb->bus,
 unin_get_config_reg(phb->config_reg, addr),
 len);
-UNIN_DPRINTF("read addr %" TARGET_FMT_plx " len %d val %x\n",
+UNIN_DPRINTF("read addr " TARGET_FMT_plx " len %d val %x\n",
  addr, len, val);
 return val;
 }
-- 
2.5.0

[Qemu-devel] [PULL 12/26] cuda: add a framework to handle commands

[David Gibson]

From: Hervé Poussineau 

Next commits will port existing CUDA commands to this framework.

Signed-off-by: Hervé Poussineau 
Reviewed-by: Mark Cave-Ayland 
Signed-off-by: David Gibson 
---
 hw/misc/macio/cuda.c | 34 ++
 1 file changed, 34 insertions(+)

diff --git a/hw/misc/macio/cuda.c b/hw/misc/macio/cuda.c
index 316c1ac..8659dc3 100644
--- a/hw/misc/macio/cuda.c
+++ b/hw/misc/macio/cuda.c
@@ -535,13 +535,47 @@ static void cuda_adb_poll(void *opaque)
(get_ticks_per_sec() / CUDA_ADB_POLL_FREQ));
 }
 
+/* description of commands */
+typedef struct CudaCommand {
+uint8_t command;
+const char *name;
+bool (*handler)(CUDAState *s,
+const uint8_t *in_args, int in_len,
+uint8_t *out_args, int *out_len);
+} CudaCommand;
+
+static const CudaCommand handlers[] = {
+};
+
 static void cuda_receive_packet(CUDAState *s,
 const uint8_t *data, int len)
 {
 uint8_t obuf[16] = { CUDA_PACKET, 0, data[0] };
 int autopoll;
+int i, out_len = 0;
 uint32_t ti;
 
+for (i = 0; i < ARRAY_SIZE(handlers); i++) {
+const CudaCommand *desc = [i];
+if (desc->command == data[0]) {
+CUDA_DPRINTF("handling command %s\n", desc->name);
+out_len = 0;
+if (desc->handler(s, data + 1, len - 1, obuf + 3, _len)) {
+cuda_send_packet_to_host(s, obuf, 3 + out_len);
+} else {
+qemu_log_mask(LOG_GUEST_ERROR,
+  "CUDA: %s: wrong parameters %d\n",
+  desc->name, len);
+obuf[0] = ERROR_PACKET;
+obuf[1] = 0x5; /* bad parameters */
+obuf[2] = CUDA_PACKET;
+obuf[3] = data[0];
+cuda_send_packet_to_host(s, obuf, 4);
+}
+return;
+}
+}
+
 switch(data[0]) {
 case CUDA_AUTOPOLL:
 autopoll = (data[1] != 0);
-- 
2.5.0

[Qemu-devel] [PULL 13/26] cuda: move unknown commands reject out of switch

[David Gibson]

From: Hervé Poussineau 

Signed-off-by: Hervé Poussineau 
Reviewed-by: Mark Cave-Ayland 
Signed-off-by: David Gibson 
---
 hw/misc/macio/cuda.c | 30 --
 1 file changed, 16 insertions(+), 14 deletions(-)

diff --git a/hw/misc/macio/cuda.c b/hw/misc/macio/cuda.c
index 8659dc3..81e34e7 100644
--- a/hw/misc/macio/cuda.c
+++ b/hw/misc/macio/cuda.c
@@ -590,15 +590,15 @@ static void cuda_receive_packet(CUDAState *s,
 }
 }
 cuda_send_packet_to_host(s, obuf, 3);
-break;
+return;
 case CUDA_GET_6805_ADDR:
 cuda_send_packet_to_host(s, obuf, 3);
-break;
+return;
 case CUDA_SET_TIME:
 ti = (((uint32_t)data[1]) << 24) + (((uint32_t)data[2]) << 16) + 
(((uint32_t)data[3]) << 8) + data[4];
 s->tick_offset = ti - (qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) / 
get_ticks_per_sec());
 cuda_send_packet_to_host(s, obuf, 3);
-break;
+return;
 case CUDA_GET_TIME:
 ti = s->tick_offset + (qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) / 
get_ticks_per_sec());
 obuf[3] = ti >> 24;
@@ -606,28 +606,28 @@ static void cuda_receive_packet(CUDAState *s,
 obuf[5] = ti >> 8;
 obuf[6] = ti;
 cuda_send_packet_to_host(s, obuf, 7);
-break;
+return;
 case CUDA_FILE_SERVER_FLAG:
 case CUDA_SET_DEVICE_LIST:
 case CUDA_SET_AUTO_RATE:
 case CUDA_SET_POWER_MESSAGES:
 cuda_send_packet_to_host(s, obuf, 3);
-break;
+return;
 case CUDA_POWERDOWN:
 cuda_send_packet_to_host(s, obuf, 3);
 qemu_system_shutdown_request();
-break;
+return;
 case CUDA_RESET_SYSTEM:
 cuda_send_packet_to_host(s, obuf, 3);
 qemu_system_reset_request();
-break;
+return;
 case CUDA_COMBINED_FORMAT_IIC:
 obuf[0] = ERROR_PACKET;
 obuf[1] = 0x5;
 obuf[2] = CUDA_PACKET;
 obuf[3] = data[0];
 cuda_send_packet_to_host(s, obuf, 4);
-break;
+return;
 case CUDA_GET_SET_IIC:
 if (len == 4) {
 cuda_send_packet_to_host(s, obuf, 3);
@@ -638,15 +638,17 @@ static void cuda_receive_packet(CUDAState *s,
 obuf[3] = data[0];
 cuda_send_packet_to_host(s, obuf, 4);
 }
-break;
+return;
 default:
-obuf[0] = ERROR_PACKET;
-obuf[1] = 0x2;
-obuf[2] = CUDA_PACKET;
-obuf[3] = data[0];
-cuda_send_packet_to_host(s, obuf, 4);
 break;
 }
+
+qemu_log_mask(LOG_GUEST_ERROR, "CUDA: unknown command 0x%02x\n", data[0]);
+obuf[0] = ERROR_PACKET;
+obuf[1] = 0x2; /* unknown command */
+obuf[2] = CUDA_PACKET;
+obuf[3] = data[0];
+cuda_send_packet_to_host(s, obuf, 4);
 }
 
 static void cuda_receive_packet_from_host(CUDAState *s,
-- 
2.5.0

[Qemu-devel] [PULL 07/26] target-ppc: Remove hack for ppc_hash64_load_hpte*() with HV KVM

[David Gibson]

With HV KVM, the guest's hash page table (HPT) is managed by the kernel and
not directly accessible to QEMU.  This means that spapr->htab is NULL
and normally env->external_htab would also be NULL for each cpu.

However, that would cause ppc_hash64_load_hpte*() to do the wrong thing in
the few cases where QEMU does need to load entries from the in-kernel HPT.
Specifically, seeing external_htab is NULL, they would look for an HPT
within the guest's address space instead.

To stop that we have an ugly hack in the pseries machine type code to
set external htab to (void *)1 instead.

This patch removes that hack by having ppc_hash64_load_hpte*() explicitly
check kvmppc_kern_htab instead, which makes more sense.

Signed-off-by: David Gibson 
Reviewed-by: Alexey Kardashevskiy 
---
 hw/ppc/spapr.c  | 7 ---
 target-ppc/mmu-hash64.h | 4 ++--
 2 files changed, 2 insertions(+), 9 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 907439a..c85a125 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1197,13 +1197,6 @@ static void spapr_cpu_reset(void *opaque)
 env->spr[SPR_HIOR] = 0;
 
 env->external_htab = (uint8_t *)spapr->htab;
-if (kvm_enabled() && !env->external_htab) {
-/*
- * HV KVM, set external_htab to 1 so our ppc_hash64_load_hpte*
- * functions do the right thing.
- */
-env->external_htab = (void *)1;
-}
 env->htab_base = -1;
 /*
  * htab_mask is the mask used to normalize hash value to PTEG index.
diff --git a/target-ppc/mmu-hash64.h b/target-ppc/mmu-hash64.h
index ab0f86b..e7d9925 100644
--- a/target-ppc/mmu-hash64.h
+++ b/target-ppc/mmu-hash64.h
@@ -102,7 +102,7 @@ static inline target_ulong ppc_hash64_load_hpte0(PowerPCCPU 
*cpu,
 uint64_t addr;
 
 addr = token + (index * HASH_PTE_SIZE_64);
-if (env->external_htab) {
+if (kvmppc_kern_htab || env->external_htab) {
 return  ldq_p((const void *)(uintptr_t)addr);
 } else {
 return ldq_phys(CPU(cpu)->as, addr);
@@ -116,7 +116,7 @@ static inline target_ulong ppc_hash64_load_hpte1(PowerPCCPU 
*cpu,
 uint64_t addr;
 
 addr = token + (index * HASH_PTE_SIZE_64) + HASH_PTE_SIZE_64/2;
-if (env->external_htab) {
+if (kvmppc_kern_htab || env->external_htab) {
 return  ldq_p((const void *)(uintptr_t)addr);
 } else {
 return ldq_phys(CPU(cpu)->as, addr);
-- 
2.5.0

[Qemu-devel] [PULL 04/26] pseries: Simplify handling of the hash page table fd

[David Gibson]

When migrating the 'pseries' machine type with KVM, we use a special fd
to access the hash page table stored within KVM.  Usually, this fd is
opened at the beginning of migration, and kept open until the migration
is complete.

However, if there is a guest reset during the migration, the fd can become
stale and we need to re-open it.  At the moment we use an 'htab_fd_stale'
flag in sPAPRMachineState to signal this, which is checked in the migration
iterators.

But that's rather ugly.  It's simpler to just close and invalidate the
fd on reset, and lazily re-open it in migration if necessary.  This patch
implements that change.

This requires a small addition to the machine state's instance_init,
so that htab_fd is initialized to -1 (telling the migration code it
needs to open it) instead of 0, which could be a valid fd.

Signed-off-by: David Gibson 
Reviewed-by: Alexey Kardashevskiy 
---
 hw/ppc/spapr.c | 86 --
 include/hw/ppc/spapr.h |  1 -
 2 files changed, 41 insertions(+), 46 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 5bd8fd3..c315715 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1024,6 +1024,32 @@ static void emulate_spapr_hypercall(PowerPCCPU *cpu)
 #define CLEAN_HPTE(_hpte)  ((*(uint64_t *)(_hpte)) &= 
tswap64(~HPTE64_V_HPTE_DIRTY))
 #define DIRTY_HPTE(_hpte)  ((*(uint64_t *)(_hpte)) |= 
tswap64(HPTE64_V_HPTE_DIRTY))
 
+/*
+ * Get the fd to access the kernel htab, re-opening it if necessary
+ */
+static int get_htab_fd(sPAPRMachineState *spapr)
+{
+if (spapr->htab_fd >= 0) {
+return spapr->htab_fd;
+}
+
+spapr->htab_fd = kvmppc_get_htab_fd(false);
+if (spapr->htab_fd < 0) {
+error_report("Unable to open fd for reading hash table from KVM: %s",
+ strerror(errno));
+}
+
+return spapr->htab_fd;
+}
+
+static void close_htab_fd(sPAPRMachineState *spapr)
+{
+if (spapr->htab_fd >= 0) {
+close(spapr->htab_fd);
+}
+spapr->htab_fd = -1;
+}
+
 static void spapr_alloc_htab(sPAPRMachineState *spapr)
 {
 long shift;
@@ -1085,10 +,7 @@ static void spapr_reset_htab(sPAPRMachineState *spapr)
 error_setg(_abort, "Requested HTAB allocation failed during 
reset");
 }
 
-/* Tell readers to update their file descriptor */
-if (spapr->htab_fd >= 0) {
-spapr->htab_fd_stale = true;
-}
+close_htab_fd(spapr);
 } else {
 memset(spapr->htab, 0, HTAB_SIZE(spapr));
 
@@ -1121,28 +1144,6 @@ static int find_unknown_sysbus_device(SysBusDevice 
*sbdev, void *opaque)
 return 0;
 }
 
-/*
- * A guest reset will cause spapr->htab_fd to become stale if being used.
- * Reopen the file descriptor to make sure the whole HTAB is properly read.
- */
-static int spapr_check_htab_fd(sPAPRMachineState *spapr)
-{
-int rc = 0;
-
-if (spapr->htab_fd_stale) {
-close(spapr->htab_fd);
-spapr->htab_fd = kvmppc_get_htab_fd(false);
-if (spapr->htab_fd < 0) {
-error_report("Unable to open fd for reading hash table from KVM: "
- "%s", strerror(errno));
-rc = -1;
-}
-spapr->htab_fd_stale = false;
-}
-
-return rc;
-}
-
 static void ppc_spapr_reset(void)
 {
 sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
@@ -1313,14 +1314,6 @@ static int htab_save_setup(QEMUFile *f, void *opaque)
 spapr->htab_first_pass = true;
 } else {
 assert(kvm_enabled());
-
-spapr->htab_fd = kvmppc_get_htab_fd(false);
-spapr->htab_fd_stale = false;
-if (spapr->htab_fd < 0) {
-fprintf(stderr, "Unable to open fd for reading hash table from 
KVM: %s\n",
-strerror(errno));
-return -1;
-}
 }
 
 
@@ -1460,6 +1453,7 @@ static int htab_save_later_pass(QEMUFile *f, 
sPAPRMachineState *spapr,
 static int htab_save_iterate(QEMUFile *f, void *opaque)
 {
 sPAPRMachineState *spapr = opaque;
+int fd;
 int rc = 0;
 
 /* Iteration header */
@@ -1468,13 +1462,12 @@ static int htab_save_iterate(QEMUFile *f, void *opaque)
 if (!spapr->htab) {
 assert(kvm_enabled());
 
-rc = spapr_check_htab_fd(spapr);
-if (rc < 0) {
-return rc;
+fd = get_htab_fd(spapr);
+if (fd < 0) {
+return fd;
 }
 
-rc = kvmppc_save_htab(f, spapr->htab_fd,
-  MAX_KVM_BUF_SIZE, MAX_ITERATION_NS);
+rc = kvmppc_save_htab(f, fd, MAX_KVM_BUF_SIZE, MAX_ITERATION_NS);
 if (rc < 0) {
 return rc;
 }
@@ -1495,6 +1488,7 @@ static int htab_save_iterate(QEMUFile *f, void *opaque)
 static int htab_save_complete(QEMUFile *f, void *opaque)
 {
 sPAPRMachineState *spapr = opaque;
+int fd;
 
 /* Iteration header */
 qemu_put_be32(f, 0);
@@ -1504,17 +1498,16 @@ static 

[Qemu-devel] [PULL 05/26] pseries: Add helper to calculate recommended hash page table size

[David Gibson]

At present we calculate the recommended hash page table (HPT) size for a
pseries guest just once in ppc_spapr_init() before allocating the HPT.
In future patches we're going to want this calculation in other places, so
this splits it out into a helper function.  While we're at it, change the
calculation to use ctz() instead of an explicit loop.

Signed-off-by: David Gibson 
Reviewed-by: Alexey Kardashevskiy 
---
 hw/ppc/spapr.c | 24 ++--
 1 file changed, 14 insertions(+), 10 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index c315715..2a81e8f 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1050,6 +1050,19 @@ static void close_htab_fd(sPAPRMachineState *spapr)
 spapr->htab_fd = -1;
 }
 
+static int spapr_hpt_shift_for_ramsize(uint64_t ramsize)
+{
+int shift;
+
+/* We aim for a hash table of size 1/128 the size of RAM (rounded
+ * up).  The PAPR recommendation is actually 1/64 of RAM size, but
+ * that's much more than is needed for Linux guests */
+shift = ctz64(pow2ceil(ramsize)) - 7;
+shift = MAX(shift, 18); /* Minimum architected size */
+shift = MIN(shift, 46); /* Maximum architected size */
+return shift;
+}
+
 static void spapr_alloc_htab(sPAPRMachineState *spapr)
 {
 long shift;
@@ -1790,16 +1803,7 @@ static void ppc_spapr_init(MachineState *machine)
 /* Setup a load limit for the ramdisk leaving room for SLOF and FDT */
 load_limit = MIN(spapr->rma_size, RTAS_MAX_ADDR) - FW_OVERHEAD;
 
-/* We aim for a hash table of size 1/128 the size of RAM.  The
- * normal rule of thumb is 1/64 the size of RAM, but that's much
- * more than needed for the Linux guests we support. */
-spapr->htab_shift = 18; /* Minimum architected size */
-while (spapr->htab_shift <= 46) {
-if ((1ULL << (spapr->htab_shift + 7)) >= machine->maxram_size) {
-break;
-}
-spapr->htab_shift++;
-}
+spapr->htab_shift = spapr_hpt_shift_for_ramsize(machine->maxram_size);
 spapr_alloc_htab(spapr);
 
 /* Set up Interrupt Controller before we create the VCPUs */
-- 
2.5.0

[Qemu-devel] [PULL 08/26] migration: ensure htab_save_first completes after timeout

[David Gibson]

htab_save_first_pass could return without finishing its work due to
timeout. The patch checks if another invocation of it is necessary and
will call it in htab_save_complete if necessary.

Signed-off-by: Jianjun Duan 
Reviewed-by: Michael Roth 
[removed overlong line]
Signed-off-by: David Gibson 
---
 hw/ppc/spapr.c | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index c85a125..276d6b4 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1312,6 +1312,7 @@ static int htab_save_setup(QEMUFile *f, void *opaque)
 static void htab_save_first_pass(QEMUFile *f, sPAPRMachineState *spapr,
  int64_t max_ns)
 {
+bool has_timeout = max_ns != -1;
 int htabslots = HTAB_SIZE(spapr) / HASH_PTE_SIZE_64;
 int index = spapr->htab_save_index;
 int64_t starttime = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
@@ -1345,7 +1346,8 @@ static void htab_save_first_pass(QEMUFile *f, 
sPAPRMachineState *spapr,
 qemu_put_buffer(f, HPTE(spapr->htab, chunkstart),
 HASH_PTE_SIZE_64 * n_valid);
 
-if ((qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - starttime) > max_ns) 
{
+if (has_timeout &&
+(qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - starttime) > max_ns) 
{
 break;
 }
 }
@@ -1498,6 +1500,9 @@ static int htab_save_complete(QEMUFile *f, void *opaque)
 }
 close_htab_fd(spapr);
 } else {
+if (spapr->htab_first_pass) {
+htab_save_first_pass(f, spapr, -1);
+}
 htab_save_later_pass(f, spapr, -1);
 }
 
-- 
2.5.0

[Qemu-devel] [PULL 03/26] target-ppc: Include missing MMU models for SDR1 in info registers

[David Gibson]

The HMP command "info registers" produces somewhat different information on
different ppc cpu variants.  For those with a hash MMU it's supposed to
include the SDR1, DAR and DSISR registers related to the MMU.  However,
the switch is missing a couple of MMU model variants, meaning we will
miss out this information on certain CPUs which should have it.

This patch corrects the oversight.  (Really these MMU model IDs need a big
cleanup, but we might as well fix the bug in the interim).

Signed-off-by: David Gibson 
Reviewed-by: Alexey Kardashevskiy 
---
 target-ppc/translate.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/target-ppc/translate.c b/target-ppc/translate.c
index ffef754..ecc85f0 100644
--- a/target-ppc/translate.c
+++ b/target-ppc/translate.c
@@ -11352,7 +11352,9 @@ void ppc_cpu_dump_state(CPUState *cs, FILE *f, 
fprintf_function cpu_fprintf,
 case POWERPC_MMU_64B:
 case POWERPC_MMU_2_03:
 case POWERPC_MMU_2_06:
+case POWERPC_MMU_2_06a:
 case POWERPC_MMU_2_07:
+case POWERPC_MMU_2_07a:
 #endif
 cpu_fprintf(f, " SDR1 " TARGET_FMT_lx "   DAR " TARGET_FMT_lx
"  DSISR " TARGET_FMT_lx "\n", env->spr[SPR_SDR1],
-- 
2.5.0

[Qemu-devel] [PULL 17/26] cuda: port POWERDOWN command to new framework

[David Gibson]

From: Hervé Poussineau 

Reviewed-by: David Gibson 
Signed-off-by: Hervé Poussineau 
Reviewed-by: Mark Cave-Ayland 
Signed-off-by: David Gibson 
---
 hw/misc/macio/cuda.c | 17 +
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/hw/misc/macio/cuda.c b/hw/misc/macio/cuda.c
index a2e31d0..9948e18 100644
--- a/hw/misc/macio/cuda.c
+++ b/hw/misc/macio/cuda.c
@@ -602,10 +602,23 @@ static bool cuda_cmd_set_device_list(CUDAState *s,
 return true;
 }
 
+static bool cuda_cmd_powerdown(CUDAState *s,
+   const uint8_t *in_data, int in_len,
+   uint8_t *out_data, int *out_len)
+{
+if (in_len != 0) {
+return false;
+}
+
+qemu_system_shutdown_request();
+return true;
+}
+
 static const CudaCommand handlers[] = {
 { CUDA_AUTOPOLL, "AUTOPOLL", cuda_cmd_autopoll },
 { CUDA_SET_AUTO_RATE, "SET_AUTO_RATE",  cuda_cmd_set_autorate },
 { CUDA_SET_DEVICE_LIST, "SET_DEVICE_LIST", cuda_cmd_set_device_list },
+{ CUDA_POWERDOWN, "POWERDOWN", cuda_cmd_powerdown },
 };
 
 static void cuda_receive_packet(CUDAState *s,
@@ -657,10 +670,6 @@ static void cuda_receive_packet(CUDAState *s,
 case CUDA_SET_POWER_MESSAGES:
 cuda_send_packet_to_host(s, obuf, 3);
 return;
-case CUDA_POWERDOWN:
-cuda_send_packet_to_host(s, obuf, 3);
-qemu_system_shutdown_request();
-return;
 case CUDA_RESET_SYSTEM:
 cuda_send_packet_to_host(s, obuf, 3);
 qemu_system_reset_request();
-- 
2.5.0

[Qemu-devel] [PULL 00/26] ppc-for-2.6 queue 20160218

[David Gibson]

The following changes since commit 3fc63c3f339a61f4e4526f88150927424744f687:

  Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging 
(2016-02-16 17:31:56 +)

are available in the git repository at:

  git://github.com/dgibson/qemu.git tags/ppc-for-2.6-20160218

for you to fetch changes up to 8a9c1b77e9df5b4a9fcc1ffe08b4fcff7b0c791f:

  hw/ppc/spapr: Halt CPU when powering off via RTAS call (2016-02-18 11:08:43 
+1100)


ppc patch queue for 2016-02-18

Currently accumulated patches for target-ppc, pseries machine type and
related devices.
  * Some cleanups to management of SDR1 and the hashed page table
  * Implementations of a number of simple PAPR hypercalls
  * Significant improvements to the Macintosh CUDA device
  * Several bugfixes


Alyssa Milburn (1):
  hw: fix some debug message format strings

David Gibson (8):
  target-ppc: Remove unused kvmppc_update_sdr1() stub
  target-ppc: Include missing MMU models for SDR1 in info registers
  pseries: Simplify handling of the hash page table fd
  pseries: Add helper to calculate recommended hash page table size
  pseries: Move hash page table allocation to reset time
  target-ppc: Remove hack for ppc_hash64_load_hpte*() with HV KVM
  migration: ensure htab_save_first completes after timeout
  pseries: Include missing pseries-2.5 compat properties in pseries-2.4

Hervé Poussineau (13):
  cuda: add a framework to handle commands
  cuda: move unknown commands reject out of switch
  cuda: port AUTOPOLL command to new framework
  cuda: port SET_AUTO_RATE command to new framework
  cuda: port SET_DEVICE_LIST command to new framework
  cuda: port POWERDOWN command to new framework
  cuda: port RESET_SYSTEM command to new framework
  cuda: port FILE_SERVER_FLAG command to new framework
  cuda: port SET_POWER_MESSAGES command to new framework
  cuda: port GET_TIME command to new framework
  cuda: port SET_TIME command to new framework
  cuda: remove GET_6805_ADDR command
  cuda: remove CUDA_GET_SET_IIC/CUDA_COMBINED_FORMAT_IIC commands

Thomas Huth (4):
  hw/ppc/spapr: Add h_set_sprg0 hypercall
  hw/ppc/spapr: Implement h_set_dabr
  hw/ppc/spapr: Implement the h_set_xdabr hypercall
  hw/ppc/spapr: Halt CPU when powering off via RTAS call

 hw/input/adb.c  |  18 ++--
 hw/misc/macio/cuda.c| 279 +++-
 hw/nvram/mac_nvram.c|   6 +-
 hw/pci-host/uninorth.c  |   4 +-
 hw/ppc/mac.h|   2 +
 hw/ppc/spapr.c  | 239 +++--
 hw/ppc/spapr_hcall.c|  62 +--
 hw/ppc/spapr_rtas.c |   1 +
 include/hw/input/adb.h  |   2 +-
 include/hw/ppc/spapr.h  |   1 -
 target-ppc/kvm_ppc.h|   5 -
 target-ppc/mmu-hash64.h |   4 +-
 target-ppc/translate.c  |   2 +
 13 files changed, 393 insertions(+), 232 deletions(-)

[Qemu-devel] [PULL 02/26] target-ppc: Remove unused kvmppc_update_sdr1() stub

[David Gibson]

This KVM stub implementation isn't used anywhere.

Signed-off-by: David Gibson 
Reviewed-by: Alexey Kardashevskiy 
---
 target-ppc/kvm_ppc.h | 5 -
 1 file changed, 5 deletions(-)

diff --git a/target-ppc/kvm_ppc.h b/target-ppc/kvm_ppc.h
index 62406ce..aaa828c 100644
--- a/target-ppc/kvm_ppc.h
+++ b/target-ppc/kvm_ppc.h
@@ -184,11 +184,6 @@ static inline uint64_t kvmppc_rma_size(uint64_t 
current_size,
 return ram_size;
 }
 
-static inline int kvmppc_update_sdr1(CPUPPCState *env)
-{
-return 0;
-}
-
 #endif /* !CONFIG_USER_ONLY */
 
 static inline bool kvmppc_has_cap_epr(void)
-- 
2.5.0

Re: [Qemu-devel] [RFC] QMP: add query-hotpluggable-cpus

[David Gibson]

On Tue, Feb 16, 2016 at 01:35:42PM +0100, Markus Armbruster wrote:
> Igor Mammedov  writes:
> 
> > On Mon, 15 Feb 2016 20:43:41 +0100
> > Markus Armbruster  wrote:
> >
> >> Igor Mammedov  writes:
> >> 
> >> > it will allow mgmt to query present and possible to hotplug CPUs
> >> > it is required from a target platform that wish to support
> >> > command to set board specific MachineClass.possible_cpus() hook,
> >> > which will return a list of possible CPUs with options
> >> > that would be needed for hotplugging possible CPUs.
> >> >
> >> > For RFC there are:
> >> >'arch_id': 'int' - mandatory unique CPU number,
> >> >   for x86 it's APIC ID for ARM it's MPIDR
> >> >'type': 'str' - CPU object type for usage with device_add
> >> >
> >> > and a set of optional fields that would allows mgmt tools
> >> > to know at what granularity and where a new CPU could be
> >> > hotplugged;
> >> > [node],[socket],[core],[thread]
> >> > Hopefully that should cover needs for CPU hotplug porposes for
> >> > magor targets and we can extend structure in future adding
> >> > more fields if it will be needed.
> >> >
> >> > also for present CPUs there is a 'cpu_link' field which
> >> > would allow mgmt inspect whatever object/abstraction
> >> > the target platform considers as CPU object.
> >> >
> >> > For RFC purposes implements only for x86 target so far.  
> >> 
> >> Adding ad hoc queries as we go won't scale.  Could this be solved by a
> >> generic introspection interface?
> > Do you mean generic QOM introspection?
> 
> Possibly, but I don't want to prematurely limit the conversation to QOM
> introspection.
> 
> > Using QOM we could have '/cpus' container and create QOM links
> > for exiting (populated links) and possible (empty links) CPUs.
> > However in that case link's name will need have a special format
> > that will convey an information necessary for mgmt to hotplug
> > a CPU object, at least:
> >   - where: [node],[socket],[core],[thread] options
> >   - optionally what CPU object to use with device_add command
> 
> Encoding information in names feels wrong.

Yeah :(.

> > Another approach to do QOM introspection would be to model hierarchy 
> > of objects like node/socket/core..., That's what Andreas
> > worked on. Only it still suffers the same issue as above
> > wrt introspection and hotplug, One can pre-create empty
> > [nodes][sockets[cores]] containers at startup but then
> > leaf nodes that could be hotplugged would be a links anyway
> > and then again we need to give them special formatted names
> > (not well documented at that mgmt could make sense of).
> > That hierarchy would need to become stable ABI once
> > mgmt will start using it and QOM tree is quite unstable
> > now for that. For some targets it involves creating dummy
> > containers like node/socket/core for x86 where just modeling
> > a thread is sufficient.
> 
> I acknowledge your concern regarding QOM tree stability.  We have QOM
> introspection commands since 1.2.  They make the QOM tree part of the
> external interface, but we've never spelled out which parts of it (if
> any) are ABI.  Until we do, parts become de facto ABI by being used in
> anger.  As a result, we don't know something's ABI until it breaks.
> 
> Andreas, do you have an opinion on proper use of QOM by external
> software?
> 
> > The similar but a bit more abstract approach was suggested
> > by David https://lists.gnu.org/archive/html/qemu-ppc/2016-02/msg0.html
> 
> Cc'ing him.

Actually I was already on the thread via my upstream email.

> If I understand the high-level idea correctly, David
> proposes to have an abstract type cpu-package with generic properties.
> Its concrete subtypes are composed of whatever components make up the
> hot-pluggable unit.

Yes, that's pretty much it.

> Management software can then use the generic properties to deal with hot
> plug without having to know about the concrete subtypes, at least to
> some useful degree.

That's the plan.

> Similarly, the generic properties suffice for implementing generic
> high-level interfaces like -smp.

Here it gets a bit fuzzier.  The idea is that the abstract type would
still make sense in a post -smp world allowing heterogenous setups.
However the concrete subtypes used for the time being are likely to
get their configuration from -smp, whether directly or indirectly.

My preferred option was for the machine type to "push" the smp
configuration into the package objects, rather than having them look
at the global directly.  However, in working with Bharata on a draft
implementation, I'm not actually sure how to do that.  So we might end
up looking at the global from the (concrete) package at least for the
time being.

At least, that's how it would work for semi-abstracted package types
as we have on some platforms.  For machine types which are supposed to
match real hardware closely, I'd could see package subtypes 

Re: [Qemu-devel] [RFC] QMP: add query-hotpluggable-cpus

[David Gibson]

On Tue, Feb 16, 2016 at 11:52:42AM +0100, Igor Mammedov wrote:
> On Tue, 16 Feb 2016 16:48:34 +1100
> David Gibson  wrote:
> 
> > On Mon, Feb 15, 2016 at 08:43:41PM +0100, Markus Armbruster wrote:
> > > Igor Mammedov  writes:
> > >   
> > > > it will allow mgmt to query present and possible to hotplug CPUs
> > > > it is required from a target platform that wish to support
> > > > command to set board specific MachineClass.possible_cpus() hook,
> > > > which will return a list of possible CPUs with options
> > > > that would be needed for hotplugging possible CPUs.
> > > >
> > > > For RFC there are:
> > > >'arch_id': 'int' - mandatory unique CPU number,
> > > >   for x86 it's APIC ID for ARM it's MPIDR
> > > >'type': 'str' - CPU object type for usage with device_add
> > > >
> > > > and a set of optional fields that would allows mgmt tools
> > > > to know at what granularity and where a new CPU could be
> > > > hotplugged;
> > > > [node],[socket],[core],[thread]
> > > > Hopefully that should cover needs for CPU hotplug porposes for
> > > > magor targets and we can extend structure in future adding
> > > > more fields if it will be needed.
> > > >
> > > > also for present CPUs there is a 'cpu_link' field which
> > > > would allow mgmt inspect whatever object/abstraction
> > > > the target platform considers as CPU object.
> > > >
> > > > For RFC purposes implements only for x86 target so far.  
> > > 
> > > Adding ad hoc queries as we go won't scale.  Could this be solved by a
> > > generic introspection interface?  
> > 
> > That's my main concern as well.
> > 
> > Igor,  did you see my post with a proposal for how to organize
> > hotpluggable packages of CPUs?  I believe that would also solve the
> > problem at hand here, by having a standard QOM location with
> > discoverable cpu objects.
> > 
> > The interface in your patch in particular would *not* solve the
> > problem of advertising to management layers what the granularity of
> > CPU hotplug is, which we absolutely need for Power.
> I've had in mind Power as well, as topology items are optional
> a query can respond with what granularity board would like
> to use and what type of object it could be hotplugged:
> 
> -> { "execute": "query-hotpluggable-cpus" }
> <- {"return": [
>  {"core": 2, "socket": 2, "arch_id": 2, "type": "power-foo-core-cpu"},
>  {"core": 1, "socket": 1, "arch_id": 1, "type": "power-foo-core-cpu"},
>  {"core": 0, "socket": 0, "arch_id": 0, "type": "power-foo-core-cpu", 
> "cpu_link": "/machine/unattached/device[3]"}
>]}

Hrm.. except your arch_id is supplied by a CPUClass hook, making it a
per-thread property, whereas here it needs to be per-core.

Other than that I guess this covers what we need for Power, however I
dislike the idea of typing the hotplug granularity to be at any fixed
level of the socket/core/thread heirarchy.  As noted elsewhere, while
all machines are likely to have some sort of similar heirarchy, giving
it fixed levels of "socket", "core" and "thread" may be limiting.

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [RFC] QMP: add query-hotpluggable-cpus

[David Gibson]

On Tue, Feb 16, 2016 at 11:36:55AM +0100, Igor Mammedov wrote:
> On Mon, 15 Feb 2016 20:43:41 +0100
> Markus Armbruster  wrote:
> 
> > Igor Mammedov  writes:
> > 
> > > it will allow mgmt to query present and possible to hotplug CPUs
> > > it is required from a target platform that wish to support
> > > command to set board specific MachineClass.possible_cpus() hook,
> > > which will return a list of possible CPUs with options
> > > that would be needed for hotplugging possible CPUs.
> > >
> > > For RFC there are:
> > >'arch_id': 'int' - mandatory unique CPU number,
> > >   for x86 it's APIC ID for ARM it's MPIDR
> > >'type': 'str' - CPU object type for usage with device_add
> > >
> > > and a set of optional fields that would allows mgmt tools
> > > to know at what granularity and where a new CPU could be
> > > hotplugged;
> > > [node],[socket],[core],[thread]
> > > Hopefully that should cover needs for CPU hotplug porposes for
> > > magor targets and we can extend structure in future adding
> > > more fields if it will be needed.
> > >
> > > also for present CPUs there is a 'cpu_link' field which
> > > would allow mgmt inspect whatever object/abstraction
> > > the target platform considers as CPU object.
> > >
> > > For RFC purposes implements only for x86 target so far.  
> > 
> > Adding ad hoc queries as we go won't scale.  Could this be solved by a
> > generic introspection interface?
> Do you mean generic QOM introspection?
> 
> Using QOM we could have '/cpus' container and create QOM links
> for exiting (populated links) and possible (empty links) CPUs.
> However in that case link's name will need have a special format
> that will convey an information necessary for mgmt to hotplug
> a CPU object, at least:
>   - where: [node],[socket],[core],[thread] options
>   - optionally what CPU object to use with device_add command

Hmm.. is it not enough to follow the link and get the topology
information by examining the target?

In the design Eduardo and I have been discussing we're actually not
planning to allow device_add to construct CPU packages - at least, not
for the time being.  The idea is that the machine type will construct
enough packages for maxcpus, and management just toggles them on and
off.

We can eventually allow construction of new packages with device_add,
but for now that gets hidden inside the platform until we've worked
out more details.

> Another approach to do QOM introspection would be to model hierarchy 
> of objects like node/socket/core..., That's what Andreas
> worked on. Only it still suffers the same issue as above
> wrt introspection and hotplug, One can pre-create empty
> [nodes][sockets[cores]] containers at startup but then
> leaf nodes that could be hotplugged would be a links anyway
> and then again we need to give them special formatted names
> (not well documented at that mgmt could make sense of).
> That hierarchy would need to become stable ABI once
> mgmt will start using it and QOM tree is quite unstable
> now for that. For some targets it involves creating dummy
> containers like node/socket/core for x86 where just modeling
> a thread is sufficient.

I'd prefer to avoid exposing the node/socket/core heirarchy through
the QOM interfaces as much as possible.  Although all systems I know
of have a heirarchy something like that, exactly what the levels may
vary, so I think it's better not to bake that into our interface.

Properties giving core/socket/node id values isn't too bad, but
building a whole tree mirroring that heirarchy seems like asking for
trouble.

> The similar but a bit more abstract approach was suggested
> by David https://lists.gnu.org/archive/html/qemu-ppc/2016-02/msg0.html
> 
> Benefit of dedicated CPU hotplug focused QMP command is that
> it can be quite abstract to suite most targets and not depend
> on how a target models CPUs internally and still provide
> information needed for hotplugging a CPU object.
> That way we can split efforts on how we model/refactor CPUs
> internally and how mgmt would work with them using
> -device/device_add.
> 

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v2 06/11] nvdimm acpi: initialize the resource used by NVDIMM ACPI

[Xiao Guangrong]

On 02/18/2016 01:26 AM, Michael S. Tsirkin wrote:

On Wed, Feb 17, 2016 at 10:04:18AM +0800, Xiao Guangrong wrote:

As for the rest could that commands go via MMIO that we usually
use for control path?


So both input data and output data go through single MMIO, we need to
introduce a protocol to pass these data, that is complex?

And is any MMIO we can reuse (more complexer？) or we should allocate this
MMIO page （the old question - where to allocated?）?

Maybe you could reuse/extend memhotplug IO interface,
or alternatively as Michael suggested add a vendor specific PCI_Config,
I'd suggest PM device for that (hw/acpi/[piix4.c|ihc9.c])
which I like even better since you won't need to care about which ports
to allocate at all.


Well, if Michael does not object, i will do it in the next version. :)


Sorry, the thread's so long by now that I'm no longer sure what does "it" refer 
to.


Never mind i saw you were busy on other loops.

"It" means the suggestion of Igor that "map each label area right after each
NVDIMM's data memory" so we do not emulate it in QEMU and is good for the 
performance
of label these are the points i like. However it also brings 
complexity/limitation for
later DSM commands supports since both dsm input & output need to go through 
single MMIO.

Your idea?

Re: [Qemu-devel] [PATCH COLO-Frame v14 35/40] COLO/filter: add each netdev a buffer filter

[Hailiang Zhang]

On 2016/2/18 11:23, Jason Wang wrote:



On 02/06/2016 05:28 PM, zhanghailiang wrote:

For COLO periodic mode, it need to buffer packets that
sent by VM, and we will not release these packets until
finish a checkpoint.

Here, we add each netdev a buffer-filter that will be controlled
by COLO. It is disabled by default, and the packets will not pass
through these filters. If users don't enable COLO while configure
qemu, these buffer-filters will not be added.

Signed-off-by: zhanghailiang 
Cc: Jason Wang 
Cc: Yang Hongyang 
---
v14:
- New patch
---
  include/migration/colo.h |  2 ++
  include/net/filter.h |  2 ++
  migration/colo-comm.c|  5 +
  migration/colo.c | 20 
  net/filter-buffer.c  |  2 --
  stubs/migration-colo.c   |  4 
  6 files changed, 33 insertions(+), 2 deletions(-)

diff --git a/include/migration/colo.h b/include/migration/colo.h
index 919b135..22b92c9 100644
--- a/include/migration/colo.h
+++ b/include/migration/colo.h
@@ -37,4 +37,6 @@ COLOMode get_colo_mode(void);
  void colo_do_failover(MigrationState *s);

  bool colo_shutdown(void);
+void colo_add_buffer_filter(const char *netdev_id, void *opaque);
+
  #endif
diff --git a/include/net/filter.h b/include/net/filter.h
index 0159080..0da0d97 100644
--- a/include/net/filter.h
+++ b/include/net/filter.h
@@ -22,6 +22,8 @@
  #define NETFILTER_CLASS(klass) \
  OBJECT_CLASS_CHECK(NetFilterClass, (klass), TYPE_NETFILTER)

+#define TYPE_FILTER_BUFFER "filter-buffer"
+
  typedef void (FilterSetup) (NetFilterState *nf, Error **errp);
  typedef void (FilterCleanup) (NetFilterState *nf);
  /*
diff --git a/migration/colo-comm.c b/migration/colo-comm.c
index 3943e94..91d873e 100644
--- a/migration/colo-comm.c
+++ b/migration/colo-comm.c
@@ -13,6 +13,7 @@

  #include 
  #include "trace.h"
+#include 

  typedef struct {
   bool colo_requested;
@@ -58,6 +59,10 @@ static const VMStateDescription colo_state = {
  void colo_info_init(void)
  {
  vmstate_register(NULL, 0, _state, _info);
+/* FIXME: Remove this after COLO switch to use colo-proxy */
+if (colo_supported()) {
+netdev_init_add_handler(colo_add_buffer_filter, NULL);
+}
  }

  bool migration_incoming_enable_colo(void)
diff --git a/migration/colo.c b/migration/colo.c
index 98a12fc..235578b 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -18,6 +18,8 @@
  #include "qemu/error-report.h"
  #include "migration/failover.h"
  #include "qapi-event.h"
+#include "net/net.h"
+#include "net/filter.h"

  static bool vmstate_loading;

@@ -380,6 +382,24 @@ static int colo_prepare_before_save(MigrationState *s)
  return ret;
  }

+void colo_add_buffer_filter(const char *netdev_id, void *opaque)
+{
+NetFilterState *nf;
+char filter_name[128];
+Error *errp = NULL;
+
+snprintf(filter_name, sizeof(filter_name),
+"%scolo", netdev_id);
+nf = netdev_add_filter(netdev_id, TYPE_FILTER_BUFFER, filter_name, false,
+   );


Using netdev_add_filter() is too heavyweight and also a overkill. Why
not simply call object_new_with_props() with what ever props you want
(e.g interval=0) here?



Make sense, i will drop the helper and fix like what you said, thanks.


+if (errp) {
+error_report_err(errp);
+return;
+}
+/* Only buffer the packets that sent out by VM */
+nf->direction = NET_FILTER_DIRECTION_RX;
+}
+
  static void colo_process_checkpoint(MigrationState *s)
  {
  QEMUSizedBuffer *buffer = NULL;
diff --git a/net/filter-buffer.c b/net/filter-buffer.c
index 58cea8f..e70d156 100644
--- a/net/filter-buffer.c
+++ b/net/filter-buffer.c
@@ -16,8 +16,6 @@
  #include "qapi-visit.h"
  #include "qom/object.h"

-#define TYPE_FILTER_BUFFER "filter-buffer"
-
  #define FILTER_BUFFER(obj) \
  OBJECT_CHECK(FilterBufferState, (obj), TYPE_FILTER_BUFFER)

diff --git a/stubs/migration-colo.c b/stubs/migration-colo.c
index 1996cd9..8e74acb 100644
--- a/stubs/migration-colo.c
+++ b/stubs/migration-colo.c
@@ -48,3 +48,7 @@ bool colo_shutdown(void)
  {
  return false;
  }
+
+void colo_add_buffer_filter(const char *netdev_id, void *opaque)
+{
+}



.

Re: [Qemu-devel] [PATCH COLO-Frame v14 37/40] COLO: enable buffer filters for PVM

[Hailiang Zhang]

On 2016/2/18 11:31, Jason Wang wrote:



On 02/06/2016 05:28 PM, zhanghailiang wrote:

Enable all buffer filters that added by COLO while
go into COLO process, and disable them while exit COLO.

Signed-off-by: zhanghailiang 
Cc: Jason Wang 
Cc: Yang Hongyang 
---
v14:
- New patch
---
  migration/colo.c | 32 
  1 file changed, 32 insertions(+)

diff --git a/migration/colo.c b/migration/colo.c
index 235578b..86a7638 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -104,10 +104,26 @@ static void secondary_vm_do_failover(void)
  }
  }

+static void colo_set_filter_status(NetFilterState *nf, void *opaque,
+   Error **errp)
+{
+char colo_filter[128];
+char *name = object_get_canonical_path_component(OBJECT(nf));
+char *status = opaque;
+
+snprintf(colo_filter, sizeof(colo_filter), "%scolo", nf->netdev_id);
+if (strcmp(colo_filter, name)) {
+return;
+}


Checking by name is not elegant. As we've discussed last time, why not
let filter-buffer track all filters with zero interval in a linked list
and just export a helper to disable and enable them all? Things will be
greatly simplified with this, and there's even no need for patch 36.




Yes, i know what you mean, but we have to add another
'QTAILQ_ENTRY() entry' like member into struct NetFilterState
if we do like that, is it acceptable ?


.

Re: [Qemu-devel] [PATCH COLO-Frame v14 37/40] COLO: enable buffer filters for PVM

[Jason Wang]

On 02/06/2016 05:28 PM, zhanghailiang wrote:
> Enable all buffer filters that added by COLO while
> go into COLO process, and disable them while exit COLO.
>
> Signed-off-by: zhanghailiang 
> Cc: Jason Wang 
> Cc: Yang Hongyang 
> ---
> v14:
> - New patch
> ---
>  migration/colo.c | 32 
>  1 file changed, 32 insertions(+)
>
> diff --git a/migration/colo.c b/migration/colo.c
> index 235578b..86a7638 100644
> --- a/migration/colo.c
> +++ b/migration/colo.c
> @@ -104,10 +104,26 @@ static void secondary_vm_do_failover(void)
>  }
>  }
>  
> +static void colo_set_filter_status(NetFilterState *nf, void *opaque,
> +   Error **errp)
> +{
> +char colo_filter[128];
> +char *name = object_get_canonical_path_component(OBJECT(nf));
> +char *status = opaque;
> +
> +snprintf(colo_filter, sizeof(colo_filter), "%scolo", nf->netdev_id);
> +if (strcmp(colo_filter, name)) {
> +return;
> +}

Checking by name is not elegant. As we've discussed last time, why not
let filter-buffer track all filters with zero interval in a linked list
and just export a helper to disable and enable them all? Things will be
greatly simplified with this, and there's even no need for patch 36.
   

Re: [Qemu-devel] [PATCH COLO-Frame v14 32/40] net/filter: Introduce a helper to add a filter to the netdev

[Hailiang Zhang]

On 2016/2/18 11:19, Jason Wang wrote:



On 02/06/2016 05:28 PM, zhanghailiang wrote:

We add a new helper function netdev_add_filter(),
this function can help adding a filter object to a netdev.

Signed-off-by: zhanghailiang 
Cc: Jason Wang 
Cc: Yang Hongyang 
---
  include/net/filter.h |  7 +++
  net/filter.c | 34 ++
  2 files changed, 41 insertions(+)

diff --git a/include/net/filter.h b/include/net/filter.h
index af3c53c..0159080 100644
--- a/include/net/filter.h
+++ b/include/net/filter.h
@@ -55,6 +55,7 @@ struct NetFilterState {
  char *netdev_id;
  NetClientState *netdev;
  NetFilterDirection direction;
+bool is_default;


I believe we've agreed that, we will remove this flag? And it seems that
it was not used by following patches.



Oops, i forgot to remove this useless codes. I will fix it in next version.


  bool enabled;
  QTAILQ_ENTRY(NetFilterState) next;
  };
@@ -74,4 +75,10 @@ ssize_t qemu_netfilter_pass_to_next(NetClientState *sender,
  int iovcnt,
  void *opaque);

+NetFilterState *netdev_add_filter(const char *netdev_id,
+  const char *filter_type,
+  const char *filter_id,
+  bool enabled,
+  Error **errp);
+
  #endif /* QEMU_NET_FILTER_H */
diff --git a/net/filter.c b/net/filter.c
index 5551cf1..dbe9399 100644
--- a/net/filter.c
+++ b/net/filter.c
@@ -177,6 +177,7 @@ static void netfilter_init(Object *obj)
  * for netfilter will be enabled.
  */
  nf->enabled = true;
+nf->is_default = false;

  object_property_add_str(obj, "netdev",
  netfilter_get_netdev_id, netfilter_set_netdev_id,
@@ -232,6 +233,39 @@ static void netfilter_complete(UserCreatable *uc, Error 
**errp)
  QTAILQ_INSERT_TAIL(>netdev->filters, nf, next);
  }

+NetFilterState *netdev_add_filter(const char *netdev_id,
+  const char *filter_type,
+  const char *filter_id,
+  bool enabled,
+  Error **errp)
+{
+NetClientState *nc = qemu_find_netdev(netdev_id);
+Object *filter;
+Error *local_err = NULL;
+
+/* FIXME: Not support multiple queues */
+if (!nc || nc->queue_index > 1) {
+return NULL;
+}
+/* Not support vhost-net */
+if (get_vhost_net(nc)) {
+return NULL;
+}


We will check those in netfilter_complete(), no?



Yes, I'd like to check this more early here.

Thanks,
hailiang


+
+filter = object_new_with_props(filter_type,
+object_get_objects_root(),
+filter_id,
+_err,
+"netdev", netdev_id,
+"status", enabled ? "enable" : "disable",
+NULL);
+if (local_err) {
+error_propagate(errp, local_err);
+return NULL;
+}
+return NETFILTER(filter);
+}
+
  static void netfilter_finalize(Object *obj)
  {
  NetFilterState *nf = NETFILTER(obj);



.

Re: [Qemu-devel] [PATCH COLO-Frame v14 31/40] net/filter: Add a 'status' property for filter object

[Hailiang Zhang]

Hi Jason,

Happy spring festival! ;)

On 2016/2/18 11:00, Jason Wang wrote:



On 02/06/2016 05:28 PM, zhanghailiang wrote:

With this property, users can control if this filter is 'enable'
or 'disable'. The default behavior for filter is enabled.

We will skip the disabled filter when delivering packets in net layer.

Signed-off-by: zhanghailiang 
Cc: Jason Wang 
Cc: Yang Hongyang 
---
  include/net/filter.h |  1 +
  net/filter.c | 45 +
  2 files changed, 46 insertions(+)

diff --git a/include/net/filter.h b/include/net/filter.h
index 5639976..af3c53c 100644
--- a/include/net/filter.h
+++ b/include/net/filter.h
@@ -55,6 +55,7 @@ struct NetFilterState {
  char *netdev_id;
  NetClientState *netdev;
  NetFilterDirection direction;
+bool enabled;
  QTAILQ_ENTRY(NetFilterState) next;
  };

diff --git a/net/filter.c b/net/filter.c
index d2a514e..5551cf1 100644
--- a/net/filter.c
+++ b/net/filter.c
@@ -17,6 +17,11 @@
  #include "qom/object_interfaces.h"
  #include "qemu/iov.h"

+static inline bool qemu_need_skip_netfilter(NetFilterState *nf)
+{
+return nf->enabled ? false : true;
+}


Suggest to rename this to qemu_netfilter_can_skip.



OK, i will fix it.


+
  ssize_t qemu_netfilter_receive(NetFilterState *nf,
 NetFilterDirection direction,
 NetClientState *sender,
@@ -25,6 +30,10 @@ ssize_t qemu_netfilter_receive(NetFilterState *nf,
 int iovcnt,
 NetPacketSent *sent_cb)
  {
+/* Don't go through the filter if it is disabled */
+if (qemu_need_skip_netfilter(nf)) {
+return 0;
+}


The code is self explained, so the commnet is useless.



OK, i will remove it.


  if (nf->direction == direction ||
  nf->direction == NET_FILTER_DIRECTION_ALL) {
  return NETFILTER_GET_CLASS(OBJECT(nf))->receive_iov(
@@ -134,8 +143,41 @@ static void netfilter_set_direction(Object *obj, int 
direction, Error **errp)
  nf->direction = direction;
  }

+static char *netfilter_get_status(Object *obj, Error **errp)
+{
+NetFilterState *nf = NETFILTER(obj);
+
+if (nf->enabled) {
+return g_strdup("enable");
+} else {
+return g_strdup("disable");
+}
+}
+
+static void netfilter_set_status(Object *obj, const char *str, Error **errp)
+{
+NetFilterState *nf = NETFILTER(obj);
+
+if (!strcmp(str, "enable")) {
+nf->enabled = true;
+} else if (!strcmp(str, "disable")) {
+nf->enabled = false;


Do we need a filter specific callback here to drain filter's queue? E.g
for filter-buffer ,need to release all the packets that has been buffered.



I don't think we need to do that here, we drain the filter's queue explicitly
when we try to change the filter's status .
Besides, it seems that, we didn't have a callback in filter layer to process
the queued packets for different types of filters.


+} else {
+error_setg(errp, "Invalid value for netfilter status, "
+ "should be 'enable' or 'disable'");
+}
+}
+
  static void netfilter_init(Object *obj)
  {
+NetFilterState *nf = NETFILTER(obj);
+
+/*
+* If not configured with 'status' property, the default status
+* for netfilter will be enabled.
+*/
+nf->enabled = true;


The code is clear too, so the comment need to be moved to qemu-options.hx.



OK, i will fix it in next version.

Thanks,
Hailiang


+
  object_property_add_str(obj, "netdev",
  netfilter_get_netdev_id, netfilter_set_netdev_id,
  NULL);
@@ -143,6 +185,9 @@ static void netfilter_init(Object *obj)
   NetFilterDirection_lookup,
   netfilter_get_direction, netfilter_set_direction,
   NULL);
+object_property_add_str(obj, "status",
+netfilter_get_status, netfilter_set_status,
+NULL);
  }

  static void netfilter_complete(UserCreatable *uc, Error **errp)



.

Re: [Qemu-devel] [PATCH COLO-Frame v14 35/40] COLO/filter: add each netdev a buffer filter

[Jason Wang]

On 02/06/2016 05:28 PM, zhanghailiang wrote:
> For COLO periodic mode, it need to buffer packets that
> sent by VM, and we will not release these packets until
> finish a checkpoint.
>
> Here, we add each netdev a buffer-filter that will be controlled
> by COLO. It is disabled by default, and the packets will not pass
> through these filters. If users don't enable COLO while configure
> qemu, these buffer-filters will not be added.
>
> Signed-off-by: zhanghailiang 
> Cc: Jason Wang 
> Cc: Yang Hongyang 
> ---
> v14:
> - New patch
> ---
>  include/migration/colo.h |  2 ++
>  include/net/filter.h |  2 ++
>  migration/colo-comm.c|  5 +
>  migration/colo.c | 20 
>  net/filter-buffer.c  |  2 --
>  stubs/migration-colo.c   |  4 
>  6 files changed, 33 insertions(+), 2 deletions(-)
>
> diff --git a/include/migration/colo.h b/include/migration/colo.h
> index 919b135..22b92c9 100644
> --- a/include/migration/colo.h
> +++ b/include/migration/colo.h
> @@ -37,4 +37,6 @@ COLOMode get_colo_mode(void);
>  void colo_do_failover(MigrationState *s);
>  
>  bool colo_shutdown(void);
> +void colo_add_buffer_filter(const char *netdev_id, void *opaque);
> +
>  #endif
> diff --git a/include/net/filter.h b/include/net/filter.h
> index 0159080..0da0d97 100644
> --- a/include/net/filter.h
> +++ b/include/net/filter.h
> @@ -22,6 +22,8 @@
>  #define NETFILTER_CLASS(klass) \
>  OBJECT_CLASS_CHECK(NetFilterClass, (klass), TYPE_NETFILTER)
>  
> +#define TYPE_FILTER_BUFFER "filter-buffer"
> +
>  typedef void (FilterSetup) (NetFilterState *nf, Error **errp);
>  typedef void (FilterCleanup) (NetFilterState *nf);
>  /*
> diff --git a/migration/colo-comm.c b/migration/colo-comm.c
> index 3943e94..91d873e 100644
> --- a/migration/colo-comm.c
> +++ b/migration/colo-comm.c
> @@ -13,6 +13,7 @@
>  
>  #include 
>  #include "trace.h"
> +#include 
>  
>  typedef struct {
>   bool colo_requested;
> @@ -58,6 +59,10 @@ static const VMStateDescription colo_state = {
>  void colo_info_init(void)
>  {
>  vmstate_register(NULL, 0, _state, _info);
> +/* FIXME: Remove this after COLO switch to use colo-proxy */
> +if (colo_supported()) {
> +netdev_init_add_handler(colo_add_buffer_filter, NULL);
> +}
>  }
>  
>  bool migration_incoming_enable_colo(void)
> diff --git a/migration/colo.c b/migration/colo.c
> index 98a12fc..235578b 100644
> --- a/migration/colo.c
> +++ b/migration/colo.c
> @@ -18,6 +18,8 @@
>  #include "qemu/error-report.h"
>  #include "migration/failover.h"
>  #include "qapi-event.h"
> +#include "net/net.h"
> +#include "net/filter.h"
>  
>  static bool vmstate_loading;
>  
> @@ -380,6 +382,24 @@ static int colo_prepare_before_save(MigrationState *s)
>  return ret;
>  }
>  
> +void colo_add_buffer_filter(const char *netdev_id, void *opaque)
> +{
> +NetFilterState *nf;
> +char filter_name[128];
> +Error *errp = NULL;
> +
> +snprintf(filter_name, sizeof(filter_name),
> +"%scolo", netdev_id);
> +nf = netdev_add_filter(netdev_id, TYPE_FILTER_BUFFER, filter_name, false,
> +   );

Using netdev_add_filter() is too heavyweight and also a overkill. Why
not simply call object_new_with_props() with what ever props you want
(e.g interval=0) here?

> +if (errp) {
> +error_report_err(errp);
> +return;
> +}
> +/* Only buffer the packets that sent out by VM */
> +nf->direction = NET_FILTER_DIRECTION_RX;
> +}
> +
>  static void colo_process_checkpoint(MigrationState *s)
>  {
>  QEMUSizedBuffer *buffer = NULL;
> diff --git a/net/filter-buffer.c b/net/filter-buffer.c
> index 58cea8f..e70d156 100644
> --- a/net/filter-buffer.c
> +++ b/net/filter-buffer.c
> @@ -16,8 +16,6 @@
>  #include "qapi-visit.h"
>  #include "qom/object.h"
>  
> -#define TYPE_FILTER_BUFFER "filter-buffer"
> -
>  #define FILTER_BUFFER(obj) \
>  OBJECT_CHECK(FilterBufferState, (obj), TYPE_FILTER_BUFFER)
>  
> diff --git a/stubs/migration-colo.c b/stubs/migration-colo.c
> index 1996cd9..8e74acb 100644
> --- a/stubs/migration-colo.c
> +++ b/stubs/migration-colo.c
> @@ -48,3 +48,7 @@ bool colo_shutdown(void)
>  {
>  return false;
>  }
> +
> +void colo_add_buffer_filter(const char *netdev_id, void *opaque)
> +{
> +}

Re: [Qemu-devel] [PATCH COLO-Frame v14 32/40] net/filter: Introduce a helper to add a filter to the netdev

[Jason Wang]

On 02/06/2016 05:28 PM, zhanghailiang wrote:
> We add a new helper function netdev_add_filter(),
> this function can help adding a filter object to a netdev.
>
> Signed-off-by: zhanghailiang 
> Cc: Jason Wang 
> Cc: Yang Hongyang 
> ---
>  include/net/filter.h |  7 +++
>  net/filter.c | 34 ++
>  2 files changed, 41 insertions(+)
>
> diff --git a/include/net/filter.h b/include/net/filter.h
> index af3c53c..0159080 100644
> --- a/include/net/filter.h
> +++ b/include/net/filter.h
> @@ -55,6 +55,7 @@ struct NetFilterState {
>  char *netdev_id;
>  NetClientState *netdev;
>  NetFilterDirection direction;
> +bool is_default;

I believe we've agreed that, we will remove this flag? And it seems that
it was not used by following patches.

>  bool enabled;
>  QTAILQ_ENTRY(NetFilterState) next;
>  };
> @@ -74,4 +75,10 @@ ssize_t qemu_netfilter_pass_to_next(NetClientState *sender,
>  int iovcnt,
>  void *opaque);
>  
> +NetFilterState *netdev_add_filter(const char *netdev_id,
> +  const char *filter_type,
> +  const char *filter_id,
> +  bool enabled,
> +  Error **errp);
> +
>  #endif /* QEMU_NET_FILTER_H */
> diff --git a/net/filter.c b/net/filter.c
> index 5551cf1..dbe9399 100644
> --- a/net/filter.c
> +++ b/net/filter.c
> @@ -177,6 +177,7 @@ static void netfilter_init(Object *obj)
>  * for netfilter will be enabled.
>  */
>  nf->enabled = true;
> +nf->is_default = false;
>  
>  object_property_add_str(obj, "netdev",
>  netfilter_get_netdev_id, netfilter_set_netdev_id,
> @@ -232,6 +233,39 @@ static void netfilter_complete(UserCreatable *uc, Error 
> **errp)
>  QTAILQ_INSERT_TAIL(>netdev->filters, nf, next);
>  }
>  
> +NetFilterState *netdev_add_filter(const char *netdev_id,
> +  const char *filter_type,
> +  const char *filter_id,
> +  bool enabled,
> +  Error **errp)
> +{
> +NetClientState *nc = qemu_find_netdev(netdev_id);
> +Object *filter;
> +Error *local_err = NULL;
> +
> +/* FIXME: Not support multiple queues */
> +if (!nc || nc->queue_index > 1) {
> +return NULL;
> +}
> +/* Not support vhost-net */
> +if (get_vhost_net(nc)) {
> +return NULL;
> +}

We will check those in netfilter_complete(), no?

> +
> +filter = object_new_with_props(filter_type,
> +object_get_objects_root(),
> +filter_id,
> +_err,
> +"netdev", netdev_id,
> +"status", enabled ? "enable" : "disable",
> +NULL);
> +if (local_err) {
> +error_propagate(errp, local_err);
> +return NULL;
> +}
> +return NETFILTER(filter);
> +}
> +
>  static void netfilter_finalize(Object *obj)
>  {
>  NetFilterState *nf = NETFILTER(obj);

Re: [Qemu-devel] [PATCH COLO-Frame v14 31/40] net/filter: Add a 'status' property for filter object

[Jason Wang]

On 02/06/2016 05:28 PM, zhanghailiang wrote:
> With this property, users can control if this filter is 'enable'
> or 'disable'. The default behavior for filter is enabled.
>
> We will skip the disabled filter when delivering packets in net layer.
>
> Signed-off-by: zhanghailiang 
> Cc: Jason Wang 
> Cc: Yang Hongyang 
> ---
>  include/net/filter.h |  1 +
>  net/filter.c | 45 +
>  2 files changed, 46 insertions(+)
>
> diff --git a/include/net/filter.h b/include/net/filter.h
> index 5639976..af3c53c 100644
> --- a/include/net/filter.h
> +++ b/include/net/filter.h
> @@ -55,6 +55,7 @@ struct NetFilterState {
>  char *netdev_id;
>  NetClientState *netdev;
>  NetFilterDirection direction;
> +bool enabled;
>  QTAILQ_ENTRY(NetFilterState) next;
>  };
>  
> diff --git a/net/filter.c b/net/filter.c
> index d2a514e..5551cf1 100644
> --- a/net/filter.c
> +++ b/net/filter.c
> @@ -17,6 +17,11 @@
>  #include "qom/object_interfaces.h"
>  #include "qemu/iov.h"
>  
> +static inline bool qemu_need_skip_netfilter(NetFilterState *nf)
> +{
> +return nf->enabled ? false : true;
> +}

Suggest to rename this to qemu_netfilter_can_skip.

> +
>  ssize_t qemu_netfilter_receive(NetFilterState *nf,
> NetFilterDirection direction,
> NetClientState *sender,
> @@ -25,6 +30,10 @@ ssize_t qemu_netfilter_receive(NetFilterState *nf,
> int iovcnt,
> NetPacketSent *sent_cb)
>  {
> +/* Don't go through the filter if it is disabled */
> +if (qemu_need_skip_netfilter(nf)) {
> +return 0;
> +}

The code is self explained, so the commnet is useless.

>  if (nf->direction == direction ||
>  nf->direction == NET_FILTER_DIRECTION_ALL) {
>  return NETFILTER_GET_CLASS(OBJECT(nf))->receive_iov(
> @@ -134,8 +143,41 @@ static void netfilter_set_direction(Object *obj, int 
> direction, Error **errp)
>  nf->direction = direction;
>  }
>  
> +static char *netfilter_get_status(Object *obj, Error **errp)
> +{
> +NetFilterState *nf = NETFILTER(obj);
> +
> +if (nf->enabled) {
> +return g_strdup("enable");
> +} else {
> +return g_strdup("disable");
> +}
> +}
> +
> +static void netfilter_set_status(Object *obj, const char *str, Error **errp)
> +{
> +NetFilterState *nf = NETFILTER(obj);
> +
> +if (!strcmp(str, "enable")) {
> +nf->enabled = true;
> +} else if (!strcmp(str, "disable")) {
> +nf->enabled = false;

Do we need a filter specific callback here to drain filter's queue? E.g
for filter-buffer ,need to release all the packets that has been buffered.

> +} else {
> +error_setg(errp, "Invalid value for netfilter status, "
> + "should be 'enable' or 'disable'");
> +}
> +}
> +
>  static void netfilter_init(Object *obj)
>  {
> +NetFilterState *nf = NETFILTER(obj);
> +
> +/*
> +* If not configured with 'status' property, the default status
> +* for netfilter will be enabled.
> +*/
> +nf->enabled = true;

The code is clear too, so the comment need to be moved to qemu-options.hx.

> +
>  object_property_add_str(obj, "netdev",
>  netfilter_get_netdev_id, netfilter_set_netdev_id,
>  NULL);
> @@ -143,6 +185,9 @@ static void netfilter_init(Object *obj)
>   NetFilterDirection_lookup,
>   netfilter_get_direction, 
> netfilter_set_direction,
>   NULL);
> +object_property_add_str(obj, "status",
> +netfilter_get_status, netfilter_set_status,
> +NULL);
>  }
>  
>  static void netfilter_complete(UserCreatable *uc, Error **errp)

Re: [Qemu-devel] [PATCH] net: netmap: probe netmap interface for virtio-net header

[Jason Wang]

On 02/05/2016 06:30 PM, Vincenzo Maffione wrote:
> Previous implementation of has_ufo, has_vnet_hdr, has_vnet_hdr_len, etc.
> did not really probe for virtio-net header support for the netmap
> interface attached to the backend. These callbacks were correct for
> VALE ports, but incorrect for hardware NICs, pipes, monitors, etc.
>
> This patch fixes the implementation to work properly with all kinds
> of netmap ports.
>
> Signed-off-by: Vincenzo Maffione 
> ---
>  net/netmap.c | 70 
> ++--
>  1 file changed, 44 insertions(+), 26 deletions(-)
>
> diff --git a/net/netmap.c b/net/netmap.c
> index 9710321..f2dcaeb 100644
> --- a/net/netmap.c
> +++ b/net/netmap.c
> @@ -323,20 +323,55 @@ static void netmap_cleanup(NetClientState *nc)
>  }
>  
>  /* Offloading manipulation support callbacks. */
> -static bool netmap_has_ufo(NetClientState *nc)
> +static int netmap_do_set_vnet_hdr_len(NetmapState *s, int len, bool 
> err_report)

Passing something like err_report usually means it was the
responsibility of caller to report. So let's remove the err_report and
let caller to decide based on the return value.

>  {
> -return true;
> +struct nmreq req;
> +int err;
> +
> +/* Issue a NETMAP_BDG_VNET_HDR command to change the virtio-net header
> + * length for the netmap adapter associated to 's->ifname'.
> + */
> +memset(, 0, sizeof(req));
> +pstrcpy(req.nr_name, sizeof(req.nr_name), s->ifname);
> +req.nr_version = NETMAP_API;
> +req.nr_cmd = NETMAP_BDG_VNET_HDR;
> +req.nr_arg1 = len;
> +err = ioctl(s->nmd->fd, NIOCREGIF, );
> +if (err) {
> +if (err_report) {
> +error_report("Unable to execute NETMAP_BDG_VNET_HDR on %s: %s",
> + s->ifname, strerror(errno));
> +}
> +return -1;
> +}
> +
> +/* Keep track of the current length. */
> +s->vnet_hdr_len = len;
> +
> +return 0;
>  }
>  
> -static bool netmap_has_vnet_hdr(NetClientState *nc)
> +static bool netmap_has_vnet_hdr_len(NetClientState *nc, int len)
>  {
> +NetmapState *s = DO_UPCAST(NetmapState, nc, nc);
> +int prev_len = s->vnet_hdr_len;
> +
> +/* Check that we can set the new length. */
> +if (netmap_do_set_vnet_hdr_len(s, len, false)) {
> +return false;
> +}
> +
> +/* Restore the previous length. */
> +netmap_do_set_vnet_hdr_len(s, prev_len, true);
> +
>  return true;
>  }
>  
> -static bool netmap_has_vnet_hdr_len(NetClientState *nc, int len)
> +/* A netmap interface that supports virtio-net headers always
> + * supports UFO, so we use this callback also for the has_ufo hook. */
> +static bool netmap_has_vnet_hdr(NetClientState *nc)
>  {
> -return len == 0 || len == sizeof(struct virtio_net_hdr) ||
> -len == sizeof(struct virtio_net_hdr_mrg_rxbuf);
> +return netmap_has_vnet_hdr_len(nc, sizeof(struct virtio_net_hdr));
>  }
>  
>  static void netmap_using_vnet_hdr(NetClientState *nc, bool enable)
> @@ -346,25 +381,8 @@ static void netmap_using_vnet_hdr(NetClientState *nc, 
> bool enable)
>  static void netmap_set_vnet_hdr_len(NetClientState *nc, int len)
>  {
>  NetmapState *s = DO_UPCAST(NetmapState, nc, nc);
> -int err;
> -struct nmreq req;
>  
> -/* Issue a NETMAP_BDG_VNET_HDR command to change the virtio-net header
> - * length for the netmap adapter associated to 's->ifname'.
> - */
> -memset(, 0, sizeof(req));
> -pstrcpy(req.nr_name, sizeof(req.nr_name), s->ifname);
> -req.nr_version = NETMAP_API;
> -req.nr_cmd = NETMAP_BDG_VNET_HDR;
> -req.nr_arg1 = len;
> -err = ioctl(s->nmd->fd, NIOCREGIF, );
> -if (err) {
> -error_report("Unable to execute NETMAP_BDG_VNET_HDR on %s: %s",
> - s->ifname, strerror(errno));
> -} else {
> -/* Keep track of the current length. */
> -s->vnet_hdr_len = len;
> -}
> +netmap_do_set_vnet_hdr_len(s, len, true);
>  }
>  
>  static void netmap_set_offload(NetClientState *nc, int csum, int tso4, int 
> tso6,
> @@ -376,7 +394,7 @@ static void netmap_set_offload(NetClientState *nc, int 
> csum, int tso4, int tso6,
>   * enables the offloadings.
>   */
>  if (!s->vnet_hdr_len) {
> -netmap_set_vnet_hdr_len(nc, sizeof(struct virtio_net_hdr));
> +netmap_do_set_vnet_hdr_len(s, sizeof(struct virtio_net_hdr), true);
>  }
>  }
>  
> @@ -388,7 +406,7 @@ static NetClientInfo net_netmap_info = {
>  .receive_iov = netmap_receive_iov,
>  .poll = netmap_poll,
>  .cleanup = netmap_cleanup,
> -.has_ufo = netmap_has_ufo,
> +.has_ufo = netmap_has_vnet_hdr,

This look suspicious, I'm not sure this is correct for netmap. May need
a comment to explain. Usually vnet hdr does not imply ufo.

>  .has_vnet_hdr = netmap_has_vnet_hdr,
>  .has_vnet_hdr_len = netmap_has_vnet_hdr_len,
>  .using_vnet_hdr = netmap_using_vnet_hdr,

Re: [Qemu-devel] [PATCH v2 3/3] migration: allow to suppress configuration section submission

[David Gibson]

On Wed, Feb 17, 2016 at 05:06:43PM +0100, Greg Kurz wrote:
> Since the addition of the configuration section in QEMU 2.4, it is impossible
> to migrate a pseries-2.3 machine back to QEMU 2.3.
> 
> This patch makes it possible thanks to a new machine property which allows to
> disable configuration section submission.
> 
> To disable submission, just add:
> 
> -machine suppress-config-section=on
> 
> Alternatively, if the target QEMU version isn't known at startup, this can
> be done later from the QEMU monitor with:
> 
> qom-set /machine suppress-config-section on
> 
> This property won't be automatically set for pseries-2.3 because it would
> then break backward migration to QEMU 2.4. If automatic behaviour is needed,
> it is up to the tooling to handle this.

As noted elsewhere, I'd actually be ok with enabling it for
pseries-2.3.  Basically we have to chose whether to work against qemu
2.3 or 2.4 out of the box, we can't have both, and it sounds like qemu
2.3 is more widely deployed than qemu 2.4.

> 
> Signed-off-by: Greg Kurz 

Reviewed-by: David Gibson 

I'm not sure whose tree these need to go in via.

> ---
>  hw/core/machine.c   |   21 +
>  include/hw/boards.h |1 +
>  migration/savevm.c  |8 +++-
>  qemu-options.hx |3 ++-
>  4 files changed, 31 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index 7203dd260bf2..ad4ecdc65787 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -327,6 +327,21 @@ static bool machine_get_require_config_section(Object 
> *obj, Error **errp)
>  return ms->require_config_section;
>  }
>  
> +static void machine_set_suppress_config_section(Object *obj, bool value,
> +Error **errp)
> +{
> +MachineState *ms = MACHINE(obj);
> +
> +ms->suppress_config_section = value;
> +}
> +
> +static bool machine_get_suppress_config_section(Object *obj, Error **errp)
> +{
> +MachineState *ms = MACHINE(obj);
> +
> +return ms->suppress_config_section;
> +}
> +
>  static int error_on_sysbus_device(SysBusDevice *sbdev, void *opaque)
>  {
>  error_report("Option '-device %s' cannot be handled by this machine",
> @@ -489,6 +504,12 @@ static void machine_initfn(Object *obj)
>  object_property_set_description(obj, "require-config-section",
>  "Set on/off to reject/accept migration 
> without configuration section",
>  NULL);
> +object_property_add_bool(obj, "suppress-config-section",
> + machine_get_suppress_config_section,
> + machine_set_suppress_config_section, NULL);
> +object_property_set_description(obj, "suppress-config-section",
> +"Set on/off to enable/disable 
> configuration section migration",
> +NULL);
>  
>  /* Register notifier when init is done for sysbus sanity checks */
>  ms->sysbus_notifier.notify = machine_init_notify;
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index d6ff1ba4c260..35aaa6493458 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -129,6 +129,7 @@ struct MachineState {
>  bool iommu;
>  bool suppress_vmdesc;
>  bool require_config_section;
> +bool suppress_config_section;
>  
>  ram_addr_t ram_size;
>  ram_addr_t maxram_size;
> diff --git a/migration/savevm.c b/migration/savevm.c
> index f8dee03a3350..3fd07efeb38b 100644
> --- a/migration/savevm.c
> +++ b/migration/savevm.c
> @@ -878,13 +878,19 @@ bool qemu_savevm_state_blocked(Error **errp)
>  return false;
>  }
>  
> +static bool should_send_configuration(void)
> +{
> +MachineState *machine = MACHINE(qdev_get_machine());
> +return !machine->suppress_config_section;
> +}
> +
>  void qemu_savevm_state_header(QEMUFile *f)
>  {
>  trace_savevm_state_header();
>  qemu_put_be32(f, QEMU_VM_FILE_MAGIC);
>  qemu_put_be32(f, QEMU_VM_FILE_VERSION);
>  
> -if (!savevm_state.skip_configuration) {
> +if (!savevm_state.skip_configuration && should_send_configuration()) {
>  qemu_put_byte(f, QEMU_VM_CONFIGURATION);
>  vmstate_save_state(f, _configuration, _state, 0);
>  }
> diff --git a/qemu-options.hx b/qemu-options.hx
> index 172471c75b1c..af08fa8ec314 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -44,7 +44,8 @@ DEF("machine", HAS_ARG, QEMU_OPTION_machine, \
>  "dea-key-wrap=on|off controls support for DEA key 
> wrapping (default=on)\n"
>  "suppress-vmdesc=on|off disables self-describing 
> migration (default=off)\n"
>  "nvdimm=on|off controls NVDIMM support (default=off)\n"
> -"require-config-section=on|off incoming migration 
> requires configuration section (default=on)\n",
> +"

Re: [Qemu-devel] [PATCH 1/4] machine: Use type_init() to register machine classes

[David Gibson]

On Tue, Feb 16, 2016 at 06:59:04PM -0200, Eduardo Habkost wrote:
> Change all machine_init() users that simply call type_register*()
> to use type_init().
> 
> Cc: Evgeny Voevodin 
> Cc: Maksim Kozlov 
> Cc: Igor Mitsyanko 
> Cc: Dmitry Solodkiy 
> Cc: Peter Maydell 
> Cc: Rob Herring 
> Cc: Andrzej Zaborowski 
> Cc: Michael Walle 
> Cc: "Hervé Poussineau" 
> Cc: Aurelien Jarno 
> Cc: Leon Alrae 
> Cc: Alexander Graf 
> Cc: David Gibson 
> Cc: Blue Swirl 
> Cc: Mark Cave-Ayland 
> Cc: Max Filippov 
> Cc: "Michael S. Tsirkin" 
> Signed-off-by: Eduardo Habkost 
> ---
>  hw/arm/exynos4_boards.c | 2 +-
>  hw/arm/gumstix.c| 2 +-
>  hw/arm/highbank.c   | 2 +-
>  hw/arm/nseries.c| 2 +-
>  hw/arm/omap_sx1.c   | 2 +-
>  hw/arm/realview.c   | 2 +-
>  hw/arm/spitz.c  | 2 +-
>  hw/arm/stellaris.c  | 2 +-
>  hw/arm/versatilepb.c| 2 +-
>  hw/arm/vexpress.c   | 2 +-
>  hw/arm/virt.c   | 2 +-
>  hw/lm32/lm32_boards.c   | 2 +-
>  hw/mips/mips_jazz.c | 2 +-
>  hw/ppc/ppc405_boards.c  | 2 +-
>  hw/ppc/spapr.c  | 2 +-
>  hw/sparc/sun4m.c| 4 
>  hw/sparc64/sun4u.c  | 4 
>  hw/xtensa/xtfpga.c  | 2 +-
>  include/hw/boards.h | 2 +-
>  include/hw/i386/pc.h| 2 +-
>  20 files changed, 18 insertions(+), 26 deletions(-)

For spapr,

Acked-by: David Gibson 

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH] net/filter-redirector:Add filter-redirector

[Jason Wang]

On 02/05/2016 02:50 PM, Zhang Chen wrote:
> From: ZhangChen 
>
> Filter-redirector is a netfilter plugin.
> It gives qemu the ability to redirect net packet.
> redirector can redirect filter's net packet to outdev.
> and redirect indev's packet to filter.
>
>  filter
>+
>|
>|
>   redirector   |
>+-+
>|   | |
>|   | |
>|   | |
>   indev ++ +>  outdev
>| |   |
>| |   |
>| |   |
>+-+
>  |
>  |
>  v
>   filter
>
> usage:
>
> -netdev tap,id=hn0
> -chardev socket,id=s0,host=ip_primary,port=X,server,nowait
> -chardev socket,id=s1,host=ip_primary,port=Y,server,nowait
> -filter-redirector,id=r0,netdev=hn0,queue=tx/rx/all,indev=s0,outdev=s1
>
> Signed-off-by: ZhangChen 
> Signed-off-by: Wen Congyang 
> ---

Thanks a lot for the patch. Like mirror, let's design a unit-test for
this. And what's more, is there any chance to unify the codes? (At least
parts of the codes could be reused).

>  net/Makefile.objs   |   1 +
>  net/filter-redirector.c | 245 
> 
>  qemu-options.hx |   6 ++
>  vl.c|   3 +-
>  4 files changed, 254 insertions(+), 1 deletion(-)
>  create mode 100644 net/filter-redirector.c
>
> diff --git a/net/Makefile.objs b/net/Makefile.objs
> index 5fa2f97..f4290a5 100644
> --- a/net/Makefile.objs
> +++ b/net/Makefile.objs
> @@ -15,3 +15,4 @@ common-obj-$(CONFIG_VDE) += vde.o
>  common-obj-$(CONFIG_NETMAP) += netmap.o
>  common-obj-y += filter.o
>  common-obj-y += filter-buffer.o
> +common-obj-y += filter-redirector.o
> diff --git a/net/filter-redirector.c b/net/filter-redirector.c
> new file mode 100644
> index 000..364e463
> --- /dev/null
> +++ b/net/filter-redirector.c
> @@ -0,0 +1,245 @@
> +/*
> + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
> + * Copyright (c) 2016 FUJITSU LIMITED
> + * Copyright (c) 2016 Intel Corporation
> + *
> + * Author: Zhang Chen 
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or
> + * later.  See the COPYING file in the top-level directory.
> + */
> +
> +#include "net/filter.h"
> +#include "net/net.h"
> +#include "qemu-common.h"
> +#include "qapi/qmp/qerror.h"
> +#include "qapi-visit.h"
> +#include "qom/object.h"
> +#include "qemu/main-loop.h"
> +#include "qemu/error-report.h"
> +#include "trace.h"
> +#include "sysemu/char.h"
> +#include "qemu/iov.h"
> +#include "qemu/sockets.h"
> +
> +#define FILTER_REDIRECTOR(obj) \
> +OBJECT_CHECK(RedirectorState, (obj), TYPE_FILTER_REDIRECTOR)
> +
> +#define TYPE_FILTER_REDIRECTOR "filter-redirector"
> +#define REDIRECT_HEADER_LEN sizeof(uint32_t)
> +
> +typedef struct RedirectorState {
> +NetFilterState parent_obj;
> +NetQueue *incoming_queue;/* guest normal net queue */

The comment looks unless and maybe even wrong when queue=rx?

> +char *indev;
> +char *outdev;
> +CharDriverState *chr_in;
> +CharDriverState *chr_out;
> +} RedirectorState;
> +
> +static ssize_t filter_redirector_send(NetFilterState *nf,
> +   const struct iovec *iov,
> +   int iovcnt)
> +{
> +RedirectorState *s = FILTER_REDIRECTOR(nf);
> +ssize_t ret = 0;
> +ssize_t size = 0;
> +uint32_t len =  0;
> +char *buf;
> +
> +size = iov_size(iov, iovcnt);
> +len = htonl(size);
> +if (!size) {
> +return 0;
> +}
> +
> +buf = g_malloc0(size);
> +iov_to_buf(iov, iovcnt, 0, buf, size);
> +ret = qemu_chr_fe_write_all(s->chr_out, (uint8_t *), sizeof(len));
> +if (ret < 0) {

Similar to mirror, need check the return value against sizeof(len). And
the code could be simplified with something like:

goto out;

...

out:
g_free(buf);
return ret;

And you can see another advantages of unifying the codes here, avoid
duplicating bugx/fixes.

> +g_free(buf);
> +return ret;
> +}
> +
> +ret = qemu_chr_fe_write_all(s->chr_out, (uint8_t *)buf, size);
> +g_free(buf);
> +return ret;
> +}
> +
> +static ssize_t filter_redirector_receive_iov(NetFilterState *nf,
> + NetClientState *sender,
> + unsigned flags,
> + const struct iovec *iov,
> + int iovcnt,
> + 

Re: [Qemu-devel] [PATCH] add CephFS support in VirtFS

[Jevon Qiao]

On 17/2/16 17:37, Daniel P. Berrange wrote:

On Wed, Feb 17, 2016 at 03:32:06PM +0800, Jevon Qiao wrote:

Hi Daniel,

Thank you for reviewing my code, please see my reply in-line.
On 15/2/16 17:17, Daniel P. Berrange wrote:

On Sun, Feb 14, 2016 at 01:06:40PM +0800, Jevon Qiao wrote:

+
+static int cephfs_parse_opts(QemuOpts *opts, struct FsDriverEntry *fse)
+{
+const char *sec_model = qemu_opt_get(opts, "security_model");
+const char *path = qemu_opt_get(opts, "path");
+
+if (!sec_model) {
+fprintf(stderr, "Invalid argument security_model specified with "
+"cephfs fsdriver\n");

Bad indent.

BTW, is there any tool I can use to check the coding style in Qemu?

As already mentioned there is 'scripts/checkpatch.pl'. If you are just
wanting to check a single patch before sending it you can run it
manually eg

git show | ./scripts/checkpatch.pl -

If you have a branch holding a whole series of patches to submit, then
it is easier to automate it. eg

  git rebase -i master -x 'git show | ./scripts/checkpatch.pl -'

Thank you for the detailed answers.

NB, the script doesn't catch all style problems, but it does a pretty
good job. So even if the script passes, don't be suprised if reviewers
point out further style issues.

Good to know:-) .

Thanks,
Jevon

Regards,
Daniel

Re: [Qemu-devel] [PATCH] add CephFS support in VirtFS

[Jevon Qiao]

On 17/2/16 17:04, Greg Kurz wrote:

On Wed, 17 Feb 2016 16:49:33 +0800
Jevon Qiao  wrote:


On 17/2/16 16:01, Greg Kurz wrote:

On Wed, 17 Feb 2016 15:32:06 +0800
Jevon Qiao  wrote:
  

Hi Daniel,

Thank you for reviewing my code, please see my reply in-line.
On 15/2/16 17:17, Daniel P. Berrange wrote:

On Sun, Feb 14, 2016 at 01:06:40PM +0800, Jevon Qiao wrote:

diff --git a/configure b/configure
index 83b40fc..cecece7 100755
--- a/configure
+++ b/configure
@@ -1372,6 +1377,7 @@ disabled with --disable-FEATURE, default is enabled if
available:
  vhost-net   vhost-net acceleration support
  spice   spice
  rbd rados block device (rbd)
+  cephfs  Ceph File System

Inconsistent vertical alignment with surrounding text

This is just a display issue, I'll send the patch with 'git send-email'
later after I address all the technical comments.

Expect reviewers to always comment on broken formatting. If you want the
review to be focused on technical aspects, the best choice is to fix the
way you send patches first.

OK, I'll re-send the patches first.


I don't ask you to resend this patch without the fixes. I just
wanted to put emphasis on the fact that correct formatting is
the first thing to have in mind before sending patches, not
just a "display issue".
Yes, I understand. I'll resend the patch with the fix which at least 
addresses Daniel's comments.

BTW, I had also answered your questions about g_new0() and a
tool to check coding style... not sure you saw that.
Sorry, I missed that part due to the long email:-( . I read it now, 
thank you for the answers.


Thanks,
Jevon

Cheers.

--
Greg


Thanks,
Jevon

  libiscsiiscsi support
  libnfs  nfs support
  smartcard   smartcard support (libcacard)
+/*
+ * Helper function for cephfs_preadv and cephfs_pwritev
+ */
+inline static ssize_t preadv_pwritev(struct ceph_mount_info *cmount, int
fd,

Your email client is mangling long lines, here and in many other
places in the file. Please either fix your email client to not
insert arbitrary line breaks, or use git send-email to submit
the patch.

Ditto.

+  const struct iovec *iov, int iov_cnt,
+  off_t offset, bool do_write)
+{
+ssize_t ret = 0;
+int i = 0;

Use size_t for iterators

I'll revise the code.

+size_t len = 0;
+void *buf, *buftmp;
+size_t bufoffset = 0;
+
+for (; i < iov_cnt; i++) {
+len += iov[i].iov_len;
+}

iov_size() does this calculation

Thanks for the suggestion.

+
+buf = malloc(len);

Use g_new0(uint8_t, len)

OK.

+if (buf == NULL) {
+errno = ENOMEM;
+return -1;
+}

and don't check ENOMEM;

Any reason for this?

https://developer.gnome.org/glib/unstable/glib-Memory-Allocation.html
  

+
+i = 0;
+buftmp = buf;
+if (do_write) {
+for (i = 0; i < iov_cnt; i++) {
+memcpy((buftmp + bufoffset), iov[i].iov_base, iov[i].iov_len);
+bufoffset += iov[i].iov_len;
+}
+ret = ceph_write(cmount, fd, buf, len, offset);
+if (ret <= 0) {
+   errno = -ret;
+   ret = -1;
+}
+} else {
+ret = ceph_read(cmount, fd, buf, len, offset);
+if (ret <= 0) {
+errno = -ret;
+ret = -1;
+} else {
+for (i = 0; i < iov_cnt; i++) {
+memcpy(iov[i].iov_base, (buftmp + bufoffset),
iov[i].iov_len);

Mangled long line again.

That's the email client issue.

+bufoffset += iov[i].iov_len;
+}
+}
+}
+
+free(buf);
+return ret;
+}
+
+static int cephfs_update_file_cred(struct ceph_mount_info *cmount,
+   const char *name, FsCred *credp)

Align the parameters on following line to the '('

I will revise the code.

+{
+int fd, ret;
+fd = ceph_open(cmount, name, O_NONBLOCK | O_NOFOLLOW, credp->fc_mode);
+if (fd < 0) {
+return fd;
+}
+ret = ceph_fchown(cmount, fd, credp->fc_uid, credp->fc_gid);
+if (ret < 0) {
+goto err_out;
+}
+ret = ceph_fchmod(cmount, fd, credp->fc_mode & 0);
+err_out:
+close(fd);
+return ret;
+}
+
+static int cephfs_lstat(FsContext *fs_ctx, V9fsPath *fs_path,
+struct stat *stbuf)
+{
+D_CEPHFS("cephfs_lstat");

All of these D_CEPHFS() lines you have are really inserting trace
points, so you should really use the QEMU trace facility instead
of a fprintf() based macro. ie add to trace-events and then
call the generated trace fnuction for your event. Then get rid
of your D_CEPHFS macro.

I will revise the code.

+int ret;
+char *path = fs_path->data;
+struct cephfs_data *cfsdata = (struct cephfs_data *)fs_ctx->private;

fs_ctx->private is 'void *' so you don't need the (struct cephfs_data *)
cast there - 'void *' casts to anything automatically. The same issue
in all the 

Re: [Qemu-devel] [PATCH v2 0/6] external backup api

[Fam Zheng]

On Wed, 02/17 20:47, Vladimir Sementsov-Ogievskiy wrote:
> What about exporting bitmap as separate nbd entity? Just implement
> block driver, which will read from bitmap? If consider only read
> access and disabled (or frozen) bitmaps it should be simple enough.

Yes, I think this idea also makes sense. The driver could implement .bdrv_read
by memcpy from the bitmap.

Fam

Re: [Qemu-devel] [PATCH v3] hw/ppc/spapr: Implement the h_page_init hypercall

[David Gibson]

On Wed, Feb 17, 2016 at 05:45:42PM +0100, Thomas Huth wrote:
> This hypercall either initializes a page with zeros, or copies
> another page.
> According to LoPAPR, the i-cache of the page should also be
> flushed if using H_ICACHE_INVALIDATE or H_ICACHE_SYNCHRONIZE,
> and the d-cache should be synchronized to the RAM if the
> H_ICACHE_SYNCHRONIZE flag is used. For this, two new functions
> are introduced, kvmppc_dcbst_range() and kvmppc_icbi()_range, which
> use the corresponding assembler instructions to flush the caches
> if running with KVM on Power. If the code runs with TCG instead,
> the code only uses tb_flush(), assuming that this will be
> enough for synchronization.
> 
> Signed-off-by: Thomas Huth 

Ugh, sorry to nitpick, but I've hit one more little issue here.

> ---
>  v3:
>  - Change H_HARDWARE return value into H_PARAMETER (which should
>be the right one according to the LoPAPR spec)
>  - The dcbst and icbi helpers now contain the for-loop, too
> 
>  PS: I'll have a look at the missing entries in the ibm,hypertas
>  property later, once this got merged.
> 
>  hw/ppc/spapr_hcall.c | 64 
> 
>  target-ppc/kvm_ppc.h | 36 +++--
>  2 files changed, 98 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c
> index 6e9b6be..6343caa 100644
> --- a/hw/ppc/spapr_hcall.c
> +++ b/hw/ppc/spapr_hcall.c
> @@ -386,6 +386,69 @@ static target_ulong h_set_xdabr(PowerPCCPU *cpu, 
> sPAPRMachineState *spapr,
>  return H_SUCCESS;
>  }
>  
> +static target_ulong h_page_init(PowerPCCPU *cpu, sPAPRMachineState *spapr,
> +target_ulong opcode, target_ulong *args)
> +{
> +target_ulong flags = args[0];
> +hwaddr dst = args[1];
> +hwaddr src = args[2];
> +hwaddr len = TARGET_PAGE_SIZE;
> +uint8_t *pdst, *psrc;
> +
> +if (flags & ~(H_ICACHE_SYNCHRONIZE | H_ICACHE_INVALIDATE
> +  | H_COPY_PAGE | H_ZERO_PAGE)) {
> +qemu_log_mask(LOG_UNIMP, "h_page_init: Bad flags (" TARGET_FMT_lx 
> "\n",
> +  flags);
> +return H_PARAMETER;
> +}
> +
> +if (!is_ram_address(spapr, dst) || (dst & ~TARGET_PAGE_MASK) != 0) {
> +return H_PARAMETER;
> +}
> +
> +/* Map-in source */
> +if (flags & H_COPY_PAGE) {
> +if (!is_ram_address(spapr, src) || (src & ~TARGET_PAGE_MASK) != 0) {
> +return H_PARAMETER;
> +}
> +psrc = cpu_physical_memory_map(src, , 0);
> +if (!psrc || len != TARGET_PAGE_SIZE) {
> +return H_PARAMETER;
> +}
> +}
> +
> +/* Map-in destination */
> +pdst = cpu_physical_memory_map(dst, , 1);
> +if (!pdst || len != TARGET_PAGE_SIZE) {
> +if (flags & H_COPY_PAGE) {
> +cpu_physical_memory_unmap(psrc, len, 0, 0);
> +}
> +return H_PARAMETER;
> +}
> +
> +if (flags & H_ZERO_PAGE) {
> +memset(pdst, 0, len);
> +}
> +if (flags & H_COPY_PAGE) {
> +memcpy(pdst, psrc, len);
> +cpu_physical_memory_unmap(psrc, len, 0, len);

So, at least on my compiler version (Fedora 23) I get one of those
irritating "variable may be used uninitialized" warnings here for
psrc.

The compiler is wrong, of course, but you could both prevent its
confusion and make the code a little straightforward if you remove the
multiple tests on flags.  I think you should be able to do that if you
restructure as:

map in dest
if H_COPY_PAGE
   map in src
   memcpy
   unmap src
else if H_ZERO_PAGE
   memset
cache sync
unmap dest

> +}
> +
> +if (kvm_enabled() && (flags & H_ICACHE_SYNCHRONIZE) != 0) {
> +kvmppc_dcbst_range(cpu, pdst, len);
> +}
> +if (flags & (H_ICACHE_SYNCHRONIZE | H_ICACHE_INVALIDATE)) {
> +if (kvm_enabled()) {
> +kvmppc_icbi_range(cpu, pdst, len);
> +} else {
> +tb_flush(CPU(cpu));
> +}
> +}
> +
> +cpu_physical_memory_unmap(pdst, len, 1, len);
> +return H_SUCCESS;
> +}
> +
>  #define FLAGS_REGISTER_VPA 0x2000ULL
>  #define FLAGS_REGISTER_DTL 0x4000ULL
>  #define FLAGS_REGISTER_SLBSHADOW   0x6000ULL
> @@ -1045,6 +1108,7 @@ static void hypercall_register_types(void)
>  spapr_register_hypercall(H_SET_SPRG0, h_set_sprg0);
>  spapr_register_hypercall(H_SET_DABR, h_set_dabr);
>  spapr_register_hypercall(H_SET_XDABR, h_set_xdabr);
> +spapr_register_hypercall(H_PAGE_INIT, h_page_init);
>  spapr_register_hypercall(H_SET_MODE, h_set_mode);
>  
>  /* "debugger" hcalls (also used by SLOF). Note: We do -not- differenciate
> diff --git a/target-ppc/kvm_ppc.h b/target-ppc/kvm_ppc.h
> index aaa828c..fd64c44 100644
> --- a/target-ppc/kvm_ppc.h
> +++ b/target-ppc/kvm_ppc.h
> @@ -249,15 +249,47 @@ static inline int kvmppc_enable_hwrng(void)
>  #endif
>  
>  #ifndef 

Re: [Qemu-devel] [PATCH v1 2/2] generic-loader: Add a generic loader

[Alistair Francis]

On Wed, Feb 17, 2016 at 4:11 PM, Eric Blake  wrote:
> On 02/17/2016 05:03 PM, Alistair Francis wrote:
>
 +++ b/hw/misc/generic-loader.c
 @@ -0,0 +1,127 @@
 +/*
 + * Generic Loader
 + *
 + * Copyright (C) 2014 Li Guang
 + * Written by Li Guang 
>>>
>>> Want to claim 2016?
>>
>> Yep, I can do that. I'm never too sure when this can be changed or
>> not. Should I add a written by as well?
>
> I'm not a lawyer, so my response may not be authoritative; in
> particular, your employer may have specific rules that you must follow
> for any code you submit that was written on your employer's time, and
> that trumps anything I say here (that is, trust your lawyers more than
> you trust me).
>
> But in general, I tend to go by the simple rule of listing the first
> year that any of the code was first developed (if you are copying
> significant portions from some other file, then use the starting year
> from that file, even if your file didn't exist back then), through the
> current year, if my change is non-trivial (more than 10 lines, or
> altering an interface), while ignoring the issue for trivial things
> (such as fixing a typo, or doing a bulk search-and-replace across the
> tree).  As for an authorship line, I tend to omit those (they quickly go
> stale, and git history is sufficient for a much more accurate picture);
> the copyright line is more important legally than any author line.

Ok, I have added a Xilinx copyright line and not bothered with a
written by line.

Thanks for your help.

Thanks,

Alistair

>
> --
> Eric Blake   eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>

Re: [Qemu-devel] [PATCH] hw/ppc/spapr: Halt CPU when powering off via RTAS call

[David Gibson]

On Wed, Feb 17, 2016 at 07:23:19PM +0100, Thomas Huth wrote:
> The LoPAPR specification defines the following for the RTAS
> power-off call: "On successful operation, does not return".
> However, the implementation in QEMU currently returns and runs
> the guest CPU again for some more cycles. This caused some
> trouble with the new ppc implementation of the kvm-unit-tests
> recently. So let's make sure that the QEMU implementation
> follows the spec, thus stop the CPU to make sure that the
> RTAS call does not return to the guest anymore.
> 
> Signed-off-by: Thomas Huth 

Applied to ppc-for-2.6, thanks.

> ---
>  hw/ppc/spapr_rtas.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
> index 07ad672..b7c5ebd 100644
> --- a/hw/ppc/spapr_rtas.c
> +++ b/hw/ppc/spapr_rtas.c
> @@ -113,6 +113,7 @@ static void rtas_power_off(PowerPCCPU *cpu, 
> sPAPRMachineState *spapr,
>  return;
>  }
>  qemu_system_shutdown_request();
> +cpu_stop_current();
>  rtas_st(rets, 0, RTAS_OUT_SUCCESS);
>  }
>  

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v1 2/2] generic-loader: Add a generic loader

[Eric Blake]

On 02/17/2016 05:03 PM, Alistair Francis wrote:

>>> +++ b/hw/misc/generic-loader.c
>>> @@ -0,0 +1,127 @@
>>> +/*
>>> + * Generic Loader
>>> + *
>>> + * Copyright (C) 2014 Li Guang
>>> + * Written by Li Guang 
>>
>> Want to claim 2016?
> 
> Yep, I can do that. I'm never too sure when this can be changed or
> not. Should I add a written by as well?

I'm not a lawyer, so my response may not be authoritative; in
particular, your employer may have specific rules that you must follow
for any code you submit that was written on your employer's time, and
that trumps anything I say here (that is, trust your lawyers more than
you trust me).

But in general, I tend to go by the simple rule of listing the first
year that any of the code was first developed (if you are copying
significant portions from some other file, then use the starting year
from that file, even if your file didn't exist back then), through the
current year, if my change is non-trivial (more than 10 lines, or
altering an interface), while ignoring the issue for trivial things
(such as fixing a typo, or doing a bulk search-and-replace across the
tree).  As for an authorship line, I tend to omit those (they quickly go
stale, and git history is sufficient for a much more accurate picture);
the copyright line is more important legally than any author line.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v1 2/2] generic-loader: Add a generic loader

[Alistair Francis]

On Wed, Feb 17, 2016 at 1:41 PM, Eric Blake  wrote:
> On 02/17/2016 02:04 PM, Alistair Francis wrote:
>> Add a generic loader to QEMU which can be used to load images or set
>> memory values.
>>
>> This only supports ARM architectures at the moment.
>>
>> Signed-off-by: Alistair Francis 
>> ---
>> Changes since RFC:
>>  - Add BE support
>>
>
>>  hw/misc/generic-loader.c | 127 
>> +++
>>  include/hw/misc/generic-loader.h |  50 +++
>>  4 files changed, 180 insertions(+)
>>  create mode 100644 hw/misc/generic-loader.c
>>  create mode 100644 include/hw/misc/generic-loader.h
>
> We really ought to improve checkpatch.pl to flag patches that add new
> files not covered by MAINTAINERS.

Adding an entry for this.

>
>> +++ b/hw/misc/generic-loader.c
>> @@ -0,0 +1,127 @@
>> +/*
>> + * Generic Loader
>> + *
>> + * Copyright (C) 2014 Li Guang
>> + * Written by Li Guang 
>
> Want to claim 2016?

Yep, I can do that. I'm never too sure when this can be changed or
not. Should I add a written by as well?

>
>>
>> +
>> +#include "hw/sysbus.h"
>> +#include "sysemu/dma.h"
>> +#include "hw/loader.h"
>> +#include "hw/misc/generic-loader.h"
>
> New .c files should include "qemu/osdep.h" first, before anything else.

Adding it

>
>> +static void generic_loader_realize(DeviceState *dev, Error **errp)
>> +{
>> +GenericLoaderState *s = GENERIC_LOADER(dev);
>> +hwaddr entry;
>> +int big_endian;
>> +int size = 0;
>> +
>> +if (s->cpu_nr != CPU_NONE) {
>> +CPUState *cs = first_cpu;
>> +int cpu_num = 0;
>> +
>> +CPU_FOREACH(cs) {
>> +if (cpu_num == s->cpu_nr) {
>> +s->cpu = cs;
>> +break;
>> +} else if (!CPU_NEXT(cs)) {
>> +error_setg(errp, "Specified boot CPU#%d is non existant",
>> +   s->cpu_nr);
>
> s/non existant/nonexistent/

Thanks, fixed

Thanks,

Alistair

>
>
>> +++ b/include/hw/misc/generic-loader.h
>> @@ -0,0 +1,50 @@
>> +/*
>> + * Generic Loader
>> + *
>> + * Copyright (C) 2014 Li Guang
>> + * Written by Li Guang 
>
> 2016
>
> --
> Eric Blake   eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>

Re: [Qemu-devel] [PATCH v2 09/14] hw/timer: QOM'ify milkymist_sysctl

[xiaoqiang zhao]

I have sent the v3, which fix this problem.

> 在 2016年2月16日，02:14，Peter Maydell  写道：
> 
>> On 27 January 2016 at 02:54, xiaoqiang zhao  wrote:
>> * split milkymist_sysctl_init into milkymist_sysctl_info.instance_init and 
>> milkymist_sysctl_realize
> 
> I think the "info" in this function name is wrong ?
> 
>> * use DeviceClass::realize instead of SysBusDeviceClass::init
> 
> Please make sure you line wrap your commit messages (at somewhere
> around 70-72 columns is usual).
> 
> Otherwise:
> Reviewed-by: Peter Maydell 
> 
> Again, you should cc the maintainer for this device.
> 
> thanks
> -- PMM
> 

Re: [Qemu-devel] KVM call for agenda for 2016-02-16

[Juan Quintela]

Eduardo Habkost  wrote:
> On Tue, Feb 16, 2016 at 12:54:57PM +0100, Christian Borntraeger wrote:
>> So my quick and dirty summary of CPU as _I_ understand it
>> (and I only have some part time bandwidth at the moment for that)
>> 
>> x86 has cpu hotplug. Some history
>> qemu-kvm had cpu_set in the past
>> qemu has cpu_add for a while now
>> libvirt code uses cpu_add cross-platform out of the box
>> 
>> proposal to use device_add. Currently this has the following issues that
>> we need to discuss:
>> - will require capability checking and dual code in libvirt (and libvirt 
>> updates)
>> - Power has some constraints that are hard to model with just device add
>>  - David Gibson proposes a two layer interface
>>  - low level: device add cpu-package
>>  - high level
>> - David Hildenbrand has some concerns regarding CPU models (with base model 
>> + feature
>>   on/off), as device_add needs instantiatable type
>> - devel_del: s390 has no interface for cpu removal (Matts latest patches 
>> reset the 
>> machine just like z/VM - until we have some interface)
>> - anything else? (cpu hotplug on ARM or MIPS?)
>> 
>> Would be good to use todays call to have a plan how to finish things soon.
>> (maybe even for 2.6)
>
> Did the call happen? I am away from work for most days during the
> next 2 weeks, so I couldn't attend.

Yeap.  I sent some minutes, but they are kind of incoherent, I hope that
Andreas or Christian fill the holes...

Later, Juan.

Re: [Qemu-devel] [PATCH v1 2/2] generic-loader: Add a generic loader

[Eric Blake]

On 02/17/2016 02:04 PM, Alistair Francis wrote:
> Add a generic loader to QEMU which can be used to load images or set
> memory values.
> 
> This only supports ARM architectures at the moment.
> 
> Signed-off-by: Alistair Francis 
> ---
> Changes since RFC:
>  - Add BE support
> 

>  hw/misc/generic-loader.c | 127 
> +++
>  include/hw/misc/generic-loader.h |  50 +++
>  4 files changed, 180 insertions(+)
>  create mode 100644 hw/misc/generic-loader.c
>  create mode 100644 include/hw/misc/generic-loader.h

We really ought to improve checkpatch.pl to flag patches that add new
files not covered by MAINTAINERS.

> +++ b/hw/misc/generic-loader.c
> @@ -0,0 +1,127 @@
> +/*
> + * Generic Loader
> + *
> + * Copyright (C) 2014 Li Guang
> + * Written by Li Guang 

Want to claim 2016?

> 
> +
> +#include "hw/sysbus.h"
> +#include "sysemu/dma.h"
> +#include "hw/loader.h"
> +#include "hw/misc/generic-loader.h"

New .c files should include "qemu/osdep.h" first, before anything else.

> +static void generic_loader_realize(DeviceState *dev, Error **errp)
> +{
> +GenericLoaderState *s = GENERIC_LOADER(dev);
> +hwaddr entry;
> +int big_endian;
> +int size = 0;
> +
> +if (s->cpu_nr != CPU_NONE) {
> +CPUState *cs = first_cpu;
> +int cpu_num = 0;
> +
> +CPU_FOREACH(cs) {
> +if (cpu_num == s->cpu_nr) {
> +s->cpu = cs;
> +break;
> +} else if (!CPU_NEXT(cs)) {
> +error_setg(errp, "Specified boot CPU#%d is non existant",
> +   s->cpu_nr);

s/non existant/nonexistent/


> +++ b/include/hw/misc/generic-loader.h
> @@ -0,0 +1,50 @@
> +/*
> + * Generic Loader
> + *
> + * Copyright (C) 2014 Li Guang
> + * Written by Li Guang 

2016

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PATCH v7 3/4] tcg: Add type for vCPU pointers

[Lluís Vilanova]

Adds the 'TCGv_env' type for pointers to 'CPUArchState' objects. The
tracing infrastructure later needs to differentiate between regular
pointers and pointers to vCPUs.

Also changes all targets to use the new 'TCGv_env' type instead of the
generic 'TCGv_ptr'. As of now, the change is merely cosmetic ('TCGv_env'
translates into 'TCGv_ptr'), but that could change in the future to
enforce the difference.

Note that a 'TCGv_env' type (for 'CPUState') is not added, since all
helpers currently receive the architecture-specific
pointer ('CPUArchState').

Signed-off-by: Lluís Vilanova 
Acked-by: Richard Henderson 
---
 target-alpha/translate.c  |2 +-
 target-arm/translate.c|2 +-
 target-arm/translate.h|2 +-
 target-cris/translate.c   |2 +-
 target-i386/translate.c   |2 +-
 target-lm32/translate.c   |2 +-
 target-m68k/translate.c   |2 +-
 target-microblaze/translate.c |2 +-
 target-mips/translate.c   |2 +-
 target-moxie/translate.c  |2 +-
 target-openrisc/translate.c   |2 +-
 target-ppc/translate.c|2 +-
 target-s390x/translate.c  |2 +-
 target-sh4/translate.c|2 +-
 target-sparc/translate.c  |5 +++--
 target-tilegx/translate.c |2 +-
 target-tricore/translate.c|2 +-
 target-unicore32/translate.c  |2 +-
 target-xtensa/translate.c |2 +-
 tcg/tcg.h |1 +
 20 files changed, 22 insertions(+), 20 deletions(-)

diff --git a/target-alpha/translate.c b/target-alpha/translate.c
index 7b798b0..5b86992 100644
--- a/target-alpha/translate.c
+++ b/target-alpha/translate.c
@@ -93,7 +93,7 @@ typedef enum {
 } ExitStatus;
 
 /* global register indexes */
-static TCGv_ptr cpu_env;
+static TCGv_env cpu_env;
 static TCGv cpu_std_ir[31];
 static TCGv cpu_fir[31];
 static TCGv cpu_pc;
diff --git a/target-arm/translate.c b/target-arm/translate.c
index f6a38bc..d2367ab 100644
--- a/target-arm/translate.c
+++ b/target-arm/translate.c
@@ -56,7 +56,7 @@
 #define IS_USER(s) (s->user)
 #endif
 
-TCGv_ptr cpu_env;
+TCGv_env cpu_env;
 /* We reuse the same 64-bit temporaries for efficiency.  */
 static TCGv_i64 cpu_V0, cpu_V1, cpu_M0;
 static TCGv_i32 cpu_R[16];
diff --git a/target-arm/translate.h b/target-arm/translate.h
index 53ef971..82e3f6b 100644
--- a/target-arm/translate.h
+++ b/target-arm/translate.h
@@ -70,7 +70,7 @@ typedef struct DisasCompare {
 } DisasCompare;
 
 /* Share the TCG temporaries common between 32 and 64 bit modes.  */
-extern TCGv_ptr cpu_env;
+extern TCGv_env cpu_env;
 extern TCGv_i32 cpu_NF, cpu_ZF, cpu_CF, cpu_VF;
 extern TCGv_i64 cpu_exclusive_addr;
 extern TCGv_i64 cpu_exclusive_val;
diff --git a/target-cris/translate.c b/target-cris/translate.c
index 2a283e0..a73176c 100644
--- a/target-cris/translate.c
+++ b/target-cris/translate.c
@@ -60,7 +60,7 @@
 #define CC_MASK_NZVC 0xf
 #define CC_MASK_RNZV 0x10e
 
-static TCGv_ptr cpu_env;
+static TCGv_env cpu_env;
 static TCGv cpu_R[16];
 static TCGv cpu_PR[16];
 static TCGv cc_x;
diff --git a/target-i386/translate.c b/target-i386/translate.c
index f7ceadd..8fe167b 100644
--- a/target-i386/translate.c
+++ b/target-i386/translate.c
@@ -59,7 +59,7 @@
 //#define MACRO_TEST   1
 
 /* global register indexes */
-static TCGv_ptr cpu_env;
+static TCGv_env cpu_env;
 static TCGv cpu_A0;
 static TCGv cpu_cc_dst, cpu_cc_src, cpu_cc_src2, cpu_cc_srcT;
 static TCGv_i32 cpu_cc_op;
diff --git a/target-lm32/translate.c b/target-lm32/translate.c
index 3877993..256a51f 100644
--- a/target-lm32/translate.c
+++ b/target-lm32/translate.c
@@ -44,7 +44,7 @@
 
 #define MEM_INDEX 0
 
-static TCGv_ptr cpu_env;
+static TCGv_env cpu_env;
 static TCGv cpu_R[32];
 static TCGv cpu_pc;
 static TCGv cpu_ie;
diff --git a/target-m68k/translate.c b/target-m68k/translate.c
index 085cb6a..7560c3a 100644
--- a/target-m68k/translate.c
+++ b/target-m68k/translate.c
@@ -50,7 +50,7 @@
 static TCGv_i32 cpu_halted;
 static TCGv_i32 cpu_exception_index;
 
-static TCGv_ptr cpu_env;
+static TCGv_env cpu_env;
 
 static char cpu_reg_names[3*8*3 + 5*4];
 static TCGv cpu_dregs[8];
diff --git a/target-microblaze/translate.c b/target-microblaze/translate.c
index 296c4d7..f944965 100644
--- a/target-microblaze/translate.c
+++ b/target-microblaze/translate.c
@@ -46,7 +46,7 @@
 (((src) >> start) & ((1 << (end - start + 1)) - 1))
 
 static TCGv env_debug;
-static TCGv_ptr cpu_env;
+static TCGv_env cpu_env;
 static TCGv cpu_R[32];
 static TCGv cpu_SR[18];
 static TCGv env_imm;
diff --git a/target-mips/translate.c b/target-mips/translate.c
index 658926d..3706176 100644
--- a/target-mips/translate.c
+++ b/target-mips/translate.c
@@ -1353,7 +1353,7 @@ enum {
 };
 
 /* global register indices */
-static TCGv_ptr cpu_env;
+static TCGv_env cpu_env;
 static TCGv cpu_gpr[32], cpu_PC;
 static TCGv cpu_HI[MIPS_DSP_ACC], cpu_LO[MIPS_DSP_ACC];
 static TCGv cpu_dspctrl, btarget, bcond;
diff --git 

[Qemu-devel] [PATCH v7 4/4] trace: Add 'vcpu' event property to trace guest vCPU

[Lluís Vilanova]

This property identifies events that trace vCPU-specific information.

It adds a "CPUState*" argument to events with the property, identifying
the vCPU raising the event. TCG translation events also have a
"TCGv_env" implicit argument that is later used as the "CPUState*"
argument at execution time.

Signed-off-by: Lluís Vilanova 
---
 docs/tracing.txt |   41 +
 include/qemu/typedefs.h  |1 
 scripts/tracetool/__init__.py|   11 +++-
 scripts/tracetool/format/h.py|3 +
 scripts/tracetool/format/tcg_h.py|   31 +++---
 scripts/tracetool/format/tcg_helper_c.py |   35 +--
 scripts/tracetool/format/tcg_helper_h.py |7 +-
 scripts/tracetool/format/tcg_helper_wrapper_h.py |5 +-
 scripts/tracetool/format/ust_events_c.py |1 
 scripts/tracetool/transform.py   |4 +
 scripts/tracetool/vcpu.py|   69 ++
 trace/control.h  |3 +
 12 files changed, 184 insertions(+), 27 deletions(-)
 create mode 100644 scripts/tracetool/vcpu.py

diff --git a/docs/tracing.txt b/docs/tracing.txt
index 3853a6a..f5b39e7 100644
--- a/docs/tracing.txt
+++ b/docs/tracing.txt
@@ -347,3 +347,44 @@ This will immediately call:
 and will generate the TCG code to call:
 
 void trace_foo(uint8_t a1, uint32_t a2);
+
+=== "vcpu" ===
+
+Identifies events that trace vCPU-specific information. It implicitly adds a
+"CPUState*" argument, and extends the tracing print format to show the vCPU
+information. If used together with the "tcg" property, it adds a second
+"TCGv_cpu" argument that must point to the per-target global TCG register that
+points to the vCPU when guest code is executed (usually the "cpu_env" 
variable).
+
+The following example events:
+
+foo(uint32_t a) "a=%x"
+vcpu bar(uint32_t a) "a=%x"
+tcg vcpu baz(uint32_t a) "a=%x", "a=%x"
+
+Can be used as:
+
+#include "trace-tcg.h"
+
+CPUArchState *env;
+TCGv_ptr cpu_env;
+
+void some_disassembly_func(...)
+{
+/* trace emitted at this point */
+trace_foo(0xd1);
+/* trace emitted at this point */
+trace_bar(ENV_GET_CPU(env), 0xd2);
+/* trace emitted at this point (env) and when guest code is executed 
(cpu_env) */
+trace_baz_tcg(ENV_GET_CPU(env), cpu_env, 0xd3);
+}
+
+If the translating vCPU has address 0xc1 and code is later executed by vCPU
+0xc2, this would be an example output:
+
+// at guest code translation
+foo a=0xd1
+bar cpu=0xc1 a=0xd2
+baz_trans cpu=0xc1 a=0xd3
+// at guest code execution
+baz_exec cpu=0xc2 a=0xd3
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index 6ed91b4..9a5ead6 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -18,6 +18,7 @@ typedef struct BusState BusState;
 typedef struct CharDriverState CharDriverState;
 typedef struct CompatProperty CompatProperty;
 typedef struct CPUAddressSpace CPUAddressSpace;
+typedef struct CPUState CPUState;
 typedef struct DeviceListener DeviceListener;
 typedef struct DeviceState DeviceState;
 typedef struct DisplayChangeListener DisplayChangeListener;
diff --git a/scripts/tracetool/__init__.py b/scripts/tracetool/__init__.py
index 26878f4..d32bfa7 100644
--- a/scripts/tracetool/__init__.py
+++ b/scripts/tracetool/__init__.py
@@ -157,7 +157,7 @@ class Event(object):
   "(?:(?:(?P\".+),)?\s*(?P\".+))?"
   "\s*")
 
-_VALID_PROPS = set(["disable", "tcg", "tcg-trans", "tcg-exec"])
+_VALID_PROPS = set(["disable", "tcg", "tcg-trans", "tcg-exec", "vcpu"])
 
 def __init__(self, name, props, fmt, args, orig=None):
 """
@@ -226,7 +226,13 @@ class Event(object):
 if "tcg" in props and isinstance(fmt, str):
 raise ValueError("Events with 'tcg' property must have two 
formats")
 
-return Event(name, props, fmt, args)
+event = Event(name, props, fmt, args)
+
+# add implicit arguments when using the 'vcpu' property
+import tracetool.vcpu
+event = tracetool.vcpu.transform_event(event)
+
+return event
 
 def __repr__(self):
 """Evaluable string representation for this object."""
@@ -281,6 +287,7 @@ def _read_events(fobj):
 event_trans.name += "_trans"
 event_trans.properties += ["tcg-trans"]
 event_trans.fmt = event.fmt[0]
+# ignore TCG arguments
 args_trans = []
 for atrans, aorig in zip(
 event_trans.transform(tracetool.transform.TCG_2_HOST).args,
diff --git a/scripts/tracetool/format/h.py b/scripts/tracetool/format/h.py
index 9b39430..2bd68a2 100644
--- a/scripts/tracetool/format/h.py
+++ b/scripts/tracetool/format/h.py
@@ -6,7 +6,7 @@ trace/generated-tracers.h
 """
 
 __author__  

[Qemu-devel] [PATCH v7 0/4] trace: Show vCPU info in guest code events

[Lluís Vilanova]

NOTE: This series should complete the framework for guest code tracing. From
  here on, other series can concurrently add actual events and improve the
  guest code tracing features and performance (e.g., control tracing
  independently on each vCPU).

This series introduces the "vcpu" property for tracing events. This property
identifies events that are tied to a particular virtual CPU (e.g., executing an
instruction).

Events with this property have an implicit vcpu argument, which is shown in the
trace. In the case of events executed at TCG translation time, two implicit
arguments are added:

* The vCPU performing the code translation (shown in the translation-time trace)
* The vCPU executing the translated code (shown in the execution-time trace)

Note that the "vcpu" and "tcg" properties are not merged into a single one,
since events can be defined that relate to a vCPU but are never raised from TCG
code (e.g., interrupts).


Changes in v7
=

* Fix the modified event copying code.
* Minor fix in commit message for patch 3.


Changes in v6
=

* Rebase on 84c0781.
* Replace the added 'TCGv_cpu' type for 'TCGv_env', since the users are really
  pointing to 'CPUArchState'.
* Add functions to simplify event argument management.
* Add minor event copy cleanup (remove long "event.original.original" chains).
* Make it easier to change arguments injected by the 'vcpu' property.


Changes in v5
=

* Rebase on 357e81c.
* Split from the previous (larger and more complex) v4 series.


Changes in v4
=

* Fix typo in commit message (Stefan Hajnoczi).
* Simplify per-vCPU tracing state initialization (Stefan Hajnoczi).
* Update copyright years.


Changes in v3
=

* Update QAPI version (Eric Blake).
* Fix '#optional' annotation in QAPI (Eric Blake).


Changes in v2
=

* Rebase on 5522a84.
* Improve patch descriptions.
* Refactor code generation into a separate patch.
* Fix forward declarations (Stefan Hajnoczi & Eduardo Habkost).
* Fix "since" tags in QAPI interface (Eric Blake).
* Unify QAPI/QMP interface with an optional 'vcpu' argument (Eric Blake).
* Fix QMP+GTK header workaround (Stefan Hajnoczi).


Signed-off-by: Lluís Vilanova 
Acked-by: Stefan Hajnoczi 
---

Lluís Vilanova (4):
  trace: Extend API to manage event arguments
  trace: Remove unnecessary intermediate event copies
  tcg: Add type for vCPU pointers
  trace: Add 'vcpu' event property to trace guest vCPU


 docs/tracing.txt |   41 +
 include/qemu/typedefs.h  |1 
 scripts/tracetool/__init__.py|   31 --
 scripts/tracetool/format/events_h.py |4 +
 scripts/tracetool/format/h.py|3 +
 scripts/tracetool/format/tcg_h.py|   33 +++
 scripts/tracetool/format/tcg_helper_c.py |   35 +--
 scripts/tracetool/format/tcg_helper_h.py |7 +-
 scripts/tracetool/format/tcg_helper_wrapper_h.py |5 +-
 scripts/tracetool/format/ust_events_c.py |1 
 scripts/tracetool/transform.py   |4 +
 scripts/tracetool/vcpu.py|   69 ++
 target-alpha/translate.c |2 -
 target-arm/translate.c   |2 -
 target-arm/translate.h   |2 -
 target-cris/translate.c  |2 -
 target-i386/translate.c  |2 -
 target-lm32/translate.c  |2 -
 target-m68k/translate.c  |2 -
 target-microblaze/translate.c|2 -
 target-mips/translate.c  |2 -
 target-moxie/translate.c |2 -
 target-openrisc/translate.c  |2 -
 target-ppc/translate.c   |2 -
 target-s390x/translate.c |2 -
 target-sh4/translate.c   |2 -
 target-sparc/translate.c |5 +-
 target-tilegx/translate.c|2 -
 target-tricore/translate.c   |2 -
 target-unicore32/translate.c |2 -
 target-xtensa/translate.c|2 -
 tcg/tcg.h|1 
 trace/control.h  |3 +
 33 files changed, 224 insertions(+), 55 deletions(-)
 create mode 100644 scripts/tracetool/vcpu.py


To: qemu-devel@nongnu.org
Cc: Stefan Hajnoczi 
Cc: Eduardo Habkost 
Cc: Eric Blake 
Cc: Alex Bennée 

[Qemu-devel] [PATCH v7 2/4] trace: Remove unnecessary intermediate event copies

[Lluís Vilanova]

The current code forces the use of a chain of ".original" dereferences,
which looks odd.

Signed-off-by: Lluís Vilanova 
---
 scripts/tracetool/__init__.py|5 ++---
 scripts/tracetool/format/events_h.py |4 ++--
 scripts/tracetool/format/tcg_h.py|4 ++--
 3 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/scripts/tracetool/__init__.py b/scripts/tracetool/__init__.py
index 0663e7f..26878f4 100644
--- a/scripts/tracetool/__init__.py
+++ b/scripts/tracetool/__init__.py
@@ -6,7 +6,7 @@ Machinery for generating tracing-related intermediate files.
 """
 
 __author__ = "Lluís Vilanova "
-__copyright__  = "Copyright 2012-2014, Lluís Vilanova "
+__copyright__  = "Copyright 2012-2016, Lluís Vilanova "
 __license__= "GPL version 2 or (at your option) any later version"
 
 __maintainer__ = "Stefan Hajnoczi"
@@ -288,13 +288,12 @@ def _read_events(fobj):
 if atrans == aorig:
 args_trans.append(atrans)
 event_trans.args = Arguments(args_trans)
-event_trans = event_trans.copy()
 
 event_exec = event.copy()
 event_exec.name += "_exec"
 event_exec.properties += ["tcg-exec"]
 event_exec.fmt = event.fmt[1]
-event_exec = event_exec.transform(tracetool.transform.TCG_2_HOST)
+event_exec.args = 
event_exec.args.transform(tracetool.transform.TCG_2_HOST)
 
 new_event = [event_trans, event_exec]
 event.event_trans, event.event_exec = new_event
diff --git a/scripts/tracetool/format/events_h.py 
b/scripts/tracetool/format/events_h.py
index 9f114a3..bbfaa5b 100644
--- a/scripts/tracetool/format/events_h.py
+++ b/scripts/tracetool/format/events_h.py
@@ -6,7 +6,7 @@ trace/generated-events.h
 """
 
 __author__ = "Lluís Vilanova "
-__copyright__  = "Copyright 2012-2014, Lluís Vilanova "
+__copyright__  = "Copyright 2012-2016, Lluís Vilanova "
 __license__= "GPL version 2 or (at your option) any later version"
 
 __maintainer__ = "Stefan Hajnoczi"
@@ -43,7 +43,7 @@ def generate(events, backend):
 if "tcg-trans" in e.properties:
 # a single define for the two "sub-events"
 out('#define TRACE_%(name)s_ENABLED %(enabled)d',
-name=e.original.original.name.upper(),
+name=e.original.name.upper(),
 enabled=enabled)
 out('#define TRACE_%s_ENABLED %d' % (e.name.upper(), enabled))
 
diff --git a/scripts/tracetool/format/tcg_h.py 
b/scripts/tracetool/format/tcg_h.py
index f676b66..0d2cf79 100644
--- a/scripts/tracetool/format/tcg_h.py
+++ b/scripts/tracetool/format/tcg_h.py
@@ -6,7 +6,7 @@ Generate .h file for TCG code generation.
 """
 
 __author__ = "Lluís Vilanova "
-__copyright__  = "Copyright 2012-2014, Lluís Vilanova "
+__copyright__  = "Copyright 2012-2016, Lluís Vilanova "
 __license__= "GPL version 2 or (at your option) any later version"
 
 __maintainer__ = "Stefan Hajnoczi"
@@ -36,7 +36,7 @@ def generate(events, backend):
 continue
 
 # get the original event definition
-e = e.original.original
+e = e.original
 
 out('static inline void %(name_tcg)s(%(args)s)',
 '{',

[Qemu-devel] [PATCH v7 1/4] trace: Extend API to manage event arguments

[Lluís Vilanova]

Lets the user to manage event arguments as a list, and simplifies
argument concatenation.

Signed-off-by: Lluís Vilanova 
---
 scripts/tracetool/__init__.py |   15 +--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/scripts/tracetool/__init__.py b/scripts/tracetool/__init__.py
index 181675f..0663e7f 100644
--- a/scripts/tracetool/__init__.py
+++ b/scripts/tracetool/__init__.py
@@ -50,9 +50,14 @@ class Arguments:
 Parameters
 --
 args :
-List of (type, name) tuples.
+List of (type, name) tuples or Arguments objects.
 """
-self._args = args
+self._args = []
+for arg in args:
+if isinstance(arg, Arguments):
+self._args.extend(arg._args)
+else:
+self._args.append(arg)
 
 def copy(self):
 """Create a new copy."""
@@ -83,6 +88,12 @@ class Arguments:
 res.append((arg_type, identifier))
 return Arguments(res)
 
+def __getitem__(self, index):
+if isinstance(index, slice):
+return Arguments(self._args[index])
+else:
+return self._args[index]
+
 def __iter__(self):
 """Iterate over the (type, name) pairs."""
 return iter(self._args)