[Qemu-devel] [PATCH v11 21/28] qapi: Convert qtype_code into qapi enum type
What's more meta than using qapi to define qapi? :) Convert qtype_code into a full-fledged[*] builtin qapi enum type, so that a subsequent patch can then use it as the discriminator type of qapi alternate types. Doing so is easiest when renaming it to qapi conventions, as QTypeCode. Fortunately, there are not many places in the tree that were actually spelling the type name out, and the judicious use of 'prefix' in the qapi defintion avoids churn to the spelling of the enum constants. To avoid circular definitions, we have to flip the order of inclusion between "qobject.h" vs. "qapi-types.h". Back in commit 28770e0, we had the latter include the former, so that we could use 'QObject *' for our implementation of 'any'. But that usage also works with only a forward declaration, whereas the definition of QType requires QTypeCode to be a complete type. [*] The type has to be builtin, rather than declared in qapi/common.json, because we want to use it for alternates even when common.json is not included. But since it is the first builtin enum type, we have to add special cases to qapi-types and qapi-visit to only emit definitions once, even when two qapi files are being compiled into the same binary (the way we already handled builtin list types like 'intList'). We may need to revisit how multiple qapi files share common types, but that's a project for another day. Signed-off-by: Eric Blake --- v11: new patch --- block/qapi.c | 4 ++-- docs/qapi-code-gen.txt | 1 + include/hw/qdev-core.h | 2 +- include/qapi/qmp/qobject.h | 19 +++ qobject/qdict.c | 2 +- scripts/qapi-types.py| 13 ++--- scripts/qapi-visit.py| 10 -- scripts/qapi.py | 7 ++- tests/qapi-schema/alternate-empty.out| 2 ++ tests/qapi-schema/comments.out | 2 ++ tests/qapi-schema/empty.out | 2 ++ tests/qapi-schema/event-case.out | 2 ++ tests/qapi-schema/flat-union-empty.out | 2 ++ tests/qapi-schema/ident-with-escape.out | 2 ++ tests/qapi-schema/include-relpath.out| 2 ++ tests/qapi-schema/include-repetition.out | 2 ++ tests/qapi-schema/include-simple.out | 2 ++ tests/qapi-schema/indented-expr.out | 2 ++ tests/qapi-schema/qapi-schema-test.out | 2 ++ tests/qapi-schema/union-clash-data.out | 2 ++ tests/qapi-schema/union-empty.out| 2 ++ 21 files changed, 58 insertions(+), 26 deletions(-) diff --git a/block/qapi.c b/block/qapi.c index ec0f513..4211f11 100644 --- a/block/qapi.c +++ b/block/qapi.c @@ -539,7 +539,7 @@ static void dump_qlist(fprintf_function func_fprintf, void *f, int indentation, int i = 0; for (entry = qlist_first(list); entry; entry = qlist_next(entry), i++) { -qtype_code type = qobject_type(entry->value); +QTypeCode type = qobject_type(entry->value); bool composite = (type == QTYPE_QDICT || type == QTYPE_QLIST); const char *format = composite ? "%*s[%i]:\n" : "%*s[%i]: "; @@ -557,7 +557,7 @@ static void dump_qdict(fprintf_function func_fprintf, void *f, int indentation, const QDictEntry *entry; for (entry = qdict_first(dict); entry; entry = qdict_next(dict, entry)) { -qtype_code type = qobject_type(entry->value); +QTypeCode type = qobject_type(entry->value); bool composite = (type == QTYPE_QDICT || type == QTYPE_QLIST); const char *format = composite ? "%*s%s:\n" : "%*s%s: "; char key[strlen(entry->key) + 1]; diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt index 54a6a7b..35301c5 100644 --- a/docs/qapi-code-gen.txt +++ b/docs/qapi-code-gen.txt @@ -163,6 +163,7 @@ The following types are predefined, and map to C as follows: accepts size suffixes bool bool JSON true or false any QObject * any JSON value + QTypeCode QTypeCode JSON string of enum QTypeCode values === Includes === diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h index 8057aed..8e7df8e 100644 --- a/include/hw/qdev-core.h +++ b/include/hw/qdev-core.h @@ -239,7 +239,7 @@ struct Property { PropertyInfo *info; int offset; uint8_t bitnr; -qtype_code qtype; +QTypeCodeqtype; int64_t defval; int arrayoffset; PropertyInfo *arrayinfo; diff --git a/include/qapi/qmp/qobject.h b/include/qapi/qmp/qobject.h index 4b96ed5..8d6322b 100644 --- a/include/qapi/qmp/qobject.h +++ b/include/qapi/qmp/qobject.h @@ -34,23 +34,10 @@ #include #include - -typedef enum { -QTYPE_NONE,/* sentinel value, no QObject has this type code */ -QTYPE_QNULL, -QTYPE_QINT, -QTYPE_QSTRING, -QTYPE_QDICT, -QTYPE_QLIST, -QTYPE_QFLOAT, -QTYPE_QBOOL, -QTYPE_MAX, -} qtype_code; - -struct QObject; +#include "qapi-types.h" typedef st
Re: [Qemu-devel] [PATCH v11 21/28] qapi: Convert qtype_code into qapi enum type
Eric Blake writes: > What's more meta than using qapi to define qapi? :) > > Convert qtype_code into a full-fledged[*] builtin qapi enum type, > so that a subsequent patch can then use it as the discriminator > type of qapi alternate types. Doing so is easiest when renaming > it to qapi conventions, as QTypeCode. Out of curiosity: why does the rename make the conversion easier? If we rename anyway, what about renaming to QType? Hmm, we burned that on a struct we use only internally in qobject/. Oh well. >Fortunately, there are not > many places in the tree that were actually spelling the type name > out, and the judicious use of 'prefix' in the qapi defintion definition > avoids churn to the spelling of the enum constants. > > To avoid circular definitions, we have to flip the order of > inclusion between "qobject.h" vs. "qapi-types.h". Back in commit > 28770e0, we had the latter include the former, so that we could > use 'QObject *' for our implementation of 'any'. But that usage > also works with only a forward declaration, whereas the > definition of QType requires QTypeCode to be a complete type. > > [*] The type has to be builtin, rather than declared in > qapi/common.json, because we want to use it for alternates even > when common.json is not included. But since it is the first > builtin enum type, we have to add special cases to qapi-types > and qapi-visit to only emit definitions once, even when two > qapi files are being compiled into the same binary (the way we > already handled builtin list types like 'intList'). We may > need to revisit how multiple qapi files share common types, > but that's a project for another day. > > Signed-off-by: Eric Blake > > --- > v11: new patch > --- > block/qapi.c | 4 ++-- > docs/qapi-code-gen.txt | 1 + > include/hw/qdev-core.h | 2 +- > include/qapi/qmp/qobject.h | 19 +++ > qobject/qdict.c | 2 +- > scripts/qapi-types.py| 13 ++--- > scripts/qapi-visit.py| 10 -- > scripts/qapi.py | 7 ++- > tests/qapi-schema/alternate-empty.out| 2 ++ > tests/qapi-schema/comments.out | 2 ++ > tests/qapi-schema/empty.out | 2 ++ > tests/qapi-schema/event-case.out | 2 ++ > tests/qapi-schema/flat-union-empty.out | 2 ++ > tests/qapi-schema/ident-with-escape.out | 2 ++ > tests/qapi-schema/include-relpath.out| 2 ++ > tests/qapi-schema/include-repetition.out | 2 ++ > tests/qapi-schema/include-simple.out | 2 ++ > tests/qapi-schema/indented-expr.out | 2 ++ > tests/qapi-schema/qapi-schema-test.out | 2 ++ > tests/qapi-schema/union-clash-data.out | 2 ++ > tests/qapi-schema/union-empty.out| 2 ++ > 21 files changed, 58 insertions(+), 26 deletions(-) > > diff --git a/block/qapi.c b/block/qapi.c > index ec0f513..4211f11 100644 > --- a/block/qapi.c > +++ b/block/qapi.c > @@ -539,7 +539,7 @@ static void dump_qlist(fprintf_function func_fprintf, > void *f, int indentation, > int i = 0; > > for (entry = qlist_first(list); entry; entry = qlist_next(entry), i++) { > -qtype_code type = qobject_type(entry->value); > +QTypeCode type = qobject_type(entry->value); > bool composite = (type == QTYPE_QDICT || type == QTYPE_QLIST); > const char *format = composite ? "%*s[%i]:\n" : "%*s[%i]: "; > > @@ -557,7 +557,7 @@ static void dump_qdict(fprintf_function func_fprintf, > void *f, int indentation, > const QDictEntry *entry; > > for (entry = qdict_first(dict); entry; entry = qdict_next(dict, entry)) { > -qtype_code type = qobject_type(entry->value); > +QTypeCode type = qobject_type(entry->value); > bool composite = (type == QTYPE_QDICT || type == QTYPE_QLIST); > const char *format = composite ? "%*s%s:\n" : "%*s%s: "; > char key[strlen(entry->key) + 1]; > diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt > index 54a6a7b..35301c5 100644 > --- a/docs/qapi-code-gen.txt > +++ b/docs/qapi-code-gen.txt > @@ -163,6 +163,7 @@ The following types are predefined, and map to C as > follows: > accepts size suffixes >bool bool JSON true or false >any QObject * any JSON value > + QTypeCode QTypeCode JSON string of enum QTypeCode values QTypeCode is currently used only internally, so the JSON values don't matter. I don't expect that to change. However, we either enforce internal use somehow, or document the JSON values. Documenting them is easier. In short, your patch is fine. > > > === Includes === > diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h > index 8057aed..8e7df8e 100644 > --- a/include/hw/qdev-core.h > +++ b/include/hw/qdev-core.h > @@ -239,7 +239,7 @@ struct Property { > PropertyInf
Re: [Qemu-devel] [PATCH v11 21/28] qapi: Convert qtype_code into qapi enum type
On 11/11/2015 09:42 AM, Markus Armbruster wrote: > Eric Blake writes: > >> What's more meta than using qapi to define qapi? :) >> >> Convert qtype_code into a full-fledged[*] builtin qapi enum type, >> so that a subsequent patch can then use it as the discriminator >> type of qapi alternate types. Doing so is easiest when renaming >> it to qapi conventions, as QTypeCode. > > Out of curiosity: why does the rename make the conversion easier? It guarantees I found all affected instances. (Although I guess the rename could be split to a separate patch from making it builtin). It makes sure that if we later tighten rules about naming, we won't have to whitelist 'qtype_code' as an anomaly to our conventions. > > If we rename anyway, what about renaming to QType? Hmm, we burned that > on a struct we use only internally in qobject/. Oh well. Internal structs are often easy to rename. So if we want to avoid the need for 'prefix', I could certainly try to achieve that (move internal QType out of the way, then rename qtype_code to QType, then make QType the builtin). Looks like this one patch just became three :) > >>Fortunately, there are not >> many places in the tree that were actually spelling the type name >> out, and the judicious use of 'prefix' in the qapi defintion > > definition I've got to quit coding late at night - my rate of typos increases :) >> +++ b/docs/qapi-code-gen.txt >> @@ -163,6 +163,7 @@ The following types are predefined, and map to C as >> follows: >> accepts size suffixes >>bool bool JSON true or false >>any QObject * any JSON value >> + QTypeCode QTypeCode JSON string of enum QTypeCode values > > QTypeCode is currently used only internally, so the JSON values don't > matter. I don't expect that to change. However, we either enforce > internal use somehow, or document the JSON values. Documenting them is > easier. > > In short, your patch is fine. > >> - >> -struct QObject; >> +#include "qapi-types.h" >> >> typedef struct QType { >> -qtype_code code; >> +QTypeCode code; >> void (*destroy)(struct QObject *); >> } QType; >> >typedef struct QObject { >const QType *type; >size_t refcnt; >} QObject; > > Note: typedef name QObject still defined here. Oh, I see what you're saying. Since qapi-types.h now has a forward declaration of the QObject typedef, this could be changed to just struct QObject { ... }; >> +++ b/scripts/qapi-types.py >> @@ -233,8 +233,14 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): >> self.defn += gen_type_cleanup(name) >> >> def visit_enum_type(self, name, info, values, prefix): >> -self._fwdecl += gen_enum(name, values, prefix) >> -self._fwdefn += gen_enum_lookup(name, values, prefix) >> +# Special case for our lone builtin enum type >> +if name == 'QTypeCode': > > Would "if not info" work? Same in qapi-visit.py below. Feels a bit hacky, since we just recently added is_implicit() to hide (and then change) the 'if not info' check on objects. Maybe an accessor is_builtin() makes more sense? But yes, same approach to both client files. >> -#include "qapi/qmp/qobject.h" >> + >> +typedef struct QObject QObject; > > Typedef name QObject now also defined here. GCC accepts this silently > without -Wpedantic, but other compilers might not. Whether we care for > such compilers or not, defining things in exactly one place is neater. > > Possible fixes: > > * Drop the typedef from qobject.h > > * Don't add it to qapi-types.h, and use struct QObject there > I favor dropping the second typedef. >> +++ b/scripts/qapi.py >> @@ -33,7 +33,7 @@ builtin_types = { >> 'uint32': 'QTYPE_QINT', >> 'uint64': 'QTYPE_QINT', >> 'size': 'QTYPE_QINT', >> -'any': None, # any qtype_code possible, actually >> +'any': None, # any QTypeCode possible, actually >> } >> > > Should we list QTypeCode here? Yeah, probably. This array is only used by the ad hoc parser, and may disappear later as we move more into check(), but we should be consistent in the meantime. > >> # Whitelist of commands allowed to return a non-dictionary >> @@ -1243,6 +1243,11 @@ class QAPISchema(object): >> self.the_empty_object_type = QAPISchemaObjectType(':empty', None, >> None, >>[], None) >> self._def_entity(self.the_empty_object_type) >> +self._def_entity(QAPISchemaEnumType('QTypeCode', None, >> +['none', 'qnull', 'qint', >> + 'qstring', 'qdict', 'qlist', >> + 'qfloat', 'qbool'], >> +'QTYPE')) >> >> def _make_implicit_enum_type(self, name, info, values): >> name = name +
Re: [Qemu-devel] [PATCH v11 21/28] qapi: Convert qtype_code into qapi enum type
Eric Blake writes: > On 11/11/2015 09:42 AM, Markus Armbruster wrote: >> Eric Blake writes: >> >>> What's more meta than using qapi to define qapi? :) >>> >>> Convert qtype_code into a full-fledged[*] builtin qapi enum type, >>> so that a subsequent patch can then use it as the discriminator >>> type of qapi alternate types. Doing so is easiest when renaming >>> it to qapi conventions, as QTypeCode. >> >> Out of curiosity: why does the rename make the conversion easier? > > It guarantees I found all affected instances. (Although I guess the > rename could be split to a separate patch from making it builtin). Well, you have to find them only because you rename, don't you? > It makes sure that if we later tighten rules about naming, we won't have > to whitelist 'qtype_code' as an anomaly to our conventions. Good point. >> If we rename anyway, what about renaming to QType? Hmm, we burned that >> on a struct we use only internally in qobject/. Oh well. > > Internal structs are often easy to rename. So if we want to avoid the > need for 'prefix', I could certainly try to achieve that (move internal > QType out of the way, then rename qtype_code to QType, then make QType > the builtin). Looks like this one patch just became three :) Not sure it's worth the bother; the patch is okay as it is. QType is overkill. Instead of typedef struct QType { qtype_code code; void (*destroy)(struct QObject *); } QType; typedef struct QObject { const QType *type; size_t refcnt; } QObject; we could simply have typedef struct QObject { QTypeCode type; size_t refcnt; } QObject; with an array mapping QTypeCode to destroy methods. We're not going to define additional types at run time. Perhaps such a change would be actually worth the bother. >>>Fortunately, there are not >>> many places in the tree that were actually spelling the type name >>> out, and the judicious use of 'prefix' in the qapi defintion >> >> definition > > I've got to quit coding late at night - my rate of typos increases :) > >>> +++ b/docs/qapi-code-gen.txt >>> @@ -163,6 +163,7 @@ The following types are predefined, and map to C as >>> follows: >>> accepts size suffixes >>>bool bool JSON true or false >>>any QObject * any JSON value >>> + QTypeCode QTypeCode JSON string of enum QTypeCode values >> >> QTypeCode is currently used only internally, so the JSON values don't >> matter. I don't expect that to change. However, we either enforce >> internal use somehow, or document the JSON values. Documenting them is >> easier. >> >> In short, your patch is fine. >> > >>> - >>> -struct QObject; >>> +#include "qapi-types.h" >>> >>> typedef struct QType { >>> -qtype_code code; >>> +QTypeCode code; >>> void (*destroy)(struct QObject *); >>> } QType; >>> >>typedef struct QObject { >>const QType *type; >>size_t refcnt; >>} QObject; >> >> Note: typedef name QObject still defined here. > > Oh, I see what you're saying. Since qapi-types.h now has a forward > declaration of the QObject typedef, this could be changed to just > > struct QObject { > ... > }; > >>> +++ b/scripts/qapi-types.py >>> @@ -233,8 +233,14 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor): >>> self.defn += gen_type_cleanup(name) >>> >>> def visit_enum_type(self, name, info, values, prefix): >>> -self._fwdecl += gen_enum(name, values, prefix) >>> -self._fwdefn += gen_enum_lookup(name, values, prefix) >>> +# Special case for our lone builtin enum type >>> +if name == 'QTypeCode': >> >> Would "if not info" work? Same in qapi-visit.py below. > > Feels a bit hacky, since we just recently added is_implicit() to hide > (and then change) the 'if not info' check on objects. Maybe an accessor > is_builtin() makes more sense? But yes, same approach to both client files. QAPISchemaEntity methods like is_implicit() or a new is_builtin() can't work here, because we lack the entity. We have one in visit_needed(), and we use its is_implicit() to skip implicit object types. We could use entity.is_builtin() to skip (some) builtins, and handle them elsewhere, but that doesn't feel like an improvement over your code. Let's take a step back and reconsider how we do builtins. >> +self._btin += gen_enum(name, values, prefix) >> +if do_builtins: >> +self.defn += gen_enum_lookup(name, values, prefix) >> +else: >> +self._fwdecl += gen_enum(name, values, prefix) >> +self._fwdefn += gen_enum_lookup(name, values, prefix) >> >> def visit_array_type(self, name, info, element_type): >> if isinstance(element_type, QAPISchemaBuiltinType): Linking generated code from multiple schemata that share names may fail, because multiple definitions of the same external symbol
Re: [Qemu-devel] [PATCH v11 21/28] qapi: Convert qtype_code into qapi enum type
On 11/12/2015 06:16 AM, Markus Armbruster wrote: > We have one in visit_needed(), and we use its is_implicit() to skip > implicit object types. We could use entity.is_builtin() to skip (some) > builtins, and handle them elsewhere, but that doesn't feel like an > improvement over your code. > > Let's take a step back and reconsider how we do builtins. > >>> +self._btin += gen_enum(name, values, prefix) >>> +if do_builtins: >>> +self.defn += gen_enum_lookup(name, values, prefix) >>> +else: >>> +self._fwdecl += gen_enum(name, values, prefix) >>> +self._fwdefn += gen_enum_lookup(name, values, prefix) >>> >>> def visit_array_type(self, name, info, element_type): >>> if isinstance(element_type, QAPISchemaBuiltinType): > > Linking generated code from multiple schemata that share names may fail, > because multiple definitions of the same external symbol exist. > > Example: two schemata both define enum BadIdea. Both generate const > char *BadIdea_lookup[] = { ... }, and we end up with two global symbols > BadIdea_lookup. > > Solution: don't do that then. Easy enough, except *all* schemata share > the builtin symbols! Solution: [...] Yes, the existing solution is a bit ugly. We don't even need the secondary test-qapi-types.h to declare builtins, since it is relying on the top-level qapi-types.h to do it, so we are just wasting disk space. > Here's an alternative solution that permits slightly code simpler > generator code, and thus avoids the need to know: > > * Generate code for builtins exactly the same as for any other entities, > i.e. get rid of self._btin and the ifdeffery. > > * If the program links just one generated schema, this just works. > > * If the program links multiple generated schemata, the programmer has > to ensure their definitions get generated just once, and their > declarations are available everywhere anyway. Straightforward method: > > - The programmer suppresses builtins *completely* for *all* schemata. > The obvious way to suppress them is to filter them out in > visit_needed(). > > - Instead, he generates them once for the *empty* schema, with a > well-known --prefix. > > - Suppressing builtins generates a suitable #include for the > well-known .h with the builtin declarations. > > - Additionally link the .c containing the builtin definitions. Reasonably nice ideas. I'll play with them later, though, in the interest of getting the existing patch queue flushed first. > > Alternatively, trade some ease-of-use for the single schema case for > ease-of-use for the multiple schemata case and fewer cases: > > * The generators either generate for a schema, or they generate builtins. > > * When they generate builtins, they always use well-known file names. > > * When they generate for a schema, they always generate the #include for > the well-known builtin .h. They never generate builtins. I'd probably go with this variant (that is, set up the makefiles to run once with -b to generate builtins, and multiple times (one per .json) without -b to generate non-builtins. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature