Re: Integrating QOM into QAPI
> On 28 Jan 2020, at 11:03, Daniel P. Berrangé wrote: > > On Mon, Jan 27, 2020 at 08:05:36PM +0100, Christophe de Dinechin wrote: >> >> >>> On 26 Jan 2020, at 16:04, Peter Maydell wrote: >>> >>> On Sun, 26 Jan 2020 at 08:10, Christophe de Dinechin >>> wrote: I’m still puzzled as to why anybody would switch to something like GObject when there is C++. >>> >>> I'm fairly strongly against using C++. >> >> Just to be clear, so am I ;-) >> >>> C++'s language design >>> is an "everything including the kitchen sink, lots of "this >>> is here for back compat but it's a bear trap", lots of new >>> stuff arriving all the time. >> >> Actually, the new stuff is not that bad, overall. >> >> I do agree C++ is an overly complicated language, and that in >> practice, there is zero chance of qemu moving to it. But that does >> not invalidate my point that creating a class in C++ is easier >> than creating a class in any C-based macro-heavy reinvention >> of basic OO concepts. >> >> (I write this after having read Paolo’s response, which points >> out IMO better reasons for GObject, which I will discuss there). >> >>> It's just too big to keep in >>> your head all at once. C has its faults, absolutely, but at >>> least it tries to be a reasonably sized vaguely coherent >>> language. >>> >>> You'd have more luck persuading me we should move to Rust: >>> at least then we'd get some clear benefits (no more buffer >>> overrun security bugs) for the upheaval :-) >> >> This is largely a myth as soon as you need to do “your own stuff”. >> Example: CVE-2019-18960, https://seclists.org/oss-sec/2019/q4/141. > > Calling it a "myth" from from that one data point is not really credible. A single failure is enough to credibly counter any “no more X” claim ;-) Also, I carefully qualified that as for “your own stuff”. IOW, if you write your own buffer management, Rust does not help. Well, dealing with guest memory forces us to have our own buffer management. > > No language is perfect & such that it can eliminate all possible CVEs. > Rust *can*, however, eliminate a very large set of bugs which lead to > memory corruption in unchecked languages like C/C++. The Rust hype today eerily reminds me so strongly of the C++ hype 20 years ago. C++ too was supposed to be a safer C. And indeed, if you stick to the standard library, you _do_ eliminate a very large set of bugs which lead to memory corruption, leaks, etc in C. So the claim C++ made was not wrong. The claim Rust makes is not wrong either. The false sense of safety that some people get from it is. > You'll still have > CVEs to deal with, but they'll be different classes of bugs, or rare > edge cases like the one you show above. How exactly was it an “edge case”? From what I saw, it was the most basic kind of range check, exactly the kind that you see in 99.9% of all range checks. > > Regards, > Daniel > -- > |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o-https://fstop138.berrange.com :| > |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| >
Re: Integrating QOM into QAPI
On Mon, Jan 27, 2020 at 08:05:36PM +0100, Christophe de Dinechin wrote: > > > > On 26 Jan 2020, at 16:04, Peter Maydell wrote: > > > > On Sun, 26 Jan 2020 at 08:10, Christophe de Dinechin > > wrote: > >> I’m still puzzled as to why anybody would switch to something like > >> GObject when there is C++. > > > > I'm fairly strongly against using C++. > > Just to be clear, so am I ;-) > > > C++'s language design > > is an "everything including the kitchen sink, lots of "this > > is here for back compat but it's a bear trap", lots of new > > stuff arriving all the time. > > Actually, the new stuff is not that bad, overall. > > I do agree C++ is an overly complicated language, and that in > practice, there is zero chance of qemu moving to it. But that does > not invalidate my point that creating a class in C++ is easier > than creating a class in any C-based macro-heavy reinvention > of basic OO concepts. > > (I write this after having read Paolo’s response, which points > out IMO better reasons for GObject, which I will discuss there). > > > It's just too big to keep in > > your head all at once. C has its faults, absolutely, but at > > least it tries to be a reasonably sized vaguely coherent > > language. > > > > You'd have more luck persuading me we should move to Rust: > > at least then we'd get some clear benefits (no more buffer > > overrun security bugs) for the upheaval :-) > > This is largely a myth as soon as you need to do “your own stuff”. > Example: CVE-2019-18960, https://seclists.org/oss-sec/2019/q4/141. Calling it a "myth" from from that one data point is not really credible. No language is perfect & such that it can eliminate all possible CVEs. Rust *can*, however, eliminate a very large set of bugs which lead to memory corruption in unchecked languages like C/C++. You'll still have CVEs to deal with, but they'll be different classes of bugs, or rare edge cases like the one you show above. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: Integrating QOM into QAPI
Christophe de Dinechin writes: >> On 26 Jan 2020, at 16:04, Peter Maydell wrote: >> >> On Sun, 26 Jan 2020 at 08:10, Christophe de Dinechin >> wrote: [...] >> You'd have more luck persuading me we should move to Rust: >> at least then we'd get some clear benefits (no more buffer >> overrun security bugs) for the upheaval :-) > > This is largely a myth as soon as you need to do “your own stuff”. > Example: CVE-2019-18960, https://seclists.org/oss-sec/2019/q4/141. I think "largely a myth" is too harsh. Yes, it's not a silver bullet to insta-slay all memory and concurrency safety vampires. It does provide useful guarantees, though. How useful exactly in practice time will tell.
Re: Integrating QOM into QAPI
> On 26 Jan 2020, at 16:04, Peter Maydell wrote: > > On Sun, 26 Jan 2020 at 08:10, Christophe de Dinechin > wrote: >> I’m still puzzled as to why anybody would switch to something like >> GObject when there is C++. > > I'm fairly strongly against using C++. Just to be clear, so am I ;-) > C++'s language design > is an "everything including the kitchen sink, lots of "this > is here for back compat but it's a bear trap", lots of new > stuff arriving all the time. Actually, the new stuff is not that bad, overall. I do agree C++ is an overly complicated language, and that in practice, there is zero chance of qemu moving to it. But that does not invalidate my point that creating a class in C++ is easier than creating a class in any C-based macro-heavy reinvention of basic OO concepts. (I write this after having read Paolo’s response, which points out IMO better reasons for GObject, which I will discuss there). > It's just too big to keep in > your head all at once. C has its faults, absolutely, but at > least it tries to be a reasonably sized vaguely coherent > language. > > You'd have more luck persuading me we should move to Rust: > at least then we'd get some clear benefits (no more buffer > overrun security bugs) for the upheaval :-) This is largely a myth as soon as you need to do “your own stuff”. Example: CVE-2019-18960, https://seclists.org/oss-sec/2019/q4/141. > > thanks > -- PMM >
Re: Integrating QOM into QAPI
> On 26 Jan 2020, at 10:11, Marc-André Lureau
> wrote:
>
> Hi
>
> On Sun, Jan 26, 2020 at 9:10 AM Christophe de Dinechin
> wrote:
>>
>>
>>
>>> On 21 Jan 2020, at 16:11, Marc-André Lureau
>>> wrote:
>>>
>>> Hi
>>>
>>> On Tue, Jan 21, 2020 at 7:01 PM Markus Armbruster wrote:
Daniel P. Berrangé writes:
> On Tue, Jan 21, 2020 at 02:36:17PM +0100, Markus Armbruster wrote:
>> Marc-André Lureau writes:
>>
>>> Hi
>>>
>>> On Tue, Jan 21, 2020 at 3:32 PM Stefan Hajnoczi
>>> wrote:
On Tue, Jan 21, 2020 at 06:42:47AM +0100, Markus Armbruster wrote:
> Stefan Hajnoczi writes:
>
>> On Wed, Jan 15, 2020 at 01:15:17PM +0100, Markus Armbruster wrote:
>>> Christophe de Dinechin writes:
> On 15 Jan 2020, at 10:20, Markus Armbruster
> wrote:
>>> * qemuMonitorJSONSetIOThread() uses it to control iothread's
>>> properties
>>> poll-max-ns, poll-grow, poll-shrink. Their use with -object is
>>> documented (in qemu-options.hx), their use with qom-set is not.
>>
>> I'm happy to use a different interface.
>>
>> Writing a boilerplate "iothread-set-poll-params" QMP command in C
>> would
>> be a step backwards.
>
> No argument.
>
>> Maybe the QAPI code generator could map something like this:
>>
>> { 'command': 'iothread-set-poll-params',
>> 'data': {
>> 'id': 'str',
>> '*max-ns': 'uint64',
>> '*grow': 'uint64',
>> '*shrink': 'uint64'
>> },
>> 'map-to-qom-set': 'IOThread'
>> }
>>
>> And turn it into QOM accessors on the IOThread object.
>
> I think a generic "set this configuration to that value" command is
> just
> fine. qom-set fails on several counts, though:
>
> * Tolerable: qom-set is not actually generic, it applies only to QOM.
>
> * qom-set lets you set tons of stuff that is not meant to be changed
> at
> run time. If it breaks your guest, you get to keep the pieces.
>
> * There is virtually no documentation on what can be set to what
> values,
> and their semantics.
>
> In its current state, QOM is a user interface superfund site.
Thoughts about a solution:
Static QOM properties should be declared via QAPI instead of
imperatively via QOM APIs. That way they are introspectable and type
information is present in the schema.
The QAPI code generator could emit a function that is callable from
.class_init(). This eliminates the need to manually call
object_class_property_add().
>>
>> We need to make up our minds what exactly we want generated. Then we
>> can design the QAPI language, and code up the generator.
>>
>> Skeleton QOM type, to help with the discussion:
>>
>> #define TYPE_FOO "foo"
>>
>> #define FOO(obj) OBJECT_CHECK(Foo, (obj), TYPE_FOO)
>> #define FOO_CLASS(klass) \
>> OBJECT_CLASS_CHECK(FooClass, (klass), TYPE_FOO)
>> #define FOO_GET_CLASS(obj) \
>> OBJECT_GET_CLASS(FooClass, (obj), TYPE_FOO)
>>
>> typedef FooClass {
>> ParentClass parent_class;
>> ... // hand-written per-class state
>> }
>>
>> struct Chardev {
>> ParentObject parent_obj;
>> ... // hand-written instance (per-object) state
>> };
>>
>> static const TypeInfo char_type_info = {
>> .name = TYPE_FOO,
>> .parent = TYPE_OBJECT,
>> .instance_size = sizeof(Foo),
>> .instance_init = ..., // methods to initialize
>> .instance_post_init = ..., // and finalize instances,
>> .instance_finalize = ..., // all optional
>> .abstract = ...,// true or false (d'oh)
>> .class_size = sizeof(FooClass),
>> .class_init = ..., // methods to initialize
>> .class_base_init = ..., // classes, optional
>> .class_data = ..., // extra argument for them
>> .interfaces = ...
>> };
>>
>> There's substantial boilerplate, with plenty of hand-written code in the
>> gaps. What of the boilerplate do we plan to generate? How do we plan
>> to fill the gaps, if any?
>
> FWIW, even without a QOM generator, we can do waaay better on the
> amount of boilerplate needed for QOM without very much work. It just
> needs a few convenience macros writing.
>
> QOM is not GObject, but is heavily inspired by it and so looking at
> GObject gives us
Re: Integrating QOM into QAPI
> On 26 Jan 2020, at 17:47, Paolo Bonzini wrote:
>
> On 26/01/20 10:11, Marc-André Lureau wrote:
>>> I’m still puzzled as to why anybody would switch to something like
>>> GObject when there is C++.
>> C++ is another level of complexity.
>>
>> Replacing QOM with GObject would mainly bring us a more solid type
>> system with better tooling/features, gobject-introspection support,
>> and remove the burden of having our own OO from QEMU code base.
>
> In fact, C++ doesn't solve any of the problems that either QOM or
> GObject try to solve. (Neither does Rust for that matter).
It does not solve all of them _easily_. I do believe that any solution
to these problems can be engineered in C++, if only because C++
is essentially a superset of C. The question is whether the result can
be made easier / safer to use and generally more elegant. I believe
that the answer is yes, see proof at end of mail.
However, before going further, glib offers way more than gobject.
So there’s that too… And I’m not saying migrating to C++ is a good
idea, I’m just trying to evaluate the various options fairly.
> Nevertheless, there is no stupid question, only stupid answers, and I
> think Christophe's remark is an example of a common misconception. In
> the hope of not making this a stupid answer, let my try to formulate
> succinctly what I think the differences are between QOM, GObject and the
> C++ object model:
Thank you for this remarkably non-stupid answer ;-) You have really
isolated, in very clear terms, the essence of the discussion.
>
> - the C++ object model (at least "old-style" C++ with virtual functions
> and the like) provides you with _the intersection_ of what QOM and
> GObject try to solve. This is what Marc-André calls "OO", and it's
> essentially virtual functions and dynamic casts. It's a relatively
> small part of both QOM and GObject, and unfortunately a wheel that
> almost every large C program ends up reinventing.
This was the part I was pointing to in my initial comment.
C++ solves that basic “OO” stuff well, and goes a little beyond QOM
or GObject in offering IMO many other benefits, e.g. wrt type safety.
>
> - Marc-André also described above what GObject provides: a fully
> introspectable type system and the tools so that _libraries_ can define
> _types that will be used from multiple programming languages_.
This kind of things has existed for a very long time. CORBA dates back
to 1991. Also, I’m not sure how important multiple programming languages
are for the qemu use-case. I believe that what matters more is remotely
accessible objects (e.g. over a socket), which in turn makes it almost
trivial to call from another language as long as you accept some kind
of serialization / deserialization process along the way.
GObject only seems marginally better for the “in same process” use case,
in the sense that it makes “objects” that can be used from any language,
indeed, but at the cost of being somewhat foreign and weird in all languages,
including C.
Look at the GClosure marshalling, for example, and compare it with the
example I give you at end of this email, and you tell me which one looks
easier and safer to use.
>
> - QOM also provides a fully introspectable type system, but with a
> different focus: it's so that _objects_ can expose _properties that will
> be accessed from multiple channels_.
Exposing the properties and making them introspectable are the
fundamental feature we need to discuss. So agreement here.
What you have not explained to my satisfaction
yet is how GObject is a better starting point for re-creating a new
externally-accessible API than some kind of wrapper built in C++
and taking advantage of modern C++ features.
> Everything else in both GObject and QOM follows from this core purpose,
> and the differences between the two follow from the differences. For
> example:
>
> - GObject's focus on multiple programming languages:
> gobject-introspection, GClosure, support for non-object types (scalar
> and GBoxed)
How much of that is actually useful to create a new usable qemu API?
>
> - QOM's focus on objects: dynamic properties, object tree, all types are
> classes
That, to me, looks fundamental, since not having it would require
a total re-architecture of the rest of qemu.
But it looks also somewhat trivial to implement in C++.
For example, obj[“id”] could return a PropertyAccessor
that lets you read or write the object knowing that it is of type
T, so you could write:
if (my_object[“id”] < 3) // Automatically checks the type
or
my_object[“id”] = 42;
The latter would call PropertyAccessor::operator=(int), which
in turn would check if property “id” exists in my_object, if it has type
“int”, and so on.
Implementation-wise, a simple std::map of BaseProperty pointers,
where each would be an instance of some
template
class Property : public BaseProperty
{
operator T(); // get
T&
Re: Integrating QOM into QAPI
On 26/01/20 10:11, Marc-André Lureau wrote: >> I’m still puzzled as to why anybody would switch to something like >> GObject when there is C++. > C++ is another level of complexity. > > Replacing QOM with GObject would mainly bring us a more solid type > system with better tooling/features, gobject-introspection support, > and remove the burden of having our own OO from QEMU code base. In fact, C++ doesn't solve any of the problems that either QOM or GObject try to solve. (Neither does Rust for that matter). Nevertheless, there is no stupid question, only stupid answers, and I think Christophe's remark is an example of a common misconception. In the hope of not making this a stupid answer, let my try to formulate succinctly what I think the differences are between QOM, GObject and the C++ object model: - the C++ object model (at least "old-style" C++ with virtual functions and the like) provides you with _the intersection_ of what QOM and GObject try to solve. This is what Marc-André calls "OO", and it's essentially virtual functions and dynamic casts. It's a relatively small part of both QOM and GObject, and unfortunately a wheel that almost every large C program ends up reinventing. - Marc-André also described above what GObject provides: a fully introspectable type system and the tools so that _libraries_ can define _types that will be used from multiple programming languages_. - QOM also provides a fully introspectable type system, but with a different focus: it's so that _objects_ can expose _properties that will be accessed from multiple channels_. Everything else in both GObject and QOM follows from this core purpose, and the differences between the two follow from the differences. For example: - GObject's focus on multiple programming languages: gobject-introspection, GClosure, support for non-object types (scalar and GBoxed) - QOM's focus on objects: dynamic properties, object tree, all types are classes - QOM's focus on properties: no introspection of methods - QOM's support for multiple channels: visitors Paolo
Re: Integrating QOM into QAPI
On Sun, 26 Jan 2020 at 08:10, Christophe de Dinechin wrote: > I’m still puzzled as to why anybody would switch to something like > GObject when there is C++. I'm fairly strongly against using C++. C++'s language design is an "everything including the kitchen sink, lots of "this is here for back compat but it's a bear trap", lots of new stuff arriving all the time. It's just too big to keep in your head all at once. C has its faults, absolutely, but at least it tries to be a reasonably sized vaguely coherent language. You'd have more luck persuading me we should move to Rust: at least then we'd get some clear benefits (no more buffer overrun security bugs) for the upheaval :-) thanks -- PMM
Re: Integrating QOM into QAPI
Hi
On Sun, Jan 26, 2020 at 9:10 AM Christophe de Dinechin
wrote:
>
>
>
> > On 21 Jan 2020, at 16:11, Marc-André Lureau
> > wrote:
> >
> > Hi
> >
> > On Tue, Jan 21, 2020 at 7:01 PM Markus Armbruster wrote:
> >>
> >> Daniel P. Berrangé writes:
> >>
> >>> On Tue, Jan 21, 2020 at 02:36:17PM +0100, Markus Armbruster wrote:
> Marc-André Lureau writes:
>
> > Hi
> >
> > On Tue, Jan 21, 2020 at 3:32 PM Stefan Hajnoczi
> > wrote:
> >>
> >> On Tue, Jan 21, 2020 at 06:42:47AM +0100, Markus Armbruster wrote:
> >>> Stefan Hajnoczi writes:
> >>>
> On Wed, Jan 15, 2020 at 01:15:17PM +0100, Markus Armbruster wrote:
> > Christophe de Dinechin writes:
> >>> On 15 Jan 2020, at 10:20, Markus Armbruster
> >>> wrote:
> > * qemuMonitorJSONSetIOThread() uses it to control iothread's
> > properties
> > poll-max-ns, poll-grow, poll-shrink. Their use with -object is
> > documented (in qemu-options.hx), their use with qom-set is not.
>
> I'm happy to use a different interface.
>
> Writing a boilerplate "iothread-set-poll-params" QMP command in C
> would
> be a step backwards.
> >>>
> >>> No argument.
> >>>
> Maybe the QAPI code generator could map something like this:
>
> { 'command': 'iothread-set-poll-params',
> 'data': {
> 'id': 'str',
> '*max-ns': 'uint64',
> '*grow': 'uint64',
> '*shrink': 'uint64'
> },
> 'map-to-qom-set': 'IOThread'
> }
>
> And turn it into QOM accessors on the IOThread object.
> >>>
> >>> I think a generic "set this configuration to that value" command is
> >>> just
> >>> fine. qom-set fails on several counts, though:
> >>>
> >>> * Tolerable: qom-set is not actually generic, it applies only to QOM.
> >>>
> >>> * qom-set lets you set tons of stuff that is not meant to be changed
> >>> at
> >>> run time. If it breaks your guest, you get to keep the pieces.
> >>>
> >>> * There is virtually no documentation on what can be set to what
> >>> values,
> >>> and their semantics.
> >>>
> >>> In its current state, QOM is a user interface superfund site.
> >>
> >> Thoughts about a solution:
> >>
> >> Static QOM properties should be declared via QAPI instead of
> >> imperatively via QOM APIs. That way they are introspectable and type
> >> information is present in the schema.
> >>
> >> The QAPI code generator could emit a function that is callable from
> >> .class_init(). This eliminates the need to manually call
> >> object_class_property_add().
>
> We need to make up our minds what exactly we want generated. Then we
> can design the QAPI language, and code up the generator.
>
> Skeleton QOM type, to help with the discussion:
>
> #define TYPE_FOO "foo"
>
> #define FOO(obj) OBJECT_CHECK(Foo, (obj), TYPE_FOO)
> #define FOO_CLASS(klass) \
> OBJECT_CLASS_CHECK(FooClass, (klass), TYPE_FOO)
> #define FOO_GET_CLASS(obj) \
> OBJECT_GET_CLASS(FooClass, (obj), TYPE_FOO)
>
> typedef FooClass {
> ParentClass parent_class;
> ... // hand-written per-class state
> }
>
> struct Chardev {
> ParentObject parent_obj;
> ... // hand-written instance (per-object) state
> };
>
> static const TypeInfo char_type_info = {
> .name = TYPE_FOO,
> .parent = TYPE_OBJECT,
> .instance_size = sizeof(Foo),
> .instance_init = ..., // methods to initialize
> .instance_post_init = ..., // and finalize instances,
> .instance_finalize = ..., // all optional
> .abstract = ...,// true or false (d'oh)
> .class_size = sizeof(FooClass),
> .class_init = ..., // methods to initialize
> .class_base_init = ..., // classes, optional
> .class_data = ..., // extra argument for them
> .interfaces = ...
> };
>
> There's substantial boilerplate, with plenty of hand-written code in the
> gaps. What of the boilerplate do we plan to generate? How do we plan
> to fill the gaps, if any?
> >>>
> >>> FWIW, even without a QOM generator, we can do waaay better on the
> >>> amount of boilerplate needed for QOM without very much work. It just
> >>> needs a few convenience macros writing.
> >>>
> >>> QOM is not GObject, but is heavily inspired by it and so looking at
> >>> GObject gives us a design pattern we can aim to match in terms of
> >>> amount of
Re: Integrating QOM into QAPI
> On 21 Jan 2020, at 16:11, Marc-André Lureau
> wrote:
>
> Hi
>
> On Tue, Jan 21, 2020 at 7:01 PM Markus Armbruster wrote:
>>
>> Daniel P. Berrangé writes:
>>
>>> On Tue, Jan 21, 2020 at 02:36:17PM +0100, Markus Armbruster wrote:
Marc-André Lureau writes:
> Hi
>
> On Tue, Jan 21, 2020 at 3:32 PM Stefan Hajnoczi
> wrote:
>>
>> On Tue, Jan 21, 2020 at 06:42:47AM +0100, Markus Armbruster wrote:
>>> Stefan Hajnoczi writes:
>>>
On Wed, Jan 15, 2020 at 01:15:17PM +0100, Markus Armbruster wrote:
> Christophe de Dinechin writes:
>>> On 15 Jan 2020, at 10:20, Markus Armbruster
>>> wrote:
> * qemuMonitorJSONSetIOThread() uses it to control iothread's
> properties
> poll-max-ns, poll-grow, poll-shrink. Their use with -object is
> documented (in qemu-options.hx), their use with qom-set is not.
I'm happy to use a different interface.
Writing a boilerplate "iothread-set-poll-params" QMP command in C would
be a step backwards.
>>>
>>> No argument.
>>>
Maybe the QAPI code generator could map something like this:
{ 'command': 'iothread-set-poll-params',
'data': {
'id': 'str',
'*max-ns': 'uint64',
'*grow': 'uint64',
'*shrink': 'uint64'
},
'map-to-qom-set': 'IOThread'
}
And turn it into QOM accessors on the IOThread object.
>>>
>>> I think a generic "set this configuration to that value" command is just
>>> fine. qom-set fails on several counts, though:
>>>
>>> * Tolerable: qom-set is not actually generic, it applies only to QOM.
>>>
>>> * qom-set lets you set tons of stuff that is not meant to be changed at
>>> run time. If it breaks your guest, you get to keep the pieces.
>>>
>>> * There is virtually no documentation on what can be set to what values,
>>> and their semantics.
>>>
>>> In its current state, QOM is a user interface superfund site.
>>
>> Thoughts about a solution:
>>
>> Static QOM properties should be declared via QAPI instead of
>> imperatively via QOM APIs. That way they are introspectable and type
>> information is present in the schema.
>>
>> The QAPI code generator could emit a function that is callable from
>> .class_init(). This eliminates the need to manually call
>> object_class_property_add().
We need to make up our minds what exactly we want generated. Then we
can design the QAPI language, and code up the generator.
Skeleton QOM type, to help with the discussion:
#define TYPE_FOO "foo"
#define FOO(obj) OBJECT_CHECK(Foo, (obj), TYPE_FOO)
#define FOO_CLASS(klass) \
OBJECT_CLASS_CHECK(FooClass, (klass), TYPE_FOO)
#define FOO_GET_CLASS(obj) \
OBJECT_GET_CLASS(FooClass, (obj), TYPE_FOO)
typedef FooClass {
ParentClass parent_class;
... // hand-written per-class state
}
struct Chardev {
ParentObject parent_obj;
... // hand-written instance (per-object) state
};
static const TypeInfo char_type_info = {
.name = TYPE_FOO,
.parent = TYPE_OBJECT,
.instance_size = sizeof(Foo),
.instance_init = ..., // methods to initialize
.instance_post_init = ..., // and finalize instances,
.instance_finalize = ..., // all optional
.abstract = ...,// true or false (d'oh)
.class_size = sizeof(FooClass),
.class_init = ..., // methods to initialize
.class_base_init = ..., // classes, optional
.class_data = ..., // extra argument for them
.interfaces = ...
};
There's substantial boilerplate, with plenty of hand-written code in the
gaps. What of the boilerplate do we plan to generate? How do we plan
to fill the gaps, if any?
>>>
>>> FWIW, even without a QOM generator, we can do waaay better on the
>>> amount of boilerplate needed for QOM without very much work. It just
>>> needs a few convenience macros writing.
>>>
>>> QOM is not GObject, but is heavily inspired by it and so looking at
>>> GObject gives us a design pattern we can aim to match in terms of
>>> amount of boilerplate.
>>>
>>> What we do manually with TypeInfo struct there has essentially always
>>> been done by a 1 line macro in GObject:
>>>
>>> G_DEFINE_TYPE(virIdentity, vir_identity, G_TYPE_OBJECT)
>>>
>>> If implementing interfaces, there's 1 extra line needed per interface
>>> to associate them.
>>>
>>>
>>>
Re: Integrating QOM into QAPI
On Sat, 25 Jan 2020 at 09:28, Paolo Bonzini wrote: > > On 25/01/20 05:44, Marc-André Lureau wrote: > > I try to find a good reason qom was chosen over gobject, and I can't > > find it. > > The main reasons were integration with QAPI, and the object tree. > Though everything I say here is a kind of reverse engineering of > Anthony's brain because there aren't really any design documents besides > what's in include/qom/object.h (and he overlooked some aspects, for > example "unparent" was introduced a few months later). I vagely recall that back at that time we were a lot less heavy in our usage of glib also, so "just use the glib version of whatever" would not have been quite as easy a sell as it might be today. Anthony's original RFC email lists some "key differences" between QOM and GObject, which presumably seemed to him at the time to be sufficient to justify not using GObject: https://lists.gnu.org/archive/html/qemu-devel/2011-07/msg01673.html thanks -- PMM
Re: Integrating QOM into QAPI
On 25/01/20 05:44, Marc-André Lureau wrote: > On 22/01/20 13:42, Marc-André Lureau wrote: > > From the top of my mind, this is the pain point when trying to use > GObject: > > - static/inlined object, not supported by GObject, unlikely to ever be > > - few users in qemu, transition possible. > > - 64k limit of GObject, for some reason, unlikely to change but I will > > take a look. Some users in qemu, code adaptation possible. > > - dynamic properties, possible in GObject with hacks, but not > > recommended and going to be deprecated from what I remember > > - "array" properties - would need extra layer/tweaks for compatibility > > - link properties - would need special handling > > - different limitations for type names and properties names > > The properties in general are very different between QOM and QAPI. They > have different limitations and features as Marc-André mentioned, but an > especially important one is the integration with QAPI visitors. This is > what allows us to support -object and object-add with the same code, and > is what separates QOM from GObject the most. > > Maybe it would be possible to build an adapter, but having written in > the past code that uses GType to do marshalling and unmarshalling, I'm > not really fond of repeating the experience... > > I agree it is one of the things that look very different from gobject. > At the same time, I think defining conventions/types or interface to > describe hierarchy isn't so difficult, and then adapting the visitors > shouldn't be either. Note that there are a few "structured" properties that export a QAPI struct rather than a scalar. Those would be a bit more complex. Links might also be tricky. Another small difference that came to mind is the different object liveness model; GObject is mostly plain old reference counting (plus the "floating reference" idea), while in QOM reference counting is secondary and lifetime is mostly dictated by the object tree and "unparenting". This is more of a conceptual difference; it could be easily retrofitted in GObject. > I try to find a good reason qom was chosen over gobject, and I can't > find it. The main reasons were integration with QAPI, and the object tree. Though everything I say here is a kind of reverse engineering of Anthony's brain because there aren't really any design documents besides what's in include/qom/object.h (and he overlooked some aspects, for example "unparent" was introduced a few months later). Overall I don't think there would be much benefit in reusing GObject. It's about 3k lines of code, quite a few would stay (those implementing the tree) and a bunch more would have to be rewritten. I don't think we'd have any use for most of the features that QOM lacks, such as signals. Also, QOM is quite well documented and we should include its documentation in docs/devel instead. That said, coccinell-ing efforts on QOM code are definitely a good idea since it's quite mature now and we know better what we need. And there are practical advantages too, it's not just code cleanups---see how your series shows the default value of the properties in -device foo,help. Paolo
Re: Integrating QOM into QAPI
Hi Le ven. 24 janv. 2020 à 19:32, Paolo Bonzini a écrit : > On 22/01/20 13:42, Marc-André Lureau wrote: > > From the top of my mind, this is the pain point when trying to use > GObject: > > - static/inlined object, not supported by GObject, unlikely to ever be > > - few users in qemu, transition possible. > > - 64k limit of GObject, for some reason, unlikely to change but I will > > take a look. Some users in qemu, code adaptation possible. > > - dynamic properties, possible in GObject with hacks, but not > > recommended and going to be deprecated from what I remember > > - "array" properties - would need extra layer/tweaks for compatibility > > - link properties - would need special handling > > - different limitations for type names and properties names > > The properties in general are very different between QOM and QAPI. They > have different limitations and features as Marc-André mentioned, but an > especially important one is the integration with QAPI visitors. This is > what allows us to support -object and object-add with the same code, and > is what separates QOM from GObject the most. > > Maybe it would be possible to build an adapter, but having written in > the past code that uses GType to do marshalling and unmarshalling, I'm > not really fond of repeating the experience... > I agree it is one of the things that look very different from gobject. At the same time, I think defining conventions/types or interface to describe hierarchy isn't so difficult, and then adapting the visitors shouldn't be either. I try to find a good reason qom was chosen over gobject, and I can't find it. >
Re: Integrating QOM into QAPI
On 22/01/20 13:42, Marc-André Lureau wrote: > From the top of my mind, this is the pain point when trying to use GObject: > - static/inlined object, not supported by GObject, unlikely to ever be > - few users in qemu, transition possible. > - 64k limit of GObject, for some reason, unlikely to change but I will > take a look. Some users in qemu, code adaptation possible. > - dynamic properties, possible in GObject with hacks, but not > recommended and going to be deprecated from what I remember > - "array" properties - would need extra layer/tweaks for compatibility > - link properties - would need special handling > - different limitations for type names and properties names The properties in general are very different between QOM and QAPI. They have different limitations and features as Marc-André mentioned, but an especially important one is the integration with QAPI visitors. This is what allows us to support -object and object-add with the same code, and is what separates QOM from GObject the most. Maybe it would be possible to build an adapter, but having written in the past code that uses GType to do marshalling and unmarshalling, I'm not really fond of repeating the experience... Paolo
Re: Integrating QOM into QAPI
Marc-André Lureau writes: > Hi > > On Wed, Jan 22, 2020 at 4:25 PM Markus Armbruster wrote: >> >> Alex Bennée writes: >> >> > Marc-André Lureau writes: >> >> Actually, we are not that far off from being able to use GObject >> >> altogether (I hacked something like that to play with), but I >> >> disgress... >> > >> > As a mostly hands off observer who mainly c's QOM code when he has to >> > I have to ask is this a long term plan? >> > >> > I've always found having our own hand rolled object system a little >> > incongruous given we lean heavily on the rest of glib. >> >> I vaguely remember claims that GObject falls short of our needs. Sadly, >> I don't remember the details. This is why major features should come >> with a design document. >> >> https://wiki.qemu.org/Features/QOM ain't: it does not mention GObject. >> I'm afraid that page has fallen too far behind the code to be useful to >> anyone not familiar with the code. > >>From the top of my mind, this is the pain point when trying to use GObject: > - static/inlined object, not supported by GObject, unlikely to ever be Lame. Okay for us as long as the pointer chasing stays off the hot paths. But that's not obvious. > - few users in qemu, transition possible. Peter challenged this idea. > - 64k limit of GObject, for some reason, unlikely to change but I will > take a look. Some users in qemu, code adaptation possible. Also lame. I figure code adaption will involve pointer chasing. Again, beware of hot paths. This one feels less likely to be a problem, though. > - dynamic properties, possible in GObject with hacks, but not > recommended and going to be deprecated from what I remember Never understood our need for them for longer than half a day after getting it explained to me. I guess that's my fault. Anyway, the less we use them, the better. Hacks exploiting deprecated GObject features would be a bad idea. Perhaps we can hack something up that doesn't. > - "array" properties - would need extra layer/tweaks for compatibility I hated them from day one, tried to kill them, lost the argument. > - link properties - would need special handling Hard or not? > - different limitations for type names and properties names What to do? Our naming rules and conventions are weakly documented. > A possible initial approach is to have all the type system and object > allocation done by GObject under the hood (what I hacked), while > keeping all the properties handled by QOM. Then, figure out a > migration to GObject properties (which are also being refactored a bit > upstream). So most QOM properties would be based on a GObject property, but some wouldn't be, correct? >If there is enough interest, I will keep investigating. But > for now, helping with meson seems more urgent. Yes. /me warily eyes the monster lurking in the review queue... [...]
Re: Integrating QOM into QAPI
Hi On Wed, Jan 22, 2020 at 5:28 PM Peter Maydell wrote: > > On Wed, 22 Jan 2020 at 12:42, Marc-André Lureau > wrote: > > From the top of my mind, this is the pain point when trying to use GObject: > > - static/inlined object, not supported by GObject, unlikely to ever be > > - few users in qemu, transition possible. > > Isn't there lots of use of this in the device emulation, or am I > misunderstanding what the restriction is? There are a bunch, from what I remember. Yes, that's a bit painful, but spatch managed to rewrite most of dev.subdev.foo to dev.subdev->foo iirc. -- Marc-André Lureau
Re: Integrating QOM into QAPI
On Wed, 22 Jan 2020 at 12:42, Marc-André Lureau wrote: > From the top of my mind, this is the pain point when trying to use GObject: > - static/inlined object, not supported by GObject, unlikely to ever be > - few users in qemu, transition possible. Isn't there lots of use of this in the device emulation, or am I misunderstanding what the restriction is? thanks -- PMM
Re: Integrating QOM into QAPI
Hi On Wed, Jan 22, 2020 at 4:25 PM Markus Armbruster wrote: > > Alex Bennée writes: > > > Marc-André Lureau writes: > >> Actually, we are not that far off from being able to use GObject > >> altogether (I hacked something like that to play with), but I > >> disgress... > > > > As a mostly hands off observer who mainly c's QOM code when he has to > > I have to ask is this a long term plan? > > > > I've always found having our own hand rolled object system a little > > incongruous given we lean heavily on the rest of glib. > > I vaguely remember claims that GObject falls short of our needs. Sadly, > I don't remember the details. This is why major features should come > with a design document. > > https://wiki.qemu.org/Features/QOM ain't: it does not mention GObject. > I'm afraid that page has fallen too far behind the code to be useful to > anyone not familiar with the code. >From the top of my mind, this is the pain point when trying to use GObject: - static/inlined object, not supported by GObject, unlikely to ever be - few users in qemu, transition possible. - 64k limit of GObject, for some reason, unlikely to change but I will take a look. Some users in qemu, code adaptation possible. - dynamic properties, possible in GObject with hacks, but not recommended and going to be deprecated from what I remember - "array" properties - would need extra layer/tweaks for compatibility - link properties - would need special handling - different limitations for type names and properties names A possible initial approach is to have all the type system and object allocation done by GObject under the hood (what I hacked), while keeping all the properties handled by QOM. Then, figure out a migration to GObject properties (which are also being refactored a bit upstream). If there is enough interest, I will keep investigating. But for now, helping with meson seems more urgent. > > >> So introducing GObject-like macros? sure! > >> > >> There are plenty of refactoring to do. The problem when touching the > >> whole code-base, imho, is review time. It may take a couple of > >> hours/days to come up with a cocci/spatch, and make various patches > >> here and there. But it takes often weeks and a lot of constant push to > >> various folks to get all the reviews (as seens by the qdev prop-ptr > >> series earlier for example). How can we better address whole code-base > >> changes? > > > > The problem with review time - especially for QOM - is having domain > > knowledge to understand what is happening. > > > > Are we happy that the existing qdev/qmp tests sufficiently exercise > > QEMU's object model? Perhaps with a little extra tweaking of the tests > > we could dump the object hierarchy and then compare it to the hierarchy > > presented after modification. That might make it easier to have > > confidence that these large scale but mostly mechanical changes don't > > change anything externally visible? > > Comparing the composition tree complete with properties and property > values before and after feels like a useful regression test. Any > takers? > -- Marc-André Lureau
Re: Integrating QOM into QAPI
Alex Bennée writes: > Marc-André Lureau writes: >> Actually, we are not that far off from being able to use GObject >> altogether (I hacked something like that to play with), but I >> disgress... > > As a mostly hands off observer who mainly c's QOM code when he has to > I have to ask is this a long term plan? > > I've always found having our own hand rolled object system a little > incongruous given we lean heavily on the rest of glib. I vaguely remember claims that GObject falls short of our needs. Sadly, I don't remember the details. This is why major features should come with a design document. https://wiki.qemu.org/Features/QOM ain't: it does not mention GObject. I'm afraid that page has fallen too far behind the code to be useful to anyone not familiar with the code. >> So introducing GObject-like macros? sure! >> >> There are plenty of refactoring to do. The problem when touching the >> whole code-base, imho, is review time. It may take a couple of >> hours/days to come up with a cocci/spatch, and make various patches >> here and there. But it takes often weeks and a lot of constant push to >> various folks to get all the reviews (as seens by the qdev prop-ptr >> series earlier for example). How can we better address whole code-base >> changes? > > The problem with review time - especially for QOM - is having domain > knowledge to understand what is happening. > > Are we happy that the existing qdev/qmp tests sufficiently exercise > QEMU's object model? Perhaps with a little extra tweaking of the tests > we could dump the object hierarchy and then compare it to the hierarchy > presented after modification. That might make it easier to have > confidence that these large scale but mostly mechanical changes don't > change anything externally visible? Comparing the composition tree complete with properties and property values before and after feels like a useful regression test. Any takers?
Re: Integrating QOM into QAPI
Marc-André Lureau writes:
> Hi
>
> On Tue, Jan 21, 2020 at 7:01 PM Markus Armbruster wrote:
>> Daniel P. Berrangé writes:
>> > On Tue, Jan 21, 2020 at 02:36:17PM +0100, Markus Armbruster wrote:
>> >> static const TypeInfo char_type_info = {
>> >> .name = TYPE_FOO,
>> >> .parent = TYPE_OBJECT,
>> >> .instance_size = sizeof(Foo),
>> >> .instance_init = ..., // methods to initialize
>> >> .instance_post_init = ..., // and finalize instances,
>> >> .instance_finalize = ..., // all optional
>> >> .abstract = ...,// true or false (d'oh)
>> >> .class_size = sizeof(FooClass),
>> >> .class_init = ..., // methods to initialize
>> >> .class_base_init = ..., // classes, optional
>> >> .class_data = ..., // extra argument for them
>> >> .interfaces = ...
>> >> };
>> >>
>> >> There's substantial boilerplate, with plenty of hand-written code in the
>> >> gaps. What of the boilerplate do we plan to generate? How do we plan
>> >> to fill the gaps, if any?
>> >
>> > FWIW, even without a QOM generator, we can do waaay better on the
>> > amount of boilerplate needed for QOM without very much work. It just
>> > needs a few convenience macros writing.
>> >
>> > QOM is not GObject, but is heavily inspired by it and so looking at
>> > GObject gives us a design pattern we can aim to match in terms of
>> > amount of boilerplate.
>> >
>> > What we do manually with TypeInfo struct there has essentially always
>> > been done by a 1 line macro in GObject:
>> >
>> > G_DEFINE_TYPE(virIdentity, vir_identity, G_TYPE_OBJECT)
>> >
>> > If implementing interfaces, there's 1 extra line needed per interface
>> > to associate them.
>> >
>> >
>> > https://developer.gnome.org/gobject/stable/gobject-Type-Information.html#G-DEFINE-TYPE:CAPS
>> >
>> >
>> > And what we do in the header file to add the 4 or more FOO_XXX macros,
>> > and the class struct and the object struct has recently been turned
>> > into a 2-liner:
>> >
>> > #define VIR_TYPE_IDENTITY vir_identity_get_type()
>> > G_DECLARE_FINAL_TYPE(virIdentity, vir_identity, VIR, IDENTITY, GObject);
>> >
>> >
>> > https://developer.gnome.org/gobject/stable/gobject-Type-Information.html#G-DECLARE-FINAL-TYPE:CAPS
>> >
>> > Or
>> >
>> > #define VIR_TYPE_IDENTITY vir_identity_get_type()
>> > G_DECLARE_DERIVABLE_TYPE(virIdentity, vir_identity, VIR, IDENTITY,
>> > GObject);
>> >
>> >
>> > https://developer.gnome.org/gobject/stable/gobject-Type-Information.html#G-DECLARE-DERIVABLE-TYPE:CAPS
>> >
>> >
>> > It would be nice to have a QOM code generator so that we can statically
>> > declare properties & parent/child/interface relationships, but for an
>> > immediate low cost win, better macros would be very useful IMHO.
>>
>> Volunteers?
>>
>
> Actually, we are not that far off from being able to use GObject
> altogether (I hacked something like that to play with), but I
> disgress...
As a mostly hands off observer who mainly c's QOM code when he has to
I have to ask is this a long term plan?
I've always found having our own hand rolled object system a little
incongruous given we lean heavily on the rest of glib.
> So introducing GObject-like macros? sure!
>
> There are plenty of refactoring to do. The problem when touching the
> whole code-base, imho, is review time. It may take a couple of
> hours/days to come up with a cocci/spatch, and make various patches
> here and there. But it takes often weeks and a lot of constant push to
> various folks to get all the reviews (as seens by the qdev prop-ptr
> series earlier for example). How can we better address whole code-base
> changes?
The problem with review time - especially for QOM - is having domain
knowledge to understand what is happening.
Are we happy that the existing qdev/qmp tests sufficiently exercise
QEMU's object model? Perhaps with a little extra tweaking of the tests
we could dump the object hierarchy and then compare it to the hierarchy
presented after modification. That might make it easier to have
confidence that these large scale but mostly mechanical changes don't
change anything externally visible?
--
Alex Bennée
Re: Integrating QOM into QAPI
On Tue, 21 Jan 2020 at 15:11, Marc-André Lureau wrote: > There are plenty of refactoring to do. The problem when touching the > whole code-base, imho, is review time. It may take a couple of > hours/days to come up with a cocci/spatch, and make various patches > here and there. But it takes often weeks and a lot of constant push to > various folks to get all the reviews (as seens by the qdev prop-ptr > series earlier for example). How can we better address whole code-base > changes? It depends. If it's literally just a cocci/spatch mechanical transformation, I think we should be OK with reviewing that transform and then applying it; we don't need to get acks from every submaintainer for that kind of thing. The prop-ptr series was harder work because it required making active non-mechanical changes to various devices, which means the specific changes needed review. PS: thanks for pushing that prop-ptr work through and especially the serial device part which was both awkward and definitely needed. thanks -- PMM
Re: Integrating QOM into QAPI
Hi
On Tue, Jan 21, 2020 at 7:01 PM Markus Armbruster wrote:
>
> Daniel P. Berrangé writes:
>
> > On Tue, Jan 21, 2020 at 02:36:17PM +0100, Markus Armbruster wrote:
> >> Marc-André Lureau writes:
> >>
> >> > Hi
> >> >
> >> > On Tue, Jan 21, 2020 at 3:32 PM Stefan Hajnoczi
> >> > wrote:
> >> >>
> >> >> On Tue, Jan 21, 2020 at 06:42:47AM +0100, Markus Armbruster wrote:
> >> >> > Stefan Hajnoczi writes:
> >> >> >
> >> >> > > On Wed, Jan 15, 2020 at 01:15:17PM +0100, Markus Armbruster wrote:
> >> >> > >> Christophe de Dinechin writes:
> >> >> > >> >> On 15 Jan 2020, at 10:20, Markus Armbruster
> >> >> > >> >> wrote:
> >> >> > >> * qemuMonitorJSONSetIOThread() uses it to control iothread's
> >> >> > >> properties
> >> >> > >> poll-max-ns, poll-grow, poll-shrink. Their use with -object is
> >> >> > >> documented (in qemu-options.hx), their use with qom-set is not.
> >> >> > >
> >> >> > > I'm happy to use a different interface.
> >> >> > >
> >> >> > > Writing a boilerplate "iothread-set-poll-params" QMP command in C
> >> >> > > would
> >> >> > > be a step backwards.
> >> >> >
> >> >> > No argument.
> >> >> >
> >> >> > > Maybe the QAPI code generator could map something like this:
> >> >> > >
> >> >> > > { 'command': 'iothread-set-poll-params',
> >> >> > > 'data': {
> >> >> > > 'id': 'str',
> >> >> > > '*max-ns': 'uint64',
> >> >> > > '*grow': 'uint64',
> >> >> > > '*shrink': 'uint64'
> >> >> > > },
> >> >> > > 'map-to-qom-set': 'IOThread'
> >> >> > > }
> >> >> > >
> >> >> > > And turn it into QOM accessors on the IOThread object.
> >> >> >
> >> >> > I think a generic "set this configuration to that value" command is
> >> >> > just
> >> >> > fine. qom-set fails on several counts, though:
> >> >> >
> >> >> > * Tolerable: qom-set is not actually generic, it applies only to QOM.
> >> >> >
> >> >> > * qom-set lets you set tons of stuff that is not meant to be changed
> >> >> > at
> >> >> > run time. If it breaks your guest, you get to keep the pieces.
> >> >> >
> >> >> > * There is virtually no documentation on what can be set to what
> >> >> > values,
> >> >> > and their semantics.
> >> >> >
> >> >> > In its current state, QOM is a user interface superfund site.
> >> >>
> >> >> Thoughts about a solution:
> >> >>
> >> >> Static QOM properties should be declared via QAPI instead of
> >> >> imperatively via QOM APIs. That way they are introspectable and type
> >> >> information is present in the schema.
> >> >>
> >> >> The QAPI code generator could emit a function that is callable from
> >> >> .class_init(). This eliminates the need to manually call
> >> >> object_class_property_add().
> >>
> >> We need to make up our minds what exactly we want generated. Then we
> >> can design the QAPI language, and code up the generator.
> >>
> >> Skeleton QOM type, to help with the discussion:
> >>
> >> #define TYPE_FOO "foo"
> >>
> >> #define FOO(obj) OBJECT_CHECK(Foo, (obj), TYPE_FOO)
> >> #define FOO_CLASS(klass) \
> >> OBJECT_CLASS_CHECK(FooClass, (klass), TYPE_FOO)
> >> #define FOO_GET_CLASS(obj) \
> >> OBJECT_GET_CLASS(FooClass, (obj), TYPE_FOO)
> >>
> >> typedef FooClass {
> >> ParentClass parent_class;
> >> ... // hand-written per-class state
> >> }
> >>
> >> struct Chardev {
> >> ParentObject parent_obj;
> >> ... // hand-written instance (per-object) state
> >> };
> >>
> >> static const TypeInfo char_type_info = {
> >> .name = TYPE_FOO,
> >> .parent = TYPE_OBJECT,
> >> .instance_size = sizeof(Foo),
> >> .instance_init = ..., // methods to initialize
> >> .instance_post_init = ..., // and finalize instances,
> >> .instance_finalize = ..., // all optional
> >> .abstract = ...,// true or false (d'oh)
> >> .class_size = sizeof(FooClass),
> >> .class_init = ..., // methods to initialize
> >> .class_base_init = ..., // classes, optional
> >> .class_data = ..., // extra argument for them
> >> .interfaces = ...
> >> };
> >>
> >> There's substantial boilerplate, with plenty of hand-written code in the
> >> gaps. What of the boilerplate do we plan to generate? How do we plan
> >> to fill the gaps, if any?
> >
> > FWIW, even without a QOM generator, we can do waaay better on the
> > amount of boilerplate needed for QOM without very much work. It just
> > needs a few convenience macros writing.
> >
> > QOM is not GObject, but is heavily inspired by it and so looking at
> > GObject gives us a design pattern we can aim to match in terms of
> > amount of boilerplate.
> >
> > What we do manually with TypeInfo struct there has essentially always
> > been done by a 1 line macro in GObject:
> >
> > G_DEFINE_TYPE(virIdentity, vir_identity, G_TYPE_OBJECT)
> >
> > If
Re: Integrating QOM into QAPI
Daniel P. Berrangé writes:
> On Tue, Jan 21, 2020 at 02:36:17PM +0100, Markus Armbruster wrote:
>> Marc-André Lureau writes:
>>
>> > Hi
>> >
>> > On Tue, Jan 21, 2020 at 3:32 PM Stefan Hajnoczi wrote:
>> >>
>> >> On Tue, Jan 21, 2020 at 06:42:47AM +0100, Markus Armbruster wrote:
>> >> > Stefan Hajnoczi writes:
>> >> >
>> >> > > On Wed, Jan 15, 2020 at 01:15:17PM +0100, Markus Armbruster wrote:
>> >> > >> Christophe de Dinechin writes:
>> >> > >> >> On 15 Jan 2020, at 10:20, Markus Armbruster
>> >> > >> >> wrote:
>> >> > >> * qemuMonitorJSONSetIOThread() uses it to control iothread's
>> >> > >> properties
>> >> > >> poll-max-ns, poll-grow, poll-shrink. Their use with -object is
>> >> > >> documented (in qemu-options.hx), their use with qom-set is not.
>> >> > >
>> >> > > I'm happy to use a different interface.
>> >> > >
>> >> > > Writing a boilerplate "iothread-set-poll-params" QMP command in C
>> >> > > would
>> >> > > be a step backwards.
>> >> >
>> >> > No argument.
>> >> >
>> >> > > Maybe the QAPI code generator could map something like this:
>> >> > >
>> >> > > { 'command': 'iothread-set-poll-params',
>> >> > > 'data': {
>> >> > > 'id': 'str',
>> >> > > '*max-ns': 'uint64',
>> >> > > '*grow': 'uint64',
>> >> > > '*shrink': 'uint64'
>> >> > > },
>> >> > > 'map-to-qom-set': 'IOThread'
>> >> > > }
>> >> > >
>> >> > > And turn it into QOM accessors on the IOThread object.
>> >> >
>> >> > I think a generic "set this configuration to that value" command is just
>> >> > fine. qom-set fails on several counts, though:
>> >> >
>> >> > * Tolerable: qom-set is not actually generic, it applies only to QOM.
>> >> >
>> >> > * qom-set lets you set tons of stuff that is not meant to be changed at
>> >> > run time. If it breaks your guest, you get to keep the pieces.
>> >> >
>> >> > * There is virtually no documentation on what can be set to what values,
>> >> > and their semantics.
>> >> >
>> >> > In its current state, QOM is a user interface superfund site.
>> >>
>> >> Thoughts about a solution:
>> >>
>> >> Static QOM properties should be declared via QAPI instead of
>> >> imperatively via QOM APIs. That way they are introspectable and type
>> >> information is present in the schema.
>> >>
>> >> The QAPI code generator could emit a function that is callable from
>> >> .class_init(). This eliminates the need to manually call
>> >> object_class_property_add().
>>
>> We need to make up our minds what exactly we want generated. Then we
>> can design the QAPI language, and code up the generator.
>>
>> Skeleton QOM type, to help with the discussion:
>>
>> #define TYPE_FOO "foo"
>>
>> #define FOO(obj) OBJECT_CHECK(Foo, (obj), TYPE_FOO)
>> #define FOO_CLASS(klass) \
>> OBJECT_CLASS_CHECK(FooClass, (klass), TYPE_FOO)
>> #define FOO_GET_CLASS(obj) \
>> OBJECT_GET_CLASS(FooClass, (obj), TYPE_FOO)
>>
>> typedef FooClass {
>> ParentClass parent_class;
>> ... // hand-written per-class state
>> }
>>
>> struct Chardev {
>> ParentObject parent_obj;
>> ... // hand-written instance (per-object) state
>> };
>>
>> static const TypeInfo char_type_info = {
>> .name = TYPE_FOO,
>> .parent = TYPE_OBJECT,
>> .instance_size = sizeof(Foo),
>> .instance_init = ..., // methods to initialize
>> .instance_post_init = ..., // and finalize instances,
>> .instance_finalize = ..., // all optional
>> .abstract = ...,// true or false (d'oh)
>> .class_size = sizeof(FooClass),
>> .class_init = ..., // methods to initialize
>> .class_base_init = ..., // classes, optional
>> .class_data = ..., // extra argument for them
>> .interfaces = ...
>> };
>>
>> There's substantial boilerplate, with plenty of hand-written code in the
>> gaps. What of the boilerplate do we plan to generate? How do we plan
>> to fill the gaps, if any?
>
> FWIW, even without a QOM generator, we can do waaay better on the
> amount of boilerplate needed for QOM without very much work. It just
> needs a few convenience macros writing.
>
> QOM is not GObject, but is heavily inspired by it and so looking at
> GObject gives us a design pattern we can aim to match in terms of
> amount of boilerplate.
>
> What we do manually with TypeInfo struct there has essentially always
> been done by a 1 line macro in GObject:
>
> G_DEFINE_TYPE(virIdentity, vir_identity, G_TYPE_OBJECT)
>
> If implementing interfaces, there's 1 extra line needed per interface
> to associate them.
>
>
> https://developer.gnome.org/gobject/stable/gobject-Type-Information.html#G-DEFINE-TYPE:CAPS
>
>
> And what we do in the header file to add the 4 or more FOO_XXX macros,
> and the class struct and the object struct has recently been turned
>
Re: Integrating QOM into QAPI (was: Making QEMU easier for management tools and applications)
On Tue, Jan 21, 2020 at 02:36:17PM +0100, Markus Armbruster wrote:
> Marc-André Lureau writes:
>
> > Hi
> >
> > On Tue, Jan 21, 2020 at 3:32 PM Stefan Hajnoczi wrote:
> >>
> >> On Tue, Jan 21, 2020 at 06:42:47AM +0100, Markus Armbruster wrote:
> >> > Stefan Hajnoczi writes:
> >> >
> >> > > On Wed, Jan 15, 2020 at 01:15:17PM +0100, Markus Armbruster wrote:
> >> > >> Christophe de Dinechin writes:
> >> > >> >> On 15 Jan 2020, at 10:20, Markus Armbruster
> >> > >> >> wrote:
> >> > >> * qemuMonitorJSONSetIOThread() uses it to control iothread's
> >> > >> properties
> >> > >> poll-max-ns, poll-grow, poll-shrink. Their use with -object is
> >> > >> documented (in qemu-options.hx), their use with qom-set is not.
> >> > >
> >> > > I'm happy to use a different interface.
> >> > >
> >> > > Writing a boilerplate "iothread-set-poll-params" QMP command in C would
> >> > > be a step backwards.
> >> >
> >> > No argument.
> >> >
> >> > > Maybe the QAPI code generator could map something like this:
> >> > >
> >> > > { 'command': 'iothread-set-poll-params',
> >> > > 'data': {
> >> > > 'id': 'str',
> >> > > '*max-ns': 'uint64',
> >> > > '*grow': 'uint64',
> >> > > '*shrink': 'uint64'
> >> > > },
> >> > > 'map-to-qom-set': 'IOThread'
> >> > > }
> >> > >
> >> > > And turn it into QOM accessors on the IOThread object.
> >> >
> >> > I think a generic "set this configuration to that value" command is just
> >> > fine. qom-set fails on several counts, though:
> >> >
> >> > * Tolerable: qom-set is not actually generic, it applies only to QOM.
> >> >
> >> > * qom-set lets you set tons of stuff that is not meant to be changed at
> >> > run time. If it breaks your guest, you get to keep the pieces.
> >> >
> >> > * There is virtually no documentation on what can be set to what values,
> >> > and their semantics.
> >> >
> >> > In its current state, QOM is a user interface superfund site.
> >>
> >> Thoughts about a solution:
> >>
> >> Static QOM properties should be declared via QAPI instead of
> >> imperatively via QOM APIs. That way they are introspectable and type
> >> information is present in the schema.
> >>
> >> The QAPI code generator could emit a function that is callable from
> >> .class_init(). This eliminates the need to manually call
> >> object_class_property_add().
>
> We need to make up our minds what exactly we want generated. Then we
> can design the QAPI language, and code up the generator.
>
> Skeleton QOM type, to help with the discussion:
>
> #define TYPE_FOO "foo"
>
> #define FOO(obj) OBJECT_CHECK(Foo, (obj), TYPE_FOO)
> #define FOO_CLASS(klass) \
> OBJECT_CLASS_CHECK(FooClass, (klass), TYPE_FOO)
> #define FOO_GET_CLASS(obj) \
> OBJECT_GET_CLASS(FooClass, (obj), TYPE_FOO)
>
> typedef FooClass {
> ParentClass parent_class;
> ... // hand-written per-class state
> }
>
> struct Chardev {
> ParentObject parent_obj;
> ... // hand-written instance (per-object) state
> };
>
> static const TypeInfo char_type_info = {
> .name = TYPE_FOO,
> .parent = TYPE_OBJECT,
> .instance_size = sizeof(Foo),
> .instance_init = ..., // methods to initialize
> .instance_post_init = ..., // and finalize instances,
> .instance_finalize = ..., // all optional
> .abstract = ...,// true or false (d'oh)
> .class_size = sizeof(FooClass),
> .class_init = ..., // methods to initialize
> .class_base_init = ..., // classes, optional
> .class_data = ..., // extra argument for them
> .interfaces = ...
> };
>
> There's substantial boilerplate, with plenty of hand-written code in the
> gaps. What of the boilerplate do we plan to generate? How do we plan
> to fill the gaps, if any?
FWIW, even without a QOM generator, we can do waaay better on the
amount of boilerplate needed for QOM without very much work. It just
needs a few convenience macros writing.
QOM is not GObject, but is heavily inspired by it and so looking at
GObject gives us a design pattern we can aim to match in terms of
amount of boilerplate.
What we do manually with TypeInfo struct there has essentially always
been done by a 1 line macro in GObject:
G_DEFINE_TYPE(virIdentity, vir_identity, G_TYPE_OBJECT)
If implementing interfaces, there's 1 extra line needed per interface
to associate them.
https://developer.gnome.org/gobject/stable/gobject-Type-Information.html#G-DEFINE-TYPE:CAPS
And what we do in the header file to add the 4 or more FOO_XXX macros,
and the class struct and the object struct has recently been turned
into a 2-liner:
#define VIR_TYPE_IDENTITY vir_identity_get_type()
G_DECLARE_FINAL_TYPE(virIdentity, vir_identity, VIR, IDENTITY, GObject);
