On Fri, Apr 09, 2010 at 09:41:39AM -0400, Chris Lalancette wrote:
> Hello,
> In response to a lot of the talk of qemud lately on qemu-devel, the
> libvirt community would like to put forward a proposal to help enable
> debug/advanced options when using various hypervisors. The goals of
> this API are:
>
> 1) To enable more rapid access to hypervisor features before proper
> libvirt API's are designed around them.
> 2) To facilitate debugging and access to advanced features that may
> not fit into the normal libvirt world-view.
>
> Caveats:
> 1) Unlike other libvirt API's, this one will explicitly *not* be
> guaranteed ABI/API compatible between libvirt updates.
I think we'd still aim to keep the public API stable. The bit that we
can't guarentee is the interactions with QEMU & the libvirt driver.
eg, if someone was using the API to send text monitor commands to
QEMU, and the next libvirt release switched to JSON mode, that use
would break. The API would still be valid, but the way they use it
might not be. Similarly if they add some custom extra command line
argument, this could potentially conflict with an extra command line
arg a subsquent libvirt release used. eg, they used -device to add
a PCI card with a specific PCI address. Then next libvirt release
specifies this same PCI address and then you get a clash.
So in both cases the API and XML format can be reasonably guarenteed
between releases. What we can't guarentee is that usage of these
features will be reliable across releases of libvirt.
> 2) Again unlike other libvirt API's, access and configuration of
> the debug section of a domain will be highly hypervisor dependent.
> 3) Application developers will be strongly discouraged from using
> this API because of the above 2 issues. To help in this, the
> API's will be in a separate library that developers will explicitly
> have to link to, and it will have a different (but largely compatible)
> wire protocol.
In terms of wire protocol, this would still have to run over the same
existing data channel, because we don't want apps to have to open up
multiple connections. Fortunately our protocol is designed to allow
this kind of extension
struct remote_message_header {
unsigned prog; /* REMOTE_PROGRAM */
unsigned vers; /* REMOTE_PROTOCOL_VERSION */
remote_procedure proc; /* REMOTE_PROC_x */
remote_message_type type;
unsigned serial; /* Serial number of message. */
remote_message_status status;
};
We currently have a single program 'REMOTE_PROGRAM' and its associated
'vers' and 'proc' numbers. What we would be doing is to define a new
program, say QEMU_PROGRAM. The 'remote_procedure proc' numbers can thus
start again from '1' without clashing with any of the existing APIs.
All the RPC marshalling/demarshalling code should work more or less
unchanged. We just hook in a different function dispatch table in the
daemon for that program.
> 4) We don't expect this API to solve all of the issues brought up
> during the qemud discussion. Our initial goal is just to give
> ready access of the qemu command-line and monitor to developers.
>
> With that being said, our initial proposal follows. We expect this
> to evolve over time as we get more feedback, but we think this
> proposal addresses at least 2 of the major pain points qemu developers
> have while trying to use libvirt.
>
> The initial debug XML for qemu would be:
I wouldn't call them 'debug' options, since that's just one of possible
use cases for this. What we're really doing there is exposing hypervisor
specific features, so we should just be upfront about that in our
description / naming.
>
> <domain type='kvm'>
> <name>myguest</name>
> ...
> <debug>
> <monitorpassthrough/>
> <commandline>
> <extra>qemu arguments</extra>
> <alter option="optname">
> <rename>newname</rename>
> <match>REGEXP</match>
> <modify>foo=on</modify>
> <extra>-bar</extra>
> </alter>
> </commandline>
> </debug>
> </domain>
The concept of command line & monitor is something that is QEMU specific
and thus is not suitable for the primary XML schema. IMHO, this needs to be
done as a separate schema, linked in via an XML namespace. For example
<domain type='kvm' xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0";>
<name>myguest</name>
...
<qemu:commandline>
<qemu:arg>-device</arg>
<qemu:arg>lsi</arg>
</qemu:commandline>
</domain>
>
> Raw access to the qemu monitor will be disabled by default; the
> <monitorpassthrough/> tag enables the ability to send QMP (or
> text, if you are using older qemu) messages straight through to the
> monitor. To do this there will be an additional API entry point
> named virDomainDebugCommand() which takes an arbitrary string
> and passes it to the monitor, and returns an arbitrary string as
> a result. Thus you could pass in either "info cpus" if using the
> text monitor or '{ "execute": "query-cpus" }' if using QMP.
Again the idea of a 'virDomainDebugCommand' API is QEMU specific, with
other hypervisors have different approaches for low level extension/
debug. For example, Xen would involve XenStore access, or XenD XMLRPC,
etc. So this should really live in a separate API namespace which is
specific to a hypervisor. For example, as a header file
#include <libvirt/libvirt-qemu.h>
Containing APIs like
int virDomainQEMUInvokeMonitor(virDomainPtr dom,
const char *command,
char **reply);
typedef virConnectQEMUDomainEventCallback(virConnectPtr conn,
virDomainPtr dom,
const char *eventname,
const char *data,
void *opaque)
int virConnectQEMUDomainEventRegister(virConnectPtr conn,
virDomainPtr dom,
const char *eventname,
virDomainQEMUMonitorCallback cb,
void *opaque);
For an add-on library
libvirt-qemu.so
I don't think there's much to be gained from having an XML element to
turn on/off use of these APIs. If an app doesn't want to use them, it
can simply not link to libvirt-qemu.so
> The <commandline><extra> tag does exactly what you might expect; appends
> the exact string to the qemu command-line.
Allowing many args at once in the <extra> blob means that libvirt will
need to parse & split this up into individual args which than then be
safely passed to 'exec'. It is better to specify one arg per element
to avoid this fragile parsing problem.
> The <alter> tag gets more interesting. The idea is that <alter> would
> allow you to modify the libvirt-generated qemu command-line in arbitrary
> ways. How this would work is probably best explained with some examples:
>
> <commandline>
> <alter option="-net">
> <rename>-netdev</rename>
> </alter>
> </commandline>
>
> In this example, all options named -net on the qemu command-line are
> renamed to -netdev.
>
> In this example, if (and only if) a -net option is seen, then -usbtablet is
> appended to the qemu command-line.
>
> <commandline>
> <alter option="-net">
> <match>\(.*name=hostnet0.*\)</match>
> <modify>\1,tap</modify>
> </alter>
> </commandline>
>
> This gets more complicated (but also more powerful). In this case, any -net
I think this alteration of existing args is faaaar too complex & fragile,
and way overkill. If the arg that libvirt generates isn't what someone
needs, then remove the bit of the guest config responsible for that and
add a complete extra arg, rather than munging the existing one.
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
