Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 10:20 AM, Daniel P. Berrange wrote: For the sake of my analogy, I'd still group cairo in with gtk, not x11. It is really just a re-implmentation of the GDK drawing layer from GTK. It still provides a portable higher level abstraction over underlying graphics & rendering systems it supports. eg, again an app might do the majority of its drawing with Cairo, but occasionally need to jump outside to use XRender or OpenGL directly for pieces that aren't supported currently in the cairo API. Yeah, that's a fair point. Regards, Anthony Liguori Regards, Daniel -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Mon, Apr 26, 2010 at 10:08:33AM -0500, Anthony Liguori wrote: > On 04/26/2010 09:54 AM, Daniel P. Berrange wrote: > >On Mon, Apr 26, 2010 at 05:34:22PM +0300, Avi Kivity wrote: > > > >>On 04/26/2010 05:25 PM, Chris Lalancette wrote: > >> > >>>Right, and you are probably one of the users this work targets. But in > >>>general, for those not very familiar with virtualization/qemu, we want > >>>to steer them far clear of this API. That goes doubly true for > >>>application > >>>developers; we want them to be able to use a stable, long-term API and > >>>not have to worry about the nitty-gritty details of the monitor. It's > >>>that > >>>latter group that we want to make sure doesn't use this API. > >>> > >>> > >>With qmp, we have a stable long term API, and the nitty-gritty details > >>are easily hidden behind a stock json parser (unfortunately some rpc > >>details remain). The command line is baroque, but the libvirt xml isn't > >>so pretty either. > >> > >>The problem is a user that starts with libvirt and outgrows its > >>featureset. Do we want them to fall back to qmp? > >> > >If were QMP were to be the standard mgmt API, then you are merely inverting > >the problem. The problem is now a user that starts with QMP and outgrows > >its featureset. > > > >We have two layers in the stack here& neither provides a superset of the > >other. As Anthony has mentioned, this has parallels with GTK (libvirt) vs > >libX11 (libQMP). Programming against GTK does not give you all the features > >you'd have if you programmed against X, but most people do not need them > >and this is offset by the benefits GTK gives in portability across OS. GTK > >does allow them to get direct access to X if really needed, but most apps > >will avoid doing that if at all possible, and when it is found to be > >neccessary new APIs are added to GTK so that direct X11 access remains a > >short-term hack. > > > >We shouldn't have to frame this as a libvirt vs QMP problem. libvirt uses > >QMP for its work, and apps use libvirt API for the vast majority of their > >needs. If direct access to QMP is needed we can now provide that for apps > >as a short term hack, until suitable APIs are added to libvirt. > > > > I agree with you (for the most part). > > The one difference here is that libgtk actually implements a full widget > kit on top of libX11. That's why people use gtk and not X11 directly. > Portability was really an after thought with gtk. > > If you look at libgtk and libcairo, instead of trying to replicate the > features of cairo within gtk, gtk encourages people to use cairo > directly for anything beyond the simplest 2d operations. For the sake of my analogy, I'd still group cairo in with gtk, not x11. It is really just a re-implmentation of the GDK drawing layer from GTK. It still provides a portable higher level abstraction over underlying graphics & rendering systems it supports. eg, again an app might do the majority of its drawing with Cairo, but occasionally need to jump outside to use XRender or OpenGL directly for pieces that aren't supported currently in the cairo API. Regards, Daniel -- |: Red Hat, Engineering, London-o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org-o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 09:54 AM, Daniel P. Berrange wrote: On Mon, Apr 26, 2010 at 05:34:22PM +0300, Avi Kivity wrote: On 04/26/2010 05:25 PM, Chris Lalancette wrote: Right, and you are probably one of the users this work targets. But in general, for those not very familiar with virtualization/qemu, we want to steer them far clear of this API. That goes doubly true for application developers; we want them to be able to use a stable, long-term API and not have to worry about the nitty-gritty details of the monitor. It's that latter group that we want to make sure doesn't use this API. With qmp, we have a stable long term API, and the nitty-gritty details are easily hidden behind a stock json parser (unfortunately some rpc details remain). The command line is baroque, but the libvirt xml isn't so pretty either. The problem is a user that starts with libvirt and outgrows its featureset. Do we want them to fall back to qmp? If were QMP were to be the standard mgmt API, then you are merely inverting the problem. The problem is now a user that starts with QMP and outgrows its featureset. We have two layers in the stack here& neither provides a superset of the other. As Anthony has mentioned, this has parallels with GTK (libvirt) vs libX11 (libQMP). Programming against GTK does not give you all the features you'd have if you programmed against X, but most people do not need them and this is offset by the benefits GTK gives in portability across OS. GTK does allow them to get direct access to X if really needed, but most apps will avoid doing that if at all possible, and when it is found to be neccessary new APIs are added to GTK so that direct X11 access remains a short-term hack. We shouldn't have to frame this as a libvirt vs QMP problem. libvirt uses QMP for its work, and apps use libvirt API for the vast majority of their needs. If direct access to QMP is needed we can now provide that for apps as a short term hack, until suitable APIs are added to libvirt. I agree with you (for the most part). The one difference here is that libgtk actually implements a full widget kit on top of libX11. That's why people use gtk and not X11 directly. Portability was really an after thought with gtk. If you look at libgtk and libcairo, instead of trying to replicate the features of cairo within gtk, gtk encourages people to use cairo directly for anything beyond the simplest 2d operations. Regards, Anthony Liguori Daniel -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Mon, Apr 26, 2010 at 05:34:22PM +0300, Avi Kivity wrote: > On 04/26/2010 05:25 PM, Chris Lalancette wrote: > >Right, and you are probably one of the users this work targets. But in > >general, for those not very familiar with virtualization/qemu, we want > >to steer them far clear of this API. That goes doubly true for application > >developers; we want them to be able to use a stable, long-term API and > >not have to worry about the nitty-gritty details of the monitor. It's that > >latter group that we want to make sure doesn't use this API. > > > > With qmp, we have a stable long term API, and the nitty-gritty details > are easily hidden behind a stock json parser (unfortunately some rpc > details remain). The command line is baroque, but the libvirt xml isn't > so pretty either. > > The problem is a user that starts with libvirt and outgrows its > featureset. Do we want them to fall back to qmp? If were QMP were to be the standard mgmt API, then you are merely inverting the problem. The problem is now a user that starts with QMP and outgrows its featureset. We have two layers in the stack here & neither provides a superset of the other. As Anthony has mentioned, this has parallels with GTK (libvirt) vs libX11 (libQMP). Programming against GTK does not give you all the features you'd have if you programmed against X, but most people do not need them and this is offset by the benefits GTK gives in portability across OS. GTK does allow them to get direct access to X if really needed, but most apps will avoid doing that if at all possible, and when it is found to be neccessary new APIs are added to GTK so that direct X11 access remains a short-term hack. We shouldn't have to frame this as a libvirt vs QMP problem. libvirt uses QMP for its work, and apps use libvirt API for the vast majority of their needs. If direct access to QMP is needed we can now provide that for apps as a short term hack, until suitable APIs are added to libvirt. Daniel -- |: Red Hat, Engineering, London-o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org-o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 05:48 PM, Anthony Liguori wrote: We could easily reuse that. Any other security context code would be custom written; so it can be written as a qemud plugin instead of a bit of code that goes before a qemu launch. I think we're mostly in agreement with respect to the need to have more control over the security context the qemu runs in. Whether it's launched via a daemon or directly I think is an implementation detail that we can debate when we get closer to an actual implementation. Good, as I haven't decided yet which side I'm on. -- error compiling committee.c: too many arguments to function -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 09:38 AM, Avi Kivity wrote: On 04/26/2010 05:28 PM, Anthony Liguori wrote: Or a library that the user-written launcher calls. Or a plugin that qemud calls. A plugin would lose the security context. It could attempt to recreate it that seems like a lot of unnecessary complexity. A plugin would create the security context instead of the launcher. Currently security contexts are created by the login process. It's not always that centralized. An initial context is created by the login process, but then later something may come along and create a network namespace as part of containerization. We could easily reuse that. Any other security context code would be custom written; so it can be written as a qemud plugin instead of a bit of code that goes before a qemu launch. I think we're mostly in agreement with respect to the need to have more control over the security context the qemu runs in. Whether it's launched via a daemon or directly I think is an implementation detail that we can debate when we get closer to an actual implementation. Regards, Anthony Liguori -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 05:28 PM, Anthony Liguori wrote: Or a library that the user-written launcher calls. Or a plugin that qemud calls. A plugin would lose the security context. It could attempt to recreate it that seems like a lot of unnecessary complexity. A plugin would create the security context instead of the launcher. Currently security contexts are created by the login process. We could easily reuse that. Any other security context code would be custom written; so it can be written as a qemud plugin instead of a bit of code that goes before a qemu launch. -- error compiling committee.c: too many arguments to function -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 05:25 PM, Chris Lalancette wrote: Right, and you are probably one of the users this work targets. But in general, for those not very familiar with virtualization/qemu, we want to steer them far clear of this API. That goes doubly true for application developers; we want them to be able to use a stable, long-term API and not have to worry about the nitty-gritty details of the monitor. It's that latter group that we want to make sure doesn't use this API. With qmp, we have a stable long term API, and the nitty-gritty details are easily hidden behind a stock json parser (unfortunately some rpc details remain). The command line is baroque, but the libvirt xml isn't so pretty either. The problem is a user that starts with libvirt and outgrows its featureset. Do we want them to fall back to qmp? -- error compiling committee.c: too many arguments to function -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Mon, Apr 26, 2010 at 09:26:55AM -0500, Anthony Liguori wrote: > On 04/26/2010 08:58 AM, Daniel P. Berrange wrote: > >On Mon, Apr 26, 2010 at 08:46:46AM -0500, Anthony Liguori wrote: > > > >>On 04/26/2010 08:41 AM, Avi Kivity wrote: > >> > >> > (3) The system management application can certainly create whatever > context it wants to launch a vm from. It's comes down to who's > responsible for creating the context the guest runs under. I think > doing that at the libvirt level takes away a ton of flexibility from > the management application. > > >>>If you want to push the flexibility slider all the way to the right > >>>you get bare qemu. It exposes 100% of qemu capabilities. And it's > >>>not so bad these days. But it's not something that can be remoted. > >>> > >>As I mentioned earlier, remoting is not a very important use-case to me. > >> > >NB, the "remote" protocol is no different from the "local" protocol > >the unprivileged app uses to talk to the privileged daemon. The only > >difference is unix sockets vs tcp sockets. We essentially get the > >remote access capability for free as part of the protocol used between > >the apps& daemon. > > > > I think Avi's concern is that if you expose direct launch, then that's > not really something you can usefully exploit in a remote protocol. > > For instance, if you're local, you can call setuid() before invoking the > launch API whereas if you're remote, you will always launch guests as > the context of the remoting daemon. > > This makes the remote API second-class in terms of functionality. It is not entirely outside the realms of possibility to consider making a way for libvirt to attach to a QEMU spawned by a 3rd party. eg, virsh qemu-attach /path/to/monitor PID Converting the QEMU config back into Libvirt XML is not trivial, but neither is it impossible. Particularly now we have the custom syntax for things libvirt can't represent, we can simply use that for bits we can't reverse map to XML. There's also the UUID/name uniqueness, but we could fail the attach operation if that wwas violated. This would let you use most of the other libvirt APIs remotely from any app, even if the VM was spawned by another one. Regards, Daniel -- |: Red Hat, Engineering, London-o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org-o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 09:25 AM, Avi Kivity wrote: On 04/26/2010 05:19 PM, Anthony Liguori wrote: On 04/26/2010 09:01 AM, Avi Kivity wrote: On 04/26/2010 04:43 PM, Anthony Liguori wrote: The reason I lean toward the direct launch model is that it gives the user a lot of flexibility in terms of using things like namespaces, DAC, cgroups, capabilities, etc. A lot of potential features are lost when you do indirect launch because you have to teach the daemon how to support each of these features. But what's the alternative? Teach the user how to do all these things? You can expose layers of API. The lowest layer makes no changes to the security context. A higher (optional) layer could do dynamic labelling. Or a library that the user-written launcher calls. Or a plugin that qemud calls. A plugin would lose the security context. It could attempt to recreate it that seems like a lot of unnecessary complexity. Regards, Anthony Liguori -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 08:54 AM, Jamie Lokier wrote: > All the features? The qemu API is quite large already (look at all > the command line options and monitor commands). I'll be very > surprised if libvirt provides all of it that obscure apps may use. > > I'm thinking of features which are relatively obscure but nonetheless > useful to a small number of deployments. Probably not enough to > justify the effort building data models, specifying the XML and remote > protocol and so on in libvirt. > > (Unless that becomes so easily mapped to qemu's API that it's almost an > automatic thing... Which sounds like QMP, doesn't it?) > > Is libvirt ever likely to go to the effort of providing all the > easily-usable API, or hooks, for: > > - sending keys to a guest, driven by a timed host script? > > - rebooting the guest while switching between USB touchpad and > mouse devices, because one of them is needed during an OS > install and the other is needed after? > > - changing the amount of RAM available to the guest at the next > reboot, for OS install needing more memory than run time, in a > scripted fashion when building new VMs from install disk images? > > - switching the guest between qemu mode and kvm mode on the next > guest reset, because qemu is faster for some things (VGA > updates) and kvm is faster for other things, so the best choice > depends on which app you need to run on that guest > > - pausing a VM, making a copy, and resuming it, so as to fork it > into two VMs (literally fork)? > > - setting up the host network container and NAT IP forwarding, on > demand as guests are stopped and started, so that it works in > the above scenario despite clashing IP addresses? > > - running a copy of the same guest, or perhaps an entire OS > install process (scripted), many times for different qemu and > qemu-kvm versions, different BIOSes, and different > almost-equivalent hardware emulations (i.e. different NIC types, > SMP count, CPU features, disk controller type, AIO/cache type) - > for testing guests and apps on them - with some paralellism? > > None of those, except perhaps the first, as what I think of as typical > virtualisation workloads, and they all seem obscure things probably > outside libvirt's remit. Probably not many users either :-) > > Yet you can do them all today with qemu and scripting the monitor, and > it's getting easier with QMP. In point of fact, you can also do some of them today more-or-less with libvirt and some scripting. However, you are right, there are probably obscure features that libvirt will *never* be able to entirely implement, either because they don't fit into libvirt's world-view or because they are too obscure or difficult to do. However, Dan's point stands; for most features that most users want to use, it is far better to spend effort creating a cross-hypervisor, long-term API than to spend too much effort on individual hypervisor hacks. > I'm just raising my hand as a potential user who might like to monitor > a bunch of active and inactive guests, remotely, see how much memory > they report using, etc. launch VNC viewer from the GUI, even choose > the target host based on load and migrate on demand, while also > needing a fair bit of non-standardness and qemu-level scripting too. > > Imho, that probably comes under the heading of apps using pass-through > or multiple QMP monitors, which use features that probably won't and > probably shouldn't ever be handled by libvirt itself. Right, and you are probably one of the users this work targets. But in general, for those not very familiar with virtualization/qemu, we want to steer them far clear of this API. That goes doubly true for application developers; we want them to be able to use a stable, long-term API and not have to worry about the nitty-gritty details of the monitor. It's that latter group that we want to make sure doesn't use this API. -- Chris Lalancette -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 08:58 AM, Daniel P. Berrange wrote: On Mon, Apr 26, 2010 at 08:46:46AM -0500, Anthony Liguori wrote: On 04/26/2010 08:41 AM, Avi Kivity wrote: (3) The system management application can certainly create whatever context it wants to launch a vm from. It's comes down to who's responsible for creating the context the guest runs under. I think doing that at the libvirt level takes away a ton of flexibility from the management application. If you want to push the flexibility slider all the way to the right you get bare qemu. It exposes 100% of qemu capabilities. And it's not so bad these days. But it's not something that can be remoted. As I mentioned earlier, remoting is not a very important use-case to me. NB, the "remote" protocol is no different from the "local" protocol the unprivileged app uses to talk to the privileged daemon. The only difference is unix sockets vs tcp sockets. We essentially get the remote access capability for free as part of the protocol used between the apps& daemon. I think Avi's concern is that if you expose direct launch, then that's not really something you can usefully exploit in a remote protocol. For instance, if you're local, you can call setuid() before invoking the launch API whereas if you're remote, you will always launch guests as the context of the remoting daemon. This makes the remote API second-class in terms of functionality. Regards, Anthony Liguori Does RHEV-M actually use the remote libvirt interface? I assume it'll talk to vdsm via some protocol and vdsm will use the local libvirt API. That is correct. That doesn't mean the remote interface isn't useful. I suspect most uses of libvirt are actually local uses. A key part of libvirt is interoperability between applications. So an app like vsdm can be using libvirt locally, while at the same time virt-top can be monitoring guests on a set of machines remotely, or virt-viewer can be run from your desktop talking to libvirt remotely to discover the VNC console details. For flexibility there are several options for remotely managing VMs. The native remote RPC system. The libvirt QPid/QMF model. The CIM binding. Application specific remote protocols. Since they all eventually talk to the same libvirtd daemon, their use is not mutually exclusive which is a good thing. Daniel -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 05:19 PM, Anthony Liguori wrote: On 04/26/2010 09:01 AM, Avi Kivity wrote: On 04/26/2010 04:43 PM, Anthony Liguori wrote: The reason I lean toward the direct launch model is that it gives the user a lot of flexibility in terms of using things like namespaces, DAC, cgroups, capabilities, etc. A lot of potential features are lost when you do indirect launch because you have to teach the daemon how to support each of these features. But what's the alternative? Teach the user how to do all these things? You can expose layers of API. The lowest layer makes no changes to the security context. A higher (optional) layer could do dynamic labelling. Or a library that the user-written launcher calls. Or a plugin that qemud calls. It's infinitely flexible, but it's not an API you can give to a management tool developer. I think the goal of a management API should be to make common things very simple to do but not preclude doing even the most advanced things. Agreed. -- error compiling committee.c: too many arguments to function -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 09:01 AM, Avi Kivity wrote: On 04/26/2010 04:43 PM, Anthony Liguori wrote: The reason I lean toward the direct launch model is that it gives the user a lot of flexibility in terms of using things like namespaces, DAC, cgroups, capabilities, etc. A lot of potential features are lost when you do indirect launch because you have to teach the daemon how to support each of these features. But what's the alternative? Teach the user how to do all these things? You can expose layers of API. The lowest layer makes no changes to the security context. A higher (optional) layer could do dynamic labelling. It's infinitely flexible, but it's not an API you can give to a management tool developer. I think the goal of a management API should be to make common things very simple to do but not preclude doing even the most advanced things. Regards, Anthony Liguori -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 04:43 PM, Anthony Liguori wrote: The reason I lean toward the direct launch model is that it gives the user a lot of flexibility in terms of using things like namespaces, DAC, cgroups, capabilities, etc. A lot of potential features are lost when you do indirect launch because you have to teach the daemon how to support each of these features. But what's the alternative? Teach the user how to do all these things? It's infinitely flexible, but it's not an API you can give to a management tool developer. -- error compiling committee.c: too many arguments to function -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Mon, Apr 26, 2010 at 08:46:46AM -0500, Anthony Liguori wrote: > On 04/26/2010 08:41 AM, Avi Kivity wrote: > > >>(3) The system management application can certainly create whatever > >>context it wants to launch a vm from. It's comes down to who's > >>responsible for creating the context the guest runs under. I think > >>doing that at the libvirt level takes away a ton of flexibility from > >>the management application. > > > >If you want to push the flexibility slider all the way to the right > >you get bare qemu. It exposes 100% of qemu capabilities. And it's > >not so bad these days. But it's not something that can be remoted. > > As I mentioned earlier, remoting is not a very important use-case to me. NB, the "remote" protocol is no different from the "local" protocol the unprivileged app uses to talk to the privileged daemon. The only difference is unix sockets vs tcp sockets. We essentially get the remote access capability for free as part of the protocol used between the apps & daemon. > Does RHEV-M actually use the remote libvirt interface? I assume it'll > talk to vdsm via some protocol and vdsm will use the local libvirt API. That is correct. That doesn't mean the remote interface isn't useful. > I suspect most uses of libvirt are actually local uses. A key part of libvirt is interoperability between applications. So an app like vsdm can be using libvirt locally, while at the same time virt-top can be monitoring guests on a set of machines remotely, or virt-viewer can be run from your desktop talking to libvirt remotely to discover the VNC console details. For flexibility there are several options for remotely managing VMs. The native remote RPC system. The libvirt QPid/QMF model. The CIM binding. Application specific remote protocols. Since they all eventually talk to the same libvirtd daemon, their use is not mutually exclusive which is a good thing. Daniel -- |: Red Hat, Engineering, London-o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org-o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 04:46 PM, Anthony Liguori wrote: (3) The system management application can certainly create whatever context it wants to launch a vm from. It's comes down to who's responsible for creating the context the guest runs under. I think doing that at the libvirt level takes away a ton of flexibility from the management application. If you want to push the flexibility slider all the way to the right you get bare qemu. It exposes 100% of qemu capabilities. And it's not so bad these days. But it's not something that can be remoted. As I mentioned earlier, remoting is not a very important use-case to me. Does RHEV-M actually use the remote libvirt interface? I assume it'll talk to vdsm via some protocol and vdsm will use the local libvirt API. Yes. I suspect most uses of libvirt are actually local uses. I expect the same, though I'm sure a design goal was to make use of libvirt be reasonable through the remote API. If we aren't able to fulfil it, much of the value of libvirt goes away. -- error compiling committee.c: too many arguments to function -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 08:41 AM, Avi Kivity wrote: Today, you have to make changes to libvirt whereas in a direct launch model, you get all of the neat security features linux supports for free. But you lose tap networking, unless you have a privileged helper. And how is the privileged helper to authenticate the qemu calling it? There are a variety of ways. My original proposal used a policy file. And I've said in the past that I don't like the idea of a qemud :-) I must have missed it. Why not? Every other hypervisor has a central management entity. Because you end up launching all guests from a single security context. Run multiple qemuds? But what you say makes sense. It's similar to the fork() /* do interesting stuff */ exec() model, compared to the spawn(..., hardcoded list of interesting stuff). Yeah, that's where I'm at. I'd eventually like libvirt to use our provided API and I can see where it would add value to the stack (by doing things like storage and network management). We do provide an API, qmp, and libvirt uses it? Yeah, but we need to support more features (like guest enumeration). What are our options? 1) qemud launches, enumerates 2) user launches, qemu registers in qemud 3) user launches, qemu registers in filesystem 4) you launched it, you enumerate it Both 2 and 3 are appealing to me. (3) The system management application can certainly create whatever context it wants to launch a vm from. It's comes down to who's responsible for creating the context the guest runs under. I think doing that at the libvirt level takes away a ton of flexibility from the management application. If you want to push the flexibility slider all the way to the right you get bare qemu. It exposes 100% of qemu capabilities. And it's not so bad these days. But it's not something that can be remoted. As I mentioned earlier, remoting is not a very important use-case to me. Does RHEV-M actually use the remote libvirt interface? I assume it'll talk to vdsm via some protocol and vdsm will use the local libvirt API. I suspect most uses of libvirt are actually local uses. Regards, Anthony Liguori -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 08:31 AM, Daniel P. Berrange wrote: What you describe is not inherant to the daemon model. This is why we have two separate models in libvirt. The system instance is pre-spawned with high privileges, to allow use of hosts resources which require high privileges to access. The session instance is auto-spawned when the app connects to libvirt, thus it inherits the privileges of the app that is using it. I don't deny that the system instance has a new attack surface, because it is running privileged. If the app needs to connect VMs to privileged resources, then the architecture has to have some privileged component to give access to those resoruces. You don't want the VM to be privileged, nor the whole management app to be privileged. The system daemon is thus the arbitrator for this privileged access. If you don't need todo anything that requires higher privileges though, just use the session instance which always matches the apps privileges. I regret saying "more secure" because I think it's a difficult concept to really quantify and that makes it hard to meaningfully discuss. The reason I lean toward the direct launch model is that it gives the user a lot of flexibility in terms of using things like namespaces, DAC, cgroups, capabilities, etc. A lot of potential features are lost when you do indirect launch because you have to teach the daemon how to support each of these features. Regards, Anthony Liguori Daniel -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 04:14 PM, Anthony Liguori wrote: IOW, libvirt does not run guests as separate users which is why it needs to deal with security in the first place. What if one user has multiple guests? isolation is still needed. Don't confuse a management application's concept of users with using separate uid's to launch guests. Then someone needs to manage those users. A user can't suid to any random user. You need someone privileged to allocate the new uid and su into it. One user per guest does not satisfy some security requirements. The 'M' in selinux stands for mandatory, which means that the entities secured can't leak information even if they want to (scenario: G1 breaks into qemu, chmods files, G2 breaks into qemu, reads files). If you're implementing a chinese firewall policy, then yes, you want to run each guest as a separate selinux context. Starting as separate users and setting DAC privileges appropriately will achieve this. But you're not always implementing that type of policy. If the guest inherits the uid, selinux context, and namespaces of whatever launches the guest, then you have the most flexibility from a security perspective. How do you launch a libvirt guest in a network namespace? How do you put it in a chroot? You pass the namespace fd and chroot fd using SCM_RIGHTS (except you probably can't do that). Today, you have to make changes to libvirt whereas in a direct launch model, you get all of the neat security features linux supports for free. But you lose tap networking, unless you have a privileged helper. And how is the privileged helper to authenticate the qemu calling it? And I've said in the past that I don't like the idea of a qemud :-) I must have missed it. Why not? Every other hypervisor has a central management entity. Because you end up launching all guests from a single security context. Run multiple qemuds? But what you say makes sense. It's similar to the fork() /* do interesting stuff */ exec() model, compared to the spawn(..., hardcoded list of interesting stuff). Yeah, that's where I'm at. I'd eventually like libvirt to use our provided API and I can see where it would add value to the stack (by doing things like storage and network management). We do provide an API, qmp, and libvirt uses it? Yeah, but we need to support more features (like guest enumeration). What are our options? 1) qemud launches, enumerates 2) user launches, qemu registers in qemud 3) user launches, qemu registers in filesystem 4) you launched it, you enumerate it That's wrong for three reasons. First, selinux is not a uid replacement (if it was libvirt could just suid $random_user before launching qemu). Second, a single user's guests should be protected from each other. Third, in many deployments, the guest's owner isn't logged in to supply the credentials, it's system management that launches the guests. (1) uid's are just one part of an applications security context. There's an selinux context, all of the various namespaces, capabilities, etc. If you use a daemon to launch a guest, you lose all of that unless you have a very sophisticated api. True. In a perfect world, we'd use SCM_RIGHTS to channel all of these to libvirt or qemud. On the other hand, users don't want to do all these things by hand. They want management to do things for them. Self launch is very flexible, but it's not an API, and cannot be used remotely. We could use qemud plugins to allow the user to customize the launch process. (2) If you want to implement a policy that only a single guest can access a single image, you can create an SELinux policy and use static labelling to achieve that. That's just one type of policy though. It's also not going to work in an environment that doesn't preserve all security labels (like direct access to volumes; /dev is on tmpfs these days). (3) The system management application can certainly create whatever context it wants to launch a vm from. It's comes down to who's responsible for creating the context the guest runs under. I think doing that at the libvirt level takes away a ton of flexibility from the management application. If you want to push the flexibility slider all the way to the right you get bare qemu. It exposes 100% of qemu capabilities. And it's not so bad these days. But it's not something that can be remoted. -- error compiling committee.c: too many arguments to function -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Mon, Apr 26, 2010 at 08:13:03AM -0500, Anthony Liguori wrote: > On 04/26/2010 04:59 AM, Daniel P. Berrange wrote: > >On Sun, Apr 25, 2010 at 08:53:17PM -0500, Anthony Liguori wrote: > > > >>On 04/25/2010 06:51 AM, Avi Kivity wrote: > >> > >>> Qemu is special due to the nonexistence of qemud. > >>> > >>>Why is sVirt implemented in libvirt? it's not the logical place for > >>>it; rather the logical place doesn't exist. > >>> > >>sVirt is not just implemented in libvirt. libvirt implements a > >>mechanism to set the context of a given domain and dynamically label > >>it's resources to isolate it. > >> > >>The reason it has to assign a context to a given domain is that all > >>domains are launched from the same security context (the libvirtd > >>context) as the original user's context (the consumer of the libvirt > >>API) has been lost via the domain socket interface. > >> > >>If you used the /session URL, then the domain would have the security > >>context of whomever created the guest which means that dynamic labelling > >>of the resources wouldn't be necessary (you would just do static > >>labelling). > >> > >That is not correct. You do *not* ever want the guests to have the same > >security context as the thing that created them, because that would allow > >the guest to access& compromise resources belonging to the management app. > > > > You assume that the management app is not smart enough to create a new > context for the guest to run in. > > >>This is certainly a more secure model and it's a feature of qemu that I > >>really wish didn't get lost in libvirt. Again, /session can do this too > >>but right now, /session really isn't usable in libvirt for qemu. > >> > >If you really want the qemu instance to inherit the context of the mgmt > >app, then you can just declare in the guest XML that it should use a > >static label, and pass in the apps' own label. This is *not* a more secure > >model though. > > > > There is more context than just selinux labelling. The problem with the > daemon model is that to create a guest, you start with a lower set of > privileges, escalate your privileges (by talking to libvirtd), then > lower privileges to launch a guest. Running a guest is essentially > running arbitrary code (since you can set the emulator path) so now > you've provided an environment where a user can launch arbitrary code as > a different user in a different security context. No matter what emulator path is provided, that will never be run with elevated privileges unless the host sysadmin has explicitly configured that capability. The QEMU binaries always run in a dedicated account which is completely unprivileged. This setuid is done *before* QEMU is even exec()'d. This is one of the reasons why we don't use the QEMU -runas argument, which would allow custom emulator binaries to potentially gain evelated privileges. > There is a new attack surface here. I think it's undeniable that there > is certainly the possibility that something goes wrong and a user will > find a way to escalate it's privileges. Of course all software has bugs. > Compare that to a direct launch model. There is not new attack > surface. The user's privileges never increase. In fact, what's most > likely to happen is that a caller will drop some of it's privileges > before launching a guest. What you describe is not inherant to the daemon model. This is why we have two separate models in libvirt. The system instance is pre-spawned with high privileges, to allow use of hosts resources which require high privileges to access. The session instance is auto-spawned when the app connects to libvirt, thus it inherits the privileges of the app that is using it. I don't deny that the system instance has a new attack surface, because it is running privileged. If the app needs to connect VMs to privileged resources, then the architecture has to have some privileged component to give access to those resoruces. You don't want the VM to be privileged, nor the whole management app to be privileged. The system daemon is thus the arbitrator for this privileged access. If you don't need todo anything that requires higher privileges though, just use the session instance which always matches the apps privileges. Daniel -- |: Red Hat, Engineering, London-o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org-o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 12:56 AM, Avi Kivity wrote: On 04/26/2010 04:53 AM, Anthony Liguori wrote: On 04/25/2010 06:51 AM, Avi Kivity wrote: It depends on what things you think are important. A lot of libvirt's complexity is based on the fact that it uses a daemon and needs to deal with the security implications of that. You don't need explicit labelling if you don't use a daemon. I don't follow. If you have multiple guests that you want off each other's turf you have to label their resources, either statically or dynamically. How is it related to a daemon being present? Because libvirt has to perform this labelling because it loses the original user's security context. If you invoke qemu with the original user's credentials that launched the guest, then you don't need to do anything special with respect to security. IOW, libvirt does not run guests as separate users which is why it needs to deal with security in the first place. What if one user has multiple guests? isolation is still needed. Don't confuse a management application's concept of users with using separate uid's to launch guests. One user per guest does not satisfy some security requirements. The 'M' in selinux stands for mandatory, which means that the entities secured can't leak information even if they want to (scenario: G1 breaks into qemu, chmods files, G2 breaks into qemu, reads files). If you're implementing a chinese firewall policy, then yes, you want to run each guest as a separate selinux context. Starting as separate users and setting DAC privileges appropriately will achieve this. But you're not always implementing that type of policy. If the guest inherits the uid, selinux context, and namespaces of whatever launches the guest, then you have the most flexibility from a security perspective. How do you launch a libvirt guest in a network namespace? How do you put it in a chroot? Today, you have to make changes to libvirt whereas in a direct launch model, you get all of the neat security features linux supports for free. And I've said in the past that I don't like the idea of a qemud :-) I must have missed it. Why not? Every other hypervisor has a central management entity. Because you end up launching all guests from a single security context. In theory, it does support this with the session urls but they are currently second-class citizens in libvirt. The remote dispatch also adds a fair bit of complexity and at least for the use-cases I'm interested in, it's not an important feature. If libvirt needs a local wrapper for interesting use cases, then it has failed. You can't have a local wrapper with the esx driver, for example. This is off-topic, but can you detail why you don't want remote dispatch (I assume we're talking about a multiple node deployment). Because there are dozens of remote management APIs and then all have a concept of agents that run on the end nodes. When fitting virtualization management into an existing management infrastructure, you are going to always use a local API. When you manage esx, do you deploy an agent? I thought it was all done via their remote APIs. Historically, people have deployed agents into the console OS. In recent versions, ESX actually includes CIM agents by default. Every typical virtualization use will eventually grow some non-typical requirements. If libvirt explicitly refuses to support qemu features, I don't see how we can recommend it - even if it satisfies a user's requirements today, what about tomorrow? what about future qemu feature, will they be exposed or not? If that is the case then we should develop qemud (which libvirt and other apps can use). (even if it isn't the case I think qemud is a good idea) Yeah, that's where I'm at. I'd eventually like libvirt to use our provided API and I can see where it would add value to the stack (by doing things like storage and network management). We do provide an API, qmp, and libvirt uses it? Yeah, but we need to support more features (like guest enumeration). The alternative is to get libvirt to just act as a thin layer to expose qemu features directly. But honestly, what's the point of libvirt if they did that? For most hypervisors, that's exactly what libvirt does. For Xen, it also bypasses Xend and the hypervisor's API, but it shouldn't really. Historically, xend was so incredibly slow (especially for frequent statistics collection) that it was a necessity. Ah, reimplement rather than fix. There's a complicated history there. Qemu is special due to the nonexistence of qemud. Why is sVirt implemented in libvirt? it's not the logical place for it; rather the logical place doesn't exist. sVirt is not just implemented in libvirt. libvirt implements a mechanism to set the context of a given domain and dynamically label it's resources to isolate it. The reason it has to assign a context to a given do
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/25/2010 09:50 AM, Avi Kivity wrote: On 04/23/2010 09:33 PM, Anthony Liguori wrote: This is a different ambiguity, about the semantic results of the commands, where as I'm refering to the execution order. If I look at a libvirt log file and see a set of JSON commands logged, I want to know that this ordering from the logs, was indeed the same as order in which qemu processed them. If you have two separate monitor connection you can't be sure of the order of execution. It is key for our bug troubleshooting that given a libvirt log file, we can replay the JSON commands again and get the same results. Two monitor connections is just increasing complexity of code without any tangible benefit. I think you're assuming direct access to the second monitor? I'm not suggesting that. I'm suggesting that libvirt is still the one submitting commands to the second monitor and that it submits those commands in lock step. What about protocol extensions? For instance, pretend libvirt doesn't support async messages, what would it do when it receives one from the user's monitor? Protocol extensions could not be supported in this model. I agree, that's unfortunate. Regards, Anthony Liguori -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 04:59 AM, Daniel P. Berrange wrote: On Sun, Apr 25, 2010 at 08:53:17PM -0500, Anthony Liguori wrote: On 04/25/2010 06:51 AM, Avi Kivity wrote: Qemu is special due to the nonexistence of qemud. Why is sVirt implemented in libvirt? it's not the logical place for it; rather the logical place doesn't exist. sVirt is not just implemented in libvirt. libvirt implements a mechanism to set the context of a given domain and dynamically label it's resources to isolate it. The reason it has to assign a context to a given domain is that all domains are launched from the same security context (the libvirtd context) as the original user's context (the consumer of the libvirt API) has been lost via the domain socket interface. If you used the /session URL, then the domain would have the security context of whomever created the guest which means that dynamic labelling of the resources wouldn't be necessary (you would just do static labelling). That is not correct. You do *not* ever want the guests to have the same security context as the thing that created them, because that would allow the guest to access& compromise resources belonging to the management app. You assume that the management app is not smart enough to create a new context for the guest to run in. This is certainly a more secure model and it's a feature of qemu that I really wish didn't get lost in libvirt. Again, /session can do this too but right now, /session really isn't usable in libvirt for qemu. If you really want the qemu instance to inherit the context of the mgmt app, then you can just declare in the guest XML that it should use a static label, and pass in the apps' own label. This is *not* a more secure model though. There is more context than just selinux labelling. The problem with the daemon model is that to create a guest, you start with a lower set of privileges, escalate your privileges (by talking to libvirtd), then lower privileges to launch a guest. Running a guest is essentially running arbitrary code (since you can set the emulator path) so now you've provided an environment where a user can launch arbitrary code as a different user in a different security context. There is a new attack surface here. I think it's undeniable that there is certainly the possibility that something goes wrong and a user will find a way to escalate it's privileges. Compare that to a direct launch model. There is not new attack surface. The user's privileges never increase. In fact, what's most likely to happen is that a caller will drop some of it's privileges before launching a guest. Regards, Anthony Liguori Daniel -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
Daniel P. Berrange wrote: > > Much better to exact a commitment from libvirt to track all QMP (and > > command line) capabilities. Instead of adding cleverness to QMP, add > > APIs to libvirt. > > Agreed. Despite adding this monitor / XML passthrough capability, we still > do not want apps to be using this at all. If there is some capability > missing that apps need then the default mode of operation is to add the > neccessary bits of libvirt. The monitor/XML pasthrough is just a short > term quick workaround until the official support is done. As such I do > not really think we need to put huge amounts of effort in the wierd > complex racey edge cases. The effort is better spent on getting the > features in libvirt. All the features? The qemu API is quite large already (look at all the command line options and monitor commands). I'll be very surprised if libvirt provides all of it that obscure apps may use. I'm thinking of features which are relatively obscure but nonetheless useful to a small number of deployments. Probably not enough to justify the effort building data models, specifying the XML and remote protocol and so on in libvirt. (Unless that becomes so easily mapped to qemu's API that it's almost an automatic thing... Which sounds like QMP, doesn't it?) Is libvirt ever likely to go to the effort of providing all the easily-usable API, or hooks, for: - sending keys to a guest, driven by a timed host script? - rebooting the guest while switching between USB touchpad and mouse devices, because one of them is needed during an OS install and the other is needed after? - changing the amount of RAM available to the guest at the next reboot, for OS install needing more memory than run time, in a scripted fashion when building new VMs from install disk images? - switching the guest between qemu mode and kvm mode on the next guest reset, because qemu is faster for some things (VGA updates) and kvm is faster for other things, so the best choice depends on which app you need to run on that guest - pausing a VM, making a copy, and resuming it, so as to fork it into two VMs (literally fork)? - setting up the host network container and NAT IP forwarding, on demand as guests are stopped and started, so that it works in the above scenario despite clashing IP addresses? - running a copy of the same guest, or perhaps an entire OS install process (scripted), many times for different qemu and qemu-kvm versions, different BIOSes, and different almost-equivalent hardware emulations (i.e. different NIC types, SMP count, CPU features, disk controller type, AIO/cache type) - for testing guests and apps on them - with some paralellism? None of those, except perhaps the first, as what I think of as typical virtualisation workloads, and they all seem obscure things probably outside libvirt's remit. Probably not many users either :-) Yet you can do them all today with qemu and scripting the monitor, and it's getting easier with QMP. Which is fine, qemu works, but it would be great to be able to see those guests and interact in the basic ways through the libvirt-based GUIs? QMP pass-through or QMP multiple monitors seems to provide most of that, although I can see libvirt getting a bit confused about which devices and how much RAM the guest has installed at different times. The bit about forking guests, I'm not sure how complicated it is to tie in to libvirt's notion of which disk images are being used, and hooking into it's network configuration to handle the clashing addresses. If those things are considered to be entirely outside libvirt's remit, that's fine with me. Fair enough: I will continue to live with ssh and vinagre. I'm just raising my hand as a potential user who might like to monitor a bunch of active and inactive guests, remotely, see how much memory they report using, etc. launch VNC viewer from the GUI, even choose the target host based on load and migrate on demand, while also needing a fair bit of non-standardness and qemu-level scripting too. Imho, that probably comes under the heading of apps using pass-through or multiple QMP monitors, which use features that probably won't and probably shouldn't ever be handled by libvirt itself. -- Jamie -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Sun, Apr 25, 2010 at 08:53:17PM -0500, Anthony Liguori wrote: > On 04/25/2010 06:51 AM, Avi Kivity wrote: > > Qemu is special due to the nonexistence of qemud. > > > >Why is sVirt implemented in libvirt? it's not the logical place for > >it; rather the logical place doesn't exist. > > sVirt is not just implemented in libvirt. libvirt implements a > mechanism to set the context of a given domain and dynamically label > it's resources to isolate it. > > The reason it has to assign a context to a given domain is that all > domains are launched from the same security context (the libvirtd > context) as the original user's context (the consumer of the libvirt > API) has been lost via the domain socket interface. > > If you used the /session URL, then the domain would have the security > context of whomever created the guest which means that dynamic labelling > of the resources wouldn't be necessary (you would just do static labelling). That is not correct. You do *not* ever want the guests to have the same security context as the thing that created them, because that would allow the guest to access & compromise resources belonging to the management app. Every guest must always have a unique context. The libvirt+sVirt dynamic labelling with unique contexts is applied even for the /session mode. The thing you save with the /session mode, is not having todo the user/group ownership management; sVirt labelling is always required. > This is certainly a more secure model and it's a feature of qemu that I > really wish didn't get lost in libvirt. Again, /session can do this too > but right now, /session really isn't usable in libvirt for qemu. If you really want the qemu instance to inherit the context of the mgmt app, then you can just declare in the guest XML that it should use a static label, and pass in the apps' own label. This is *not* a more secure model though. Daniel -- |: Red Hat, Engineering, London-o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org-o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
2010/4/26 Avi Kivity : > [...] >> In theory, it does support this with the session urls but they are currently second-class citizens in libvirt. Â The remote dispatch also adds a fair bit of complexity and at least for the use-cases I'm interested in, it's not an important feature. >>> >>> If libvirt needs a local wrapper for interesting use cases, then it has >>> failed. Â You can't have a local wrapper with the esx driver, for example. >>> >>> This is off-topic, but can you detail why you don't want remote dispatch >>> (I assume we're talking about a multiple node deployment). >> >> Because there are dozens of remote management APIs and then all have a >> concept of agents that run on the end nodes. Â When fitting virtualization >> management into an existing management infrastructure, you are going to >> always use a local API. > > When you manage esx, do you deploy an agent? Â I thought it was all done via > their remote APIs. > No, you don't deploy any agents. If you manage an ESX server using VMware tools only, then you use VMware management tools on the client PC. This tools directly use the SOAP based remote API provided by the ESX server. The ESX driver in libvirt does the same and uses this remote API as the VMware management tools. No libvirt specific agents are installed on the ESX server and no VMware specific agents (except libvirt itself) are installed on the client PC. Matthias -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/26/2010 04:53 AM, Anthony Liguori wrote: On 04/25/2010 06:51 AM, Avi Kivity wrote: It depends on what things you think are important. A lot of libvirt's complexity is based on the fact that it uses a daemon and needs to deal with the security implications of that. You don't need explicit labelling if you don't use a daemon. I don't follow. If you have multiple guests that you want off each other's turf you have to label their resources, either statically or dynamically. How is it related to a daemon being present? Because libvirt has to perform this labelling because it loses the original user's security context. If you invoke qemu with the original user's credentials that launched the guest, then you don't need to do anything special with respect to security. IOW, libvirt does not run guests as separate users which is why it needs to deal with security in the first place. What if one user has multiple guests? isolation is still needed. One user per guest does not satisfy some security requirements. The 'M' in selinux stands for mandatory, which means that the entities secured can't leak information even if they want to (scenario: G1 breaks into qemu, chmods files, G2 breaks into qemu, reads files). This is really the qemu model (as opposed to the xend model). (and the qemud model). And I've said in the past that I don't like the idea of a qemud :-) I must have missed it. Why not? Every other hypervisor has a central management entity. In theory, it does support this with the session urls but they are currently second-class citizens in libvirt. The remote dispatch also adds a fair bit of complexity and at least for the use-cases I'm interested in, it's not an important feature. If libvirt needs a local wrapper for interesting use cases, then it has failed. You can't have a local wrapper with the esx driver, for example. This is off-topic, but can you detail why you don't want remote dispatch (I assume we're talking about a multiple node deployment). Because there are dozens of remote management APIs and then all have a concept of agents that run on the end nodes. When fitting virtualization management into an existing management infrastructure, you are going to always use a local API. When you manage esx, do you deploy an agent? I thought it was all done via their remote APIs. Every typical virtualization use will eventually grow some non-typical requirements. If libvirt explicitly refuses to support qemu features, I don't see how we can recommend it - even if it satisfies a user's requirements today, what about tomorrow? what about future qemu feature, will they be exposed or not? If that is the case then we should develop qemud (which libvirt and other apps can use). (even if it isn't the case I think qemud is a good idea) Yeah, that's where I'm at. I'd eventually like libvirt to use our provided API and I can see where it would add value to the stack (by doing things like storage and network management). We do provide an API, qmp, and libvirt uses it? That's not what the libvirt community wants to do. We're very bias. We've made decisions about how features should be exposed and what features should be included. We want all of those features exposed exactly how we've implemented them because we think it's the right way. I'm not sure there's an obvious way forward unless we decide that there is going to be two ways to interact with qemu. One way is through the libvirt world-view and the other is through a more qemu centric view. The problem then becomes allowing those two models to co-exist happily together. I don't think there's a point in managing qemu through libvirt and directly in parallel. It means a user has to learn both APIs, and for every operation they need to check both to see what's the best way of exploiting the feature. There will invariably be some friction. Layers need to stack on top of each other, not live side by side or bypass each other. I agree with you theoretically but practically, I think it's immensely useful as a stop-gap. Sure. But please lets not start being clever with transactions and atomic operations and stuff, it has to come with a label that says, if you're using this, then something is wrong. The alternative is to get libvirt to just act as a thin layer to expose qemu features directly. But honestly, what's the point of libvirt if they did that? For most hypervisors, that's exactly what libvirt does. For Xen, it also bypasses Xend and the hypervisor's API, but it shouldn't really. Historically, xend was so incredibly slow (especially for frequent statistics collection) that it was a necessity. Ah, reimplement rather than fix. Qemu is special due to the nonexistence of qemud. Why is sVirt implemented in libvirt? it's not the logical place for it; rather the logical place doesn't exist. sVirt is not just imple
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/25/2010 06:51 AM, Avi Kivity wrote: It depends on what things you think are important. A lot of libvirt's complexity is based on the fact that it uses a daemon and needs to deal with the security implications of that. You don't need explicit labelling if you don't use a daemon. I don't follow. If you have multiple guests that you want off each other's turf you have to label their resources, either statically or dynamically. How is it related to a daemon being present? Because libvirt has to perform this labelling because it loses the original user's security context. If you invoke qemu with the original user's credentials that launched the guest, then you don't need to do anything special with respect to security. IOW, libvirt does not run guests as separate users which is why it needs to deal with security in the first place. This is really the qemu model (as opposed to the xend model). (and the qemud model). And I've said in the past that I don't like the idea of a qemud :-) In theory, it does support this with the session urls but they are currently second-class citizens in libvirt. The remote dispatch also adds a fair bit of complexity and at least for the use-cases I'm interested in, it's not an important feature. If libvirt needs a local wrapper for interesting use cases, then it has failed. You can't have a local wrapper with the esx driver, for example. This is off-topic, but can you detail why you don't want remote dispatch (I assume we're talking about a multiple node deployment). Because there are dozens of remote management APIs and then all have a concept of agents that run on the end nodes. When fitting virtualization management into an existing management infrastructure, you are going to always use a local API. If you use a remote management API, you then have to worry about security, authorization, and authentication. Just supporting SASL really isn't enough because there are a lot fewer actual deployments of kerberos out there (most of them are actually Active Directory and SASL alone is not enough to interoperate with AD). 3) Provide a unified and interoperable view of the world for non-libvirt and libvirt users This problem can be solved by the non-libvirt users adopting libvirt, or the libvirt users dropping libvirt. I don't understand why we need to add interoperability between users who choose an interoperability library and users who don't choose an interoperability library. What I'd like to avoid is user confusion. Should a user use libvirt or libqemu? If they make a decision to use libqemu and then down the road want to use libvirt, how hard is it to switch? Fragmentation hurts the ecosystem and discourages good applications from existing. I think it's our responsibility to ensure there's a good management API that exists for qemu that we can actively recommend to our users. libvirt is very good at typical virtualization uses of qemu but qemu is much more than just that and has lots of advanced features. Every typical virtualization use will eventually grow some non-typical requirements. If libvirt explicitly refuses to support qemu features, I don't see how we can recommend it - even if it satisfies a user's requirements today, what about tomorrow? what about future qemu feature, will they be exposed or not? If that is the case then we should develop qemud (which libvirt and other apps can use). (even if it isn't the case I think qemud is a good idea) Yeah, that's where I'm at. I'd eventually like libvirt to use our provided API and I can see where it would add value to the stack (by doing things like storage and network management). That's not what the libvirt community wants to do. We're very bias. We've made decisions about how features should be exposed and what features should be included. We want all of those features exposed exactly how we've implemented them because we think it's the right way. I'm not sure there's an obvious way forward unless we decide that there is going to be two ways to interact with qemu. One way is through the libvirt world-view and the other is through a more qemu centric view. The problem then becomes allowing those two models to co-exist happily together. I don't think there's a point in managing qemu through libvirt and directly in parallel. It means a user has to learn both APIs, and for every operation they need to check both to see what's the best way of exploiting the feature. There will invariably be some friction. Layers need to stack on top of each other, not live side by side or bypass each other. I agree with you theoretically but practically, I think it's immensely useful as a stop-gap. The alternative is to get libvirt to just act as a thin layer to expose qemu features directly. But honestly, what's the point of libvirt if they did that? For most hypervisors, that's exactly what libvirt does
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/23/2010 09:33 PM, Anthony Liguori wrote: This is a different ambiguity, about the semantic results of the commands, where as I'm refering to the execution order. If I look at a libvirt log file and see a set of JSON commands logged, I want to know that this ordering from the logs, was indeed the same as order in which qemu processed them. If you have two separate monitor connection you can't be sure of the order of execution. It is key for our bug troubleshooting that given a libvirt log file, we can replay the JSON commands again and get the same results. Two monitor connections is just increasing complexity of code without any tangible benefit. I think you're assuming direct access to the second monitor? I'm not suggesting that. I'm suggesting that libvirt is still the one submitting commands to the second monitor and that it submits those commands in lock step. What about protocol extensions? For instance, pretend libvirt doesn't support async messages, what would it do when it receives one from the user's monitor? -- error compiling committee.c: too many arguments to function -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/25/2010 06:39 AM, Anthony Liguori wrote: On 04/24/2010 04:46 AM, Avi Kivity wrote: On 04/23/2010 09:29 PM, Anthony Liguori wrote: Maybe. We'll still have issues. For example, sVirt: if a QMP command names a labeled resource, the non-libvirt user will have no way of knowing how to label it. This is orthogonal to QMP and has to do strictly with how libvirt prepares a resource for qemu. It's not orthogonal. If you allow qmp access behind libvirt's back, it's a problem that you will have. My point was, if libvirt is just exposing raw qemu features, then it should be possible for qemu to arbitrate concurrent access. If libvirt implements features on top of qemu, then no other third party will be able to co-exist with those features without interacting with qemu. It's an impossible problem for qemu to solve (arbitrating access to state stored in a third party management app). If libvirt implement features (like sVirt or network configuration) then it is indeed impossible for qemu to arbitrate. If we take all those features into qemu[d], then it becomes possible to arbitrate so long as the libvirt and the other management app don't step on each others toes. But that's impossible to guarantee if you upgrade your libvirt while keeping the other app unchanged. 1) Allow libvirt users to access features of qemu that are not exposed through libvirt That's an artificial problem. If libvirt exposes all features, you don't need to solve it. It won't. Otherwise, we wouldn't be having this discussion. Then libvirt will fade into uselessness. A successful app using libvirt will grow, and will have new requirements. As soon as libvirt doesn't meet those new requirements, the app will need to talk to qemu[d] directly. Once it does that, it may as well use qemu[d] for everything; if you can talk QMP and generate qemu command lines, there's not much that libvirt buys you. Even the cross-hypervisor support is not that hard to implement, especially if you only need to satisfy your own requirements. 2) Provide a means for non-libvirt users to interact with qemu We have qmp. It doesn't do multiple guest management. I think it's reasonable to have a qemud which does (and also does sVirt and the zillion other things libvirt does) provided we remove them from libvirt (long term). The only problem is that it's a lot of effort. It depends on what things you think are important. A lot of libvirt's complexity is based on the fact that it uses a daemon and needs to deal with the security implications of that. You don't need explicit labelling if you don't use a daemon. I don't follow. If you have multiple guests that you want off each other's turf you have to label their resources, either statically or dynamically. How is it related to a daemon being present? This is really the qemu model (as opposed to the xend model). (and the qemud model). In theory, it does support this with the session urls but they are currently second-class citizens in libvirt. The remote dispatch also adds a fair bit of complexity and at least for the use-cases I'm interested in, it's not an important feature. If libvirt needs a local wrapper for interesting use cases, then it has failed. You can't have a local wrapper with the esx driver, for example. This is off-topic, but can you detail why you don't want remote dispatch (I assume we're talking about a multiple node deployment). 3) Provide a unified and interoperable view of the world for non-libvirt and libvirt users This problem can be solved by the non-libvirt users adopting libvirt, or the libvirt users dropping libvirt. I don't understand why we need to add interoperability between users who choose an interoperability library and users who don't choose an interoperability library. What I'd like to avoid is user confusion. Should a user use libvirt or libqemu? If they make a decision to use libqemu and then down the road want to use libvirt, how hard is it to switch? Fragmentation hurts the ecosystem and discourages good applications from existing. I think it's our responsibility to ensure there's a good management API that exists for qemu that we can actively recommend to our users. libvirt is very good at typical virtualization uses of qemu but qemu is much more than just that and has lots of advanced features. Every typical virtualization use will eventually grow some non-typical requirements. If libvirt explicitly refuses to support qemu features, I don't see how we can recommend it - even if it satisfies a user's requirements today, what about tomorrow? what about future qemu feature, will they be exposed or not? If that is the case then we should develop qemud (which libvirt and other apps can use). (even if it isn't the case I think qemud is a good idea) For (1), we all agree that the best case scenario would be for libvirt to support every qemu fe
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/24/2010 04:46 AM, Avi Kivity wrote: On 04/23/2010 09:29 PM, Anthony Liguori wrote: Maybe. We'll still have issues. For example, sVirt: if a QMP command names a labeled resource, the non-libvirt user will have no way of knowing how to label it. This is orthogonal to QMP and has to do strictly with how libvirt prepares a resource for qemu. It's not orthogonal. If you allow qmp access behind libvirt's back, it's a problem that you will have. My point was, if libvirt is just exposing raw qemu features, then it should be possible for qemu to arbitrate concurrent access. If libvirt implements features on top of qemu, then no other third party will be able to co-exist with those features without interacting with qemu. It's an impossible problem for qemu to solve (arbitrating access to state stored in a third party management app). 1) Allow libvirt users to access features of qemu that are not exposed through libvirt That's an artificial problem. If libvirt exposes all features, you don't need to solve it. It won't. Otherwise, we wouldn't be having this discussion. 2) Provide a means for non-libvirt users to interact with qemu We have qmp. It doesn't do multiple guest management. I think it's reasonable to have a qemud which does (and also does sVirt and the zillion other things libvirt does) provided we remove them from libvirt (long term). The only problem is that it's a lot of effort. It depends on what things you think are important. A lot of libvirt's complexity is based on the fact that it uses a daemon and needs to deal with the security implications of that. You don't need explicit labelling if you don't use a daemon. This is really the qemu model (as opposed to the xend model). In theory, it does support this with the session urls but they are currently second-class citizens in libvirt. The remote dispatch also adds a fair bit of complexity and at least for the use-cases I'm interested in, it's not an important feature. 3) Provide a unified and interoperable view of the world for non-libvirt and libvirt users This problem can be solved by the non-libvirt users adopting libvirt, or the libvirt users dropping libvirt. I don't understand why we need to add interoperability between users who choose an interoperability library and users who don't choose an interoperability library. What I'd like to avoid is user confusion. Should a user use libvirt or libqemu? If they make a decision to use libqemu and then down the road want to use libvirt, how hard is it to switch? Fragmentation hurts the ecosystem and discourages good applications from existing. I think it's our responsibility to ensure there's a good management API that exists for qemu that we can actively recommend to our users. libvirt is very good at typical virtualization uses of qemu but qemu is much more than just that and has lots of advanced features. For (1), we all agree that the best case scenario would be for libvirt to support every qemu feature. I think we can also all agree though that this is not really practical and certainly not practical for developers since there is a development cost associated with libvirt support (to model an API appropriately). All except me, perhaps. We already have two layers of feature modeling: first, we mostly emulate real life, not invent new features. PCI hotplug existed long before qemu had support for it. Second, we do give some thought into how we expose it through QMP. libvirt doesn't have to invent it again, it only has to expose it through its lovely xml and C APIs. That's not what the libvirt community wants to do. We're very bias. We've made decisions about how features should be exposed and what features should be included. We want all of those features exposed exactly how we've implemented them because we think it's the right way. I'm not sure there's an obvious way forward unless we decide that there is going to be two ways to interact with qemu. One way is through the libvirt world-view and the other is through a more qemu centric view. The problem then becomes allowing those two models to co-exist happily together. The alternative is to get libvirt to just act as a thin layer to expose qemu features directly. But honestly, what's the point of libvirt if they did that? For the most part, the pool management is not virtualization specific and you could have libvirt provide that functionality without it knowing a thing about qemu. The new API proposed addresses (1) by allowing a user to drill down to the QMP context. It's a good solution IMHO and I think we all agree that there's an inherent risk to this that users will have to evaluate on a case-by-case basis. It's a good stop-gap though. Agree. (2) is largely addressed by QMP and a config file. I'd like to see a nice C library, but I think a lot of other folks are happy with JSON support in higher le
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/23/2010 09:29 PM, Anthony Liguori wrote: Maybe. We'll still have issues. For example, sVirt: if a QMP command names a labeled resource, the non-libvirt user will have no way of knowing how to label it. This is orthogonal to QMP and has to do strictly with how libvirt prepares a resource for qemu. It's not orthogonal. If you allow qmp access behind libvirt's back, it's a problem that you will have. Much better to exact a commitment from libvirt to track all QMP (and command line) capabilities. Instead of adding cleverness to QMP, add APIs to libvirt. Let's step back for a minute because I think we're missing the forest through the trees. We're trying to address a few distinct problems: 1) Allow libvirt users to access features of qemu that are not exposed through libvirt That's an artificial problem. If libvirt exposes all features, you don't need to solve it. 2) Provide a means for non-libvirt users to interact with qemu We have qmp. It doesn't do multiple guest management. I think it's reasonable to have a qemud which does (and also does sVirt and the zillion other things libvirt does) provided we remove them from libvirt (long term). The only problem is that it's a lot of effort. 3) Provide a unified and interoperable view of the world for non-libvirt and libvirt users This problem can be solved by the non-libvirt users adopting libvirt, or the libvirt users dropping libvirt. I don't understand why we need to add interoperability between users who choose an interoperability library and users who don't choose an interoperability library. For (1), we all agree that the best case scenario would be for libvirt to support every qemu feature. I think we can also all agree though that this is not really practical and certainly not practical for developers since there is a development cost associated with libvirt support (to model an API appropriately). All except me, perhaps. We already have two layers of feature modeling: first, we mostly emulate real life, not invent new features. PCI hotplug existed long before qemu had support for it. Second, we do give some thought into how we expose it through QMP. libvirt doesn't have to invent it again, it only has to expose it through its lovely xml and C APIs. The new API proposed addresses (1) by allowing a user to drill down to the QMP context. It's a good solution IMHO and I think we all agree that there's an inherent risk to this that users will have to evaluate on a case-by-case basis. It's a good stop-gap though. Agree. (2) is largely addressed by QMP and a config file. I'd like to see a nice C library, but I think a lot of other folks are happy with JSON support in higher level languages. I agree with them. C is a pretty bad choice for managing qemu (or even, C is a pretty bad choice). (3) is the place where there are still potential challenges. I think at the very least, our goal should be to enable conversion from (2) and (1) to be as easy as possible. That's why I have proposed implementing a C library for the JSON transport because we could plumb that through the new libvirt API. This would allow a user to very quickly port an application from QMP to libvirt. In order to do this, we need the libvirt API to expose a dedicated monitor because we'll need to be able to manipulate events and negotiate features. Most likely any application that talks QMP will hide the protocol behind a function call interface anyway. Beyond simple porting, there's a secondary question of having non-libvirt apps co-exist with libvirt apps. I think it's a good long term goal, but I don't think we should worry too much about it now. libvirt needs to either support all but the most esoteric use cases, or to get out of the way completely. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/23/2010 09:21 AM, Daniel P. Berrange wrote: Say libvirt is running a 'offline core dump' operation. This consists of us invoking stop migrate exec:cat> foo.dump cont I don't want other debug commands accidentally being issued in between these steps. These 3 commands are in essence considered transactional block. Internally our locking model ensures that no other APIs can be invoked in this monitor connection during this sequence. Thus if someone uses the libvirt debug API to issue a monitor command, thus command will be blocked until the last 'cont' command is issued here. This provides us a known command ordering in the logs. Ah, so what'd I'd suggest is that you still use two monitors but that you use the same locking for both. The reason a second monitor is useful is that you can allow different event masks to be setup on the second monitor which I think a QMP client will expect. I don't see any trouble with respect to command ordering here. libvirt is ultimately the one sending commands to monitor. You know which one was sent and which one completed in what order. With a single monitor connection, libvirt's current locking model for the monitor ensures that QMP monitor commands are reliably serialized onto the wire, giving unambiguous behaviour. Except that libvirt doesn't know what the side effects of the debug commands are so it's intrinsically ambiguous :-) This is a different ambiguity, about the semantic results of the commands, where as I'm refering to the execution order. If I look at a libvirt log file and see a set of JSON commands logged, I want to know that this ordering from the logs, was indeed the same as order in which qemu processed them. If you have two separate monitor connection you can't be sure of the order of execution. It is key for our bug troubleshooting that given a libvirt log file, we can replay the JSON commands again and get the same results. Two monitor connections is just increasing complexity of code without any tangible benefit. I think you're assuming direct access to the second monitor? I'm not suggesting that. I'm suggesting that libvirt is still the one submitting commands to the second monitor and that it submits those commands in lock step. Regards, Anthony Liguori -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/23/2010 09:24 AM, Avi Kivity wrote: On 04/23/2010 04:48 PM, Anthony Liguori wrote: On 04/23/2010 07:48 AM, Avi Kivity wrote: On 04/22/2010 09:49 PM, Anthony Liguori wrote: real API. Say, adding a device libvirt doesn't know about or stopping the VM while libvirt thinks it's still running or anything like that. Another problem is issuing Monitor commands that could confuse libvirt's We need to make libvirt and qemu smarter. We already face this problem today with multiple libvirt users. This is why sophisticated management mechanisms (like LDAP) have mechanisms to do transactions or at least a series of atomic operations. And people said qmp/json was overengineered... But seriously, transactions won't help anything. qemu maintains state, and when you have two updaters touching a shared variable not excepting each other to, things break, no matter how much locking there is. Let's consider some concrete examples. I'm using libvirt and QMP and in QMP, I want to hot unplug a device. Today, I do this by listing the pci devices, and issuing a pci_del that takes a PCI address. This is intrinsically racy though because in the worst case scenario, in between when I enumerate pci devices and do the pci_del in QMP, in libvirt, I've done a pci_del and then a pci_add within libvirt of a completely different device. Obviously you should do the pci_del through libvirt. Once libvirt supports an API, use it. It was just an example... There are a few ways to solve this, the simplest being that we give devices unique ids that are never reused and instead of pci_del taking a pci bus address, it takes a device id. That would address this race. You can get very far by just being clever about unique ids and notifications. There are some cases where a true RMW may be required but I can't really think of one off hand. The way LDAP addresses this is that it has a batched operation and a simple set of boolean comparison operations. This lets you execute a batched operation that will do a RMW. I'm sure we can be very clever, but I'd rather direct this cleverness to qemu core issues, not to the QMP (which in turn requires that users be clever to use it correctly). QMP is a low bandwidth protocol, so races will never show up in testing. We're laying mines here for users to step on that we will never encounter ourselves. The only way that separate monitors could work is if they touch completely separate state, which is difficult to ensure if you upgrade your libvirt. I don't think this is as difficult of a problem as you think it is. If you look at Active Directory and the whole set of management tools based on it, they certainly allow concurrent management applications. You can certainly get into trouble still but with just some careful considerations, you can make two management applications work together 90% of the time without much fuss on the applications part. Maybe. We'll still have issues. For example, sVirt: if a QMP command names a labeled resource, the non-libvirt user will have no way of knowing how to label it. This is orthogonal to QMP and has to do strictly with how libvirt prepares a resource for qemu. Much better to exact a commitment from libvirt to track all QMP (and command line) capabilities. Instead of adding cleverness to QMP, add APIs to libvirt. Let's step back for a minute because I think we're missing the forest through the trees. We're trying to address a few distinct problems: 1) Allow libvirt users to access features of qemu that are not exposed through libvirt 2) Provide a means for non-libvirt users to interact with qemu 3) Provide a unified and interoperable view of the world for non-libvirt and libvirt users For (1), we all agree that the best case scenario would be for libvirt to support every qemu feature. I think we can also all agree though that this is not really practical and certainly not practical for developers since there is a development cost associated with libvirt support (to model an API appropriately). The new API proposed addresses (1) by allowing a user to drill down to the QMP context. It's a good solution IMHO and I think we all agree that there's an inherent risk to this that users will have to evaluate on a case-by-case basis. It's a good stop-gap though. (2) is largely addressed by QMP and a config file. I'd like to see a nice C library, but I think a lot of other folks are happy with JSON support in higher level languages. (3) is the place where there are still potential challenges. I think at the very least, our goal should be to enable conversion from (2) and (1) to be as easy as possible. That's why I have proposed implementing a C library for the JSON transport because we could plumb that through the new libvirt API. This would allow a user to very quickly port an application from QMP to libvirt. In order to do this, we need the li
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
Anthony Liguori writes: > On 04/23/2010 07:48 AM, Avi Kivity wrote: >> On 04/22/2010 09:49 PM, Anthony Liguori wrote: real API. Say, adding a device libvirt doesn't know about or stopping the VM while libvirt thinks it's still running or anything like that. >>> Another problem is issuing Monitor commands that could confuse >>> libvirt's >>> >>> We need to make libvirt and qemu smarter. >>> >>> We already face this problem today with multiple libvirt users. >>> This is why sophisticated management mechanisms (like LDAP) have >>> mechanisms to do transactions or at least a series of atomic >>> operations. >> >> And people said qmp/json was overengineered... >> >> But seriously, transactions won't help anything. qemu maintains >> state, and when you have two updaters touching a shared variable not >> excepting each other to, things break, no matter how much locking >> there is. > > Let's consider some concrete examples. I'm using libvirt and QMP and > in QMP, I want to hot unplug a device. > > Today, I do this by listing the pci devices, and issuing a pci_del > that takes a PCI address. This is intrinsically racy though because > in the worst case scenario, in between when I enumerate pci devices > and do the pci_del in QMP, in libvirt, I've done a pci_del and then a > pci_add within libvirt of a completely different device. > > There are a few ways to solve this, the simplest being that we give > devices unique ids that are never reused and instead of pci_del taking > a pci bus address, it takes a device id. That would address this > race. For what it's worth, that's how device_del works, except the ID is chosen by the user, who is free to reuse them. Really tangential to this thread, but it's worth repeating anyway: use device_add & friends, not pci_add & friends. [...] -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Fri, Apr 23, 2010 at 05:24:34PM +0300, Avi Kivity wrote: > On 04/23/2010 04:48 PM, Anthony Liguori wrote: > >On 04/23/2010 07:48 AM, Avi Kivity wrote: > >>On 04/22/2010 09:49 PM, Anthony Liguori wrote: > real API. Say, adding a device libvirt doesn't know about or > stopping the VM > while libvirt thinks it's still running or anything like that. > >>> Another problem is issuing Monitor commands that could confuse > >>>libvirt's > >>> > >>>We need to make libvirt and qemu smarter. > >>> > >>>We already face this problem today with multiple libvirt users. > >>>This is why sophisticated management mechanisms (like LDAP) have > >>>mechanisms to do transactions or at least a series of atomic > >>>operations. > >> > >>And people said qmp/json was overengineered... > >> > >>But seriously, transactions won't help anything. qemu maintains > >>state, and when you have two updaters touching a shared variable not > >>excepting each other to, things break, no matter how much locking > >>there is. > > > >Let's consider some concrete examples. I'm using libvirt and QMP and > >in QMP, I want to hot unplug a device. > > > >Today, I do this by listing the pci devices, and issuing a pci_del > >that takes a PCI address. This is intrinsically racy though because > >in the worst case scenario, in between when I enumerate pci devices > >and do the pci_del in QMP, in libvirt, I've done a pci_del and then a > >pci_add within libvirt of a completely different device. > > Obviously you should do the pci_del through libvirt. Once libvirt > supports an API, use it. > > > > >There are a few ways to solve this, the simplest being that we give > >devices unique ids that are never reused and instead of pci_del taking > >a pci bus address, it takes a device id. That would address this race. > > > >You can get very far by just being clever about unique ids and > >notifications. There are some cases where a true RMW may be required > >but I can't really think of one off hand. The way LDAP addresses this > >is that it has a batched operation and a simple set of boolean > >comparison operations. This lets you execute a batched operation that > >will do a RMW. > > I'm sure we can be very clever, but I'd rather direct this cleverness to > qemu core issues, not to the QMP (which in turn requires that users be > clever to use it correctly). QMP is a low bandwidth protocol, so races > will never show up in testing. We're laying mines here for users to > step on that we will never encounter ourselves. > > > > >> The only way that separate monitors could work is if they touch > >>completely separate state, which is difficult to ensure if you > >>upgrade your libvirt. > >> > > > >I don't think this is as difficult of a problem as you think it is. > >If you look at Active Directory and the whole set of management tools > >based on it, they certainly allow concurrent management applications. > >You can certainly get into trouble still but with just some careful > >considerations, you can make two management applications work together > >90% of the time without much fuss on the applications part. > > Maybe. We'll still have issues. For example, sVirt: if a QMP command > names a labeled resource, the non-libvirt user will have no way of > knowing how to label it. > > Much better to exact a commitment from libvirt to track all QMP (and > command line) capabilities. Instead of adding cleverness to QMP, add > APIs to libvirt. Agreed. Despite adding this monitor / XML passthrough capability, we still do not want apps to be using this at all. If there is some capability missing that apps need then the default mode of operation is to add the neccessary bits of libvirt. The monitor/XML pasthrough is just a short term quick workaround until the official support is done. As such I do not really think we need to put huge amounts of effort in the wierd complex racey edge cases. The effort is better spent on getting the features in libvirt. Regards, Daniel -- |: Red Hat, Engineering, London-o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org-o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Fri, Apr 23, 2010 at 08:48:51AM -0500, Anthony Liguori wrote: > On 04/23/2010 07:48 AM, Avi Kivity wrote: > >On 04/22/2010 09:49 PM, Anthony Liguori wrote: > >>>real API. Say, adding a device libvirt doesn't know about or > >>>stopping the VM > >>>while libvirt thinks it's still running or anything like that. > >> Another problem is issuing Monitor commands that could confuse > >>libvirt's > >> > >>We need to make libvirt and qemu smarter. > >> > >>We already face this problem today with multiple libvirt users. This > >>is why sophisticated management mechanisms (like LDAP) have > >>mechanisms to do transactions or at least a series of atomic operations. > > > >And people said qmp/json was overengineered... > > > >But seriously, transactions won't help anything. qemu maintains > >state, and when you have two updaters touching a shared variable not > >excepting each other to, things break, no matter how much locking > >there is. > > Let's consider some concrete examples. I'm using libvirt and QMP and in > QMP, I want to hot unplug a device. > > Today, I do this by listing the pci devices, and issuing a pci_del that > takes a PCI address. This is intrinsically racy though because in the > worst case scenario, in between when I enumerate pci devices and do the > pci_del in QMP, in libvirt, I've done a pci_del and then a pci_add > within libvirt of a completely different device. This is what already happens with any QEMU >= 0.12, where libvirt uses the new -device syntax with its 'id' parameter for all devices, and then uses 'device_id $ID' for unplug. The app still has to be careful it doesn't try to add a device using the same naming scheme as a device libvirt uses. This is easy enough though if they prepend some random string to all device IDs thy use, that won't clash with libvirt device ID names. Regards, Daniel -- |: Red Hat, Engineering, London-o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org-o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/23/2010 04:48 PM, Anthony Liguori wrote: On 04/23/2010 07:48 AM, Avi Kivity wrote: On 04/22/2010 09:49 PM, Anthony Liguori wrote: real API. Say, adding a device libvirt doesn't know about or stopping the VM while libvirt thinks it's still running or anything like that. Another problem is issuing Monitor commands that could confuse libvirt's We need to make libvirt and qemu smarter. We already face this problem today with multiple libvirt users. This is why sophisticated management mechanisms (like LDAP) have mechanisms to do transactions or at least a series of atomic operations. And people said qmp/json was overengineered... But seriously, transactions won't help anything. qemu maintains state, and when you have two updaters touching a shared variable not excepting each other to, things break, no matter how much locking there is. Let's consider some concrete examples. I'm using libvirt and QMP and in QMP, I want to hot unplug a device. Today, I do this by listing the pci devices, and issuing a pci_del that takes a PCI address. This is intrinsically racy though because in the worst case scenario, in between when I enumerate pci devices and do the pci_del in QMP, in libvirt, I've done a pci_del and then a pci_add within libvirt of a completely different device. Obviously you should do the pci_del through libvirt. Once libvirt supports an API, use it. There are a few ways to solve this, the simplest being that we give devices unique ids that are never reused and instead of pci_del taking a pci bus address, it takes a device id. That would address this race. You can get very far by just being clever about unique ids and notifications. There are some cases where a true RMW may be required but I can't really think of one off hand. The way LDAP addresses this is that it has a batched operation and a simple set of boolean comparison operations. This lets you execute a batched operation that will do a RMW. I'm sure we can be very clever, but I'd rather direct this cleverness to qemu core issues, not to the QMP (which in turn requires that users be clever to use it correctly). QMP is a low bandwidth protocol, so races will never show up in testing. We're laying mines here for users to step on that we will never encounter ourselves. The only way that separate monitors could work is if they touch completely separate state, which is difficult to ensure if you upgrade your libvirt. I don't think this is as difficult of a problem as you think it is. If you look at Active Directory and the whole set of management tools based on it, they certainly allow concurrent management applications. You can certainly get into trouble still but with just some careful considerations, you can make two management applications work together 90% of the time without much fuss on the applications part. Maybe. We'll still have issues. For example, sVirt: if a QMP command names a labeled resource, the non-libvirt user will have no way of knowing how to label it. Much better to exact a commitment from libvirt to track all QMP (and command line) capabilities. Instead of adding cleverness to QMP, add APIs to libvirt. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Fri, Apr 23, 2010 at 08:40:49AM -0500, Anthony Liguori wrote: > On 04/23/2010 05:28 AM, Daniel P. Berrange wrote: > >On Thu, Apr 22, 2010 at 01:45:27PM -0500, Anthony Liguori wrote: > > > >>On 04/09/2010 09:27 AM, Daniel P. Berrange wrote: > >> > >>>On Fri, Apr 09, 2010 at 09:41:39AM -0400, Chris Lalancette wrote: > >>> > >>> > >>> > > myguest > ... > > > > qemu arguments > > newname > REGEXP > foo=on > -bar > > > > > > > >>>The concept of command line& monitor is something that is QEMU specific > >>>and thus is not suitable for the primary XML schema. IMHO, this needs to > >>>be > >>>done as a separate schema, linked in via an XML namespace. For example > >>> > >>>>>> xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0";> > >>> myguest > >>> ... > >>> > >>> -device > >>> lsi > >>> > >>> > >>> > >>> > >>I think it's problematic to focus too much on command line arguments. > >>We are not introducing new command line arguments to qemu for the most > >>part that aren't usable in the config file. > >> > >Currently libvirt doesn't use any config file at all, so command line > >is the only possible option. > > The config file is really just a structured command line argument so the > two are really equivalent. > > From an XML perspective, the advantage of using the config file format > is that it's very well structured whereas the command line arguments > aren't. It's the difference between: > > > -drive > file=foo.img,if=virtio,cache=off > > > And: > > > foo.img > virtio > off > > > The later being much more friendly for things like XPath and XQuery. > > > We're going to evaluate switching to the > >config file at some point& so can easily add further XML options to > >allow direct setting of config file entries at that point. > > I don't necessary think there's a super compelling reason for you to > switch to pure config. Some things you'll have to do via -device and > -set but it's still reasonable to keep using the command line. A key > requirement of the config file effort was that everything that can be > done via config file be doable via command line. The main reason we're interested in the config is that it uses a better structured format, so we don't have such hairy escaping problems to deal with. Also, the command line is getting outrageously long these days with -device syntax. > >>With respect to injecting QMP commands directly, I think the proposed > >>debug API is probably reasonable. We could build a libqemu that used > >>that API as a transport which means that one could use libqemu and > >>libvirt simultaneously which is certainly a key requirement of mine. > >> > >>I think it's important that it's a dedicated monitor session though. It > >>shouldn't just be injecting commands within an existing QMP session IMHO. > >> > >I think the opposite actually. If libvirt had two open monitor connections, > >one for normal use& one for injection, then its open to racey usage where > >2 monitor commands may be issued concurrently& it is tricky to determine > >just which will be processed first, with all the scope for unexpected > >behaviour this entails. > > Can you give an explicit example? The nature of this debug extension is > such that I don't think 1 monitor really helps this problem... Say libvirt is running a 'offline core dump' operation. This consists of us invoking stop migrate exec:cat > foo.dump cont I don't want other debug commands accidentally being issued in between these steps. These 3 commands are in essence considered transactional block. Internally our locking model ensures that no other APIs can be invoked in this monitor connection during this sequence. Thus if someone uses the libvirt debug API to issue a monitor command, thus command will be blocked until the last 'cont' command is issued here. This provides us a known command ordering in the logs. > > With a single monitor connection, libvirt's current > >locking model for the monitor ensures that QMP monitor commands are > >reliably > >serialized onto the wire, giving unambiguous behaviour. > > Except that libvirt doesn't know what the side effects of the debug > commands are so it's intrinsically ambiguous :-) This is a different ambiguity, about the semantic results of the commands, where as I'm refering to the execution order. If I look at a libvirt log file and see a set of JSON commands logged, I want to know that this ordering from the logs, was indeed the same as order in which qemu processed them. If you have two separate monitor connection you can't be sure of the order of execution. It is key for our bug troubleshooting that given a libvirt log file, we can replay the JSON commands again and get the same results.
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/23/2010 07:48 AM, Avi Kivity wrote: On 04/22/2010 09:49 PM, Anthony Liguori wrote: real API. Say, adding a device libvirt doesn't know about or stopping the VM while libvirt thinks it's still running or anything like that. Another problem is issuing Monitor commands that could confuse libvirt's We need to make libvirt and qemu smarter. We already face this problem today with multiple libvirt users. This is why sophisticated management mechanisms (like LDAP) have mechanisms to do transactions or at least a series of atomic operations. And people said qmp/json was overengineered... But seriously, transactions won't help anything. qemu maintains state, and when you have two updaters touching a shared variable not excepting each other to, things break, no matter how much locking there is. Let's consider some concrete examples. I'm using libvirt and QMP and in QMP, I want to hot unplug a device. Today, I do this by listing the pci devices, and issuing a pci_del that takes a PCI address. This is intrinsically racy though because in the worst case scenario, in between when I enumerate pci devices and do the pci_del in QMP, in libvirt, I've done a pci_del and then a pci_add within libvirt of a completely different device. There are a few ways to solve this, the simplest being that we give devices unique ids that are never reused and instead of pci_del taking a pci bus address, it takes a device id. That would address this race. You can get very far by just being clever about unique ids and notifications. There are some cases where a true RMW may be required but I can't really think of one off hand. The way LDAP addresses this is that it has a batched operation and a simple set of boolean comparison operations. This lets you execute a batched operation that will do a RMW. The only way that separate monitors could work is if they touch completely separate state, which is difficult to ensure if you upgrade your libvirt. I don't think this is as difficult of a problem as you think it is. If you look at Active Directory and the whole set of management tools based on it, they certainly allow concurrent management applications. You can certainly get into trouble still but with just some careful considerations, you can make two management applications work together 90% of the time without much fuss on the applications part. Regards, Anthony Liguori -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/23/2010 05:28 AM, Daniel P. Berrange wrote: On Thu, Apr 22, 2010 at 01:45:27PM -0500, Anthony Liguori wrote: On 04/09/2010 09:27 AM, Daniel P. Berrange wrote: On Fri, Apr 09, 2010 at 09:41:39AM -0400, Chris Lalancette wrote: myguest ... qemu arguments newname REGEXP foo=on -bar The concept of command line& monitor is something that is QEMU specific and thus is not suitable for the primary XML schema. IMHO, this needs to be done as a separate schema, linked in via an XML namespace. For example http://libvirt.org/schemas/domain/qemu/1.0";> myguest ... -device lsi I think it's problematic to focus too much on command line arguments. We are not introducing new command line arguments to qemu for the most part that aren't usable in the config file. Currently libvirt doesn't use any config file at all, so command line is the only possible option. The config file is really just a structured command line argument so the two are really equivalent. From an XML perspective, the advantage of using the config file format is that it's very well structured whereas the command line arguments aren't. It's the difference between: -drive file=foo.img,if=virtio,cache=off And: foo.img virtio off The later being much more friendly for things like XPath and XQuery. We're going to evaluate switching to the config file at some point& so can easily add further XML options to allow direct setting of config file entries at that point. I don't necessary think there's a super compelling reason for you to switch to pure config. Some things you'll have to do via -device and -set but it's still reasonable to keep using the command line. A key requirement of the config file effort was that everything that can be done via config file be doable via command line. With respect to injecting QMP commands directly, I think the proposed debug API is probably reasonable. We could build a libqemu that used that API as a transport which means that one could use libqemu and libvirt simultaneously which is certainly a key requirement of mine. I think it's important that it's a dedicated monitor session though. It shouldn't just be injecting commands within an existing QMP session IMHO. I think the opposite actually. If libvirt had two open monitor connections, one for normal use& one for injection, then its open to racey usage where 2 monitor commands may be issued concurrently& it is tricky to determine just which will be processed first, with all the scope for unexpected behaviour this entails. Can you give an explicit example? The nature of this debug extension is such that I don't think 1 monitor really helps this problem... With a single monitor connection, libvirt's current locking model for the monitor ensures that QMP monitor commands are reliably serialized onto the wire, giving unambiguous behaviour. Except that libvirt doesn't know what the side effects of the debug commands are so it's intrinsically ambiguous :-) Regards, Anthony Liguori Regards, Daniel -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/22/2010 09:49 PM, Anthony Liguori wrote: real API. Say, adding a device libvirt doesn't know about or stopping the VM while libvirt thinks it's still running or anything like that. Another problem is issuing Monitor commands that could confuse libvirt's We need to make libvirt and qemu smarter. We already face this problem today with multiple libvirt users. This is why sophisticated management mechanisms (like LDAP) have mechanisms to do transactions or at least a series of atomic operations. And people said qmp/json was overengineered... But seriously, transactions won't help anything. qemu maintains state, and when you have two updaters touching a shared variable not excepting each other to, things break, no matter how much locking there is. The only way that separate monitors could work is if they touch completely separate state, which is difficult to ensure if you upgrade your libvirt. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Thu, Apr 22, 2010 at 01:47:55PM -0500, Anthony Liguori wrote: > On 04/12/2010 07:23 AM, Jamie Lokier wrote: > >Daniel Veillard wrote: > > > >>On Sun, Apr 11, 2010 at 11:17:38PM +0100, Jamie Lokier wrote: > >> > >>>It's not that hard to write this for trivial extra options: > >>> > >>>/bin/sh -c 'qemu "$0" "$@" -extra-flag' > >>> > >>>(if that works). > >>> > >> That won't work because we expect the emulator to be a path to > >>the emulator program, so yes that has to be done in the wrapper script. > >> > >Ok. Being able to pass arguments would be about infinitely more > >useful, so that you don't need a separate wrapper script for every > >individual guest configuration - you can pass options to the single > >wrapper for each variation. > > > >Even/path/to/wrapper qemu-0.11 is a big improvement > >over/path/to/wrapper-guest-config-6-with-qemu-0.11 > >;-) > > > >Some simple but versatile hook ideas: > > > >- (no space splitting, one option, appended) > >- (space splitting multiple options) > >- > >- > >-VALUE > > > > I'd strongly suggest not focusing on manipulating command line > arguments. I think in the not too distant future, few people will pass > command line arguments as opposed to just using configuration files. Agreed, we're not having anything in the XML for modifying command line arguments explicitly. In tiny edge case where someone actually needs that, QEMU already provides the '-set OPTION' command line arg that you can use to override any qdev property. Daniel -- |: Red Hat, Engineering, London-o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org-o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Thu, Apr 22, 2010 at 01:45:27PM -0500, Anthony Liguori wrote: > On 04/09/2010 09:27 AM, Daniel P. Berrange wrote: > >On Fri, Apr 09, 2010 at 09:41:39AM -0400, Chris Lalancette wrote: > > > > > >> > >> myguest > >> ... > >> > >> > >> > >> qemu arguments > >> > >> newname > >> REGEXP > >> foo=on > >> -bar > >> > >> > >> > >> > >> > >The concept of command line& monitor is something that is QEMU specific > >and thus is not suitable for the primary XML schema. IMHO, this needs to be > >done as a separate schema, linked in via an XML namespace. For example > > > >> xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0";> > > myguest > > ... > > > > -device > > lsi > > > > > > > > I think it's problematic to focus too much on command line arguments. > We are not introducing new command line arguments to qemu for the most > part that aren't usable in the config file. Currently libvirt doesn't use any config file at all, so command line is the only possible option. We're going to evaluate switching to the config file at some point & so can easily add further XML options to allow direct setting of config file entries at that point. There's no problem supporting both command line & config file syntax in the XML. The command line does have the immediate advantage that 99% of all documentation about QEMU on the web today refers to command line args so that's what people are most familiar with. > With respect to injecting QMP commands directly, I think the proposed > debug API is probably reasonable. We could build a libqemu that used > that API as a transport which means that one could use libqemu and > libvirt simultaneously which is certainly a key requirement of mine. > > I think it's important that it's a dedicated monitor session though. It > shouldn't just be injecting commands within an existing QMP session IMHO. I think the opposite actually. If libvirt had two open monitor connections, one for normal use & one for injection, then its open to racey usage where 2 monitor commands may be issued concurrently & it is tricky to determine just which will be processed first, with all the scope for unexpected behaviour this entails. With a single monitor connection, libvirt's current locking model for the monitor ensures that QMP monitor commands are reliably serialized onto the wire, giving unambiguous behaviour. Regards, Daniel -- |: Red Hat, Engineering, London-o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org-o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/22/10 20:47, Anthony Liguori wrote: > On 04/12/2010 07:23 AM, Jamie Lokier wrote: >> Some simple but versatile hook ideas: >> >> - (no space splitting, one option, >> appended) >> - (space splitting multiple options) >> - >> - >> -VALUE > > I'd strongly suggest not focusing on manipulating command line > arguments. I think in the not too distant future, few people will pass > command line arguments as opposed to just using configuration files. > > We are very close to being able to cover 90% of current users via the > config file. There will always be a place for adding command line parameters for debugging purposes. When you try to debug a problem, the first port of call is to look at the command line arguments libvirt uses to launch qemu and replicate those with addons. It's a nightmare today and you end up losing networking and other things. Having an optional debug options thing that one can set quickly from virt-manager would make it so much easier to retry different settings fast while replicating the setup that is normally run. Having to manually edit xml files for making changes is just not a viable solution. Speaking of which, one problem with the current XML format is that it relies on hard-coding the path to QEMU. Instead if should rely on the system default and provide an optional tag for debugging purposes. That would make it easier to migrate specifications from one OS to another, without manually having to edit the QEMU tag. Cheers, Jes -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/22/2010 01:45 PM, Anthony Liguori wrote: Which in turn, could be embedded as: myguest ... Operon_G3 5 AuthenticAMD With respect to injecting QMP commands directly, I think the proposed debug API is probably reasonable. We could build a libqemu that used that API as a transport which means that one could use libqemu and libvirt simultaneously which is certainly a key requirement of mine. I think it's important that it's a dedicated monitor session though. It shouldn't just be injecting commands within an existing QMP session IMHO. Another important thing to mention is that this type of conversion has two purposes. A user can inject qemu config snippets into the libvirt description, but libvirt can also inject qemu config snippets that it cannot represent via it's own XML for the user. For instance, let's say that a user adds a disk via QMP with an option that libvirt doesn't recognize. When a virsh dumpxml is executed, libvirt could display the XML using the tag instead of the tag that it normally would use. Regards, Anthony Liguori Regards, Anthony Liguori -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/12/2010 10:20 AM, Luiz Capitulino wrote: On Fri, 9 Apr 2010 15:27:17 +0100 "Daniel P. Berrange" wrote: On Fri, Apr 09, 2010 at 09:41:39AM -0400, Chris Lalancette wrote: Hello, In response to a lot of the talk of qemud lately on qemu-devel, the libvirt community would like to put forward a proposal to help enable debug/advanced options when using various hypervisors. The goals of this API are: 1) To enable more rapid access to hypervisor features before proper libvirt API's are designed around them. 2) To facilitate debugging and access to advanced features that may not fit into the normal libvirt world-view. Caveats: 1) Unlike other libvirt API's, this one will explicitly *not* be guaranteed ABI/API compatible between libvirt updates. I think we'd still aim to keep the public API stable. The bit that we can't guarentee is the interactions with QEMU& the libvirt driver. eg, if someone was using the API to send text monitor commands to QEMU, and the next libvirt release switched to JSON mode, that use would break. The API would still be valid, but the way they use it might not be. Similarly if they add some custom extra command line argument, this could potentially conflict with an extra command line arg a subsquent libvirt release used. eg, they used -device to add a PCI card with a specific PCI address. Then next libvirt release specifies this same PCI address and then you get a clash. Another problem is issuing Monitor commands that could confuse libvirt's real API. Say, adding a device libvirt doesn't know about or stopping the VM while libvirt thinks it's still running or anything like that. We need to make libvirt and qemu smarter. We already face this problem today with multiple libvirt users. This is why sophisticated management mechanisms (like LDAP) have mechanisms to do transactions or at least a series of atomic operations. Regards, Anthony Liguori When I suggested the monitor passthrough idea, I thought it could be a workaround for the feature time lag problem, but as stated by Avi, I'm afraid that this can potentially cause more problems than it solves. It's very desirable for debugging (item 2 above) and maybe to try new things, but people couldn't use it for anything serious, which is the major issue in this subject (item 1 above). Now, Anthony's idea of having libqemu has the same issues, which seems to mean that the only way to really fix the time lag is to fix the time lag :) Raw access to the qemu monitor will be disabled by default; the tag enables the ability to send QMP (or text, if you are using older qemu) messages straight through to the monitor. To do this there will be an additional API entry point named virDomainDebugCommand() which takes an arbitrary string and passes it to the monitor, and returns an arbitrary string as a result. Thus you could pass in either "info cpus" if using the text monitor or '{ "execute": "query-cpus" }' if using QMP. Again the idea of a 'virDomainDebugCommand' API is QEMU specific, with other hypervisors have different approaches for low level extension/ debug. For example, Xen would involve XenStore access, or XenD XMLRPC, etc. So this should really live in a separate API namespace which is specific to a hypervisor. For example, as a header file #include Containing APIs like int virDomainQEMUInvokeMonitor(virDomainPtr dom, const char *command, char **reply); typedef virConnectQEMUDomainEventCallback(virConnectPtr conn, virDomainPtr dom, const char *eventname, const char *data, void *opaque) int virConnectQEMUDomainEventRegister(virConnectPtr conn, virDomainPtr dom, const char *eventname, virDomainQEMUMonitorCallback cb, void *opaque); What about all the wonders of json? This will force clients to have their own json<-> text converter, maybe we could provide two APIs, just like qemu driver works internally? -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/12/2010 07:23 AM, Jamie Lokier wrote: Daniel Veillard wrote: On Sun, Apr 11, 2010 at 11:17:38PM +0100, Jamie Lokier wrote: It's not that hard to write this for trivial extra options: /bin/sh -c 'qemu "$0" "$@" -extra-flag' (if that works). That won't work because we expect the emulator to be a path to the emulator program, so yes that has to be done in the wrapper script. Ok. Being able to pass arguments would be about infinitely more useful, so that you don't need a separate wrapper script for every individual guest configuration - you can pass options to the single wrapper for each variation. Even/path/to/wrapper qemu-0.11 is a big improvement over/path/to/wrapper-guest-config-6-with-qemu-0.11 ;-) Some simple but versatile hook ideas: - (no space splitting, one option, appended) - (space splitting multiple options) - - -VALUE I'd strongly suggest not focusing on manipulating command line arguments. I think in the not too distant future, few people will pass command line arguments as opposed to just using configuration files. We are very close to being able to cover 90% of current users via the config file. Regards, Anthony Liguori -/path/to/script This last one is to call the script, but pass all the options including the path to whichever version of qemu/kvm would get selected by libvirt. (I presume it has some selection mechanism, given the variety of guests which break with each version of qemu and/or qemu-kvm and/or kvm?) (It is most unfortunate that XML forces those long prefixes everywhere due to lack of contextual naming, unlike less verbose config formats :-/) Parsing libvirt output and having to guess which option corresponds to what from the libvirt config sounds very fragile and also a rather large amount of effort for something which should be easy. it's not that easy because we change the way we build the command line for qemu as qemu evolves. for example before and after the support for -device there have been a lot of changes. I agree, which is why it's much less fragile if libvirt provides the individual parts to hooks/wrappers, if that's asked, rather than forcing the wrapper to parse libvirts output and guess what libvirt does from version to version. For sure, before and after -device, any script which is involved with those options will probably have to change. But that's quite unusual. Most usually I'd expect things like the order of -device options might change depending on how libvirt pulled together it's backing resources, for example. -- Jamie -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/09/2010 09:27 AM, Daniel P. Berrange wrote: On Fri, Apr 09, 2010 at 09:41:39AM -0400, Chris Lalancette wrote: Hello, In response to a lot of the talk of qemud lately on qemu-devel, the libvirt community would like to put forward a proposal to help enable debug/advanced options when using various hypervisors. The goals of this API are: 1) To enable more rapid access to hypervisor features before proper libvirt API's are designed around them. 2) To facilitate debugging and access to advanced features that may not fit into the normal libvirt world-view. Caveats: 1) Unlike other libvirt API's, this one will explicitly *not* be guaranteed ABI/API compatible between libvirt updates. I think we'd still aim to keep the public API stable. The bit that we can't guarentee is the interactions with QEMU& the libvirt driver. eg, if someone was using the API to send text monitor commands to QEMU, and the next libvirt release switched to JSON mode, that use would break. The API would still be valid, but the way they use it might not be. Similarly if they add some custom extra command line argument, this could potentially conflict with an extra command line arg a subsquent libvirt release used. eg, they used -device to add a PCI card with a specific PCI address. Then next libvirt release specifies this same PCI address and then you get a clash. So in both cases the API and XML format can be reasonably guarenteed between releases. What we can't guarentee is that usage of these features will be reliable across releases of libvirt. 2) Again unlike other libvirt API's, access and configuration of the debug section of a domain will be highly hypervisor dependent. 3) Application developers will be strongly discouraged from using this API because of the above 2 issues. To help in this, the API's will be in a separate library that developers will explicitly have to link to, and it will have a different (but largely compatible) wire protocol. In terms of wire protocol, this would still have to run over the same existing data channel, because we don't want apps to have to open up multiple connections. Fortunately our protocol is designed to allow this kind of extension struct remote_message_header { unsigned prog; /* REMOTE_PROGRAM */ unsigned vers; /* REMOTE_PROTOCOL_VERSION */ remote_procedure proc; /* REMOTE_PROC_x */ remote_message_type type; unsigned serial;/* Serial number of message. */ remote_message_status status; }; We currently have a single program 'REMOTE_PROGRAM' and its associated 'vers' and 'proc' numbers. What we would be doing is to define a new program, say QEMU_PROGRAM. The 'remote_procedure proc' numbers can thus start again from '1' without clashing with any of the existing APIs. All the RPC marshalling/demarshalling code should work more or less unchanged. We just hook in a different function dispatch table in the daemon for that program. 4) We don't expect this API to solve all of the issues brought up during the qemud discussion. Our initial goal is just to give ready access of the qemu command-line and monitor to developers. With that being said, our initial proposal follows. We expect this to evolve over time as we get more feedback, but we think this proposal addresses at least 2 of the major pain points qemu developers have while trying to use libvirt. The initial debug XML for qemu would be: I wouldn't call them 'debug' options, since that's just one of possible use cases for this. What we're really doing there is exposing hypervisor specific features, so we should just be upfront about that in our description / naming. myguest ... qemu arguments newname REGEXP foo=on -bar The concept of command line& monitor is something that is QEMU specific and thus is not suitable for the primary XML schema. IMHO, this needs to be done as a separate schema, linked in via an XML namespace. For example http://libvirt.org/schemas/domain/qemu/1.0";> myguest ... -device lsi I think it's problematic to focus too much on command line arguments. We are not introducing new command line arguments to qemu for the most part that aren't usable in the config file. What I really think we would like, is a way to express the qemu configuration file in terms of XML and then to use snippets of that within an XML namespace. For instance: [cpudef] name = "Opteron_G3" level = "5" vendor = "AuthenticAMD" ... Could convert to: Operon_G3 5 AuthenticAMD ... Which in turn, could be embedded as: myguest ... Operon_G3 5 AuthenticAMD With respect to injecting QMP commands directly, I think the proposed debug API is probably reasonable. We could build a libqemu that used that API as a transport which
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Fri, 9 Apr 2010 15:27:17 +0100 "Daniel P. Berrange" wrote: > On Fri, Apr 09, 2010 at 09:41:39AM -0400, Chris Lalancette wrote: > > Hello, > > In response to a lot of the talk of qemud lately on qemu-devel, the > > libvirt community would like to put forward a proposal to help enable > > debug/advanced options when using various hypervisors. The goals of > > this API are: > > > > 1) To enable more rapid access to hypervisor features before proper > > libvirt API's are designed around them. > > 2) To facilitate debugging and access to advanced features that may > > not fit into the normal libvirt world-view. > > > > Caveats: > > 1) Unlike other libvirt API's, this one will explicitly *not* be > > guaranteed ABI/API compatible between libvirt updates. > > I think we'd still aim to keep the public API stable. The bit that we > can't guarentee is the interactions with QEMU & the libvirt driver. > eg, if someone was using the API to send text monitor commands to > QEMU, and the next libvirt release switched to JSON mode, that use > would break. The API would still be valid, but the way they use it > might not be. Similarly if they add some custom extra command line > argument, this could potentially conflict with an extra command line > arg a subsquent libvirt release used. eg, they used -device to add > a PCI card with a specific PCI address. Then next libvirt release > specifies this same PCI address and then you get a clash. Another problem is issuing Monitor commands that could confuse libvirt's real API. Say, adding a device libvirt doesn't know about or stopping the VM while libvirt thinks it's still running or anything like that. When I suggested the monitor passthrough idea, I thought it could be a workaround for the feature time lag problem, but as stated by Avi, I'm afraid that this can potentially cause more problems than it solves. It's very desirable for debugging (item 2 above) and maybe to try new things, but people couldn't use it for anything serious, which is the major issue in this subject (item 1 above). Now, Anthony's idea of having libqemu has the same issues, which seems to mean that the only way to really fix the time lag is to fix the time lag :) > > Raw access to the qemu monitor will be disabled by default; the > > tag enables the ability to send QMP (or > > text, if you are using older qemu) messages straight through to the > > monitor. To do this there will be an additional API entry point > > named virDomainDebugCommand() which takes an arbitrary string > > and passes it to the monitor, and returns an arbitrary string as > > a result. Thus you could pass in either "info cpus" if using the > > text monitor or '{ "execute": "query-cpus" }' if using QMP. > > Again the idea of a 'virDomainDebugCommand' API is QEMU specific, with > other hypervisors have different approaches for low level extension/ > debug. For example, Xen would involve XenStore access, or XenD XMLRPC, > etc. So this should really live in a separate API namespace which is > specific to a hypervisor. For example, as a header file > > #include > > Containing APIs like > > int virDomainQEMUInvokeMonitor(virDomainPtr dom, > const char *command, > char **reply); > > typedef virConnectQEMUDomainEventCallback(virConnectPtr conn, > virDomainPtr dom, > const char *eventname, > const char *data, > void *opaque) > int virConnectQEMUDomainEventRegister(virConnectPtr conn, > virDomainPtr dom, > const char *eventname, > virDomainQEMUMonitorCallback cb, > void *opaque); What about all the wonders of json? This will force clients to have their own json <-> text converter, maybe we could provide two APIs, just like qemu driver works internally? -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Mon, Apr 12, 2010 at 09:56:50AM -0400, Chris Lalancette wrote: > On 04/12/2010 08:41 AM, Daniel P. Berrange wrote: > >>> I don't think there's much to be gained from having an XML element to > >>> turn on/off use of these APIs. If an app doesn't want to use them, it > >>> can simply not link to libvirt-qemu.so > >> > >> The reason I wanted to do this was mostly for debug/support reasons. > >> That is, with this element in place we can easily tell from the dumpxml > >> output whether a person was using the "unreliable" API's, and thus we can > >> tell them to try and reproduce without that in place. > > > > That doesn't tell you whether they have actually used any API or not. > > It is also inconvenient if you start a guest without it, and only later > > realize you want to use the extra APIs. If we want to track the actual > > usage, then the first time a direct monitor command is issued, we should > > simply log a warning message. > > The problem with logging a message is that it is easy to lose it. What I'm > trying to avoid here is debugging somebody's setup for hours only to find > out that they did a "pci_del" behind libvirt's back. Maybe we can just make > the > a read-only flag; it is ignored in the parsing, but > it is set by "GetXMLDesc" when it detects that the virDomainQemuInvokeMonitor > has been called. If the logging message is at level 'WARN' it will be included in syslog for libvirtd by default, which is basic support data always collected. The guest XML is a description of the guest configuration, which is not a place for ad-hoc flags related to API usage. Daniel -- |: Red Hat, Engineering, London-o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org-o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/12/2010 08:41 AM, Daniel P. Berrange wrote: >>> I don't think there's much to be gained from having an XML element to >>> turn on/off use of these APIs. If an app doesn't want to use them, it >>> can simply not link to libvirt-qemu.so >> >> The reason I wanted to do this was mostly for debug/support reasons. >> That is, with this element in place we can easily tell from the dumpxml >> output whether a person was using the "unreliable" API's, and thus we can >> tell them to try and reproduce without that in place. > > That doesn't tell you whether they have actually used any API or not. > It is also inconvenient if you start a guest without it, and only later > realize you want to use the extra APIs. If we want to track the actual > usage, then the first time a direct monitor command is issued, we should > simply log a warning message. The problem with logging a message is that it is easy to lose it. What I'm trying to avoid here is debugging somebody's setup for hours only to find out that they did a "pci_del" behind libvirt's back. Maybe we can just make the a read-only flag; it is ignored in the parsing, but it is set by "GetXMLDesc" when it detects that the virDomainQemuInvokeMonitor has been called. -- Chris Lalancette -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Mon, Apr 12, 2010 at 01:23:08PM +0100, Jamie Lokier wrote: > Daniel Veillard wrote: > > On Sun, Apr 11, 2010 at 11:17:38PM +0100, Jamie Lokier wrote: > > > Parsing libvirt output and having to guess which option corresponds to > > > what from the libvirt config sounds very fragile and also a rather > > > large amount of effort for something which should be easy. > > > > it's not that easy because we change the way we build the command line > > for qemu as qemu evolves. for example before and after the support for > > -device there have been a lot of changes. > > I agree, which is why it's much less fragile if libvirt provides the > individual parts to hooks/wrappers, if that's asked, rather than > forcing the wrapper to parse libvirts output and guess what libvirt > does from version to version. > > For sure, before and after -device, any script which is involved with > those options will probably have to change. But that's quite unusual. I could easily see this changing again in the not too distant future. eg, we might switch to writing out a temporary config file with all options and just passing qemu -readonly /var/lib/libvirt/qemu/guest.cfg to avoid the QEMU arg parsing / escaping problems. Or we might stop using command line at all for devices, and use monitor device_add to setup all devices. Allowing ability to append extra args to the QEMU command line is reasonably safe, while support for hooking into existing command line args is not. Daniel -- |: Red Hat, Engineering, London-o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org-o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Sun, Apr 11, 2010 at 11:17:38PM +0100, Jamie Lokier wrote: > Richard W.M. Jones wrote: > > On Fri, Apr 09, 2010 at 10:06:51PM +0100, Jamie Lokier wrote: > > > Daniel P. Berrange wrote: > > > > I think this alteration of existing args is fr too complex & > > > > fragile, > > > > and way overkill. > > > > > > Would it not be simpler, for the target audience, for the config to > > > contain a one-line shell script to transform particular matched > > > arguments in any way that's wanted? > > > > I might be missing something but I thought you could already do this > > in current libvirt. ie. Change the element [1] to point to > > your shell script: > > > > /usr/local/bin/my-qemu-wrapper.sh > > > > The wrapper should get called with the command line arguments and you > > can alter them however you like. > > > > Isn't that right, Dan? > > That forces the shell script to be relatively complicated and > recognise all qemu arguments (including version specific ones) to skip > the ones with parameters - and to know where to look for the real > target executable is. > > It's not a very friendly way to add or change parts of specific > subsystems, for example modifying a blockdev parameter, but it can be > done that way if necessary. > > It's not that hard to write this for trivial extra options: > >/bin/sh -c 'qemu "$0" "$@" -extra-flag' > > (if that works). > > But for things like modifications to particular devices, wouldn't it > be better to associate "add cache=special option" to a section about > one of the blockdevs which libvirt knows about, rather than the awful > prospect of parsing libvirt's output to guess which of three -device > arguments corresponds with the second SCSI drive mentioned in the > libvirt config - which might not necessarily be the second on the > command line, even. Every device has a unique alias in the XML, which maps onto the 'id=' parameter for QEMU so you can match up them. We need to consider the tradeoff between what's likely to be neccessary vs cost of supporting the extensions. Allowing addition of arbitrary extra command lines is a trivial extension with low support cost & covers the most pressing feature timelag problems. Munging existing dev options is a less common scenario with significantly higher support burden which I don't really consider worth explicitly supporting beyond the existing ability to add a wrapper script. Daniel -- |: Red Hat, Engineering, London-o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org-o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
Daniel Veillard wrote: > On Sun, Apr 11, 2010 at 11:17:38PM +0100, Jamie Lokier wrote: > > It's not that hard to write this for trivial extra options: > > > >/bin/sh -c 'qemu "$0" "$@" -extra-flag' > > > > (if that works). > > That won't work because we expect the emulator to be a path to > the emulator program, so yes that has to be done in the wrapper script. Ok. Being able to pass arguments would be about infinitely more useful, so that you don't need a separate wrapper script for every individual guest configuration - you can pass options to the single wrapper for each variation. Even /path/to/wrapper qemu-0.11 is a big improvement over /path/to/wrapper-guest-config-6-with-qemu-0.11 ;-) Some simple but versatile hook ideas: - (no space splitting, one option, appended) - (space splitting multiple options) - - - VALUE - /path/to/script This last one is to call the script, but pass all the options including the path to whichever version of qemu/kvm would get selected by libvirt. (I presume it has some selection mechanism, given the variety of guests which break with each version of qemu and/or qemu-kvm and/or kvm?) (It is most unfortunate that XML forces those long prefixes everywhere due to lack of contextual naming, unlike less verbose config formats :-/) > > Parsing libvirt output and having to guess which option corresponds to > > what from the libvirt config sounds very fragile and also a rather > > large amount of effort for something which should be easy. > > it's not that easy because we change the way we build the command line > for qemu as qemu evolves. for example before and after the support for > -device there have been a lot of changes. I agree, which is why it's much less fragile if libvirt provides the individual parts to hooks/wrappers, if that's asked, rather than forcing the wrapper to parse libvirts output and guess what libvirt does from version to version. For sure, before and after -device, any script which is involved with those options will probably have to change. But that's quite unusual. Most usually I'd expect things like the order of -device options might change depending on how libvirt pulled together it's backing resources, for example. -- Jamie -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Sun, Apr 11, 2010 at 11:17:38PM +0100, Jamie Lokier wrote: > Richard W.M. Jones wrote: > > On Fri, Apr 09, 2010 at 10:06:51PM +0100, Jamie Lokier wrote: > > > Daniel P. Berrange wrote: > > > > I think this alteration of existing args is fr too complex & > > > > fragile, > > > > and way overkill. > > > > > > Would it not be simpler, for the target audience, for the config to > > > contain a one-line shell script to transform particular matched > > > arguments in any way that's wanted? > > > > I might be missing something but I thought you could already do this > > in current libvirt. ie. Change the element [1] to point to > > your shell script: > > > > /usr/local/bin/my-qemu-wrapper.sh > > > > The wrapper should get called with the command line arguments and you > > can alter them however you like. > > > > Isn't that right, Dan? > > That forces the shell script to be relatively complicated and > recognise all qemu arguments (including version specific ones) to skip > the ones with parameters - and to know where to look for the real > target executable is. > > It's not a very friendly way to add or change parts of specific > subsystems, for example modifying a blockdev parameter, but it can be > done that way if necessary. > > It's not that hard to write this for trivial extra options: > >/bin/sh -c 'qemu "$0" "$@" -extra-flag' > > (if that works). That won't work because we expect the emulator to be a path to the emulator program, so yes that has to be done in the wrapper script. > Parsing libvirt output and having to guess which option corresponds to > what from the libvirt config sounds very fragile and also a rather > large amount of effort for something which should be easy. it's not that easy because we change the way we build the command line for qemu as qemu evolves. for example before and after the support for -device there have been a lot of changes. > And then there's modifying how network tap devices are set up and > other networking, outside qemu, but requested by libvirt. Doing that > in sounds like a bad idea, but script hooks are the only > sane way to make networking fit every environment and requirement. > Ideally without forcing the hook writer to reimplement the normal > network setup from scratch just to tweak it a bit. I apologise if > libvirt already provides such hooks - I haven't looked at that part of > it. Well it's very new, and I still didn't wrote the documentation for it so that's notmal, not your fault :-) Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ dan...@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/ -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
Richard W.M. Jones wrote: > On Fri, Apr 09, 2010 at 10:06:51PM +0100, Jamie Lokier wrote: > > Daniel P. Berrange wrote: > > > I think this alteration of existing args is fr too complex & fragile, > > > and way overkill. > > > > Would it not be simpler, for the target audience, for the config to > > contain a one-line shell script to transform particular matched > > arguments in any way that's wanted? > > I might be missing something but I thought you could already do this > in current libvirt. ie. Change the element [1] to point to > your shell script: > > /usr/local/bin/my-qemu-wrapper.sh > > The wrapper should get called with the command line arguments and you > can alter them however you like. > > Isn't that right, Dan? That forces the shell script to be relatively complicated and recognise all qemu arguments (including version specific ones) to skip the ones with parameters - and to know where to look for the real target executable is. It's not a very friendly way to add or change parts of specific subsystems, for example modifying a blockdev parameter, but it can be done that way if necessary. It's not that hard to write this for trivial extra options: /bin/sh -c 'qemu "$0" "$@" -extra-flag' (if that works). But for things like modifications to particular devices, wouldn't it be better to associate "add cache=special option" to a section about one of the blockdevs which libvirt knows about, rather than the awful prospect of parsing libvirt's output to guess which of three -device arguments corresponds with the second SCSI drive mentioned in the libvirt config - which might not necessarily be the second on the command line, even. Parsing libvirt output and having to guess which option corresponds to what from the libvirt config sounds very fragile and also a rather large amount of effort for something which should be easy. And then there's modifying how network tap devices are set up and other networking, outside qemu, but requested by libvirt. Doing that in sounds like a bad idea, but script hooks are the only sane way to make networking fit every environment and requirement. Ideally without forcing the hook writer to reimplement the normal network setup from scratch just to tweak it a bit. I apologise if libvirt already provides such hooks - I haven't looked at that part of it. -- Jamie -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On Fri, Apr 09, 2010 at 10:06:51PM +0100, Jamie Lokier wrote: > Daniel P. Berrange wrote: > > I think this alteration of existing args is fr too complex & fragile, > > and way overkill. > > Would it not be simpler, for the target audience, for the config to > contain a one-line shell script to transform particular matched > arguments in any way that's wanted? I might be missing something but I thought you could already do this in current libvirt. ie. Change the element [1] to point to your shell script: /usr/local/bin/my-qemu-wrapper.sh The wrapper should get called with the command line arguments and you can alter them however you like. Isn't that right, Dan? Rich. [1] http://libvirt.org/formatdomain.html#elementsDevices -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming blog: http://rwmj.wordpress.com Fedora now supports 80 OCaml packages (the OPEN alternative to F#) http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/09/2010 11:30 PM, Eric Blake wrote: Yeah, having the ability to specify an optional wrapper script, that receives the name of the normal interpreter and all the arguments the normal interpreter would have been given, sounds like the ultimate in flexibility at a minimum of xml. You can also just have it piped into sed. Paolo -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
On 04/09/2010 03:06 PM, Jamie Lokier wrote: > Daniel P. Berrange wrote: >> I think this alteration of existing args is fr too complex & fragile, >> and way overkill. > > Would it not be simpler, for the target audience, for the config to > contain a one-line shell script to transform particular matched > arguments in any way that's wanted? Yeah, having the ability to specify an optional wrapper script, that receives the name of the normal interpreter and all the arguments the normal interpreter would have been given, sounds like the ultimate in flexibility at a minimum of xml. It comes at the expense of pushing the burden of flexibility onto the shoulders of whoever writes the wrapper script, but that seems okay to me. -- Eric Blake ebl...@redhat.com+1-801-349-2682 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
Daniel P. Berrange wrote: > I think this alteration of existing args is fr too complex & fragile, > and way overkill. Would it not be simpler, for the target audience, for the config to contain a one-line shell script to transform particular matched arguments in any way that's wanted? > If the arg that libvirt generates isn't what someone > needs, then remove the bit of the guest config responsible for that and > add a complete extra arg, rather than munging the existing one. Problem with that: You might want libvirt to think it has a device that it's controlling, still show in the GUI, still tie up with the rest of the stack, still do the management setup, but you just want to change some qemu-specific parameters. For example, cache=somethingelse for block devices, iptables=tobedefined for net devices, usbimplementation=hack for USB devices. -- Jamie -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list