Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
Hi, I think that would work well for spice. Spice uses shared memory from the pci device for both the framebuffer and surfaces/commands, but this is Is that the only DMA do you do? That's good for this model. Yes. Spice does both reads and writes though, so a way to tag pages as dirty is needed. cheers, Gerd
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 04/26/2011 04:14 AM, Gerd Hoffmann wrote: Hi, I think that would work well for spice. Spice uses shared memory from the pci device for both the framebuffer and surfaces/commands, but this is Is that the only DMA do you do? That's good for this model. Yes. Spice does both reads and writes though, so a way to tag pages as dirty is needed. Just implementing Spice as it currently is in a separate process isn't going to be useful IMHO. I would think that the best approach would be to parse all of the ring requests in QEMU itself, and issue higher level commands to a separate process. You can still have the video memory segment mapped in a separate process but QEMU should know enough about what's going on to take care of dirtying the memory. Sort of like how we deal with SCSI passthrough. We interpret enough of the command to hand it off to something else and then handle the return logic. Having QEMU as an intermediary is important to preserve our current security model. We shouldn't be passing unsanitized guest input to an external process. Regards, Anthony Liguori cheers, Gerd
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/02/11 14:49, Michael Roth wrote: On 03/02/2011 07:18 AM, Jes Sorensen wrote: I think we need two types for sure, even for the video case, we will still need a control channel as well. However, I don't think it is desirable to split things up more than we have to, so if we can keep it within one client process that is good. Maybe there are cases where it makes sense to split it into more processes, I could be convinced, but I think we really need to be careful making it too much of a complex mess either. Yup, if it's doable I'd prefer a single client process as well. Just hard to predict how difficult it'll be to support 2 or more mechanisms. Although, I'd imagine we'd end up with something like qemu's io loop, with event-driven shmem and fd-based work, which does seem doable. That is pretty much what I had in mind. Will have to see how it works out, but I think it is very feasible :) Cheers, Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 02/28/11 18:44, Anthony Liguori wrote: On Feb 28, 2011 10:44 AM, Jes Sorensen jes.soren...@redhat.com wrote: Separating host-side virtagent and other tasks from core QEMU = To improve auditing of the core QEMU code, it would be ideal to be able to separate the core QEMU functionality from utility functionality by moving the utility functionality into its own process. This process will be referred to as the QEMU client below. Separating as in moving code outside of qemu.git, making code optionally built in, making code optionally built in or loadable as a separate executable that is automatically launched, or making code always built outside the main executable? I'm very nervous about having a large number of daemons necessary to run QEMU. I think a reasonable approach would be a single front-end daemond. Once QAPI is merged, there is a very incremental approach we can take for this sort of work. Take your favorite subsystem (like gdbstub or SDL) and make it only use QMP apis. Once we're only using QMP internally in a subsystem, then building it out of the main QEMU and using libqmp should be fairly easy. Hi Anthony, Back from a day off playing with power drills and concrete walls :) Sorry I should have made it a little more clear, it was obvious to me, but not written down: The idea is to keep everything as part of the QEMU package, ie. part of qemu.git. My idea is really to have one QEMU host daemon and one QEMU client, which provides the various services. You type make and you get two binaries instead of one. We could allow other daemons to connect to the host daemon, but that wouldn't be a primary target for this in my book, and I am not sure we really want to do this. It is absolutely vital for me that we do not make things much more complicated for users with this move. I don't want to get into a situation where we start forcing external packages or daemons in order to run basic QEMU. If we start requiring such, we have failed! However, I think it is a reasonable compromise to have one daemon you launch, and then a simple client you launch straight after which will then provide the same/similar views and interfaces that users are used to when they launch current QEMU (monitor, vnc server etc). I hope this clarifies things. Cheers, Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/01/11 13:07, Dor Laor wrote: On 02/28/2011 07:44 PM, Anthony Liguori wrote: I'm very nervous about having a large number of daemons necessary to run QEMU. I think a reasonable approach would be a single front-end daemond. s/daemon/son processes/ Qemu is the one that should spawn them and they should be transparent from the management. This way running qemu stays the same and qemu just need to add the logic to get a SIGCHILD and potentially re-execute an dead son process. I disagree here, I do not want to require QEMU to spawn the new processes. Having a daemon you can use to connect will provide more flexibility and isn't unreasonably complex. Cheers, Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/01/11 15:25, Dor Laor wrote: On 03/01/2011 02:40 PM, Anthony Liguori wrote: On Mar 1, 2011 7:07 AM, Dor Laor dl...@redhat.com Qemu is the one that should spawn them and they should be transparent from the management. This way running qemu stays the same and qemu just need to add the logic to get a SIGCHILD and potentially re-execute an dead son process. Spice is the logical place to start, no? It's the largest single dependency we have and it does some scary things with qemu_mutex. I would use spice as a way to prove the concept. I agree it is desirable to the this for spice but it is allot more complex than virtagent isolation. Spice is performance sensitive and contains much more state. It needs to access the guest memory for reading the surfaces. It can be solved but needs some major changes. Adding spice-devel to the discussion. I had a few thoughts about this already, which I think will work for both spice and vnc. What we could do is to expose the video memory via shared memory. That way a spice or vnc daemon could get direct access to the memory, this would limit communication to keyboard/mouse events, as well as video mode info, and possibly notifications to the client about which ranges of memory have been updated. Using shared memory this way should allow us to implement the video clients without performance loss, in fact it should be beneficial since it would allow them to run fully separate from the host daemon. Cheers, Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/01/11 15:25, Dor Laor wrote: On 03/01/2011 02:40 PM, Anthony Liguori wrote: Spice is the logical place to start, no? It's the largest single dependency we have and it does some scary things with qemu_mutex. I would use spice as a way to prove the concept. I agree it is desirable to the this for spice but it is allot more complex than virtagent isolation. Spice is performance sensitive and contains much more state. It needs to access the guest memory for reading the surfaces. It can be solved but needs some major changes. Adding spice-devel to the discussion. Please don't add broken misconfigured mailing lists, which require moderator or subscription, to discussions on public lists. Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/02/2011 12:28 PM, Jes Sorensen wrote: On 03/01/11 15:25, Dor Laor wrote: On 03/01/2011 02:40 PM, Anthony Liguori wrote: Spice is the logical place to start, no? It's the largest single dependency we have and it does some scary things with qemu_mutex. I would use spice as a way to prove the concept. I agree it is desirable to the this for spice but it is allot more complex than virtagent isolation. Spice is performance sensitive and contains much more state. It needs to access the guest memory for reading the surfaces. It can be solved but needs some major changes. Adding spice-devel to the discussion. Please don't add broken misconfigured mailing lists, which require moderator or subscription, to discussions on public lists. Isn't it simpler to ask the spice list maintainer (Alon Levy, CCed) to do it? Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/02/11 11:42, Dor Laor wrote: On 03/02/2011 12:28 PM, Jes Sorensen wrote: On 03/01/11 15:25, Dor Laor wrote: I agree it is desirable to the this for spice but it is allot more complex than virtagent isolation. Spice is performance sensitive and contains much more state. It needs to access the guest memory for reading the surfaces. It can be solved but needs some major changes. Adding spice-devel to the discussion. Please don't add broken misconfigured mailing lists, which require moderator or subscription, to discussions on public lists. Isn't it simpler to ask the spice list maintainer (Alon Levy, CCed) to do it? We cannot have a discussion on a mailing list if we constantly have to wait for the list maintainer to approve it. There is really zero justification for closing a development list of an open source project. Spam is handled in other ways, this doesn't solve it. Cheers, Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/02/2011 12:25 PM, Jes Sorensen wrote: On 03/01/11 15:25, Dor Laor wrote: On 03/01/2011 02:40 PM, Anthony Liguori wrote: On Mar 1, 2011 7:07 AM, Dor Laordl...@redhat.com Qemu is the one that should spawn them and they should be transparent from the management. This way running qemu stays the same and qemu just need to add the logic to get a SIGCHILD and potentially re-execute an dead son process. Spice is the logical place to start, no? It's the largest single dependency we have and it does some scary things with qemu_mutex. I would use spice as a way to prove the concept. I agree it is desirable to the this for spice but it is allot more complex than virtagent isolation. Spice is performance sensitive and contains much more state. It needs to access the guest memory for reading the surfaces. It can be solved but needs some major changes. Adding spice-devel to the discussion. I had a few thoughts about this already, which I think will work for both spice and vnc. What we could do is to expose the video memory via shared memory. That way a spice or vnc daemon could get direct access to the memory, this would limit communication to keyboard/mouse events, as well as video mode info, and possibly notifications to the client about which ranges of memory have been updated. Using shared memory this way should allow us to implement the video clients without performance loss, in fact it should be beneficial since it would allow them to run fully separate from the host daemon. Why do you call it a daemon? Each VM instance should have only one, the 'host daemon' naming is misleading. The proper solution long term is to sandbox qemu in a way that there privileged mode and non privileged mode. It might be implemented using separate address space or not. Most operations like vnc/rpc/spice/usb should be run with less privileges. The main issue is that doing it right will take time and we'll want virt-agent be merged before the long term solution is ready. The best approach would be gradual development Cheers, Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On Wed, Mar 02, 2011 at 11:25:44AM +0100, Jes Sorensen wrote: On 03/01/11 15:25, Dor Laor wrote: On 03/01/2011 02:40 PM, Anthony Liguori wrote: On Mar 1, 2011 7:07 AM, Dor Laor dl...@redhat.com Qemu is the one that should spawn them and they should be transparent from the management. This way running qemu stays the same and qemu just need to add the logic to get a SIGCHILD and potentially re-execute an dead son process. Spice is the logical place to start, no? It's the largest single dependency we have and it does some scary things with qemu_mutex. I would use spice as a way to prove the concept. I agree it is desirable to the this for spice but it is allot more complex than virtagent isolation. Spice is performance sensitive and contains much more state. It needs to access the guest memory for reading the surfaces. It can be solved but needs some major changes. Adding spice-devel to the discussion. I had a few thoughts about this already, which I think will work for both spice and vnc. What we could do is to expose the video memory via shared memory. That way a spice or vnc daemon could get direct access to the memory, this would limit communication to keyboard/mouse events, as well as video mode info, and possibly notifications to the client about which ranges of memory have been updated. Using shared memory this way should allow us to implement the video clients without performance loss, in fact it should be beneficial since it would allow them to run fully separate from the host daemon. I think that would work well for spice. Spice uses shared memory from the pci device for both the framebuffer and surfaces/commands, but this is not really relevant at this level. What about IO and irq? that would add additional latencies, no? because each io exit would need to be ipc'ed over to the spice/vnc process? and same way in the other direction, requesting qemu to trigger an interrupt in the next vm entry. Cheers, Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/02/11 11:56, Dor Laor wrote: On 03/02/2011 12:25 PM, Jes Sorensen wrote: On 03/01/11 15:25, Dor Laor wrote: Using shared memory this way should allow us to implement the video clients without performance loss, in fact it should be beneficial since it would allow them to run fully separate from the host daemon. Why do you call it a daemon? Each VM instance should have only one, the 'host daemon' naming is misleading. I refer to it as a daemon because it is something the client(s) will connect to. But yes, there will be a daemon per VM. The proper solution long term is to sandbox qemu in a way that there privileged mode and non privileged mode. It might be implemented using separate address space or not. Most operations like vnc/rpc/spice/usb should be run with less privileges. The main issue is that doing it right will take time and we'll want virt-agent be merged before the long term solution is ready. The best approach would be gradual development Yes I agree, I don't think this will happen overnight, and blocking virtagent with this would be bad. Cheers, Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/02/11 11:58, Alon Levy wrote: On Wed, Mar 02, 2011 at 11:25:44AM +0100, Jes Sorensen wrote: I had a few thoughts about this already, which I think will work for both spice and vnc. What we could do is to expose the video memory via shared memory. That way a spice or vnc daemon could get direct access to the memory, this would limit communication to keyboard/mouse events, as well as video mode info, and possibly notifications to the client about which ranges of memory have been updated. Using shared memory this way should allow us to implement the video clients without performance loss, in fact it should be beneficial since it would allow them to run fully separate from the host daemon. I think that would work well for spice. Spice uses shared memory from the pci device for both the framebuffer and surfaces/commands, but this is not really relevant at this level. What about IO and irq? that would add additional latencies, no? because each io exit would need to be ipc'ed over to the spice/vnc process? and same way in the other direction, requesting qemu to trigger an interrupt in the next vm entry. I am glad the shmem approach will work for Spice. There would be something there with IRQs etc, I agree. but there are methods to help that too, we could use a shared memory event ring for example. Cheers, Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/02/2011 12:58 PM, Alon Levy wrote: On Wed, Mar 02, 2011 at 11:25:44AM +0100, Jes Sorensen wrote: On 03/01/11 15:25, Dor Laor wrote: On 03/01/2011 02:40 PM, Anthony Liguori wrote: On Mar 1, 2011 7:07 AM, Dor Laordl...@redhat.com Qemu is the one that should spawn them and they should be transparent from the management. This way running qemu stays the same and qemu just need to add the logic to get a SIGCHILD and potentially re-execute an dead son process. Spice is the logical place to start, no? It's the largest single dependency we have and it does some scary things with qemu_mutex. I would use spice as a way to prove the concept. I agree it is desirable to the this for spice but it is allot more complex than virtagent isolation. Spice is performance sensitive and contains much more state. It needs to access the guest memory for reading the surfaces. It can be solved but needs some major changes. Adding spice-devel to the discussion. I had a few thoughts about this already, which I think will work for both spice and vnc. What we could do is to expose the video memory via shared memory. That way a spice or vnc daemon could get direct access to the memory, this would limit communication to keyboard/mouse events, as well as video mode info, and possibly notifications to the client about which ranges of memory have been updated. Using shared memory this way should allow us to implement the video clients without performance loss, in fact it should be beneficial since it would allow them to run fully separate from the host daemon. I think that would work well for spice. Spice uses shared memory from the pci device for both the framebuffer and surfaces/commands, but this is Is that the only DMA do you do? That's good for this model. not really relevant at this level. What about IO and irq? that would add additional latencies, no? because each io exit would need to be ipc'ed over to the spice/vnc process? and same way in the other direction, requesting qemu to trigger an interrupt in the next vm entry. The qxl device can be in the privileged qemu (as a start) and it will handle irqs directly. Even today you need to notify the spice server thread, so nothing will change Cheers, Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On Wed, Mar 02, 2011 at 01:04:58PM +0200, Dor Laor wrote: On 03/02/2011 12:58 PM, Alon Levy wrote: On Wed, Mar 02, 2011 at 11:25:44AM +0100, Jes Sorensen wrote: On 03/01/11 15:25, Dor Laor wrote: On 03/01/2011 02:40 PM, Anthony Liguori wrote: On Mar 1, 2011 7:07 AM, Dor Laordl...@redhat.com Qemu is the one that should spawn them and they should be transparent from the management. This way running qemu stays the same and qemu just need to add the logic to get a SIGCHILD and potentially re-execute an dead son process. Spice is the logical place to start, no? It's the largest single dependency we have and it does some scary things with qemu_mutex. I would use spice as a way to prove the concept. I agree it is desirable to the this for spice but it is allot more complex than virtagent isolation. Spice is performance sensitive and contains much more state. It needs to access the guest memory for reading the surfaces. It can be solved but needs some major changes. Adding spice-devel to the discussion. I had a few thoughts about this already, which I think will work for both spice and vnc. What we could do is to expose the video memory via shared memory. That way a spice or vnc daemon could get direct access to the memory, this would limit communication to keyboard/mouse events, as well as video mode info, and possibly notifications to the client about which ranges of memory have been updated. Using shared memory this way should allow us to implement the video clients without performance loss, in fact it should be beneficial since it would allow them to run fully separate from the host daemon. I think that would work well for spice. Spice uses shared memory from the pci device for both the framebuffer and surfaces/commands, but this is Is that the only DMA do you do? That's good for this model. Yes. We have two memory bars (another for rom) and an io bar. The bars contain the framebuffer (primary surface backing store, also used in vga mode), the non primary surfaces backing store and the rings (display, cursor, and release). In qxl (native) mode the rings are read/written to all the time, the rest by demand. In vga mode it's identical to vnc, the framebuffer is written from the guest and we track the dirty regions. not really relevant at this level. What about IO and irq? that would add additional latencies, no? because each io exit would need to be ipc'ed over to the spice/vnc process? and same way in the other direction, requesting qemu to trigger an interrupt in the next vm entry. The qxl device can be in the privileged qemu (as a start) and it will handle irqs directly. Even today you need to notify the spice server thread, so nothing will change ok, so qxl remains in qemu process. But all those io exits (well, most) end up as calls to spice-server, now in another process. So latency wise it's the same. Cheers, Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/02/2011 04:19 AM, Jes Sorensen wrote: On 02/28/11 18:44, Anthony Liguori wrote: On Feb 28, 2011 10:44 AM, Jes Sorensenjes.soren...@redhat.com wrote: Separating host-side virtagent and other tasks from core QEMU = To improve auditing of the core QEMU code, it would be ideal to be able to separate the core QEMU functionality from utility functionality by moving the utility functionality into its own process. This process will be referred to as the QEMU client below. Separating as in moving code outside of qemu.git, making code optionally built in, making code optionally built in or loadable as a separate executable that is automatically launched, or making code always built outside the main executable? I'm very nervous about having a large number of daemons necessary to run QEMU. I think a reasonable approach would be a single front-end daemond. Once QAPI is merged, there is a very incremental approach we can take for this sort of work. Take your favorite subsystem (like gdbstub or SDL) and make it only use QMP apis. Once we're only using QMP internally in a subsystem, then building it out of the main QEMU and using libqmp should be fairly easy. Hi Anthony, Back from a day off playing with power drills and concrete walls :) Sorry I should have made it a little more clear, it was obvious to me, but not written down: The idea is to keep everything as part of the QEMU package, ie. part of qemu.git. My idea is really to have one QEMU host daemon and one QEMU client, which provides the various services. You type make and you get two binaries instead of one. We could allow other daemons to connect to the host daemon, but that wouldn't be a primary target for this in my book, and I am not sure we really want to do this. It is absolutely vital for me that we do not make things much more complicated for users with this move. I don't want to get into a situation where we start forcing external packages or daemons in order to run basic QEMU. If we start requiring such, we have failed! However, I think it is a reasonable compromise to have one daemon you launch, and then a simple client you launch straight after which will then provide the same/similar views and interfaces that users are used to when they launch current QEMU (monitor, vnc server etc). I think the challenge with this approach is defining an IPC mechanism that is robust enough to support it. For instance, on one end of the spectrum we have Spice, where shared memory paired with some mechanism for IO notification between the client/server might make sense. But then we have things like the human monitor where a socket interface or pipe is clearly more the more straight-forward route. We may find that it more desirable to have multiple classes of client components, each contain within a client process with it's own IPC mechanism. Although, multiple IPC mechanisms doesn't sound particularly enticing either...but 2 might not be so unreasonable. I hope this clarifies things. Cheers, Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/02/11 14:13, Michael Roth wrote: On 03/02/2011 04:19 AM, Jes Sorensen wrote: It is absolutely vital for me that we do not make things much more complicated for users with this move. I don't want to get into a situation where we start forcing external packages or daemons in order to run basic QEMU. If we start requiring such, we have failed! However, I think it is a reasonable compromise to have one daemon you launch, and then a simple client you launch straight after which will then provide the same/similar views and interfaces that users are used to when they launch current QEMU (monitor, vnc server etc). I think the challenge with this approach is defining an IPC mechanism that is robust enough to support it. For instance, on one end of the spectrum we have Spice, where shared memory paired with some mechanism for IO notification between the client/server might make sense. But then we have things like the human monitor where a socket interface or pipe is clearly more the more straight-forward route. We may find that it more desirable to have multiple classes of client components, each contain within a client process with it's own IPC mechanism. Although, multiple IPC mechanisms doesn't sound particularly enticing either...but 2 might not be so unreasonable. Hi Michael, I think we need two types for sure, even for the video case, we will still need a control channel as well. However, I don't think it is desirable to split things up more than we have to, so if we can keep it within one client process that is good. Maybe there are cases where it makes sense to split it into more processes, I could be convinced, but I think we really need to be careful making it too much of a complex mess either. Cheers, Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/02/2011 07:18 AM, Jes Sorensen wrote: On 03/02/11 14:13, Michael Roth wrote: On 03/02/2011 04:19 AM, Jes Sorensen wrote: It is absolutely vital for me that we do not make things much more complicated for users with this move. I don't want to get into a situation where we start forcing external packages or daemons in order to run basic QEMU. If we start requiring such, we have failed! However, I think it is a reasonable compromise to have one daemon you launch, and then a simple client you launch straight after which will then provide the same/similar views and interfaces that users are used to when they launch current QEMU (monitor, vnc server etc). I think the challenge with this approach is defining an IPC mechanism that is robust enough to support it. For instance, on one end of the spectrum we have Spice, where shared memory paired with some mechanism for IO notification between the client/server might make sense. But then we have things like the human monitor where a socket interface or pipe is clearly more the more straight-forward route. We may find that it more desirable to have multiple classes of client components, each contain within a client process with it's own IPC mechanism. Although, multiple IPC mechanisms doesn't sound particularly enticing either...but 2 might not be so unreasonable. Hi Michael, I think we need two types for sure, even for the video case, we will still need a control channel as well. However, I don't think it is desirable to split things up more than we have to, so if we can keep it within one client process that is good. Maybe there are cases where it makes sense to split it into more processes, I could be convinced, but I think we really need to be careful making it too much of a complex mess either. Yup, if it's doable I'd prefer a single client process as well. Just hard to predict how difficult it'll be to support 2 or more mechanisms. Although, I'd imagine we'd end up with something like qemu's io loop, with event-driven shmem and fd-based work, which does seem doable. Cheers, Jes
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 02/28/2011 07:44 PM, Anthony Liguori wrote: On Feb 28, 2011 10:44 AM, Jes Sorensen jes.soren...@redhat.com mailto:jes.soren...@redhat.com wrote: Hi, On last week's call we discussed the issue of splitting non core features of QEMU into it's own process to reduce the security risks etc. I wrote up a summary of my thoughts on this to try to cover the various issues. Feedback welcome and hopefully we can continue the discussion on a future call - maybe next week? I would like to be part of the discussion, but it's a public holiday here March 1st, so I won't be on that call. Cheers, Jes Separating host-side virtagent and other tasks from core QEMU = To improve auditing of the core QEMU code, it would be ideal to be able to separate the core QEMU functionality from utility functionality by moving the utility functionality into its own process. This process will be referred to as the QEMU client below. Separating as in moving code outside of qemu.git, making code optionally built in, making code optionally built in or loadable as a separate executable that is automatically launched, or making code always built outside the main executable? I'm very nervous about having a large number of daemons necessary to run QEMU. I think a reasonable approach would be a single front-end daemond. s/daemon/son processes/ Qemu is the one that should spawn them and they should be transparent from the management. This way running qemu stays the same and qemu just need to add the logic to get a SIGCHILD and potentially re-execute an dead son process. Once QAPI is merged, there is a very incremental approach we can take for this sort of work. Take your favorite subsystem (like gdbstub or SDL) and make it only use QMP apis. Once we're only using QMP internally in a subsystem, then building it out of the main QEMU and using libqmp should be fairly easy. Regards, Anthony Liguori Components which are candidates for moving out of QEMU include: - virtagent - vnc server (and other graphical displays such as SDL, spice and curses) - human monitor These are all much harder than they may seem. There's a ton of QMP functions that would be needed before we could even try to do this. The idea is to have QEMU launch as a daemon, and then allow for one of more client processes to connect to it. These will then offer the various services. The main issue to discuss is how to handle various state information, reconnects, and migration. Security The primary reason for this discussion is security, however there are other benefits from this split, as will be mentioned below. During a demo of virtagent I hit a case where a bug in the agent command handling code caused a crash of the host QEMU process. While it is probably a simple bug, it shows how adding more complexity to the QEMU process increases the risk of adding security problems that could potentially be exploited by a hostile guest. By splitting non core functionality into a QEMU client process, the host process will be isolated from a large number of potential security problems, ie. in case a client process is killed or crashes, it should not affect the main QEMU process. In addition it makes it easier to audit the core QEMU functionality. virtagent = In short virtagent provides a set of simple commands, most of which do not have state associated with them. These include shutdown, ping, fsfreeze/fsthaw, etc. Other commands might be multi-stage commands which could fail if the client is disconnected from the daemon while the command is in progress. These include copy-paste and file copy. vnc server == The vnc server simply needs a connection to the video memory of the QEMU process, video mode state, as well as channels for sending keyboard and mouse events. It is stateless by nature and supports reconnects. This applies to the other graphical display engines (SDL, spice, and curses) as well. Human monitor = The human monitor is effectively stateless. It issues commands and prints the result. There is no state in the monitor and it can be built directly on top of QMP. An additional benefit here is that it would allow for multiple monitors. Disconnects === It must be possible for a client process getting killed or disconnected from the QEMU process, in which case is should be possible to launch a new client that connects to the QEMU process. In this case, commands needs to be provided allowing the client process to query the QEMU process and virtagent for current state information. In-progress commands may fail, and will need to be re-run, such as copy-paste and and file copy. However neither of these are vital commands and a re-run of such commands is acceptable behavior.
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On Mar 1, 2011 7:07 AM, Dor Laor dl...@redhat.com wrote: On 02/28/2011 07:44 PM, Anthony Liguori wrote: On Feb 28, 2011 10:44 AM, Jes Sorensen jes.soren...@redhat.com mailto:jes.soren...@redhat.com wrote: Hi, On last week's call we discussed the issue of splitting non core features of QEMU into it's own process to reduce the security risks etc. I wrote up a summary of my thoughts on this to try to cover the various issues. Feedback welcome and hopefully we can continue the discussion on a future call - maybe next week? I would like to be part of the discussion, but it's a public holiday here March 1st, so I won't be on that call. Cheers, Jes Separating host-side virtagent and other tasks from core QEMU = To improve auditing of the core QEMU code, it would be ideal to be able to separate the core QEMU functionality from utility functionality by moving the utility functionality into its own process. This process will be referred to as the QEMU client below. Separating as in moving code outside of qemu.git, making code optionally built in, making code optionally built in or loadable as a separate executable that is automatically launched, or making code always built outside the main executable? I'm very nervous about having a large number of daemons necessary to run QEMU. I think a reasonable approach would be a single front-end daemond. s/daemon/son processes/ Qemu is the one that should spawn them and they should be transparent from the management. This way running qemu stays the same and qemu just need to add the logic to get a SIGCHILD and potentially re-execute an dead son process. Spice is the logical place to start, no? It's the largest single dependency we have and it does some scary things with qemu_mutex. I would use spice as a way to prove the concept. Regards, Anthony Liguori Once QAPI is merged, there is a very incremental approach we can take for this sort of work. Take your favorite subsystem (like gdbstub or SDL) and make it only use QMP apis. Once we're only using QMP internally in a subsystem, then building it out of the main QEMU and using libqmp should be fairly easy. Regards, Anthony Liguori Components which are candidates for moving out of QEMU include: - virtagent - vnc server (and other graphical displays such as SDL, spice and curses) - human monitor These are all much harder than they may seem. There's a ton of QMP functions that would be needed before we could even try to do this. The idea is to have QEMU launch as a daemon, and then allow for one of more client processes to connect to it. These will then offer the various services. The main issue to discuss is how to handle various state information, reconnects, and migration. Security The primary reason for this discussion is security, however there are other benefits from this split, as will be mentioned below. During a demo of virtagent I hit a case where a bug in the agent command handling code caused a crash of the host QEMU process. While it is probably a simple bug, it shows how adding more complexity to the QEMU process increases the risk of adding security problems that could potentially be exploited by a hostile guest. By splitting non core functionality into a QEMU client process, the host process will be isolated from a large number of potential security problems, ie. in case a client process is killed or crashes, it should not affect the main QEMU process. In addition it makes it easier to audit the core QEMU functionality. virtagent = In short virtagent provides a set of simple commands, most of which do not have state associated with them. These include shutdown, ping, fsfreeze/fsthaw, etc. Other commands might be multi-stage commands which could fail if the client is disconnected from the daemon while the command is in progress. These include copy-paste and file copy. vnc server == The vnc server simply needs a connection to the video memory of the QEMU process, video mode state, as well as channels for sending keyboard and mouse events. It is stateless by nature and supports reconnects. This applies to the other graphical display engines (SDL, spice, and curses) as well. Human monitor = The human monitor is effectively stateless. It issues commands and prints the result. There is no state in the monitor and it can be built directly on top of QMP. An additional benefit here is that it would allow for multiple monitors. Disconnects === It must be possible for a client process getting killed or disconnected from the QEMU process, in which case is should be possible to launch a new client that connects to the QEMU
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/01/2011 02:40 PM, Anthony Liguori wrote: On Mar 1, 2011 7:07 AM, Dor Laor dl...@redhat.com mailto:dl...@redhat.com wrote: On 02/28/2011 07:44 PM, Anthony Liguori wrote: On Feb 28, 2011 10:44 AM, Jes Sorensen jes.soren...@redhat.com mailto:jes.soren...@redhat.com mailto:jes.soren...@redhat.com mailto:jes.soren...@redhat.com wrote: Hi, On last week's call we discussed the issue of splitting non core features of QEMU into it's own process to reduce the security risks etc. I wrote up a summary of my thoughts on this to try to cover the various issues. Feedback welcome and hopefully we can continue the discussion on a future call - maybe next week? I would like to be part of the discussion, but it's a public holiday here March 1st, so I won't be on that call. Cheers, Jes Separating host-side virtagent and other tasks from core QEMU = To improve auditing of the core QEMU code, it would be ideal to be able to separate the core QEMU functionality from utility functionality by moving the utility functionality into its own process. This process will be referred to as the QEMU client below. Separating as in moving code outside of qemu.git, making code optionally built in, making code optionally built in or loadable as a separate executable that is automatically launched, or making code always built outside the main executable? I'm very nervous about having a large number of daemons necessary to run QEMU. I think a reasonable approach would be a single front-end daemond. s/daemon/son processes/ Qemu is the one that should spawn them and they should be transparent from the management. This way running qemu stays the same and qemu just need to add the logic to get a SIGCHILD and potentially re-execute an dead son process. Spice is the logical place to start, no? It's the largest single dependency we have and it does some scary things with qemu_mutex. I would use spice as a way to prove the concept. I agree it is desirable to the this for spice but it is allot more complex than virtagent isolation. Spice is performance sensitive and contains much more state. It needs to access the guest memory for reading the surfaces. It can be solved but needs some major changes. Adding spice-devel to the discussion. Will virtagent be kept out of tree till we merge spice? Regards, Anthony Liguori Once QAPI is merged, there is a very incremental approach we can take for this sort of work. Take your favorite subsystem (like gdbstub or SDL) and make it only use QMP apis. Once we're only using QMP internally in a subsystem, then building it out of the main QEMU and using libqmp should be fairly easy. Regards, Anthony Liguori Components which are candidates for moving out of QEMU include: - virtagent - vnc server (and other graphical displays such as SDL, spice and curses) - human monitor These are all much harder than they may seem. There's a ton of QMP functions that would be needed before we could even try to do this. The idea is to have QEMU launch as a daemon, and then allow for one of more client processes to connect to it. These will then offer the various services. The main issue to discuss is how to handle various state information, reconnects, and migration. Security The primary reason for this discussion is security, however there are other benefits from this split, as will be mentioned below. During a demo of virtagent I hit a case where a bug in the agent command handling code caused a crash of the host QEMU process. While it is probably a simple bug, it shows how adding more complexity to the QEMU process increases the risk of adding security problems that could potentially be exploited by a hostile guest. By splitting non core functionality into a QEMU client process, the host process will be isolated from a large number of potential security problems, ie. in case a client process is killed or crashes, it should not affect the main QEMU process. In addition it makes it easier to audit the core QEMU functionality. virtagent = In short virtagent provides a set of simple commands, most of which do not have state associated with them. These include shutdown, ping, fsfreeze/fsthaw, etc. Other commands might be multi-stage commands which could fail if the client is disconnected from the daemon while the command is in progress. These include copy-paste and file copy. vnc server == The vnc server simply needs a connection to the video memory of the QEMU process, video mode state, as well as channels for sending keyboard and mouse events. It is stateless by nature and supports reconnects. This applies to the other graphical display engines (SDL,
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On 03/01/2011 09:25 AM, Dor Laor wrote: On 03/01/2011 02:40 PM, Anthony Liguori wrote: On Mar 1, 2011 7:07 AM, Dor Laor dl...@redhat.com mailto:dl...@redhat.com wrote: On 02/28/2011 07:44 PM, Anthony Liguori wrote: On Feb 28, 2011 10:44 AM, Jes Sorensen jes.soren...@redhat.com mailto:jes.soren...@redhat.com mailto:jes.soren...@redhat.com mailto:jes.soren...@redhat.com wrote: Hi, On last week's call we discussed the issue of splitting non core features of QEMU into it's own process to reduce the security risks etc. I wrote up a summary of my thoughts on this to try to cover the various issues. Feedback welcome and hopefully we can continue the discussion on a future call - maybe next week? I would like to be part of the discussion, but it's a public holiday here March 1st, so I won't be on that call. Cheers, Jes Separating host-side virtagent and other tasks from core QEMU = To improve auditing of the core QEMU code, it would be ideal to be able to separate the core QEMU functionality from utility functionality by moving the utility functionality into its own process. This process will be referred to as the QEMU client below. Separating as in moving code outside of qemu.git, making code optionally built in, making code optionally built in or loadable as a separate executable that is automatically launched, or making code always built outside the main executable? I'm very nervous about having a large number of daemons necessary to run QEMU. I think a reasonable approach would be a single front-end daemond. s/daemon/son processes/ Qemu is the one that should spawn them and they should be transparent from the management. This way running qemu stays the same and qemu just need to add the logic to get a SIGCHILD and potentially re-execute an dead son process. Spice is the logical place to start, no? It's the largest single dependency we have and it does some scary things with qemu_mutex. I would use spice as a way to prove the concept. I agree it is desirable to the this for spice but it is allot more complex than virtagent isolation. Spice is performance sensitive and contains much more state. It needs to access the guest memory for reading the surfaces. It can be solved but needs some major changes. Adding spice-devel to the discussion. Yeah, but the viability of this mechanism is dependent on whether it can support something that's complex, just like Spice. Considering that the smaller pieces like VNC or virt-agent are at most a couple thousand of lines of code, whereas Spice is close to a hundred thousand lines, the benefit from moving Spice to a separate address space is significantly higher. Will virtagent be kept out of tree till we merge spice? Nothing should be held up from being merged because of an effort to put things in a separate address space. But that's not to say that virt-agent is something that's going to get merged tomorrow. I don't know that there is even an agreed upon architecture at the moment. Regards, Anthony Liguori
Re: [Qemu-devel] QEMU: Discussion of separating core functionality vs supportive features
On Feb 28, 2011 10:44 AM, Jes Sorensen jes.soren...@redhat.com wrote: Hi, On last week's call we discussed the issue of splitting non core features of QEMU into it's own process to reduce the security risks etc. I wrote up a summary of my thoughts on this to try to cover the various issues. Feedback welcome and hopefully we can continue the discussion on a future call - maybe next week? I would like to be part of the discussion, but it's a public holiday here March 1st, so I won't be on that call. Cheers, Jes Separating host-side virtagent and other tasks from core QEMU = To improve auditing of the core QEMU code, it would be ideal to be able to separate the core QEMU functionality from utility functionality by moving the utility functionality into its own process. This process will be referred to as the QEMU client below. Separating as in moving code outside of qemu.git, making code optionally built in, making code optionally built in or loadable as a separate executable that is automatically launched, or making code always built outside the main executable? I'm very nervous about having a large number of daemons necessary to run QEMU. I think a reasonable approach would be a single front-end daemond. Once QAPI is merged, there is a very incremental approach we can take for this sort of work. Take your favorite subsystem (like gdbstub or SDL) and make it only use QMP apis. Once we're only using QMP internally in a subsystem, then building it out of the main QEMU and using libqmp should be fairly easy. Regards, Anthony Liguori Components which are candidates for moving out of QEMU include: - virtagent - vnc server (and other graphical displays such as SDL, spice and curses) - human monitor These are all much harder than they may seem. There's a ton of QMP functions that would be needed before we could even try to do this. The idea is to have QEMU launch as a daemon, and then allow for one of more client processes to connect to it. These will then offer the various services. The main issue to discuss is how to handle various state information, reconnects, and migration. Security The primary reason for this discussion is security, however there are other benefits from this split, as will be mentioned below. During a demo of virtagent I hit a case where a bug in the agent command handling code caused a crash of the host QEMU process. While it is probably a simple bug, it shows how adding more complexity to the QEMU process increases the risk of adding security problems that could potentially be exploited by a hostile guest. By splitting non core functionality into a QEMU client process, the host process will be isolated from a large number of potential security problems, ie. in case a client process is killed or crashes, it should not affect the main QEMU process. In addition it makes it easier to audit the core QEMU functionality. virtagent = In short virtagent provides a set of simple commands, most of which do not have state associated with them. These include shutdown, ping, fsfreeze/fsthaw, etc. Other commands might be multi-stage commands which could fail if the client is disconnected from the daemon while the command is in progress. These include copy-paste and file copy. vnc server == The vnc server simply needs a connection to the video memory of the QEMU process, video mode state, as well as channels for sending keyboard and mouse events. It is stateless by nature and supports reconnects. This applies to the other graphical display engines (SDL, spice, and curses) as well. Human monitor = The human monitor is effectively stateless. It issues commands and prints the result. There is no state in the monitor and it can be built directly on top of QMP. An additional benefit here is that it would allow for multiple monitors. Disconnects === It must be possible for a client process getting killed or disconnected from the QEMU process, in which case is should be possible to launch a new client that connects to the QEMU process. In this case, commands needs to be provided allowing the client process to query the QEMU process and virtagent for current state information. In-progress commands may fail, and will need to be re-run, such as copy-paste and and file copy. However neither of these are vital commands and a re-run of such commands is acceptable behavior. Migration = Given that migration moves the guest to a new QEMU process, normally on a different host, any connection from management tools to the monitor, QMP sockets, virtio-serial, etc. require a new connection through the new QEMU process. Stateless connections, such as the monitor, QMP and the vnc-server handles reconnects without problems, and should not have any issues during migration that are different from the
