Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-04-08 Thread Antoine Martin
Avi Kivity wrote: > On 03/24/2010 06:40 PM, Joerg Roedel wrote: >> >>> Looks trivial to find a guest, less so with enumerating (still doable). >>> >> Not so trival and even more likely to break. Even it perf has the pid of >> the process and wants to find the directory it has to do: >> >> 1.

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-25 Thread Zhang, Yanmin
On Wed, 2010-03-24 at 20:20 +0200, Avi Kivity wrote: > On 03/24/2010 07:47 PM, Arnaldo Carvalho de Melo wrote: > > Em Wed, Mar 24, 2010 at 06:09:30PM +0200, Avi Kivity escreveu: > > > >> Doesn't perf already has a dependency on naming conventions for finding > >> debug information? > >> >

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Arnaldo Carvalho de Melo
Em Wed, Mar 24, 2010 at 08:20:10PM +0200, Avi Kivity escreveu: > On 03/24/2010 07:47 PM, Arnaldo Carvalho de Melo wrote: >> Em Wed, Mar 24, 2010 at 06:09:30PM +0200, Avi Kivity escreveu: >> >>> Doesn't perf already has a dependency on naming conventions for finding >>> debug information? >>>

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 07:47 PM, Arnaldo Carvalho de Melo wrote: Em Wed, Mar 24, 2010 at 06:09:30PM +0200, Avi Kivity escreveu: Doesn't perf already has a dependency on naming conventions for finding debug information? It looks at several places, from most symbol rich (/usr/lib/debug/, aka -de

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Arnaldo Carvalho de Melo
Em Wed, Mar 24, 2010 at 06:09:30PM +0200, Avi Kivity escreveu: > Doesn't perf already has a dependency on naming conventions for finding > debug information? It looks at several places, from most symbol rich (/usr/lib/debug/, aka -debuginfo packages, where we have full symtabs) to poorest (the p

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 06:47 PM, Avi Kivity wrote: It's true. If the kernel provides something, there are fewer things that can break. But if your system is so broken that you can't resolve uids, fix that before running perf. Must we design perf for that case? After all, 'ls -l' will break under

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 06:45 PM, Joerg Roedel wrote: That's just what I want to do. Leave it in userspace and then they can deal with it without telling us about it. They can't do that with a directory in /proc? I don't know. -- error compiling committee.c: too many arguments to function

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 06:40 PM, Joerg Roedel wrote: Looks trivial to find a guest, less so with enumerating (still doable). Not so trival and even more likely to break. Even it perf has the pid of the process and wants to find the directory it has to do: 1. Get the uid of the process 2. Find th

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Peter Zijlstra
On Wed, 2010-03-24 at 17:23 +0100, Joerg Roedel wrote: > On Wed, Mar 24, 2010 at 05:03:42PM +0100, Peter Zijlstra wrote: > > On Wed, 2010-03-24 at 16:01 +0100, Joerg Roedel wrote: > > > > > What I meant was: perf-kernel puts the guest-name into every sample and > > > perf-userspace accesses /sys/k

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Joerg Roedel
On Wed, Mar 24, 2010 at 06:32:51PM +0200, Avi Kivity wrote: > On 03/24/2010 06:31 PM, Joerg Roedel wrote: > That's just what I want to do. Leave it in userspace and then they can > deal with it without telling us about it. They can't do that with a directory in /proc? -- To unsubscribe from t

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Joerg Roedel
On Wed, Mar 24, 2010 at 06:09:30PM +0200, Avi Kivity wrote: > On 03/24/2010 05:59 PM, Joerg Roedel wrote: >> >> I am not tied to /sys/kvm. We could also use /proc//kvm/ for example. This would keep anything in the process space (except for the global list of VMs which we should h

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 06:31 PM, Joerg Roedel wrote: On Wed, Mar 24, 2010 at 06:20:38PM +0200, Avi Kivity wrote: On 03/24/2010 06:17 PM, Joerg Roedel wrote: But is this not only one entity more for sVirt to handle? I would leave that decision to the sVirt developers. Does attaching the same la

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Joerg Roedel
On Wed, Mar 24, 2010 at 06:20:38PM +0200, Avi Kivity wrote: > On 03/24/2010 06:17 PM, Joerg Roedel wrote: >> But is this not only one entity more for >> sVirt to handle? I would leave that decision to the sVirt developers. >> Does attaching the same label as for the VM resources mean that root >> c

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Joerg Roedel
On Wed, Mar 24, 2010 at 05:03:42PM +0100, Peter Zijlstra wrote: > On Wed, 2010-03-24 at 16:01 +0100, Joerg Roedel wrote: > > > What I meant was: perf-kernel puts the guest-name into every sample and > > perf-userspace accesses /sys/kvm/guest_name/fs/ later to resolve the > > symbols. I leave the q

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 06:17 PM, Joerg Roedel wrote: On Wed, Mar 24, 2010 at 05:52:54PM +0200, Avi Kivity wrote: On 03/24/2010 05:50 PM, Joerg Roedel wrote: If we go the /proc//kvm way then the directory should probably inherit the label from /proc//? That's a security policy. The se

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 06:03 PM, Peter Zijlstra wrote: On Wed, 2010-03-24 at 16:01 +0100, Joerg Roedel wrote: What I meant was: perf-kernel puts the guest-name into every sample and perf-userspace accesses /sys/kvm/guest_name/fs/ later to resolve the symbols. I leave the question of how the guest-fs

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Joerg Roedel
On Wed, Mar 24, 2010 at 05:52:54PM +0200, Avi Kivity wrote: > On 03/24/2010 05:50 PM, Joerg Roedel wrote: >> If we go the /proc//kvm way then the directory should probably >> inherit the label from /proc//? > > That's a security policy. The security people like their policies > outside the kerne

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 05:59 PM, Joerg Roedel wrote: I am not tied to /sys/kvm. We could also use /proc//kvm/ for example. This would keep anything in the process space (except for the global list of VMs which we should have anyway). How about ~/.qemu/guests/$pid? That makes it hard

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Peter Zijlstra
On Wed, 2010-03-24 at 16:01 +0100, Joerg Roedel wrote: > What I meant was: perf-kernel puts the guest-name into every sample and > perf-userspace accesses /sys/kvm/guest_name/fs/ later to resolve the > symbols. I leave the question of how the guest-fs is exposed to the host > out of this discussio

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Joerg Roedel
On Wed, Mar 24, 2010 at 05:49:42PM +0200, Avi Kivity wrote: > On 03/24/2010 05:46 PM, Joerg Roedel wrote: >> On Wed, Mar 24, 2010 at 05:12:55PM +0200, Avi Kivity wrote: >> >>> On 03/24/2010 05:01 PM, Joerg Roedel wrote: >>> $ cd /sys/kvm/guest0 $ ls -l -r 1 root roo

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 05:50 PM, Joerg Roedel wrote: On Wed, Mar 24, 2010 at 05:43:31PM +0200, Avi Kivity wrote: On 03/24/2010 05:37 PM, Joerg Roedel wrote: Even better. So a guest which breaks out can't even access its own /sys/kvm/ directory. Perfect, it doesn't need that access anyway.

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Joerg Roedel
On Wed, Mar 24, 2010 at 05:43:31PM +0200, Avi Kivity wrote: > On 03/24/2010 05:37 PM, Joerg Roedel wrote: >> Even better. So a guest which breaks out can't even access its own >> /sys/kvm/ directory. Perfect, it doesn't need that access anyway. > > But what security label does that directory have?

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 05:46 PM, Joerg Roedel wrote: On Wed, Mar 24, 2010 at 05:12:55PM +0200, Avi Kivity wrote: On 03/24/2010 05:01 PM, Joerg Roedel wrote: $ cd /sys/kvm/guest0 $ ls -l -r 1 root root 0 2009-08-17 12:05 name dr-x-- 1 root root 0 2009-08-17 12:05 fs $ cat name guest

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Joerg Roedel
On Wed, Mar 24, 2010 at 05:12:55PM +0200, Avi Kivity wrote: > On 03/24/2010 05:01 PM, Joerg Roedel wrote: >> $ cd /sys/kvm/guest0 >> $ ls -l >> -r 1 root root 0 2009-08-17 12:05 name >> dr-x-- 1 root root 0 2009-08-17 12:05 fs >> $ cat name >> guest0 >> $ # ... >> >> The fs/ directory i

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 05:37 PM, Joerg Roedel wrote: No it can't. With sVirt every single VM has a custom security label and the policy only allows it access to disks / files with a matching label, and prevents it attacking any other VMs or processes on the host. THis confines the scope of any exploit i

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Joerg Roedel
On Wed, Mar 24, 2010 at 03:26:53PM +, Daniel P. Berrange wrote: > On Wed, Mar 24, 2010 at 04:01:37PM +0100, Joerg Roedel wrote: > > >> An approach like: "The files are owned and only readable by the same > > >> user that started the vm." might be a good start. So a user can measure > > >> its o

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Daniel P. Berrange
On Wed, Mar 24, 2010 at 04:01:37PM +0100, Joerg Roedel wrote: > >> An approach like: "The files are owned and only readable by the same > >> user that started the vm." might be a good start. So a user can measure > >> its own guests and root can measure all of them. > > > > That's not how sVirt wor

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 05:01 PM, Joerg Roedel wrote: But when I weigh the benefit of truly transparent system-wide perf integration for users who don't use libvirt but do use perf, versus the cost of transforming kvm from a single-process API to a system-wide API with all the complications that I've l

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 04:24 PM, Alexander Graf wrote: Avi Kivity wrote: On 03/24/2010 03:53 PM, Alexander Graf wrote: Someone needs to know about the new guest to fetch its symbols. Or do you want that part in the kernel too? How about we add a virtio "guest file system a

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Joerg Roedel
On Wed, Mar 24, 2010 at 03:57:39PM +0200, Avi Kivity wrote: > On 03/24/2010 03:46 PM, Joerg Roedel wrote: >> Someone who uses libvirt and virt-manager by default is probably not >> interested in this feature at the same level a kvm developer is. And >> developers tend not to use libvirt for low-le

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Alexander Graf
Avi Kivity wrote: > On 03/24/2010 03:53 PM, Alexander Graf wrote: >> >>> Someone needs to know about the new guest to fetch its symbols. Or do >>> you want that part in the kernel too? >>> >> >> How about we add a virtio "guest file system access" device? The guest >> would then expose its o

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 03:53 PM, Alexander Graf wrote: Someone needs to know about the new guest to fetch its symbols. Or do you want that part in the kernel too? How about we add a virtio "guest file system access" device? The guest would then expose its own file system using that device. On

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 03:46 PM, Joerg Roedel wrote: On Wed, Mar 24, 2010 at 03:05:02PM +0200, Avi Kivity wrote: On 03/24/2010 02:50 PM, Joerg Roedel wrote: I don't want the tool for myself only. A typical perf user expects that it works transparent. A typical kvm user uses libvir

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Alexander Graf
Avi Kivity wrote: > On 03/24/2010 02:50 PM, Joerg Roedel wrote: >> >>> You can always provide the kernel and module paths as command line >>> parameters. It just won't be transparently usable, but if you're using >>> qemu from the command line, presumably you can live with that. >>> >> I don

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Joerg Roedel
On Wed, Mar 24, 2010 at 03:05:02PM +0200, Avi Kivity wrote: > On 03/24/2010 02:50 PM, Joerg Roedel wrote: >> I don't want the tool for myself only. A typical perf user expects that >> it works transparent. > > A typical kvm user uses libvirt, so we can integrate it with that. Someone who uses lib

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 02:50 PM, Joerg Roedel wrote: You can always provide the kernel and module paths as command line parameters. It just won't be transparently usable, but if you're using qemu from the command line, presumably you can live with that. I don't want the tool for myself only. A t

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Joerg Roedel
On Wed, Mar 24, 2010 at 02:08:17PM +0200, Avi Kivity wrote: > On 03/24/2010 01:59 PM, Joerg Roedel wrote: > You can always provide the kernel and module paths as command line > parameters. It just won't be transparently usable, but if you're using > qemu from the command line, presumably you

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 01:59 PM, Joerg Roedel wrote: On Wed, Mar 24, 2010 at 06:57:47AM +0200, Avi Kivity wrote: On 03/23/2010 08:21 PM, Joerg Roedel wrote: This enumeration is a very small and non-intrusive feature. Making it aware of namespaces is easy too. It's easier (and safer a

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Paolo Bonzini
On 03/22/2010 08:13 AM, Avi Kivity wrote: (btw, why are you interested in desktop-on-desktop? one use case is developers, which don't really need fancy GUIs; a second is people who test out distributions, but that doesn't seem to be a huge population; and a third is people running Windows for s

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Joerg Roedel
On Wed, Mar 24, 2010 at 06:57:47AM +0200, Avi Kivity wrote: > On 03/23/2010 08:21 PM, Joerg Roedel wrote: >> This enumeration is a very small and non-intrusive feature. Making it >> aware of namespaces is easy too. >> > > It's easier (and safer and all the other boring bits) not to do it at >

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Andi Kleen
Avi Kivity writes: > On 03/24/2010 09:38 AM, Andi Kleen wrote: >>> If you're profiling a single guest it makes more sense to do this from >>> inside the guest - you can profile userspace as well as the kernel. >>> >> I'm interested in debugging the guest without guest cooperation. >> >> In

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Avi Kivity
On 03/24/2010 09:38 AM, Andi Kleen wrote: If you're profiling a single guest it makes more sense to do this from inside the guest - you can profile userspace as well as the kernel. I'm interested in debugging the guest without guest cooperation. In many cases qemu's new gdb stub works for

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Andi Kleen
> If you're profiling a single guest it makes more sense to do this from > inside the guest - you can profile userspace as well as the kernel. I'm interested in debugging the guest without guest cooperation. In many cases qemu's new gdb stub works for that, but in some cases I would prefer instr

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-23 Thread Avi Kivity
On 03/24/2010 07:09 AM, Andi Kleen wrote: Joerg Roedel writes: Sidenote: I really think we should come to a conclusion about the concept. KVM integration into perf is very useful feature to analyze virtualization workloads. Agreed. I especially would like to see

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-23 Thread Andi Kleen
Joerg Roedel writes: > > Sidenote: I really think we should come to a conclusion about the > concept. KVM integration into perf is very useful feature to > analyze virtualization workloads. Agreed. I especially would like to see instruction/branch tracing working this way. This

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-23 Thread Avi Kivity
On 03/23/2010 08:21 PM, Joerg Roedel wrote: On Tue, Mar 23, 2010 at 06:39:58PM +0200, Avi Kivity wrote: On 03/23/2010 04:06 PM, Joerg Roedel wrote: And this system wide entity is the kvm module. It creates instances of 'struct kvm' and destroys them. I see no problem if we just at

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-23 Thread Javier Guerra Giraldez
On Tue, Mar 23, 2010 at 2:21 PM, Joerg Roedel wrote: > On Tue, Mar 23, 2010 at 06:39:58PM +0200, Avi Kivity wrote: >> So, two users can't have a guest named MyGuest each?  What about >> namespace support?  There's a lot of work in virtualizing all kernel >> namespaces, you're adding to that. > > T

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-23 Thread Peter Zijlstra
On Tue, 2010-03-23 at 19:21 +0100, Joerg Roedel wrote: > Sidenote: I really think we should come to a conclusion about the > concept. KVM integration into perf is very useful feature to > analyze virtualization workloads. I always start my things with bare kvm, It would be very

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-23 Thread Joerg Roedel
On Tue, Mar 23, 2010 at 06:39:58PM +0200, Avi Kivity wrote: > On 03/23/2010 04:06 PM, Joerg Roedel wrote: >> And this system wide entity is the kvm module. It creates instances of >> 'struct kvm' and destroys them. I see no problem if we just attach a >> name to every instance with a good default

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-23 Thread Avi Kivity
On 03/23/2010 04:06 PM, Joerg Roedel wrote: On Mon, Mar 22, 2010 at 05:06:17PM -0500, Anthony Liguori wrote: There always needs to be a system wide entity. There are two ways to enumerate instances from that system wide entity. You can centralize the creation of instances and there by main

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-23 Thread Anthony Liguori
On 03/23/2010 04:07 AM, Avi Kivity wrote: On 03/23/2010 12:06 AM, Anthony Liguori wrote: Having qemu enumerate guests one way or another is not a good idea IMO since it is focused on one guest and doesn't have a system-wide entity. There always needs to be a system wide entity. There are tw

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-23 Thread Joerg Roedel
On Mon, Mar 22, 2010 at 05:06:17PM -0500, Anthony Liguori wrote: > There always needs to be a system wide entity. There are two ways to > enumerate instances from that system wide entity. You can centralize > the creation of instances and there by maintain an list of current > instances. Y

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-23 Thread Bernd Petrovitsch
On Mon, 2010-03-22 at 21:21 +0100, Ingo Molnar wrote: [...] > Have you made thoughts about why that might be so? Yes. Forword: I assume with "GUI" you mean "a user interface for the classical desktop user with next to no interest in learning details or basics". That doesn't mean the classical des

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-23 Thread Antoine Martin
On 03/23/2010 05:13 PM, Kevin Wolf wrote: Am 22.03.2010 23:06, schrieb Anthony Liguori: On 03/22/2010 02:47 PM, Avi Kivity wrote: Having qemu enumerate guests one way or another is not a good idea IMO since it is focused on one guest and doesn't have a system-wide entity. The

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-23 Thread Kevin Wolf
Am 22.03.2010 23:06, schrieb Anthony Liguori: > On 03/22/2010 02:47 PM, Avi Kivity wrote: >> Having qemu enumerate guests one way or another is not a good idea IMO >> since it is focused on one guest and doesn't have a system-wide entity. > > There always needs to be a system wide entity. There

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-23 Thread Olivier Galibert
On Mon, Mar 22, 2010 at 03:54:37PM +0100, Ingo Molnar wrote: > Yes, i thought Qemu would be a prime candidate to be the baseline for > tools/kvm/, but i guess that has become socially impossible now after this > flamewar. It's not a big problem in the big scheme of things: tools/kvm/ is > best g

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-23 Thread Avi Kivity
On 03/23/2010 12:06 AM, Anthony Liguori wrote: Having qemu enumerate guests one way or another is not a good idea IMO since it is focused on one guest and doesn't have a system-wide entity. There always needs to be a system wide entity. There are two ways to enumerate instances from that sy

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Anthony Liguori
On 03/22/2010 02:47 PM, Avi Kivity wrote: On 03/22/2010 09:27 PM, Ingo Molnar wrote: If your position basically boils down to, we can't trust userspace and we can always trust the kernel, I want to eliminate any userspace path, then I can't really help you out. Why would you want to 'help me o

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Daniel P. Berrange
On Tue, Mar 23, 2010 at 03:00:28AM +0700, Antoine Martin wrote: > On 03/23/2010 02:15 AM, Anthony Liguori wrote: > >On 03/22/2010 12:55 PM, Avi Kivity wrote: > >>>Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by > >>>Anthony. > >>>There's numerous ways that this can break: > >>

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Avi Kivity
On 03/22/2010 10:46 PM, Ingo Molnar wrote: You should instead read the long list of disadvantages above, invert them and list then as advantages for the kernel-based vcpu enumeration solution, apply common sense and go admit to yourself that indeed in this situation a kernel provided enumeratio

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Ingo Molnar
* Avi Kivity wrote: > On 03/22/2010 09:27 PM, Ingo Molnar wrote: > > > >> If your position basically boils down to, we can't trust userspace > >> and we can always trust the kernel, I want to eliminate any > >> userspace path, then I can't really help you out. > > > > Why would you want to 'help

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Avi Kivity
On 03/22/2010 10:35 PM, Ingo Molnar wrote: And your point is that such vcpus should be excluded from profiling just because they fall outside the Qemu/libvirt umbrella? That is a ridiculous position. Non-guest vcpus will not be able to provide Linux-style symbols. And why do y

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Avi Kivity
On 03/22/2010 10:32 PM, Ingo Molnar wrote: * Anthony Liguori wrote: On 03/22/2010 02:22 PM, Ingo Molnar wrote: Transitive had a product that was using a KVM context to run their binary translator which allowed them full access to the host processes virtual address space range. In t

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Avi Kivity
On 03/22/2010 10:29 PM, Ingo Molnar wrote: * Avi Kivity wrote: I think you didnt understand my point. I am talking about 'perf kvm top' hanging if Qemu hangs. Use non-blocking I/O, report that guest as dead. No point in profiling it, it isn't making any progress. Erm, at w

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Avi Kivity
On 03/22/2010 10:21 PM, Ingo Molnar wrote: * Alexander Graf wrote: Furthermore, another negative effect is that many times features are implemented not in their technically best way, but in a way to keep them local to the project that originates them. This is done to keep deployment latenc

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Ingo Molnar
* Avi Kivity wrote: > On 03/22/2010 09:22 PM, Ingo Molnar wrote: > > > >> Transitive had a product that was using a KVM context to run their binary > >> translator which allowed them full access to the host processes virtual > >> address space range. In this case, there is no kernel and there

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Ingo Molnar
* Anthony Liguori wrote: > On 03/22/2010 02:22 PM, Ingo Molnar wrote: > >>Transitive had a product that was using a KVM context to run their > >>binary translator which allowed them full access to the host > >>processes virtual address space range. In this case, there is no > >>kernel and there

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Ingo Molnar
* Avi Kivity wrote: > > I think you didnt understand my point. I am talking about 'perf kvm top' > > hanging if Qemu hangs. > > Use non-blocking I/O, report that guest as dead. No point in profiling it, > it isn't making any progress. Erm, at what point do i decide that a guest is 'dead' ve

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Ingo Molnar
* Alexander Graf wrote: > > Furthermore, another negative effect is that many times features are > > implemented not in their technically best way, but in a way to keep them > > local to the project that originates them. This is done to keep deployment > > latencies and general contribution o

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Antoine Martin
On 03/23/2010 02:54 AM, Ingo Molnar wrote: * Alexander Graf wrote Yes. I think the point was that every layer in between brings potential slowdown and loss of features. Exactly. The more 'fragmented' a project is into sub-projects, without a single, unified, functional reference implemen

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Avi Kivity
On 03/22/2010 10:06 PM, Ingo Molnar wrote: * Avi Kivity wrote: On 03/22/2010 09:20 PM, Ingo Molnar wrote: * Avi Kivity wrote: Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by Anthony. There's numerous ways that this can break: I don't like

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Ingo Molnar
* Avi Kivity wrote: > On 03/22/2010 09:20 PM, Ingo Molnar wrote: > >* Avi Kivity wrote: > > > >>>Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by > >>>Anthony. There's numerous ways that this can break: > >>I don't like it either. We have libvirt for enumerating guests. > >W

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Antoine Martin
On 03/23/2010 02:15 AM, Anthony Liguori wrote: On 03/22/2010 12:55 PM, Avi Kivity wrote: Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by Anthony. There's numerous ways that this can break: I don't like it either. We have libvirt for enumerating guests. We're stuck in a

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Alexander Graf
On 22.03.2010, at 20:54, Ingo Molnar wrote: > > * Alexander Graf wrote: > >> Yes. I think the point was that every layer in between brings potential >> slowdown and loss of features. > > Exactly. The more 'fragmented' a project is into sub-projects, without a > single, unified, functional r

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Ingo Molnar
* Alexander Graf wrote: > Yes. I think the point was that every layer in between brings potential > slowdown and loss of features. Exactly. The more 'fragmented' a project is into sub-projects, without a single, unified, functional reference implementation in the center of it, the longer it

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Ingo Molnar
* Joerg Roedel wrote: > On Mon, Mar 22, 2010 at 05:32:15PM +0100, Ingo Molnar wrote: > > I dont know how you can find the situation of Alpha comparable, which is a > > legacy architecture for which no new CPU was manufactored in the past ~10 > > years. > > > > The negative effects of physical

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Avi Kivity
On 03/22/2010 09:27 PM, Ingo Molnar wrote: If your position basically boils down to, we can't trust userspace and we can always trust the kernel, I want to eliminate any userspace path, then I can't really help you out. Why would you want to 'help me out'? I can tell a good solution from

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Avi Kivity
On 03/22/2010 09:22 PM, Ingo Molnar wrote: Transitive had a product that was using a KVM context to run their binary translator which allowed them full access to the host processes virtual address space range. In this case, there is no kernel and there are no devices. And your point is

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Avi Kivity
On 03/22/2010 09:20 PM, Ingo Molnar wrote: * Avi Kivity wrote: Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by Anthony. There's numerous ways that this can break: I don't like it either. We have libvirt for enumerating guests. Which has pretty much the

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Alexander Graf
On 22.03.2010, at 20:31, Daniel P. Berrange wrote: > On Mon, Mar 22, 2010 at 02:15:35PM -0500, Anthony Liguori wrote: >> On 03/22/2010 12:55 PM, Avi Kivity wrote: Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by Anthony. There's numerous ways that this can break

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Anthony Liguori
On 03/22/2010 02:31 PM, Daniel P. Berrange wrote: On Mon, Mar 22, 2010 at 02:15:35PM -0500, Anthony Liguori wrote: On 03/22/2010 12:55 PM, Avi Kivity wrote: Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by Anthony. There's numerous ways that this can break:

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Daniel P. Berrange
On Mon, Mar 22, 2010 at 02:15:35PM -0500, Anthony Liguori wrote: > On 03/22/2010 12:55 PM, Avi Kivity wrote: > >>Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by > >>Anthony. > >>There's numerous ways that this can break: > > > >I don't like it either. We have libvirt for enume

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Avi Kivity
On 03/22/2010 09:20 PM, Joerg Roedel wrote: Why doesnt it solve the bisectability problem? The kernel repo is supposed to be bisectable so that problem would be solved. Because Marcelo and Avi try to keep as close to upstream qemu as possible. So the qemu repo is regularly merged in qemu

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Anthony Liguori
On 03/22/2010 02:22 PM, Ingo Molnar wrote: Transitive had a product that was using a KVM context to run their binary translator which allowed them full access to the host processes virtual address space range. In this case, there is no kernel and there are no devices. And your point is th

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Andrea Arcangeli
On Mon, Mar 22, 2010 at 08:10:28PM +0100, Ingo Molnar wrote: > I posit that it's both: and that priorities can be communicated - if only you > try as a maintainer. All i'm suggesting is to add 'usable, unified > user-space' > to the list of unfun priorities, because it's possible and because it

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Ingo Molnar
* Anthony Liguori wrote: > On 03/22/2010 12:34 PM, Ingo Molnar wrote: > >This is really just the much-discredited microkernel approach for keeping > >global enumeration data that should be kept by the kernel ... > > > >Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by Anthony.

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Avi Kivity
On 03/22/2010 09:10 PM, Ingo Molnar wrote: * Avi Kivity wrote: On 03/22/2010 06:32 PM, Ingo Molnar wrote: So, what do you think creates code communities and keeps them alive? Developers and code. And the wellbeing of developers are primarily influenced by the repository structure an

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Ingo Molnar
* Anthony Liguori wrote: > On 03/22/2010 12:34 PM, Ingo Molnar wrote: > >* Avi Kivity wrote: > > > > - Easy default reference to guest instances, and a way for tools to > >reference them symbolically as well in the multi-guest case. > > Preferably > >something trustabl

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Joerg Roedel
On Mon, Mar 22, 2010 at 05:32:15PM +0100, Ingo Molnar wrote: > I dont know how you can find the situation of Alpha comparable, which is a > legacy architecture for which no new CPU was manufactored in the past ~10 > years. > > The negative effects of physical obscolescence cannot be overcome eve

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Ingo Molnar
* Avi Kivity wrote: > > Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by > > Anthony. There's numerous ways that this can break: > > I don't like it either. We have libvirt for enumerating guests. Which has pretty much the same problems to the ${HOME}/.qemu/qmp/ solution,

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Anthony Liguori
On 03/22/2010 02:10 PM, Ingo Molnar wrote: I posit that it's both: and that priorities can be communicated - if only you try as a maintainer. All i'm suggesting is to add 'usable, unified user-space' to the list of unfun priorities, because it's possible and because it matters. I've spent

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Anthony Liguori
On 03/22/2010 12:55 PM, Avi Kivity wrote: Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by Anthony. There's numerous ways that this can break: I don't like it either. We have libvirt for enumerating guests. We're stuck in a rut with libvirt and I think a lot of the dissa

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Ingo Molnar
* Avi Kivity wrote: > On 03/22/2010 06:32 PM, Ingo Molnar wrote: > > > > So, what do you think creates code communities and keeps them alive? > > Developers and code. And the wellbeing of developers are primarily > > influenced by the repository structure and by the development/maintenance >

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Avi Kivity
On 03/22/2010 04:54 PM, Ingo Molnar wrote: * Pekka Enberg wrote: Hi Avi, On Mon, Mar 22, 2010 at 2:49 PM, Avi Kivity wrote: Seems like perf is also split, with sysprof being developed outside the kernel. ?Will you bring sysprof into the kernel? ?Will every feature be duplicated in

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Avi Kivity
On 03/22/2010 08:10 PM, Pekka Enberg wrote: On Mon, Mar 22, 2010 at 8:04 PM, Avi Kivity wrote: That said, pulling 400 KLOC of code into the kernel sounds really excessive. Would we need all that if we just do native virtualization and no actual emulation? What is native virtualizat

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Anthony Liguori
On 03/22/2010 12:34 PM, Ingo Molnar wrote: This is really just the much-discredited microkernel approach for keeping global enumeration data that should be kept by the kernel ... Lets look at the ${HOME}/.qemu/qmp/ enumeration method suggested by Anthony. There's numerous ways that this can brea

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Anthony Liguori
On 03/22/2010 12:34 PM, Ingo Molnar wrote: * Avi Kivity wrote: - Easy default reference to guest instances, and a way for tools to reference them symbolically as well in the multi-guest case. Preferably something trustable and kernel-provided - not some indirect information l

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Anthony Liguori
On 03/22/2010 12:11 PM, Ingo Molnar wrote: * Anthony Liguori wrote: - Easy default reference to guest instances, and a way for tools to reference them symbolically as well in the multi-guest case. Preferably something trustable and kernel-provided - not some indirect information

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Anthony Liguori
On 03/22/2010 11:59 AM, Ingo Molnar wrote: Ok, that sounds interesting! I'd rather see some raw mechanism that 'perf kvm' could use instead of having to require yet another library (which generally dampens adoption of a tool). So i think we can work from there. You can access the protocol

Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-22 Thread Avi Kivity
On 03/22/2010 04:47 PM, Ingo Molnar wrote: If you are interested in the first-hand experience of the people who are doing the perf work then here it is: by far the biggest reason for perf success and perf usability is the integration of the user-space tooling with the kernel-space bits, into a

  1   2   3   >