Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-16 Thread LluĂ­s Vilanova
Richard W M Jones writes: > On Sat, Oct 15, 2011 at 10:00:02AM +, Blue Swirl wrote: >> I don't understand why PC can't use the same way of loading initrd by >> QEMU to guest memory before boot as Sparc32 uses. It should even be >> possible to deduplicate the kernel and initrd images: improve t

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-15 Thread Richard W.M. Jones
On Sat, Oct 15, 2011 at 10:00:02AM +, Blue Swirl wrote: > I don't understand why PC can't use the same way of loading initrd by > QEMU to guest memory before boot as Sparc32 uses. It should even be > possible to deduplicate the kernel and initrd images: improve the > loader to use mmap() for lo

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-15 Thread Kevin O'Connor
On Tue, Oct 11, 2011 at 08:17:28AM -0500, Anthony Liguori wrote: > On 10/11/2011 04:55 AM, Avi Kivity wrote: > >On 10/11/2011 11:50 AM, Gleb Natapov wrote: > >>But loading MBs of data through fw_cfg interface is just abusing it. > >>You wouldn't use pio on real HW to move megabytes of data and expe

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-15 Thread Blue Swirl
On Tue, Oct 11, 2011 at 8:23 AM, Daniel P. Berrange wrote: > On Mon, Oct 10, 2011 at 09:01:52PM +0200, Alexander Graf wrote: >> >> On 10.10.2011, at 20:53, Anthony Liguori wrote: >> >> > On 10/10/2011 12:08 PM, Daniel P. Berrange wrote: >> >> With the attached patches applied to QEMU and SeaBios,

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Daniel P. Berrange
On Tue, Oct 11, 2011 at 09:33:49AM -0500, Anthony Liguori wrote: > On 10/11/2011 09:01 AM, Daniel P. Berrange wrote: > >On Tue, Oct 11, 2011 at 08:19:14AM -0500, Anthony Liguori wrote: > >>On 10/11/2011 08:14 AM, Alexander Graf wrote: > >>>And I don't see the point why we would have to shoot ye

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Alexander Graf
On 11.10.2011, at 16:33, Anthony Liguori wrote: > On 10/11/2011 09:01 AM, Daniel P. Berrange wrote: >> On Tue, Oct 11, 2011 at 08:19:14AM -0500, Anthony Liguori wrote: >>> On 10/11/2011 08:14 AM, Alexander Graf wrote: And I don't see the point why we would have to shoot yet another hole

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Anthony Liguori
On 10/11/2011 09:01 AM, Daniel P. Berrange wrote: On Tue, Oct 11, 2011 at 08:19:14AM -0500, Anthony Liguori wrote: On 10/11/2011 08:14 AM, Alexander Graf wrote: And I don't see the point why we would have to shoot yet another hole into the guest just because we're too unwilling to make an inte

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Daniel P. Berrange
On Tue, Oct 11, 2011 at 08:19:14AM -0500, Anthony Liguori wrote: > On 10/11/2011 08:14 AM, Alexander Graf wrote: > >And I don't see the point why we would have to shoot yet another hole > >into the guest just because we're too unwilling to make an interface > >that's perfectly valid h

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Anthony Liguori
On 10/11/2011 08:45 AM, Avi Kivity wrote: On 10/11/2011 03:29 PM, Anthony Liguori wrote: On 10/11/2011 08:23 AM, Avi Kivity wrote: On 10/11/2011 03:19 PM, Anthony Liguori wrote: No, DMA has a lot bigger granularities in kvm/user interaction. We can easily DMA a 50MB region with a single kvm/us

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Avi Kivity
On 10/11/2011 03:29 PM, Anthony Liguori wrote: > On 10/11/2011 08:23 AM, Avi Kivity wrote: >> On 10/11/2011 03:19 PM, Anthony Liguori wrote: No, DMA has a lot bigger granularities in kvm/user interaction. We can easily DMA a 50MB region with a single kvm/user exit. For PIO we can at

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Anthony Liguori
On 10/11/2011 08:23 AM, Avi Kivity wrote: On 10/11/2011 03:19 PM, Anthony Liguori wrote: No, DMA has a lot bigger granularities in kvm/user interaction. We can easily DMA a 50MB region with a single kvm/user exit. For PIO we can at most do page granularity. So make a proper PCI device for ker

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Gleb Natapov
On Tue, Oct 11, 2011 at 03:23:36PM +0200, Avi Kivity wrote: > On 10/11/2011 03:19 PM, Anthony Liguori wrote: > >> No, DMA has a lot bigger granularities in kvm/user interaction. We > >> can easily DMA a 50MB region with a single kvm/user exit. For PIO we > >> can at most do page granularity. > > >

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Avi Kivity
On 10/11/2011 03:19 PM, Anthony Liguori wrote: >> No, DMA has a lot bigger granularities in kvm/user interaction. We >> can easily DMA a 50MB region with a single kvm/user exit. For PIO we >> can at most do page granularity. > > > So make a proper PCI device for kernel loading. It's a much more >

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Gleb Natapov
On Tue, Oct 11, 2011 at 08:17:28AM -0500, Anthony Liguori wrote: > On 10/11/2011 04:55 AM, Avi Kivity wrote: > >On 10/11/2011 11:50 AM, Gleb Natapov wrote: > >>On Tue, Oct 11, 2011 at 11:26:14AM +0200, Avi Kivity wrote: > >>> rep/ins is exactly like dma+wait for this use case: provide an > >>> addr

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Anthony Liguori
On 10/11/2011 08:14 AM, Alexander Graf wrote: And I don't see the point why we would have to shoot yet another hole into the guest just because we're too unwilling to make an interface that's perfectly valid horribly slow. rep/ins is exactly like dma+wait for this use case: provide an address

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Gleb Natapov
On Tue, Oct 11, 2011 at 03:14:52PM +0200, Alexander Graf wrote: > > rep/ins is effectively equivalent to DMA except in how it's handled within > > QEMU. > > No, DMA has a lot bigger granularities in kvm/user interaction. We can easily > DMA a 50MB region with a single kvm/user exit. For PIO we c

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Anthony Liguori
On 10/11/2011 04:55 AM, Avi Kivity wrote: On 10/11/2011 11:50 AM, Gleb Natapov wrote: On Tue, Oct 11, 2011 at 11:26:14AM +0200, Avi Kivity wrote: > rep/ins is exactly like dma+wait for this use case: provide an > address, get a memory image in return. There's no need to add > another interface,

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Alexander Graf
On 11.10.2011, at 15:12, Anthony Liguori wrote: > On 10/11/2011 04:38 AM, Alexander Graf wrote: >> >> On 11.10.2011, at 11:26, Avi Kivity wrote: >> >>> On 10/11/2011 11:19 AM, Alexander Graf wrote: >> >> Of this, 1.4 seconds is the time required by LinuxBoot to copy the >> kernel

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Anthony Liguori
On 10/11/2011 04:38 AM, Alexander Graf wrote: On 11.10.2011, at 11:26, Avi Kivity wrote: On 10/11/2011 11:19 AM, Alexander Graf wrote: Of this, 1.4 seconds is the time required by LinuxBoot to copy the kernel+initrd. If I used an uncompressed initrd, which I really want to, to avoid de

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Gleb Natapov
On Tue, Oct 11, 2011 at 11:59:45AM +0200, Avi Kivity wrote: > On 10/11/2011 11:56 AM, Gleb Natapov wrote: > >On Tue, Oct 11, 2011 at 11:49:16AM +0200, Avi Kivity wrote: > >> >Whatever we do, the interface will never be as fast as DMA. We will > >> always have to do sanity / permission checks for

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Daniel P. Berrange
On Tue, Oct 11, 2011 at 11:50:01AM +0200, Avi Kivity wrote: > On 10/11/2011 11:49 AM, Daniel P. Berrange wrote: > >On Tue, Oct 11, 2011 at 11:39:36AM +0200, Avi Kivity wrote: > >> On 10/11/2011 11:27 AM, Daniel P. Berrange wrote: > >> >> >For comparison I also did a test building a bootable ISO

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Avi Kivity
On 10/11/2011 11:56 AM, Gleb Natapov wrote: On Tue, Oct 11, 2011 at 11:49:16AM +0200, Avi Kivity wrote: > >Whatever we do, the interface will never be as fast as DMA. We will always have to do sanity / permission checks for every IO operation, can batch up only so many IO requests and in QEMU

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Gleb Natapov
On Tue, Oct 11, 2011 at 11:49:16AM +0200, Avi Kivity wrote: > >Whatever we do, the interface will never be as fast as DMA. We will always > >have to do sanity / permission checks for every IO operation, can batch up > >only so many IO requests and in QEMU again have to call our callbacks in a >

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Avi Kivity
On 10/11/2011 11:50 AM, Gleb Natapov wrote: On Tue, Oct 11, 2011 at 11:26:14AM +0200, Avi Kivity wrote: > rep/ins is exactly like dma+wait for this use case: provide an > address, get a memory image in return. There's no need to add > another interface, we should just optimize the existing on

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Gleb Natapov
On Tue, Oct 11, 2011 at 11:26:14AM +0200, Avi Kivity wrote: > rep/ins is exactly like dma+wait for this use case: provide an > address, get a memory image in return. There's no need to add > another interface, we should just optimize the existing one. > rep/ins cannot be optimized to be as effici

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Avi Kivity
On 10/11/2011 11:49 AM, Daniel P. Berrange wrote: On Tue, Oct 11, 2011 at 11:39:36AM +0200, Avi Kivity wrote: > On 10/11/2011 11:27 AM, Daniel P. Berrange wrote: > >> >For comparison I also did a test building a bootable ISO using ISOLinux. > >> >This required 700 ms for the boot time, wh

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Avi Kivity
On 10/11/2011 11:38 AM, Alexander Graf wrote: > >> and gets you in a pretty bizarre state when doing updates of your host files, since then you have 2 different paths: full boot and restore. That's yet another potential source for bugs. > > Typically you'd check the timestamps to make sure yo

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Daniel P. Berrange
On Tue, Oct 11, 2011 at 11:39:36AM +0200, Avi Kivity wrote: > On 10/11/2011 11:27 AM, Daniel P. Berrange wrote: > >> >For comparison I also did a test building a bootable ISO using ISOLinux. > >> >This required 700 ms for the boot time, which is appoximately 1/2 the > >> >time reqiured for direc

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Avi Kivity
On 10/11/2011 11:27 AM, Daniel P. Berrange wrote: > > One thing we can do is boot a guest and immediately snapshot it, > before it runs any application specific code. Subsequent > invocations will MAP_PRIVATE the memory image and COW their way. > This avoids the kernel initialization time as

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Alexander Graf
On 11.10.2011, at 11:26, Avi Kivity wrote: > On 10/11/2011 11:19 AM, Alexander Graf wrote: >> >> >> >> Of this, 1.4 seconds is the time required by LinuxBoot to copy the >> >> kernel+initrd. If I used an uncompressed initrd, which I really want >> >> to, to avoid decompression overhead, this i

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Daniel P. Berrange
On Tue, Oct 11, 2011 at 11:15:05AM +0200, Avi Kivity wrote: > On 10/11/2011 10:23 AM, Daniel P. Berrange wrote: > > - Application sandbox, directly boots the regular host's kernel and > >a custom initrd image. The initrd does not contain any files except > >for the 9p kernel modules and a

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Avi Kivity
On 10/11/2011 11:19 AM, Alexander Graf wrote: >> >> Of this, 1.4 seconds is the time required by LinuxBoot to copy the >> kernel+initrd. If I used an uncompressed initrd, which I really want >> to, to avoid decompression overhead, this increases to ~1.7 seconds. >> So the LinuxBoot ROM is ~60

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Alexander Graf
On 11.10.2011, at 11:15, Avi Kivity wrote: > On 10/11/2011 10:23 AM, Daniel P. Berrange wrote: >> - Application sandbox, directly boots the regular host's kernel and >>a custom initrd image. The initrd does not contain any files except >>for the 9p kernel modules and a custom init binary

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Avi Kivity
On 10/11/2011 10:23 AM, Daniel P. Berrange wrote: - Application sandbox, directly boots the regular host's kernel and a custom initrd image. The initrd does not contain any files except for the 9p kernel modules and a custom init binary, which mounts the guest root FS from a 9p file

Re: [Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Richard W.M. Jones
On Tue, Oct 11, 2011 at 09:23:15AM +0100, Daniel P. Berrange wrote: > - libguestfs directly boots its appliance using the regular host's >kernel image and a custom built initrd image. The initrd does >not contain the entire appliance, just enough to boot up and >dynamically read files

[Qemu-devel] Slow kernel/initrd loading via fw_cfg; Was Re: Hack integrating SeaBios / LinuxBoot option rom with QEMU trace backends

2011-10-11 Thread Daniel P. Berrange
On Mon, Oct 10, 2011 at 09:01:52PM +0200, Alexander Graf wrote: > > On 10.10.2011, at 20:53, Anthony Liguori wrote: > > > On 10/10/2011 12:08 PM, Daniel P. Berrange wrote: > >> With the attached patches applied to QEMU and SeaBios, the attached > >> systemtap script can be used to debug timings i