Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Avi Kivity
On 09/12/2010 07:19 PM, Anthony Liguori wrote: On 09/12/2010 11:45 AM, Avi Kivity wrote: Streaming relies on copy-on-read to do the writing. Ah. You can avoid the copy-on-read implementation in the block format driver and do it completely in generic code. Copy on read takes advantage of

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Anthony Liguori
On 09/12/2010 11:45 AM, Avi Kivity wrote: Streaming relies on copy-on-read to do the writing. Ah. You can avoid the copy-on-read implementation in the block format driver and do it completely in generic code. Copy on read takes advantage of temporal locality. You wouldn't want to stream

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Avi Kivity
On 09/12/2010 05:23 PM, Anthony Liguori wrote: On 09/12/2010 08:40 AM, Avi Kivity wrote: Why would it serialize all I/O operations? It's just like another vcpu issuing reads. Because the block layer isn't re-entrant. A threaded block layer is reentrant. Of course pushing the thing into a

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Anthony Liguori
On 09/12/2010 08:40 AM, Avi Kivity wrote: Why would it serialize all I/O operations? It's just like another vcpu issuing reads. Because the block layer isn't re-entrant. What you basically do is: stream_step_three(): complete() stream_step_two(offset, length): bdrv_aio_readv(offset,

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Avi Kivity
On 09/12/2010 03:25 PM, Anthony Liguori wrote: On 09/12/2010 07:41 AM, Avi Kivity wrote: On 09/07/2010 05:57 PM, Anthony Liguori wrote: I agree that streaming should be generic, like block migration. The trivial generic implementation is: void bdrv_stream(BlockDriverState* bs) { for (s

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Anthony Liguori
On 09/12/2010 07:41 AM, Avi Kivity wrote: On 09/07/2010 05:57 PM, Anthony Liguori wrote: I agree that streaming should be generic, like block migration. The trivial generic implementation is: void bdrv_stream(BlockDriverState* bs) { for (sector = 0; sector< bdrv_getlength(bs); sector +=

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Avi Kivity
On 09/07/2010 05:57 PM, Anthony Liguori wrote: I agree that streaming should be generic, like block migration. The trivial generic implementation is: void bdrv_stream(BlockDriverState* bs) { for (sector = 0; sector< bdrv_getlength(bs); sector += n) { if (!bdrv_is_allocated(bs, s

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-12 Thread Avi Kivity
On 09/07/2010 04:41 PM, Anthony Liguori wrote: Hi, We've got copy-on-read and image streaming working in QED and before going much further, I wanted to bounce some interfaces off of the libvirt folks to make sure our final interface makes sense. Here's the basic idea: Today, you can create

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-08 Thread Kevin Wolf
Am 07.09.2010 17:09, schrieb Stefan Hajnoczi: > Right, I'm a little hesitant to get too far into discussing the > management interface because I remember long threads about polling and > async. I never fully read them but I bet some wisdom came out of them > that applies here. > > There are two w

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 10:09 AM, Stefan Hajnoczi wrote: Right, so that argues for an incremental interface like I started with :-) BTW, this whole discussion is also relevant for other background tasks like online defragmentation so keep that use-case in mind too. Right, I'm a little hesitant to g

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Kevin Wolf
Am 07.09.2010 17:11, schrieb Anthony Liguori: > On 09/07/2010 10:02 AM, Kevin Wolf wrote: >> Am 07.09.2010 16:49, schrieb Anthony Liguori: >> Shouldn't it be a runtime option? You can use the very same image with copy-on-read or copy-on-write and it will behave the same (execpt for >>

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 10:39 AM, Kevin Wolf wrote: Am 07.09.2010 17:30, schrieb Anthony Liguori: On 09/07/2010 10:20 AM, Kevin Wolf wrote: Am 07.09.2010 17:11, schrieb Anthony Liguori: Copy-on-read is, in many cases, a property of the backing file because it suggests that the backing

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Kevin Wolf
Am 07.09.2010 17:30, schrieb Anthony Liguori: > On 09/07/2010 10:20 AM, Kevin Wolf wrote: >> Am 07.09.2010 17:11, schrieb Anthony Liguori: >>> Copy-on-read is, in many cases, a property of the backing file because >>> it suggests that the backing file is either very slow or potentially >>> volatile

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 10:20 AM, Kevin Wolf wrote: Am 07.09.2010 17:11, schrieb Anthony Liguori: On 09/07/2010 10:02 AM, Kevin Wolf wrote: Am 07.09.2010 16:49, schrieb Anthony Liguori: Shouldn't it be a runtime option? You can use the very same image with copy-on-read or copy-on-writ

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 10:05 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 3:57 PM, Anthony Liguori wrote: On 09/07/2010 09:49 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 3:34 PM, Kevin Wolfwrote: Am 07.09.2010 15:41, schrieb Anthony Liguori: Hi, We've go

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Stefan Hajnoczi
On Tue, Sep 7, 2010 at 4:00 PM, Anthony Liguori wrote: > On 09/07/2010 09:55 AM, Stefan Hajnoczi wrote: >> >> On Tue, Sep 7, 2010 at 3:51 PM, Anthony Liguori >>  wrote: >> >>> >>> On 09/07/2010 09:33 AM, Stefan Hajnoczi wrote: >>> On Tue, Sep 7, 2010 at 2:41 PM, Anthony Liguori  

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 10:02 AM, Kevin Wolf wrote: Am 07.09.2010 16:49, schrieb Anthony Liguori: Shouldn't it be a runtime option? You can use the very same image with copy-on-read or copy-on-write and it will behave the same (execpt for performance), so it's not an inherent feature of the image file

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Stefan Hajnoczi
On Tue, Sep 7, 2010 at 3:57 PM, Anthony Liguori wrote: > On 09/07/2010 09:49 AM, Stefan Hajnoczi wrote: >> >> On Tue, Sep 7, 2010 at 3:34 PM, Kevin Wolf  wrote: >> >>> >>> Am 07.09.2010 15:41, schrieb Anthony Liguori: >>> Hi, We've got copy-on-read and image streaming working i

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 09:55 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 3:51 PM, Anthony Liguori wrote: On 09/07/2010 09:33 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 2:41 PM, Anthony Liguori wrote: The interface for copy-on-read is just an option within qemu-img

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Kevin Wolf
Am 07.09.2010 16:49, schrieb Anthony Liguori: >> Shouldn't it be a runtime option? You can use the very same image with >> copy-on-read or copy-on-write and it will behave the same (execpt for >> performance), so it's not an inherent feature of the image file. >> > > The way it's implemented i

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Stefan Hajnoczi
On Tue, Sep 7, 2010 at 3:51 PM, Anthony Liguori wrote: > On 09/07/2010 09:33 AM, Stefan Hajnoczi wrote: >> >> On Tue, Sep 7, 2010 at 2:41 PM, Anthony Liguori >>  wrote: >> >>> >>> The interface for copy-on-read is just an option within qemu-img create. >>>  Streaming, on the other hand, requires

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Kevin Wolf
Am 07.09.2010 15:41, schrieb Anthony Liguori: > Hi, > > We've got copy-on-read and image streaming working in QED and before > going much further, I wanted to bounce some interfaces off of the > libvirt folks to make sure our final interface makes sense. > > Here's the basic idea: > > Today, y

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Stefan Hajnoczi
On Tue, Sep 7, 2010 at 3:34 PM, Kevin Wolf wrote: > Am 07.09.2010 15:41, schrieb Anthony Liguori: >> Hi, >> >> We've got copy-on-read and image streaming working in QED and before >> going much further, I wanted to bounce some interfaces off of the >> libvirt folks to make sure our final interface

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Stefan Hajnoczi
On Tue, Sep 7, 2010 at 2:41 PM, Anthony Liguori wrote: > The interface for copy-on-read is just an option within qemu-img create. >  Streaming, on the other hand, requires a bit more thought.  Today, I have a > monitor command that does the following: > > stream > > Which will try to stream the

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 09:49 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 3:34 PM, Kevin Wolf wrote: Am 07.09.2010 15:41, schrieb Anthony Liguori: Hi, We've got copy-on-read and image streaming working in QED and before going much further, I wanted to bounce some interfaces off of the l

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 09:33 AM, Stefan Hajnoczi wrote: On Tue, Sep 7, 2010 at 2:41 PM, Anthony Liguori wrote: The interface for copy-on-read is just an option within qemu-img create. Streaming, on the other hand, requires a bit more thought. Today, I have a monitor command that does the followi

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 09:34 AM, Kevin Wolf wrote: Am 07.09.2010 15:41, schrieb Anthony Liguori: Hi, We've got copy-on-read and image streaming working in QED and before going much further, I wanted to bounce some interfaces off of the libvirt folks to make sure our final interface makes sense. Her

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Anthony Liguori
On 09/07/2010 09:01 AM, Alexander Graf wrote: I'm torn here too. Why not expose both? Have a qemu internal daemon available that gets a sleep time as parameter and an external "pull sectors" command. We'll see which one is more useful, but I don't think it's too much code to justify only having

Re: [libvirt] [Qemu-devel] QEMU interfaces for image streaming and post-copy block migration

2010-09-07 Thread Alexander Graf
On 07.09.2010, at 15:41, Anthony Liguori wrote: > Hi, > > We've got copy-on-read and image streaming working in QED and before going > much further, I wanted to bounce some interfaces off of the libvirt folks to > make sure our final interface makes sense. > > Here's the basic idea: > > Toda