Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-21 Thread Stefan Hajnoczi
On Tue, Feb 21, 2012 at 4:01 PM, Markus Armbruster wrote: > Anthony Liguori writes: > >> On 02/07/2012 07:50 AM, Stefan Hajnoczi wrote: >>> On Tue, Feb 7, 2012 at 1:34 PM, Kevin Wolf  wrote: Am 07.02.2012 11:29, schrieb Ori Mamluk: > Repagent is a new module that allows an external repli

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-21 Thread Markus Armbruster
Anthony Liguori writes: > On 02/07/2012 07:50 AM, Stefan Hajnoczi wrote: >> On Tue, Feb 7, 2012 at 1:34 PM, Kevin Wolf wrote: >>> Am 07.02.2012 11:29, schrieb Ori Mamluk: Repagent is a new module that allows an external replication system to replicate a volume of a Qemu VM. >> >> I rec

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-08 Thread Stefan Hajnoczi
On Wed, Feb 8, 2012 at 2:55 PM, Paolo Bonzini wrote: > On 02/08/2012 03:39 PM, Stefan Hajnoczi wrote: >> >> >>> >  What about taking the existing Ceph/RBD driver in QEMU and changing it >>> > to >>> >  support arbitrary image formats rather than just raw?  That sounds >>> > much much >>> >  easier

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-08 Thread Paolo Bonzini
On 02/08/2012 03:39 PM, Stefan Hajnoczi wrote: > What about taking the existing Ceph/RBD driver in QEMU and changing it to > support arbitrary image formats rather than just raw? That sounds much much > easier. The main advantage is that Ceph has a user-space library for use in > the repl

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-08 Thread Stefan Hajnoczi
On Wed, Feb 8, 2012 at 12:46 PM, Paolo Bonzini wrote: > On 02/08/2012 01:03 PM, Stefan Hajnoczi wrote: >> >> If you intend to run an unmodified drbd server on the rephub, then it >> may not be possible to get point-in-time backups.  (Although this >> probably depends since things like btrfs or zfs

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-08 Thread Paolo Bonzini
On 02/08/2012 01:03 PM, Stefan Hajnoczi wrote: If you intend to run an unmodified drbd server on the rephub, then it may not be possible to get point-in-time backups. (Although this probably depends since things like btrfs or zfs may allow you to get back to arbitrary transactions or timestamps.

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-08 Thread Orit Wasserman
On 02/07/2012 04:45 PM, Ori Mamluk wrote: > On 07/02/2012 15:34, Kevin Wolf wrote: >> Am 07.02.2012 11:29, schrieb Ori Mamluk: >>> Repagent is a new module that allows an external replication system to >>> replicate a volume of a Qemu VM. >>> >>> This RFC patch adds the repagent client module to Qe

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-08 Thread Orit Wasserman
On 02/07/2012 04:05 PM, Paolo Bonzini wrote: > On 02/07/2012 02:50 PM, Stefan Hajnoczi wrote: >>> I guess we might need to attach multiple QMP monitors for this to work >>> (one for libvirt, one for the rephub). I'm not sure if there is a >>> fundamental problem with this or if it just needs to be

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-08 Thread Stefan Hajnoczi
On Wed, Feb 8, 2012 at 6:10 AM, Ori Mamluk wrote: > On 07/02/2012 17:47, Paolo Bonzini wrote: >> But I'm also starting to wonder whether it would be simpler to use >> existing replication code.  DRBD is more feature-rich, and you can use it >> over loopback or NBD devices (respectively raw and non

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-08 Thread Stefan Hajnoczi
2012/2/8 Dor Laor : > On 02/08/2012 08:10 AM, Ori Mamluk wrote: >> >> 2. drbd is 'below' all the Qemu block layers - if the protected volume >> is qcow2 then drbd doesn't get the raw IOs, right? > > > That's one of the major caveats in drbd/iscsi/nbd - there is no support for > block level snapshot

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-08 Thread Luiz Capitulino
On Tue, 07 Feb 2012 14:34:12 +0100 Kevin Wolf wrote: > The other message types could possibly be implemented as QMP commands. I > guess we might need to attach multiple QMP monitors for this to work > (one for libvirt, one for the rephub). I'm not sure if there is a > fundamental problem with thi

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-08 Thread Stefan Hajnoczi
On Wed, Feb 8, 2012 at 6:10 AM, Ori Mamluk wrote: > 2. drbd is 'below' all the Qemu block layers - if the protected volume is > qcow2 then drbd doesn't get the raw IOs, right? No, if you look at the layers again: Guest -> QEMU -> drbd -> local NBD server The local NBD server runs the qcow2, qed

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-08 Thread Kevin Wolf
Am 08.02.2012 10:47, schrieb Ori Mamluk: > On 08/02/2012 10:55, Kevin Wolf wrote: >> Am 08.02.2012 07:10, schrieb Ori Mamluk: >>> On 07/02/2012 17:47, Paolo Bonzini wrote: Why can't QEMU itself stream the full volume in the background, and send that together with any new I/O? Is it

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-08 Thread Ori Mamluk
On 08/02/2012 10:55, Kevin Wolf wrote: Am 08.02.2012 07:10, schrieb Ori Mamluk: On 07/02/2012 17:47, Paolo Bonzini wrote: Why can't QEMU itself stream the full volume in the background, and send that together with any new I/O? Is it because the rephub knows which parts are out-of-date and nee

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-08 Thread Kevin Wolf
Am 08.02.2012 07:10, schrieb Ori Mamluk: > On 07/02/2012 17:47, Paolo Bonzini wrote: >> On 02/07/2012 03:48 PM, Ori Mamluk wrote: The current streaming code in QEMU only deals with the former. Streaming to a remote server would not be supported. >>> I need it at the same time. The Re

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-08 Thread Dor Laor
On 02/08/2012 08:10 AM, Ori Mamluk wrote: On 07/02/2012 17:47, Paolo Bonzini wrote: On 02/07/2012 03:48 PM, Ori Mamluk wrote: The current streaming code in QEMU only deals with the former. Streaming to a remote server would not be supported. I need it at the same time. The Rephub reads either

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Ori Mamluk
On 07/02/2012 17:47, Paolo Bonzini wrote: On 02/07/2012 03:48 PM, Ori Mamluk wrote: The current streaming code in QEMU only deals with the former. Streaming to a remote server would not be supported. I need it at the same time. The Rephub reads either the full volume or parts of, and concurren

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Dor Laor
On 02/07/2012 02:12 PM, Anthony Liguori wrote: Hi, On 02/07/2012 04:29 AM, Ori Mamluk wrote: Repagent is a new module that allows an external replication system to replicate a volume of a Qemu VM. This RFC patch adds the repagent client module to Qemu. Please read http://wiki.qemu.org/Contri

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Anthony Liguori
On 02/07/2012 09:20 AM, Stefan Hajnoczi wrote: On Tue, Feb 7, 2012 at 2:59 PM, Anthony Liguori wrote: On 02/07/2012 07:50 AM, Stefan Hajnoczi wrote: On Tue, Feb 7, 2012 at 1:34 PM, Kevin Wolfwrote: Am 07.02.2012 11:29, schrieb Ori Mamluk: Repagent is a new module that allows an extern

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Paolo Bonzini
On 02/07/2012 03:48 PM, Ori Mamluk wrote: The current streaming code in QEMU only deals with the former. Streaming to a remote server would not be supported. I need it at the same time. The Rephub reads either the full volume or parts of, and concurrently protects new IOs. Why can't QEMU itse

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Kevin Wolf
Am 07.02.2012 15:40, schrieb Paolo Bonzini: > On 02/07/2012 03:06 PM, Ori Mamluk wrote: >> The main issue about it is that the Rephub also needs the other >> direction - to read the protected volume. I get the feeling that with >> live block copy and NBD there's probably something that might fit >>

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Stefan Hajnoczi
On Tue, Feb 7, 2012 at 2:59 PM, Anthony Liguori wrote: > On 02/07/2012 07:50 AM, Stefan Hajnoczi wrote: >> >> On Tue, Feb 7, 2012 at 1:34 PM, Kevin Wolf  wrote: >>> >>> Am 07.02.2012 11:29, schrieb Ori Mamluk: Repagent is a new module that allows an external replication system to re

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Anthony Liguori
On 02/07/2012 08:40 AM, Paolo Bonzini wrote: On 02/07/2012 03:06 PM, Ori Mamluk wrote: The main issue about it is that the Rephub also needs the other direction - to read the protected volume. I get the feeling that with live block copy and NBD there's probably something that might fit this need

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Anthony Liguori
On 02/07/2012 07:50 AM, Stefan Hajnoczi wrote: On Tue, Feb 7, 2012 at 1:34 PM, Kevin Wolf wrote: Am 07.02.2012 11:29, schrieb Ori Mamluk: Repagent is a new module that allows an external replication system to replicate a volume of a Qemu VM. I recently joked with Kevin that QEMU is on its wa

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Ori Mamluk
On 07/02/2012 16:40, Paolo Bonzini wrote: On 02/07/2012 03:06 PM, Ori Mamluk wrote: The main issue about it is that the Rephub also needs the other direction - to read the protected volume. I get the feeling that with live block copy and NBD there's probably something that might fit this need, n

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Ori Mamluk
On 07/02/2012 15:34, Kevin Wolf wrote: Am 07.02.2012 11:29, schrieb Ori Mamluk: Repagent is a new module that allows an external replication system to replicate a volume of a Qemu VM. This RFC patch adds the repagent client module to Qemu. Documentation of the module role and API is in the p

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Paolo Bonzini
On 02/07/2012 03:06 PM, Ori Mamluk wrote: The main issue about it is that the Rephub also needs the other direction - to read the protected volume. I get the feeling that with live block copy and NBD there's probably something that might fit this need, no? Yes, with two NBD sockets you could do

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Ori Mamluk
On 07/02/2012 15:50, Stefan Hajnoczi wrote: First let me say that I'm not completely used to the inline replies - so I initially missed some of your mail before. On Tue, Feb 7, 2012 at 1:34 PM, Kevin Wolf wrote: Am 07.02.2012 11:29, schrieb Ori Mamluk: Repagent is a new module that allows an

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Ori Mamluk
guori Cc: Ori Mamluk; Kevin Wolf; qemu-devel@nongnu.org Subject: Re: [Qemu-devel] [RFC PATCH] replication agent module On 02/07/2012 02:12 PM, Anthony Liguori wrote: Hi, On 02/07/2012 04:29 AM, Ori Mamluk wrote: Repagent is a new module that allows an external replication system to replicate a volume

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Paolo Bonzini
On 02/07/2012 02:50 PM, Stefan Hajnoczi wrote: I guess we might need to attach multiple QMP monitors for this to work (one for libvirt, one for the rephub). I'm not sure if there is a fundamental problem with this or if it just needs to be done. Agreed. You can already query block devices usin

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Paolo Bonzini
On 02/07/2012 02:50 PM, Stefan Hajnoczi wrote: We still haven't figured out a nice way to make image files full-fledged Linux block devices, so we're reimplementing all the block code in QEMU userspace. What about my favorite hammer, NBD? Paolo

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Stefan Hajnoczi
On Tue, Feb 7, 2012 at 1:34 PM, Kevin Wolf wrote: > Am 07.02.2012 11:29, schrieb Ori Mamluk: >> Repagent is a new module that allows an external replication system to >> replicate a volume of a Qemu VM. I recently joked with Kevin that QEMU is on its way to reimplementing the Linux block and devi

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Kevin Wolf
Am 07.02.2012 11:29, schrieb Ori Mamluk: > Repagent is a new module that allows an external replication system to > replicate a volume of a Qemu VM. > > This RFC patch adds the repagent client module to Qemu. > > > > Documentation of the module role and API is in the patch at > replication/qem

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Anthony Liguori
to:dl...@redhat.com] Sent: Tuesday, February 07, 2012 2:25 PM To: Anthony Liguori Cc: Ori Mamluk; Kevin Wolf; qemu-devel@nongnu.org Subject: Re: [Qemu-devel] [RFC PATCH] replication agent module On 02/07/2012 02:12 PM, Anthony Liguori wrote: Hi, On 02/07/2012 04:29 AM, Ori Mamluk wrote: Repagent

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Ori Mamluk
nction I call from a new thread context is bdrv_aio_readv. -Original Message- From: Dor Laor [mailto:dl...@redhat.com] Sent: Tuesday, February 07, 2012 2:25 PM To: Anthony Liguori Cc: Ori Mamluk; Kevin Wolf; qemu-devel@nongnu.org Subject: Re: [Qemu-devel] [RFC PATCH] replication agent module

Re: [Qemu-devel] [RFC PATCH] replication agent module

2012-02-07 Thread Anthony Liguori
Hi, On 02/07/2012 04:29 AM, Ori Mamluk wrote: Repagent is a new module that allows an external replication system to replicate a volume of a Qemu VM. This RFC patch adds the repagent client module to Qemu. Please read http://wiki.qemu.org/Contribute/SubmitAPatch In particular, use a tool lik