On Sun, Mar 10, 2013 at 10:32 AM, Dietmar Maurer wrote:
>> >> What I am looking for is a stripped down patch series with just a
>> >> backup block job (no backup archive writer or migration code). That
>> >> would be easily merged and saves you front rebasing this series as QEMU
>> changes.
>> >
> >> What I am looking for is a stripped down patch series with just a
> >> backup block job (no backup archive writer or migration code). That
> >> would be easily merged and saves you front rebasing this series as QEMU
> changes.
> >
> > That is Patch 2/6?
>
> Yes. I sent an RFC series that sh
On Fri, Mar 8, 2013 at 6:44 PM, Dietmar Maurer wrote:
>> >> This is a strong indicator that the backup archive code should
>> >>live outside QEMU. I doesn't make sense for proxmox, oVirt,
>> >>OpenStack, and others to each maintain their backup archive code
>> >>inside qemu.git, tied
> >> For these reasons, I'm against putting backup archive code into QEMU.
> >
> > That is OK for me - I already maintain the code outside of qemu.
>
> Does this mean you will keep this patch series out-of-tree?
You are 'against putting backup archive code into QEMU', so I need to
maintain it out
> >> 1. QEMU can neither backup nor restore without help from the management
> >>tool.
> >
> > Backup works perfectly with the current patches. You can easily
> > trigger a backup using a HMP command. This is not really important, but
> > works.
>
> If you send me a VMA file I can't restore i
On Fri, Mar 8, 2013 at 12:01 PM, Dietmar Maurer wrote:
>> > Anyways, and additional RPC layer always adds overhead, and It can be
>> completely avoided.
>> > Maybe not much, but I want to make backup as efficient as possible.
>>
>> The drawbacks outweight the performance advantage:
>
> not for me.
> > Anyways, and additional RPC layer always adds overhead, and It can be
> completely avoided.
> > Maybe not much, but I want to make backup as efficient as possible.
>
> The drawbacks outweight the performance advantage:
not for me.
> 1. QEMU can neither backup nor restore without help from th
On Thu, Mar 07, 2013 at 09:28:40AM +, Dietmar Maurer wrote:
> > > When we run backup, we need to read such block on every write from the
> > guest.
> > > So if we increase block size we get additional delays.
> >
> > Don't increase the bitmap block size.
> >
> > Just let the block job do larg
> > When we run backup, we need to read such block on every write from the
> guest.
> > So if we increase block size we get additional delays.
>
> Don't increase the bitmap block size.
>
> Just let the block job do larger reads. This is the bulk of the I/O
> workload. You
> can use large reads
> > > > You can make that as complex as you want. I simply do not need that.
> > >
> > > Then you also don't need the performance that you lose by using NBD.
> >
> > Why exactly?
>
> Because your format kills more performance than any NBD connection could.
That is not true.
On Wed, Mar 06, 2013 at 02:42:57PM +, Dietmar Maurer wrote:
> > > > Maybe you'd better use a different output format that doesn't
> > > > restrict you to 64k writes.
> > >
> > > The output format is not really the restriction. The problem is that
> > > an additional IPC layer add overhead, an d
Am 06.03.2013 um 18:39 hat Dietmar Maurer geschrieben:
> > > > How about variable block sizes? I mean this is a stream format that
> > > > has a header for each block anyway. Include a size there and be done.
> > >
> > > You can make that as complex as you want. I simply do not need that.
> >
> >
> > > How about variable block sizes? I mean this is a stream format that
> > > has a header for each block anyway. Include a size there and be done.
> >
> > You can make that as complex as you want. I simply do not need that.
>
> Then you also don't need the performance that you lose by using NBD
Am 06.03.2013 um 16:33 hat Dietmar Maurer geschrieben:
> > > When we run backup, we need to read such block on every write from the
> > guest.
> > > So if we increase block size we get additional delays.
> >
> > How about variable block sizes? I mean this is a stream format that has a
> > header
> > When we run backup, we need to read such block on every write from the
> guest.
> > So if we increase block size we get additional delays.
>
> How about variable block sizes? I mean this is a stream format that has a
> header
> for each block anyway. Include a size there and be done.
You can
Am 06.03.2013 um 15:42 hat Dietmar Maurer geschrieben:
> > > > Maybe you'd better use a different output format that doesn't
> > > > restrict you to 64k writes.
> > >
> > > The output format is not really the restriction. The problem is that
> > > an additional IPC layer add overhead, an d I do not
> > > Maybe you'd better use a different output format that doesn't
> > > restrict you to 64k writes.
> >
> > The output format is not really the restriction. The problem is that
> > an additional IPC layer add overhead, an d I do not want that (because it is
> totally unnecessary).
>
> I missed t
On Mon, Mar 04, 2013 at 02:33:16PM +, Dietmar Maurer wrote:
> > > > Is it using 64 KB writes and have you tried 256 KB writes?
> > >
> > > I use a modified 'qemu-img convert' at 64KB block size (I need 64KB for
> > backup).
> >
> > Maybe you'd better use a different output format that doesn't
> > > Is it using 64 KB writes and have you tried 256 KB writes?
> >
> > I use a modified 'qemu-img convert' at 64KB block size (I need 64KB for
> backup).
>
> Maybe you'd better use a different output format that doesn't restrict you to
> 64k writes.
The output format is not really the restricti
Am 04.03.2013 um 14:16 hat Dietmar Maurer geschrieben:
> > What are the details of the test?
> >
> > Is it using 64 KB writes and have you tried 256 KB writes?
>
> I use a modified 'qemu-img convert' at 64KB block size (I need 64KB for
> backup).
Maybe you'd better use a different output format
> What are the details of the test?
>
> Is it using 64 KB writes and have you tried 256 KB writes?
I use a modified 'qemu-img convert' at 64KB block size (I need 64KB for backup).
On Thu, Feb 28, 2013 at 03:24:27PM +, Dietmar Maurer wrote:
> > > Unfortunately, NBD add considerable overheads. I guess the socket
> > communications copies data.
> > > This is really unnecessary if I can write directly to the output stream.
> >
> > The disk is the bottleneck, not memory band
> The disk is the bottleneck, not memory bandwidth. Hard disks only do
> 10-100 MB/sec and SSDs only do a couple 100 MB/sec. Memory copy is
> insignificant compared to the I/O activity required to copy out the entire
> disk
> image, not to mention delaying guest writes until we read the original
> > Unfortunately, NBD add considerable overheads. I guess the socket
> communications copies data.
> > This is really unnecessary if I can write directly to the output stream.
>
> The disk is the bottleneck, not memory bandwidth. Hard disks only do
> 10-100 MB/sec and SSDs only do a couple 100 M
On Wed, Feb 27, 2013 at 03:50:53PM +, Dietmar Maurer wrote:
> > NBD enables interprocess communication - any form of IPC requires a protocol
> > and NBD is quite a trivial one. What is a simpler way of talking to a
> > backup
> > server?
>
> Unfortunately, NBD add considerable overheads. I g
> NBD enables interprocess communication - any form of IPC requires a protocol
> and NBD is quite a trivial one. What is a simpler way of talking to a backup
> server?
Unfortunately, NBD add considerable overheads. I guess the socket
communications copies data.
This is really unnecessary if I ca
On Fri, Feb 22, 2013 at 10:14:03AM +, Dietmar Maurer wrote:
> > > > QEMU must provide the mechanism for point-in-time backups of block
> > > > devices - your backup block job implements this.
> > > >
> > > > Where I disagree with this patch series is that you put the
> > > > management tool- sp
> > The management tooI just needs to convert the config - looks quite easy to
> me.
>
> It's not an easy problem. This is why there is no central vm-images.com where
> everyone can share/sell virtual appliances. You cannot trivially convert
> between
> VMware, oVirt, proxmox, Xen, EC2, etc.
F
On Thu, Feb 21, 2013 at 03:48:57PM +, Dietmar Maurer wrote:
> > > In future, we can allow to pass multiple config files - the vma
> > > archive format can already handle that.
> >
> > My point is that QEMU has no business dealing with the management tool's VM
> > configuration file. And I thi
On Thu, 21 Feb 2013 14:47:08 +0100
Stefan Hajnoczi wrote:
> On Thu, Feb 21, 2013 at 06:21:32AM +, Dietmar Maurer wrote:
> > > > +##
> > > > +# @backup:
> > > > +#
> > > > +# Starts a VM backup.
> > > > +#
> > > > +# @backup-file: the backup file name
> > > > +#
> > > > +# @format: format of t
> > In future, we can allow to pass multiple config files - the vma
> > archive format can already handle that.
>
> My point is that QEMU has no business dealing with the management tool's VM
> configuration file. And I think the management tool-specific archive format
> also
> shouldn't be in Q
On Thu, Feb 21, 2013 at 06:21:32AM +, Dietmar Maurer wrote:
> > > +##
> > > +# @backup:
> > > +#
> > > +# Starts a VM backup.
> > > +#
> > > +# @backup-file: the backup file name
> > > +#
> > > +# @format: format of the backup file
> > > +#
> > > +# @config-filename: #optional name of a configu
On Thu, Feb 21, 2013 at 06:21:32AM +, Dietmar Maurer wrote:
> > > +/* add configuration file to archive */
> > > +if (has_config_file) {
> > > +char *cdata = NULL;
> > > +gsize clen = 0;
> > > +GError *err = NULL;
> > > +if (!g_file_get_contents(config_fi
> > Another option would be to simply dump
> > to the output fh (pipe), and an
> > external binary saves the data. That way we could move the whole archive
> format related code out of qemu.
>
> That sounds like the NBD option - write the backup to an NBD disk image.
> The NBD server process can
On Wed, Feb 20, 2013 at 10:32:00AM +0100, Dietmar Maurer wrote:
> We use a generic BackupDriver struct to encapsulate all archive format
> related function.
>
> Another option would be to simply dump to
> the output fh (pipe), and an external binary saves the data. That way we
> could move the wh
We use a generic BackupDriver struct to encapsulate all archive format
related function.
Another option would be to simply dump to
the output fh (pipe), and an external binary saves the data. That way we
could move the whole archive format related code out of qemu.
Signed-off-by: Dietmar Maurer
36 matches
Mail list logo
