Re: [Qemu-devel] [RFC PATCH 00/16] Qemu Bit Map (QBM) - an overlay format for persistent dirty bitmap

[Fam Zheng]

On Tue, 02/23 18:43, Kevin Wolf wrote:
> Am 23.02.2016 um 04:40 hat Fam Zheng geschrieben:
> > (I'm Cc'ing a few more people here just in case they have different visions
> > about raw image use cases.)
> > 
> > On Mon, 02/22 15:24, Kevin Wolf wrote:
> > > Am 26.01.2016 um 11:38 hat Fam Zheng geschrieben:
> > > > This series introduces a simple format to enable support of persistence 
> > > > of
> > > > block dirty bitmaps. Block dirty bitmap is the tool to achieve 
> > > > incremental
> > > > backup, and persistence of block dirty bitmap makes incrememtal backup 
> > > > possible
> > > > across VM shutdowns, where existing in-memory dirty bitmaps cannot 
> > > > survive.
> > > > 
> > > > When user creates a "persisted" dirty bitmap, the QBM driver will 
> > > > create a
> > > > binary file and synchronize it with the existing in-memory block dirty 
> > > > bitmap
> > > > (BdrvDirtyBitmap). When the VM is powered down, the binary file has all 
> > > > the
> > > > bits saved on disk, which will be loaded and used to initialize the 
> > > > in-memory
> > > > block dirty bitmap next time the guest is started.
> > > > 
> > > > The idea of the format is to reuse as much existing infrastructure as 
> > > > possible
> > > > and avoid introducing complex data structures - it works with any image 
> > > > format,
> > > > by gluing it together plain bitmap files with a json descriptor file. 
> > > > The
> > > > advantage of this approach over extending existing formats, such as 
> > > > qcow2, is
> > > > that the new feature is implemented by an orthogonal driver, in a format
> > > > agnostic way. This way, even raw images can have their persistent dirty
> > > > bitmaps.  (And you will notice in this series, with a little forging to 
> > > > the
> > > > spec, raw images can also have backing files through a QBM overlay!)
> > > > 
> > > > Rather than superseding it, this intends to be coexistent in parallel 
> > > > with the
> > > > qcow2 bitmap extension that Vladimir is working on.  The block driver 
> > > > interface
> > > > changes in this series also try to be generic and compatible for both 
> > > > drivers.
> > > 
> > > So as I already told Fam last week, before we discuss any technical
> > > details here, we first need to discuss whether this is even the right
> > > thing to do. Currently I'm doubtful, as this is another attempt to
> > > introduce a new native image format in qemu.
> > > 
> > > Let's recap the image formats and what we tell users about them today:
> > > 
> > > * qcow2: This is the default choice for disk images. It gives you access
> > >   to all of the features in qemu at a good performance. If it doesn't
> > >   perform well in your case, we'll fix it.
> > > 
> > > * raw: Use this when you need absolute performance and don't need any
> > >   features from an image format, so you want to get any complexity just
> > >   out of the way and pass requests as directly as possible from the
> > >   guest device to the host kernel.
> > > 
> > > * Anything else: Only use them to convert into raw or qcow2.
> > > 
> > > Now using bitmaps is clearly on the "features" side, which suggests that
> > > qcow2 is the format of choice for this. If you want to introduce a new
> > > format, you need to justify it with evidence that...
> > > 
> > > 1. there is a relevant use case that qcow2 doesn't cover
> > > 2. qcow2 can't be fixed/enhanced to cover the use case
> > > 
> > > The one thing that people have claimed in the past that qcow2 can't
> > > provide is enough performance. This is where QED tried to come in and
> > > promised a compromise between performance (then a bit faster than qcow2)
> > > and features (almost none, but supports backing files). We all know that
> > > it was a failure because you had to sacrifice features and still the
> > > idea that qcow2 couldn't be fixed was wrong, so today we have a QED
> > > driver that is much slower than qcow2 despite having less features.
> > > 
> > > Now for QBM. First, let's have a look at the image format that it can be
> > > used with. qcow2 doesn't need it if we continue with Vladimir's
> > > extension. Other non-raw formats are only supposed to be used for
> > > conversion. The only thing that's really left is raw.
> > 
> > Yes, I agree with this point.
> > 
> > > Now adding a
> > > feature only for raw, as a compromise between features and performance,
> > > looks an awful lot like what QED tried. We don't want to go there.
> > > 
> > > Even if we wanted to support persistent dirty bitmaps with raw images
> > > (which has to be discussed based on use cases), it's still questionable
> > > whether we need a new image format with JSON descriptor files instead of
> > > just raw bitmaps that can be added with a QMP command.
> > > 
> > 
> > I don't think QMP interface alone is enough, in persistent backup use case,
> > when starting a guest, command line interface is more appropriate to 
> > continue
> > dirty trackings that were enabled during shutdown.
> 

Re: [Qemu-devel] [RFC PATCH 00/16] Qemu Bit Map (QBM) - an overlay format for persistent dirty bitmap

[Kevin Wolf]

Am 23.02.2016 um 04:40 hat Fam Zheng geschrieben:
> (I'm Cc'ing a few more people here just in case they have different visions
> about raw image use cases.)
> 
> On Mon, 02/22 15:24, Kevin Wolf wrote:
> > Am 26.01.2016 um 11:38 hat Fam Zheng geschrieben:
> > > This series introduces a simple format to enable support of persistence of
> > > block dirty bitmaps. Block dirty bitmap is the tool to achieve incremental
> > > backup, and persistence of block dirty bitmap makes incrememtal backup 
> > > possible
> > > across VM shutdowns, where existing in-memory dirty bitmaps cannot 
> > > survive.
> > > 
> > > When user creates a "persisted" dirty bitmap, the QBM driver will create a
> > > binary file and synchronize it with the existing in-memory block dirty 
> > > bitmap
> > > (BdrvDirtyBitmap). When the VM is powered down, the binary file has all 
> > > the
> > > bits saved on disk, which will be loaded and used to initialize the 
> > > in-memory
> > > block dirty bitmap next time the guest is started.
> > > 
> > > The idea of the format is to reuse as much existing infrastructure as 
> > > possible
> > > and avoid introducing complex data structures - it works with any image 
> > > format,
> > > by gluing it together plain bitmap files with a json descriptor file. The
> > > advantage of this approach over extending existing formats, such as 
> > > qcow2, is
> > > that the new feature is implemented by an orthogonal driver, in a format
> > > agnostic way. This way, even raw images can have their persistent dirty
> > > bitmaps.  (And you will notice in this series, with a little forging to 
> > > the
> > > spec, raw images can also have backing files through a QBM overlay!)
> > > 
> > > Rather than superseding it, this intends to be coexistent in parallel 
> > > with the
> > > qcow2 bitmap extension that Vladimir is working on.  The block driver 
> > > interface
> > > changes in this series also try to be generic and compatible for both 
> > > drivers.
> > 
> > So as I already told Fam last week, before we discuss any technical
> > details here, we first need to discuss whether this is even the right
> > thing to do. Currently I'm doubtful, as this is another attempt to
> > introduce a new native image format in qemu.
> > 
> > Let's recap the image formats and what we tell users about them today:
> > 
> > * qcow2: This is the default choice for disk images. It gives you access
> >   to all of the features in qemu at a good performance. If it doesn't
> >   perform well in your case, we'll fix it.
> > 
> > * raw: Use this when you need absolute performance and don't need any
> >   features from an image format, so you want to get any complexity just
> >   out of the way and pass requests as directly as possible from the
> >   guest device to the host kernel.
> > 
> > * Anything else: Only use them to convert into raw or qcow2.
> > 
> > Now using bitmaps is clearly on the "features" side, which suggests that
> > qcow2 is the format of choice for this. If you want to introduce a new
> > format, you need to justify it with evidence that...
> > 
> > 1. there is a relevant use case that qcow2 doesn't cover
> > 2. qcow2 can't be fixed/enhanced to cover the use case
> > 
> > The one thing that people have claimed in the past that qcow2 can't
> > provide is enough performance. This is where QED tried to come in and
> > promised a compromise between performance (then a bit faster than qcow2)
> > and features (almost none, but supports backing files). We all know that
> > it was a failure because you had to sacrifice features and still the
> > idea that qcow2 couldn't be fixed was wrong, so today we have a QED
> > driver that is much slower than qcow2 despite having less features.
> > 
> > Now for QBM. First, let's have a look at the image format that it can be
> > used with. qcow2 doesn't need it if we continue with Vladimir's
> > extension. Other non-raw formats are only supposed to be used for
> > conversion. The only thing that's really left is raw.
> 
> Yes, I agree with this point.
> 
> > Now adding a
> > feature only for raw, as a compromise between features and performance,
> > looks an awful lot like what QED tried. We don't want to go there.
> > 
> > Even if we wanted to support persistent dirty bitmaps with raw images
> > (which has to be discussed based on use cases), it's still questionable
> > whether we need a new image format with JSON descriptor files instead of
> > just raw bitmaps that can be added with a QMP command.
> > 
> 
> I don't think QMP interface alone is enough, in persistent backup use case,
> when starting a guest, command line interface is more appropriate to continue
> dirty trackings that were enabled during shutdown.

Yes, I was sloppy. Maybe s/QMP command/runtime option/ gets closer.

> I'd justify in two parts, one is "why" and the other is "how".
> 
> So to answer why.  The reason I worked on QBM is because I feel it wrong to
> leaving raw behind. Ceph and LVM users use raw format. 

Re: [Qemu-devel] [RFC PATCH 00/16] Qemu Bit Map (QBM) - an overlay format for persistent dirty bitmap

[Kevin Wolf]

Am 23.02.2016 um 10:14 hat Markus Armbruster geschrieben:
> Kevin Wolf  writes:
> 
> > Am 26.01.2016 um 11:38 hat Fam Zheng geschrieben:
> >> This series introduces a simple format to enable support of persistence of
> >> block dirty bitmaps. Block dirty bitmap is the tool to achieve incremental
> >> backup, and persistence of block dirty bitmap makes incrememtal backup 
> >> possible
> >> across VM shutdowns, where existing in-memory dirty bitmaps cannot survive.
> >> 
> >> When user creates a "persisted" dirty bitmap, the QBM driver will create a
> >> binary file and synchronize it with the existing in-memory block dirty 
> >> bitmap
> >> (BdrvDirtyBitmap). When the VM is powered down, the binary file has all the
> >> bits saved on disk, which will be loaded and used to initialize the 
> >> in-memory
> >> block dirty bitmap next time the guest is started.
> >> 
> >> The idea of the format is to reuse as much existing infrastructure as 
> >> possible
> >> and avoid introducing complex data structures - it works with any image 
> >> format,
> >> by gluing it together plain bitmap files with a json descriptor file. The
> >> advantage of this approach over extending existing formats, such as qcow2, 
> >> is
> >> that the new feature is implemented by an orthogonal driver, in a format
> >> agnostic way. This way, even raw images can have their persistent dirty
> >> bitmaps.  (And you will notice in this series, with a little forging to the
> >> spec, raw images can also have backing files through a QBM overlay!)
> >> 
> >> Rather than superseding it, this intends to be coexistent in parallel with 
> >> the
> >> qcow2 bitmap extension that Vladimir is working on.  The block driver 
> >> interface
> >> changes in this series also try to be generic and compatible for both 
> >> drivers.
> >
> > So as I already told Fam last week, before we discuss any technical
> > details here, we first need to discuss whether this is even the right
> > thing to do.
> 
> Yes, this must come first.
> 
> >  Currently I'm doubtful, as this is another attempt to
> > introduce a new native image format in qemu.
> >
> > Let's recap the image formats and what we tell users about them today:
> >
> > * qcow2: This is the default choice for disk images. It gives you access
> >   to all of the features in qemu at a good performance. If it doesn't
> >   perform well in your case, we'll fix it.
> 
> Rather: we'll fix it if we can.

Right. The assumption is so far that we generally can. If it turns out
at some point that we can't improve it sufficiently and a new format
could improve it, then the whole approach of having only raw and qcow2
is indeed in question.

If you look at how different the various VMDK subformats are, though, it
seems that we still have more than enough maneuvering room. I think
having optional features in the format are preferrable to multiple
user-visible formats. For most people, VMDK is just VMDK, and there is
no reason why they should know about the subformats. Similarly, it would
be good if qemu users didn't have to know about qcow2, qed, fvd, qbm...

> > * raw: Use this when you need absolute performance and don't need any
> >   features from an image format, so you want to get any complexity just
> >   out of the way and pass requests as directly as possible from the
> >   guest device to the host kernel.
> >
> > * Anything else: Only use them to convert into raw or qcow2.
> >
> > Now using bitmaps is clearly on the "features" side, which suggests that
> > qcow2 is the format of choice for this.
> 
> I'd agree with a general "extra feature suggests QCOW2" maxim, with
> stress on "suggests".
> 
> However, the "extraness" of bitmaps is perhaps less clear than for other
> features.  Bitmap-like things occur not just in formats: sparse files,
> thinly provisioned SCSI devices, ...
> 
> > If you want to introduce a new
> > format, you need to justify it with evidence that...
> >
> > 1. there is a relevant use case that qcow2 doesn't cover
> > 2. qcow2 can't be fixed/enhanced to cover the use case
> >
> > The one thing that people have claimed in the past that qcow2 can't
> > provide is enough performance. This is where QED tried to come in and
> > promised a compromise between performance (then a bit faster than qcow2)
> > and features (almost none, but supports backing files). We all know that
> > it was a failure because you had to sacrifice features and still the
> > idea that qcow2 couldn't be fixed was wrong, so today we have a QED
> > driver that is much slower than qcow2 despite having less features.
> 
> Yes.  We thought QCOW2 could not be made to perform[*], until you did.
> 
> New storage hardware will bring back performance pressure with a
> vengeance, though.

Then we'll have to address them. Not only for dirty bitmaps, but also
for snapshots and all the other features you only get with an image
format.

> > Now for QBM. First, let's 

Re: [Qemu-devel] [RFC PATCH 00/16] Qemu Bit Map (QBM) - an overlay format for persistent dirty bitmap

[Markus Armbruster]

Kevin Wolf  writes:

> Am 26.01.2016 um 11:38 hat Fam Zheng geschrieben:
>> This series introduces a simple format to enable support of persistence of
>> block dirty bitmaps. Block dirty bitmap is the tool to achieve incremental
>> backup, and persistence of block dirty bitmap makes incrememtal backup 
>> possible
>> across VM shutdowns, where existing in-memory dirty bitmaps cannot survive.
>> 
>> When user creates a "persisted" dirty bitmap, the QBM driver will create a
>> binary file and synchronize it with the existing in-memory block dirty bitmap
>> (BdrvDirtyBitmap). When the VM is powered down, the binary file has all the
>> bits saved on disk, which will be loaded and used to initialize the in-memory
>> block dirty bitmap next time the guest is started.
>> 
>> The idea of the format is to reuse as much existing infrastructure as 
>> possible
>> and avoid introducing complex data structures - it works with any image 
>> format,
>> by gluing it together plain bitmap files with a json descriptor file. The
>> advantage of this approach over extending existing formats, such as qcow2, is
>> that the new feature is implemented by an orthogonal driver, in a format
>> agnostic way. This way, even raw images can have their persistent dirty
>> bitmaps.  (And you will notice in this series, with a little forging to the
>> spec, raw images can also have backing files through a QBM overlay!)
>> 
>> Rather than superseding it, this intends to be coexistent in parallel with 
>> the
>> qcow2 bitmap extension that Vladimir is working on.  The block driver 
>> interface
>> changes in this series also try to be generic and compatible for both 
>> drivers.
>
> So as I already told Fam last week, before we discuss any technical
> details here, we first need to discuss whether this is even the right
> thing to do.

Yes, this must come first.

>  Currently I'm doubtful, as this is another attempt to
> introduce a new native image format in qemu.
>
> Let's recap the image formats and what we tell users about them today:
>
> * qcow2: This is the default choice for disk images. It gives you access
>   to all of the features in qemu at a good performance. If it doesn't
>   perform well in your case, we'll fix it.

Rather: we'll fix it if we can.

> * raw: Use this when you need absolute performance and don't need any
>   features from an image format, so you want to get any complexity just
>   out of the way and pass requests as directly as possible from the
>   guest device to the host kernel.
>
> * Anything else: Only use them to convert into raw or qcow2.
>
> Now using bitmaps is clearly on the "features" side, which suggests that
> qcow2 is the format of choice for this.

I'd agree with a general "extra feature suggests QCOW2" maxim, with
stress on "suggests".

However, the "extraness" of bitmaps is perhaps less clear than for other
features.  Bitmap-like things occur not just in formats: sparse files,
thinly provisioned SCSI devices, ...

> If you want to introduce a new
> format, you need to justify it with evidence that...
>
> 1. there is a relevant use case that qcow2 doesn't cover
> 2. qcow2 can't be fixed/enhanced to cover the use case
>
> The one thing that people have claimed in the past that qcow2 can't
> provide is enough performance. This is where QED tried to come in and
> promised a compromise between performance (then a bit faster than qcow2)
> and features (almost none, but supports backing files). We all know that
> it was a failure because you had to sacrifice features and still the
> idea that qcow2 couldn't be fixed was wrong, so today we have a QED
> driver that is much slower than qcow2 despite having less features.

Yes.  We thought QCOW2 could not be made to perform[*], until you did.

New storage hardware will bring back performance pressure with a
vengeance, though.

> Now for QBM. First, let's have a look at the image format that it can be
> used with. qcow2 doesn't need it if we continue with Vladimir's
> extension. Other non-raw formats are only supposed to be used for
> conversion. The only thing that's really left is raw. Now adding a
> feature only for raw, as a compromise between features and performance,
> looks an awful lot like what QED tried. We don't want to go there.

A possible difference: complexity.

Adding another QEMU-native format in QCOW2's complexity class would be
highly problematic.  We tried with QED, because we thought we'd need it
to support different tradeoffs, but it turned out to be a dead end.

Doesn't mean there's absolutely no space for a *simple* format to
support different tradeoffs.  Is QBM simple enough?  Will it stay simple
enough?

> Even if we wanted to support persistent dirty bitmaps with raw images
> (which has to be discussed based on use cases), it's still questionable
> whether we need a new image format with JSON descriptor files instead of
> just raw bitmaps that can be added with 

Re: [Qemu-devel] [RFC PATCH 00/16] Qemu Bit Map (QBM) - an overlay format for persistent dirty bitmap

[Fam Zheng]

(I'm Cc'ing a few more people here just in case they have different visions
about raw image use cases.)

On Mon, 02/22 15:24, Kevin Wolf wrote:
> Am 26.01.2016 um 11:38 hat Fam Zheng geschrieben:
> > This series introduces a simple format to enable support of persistence of
> > block dirty bitmaps. Block dirty bitmap is the tool to achieve incremental
> > backup, and persistence of block dirty bitmap makes incrememtal backup 
> > possible
> > across VM shutdowns, where existing in-memory dirty bitmaps cannot survive.
> > 
> > When user creates a "persisted" dirty bitmap, the QBM driver will create a
> > binary file and synchronize it with the existing in-memory block dirty 
> > bitmap
> > (BdrvDirtyBitmap). When the VM is powered down, the binary file has all the
> > bits saved on disk, which will be loaded and used to initialize the 
> > in-memory
> > block dirty bitmap next time the guest is started.
> > 
> > The idea of the format is to reuse as much existing infrastructure as 
> > possible
> > and avoid introducing complex data structures - it works with any image 
> > format,
> > by gluing it together plain bitmap files with a json descriptor file. The
> > advantage of this approach over extending existing formats, such as qcow2, 
> > is
> > that the new feature is implemented by an orthogonal driver, in a format
> > agnostic way. This way, even raw images can have their persistent dirty
> > bitmaps.  (And you will notice in this series, with a little forging to the
> > spec, raw images can also have backing files through a QBM overlay!)
> > 
> > Rather than superseding it, this intends to be coexistent in parallel with 
> > the
> > qcow2 bitmap extension that Vladimir is working on.  The block driver 
> > interface
> > changes in this series also try to be generic and compatible for both 
> > drivers.
> 
> So as I already told Fam last week, before we discuss any technical
> details here, we first need to discuss whether this is even the right
> thing to do. Currently I'm doubtful, as this is another attempt to
> introduce a new native image format in qemu.
> 
> Let's recap the image formats and what we tell users about them today:
> 
> * qcow2: This is the default choice for disk images. It gives you access
>   to all of the features in qemu at a good performance. If it doesn't
>   perform well in your case, we'll fix it.
> 
> * raw: Use this when you need absolute performance and don't need any
>   features from an image format, so you want to get any complexity just
>   out of the way and pass requests as directly as possible from the
>   guest device to the host kernel.
> 
> * Anything else: Only use them to convert into raw or qcow2.
> 
> Now using bitmaps is clearly on the "features" side, which suggests that
> qcow2 is the format of choice for this. If you want to introduce a new
> format, you need to justify it with evidence that...
> 
> 1. there is a relevant use case that qcow2 doesn't cover
> 2. qcow2 can't be fixed/enhanced to cover the use case
> 
> The one thing that people have claimed in the past that qcow2 can't
> provide is enough performance. This is where QED tried to come in and
> promised a compromise between performance (then a bit faster than qcow2)
> and features (almost none, but supports backing files). We all know that
> it was a failure because you had to sacrifice features and still the
> idea that qcow2 couldn't be fixed was wrong, so today we have a QED
> driver that is much slower than qcow2 despite having less features.
> 
> Now for QBM. First, let's have a look at the image format that it can be
> used with. qcow2 doesn't need it if we continue with Vladimir's
> extension. Other non-raw formats are only supposed to be used for
> conversion. The only thing that's really left is raw.

Yes, I agree with this point.

> Now adding a
> feature only for raw, as a compromise between features and performance,
> looks an awful lot like what QED tried. We don't want to go there.
> 
> Even if we wanted to support persistent dirty bitmaps with raw images
> (which has to be discussed based on use cases), it's still questionable
> whether we need a new image format with JSON descriptor files instead of
> just raw bitmaps that can be added with a QMP command.
> 

I don't think QMP interface alone is enough, in persistent backup use case,
when starting a guest, command line interface is more appropriate to continue
dirty trackings that were enabled during shutdown.

I'd justify in two parts, one is "why" and the other is "how".

So to answer why.  The reason I worked on QBM is because I feel it wrong to
leaving raw behind. Ceph and LVM users use raw format. You could technically
use qcow2 with ceph but that is discouraged[1] or even refused by openstack[2].
We've seen qcow2 on top of LVs but that is not the dominance.

The scope of "features" for which we tell users they have to use qcow2 should
those that are format specific, not "block features" in general.  Backing file,

Re: [Qemu-devel] [RFC PATCH 00/16] Qemu Bit Map (QBM) - an overlay format for persistent dirty bitmap

[Kevin Wolf]

Am 26.01.2016 um 11:38 hat Fam Zheng geschrieben:
> This series introduces a simple format to enable support of persistence of
> block dirty bitmaps. Block dirty bitmap is the tool to achieve incremental
> backup, and persistence of block dirty bitmap makes incrememtal backup 
> possible
> across VM shutdowns, where existing in-memory dirty bitmaps cannot survive.
> 
> When user creates a "persisted" dirty bitmap, the QBM driver will create a
> binary file and synchronize it with the existing in-memory block dirty bitmap
> (BdrvDirtyBitmap). When the VM is powered down, the binary file has all the
> bits saved on disk, which will be loaded and used to initialize the in-memory
> block dirty bitmap next time the guest is started.
> 
> The idea of the format is to reuse as much existing infrastructure as possible
> and avoid introducing complex data structures - it works with any image 
> format,
> by gluing it together plain bitmap files with a json descriptor file. The
> advantage of this approach over extending existing formats, such as qcow2, is
> that the new feature is implemented by an orthogonal driver, in a format
> agnostic way. This way, even raw images can have their persistent dirty
> bitmaps.  (And you will notice in this series, with a little forging to the
> spec, raw images can also have backing files through a QBM overlay!)
> 
> Rather than superseding it, this intends to be coexistent in parallel with the
> qcow2 bitmap extension that Vladimir is working on.  The block driver 
> interface
> changes in this series also try to be generic and compatible for both drivers.

So as I already told Fam last week, before we discuss any technical
details here, we first need to discuss whether this is even the right
thing to do. Currently I'm doubtful, as this is another attempt to
introduce a new native image format in qemu.

Let's recap the image formats and what we tell users about them today:

* qcow2: This is the default choice for disk images. It gives you access
  to all of the features in qemu at a good performance. If it doesn't
  perform well in your case, we'll fix it.

* raw: Use this when you need absolute performance and don't need any
  features from an image format, so you want to get any complexity just
  out of the way and pass requests as directly as possible from the
  guest device to the host kernel.

* Anything else: Only use them to convert into raw or qcow2.

Now using bitmaps is clearly on the "features" side, which suggests that
qcow2 is the format of choice for this. If you want to introduce a new
format, you need to justify it with evidence that...

1. there is a relevant use case that qcow2 doesn't cover
2. qcow2 can't be fixed/enhanced to cover the use case

The one thing that people have claimed in the past that qcow2 can't
provide is enough performance. This is where QED tried to come in and
promised a compromise between performance (then a bit faster than qcow2)
and features (almost none, but supports backing files). We all know that
it was a failure because you had to sacrifice features and still the
idea that qcow2 couldn't be fixed was wrong, so today we have a QED
driver that is much slower than qcow2 despite having less features.

Now for QBM. First, let's have a look at the image format that it can be
used with. qcow2 doesn't need it if we continue with Vladimir's
extension. Other non-raw formats are only supposed to be used for
conversion. The only thing that's really left is raw. Now adding a
feature only for raw, as a compromise between features and performance,
looks an awful lot like what QED tried. We don't want to go there.

Even if we wanted to support persistent dirty bitmaps with raw images
(which has to be discussed based on use cases), it's still questionable
whether we need a new image format with JSON descriptor files instead of
just raw bitmaps that can be added with a QMP command.


tl;dr: Where is the justification for a new image format? You need a
good one.

Kevin

[Qemu-devel] [RFC PATCH 00/16] Qemu Bit Map (QBM) - an overlay format for persistent dirty bitmap

[Fam Zheng]

Hi all,

This series introduces a simple format to enable support of persistence of
block dirty bitmaps. Block dirty bitmap is the tool to achieve incremental
backup, and persistence of block dirty bitmap makes incrememtal backup possible
across VM shutdowns, where existing in-memory dirty bitmaps cannot survive.

When user creates a "persisted" dirty bitmap, the QBM driver will create a
binary file and synchronize it with the existing in-memory block dirty bitmap
(BdrvDirtyBitmap). When the VM is powered down, the binary file has all the
bits saved on disk, which will be loaded and used to initialize the in-memory
block dirty bitmap next time the guest is started.

The idea of the format is to reuse as much existing infrastructure as possible
and avoid introducing complex data structures - it works with any image format,
by gluing it together plain bitmap files with a json descriptor file. The
advantage of this approach over extending existing formats, such as qcow2, is
that the new feature is implemented by an orthogonal driver, in a format
agnostic way. This way, even raw images can have their persistent dirty
bitmaps.  (And you will notice in this series, with a little forging to the
spec, raw images can also have backing files through a QBM overlay!)

Rather than superseding it, this intends to be coexistent in parallel with the
qcow2 bitmap extension that Vladimir is working on.  The block driver interface
changes in this series also try to be generic and compatible for both drivers.

The format's specification is added to docs/specs/, see patch 1.

Patches 2-7 are necessary block layer changes in order be friendly to
persistent dirty bitmap drivers.

Patches 8, 9 and 11 extends the QMP interface to expose the added feature.

Patch 10 implements the driver. (todo: checksum extension for image/bitmap
integrity check)

Patch 12 - 16 are the tests I have for QBM so far. I'm sure more can be added
as questions emerge. :)

The series applies on top of my "qemu-img map" series and "meta bitmap" series:

https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg04866.html
https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg03656.html

If you feel like to play with it, git branch is also available at:

https://github.com/famz/qemu qbm

Comments are welcome!

Fam


Fam Zheng (16):
  doc: Add QBM format specification
  block: Set dirty before doing write
  block: Allow .bdrv_close callback to release dirty bitmaps
  block: Move filename_decompose to block.c
  block: Make bdrv_get_cluster_size public
  block: Introduce bdrv_dirty_bitmap_set_persistent
  block: Only swap non-persistent dirty bitmaps
  qmp: Add optional parameter "persistent" in block-dirty-bitmap-add
  qmp: Add block-dirty-bitmap-set-persistent
  qbm: Implement format driver
  qapi: Add "qbm" as a generic cow format driver
  iotests: Add qbm format to 041
  iotests: Add qbm to case 097
  iotests: Add qbm to applicable test cases
  iotests: Add qbm specific test case 140
  iotests: Add persistent bitmap test case 141

 block.c  |   54 +-
 block/Makefile.objs  |1 +
 block/dirty-bitmap.c |   63 ++
 block/io.c   |6 +-
 block/qbm.c  | 1315 ++
 block/vmdk.c |   40 --
 blockdev.c   |   28 +-
 docs/specs/qbm.md|  118 
 include/block/block.h|4 +-
 include/block/block_int.h|8 +
 include/block/dirty-bitmap.h |5 +
 qapi/block-core.json |   31 +-
 qmp-commands.hx  |   34 +-
 tests/qemu-iotests/004   |2 +-
 tests/qemu-iotests/017   |2 +-
 tests/qemu-iotests/018   |2 +-
 tests/qemu-iotests/019   |2 +-
 tests/qemu-iotests/020   |2 +-
 tests/qemu-iotests/024   |2 +-
 tests/qemu-iotests/025   |2 +-
 tests/qemu-iotests/027   |2 +-
 tests/qemu-iotests/028   |2 +-
 tests/qemu-iotests/030   |2 +-
 tests/qemu-iotests/034   |2 +-
 tests/qemu-iotests/037   |2 +-
 tests/qemu-iotests/038   |2 +-
 tests/qemu-iotests/040   |2 +-
 tests/qemu-iotests/041   |   18 +-
 tests/qemu-iotests/050   |2 +-
 tests/qemu-iotests/055   |2 +-
 tests/qemu-iotests/056   |2 +-
 tests/qemu-iotests/069   |2 +-
 tests/qemu-iotests/072   |2 +-
 tests/qemu-iotests/086   |2 +-
 tests/qemu-iotests/095   |2 +-
 tests/qemu-iotests/096   |2 +-
 tests/qemu-iotests/097   |4 +-
 tests/qemu-iotests/099   |2 +-
 tests/qemu-iotests/110   |2 +-
 tests/qemu-iotests/129   |2 +-
 tests/qemu-iotests/132   |2 +-
 tests/qemu-iotests/139   |2 +-
 tests/qemu-iotests/140   |   80 +++
 tests/qemu-iotests/140.out   |  145 +
 tests/qemu-iotests/141   |   62 ++
 tests/qemu-iotests/141.out   |5 +
 tests/qemu-iotests/common|6 +
 tests/qemu-iotests/group |2