Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Fam Zheng]

On Tue, 09/05 15:27, Kevin Wolf wrote:
> Am 05.09.2017 um 15:18 hat Fam Zheng geschrieben:
> > On Tue, 09/05 15:01, Kevin Wolf wrote:
> > > Am 28.08.2017 um 04:57 hat Fam Zheng geschrieben:
> > > > On Fri, 08/25 15:44, Max Reitz wrote:
> > > > > Well, OK.  The main argument against supporting anything but qcow2 is
> > > > > "if you want features, use qcow2; and we are working on making qcow2 
> > > > > as
> > > > > fast as possible."  I think that's a very good argument still.  At 
> > > > > some
> > > > > point I (and probably others, too) had the idea of making qcow2 files 
> > > > > in
> > > > > raw layout: 
> > > > 
> > > > Yes! I think this idea makes a whole lot of sense, too. Metadata tables 
> > > > can be
> > > > generated so old implementation can still use it.
> > > 
> > > Maybe a nice way to attack this would be "huge pages", i.e. have a L1
> > > entry flag that tells "this points directly to a huge cluster instead of
> > > an L2 table". Gives us 512 MB clusters with a 64k cluster size, or a
> > > maximum of 512 GB clusters with a 2 MB cluster size.
> > > 
> > > Huge clusters would only be used by qemu-img create if the respective
> > > option is given, and only with preallocation.
> > 
> > Then this image is not usable on old QEMUs, right? So this is going to be an
> > incompatible_feature, whereas with the linear mapping L1/L2 tables 
> > approach, it
> > can be an autoclear_feature.
> 
> Ah yes, that's true. Maybe that's important enough that just a single
> global flag is better, even if it's less flexible.
> 
> Of course, this approach also means that we still need to write all of
> the L2 tables and use disk space for them, even though newer versions
> will never look at them.

To speculate^Wmitigate, "qemu-img create" is free to pick a larger cluster size
when this feature is used?

Fam

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Kevin Wolf]

Am 05.09.2017 um 15:18 hat Fam Zheng geschrieben:
> On Tue, 09/05 15:01, Kevin Wolf wrote:
> > Am 28.08.2017 um 04:57 hat Fam Zheng geschrieben:
> > > On Fri, 08/25 15:44, Max Reitz wrote:
> > > > Well, OK.  The main argument against supporting anything but qcow2 is
> > > > "if you want features, use qcow2; and we are working on making qcow2 as
> > > > fast as possible."  I think that's a very good argument still.  At some
> > > > point I (and probably others, too) had the idea of making qcow2 files in
> > > > raw layout: 
> > > 
> > > Yes! I think this idea makes a whole lot of sense, too. Metadata tables 
> > > can be
> > > generated so old implementation can still use it.
> > 
> > Maybe a nice way to attack this would be "huge pages", i.e. have a L1
> > entry flag that tells "this points directly to a huge cluster instead of
> > an L2 table". Gives us 512 MB clusters with a 64k cluster size, or a
> > maximum of 512 GB clusters with a 2 MB cluster size.
> > 
> > Huge clusters would only be used by qemu-img create if the respective
> > option is given, and only with preallocation.
> 
> Then this image is not usable on old QEMUs, right? So this is going to be an
> incompatible_feature, whereas with the linear mapping L1/L2 tables approach, 
> it
> can be an autoclear_feature.

Ah yes, that's true. Maybe that's important enough that just a single
global flag is better, even if it's less flexible.

Of course, this approach also means that we still need to write all of
the L2 tables and use disk space for them, even though newer versions
will never look at them.

Kevin

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Fam Zheng]

On Tue, 09/05 15:01, Kevin Wolf wrote:
> Am 28.08.2017 um 04:57 hat Fam Zheng geschrieben:
> > On Fri, 08/25 15:44, Max Reitz wrote:
> > > Well, OK.  The main argument against supporting anything but qcow2 is
> > > "if you want features, use qcow2; and we are working on making qcow2 as
> > > fast as possible."  I think that's a very good argument still.  At some
> > > point I (and probably others, too) had the idea of making qcow2 files in
> > > raw layout: 
> > 
> > Yes! I think this idea makes a whole lot of sense, too. Metadata tables can 
> > be
> > generated so old implementation can still use it.
> 
> Maybe a nice way to attack this would be "huge pages", i.e. have a L1
> entry flag that tells "this points directly to a huge cluster instead of
> an L2 table". Gives us 512 MB clusters with a 64k cluster size, or a
> maximum of 512 GB clusters with a 2 MB cluster size.
> 
> Huge clusters would only be used by qemu-img create if the respective
> option is given, and only with preallocation.

Then this image is not usable on old QEMUs, right? So this is going to be an
incompatible_feature, whereas with the linear mapping L1/L2 tables approach, it
can be an autoclear_feature.

Fam

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Kevin Wolf]

Am 28.08.2017 um 04:57 hat Fam Zheng geschrieben:
> On Fri, 08/25 15:44, Max Reitz wrote:
> > Well, OK.  The main argument against supporting anything but qcow2 is
> > "if you want features, use qcow2; and we are working on making qcow2 as
> > fast as possible."  I think that's a very good argument still.  At some
> > point I (and probably others, too) had the idea of making qcow2 files in
> > raw layout: 
> 
> Yes! I think this idea makes a whole lot of sense, too. Metadata tables can be
> generated so old implementation can still use it.

Maybe a nice way to attack this would be "huge pages", i.e. have a L1
entry flag that tells "this points directly to a huge cluster instead of
an L2 table". Gives us 512 MB clusters with a 64k cluster size, or a
maximum of 512 GB clusters with a 2 MB cluster size.

Huge clusters would only be used by qemu-img create if the respective
option is given, and only with preallocation.

Kevin

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Yaniv Lavi (Dary)]

YANIV LAVI (YANIV DARY)

SENIOR TECHNICAL PRODUCT MANAGER

Red Hat Israel Ltd. 

34 Jerusalem Road, Building A, 1st floor

Ra'anana, Israel 4350109

yl...@redhat.comT: +972-9-7692306/8272306 F: +972-9-7692223IM: ylavi
  TRIED. TESTED. TRUSTED. 
@redhatnews    Red Hat
   Red Hat



On Thu, Aug 31, 2017 at 12:25 AM, John Snow  wrote:

>
>
> On 08/30/2017 08:58 AM, Yaniv Lavi (Dary) wrote:
> >
> >
> > We had no reason to switch to anything else so far and I'm sure this
> > option was not available when we started supporting raw format.
> >
> >
>
> Yeah, they don't exist yet...! I've looped you in to see if the proposal
> being discussed would alleviate the need for bitmaps for "other formats"
> if we can offer a "raw mode qcow2."
>

As we our direction is to move a file system/LUN per disk I don't think
adding a new type disk image is the right priority.
The below option of separate file sound like the right direction.


>
> At the moment I am going to still try to add bitmaps to other formats
> (through the use of a qcow2 wrapper) but it sounds like a raw-layout
> qcow2 might provide some benefits too.
>
> -js
>

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[John Snow]

On 08/30/2017 08:58 AM, Yaniv Lavi (Dary) wrote:
> 
> 
> We had no reason to switch to anything else so far and I'm sure this
> option was not available when we started supporting raw format. 
>  
> 

Yeah, they don't exist yet...! I've looped you in to see if the proposal
being discussed would alleviate the need for bitmaps for "other formats"
if we can offer a "raw mode qcow2."

At the moment I am going to still try to add bitmaps to other formats
(through the use of a qcow2 wrapper) but it sounds like a raw-layout
qcow2 might provide some benefits too.

-js

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Daniel P. Berrange]

On Wed, Aug 30, 2017 at 02:36:11PM +0100, Stefan Hajnoczi wrote:
> On Tue, Aug 22, 2017 at 03:07:04PM -0400, John Snow wrote:
> > (3) Add either a new flag that turns qcow2's backing file into a full
> > R/W backing file, or add a new extension to qcow2 entirely (bypassing
> > the traditional backing file mechanism to avoid confusion for older
> > tooling) that adds a new read-write backing file field.
> > 
> > This RW backing file field will be used for all reads AND writes; the
> > qcow2 in question becomes a metadata container on top of the BDS chain.
> > We can re-use Vladimir's bitmap persistence extension to save bitmaps in
> > a qcow2 shell.
> > 
> > The qcow2 becomes effectively a metadata cache for a new (essentially)
> > filter node that handles features such as bitmaps. This could also be
> > used to provide allocation map data for RAW files and other goodies down
> > the road.
> > 
> > Hopefully this achieves our desire to not create new formats AND our
> > desire to concentrate features (and debugging, testing, etc) into qcow2,
> > while allowing users to "have bitmaps with raw files."
> > 
> > Of course, in this scenario, users now have two files: a qcow2 wrapper
> > and the actual raw file in question; but regardless of how we were going
> > to solve this, a raw file necessitates an external file of some sort,
> > else we give up the idea that it was a raw file.
> 
> There is some complexity here for management tools:
> 
> If the underlying image is resized, who resizes the qcow2 and how do
> they know to do it?
> 
> If QEMU's resize/truncate command it used, does first try to resize the
> underlying image and then resize the qcow2?  This is probably the sanest
> approach.
> 
> If the underlying image is moved to a new location, does the qcow2 file
> need to be modified and who does that?
> 
> Management tools need to figure out how to represent manage this extra
> qcow2 file.  The easiest solution is to punt it to the user and treat it
> as part of a backing file chain.  If the management tool wants to
> automatically manage the qcow2 so the user just specifies the underlying
> image and enables the persistent bitmap checkbox, then it becomes more
> complicated.

Indeed, I don't think it is practical to have libvirt / QEMU automagically
create a qcow2 overlay on disk. Something has to decide where this would
be stored. You might say just put it alongside the raw file, but it might
not be a local file at all, it could be a NBD, or RBD raw "file". So do
we create  local qcow2 file, or store a qcow2 file inside another RBD
volume to hold the persistent bitmap. This kind of decision needs to be
made by the mgmt app since only it knows about its storage mgmt model.
At this point you might as well just let the mgmt app take care of it
all and not try to do anything magical with qcow2 overlays in libvirt/QEMU

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Yaniv Lavi (Dary)]

YANIV LAVI (YANIV DARY)

SENIOR TECHNICAL PRODUCT MANAGER

Red Hat Israel Ltd. 

34 Jerusalem Road, Building A, 1st floor

Ra'anana, Israel 4350109

yl...@redhat.comT: +972-9-7692306/8272306 F: +972-9-7692223IM: ylavi
  TRIED. TESTED. TRUSTED. 
@redhatnews    Red Hat
   Red Hat



On Wed, Aug 30, 2017 at 1:35 PM, Max Reitz  wrote:

> On 2017-08-29 11:26, Yaniv Lavi (Dary) wrote:
> >
> >
> > YANIV LAVI (YANIV DARY)
> >
> > SENIOR TECHNICAL PRODUCT MANAGER
> >
> > Red Hat Israel Ltd. 
> >
> > 34 Jerusalem Road, Building A, 1st floor
> >
> > Ra'anana, Israel 4350109
> >
> > yl...@redhat.com T: +972-9-7692306
> > /8272306  F: +972-9-7692223
> > IM: ylavi
> >
> >   TRIED. TESTED. TRUSTED. <
> https://redhat.com/trusted>
> >
> > @redhatnews    Red Hat
> >    Red Hat
> > 
> >
> >
> > On Mon, Aug 28, 2017 at 9:11 PM, John Snow  > > wrote:
> >
> >
> >
> > On 08/27/2017 10:57 PM, Fam Zheng wrote:
> > > On Fri, 08/25 15:44, Max Reitz wrote:
> > >> Well, OK.  The main argument against supporting anything but
> qcow2 is
> > >> "if you want features, use qcow2; and we are working on making
> > qcow2 as
> > >> fast as possible."  I think that's a very good argument still.
> > At some
> > >> point I (and probably others, too) had the idea of making qcow2
> > files in
> > >> raw layout:
> > >
> > >
> > > Yes! I think this idea makes a whole lot of sense, too. Metadata
> > tables can be
> > > generated so old implementation can still use it.
> > >
> > > Fam
> > >
> > >> Have the data as a blob, just like a raw file, padded by
> > >> metadata around it.  An autoclear flag would specify that the
> > qcow2 file
> > >> is in this format, and if so, you could simply access it like a
> > raw file
> > >> and should have exactly the same speed as a raw file.  Maybe that
> > would
> > >> solve this whole issue, too?
> >
> > I wonder if this would be sufficient to alleviate the desire to use
> raw
> > files...
> >
> > (Eh, well, realistically, someone's still always going to ask if they
> > can use various features with non-qcow2 files...)
> >
> > Nir, Yaniv; any input?
> >
> >
> > We are using raw format for performance reasons.
>
> Using raw layout for the data in a qcow2 file would give you exactly the
> same performance as raw.
>

We had no reason to switch to anything else so far and I'm sure this option
was not available when we started supporting raw format.


>
> (Or better "should", but I can't think of a reason why it would not.)
>
> Max
>
> > As we have many customers that currently use this format, not support it
> > would be a blocker the use of the feature.
> > At a minimum we would require ability to convert raw to qcow2 raw-layout.
> >
> > Please also consider that we are planning to go on the OSP route of LUN
> > per disk and would still want the tracking to work.
> > I makes sense that for that and raw format you will be able to save the
> > mapping to another file other than a qcow.
> >
> >
> >
> > (Context: We're debating how to add persistent bitmaps to raw files
> as I
> > was informed that RHV was 'asking about it.' Max is reminding me
> there
> > is a proposal for a style of QCOW2 that uses a raw layout for data,
> > mitigating or eliminating any performance hits related to the L2
> cache.
> > What I am not aware of is why RHV would use raw files for any
> purpose.
> > Is it performance? Simplicity? Could RHV use a raw-layout qcow2?)
> >
> > --js
> >
> >
>
>
>

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Max Reitz]

On 2017-08-29 23:02, John Snow wrote:
> 
> 
> On 08/29/2017 10:30 AM, Eric Blake wrote:
>> On 08/28/2017 08:18 PM, John Snow wrote:
> We'd have to develop a new syntax for specifying these resources that
> can be stored in a qcow2 file,

 It's called the json-pseudo-protocol and was developed exactly for this.

>>>
>>> That's what I was hinting at for "or otherwise co-opt an existing
>>> syntax" but I was unaware that it was intended for "exactly" this.
>>>
>>> Do we actually use it in any on-disk format, currently? qcow2 only lets
>>> you specify simple filenames in the qcow2 metadata, right?
>>
>> You can specify json-pseudo backing names both on the command line AND
>> embedded in the qcow2 file itself (within the length limits imposed by
>> the qcow2 header size).  Yes, this means it is already possible to
>> create qcow2 files that can only be opened by qemu (or else teaching
>> your alternative program how to parse qemu's json-pseudo-protocol).
>>
>>>
>or otherwise co-opt an existing syntax
> in-use by QEMU. This syntax would likely be useful only to QEMU, which
> would steer the qcow2 format in a direction not too useful by other
> emulators, and qcow2 is an open format, so we may want to avoid this.

 Storing a file name in the backing link field that cannot be interpreted
 by other programs is in my opinion still very much better than not
 storing any information whatsoever, because in the former case other
 programs can at least say "sorry, I have no idea what this means" (or
 maybe they can indeed interpret it, who knows), whereas in the latter
 they may not even know that the qcow2 image is incomplete.

>>>
>>> I don't disagree personally, but I seem to recall that Kevin was adamant
>>> that the qcow2 bitmap extension should remain useful and semantically
>>> meaningful to third parties, so I try to keep that in mind. Maybe I
>>> should let him chime in instead of try to "concern troll" my own
>>> suggestions into the ground.
>>
>> We already have that situation today, but you are right to worry about
>> whether making it even more prevalent is something we can try to minimize.
>>
> 
> Proposal distillate:
> 
> (1) Specify relationship on CLI, QCOW becomes a bitmap-child of any
> arbitrary node.
> 
> Pros: Easy to implement
>   Adds bitmap support to literally everything
> Cons: Bitmap has no semantic link to data it describes
>   Relationship must be re-specified every launch
>   Max and Kevin are firmly NACK
> 
> (2) Raw file becomes a R/W backing file of the QCOW2, implemented as
> either a bitmap-child or a more traditional backing file that can
> additionally service writes
> 
> Pros: Easy to understand
>   relationship between files exists outside of the QEMU process
>   Adds bitmap support to just about everything that can be expressed
> via JSON
> Cons: All but necessitates QEMU-specific syntax in a qcow2 file
>   Depending on implementation, possibly messy in bdrv_open
>   Adding bitmaps to non-qcow2 files after open makes the launch CLI
> invalid for future launches (Not any different to snapshots.)
> 
> (3) Add a raw-like mapping mode to QCOW2 instead, skipping the whole affair
> 
> Pros: Adds a nice, performant hybrid mode to qcow2
>   Solves the problem of "bitmaps for raw" more or less
>   Avoids bdrv_open() complications
>   Avoids writing qemu-specific jargon in qcow2 files
> Cons: Doesn't actually add arbitrary bitmaps to any file format
>   Users are still gonna ask for bitmaps for other formats anyway
> 
> 
> 
> I think I like 2 or 3 -- or perhaps indeed two AND three. The qcow2-raw
> mode sounds like something we ought to have anyway. I'll try to start an
> RFC.

I think the only main thing that's missing for implementing (3) is a way
to mark allocated clusters to be fall-through-to-backing-file.

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Max Reitz]

On 2017-08-29 03:18, John Snow wrote:
> 
> 
> On 08/25/2017 09:44 AM, Max Reitz wrote:
>> On 2017-08-25 02:55, John Snow wrote:

[...]

>>> In (1), we avoid saving or specifying the relationship between these two
>>> data stores in any way. This is certainly easy to do, and will save us
>>> some headache on the CLI. As a downside, we now have random orphaned
>>> files that aren't very interesting or useful on their own. The
>>> likelihood for desync between metadata and data increases. The use of
>>> management software is all but necessitated.
>>>
>>> In (2) We have to now specify, with a dizzying long list of
>>> possibilities, the location of the implementation data. qcow2 only has a
>>> filename for backing files presently, but this is likely inadequate.
>>> What if the data store isn't a locally kept file? What if it's a socket,
>>> or a stream, or literally anything else?
>>
>> I don't see the difference.  In (1), your data image gets a "bitmap" or
>> "metadata" child.  In (2), your qcow2 image gets the usual "backing"
>> child, or maybe call it "passthrough" or whatever, if you want to make
>> the difference more explicit than just passing an option to the qcow2
>> image to pass writes to its backing file.
>>
> 
> The difference in the relationship in-memory is actually kind of
> uninteresting.
> 
> The difference as I see it is primarily how we specify the relationship
> between the qcow2 and the implementation storage; in (1) It's defined
> on-disk, and in (2) It's defined via CLI only, so
> 
> (1) Incurs a cost of having to define the link syntax (possibly causing
> a rather qemu-specific syntax), and
> (2) Avoids that cost, but leaves the data on-disk unrelated, which you hate.

What I meant was that in the end both have some parent-child
relationship in the BDS graph, so you can freely define both on the CLI.
So for (1), it usually is defined on-disk, yes, but you can also define
it on the command line -- just like it works for backing files.

>>> We'd have to develop a new syntax for specifying these resources that
>>> can be stored in a qcow2 file,
>>
>> It's called the json-pseudo-protocol and was developed exactly for this.
>>
> 
> That's what I was hinting at for "or otherwise co-opt an existing
> syntax" but I was unaware that it was intended for "exactly" this.
> 
> Do we actually use it in any on-disk format, currently? qcow2 only lets
> you specify simple filenames in the qcow2 metadata, right?

I don't think it's specified.  And I'm pretty sure that people would
agree that e.g. HTTP URLs are fine to specify in the backing link field.
 So I think that non-simple filenames are fine, too, but the question of
course is where to draw the line.

I'd argue that every time your start to allow filenames to have
protocols, you end up with some that not all programs can interpret;
like the json: protocol.

>>>or otherwise co-opt an existing syntax
>>> in-use by QEMU. This syntax would likely be useful only to QEMU, which
>>> would steer the qcow2 format in a direction not too useful by other
>>> emulators, and qcow2 is an open format, so we may want to avoid this.
>>
>> Storing a file name in the backing link field that cannot be interpreted
>> by other programs is in my opinion still very much better than not
>> storing any information whatsoever, because in the former case other
>> programs can at least say "sorry, I have no idea what this means" (or
>> maybe they can indeed interpret it, who knows), whereas in the latter
>> they may not even know that the qcow2 image is incomplete.
>>
> 
> I don't disagree personally, but I seem to recall that Kevin was adamant
> that the qcow2 bitmap extension should remain useful and semantically
> meaningful to third parties, so I try to keep that in mind. Maybe I
> should let him chime in instead of try to "concern troll" my own
> suggestions into the ground.

I seem to recall that I was adamant, too. :-)

All I'm saying is if you allow qcow2 to have writable backing files, it
doesn't make it any less meaningful to 3rd party programs than the
backing link field currently is.

>> Also note that we are making an effort to be able to generate plain file
>> names (such as URLs which should be usable by other programs) whenever
>> possible.
>>
> 
> Noted. Do we have a useful discriminator anywhere that allows us to
> easily check if a filename/locator/URI/whatever is in an accepted
> format, or if we still have QEMU-specific garbage?

Are NBD URLs qemu-specific? :-)

I think protocol URLs are just always icky.  If it's json: (when
bs->exact_filename is empty), you can be sure it's qemu-specific --
otherwise, it may be.

> We could always just disallow QEMU-specific protocol-talk from getting
> written and allow this only for configurations that QEMU understands to
> be universal...

I'd say it's too late for that, and also that json: was introduced
precisely to be able to get full BDS graph configuration into a backing
filename 

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Max Reitz]

On 2017-08-29 11:26, Yaniv Lavi (Dary) wrote:
> 
> 
> YANIV LAVI (YANIV DARY)
> 
> SENIOR TECHNICAL PRODUCT MANAGER
> 
> Red Hat Israel Ltd. 
> 
> 34 Jerusalem Road, Building A, 1st floor
> 
> Ra'anana, Israel 4350109
> 
> yl...@redhat.com     T: +972-9-7692306
> /8272306      F: +972-9-7692223
>     IM: ylavi
> 
>   TRIED. TESTED. TRUSTED. 
> 
> @redhatnews    Red Hat
>    Red Hat
> 
> 
> 
> On Mon, Aug 28, 2017 at 9:11 PM, John Snow  > wrote:
> 
> 
> 
> On 08/27/2017 10:57 PM, Fam Zheng wrote:
> > On Fri, 08/25 15:44, Max Reitz wrote:
> >> Well, OK.  The main argument against supporting anything but qcow2 is
> >> "if you want features, use qcow2; and we are working on making
> qcow2 as
> >> fast as possible."  I think that's a very good argument still. 
> At some
> >> point I (and probably others, too) had the idea of making qcow2
> files in
> >> raw layout:
> >
> >
> > Yes! I think this idea makes a whole lot of sense, too. Metadata
> tables can be
> > generated so old implementation can still use it.
> >
> > Fam
> >
> >> Have the data as a blob, just like a raw file, padded by
> >> metadata around it.  An autoclear flag would specify that the
> qcow2 file
> >> is in this format, and if so, you could simply access it like a
> raw file
> >> and should have exactly the same speed as a raw file.  Maybe that
> would
> >> solve this whole issue, too?
> 
> I wonder if this would be sufficient to alleviate the desire to use raw
> files...
> 
> (Eh, well, realistically, someone's still always going to ask if they
> can use various features with non-qcow2 files...)
> 
> Nir, Yaniv; any input?
> 
> 
> We are using raw format for performance reasons.

Using raw layout for the data in a qcow2 file would give you exactly the
same performance as raw.

(Or better "should", but I can't think of a reason why it would not.)

Max

> As we have many customers that currently use this format, not support it
> would be a blocker the use of the feature.
> At a minimum we would require ability to convert raw to qcow2 raw-layout.
> 
> Please also consider that we are planning to go on the OSP route of LUN
> per disk and would still want the tracking to work.
> I makes sense that for that and raw format you will be able to save the
> mapping to another file other than a qcow.
>  
> 
> 
> (Context: We're debating how to add persistent bitmaps to raw files as I
> was informed that RHV was 'asking about it.' Max is reminding me there
> is a proposal for a style of QCOW2 that uses a raw layout for data,
> mitigating or eliminating any performance hits related to the L2 cache.
> What I am not aware of is why RHV would use raw files for any purpose.
> Is it performance? Simplicity? Could RHV use a raw-layout qcow2?)
> 
> --js
> 
> 




signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[John Snow]

On 08/29/2017 10:30 AM, Eric Blake wrote:
> On 08/28/2017 08:18 PM, John Snow wrote:
 We'd have to develop a new syntax for specifying these resources that
 can be stored in a qcow2 file,
>>>
>>> It's called the json-pseudo-protocol and was developed exactly for this.
>>>
>>
>> That's what I was hinting at for "or otherwise co-opt an existing
>> syntax" but I was unaware that it was intended for "exactly" this.
>>
>> Do we actually use it in any on-disk format, currently? qcow2 only lets
>> you specify simple filenames in the qcow2 metadata, right?
> 
> You can specify json-pseudo backing names both on the command line AND
> embedded in the qcow2 file itself (within the length limits imposed by
> the qcow2 header size).  Yes, this means it is already possible to
> create qcow2 files that can only be opened by qemu (or else teaching
> your alternative program how to parse qemu's json-pseudo-protocol).
> 
>>
or otherwise co-opt an existing syntax
 in-use by QEMU. This syntax would likely be useful only to QEMU, which
 would steer the qcow2 format in a direction not too useful by other
 emulators, and qcow2 is an open format, so we may want to avoid this.
>>>
>>> Storing a file name in the backing link field that cannot be interpreted
>>> by other programs is in my opinion still very much better than not
>>> storing any information whatsoever, because in the former case other
>>> programs can at least say "sorry, I have no idea what this means" (or
>>> maybe they can indeed interpret it, who knows), whereas in the latter
>>> they may not even know that the qcow2 image is incomplete.
>>>
>>
>> I don't disagree personally, but I seem to recall that Kevin was adamant
>> that the qcow2 bitmap extension should remain useful and semantically
>> meaningful to third parties, so I try to keep that in mind. Maybe I
>> should let him chime in instead of try to "concern troll" my own
>> suggestions into the ground.
> 
> We already have that situation today, but you are right to worry about
> whether making it even more prevalent is something we can try to minimize.
> 

Proposal distillate:

(1) Specify relationship on CLI, QCOW becomes a bitmap-child of any
arbitrary node.

Pros: Easy to implement
  Adds bitmap support to literally everything
Cons: Bitmap has no semantic link to data it describes
  Relationship must be re-specified every launch
  Max and Kevin are firmly NACK

(2) Raw file becomes a R/W backing file of the QCOW2, implemented as
either a bitmap-child or a more traditional backing file that can
additionally service writes

Pros: Easy to understand
  relationship between files exists outside of the QEMU process
  Adds bitmap support to just about everything that can be expressed
via JSON
Cons: All but necessitates QEMU-specific syntax in a qcow2 file
  Depending on implementation, possibly messy in bdrv_open
  Adding bitmaps to non-qcow2 files after open makes the launch CLI
invalid for future launches (Not any different to snapshots.)

(3) Add a raw-like mapping mode to QCOW2 instead, skipping the whole affair

Pros: Adds a nice, performant hybrid mode to qcow2
  Solves the problem of "bitmaps for raw" more or less
  Avoids bdrv_open() complications
  Avoids writing qemu-specific jargon in qcow2 files
Cons: Doesn't actually add arbitrary bitmaps to any file format
  Users are still gonna ask for bitmaps for other formats anyway



I think I like 2 or 3 -- or perhaps indeed two AND three. The qcow2-raw
mode sounds like something we ought to have anyway. I'll try to start an
RFC.

--js

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Eric Blake]

On 08/28/2017 08:18 PM, John Snow wrote:
>>> We'd have to develop a new syntax for specifying these resources that
>>> can be stored in a qcow2 file,
>>
>> It's called the json-pseudo-protocol and was developed exactly for this.
>>
> 
> That's what I was hinting at for "or otherwise co-opt an existing
> syntax" but I was unaware that it was intended for "exactly" this.
> 
> Do we actually use it in any on-disk format, currently? qcow2 only lets
> you specify simple filenames in the qcow2 metadata, right?

You can specify json-pseudo backing names both on the command line AND
embedded in the qcow2 file itself (within the length limits imposed by
the qcow2 header size).  Yes, this means it is already possible to
create qcow2 files that can only be opened by qemu (or else teaching
your alternative program how to parse qemu's json-pseudo-protocol).

> 
>>>or otherwise co-opt an existing syntax
>>> in-use by QEMU. This syntax would likely be useful only to QEMU, which
>>> would steer the qcow2 format in a direction not too useful by other
>>> emulators, and qcow2 is an open format, so we may want to avoid this.
>>
>> Storing a file name in the backing link field that cannot be interpreted
>> by other programs is in my opinion still very much better than not
>> storing any information whatsoever, because in the former case other
>> programs can at least say "sorry, I have no idea what this means" (or
>> maybe they can indeed interpret it, who knows), whereas in the latter
>> they may not even know that the qcow2 image is incomplete.
>>
> 
> I don't disagree personally, but I seem to recall that Kevin was adamant
> that the qcow2 bitmap extension should remain useful and semantically
> meaningful to third parties, so I try to keep that in mind. Maybe I
> should let him chime in instead of try to "concern troll" my own
> suggestions into the ground.

We already have that situation today, but you are right to worry about
whether making it even more prevalent is something we can try to minimize.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Yaniv Lavi (Dary)]

YANIV LAVI (YANIV DARY)

SENIOR TECHNICAL PRODUCT MANAGER

Red Hat Israel Ltd. 

34 Jerusalem Road, Building A, 1st floor

Ra'anana, Israel 4350109

yl...@redhat.comT: +972-9-7692306/8272306 F: +972-9-7692223IM: ylavi
  TRIED. TESTED. TRUSTED. 
@redhatnews    Red Hat
   Red Hat



On Mon, Aug 28, 2017 at 9:11 PM, John Snow  wrote:

>
>
> On 08/27/2017 10:57 PM, Fam Zheng wrote:
> > On Fri, 08/25 15:44, Max Reitz wrote:
> >> Well, OK.  The main argument against supporting anything but qcow2 is
> >> "if you want features, use qcow2; and we are working on making qcow2 as
> >> fast as possible."  I think that's a very good argument still.  At some
> >> point I (and probably others, too) had the idea of making qcow2 files in
> >> raw layout:
> >
> >
> > Yes! I think this idea makes a whole lot of sense, too. Metadata tables
> can be
> > generated so old implementation can still use it.
> >
> > Fam
> >
> >> Have the data as a blob, just like a raw file, padded by
> >> metadata around it.  An autoclear flag would specify that the qcow2 file
> >> is in this format, and if so, you could simply access it like a raw file
> >> and should have exactly the same speed as a raw file.  Maybe that would
> >> solve this whole issue, too?
>
> I wonder if this would be sufficient to alleviate the desire to use raw
> files...
>
> (Eh, well, realistically, someone's still always going to ask if they
> can use various features with non-qcow2 files...)
>
> Nir, Yaniv; any input?
>

We are using raw format for performance reasons.
As we have many customers that currently use this format, not support it
would be a blocker the use of the feature.
At a minimum we would require ability to convert raw to qcow2 raw-layout.

Please also consider that we are planning to go on the OSP route of LUN per
disk and would still want the tracking to work.
I makes sense that for that and raw format you will be able to save the
mapping to another file other than a qcow.


>
> (Context: We're debating how to add persistent bitmaps to raw files as I
> was informed that RHV was 'asking about it.' Max is reminding me there
> is a proposal for a style of QCOW2 that uses a raw layout for data,
> mitigating or eliminating any performance hits related to the L2 cache.
> What I am not aware of is why RHV would use raw files for any purpose.
> Is it performance? Simplicity? Could RHV use a raw-layout qcow2?)
>
> --js
>

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[John Snow]

On 08/25/2017 09:44 AM, Max Reitz wrote:
> On 2017-08-25 02:55, John Snow wrote:
>> Sorry in advance for :words: ...
>>
>> On 08/23/2017 02:04 PM, Vladimir Sementsov-Ogievskiy wrote:
>>> 23.08.2017 11:59, Vladimir Sementsov-Ogievskiy wrote:
 22.08.2017 22:07, John Snow wrote:

[snip]


 Should there be some problems with internal snapshots and other things?
> 
> I'd suspect you get exactly the same problems when using internal
> snapshots together with backing files.  Imagine a newly created overlay
> file and taking a snapshot.  This should give you exactly the same
> issue, doesn't it?
> 



>>> Hm. looks like that this backing file should not only receive all reads
>>> and writes, but almost everything ->bdrv_ handlers, except bitmap
>>> related of course.
> 
> How so?  Shouldn't it just work like a backing file, except it also
> receives writes instead of just reads?
> 
>>>This doesn't seems simple to implement. Especially if
>>> imaging some not-raw feature-full format under this thin qcow2 layer..
>>> Or we can restrict this RW backing file to be raw-only?
>>>
>>>
>>
>> The idea would really be to support any arbitrary data store, so I
>> wouldn't want to restrict it to just raw.
>>
>> You're right though, this might be a kind of messy approach.
>>
>> [From your other mail:]
>>
>>>
>>> So, anyway, I see only two differences (from the outside) between this 
>>> approach and just a separate bitmap-only qcow2 without a data:
>>>
>>
>> It's very delicately similar, yes.
>>
>>> 1. in RW-backing approach qcow2-bitmap file has a link to data file (as a 
>>> backing). It looks good.
> 
> And this is rather important to me.
> 

Good to know. Some good solid opinions to work around. ;)

>> Right. The information necessary to establish a link between the bitmap
>> data and the data being described is fully contained within a file fully
>> specified by the QCOW2 spec.
>>
>>> 2. in RW-backing approach qcow2-bitmap file is a top of the virtual disk, 
>>> in separate-file approach it is an option of the real data drive. In my 
>>> opinion the second is more clean for users ("to add this feature you should 
>>> use other file as your disk" vs "to add this feature you should add an 
>>> option to your disk description")
> 
> I'd argue it's rather: "You cannot use this feature unless your format
> supports it.  The only format supporting persistent bitmaps currently is
> qcow2.  To use persistent bitmaps with other formats you can attach them
> as R/W backing files to an empty qcow2 file, though."
> 
> So the difference is that you are saying it's a feature that is added to
> a non-qcow2 image whereas I'm saying it's a feature that only a qcow2
> image can provide (currently).
> 
>> This puts us a little closer to the original idea that was rejected by
>> Kevin at the time. To recap:
>>
>> "1": Use qcow2 as a container. This was rejected because we didn't want
>> qcow2 containing data with no semantic relationship to the qcow2
>> container or to each other. The way it sounds like you're proposing it,
>> though, it would be one-qcow2-with-bitmaps-per-drive, so the data would
>> at least stay strictly related, but it would be meaningless outside of
>> QEMU itself. I think this is something that Kevin wanted to avoid, but I
>> can't speak for him.
>>
>> It's certainly not beyond the realm of management software to remember
>> to correlate a qcow2 metadata file alongside its actual data stores
>> whenever it needed to do so, but it does mean the introduction of a
>> feature that essentially requires the use of management software, which
>> sees resistance in the community at times.
>>
>> In this model, you'd probably have the raw drive at the top, with the
>> qcow2-with-bitmaps as a child node with some kind of new named child
>> relationship. All IO stays at the root node, but the bitmap method
>> handlers would know to look for this special bitmap-child. It shouldn't
>> be too hard to implement.
> 
> I'd still like to throw in how much I dislike this approach, and I can't
> really think of a way to make it palatable to me.  Not even "just write
> the file name of the image the bitmaps cover into the qcow2 file" sounds
> good to me, because then it still is basically unrelated data.
> 

Understood. It's something I'd like to avoid too, but I have some real
concerns about implementation of that semantic link.

> The only approach that I might see myself liking is to indeed add a flag
> or whatever to say a qcow2 backing image is supposed to be R/W; and then
> (after somehow verifying that the qcow2 image itself is empty) just make
> qemu interpret this as "load the backing file as the real disk and
> attach the qcow2 image as a 'metadata' child" or whatever.  But I fear
> this gets uglier and uglier because how qemu loads the files will then
> depend on whether the overlay is empty or not, and this may be very
> confusing.
> 

Right, it makes opening a little more convoluted 

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[John Snow]

On 08/27/2017 10:57 PM, Fam Zheng wrote:
> On Fri, 08/25 15:44, Max Reitz wrote:
>> Well, OK.  The main argument against supporting anything but qcow2 is
>> "if you want features, use qcow2; and we are working on making qcow2 as
>> fast as possible."  I think that's a very good argument still.  At some
>> point I (and probably others, too) had the idea of making qcow2 files in
>> raw layout: 
> 
> 
> Yes! I think this idea makes a whole lot of sense, too. Metadata tables can be
> generated so old implementation can still use it.
> 
> Fam
> 
>> Have the data as a blob, just like a raw file, padded by
>> metadata around it.  An autoclear flag would specify that the qcow2 file
>> is in this format, and if so, you could simply access it like a raw file
>> and should have exactly the same speed as a raw file.  Maybe that would
>> solve this whole issue, too?

I wonder if this would be sufficient to alleviate the desire to use raw
files...

(Eh, well, realistically, someone's still always going to ask if they
can use various features with non-qcow2 files...)

Nir, Yaniv; any input?

(Context: We're debating how to add persistent bitmaps to raw files as I
was informed that RHV was 'asking about it.' Max is reminding me there
is a proposal for a style of QCOW2 that uses a raw layout for data,
mitigating or eliminating any performance hits related to the L2 cache.
What I am not aware of is why RHV would use raw files for any purpose.
Is it performance? Simplicity? Could RHV use a raw-layout qcow2?)

--js

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Fam Zheng]

On Fri, 08/25 15:44, Max Reitz wrote:
> Well, OK.  The main argument against supporting anything but qcow2 is
> "if you want features, use qcow2; and we are working on making qcow2 as
> fast as possible."  I think that's a very good argument still.  At some
> point I (and probably others, too) had the idea of making qcow2 files in
> raw layout: 


Yes! I think this idea makes a whole lot of sense, too. Metadata tables can be
generated so old implementation can still use it.

Fam

> Have the data as a blob, just like a raw file, padded by
> metadata around it.  An autoclear flag would specify that the qcow2 file
> is in this format, and if so, you could simply access it like a raw file
> and should have exactly the same speed as a raw file.  Maybe that would
> solve this whole issue, too?

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Max Reitz]

On 2017-08-25 02:55, John Snow wrote:
> Sorry in advance for :words: ...
> 
> On 08/23/2017 02:04 PM, Vladimir Sementsov-Ogievskiy wrote:
>> 23.08.2017 11:59, Vladimir Sementsov-Ogievskiy wrote:
>>> 22.08.2017 22:07, John Snow wrote:
 Well, we knew we'd want this sooner or later. I've got some pings
 downstream over whether or not we support persistent bitmaps for
 non-qcow2 formats.

 Currently: no, we don't.

 We tried two different ideas for storage agnostic bitmap persistence:


 (1) Using qcow2 as a storage container format (similar to VM save
 states) for information not associated with that particular drive.

 This didn't go over so well; we decided it was too messy logistically to
 manage data in any meaningful sense in a file that didn't share any
 semantic relationship to the qcow2 in question.

 Well, "We" decided is more like "Kevin and Max" decided. They are right,
 though.


 (2) Using a new proto-wrapper format that Fam Zheng designed that serves
 as a simple top-layer redirect for metadata that allows us to associate
 raw bitmap data with an arbitrary backing image.

 This also didn't go over so well, the desire for a "new format" was
 pretty thin, even if it was only a pseudo-format.


 We'd still like to be able to use bitmaps with non-qcow2 formats
 however, and people are going to keep asking. So here's a third thought:


 (3) Add either a new flag that turns qcow2's backing file into a full
 R/W backing file, or add a new extension to qcow2 entirely (bypassing
 the traditional backing file mechanism to avoid confusion for older
 tooling) that adds a new read-write backing file field.

 This RW backing file field will be used for all reads AND writes; the
 qcow2 in question becomes a metadata container on top of the BDS chain.
 We can re-use Vladimir's bitmap persistence extension to save bitmaps in
 a qcow2 shell.

 The qcow2 becomes effectively a metadata cache for a new (essentially)
 filter node that handles features such as bitmaps. This could also be
 used to provide allocation map data for RAW files and other goodies down
 the road.

 Hopefully this achieves our desire to not create new formats AND our
 desire to concentrate features (and debugging, testing, etc) into qcow2,
 while allowing users to "have bitmaps with raw files."

 Of course, in this scenario, users now have two files: a qcow2 wrapper
 and the actual raw file in question; but regardless of how we were going
 to solve this, a raw file necessitates an external file of some sort,
 else we give up the idea that it was a raw file.


 Sound good?
>>>
>>> Looks interesting. It is a clean reusing of qcow2-bitmaps without any
>>> modifications to them.
>>>
>>> Should there be some problems with internal snapshots and other things?

I'd suspect you get exactly the same problems when using internal
snapshots together with backing files.  Imagine a newly created overlay
file and taking a snapshot.  This should give you exactly the same
issue, doesn't it?

>>>
>>>
>>>
>> Hm. looks like that this backing file should not only receive all reads
>> and writes, but almost everything ->bdrv_ handlers, except bitmap
>> related of course.

How so?  Shouldn't it just work like a backing file, except it also
receives writes instead of just reads?

>>This doesn't seems simple to implement. Especially if
>> imaging some not-raw feature-full format under this thin qcow2 layer..
>> Or we can restrict this RW backing file to be raw-only?
>>
>>
> 
> The idea would really be to support any arbitrary data store, so I
> wouldn't want to restrict it to just raw.
> 
> You're right though, this might be a kind of messy approach.
> 
> [From your other mail:]
> 
>>
>> So, anyway, I see only two differences (from the outside) between this 
>> approach and just a separate bitmap-only qcow2 without a data:
>>
> 
> It's very delicately similar, yes.
> 
>> 1. in RW-backing approach qcow2-bitmap file has a link to data file (as a 
>> backing). It looks good.

And this is rather important to me.

> Right. The information necessary to establish a link between the bitmap
> data and the data being described is fully contained within a file fully
> specified by the QCOW2 spec.
> 
>> 2. in RW-backing approach qcow2-bitmap file is a top of the virtual disk, in 
>> separate-file approach it is an option of the real data drive. In my opinion 
>> the second is more clean for users ("to add this feature you should use 
>> other file as your disk" vs "to add this feature you should add an option to 
>> your disk description")

I'd argue it's rather: "You cannot use this feature unless your format
supports it.  The only format supporting persistent bitmaps currently is
qcow2.  To use persistent bitmaps with other 

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[Vladimir Sementsov-Ogievskiy]

25.08.2017 03:55, John Snow wrote:

Sorry in advance for :words: ...

On 08/23/2017 02:04 PM, Vladimir Sementsov-Ogievskiy wrote:

23.08.2017 11:59, Vladimir Sementsov-Ogievskiy wrote:

22.08.2017 22:07, John Snow wrote:

Well, we knew we'd want this sooner or later. I've got some pings
downstream over whether or not we support persistent bitmaps for
non-qcow2 formats.

Currently: no, we don't.

We tried two different ideas for storage agnostic bitmap persistence:


(1) Using qcow2 as a storage container format (similar to VM save
states) for information not associated with that particular drive.

This didn't go over so well; we decided it was too messy logistically to
manage data in any meaningful sense in a file that didn't share any
semantic relationship to the qcow2 in question.

Well, "We" decided is more like "Kevin and Max" decided. They are right,
though.


(2) Using a new proto-wrapper format that Fam Zheng designed that serves
as a simple top-layer redirect for metadata that allows us to associate
raw bitmap data with an arbitrary backing image.

This also didn't go over so well, the desire for a "new format" was
pretty thin, even if it was only a pseudo-format.


We'd still like to be able to use bitmaps with non-qcow2 formats
however, and people are going to keep asking. So here's a third thought:


(3) Add either a new flag that turns qcow2's backing file into a full
R/W backing file, or add a new extension to qcow2 entirely (bypassing
the traditional backing file mechanism to avoid confusion for older
tooling) that adds a new read-write backing file field.

This RW backing file field will be used for all reads AND writes; the
qcow2 in question becomes a metadata container on top of the BDS chain.
We can re-use Vladimir's bitmap persistence extension to save bitmaps in
a qcow2 shell.

The qcow2 becomes effectively a metadata cache for a new (essentially)
filter node that handles features such as bitmaps. This could also be
used to provide allocation map data for RAW files and other goodies down
the road.

Hopefully this achieves our desire to not create new formats AND our
desire to concentrate features (and debugging, testing, etc) into qcow2,
while allowing users to "have bitmaps with raw files."

Of course, in this scenario, users now have two files: a qcow2 wrapper
and the actual raw file in question; but regardless of how we were going
to solve this, a raw file necessitates an external file of some sort,
else we give up the idea that it was a raw file.


Sound good?

Looks interesting. It is a clean reusing of qcow2-bitmaps without any
modifications to them.

Should there be some problems with internal snapshots and other things?




Hm. looks like that this backing file should not only receive all reads
and writes, but almost everything ->bdrv_ handlers, except bitmap
related of course. This doesn't seems simple to implement. Especially if
imaging some not-raw feature-full format under this thin qcow2 layer..
Or we can restrict this RW backing file to be raw-only?



The idea would really be to support any arbitrary data store, so I
wouldn't want to restrict it to just raw.

You're right though, this might be a kind of messy approach.

[From your other mail:]


So, anyway, I see only two differences (from the outside) between this approach 
and just a separate bitmap-only qcow2 without a data:


It's very delicately similar, yes.


1. in RW-backing approach qcow2-bitmap file has a link to data file (as a 
backing). It looks good.


Right. The information necessary to establish a link between the bitmap
data and the data being described is fully contained within a file fully
specified by the QCOW2 spec.


2. in RW-backing approach qcow2-bitmap file is a top of the virtual disk, in separate-file approach 
it is an option of the real data drive. In my opinion the second is more clean for users ("to 
add this feature you should use other file as your disk" vs "to add this feature you 
should add an option to your disk description")

This puts us a little closer to the original idea that was rejected by
Kevin at the time. To recap:

"1": Use qcow2 as a container. This was rejected because we didn't want
qcow2 containing data with no semantic relationship to the qcow2
container or to each other. The way it sounds like you're proposing it,
though, it would be one-qcow2-with-bitmaps-per-drive, so the data would
at least stay strictly related, but it would be meaningless outside of
QEMU itself. I think this is something that Kevin wanted to avoid, but I
can't speak for him.

It's certainly not beyond the realm of management software to remember
to correlate a qcow2 metadata file alongside its actual data stores
whenever it needed to do so, but it does mean the introduction of a
feature that essentially requires the use of management software, which
sees resistance in the community at times.

In this model, you'd probably have the raw drive at the top, with the
qcow2-with-bitmaps as a child node 

Re: [Qemu-block] [Qemu-devel] Persistent bitmaps for non-qcow2 formats

[John Snow]

Sorry in advance for :words: ...

On 08/23/2017 02:04 PM, Vladimir Sementsov-Ogievskiy wrote:
> 23.08.2017 11:59, Vladimir Sementsov-Ogievskiy wrote:
>> 22.08.2017 22:07, John Snow wrote:
>>> Well, we knew we'd want this sooner or later. I've got some pings
>>> downstream over whether or not we support persistent bitmaps for
>>> non-qcow2 formats.
>>>
>>> Currently: no, we don't.
>>>
>>> We tried two different ideas for storage agnostic bitmap persistence:
>>>
>>>
>>> (1) Using qcow2 as a storage container format (similar to VM save
>>> states) for information not associated with that particular drive.
>>>
>>> This didn't go over so well; we decided it was too messy logistically to
>>> manage data in any meaningful sense in a file that didn't share any
>>> semantic relationship to the qcow2 in question.
>>>
>>> Well, "We" decided is more like "Kevin and Max" decided. They are right,
>>> though.
>>>
>>>
>>> (2) Using a new proto-wrapper format that Fam Zheng designed that serves
>>> as a simple top-layer redirect for metadata that allows us to associate
>>> raw bitmap data with an arbitrary backing image.
>>>
>>> This also didn't go over so well, the desire for a "new format" was
>>> pretty thin, even if it was only a pseudo-format.
>>>
>>>
>>> We'd still like to be able to use bitmaps with non-qcow2 formats
>>> however, and people are going to keep asking. So here's a third thought:
>>>
>>>
>>> (3) Add either a new flag that turns qcow2's backing file into a full
>>> R/W backing file, or add a new extension to qcow2 entirely (bypassing
>>> the traditional backing file mechanism to avoid confusion for older
>>> tooling) that adds a new read-write backing file field.
>>>
>>> This RW backing file field will be used for all reads AND writes; the
>>> qcow2 in question becomes a metadata container on top of the BDS chain.
>>> We can re-use Vladimir's bitmap persistence extension to save bitmaps in
>>> a qcow2 shell.
>>>
>>> The qcow2 becomes effectively a metadata cache for a new (essentially)
>>> filter node that handles features such as bitmaps. This could also be
>>> used to provide allocation map data for RAW files and other goodies down
>>> the road.
>>>
>>> Hopefully this achieves our desire to not create new formats AND our
>>> desire to concentrate features (and debugging, testing, etc) into qcow2,
>>> while allowing users to "have bitmaps with raw files."
>>>
>>> Of course, in this scenario, users now have two files: a qcow2 wrapper
>>> and the actual raw file in question; but regardless of how we were going
>>> to solve this, a raw file necessitates an external file of some sort,
>>> else we give up the idea that it was a raw file.
>>>
>>>
>>> Sound good?
>>
>> Looks interesting. It is a clean reusing of qcow2-bitmaps without any
>> modifications to them.
>>
>> Should there be some problems with internal snapshots and other things?
>>
>>
>>
> Hm. looks like that this backing file should not only receive all reads
> and writes, but almost everything ->bdrv_ handlers, except bitmap
> related of course. This doesn't seems simple to implement. Especially if
> imaging some not-raw feature-full format under this thin qcow2 layer..
> Or we can restrict this RW backing file to be raw-only?
> 
> 

The idea would really be to support any arbitrary data store, so I
wouldn't want to restrict it to just raw.

You're right though, this might be a kind of messy approach.

[From your other mail:]

> 
> So, anyway, I see only two differences (from the outside) between this 
> approach and just a separate bitmap-only qcow2 without a data:
> 

It's very delicately similar, yes.

> 1. in RW-backing approach qcow2-bitmap file has a link to data file (as a 
> backing). It looks good.
> 

Right. The information necessary to establish a link between the bitmap
data and the data being described is fully contained within a file fully
specified by the QCOW2 spec.

> 2. in RW-backing approach qcow2-bitmap file is a top of the virtual disk, in 
> separate-file approach it is an option of the real data drive. In my opinion 
> the second is more clean for users ("to add this feature you should use other 
> file as your disk" vs "to add this feature you should add an option to your 
> disk description")

This puts us a little closer to the original idea that was rejected by
Kevin at the time. To recap:

"1": Use qcow2 as a container. This was rejected because we didn't want
qcow2 containing data with no semantic relationship to the qcow2
container or to each other. The way it sounds like you're proposing it,
though, it would be one-qcow2-with-bitmaps-per-drive, so the data would
at least stay strictly related, but it would be meaningless outside of
QEMU itself. I think this is something that Kevin wanted to avoid, but I
can't speak for him.

It's certainly not beyond the realm of management software to remember
to correlate a qcow2 metadata file alongside its actual data stores
whenever it needed to do so, but it does mean the