Re: [Xen-devel] [RFC V9 4/4] domain snapshot design: libxl/libxlu

2015-01-09 Thread Chun Yan Liu


 On 1/8/2015 at 08:11 PM, in message 1420719107.19787.53.ca...@citrix.com, 
 Ian
Campbell ian.campb...@citrix.com wrote: 
 On Mon, 2014-12-22 at 02:36 -0700, Chun Yan Liu wrote: 
b). For internal snapshot, like qcow2, lvm too. For lvm, it doesn't  
 support  
snapshot of snapshot, so out of scope. For qcow2, delete any disk  
 snapshot  
won't affect others.  
 
   For either internal or external if you are removing a snapshot from the  
   middle of a chain which ends in one or more active disks, then surely  
   the disk backend associated with those domains need to get some sort of  
   notification, otherwise they would need to be written *very* carefully  
   in order to be able to cope with disk metadata changing under their  
   feet.  
 
   Are you saying that the qemu/qcow implementation has indeed been written  
   with this in mind and can cope with arbitrary other processes modifying  
   the qcow metadata under their feet?  
   
  Yes. 
   
  I add qemu-devel Kevin and Stefan in this thread in case my understanding 
  has somewhere wrong. 
   
  Kevin  Stefan, 
   
  About the qcow2 snapshot implementation,  in following snapshot chain case, 
  if we delete SNAPSHOT A, will it affect domain 1 and domain 2 which uses 
  SNAPSHOT B and SNAPSHOT C? 
   
  From my understanding, creating a snapshot will increases refcount of  
 original data, 
  deleting a snapshot only descreases the refcount (won't delete data until  
 the refcount 
  becomes 0), so I think it won't affect domain 1 and domain 2. 
  Is that right? 
  
 I'm not worried about the data being deleted (I'm sure qcow2 will get 
 that right), but rather about the snapshot chain being collapsed and 
 therefore the metadata (e.g. the tables of which block is in which 
 backing file, and perhaps the location of the data itself) changing 
 while a domain is running, e.g. 
  
 BASE---SNAPSHOT A---SNAPSHOT B --- domain 1  
  `--SNAPSHOT C --- domain 2  
  
 becoming  
  
 BASESNAPSHOT B --- domain 1  
  `--SNAPSHOT C --- domain 2  
  
 (essentially changing B and C's tables to accommodate the lack of A) 
  
 For an internal snapshot I can see that it would be sensible (and easy) 
 to keep A around as a ghost, to avoid this case, and the need to 
 perhaps move data around or duplicate it. 

This is what we talked about a lot :-)

If I understand correctly, qcow2 internal snapshot implementation
should be like this:
Taking a snapshot is increasing a new table which records the data
clusters index in this snapshot, at the same time increase the refcount
of those data clusters;
Teleting a snapshot only decrease the refcount (not deleting the data,
only when refcount=0 the data will be deleted.) and remove the map table.

So, I think deleting a snapshot even within a snapshot chain, won't
affect other snapshots. ( Otherwise, even if that's not the case, how can
we move internal data around or duplicate it? )

  
 If A were external though an admin might think they could delete the 
 file...

About external, generally, external disk snapshot are made by backing file,
qemu, vhd-util and lvm are all done that way. Like:
snapshot A - snapshot B - Current image
A is B's backing file, B is current image's backing file, as backing file, it
should not be changed, otherwise if A is changed or deleted, B is corrupted.
So, a delete operation is actually a merge process, like 'virsh blockcommit':
shorten disk image chain by live merging the current active disk content.

To external disk snapshot: revert (may need to clone the snapshot out)
and delete (merge in fact as mentioned above). The complexity is
internal/external mixing case (take internal snapshot, revert, take
external snapshot, ...)

Chunyan


 Ian. 
  
  
  


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC V9 4/4] domain snapshot design: libxl/libxlu

2015-01-08 Thread Ian Campbell
On Mon, 2014-12-22 at 02:36 -0700, Chun Yan Liu wrote:
   b). For internal snapshot, like qcow2, lvm too. For lvm, it doesn't 
   support 
   snapshot of snapshot, so out of scope. For qcow2, delete any disk 
   snapshot 
   won't affect others. 
   
  For either internal or external if you are removing a snapshot from the 
  middle of a chain which ends in one or more active disks, then surely 
  the disk backend associated with those domains need to get some sort of 
  notification, otherwise they would need to be written *very* carefully 
  in order to be able to cope with disk metadata changing under their 
  feet. 
   
  Are you saying that the qemu/qcow implementation has indeed been written 
  with this in mind and can cope with arbitrary other processes modifying 
  the qcow metadata under their feet? 
 
 Yes.
 
 I add qemu-devel Kevin and Stefan in this thread in case my understanding
 has somewhere wrong.
 
 Kevin  Stefan,
 
 About the qcow2 snapshot implementation,  in following snapshot chain case,
 if we delete SNAPSHOT A, will it affect domain 1 and domain 2 which uses
 SNAPSHOT B and SNAPSHOT C?
 
 From my understanding, creating a snapshot will increases refcount of 
 original data,
 deleting a snapshot only descreases the refcount (won't delete data until the 
 refcount
 becomes 0), so I think it won't affect domain 1 and domain 2.
 Is that right?

I'm not worried about the data being deleted (I'm sure qcow2 will get
that right), but rather about the snapshot chain being collapsed and
therefore the metadata (e.g. the tables of which block is in which
backing file, and perhaps the location of the data itself) changing
while a domain is running, e.g.

BASE---SNAPSHOT A---SNAPSHOT B --- domain 1 
 `--SNAPSHOT C --- domain 2 

becoming 

BASESNAPSHOT B --- domain 1 
 `--SNAPSHOT C --- domain 2 

(essentially changing B and C's tables to accommodate the lack of A)

For an internal snapshot I can see that it would be sensible (and easy)
to keep A around as a ghost, to avoid this case, and the need to
perhaps move data around or duplicate it.

If A were external though an admin might think they could delete the
file...

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC V9 4/4] domain snapshot design: libxl/libxlu

2014-12-22 Thread Chun Yan Liu


 On 12/19/2014 at 06:38 PM, in message 
 1418985490.20028.27.ca...@citrix.com,
Ian Campbell ian.campb...@citrix.com wrote: 
 On Thu, 2014-12-18 at 23:58 -0700, Chun Yan Liu wrote: 
   
   On 12/18/2014 at 11:27 PM, in message  
 1418916436.11882.101.ca...@citrix.com, 
  Ian Campbell ian.campb...@citrix.com wrote:  
   On Tue, 2014-12-16 at 14:32 +0800, Chunyan Liu wrote:  
Changes to V8:  
  * remove libxl_domain_snapshot_create/delete/revert API  
  * export disk snapshot functionality for both xl and libvirt usage  
  
 
 ===  
Libxl/libxlu Design  
  
1. New Structures  
  
libxl_disk_snapshot = Struct(disk_snapshot,[  
# target disk  
(disk,libxl_device_disk),  
  
# disk snapshot name  
(name,string),  
  
# internal/external disk snapshot?  
(external,bool),  
  
# for external disk snapshot, specify following two field  
(external_format, string),  
(external_path,   string),  
 
   Should this be a KeyedUnion over a new LIBXL_DISK_SNAPSHOT_KIND enum  
   (with values INTERNAL and EXTERNAL)? 
  The KeyedUnion seems to be unnecessary. Only EXTERNAL has data items, 
  INTERNAL doesn't, and no third types. 
   
   This would automatically make the  
   binding between external==true and the fields which depend on that.  
 
   external_format should be of type libxl_disk_format, unless it is  
   referring to something else?  
   
  Yes. That's right. I'll update. 
   
 
   Is it possible for format to differ from the format of the underlying  
   disk? Perhaps taking a snapshot of a raw disk as a qcow? 
   
  This is related to implementation details. As I understand qemu's 
  implementation, taking an external disk snapshot is actually a way: 
  origin domain disk: a raw disk 
  external= true, external_format: qcow2, external_path: test 
  a), create a qcow2 file (test.qcow2) with  backing file (the raw disk) 
  b), replace domain disk, now domain uses test.qcow2 (the raw disk 
   is actually to be the snapshot) 
   
  So, I think the external_format can only be those supporting backing file. 
  
 Not sure what you mean here. 
  
 What about a phy snapshot via lvm snapshotting? 
  
   In any case  
   passing in UNKNOWN and letting libxl choose (probably by picking the  
   same as the underlying disk) should be supported. 
   
  If external_format is not passed (NULL), by default, we will use qcow2. 
  
 I think you need to base this on the type of the original disk, if it is 
 e.g. vhd then making a qcow snapshot seems a bit odd. 
  
   
 
/*  This API might not be used by xl, since xl won't take care of  
 deleting  
 *  snapshots. But for libvirt, since libvirt manages snapshots and 
will  
 *  delete snapshot, this API will be used.  
 */  
int libxl_disk_snapshot_delete(libxl_ctx *ctx, uint32_t domid,  
   libxl_disk_snapshot *snapshot, int nb);  
 
   The three usecases I mentioned in the previous mail are important here,  
   because depending on which usecases you are considering there maybe a  
   many to one relationship between domains and a given snapshot (gold  
   image case). This interface cannot support that I think. 
   
  I'm not quite clear about the three usecases, especially the 3rd usercase, 
  so really not sure what's the requirement towards deleting disk snapshot. 
  
 I hope my reply to the previous mail helped clear this up a bit. The 
 reason deleting a disk is interesting is because that is what you would 
 do after the backup was finished. 
  
   When we discussed this in previous iterations I suggested a libxl  
   command to tell a VM that it needed to reexamine its disks to see if any  
   of the chains had changed. I'm sure that's not the only potential answer  
   though. 

  About delete disk snapshot in a snapshot chain, whether we need to do 
  extra work to avoid data break, it can be discussed: 
  a). For external snapshots, usually it's based on backing file chain, qemu 
  does this, vhd-util does this. In this case, to delete a domain snapshot, 
  one doesn't need to do anything to disk (no need to delete disk snapshot 
  at all). Downside is, there might be a long backing chain. 
  
 I'm not sure what you mean here I'm afraid. If you are deleting a domain 
 snapshot why do you not want to delete the disk snapshots associated 
 with it? 
  
  b). For internal snapshot, like qcow2, lvm too. For lvm, it doesn't support 
  snapshot of snapshot, so out of scope. For qcow2, delete any disk snapshot 
  won't affect others. 
  
 For either internal or external if you are removing a snapshot from the 
 middle of a chain which ends in one or more active disks, then surely 
 the disk backend associated with those domains need to get some sort of 
 

Re: [Xen-devel] [RFC V9 4/4] domain snapshot design: libxl/libxlu

2014-12-22 Thread Chun Yan Liu


 On 12/19/2014 at 06:38 PM, in message 
 1418985490.20028.27.ca...@citrix.com,
Ian Campbell ian.campb...@citrix.com wrote: 
 On Thu, 2014-12-18 at 23:58 -0700, Chun Yan Liu wrote: 
   
   On 12/18/2014 at 11:27 PM, in message  
 1418916436.11882.101.ca...@citrix.com, 
  Ian Campbell ian.campb...@citrix.com wrote:  
   On Tue, 2014-12-16 at 14:32 +0800, Chunyan Liu wrote:  
Changes to V8:  
  * remove libxl_domain_snapshot_create/delete/revert API  
  * export disk snapshot functionality for both xl and libvirt usage  
  
 
 ===  
Libxl/libxlu Design  
  
1. New Structures  
  
libxl_disk_snapshot = Struct(disk_snapshot,[  
# target disk  
(disk,libxl_device_disk),  
  
# disk snapshot name  
(name,string),  
  
# internal/external disk snapshot?  
(external,bool),  
  
# for external disk snapshot, specify following two field  
(external_format, string),  
(external_path,   string),  
 
   Should this be a KeyedUnion over a new LIBXL_DISK_SNAPSHOT_KIND enum  
   (with values INTERNAL and EXTERNAL)? 
  The KeyedUnion seems to be unnecessary. Only EXTERNAL has data items, 
  INTERNAL doesn't, and no third types. 
   
   This would automatically make the  
   binding between external==true and the fields which depend on that.  
 
   external_format should be of type libxl_disk_format, unless it is  
   referring to something else?  
   
  Yes. That's right. I'll update. 
   
 
   Is it possible for format to differ from the format of the underlying  
   disk? Perhaps taking a snapshot of a raw disk as a qcow? 
   
  This is related to implementation details. As I understand qemu's 
  implementation, taking an external disk snapshot is actually a way: 
  origin domain disk: a raw disk 
  external= true, external_format: qcow2, external_path: test 
  a), create a qcow2 file (test.qcow2) with  backing file (the raw disk) 
  b), replace domain disk, now domain uses test.qcow2 (the raw disk 
   is actually to be the snapshot) 
   
  So, I think the external_format can only be those supporting backing file.

Well, yeah, I should correct. This is only valid for creating external snapshot 
by
'qemu-img' tool, not fit for lvm and vhd, which have  their own snapshot
functionality tools.

For lvm, the snapshot can be done by 'lvcreate snapshot', snapshot file is also
'lvm' format;

For vhd, the snapshot can be done by 'vht-util snapshot' , then snapshot file
is still vhd format; snapshot also can be done by 'qemu-img snapshot', then the
external format should be a format supported by qemu-img and supporting
backing file.

  
 Not sure what you mean here. 
  
 What about a phy snapshot via lvm snapshotting? 
  
   In any case  
   passing in UNKNOWN and letting libxl choose (probably by picking the  
   same as the underlying disk) should be supported. 
   
  If external_format is not passed (NULL), by default, we will use qcow2. 
  
 I think you need to base this on the type of the original disk, if it is 
 e.g. vhd then making a qcow snapshot seems a bit odd. 

Agree. For vhd and lvm which have tools other than 'qemu-img', should be
treated differently. For those creating snapshot by 'qemu-img', I think using
'qcow2' by default is reasonable according to qemu's implementation.

- Chunyan

  
 
/*  This API might not be used by xl, since xl won't take care of  
 deleting  
 *  snapshots. But for libvirt, since libvirt manages snapshots and 
will  
 *  delete snapshot, this API will be used.  
 */  
int libxl_disk_snapshot_delete(libxl_ctx *ctx, uint32_t domid,  
   libxl_disk_snapshot *snapshot, int nb);  
 
   The three usecases I mentioned in the previous mail are important here,  
   because depending on which usecases you are considering there maybe a  
   many to one relationship between domains and a given snapshot (gold  
   image case). This interface cannot support that I think. 
   
  I'm not quite clear about the three usecases, especially the 3rd usercase, 
  so really not sure what's the requirement towards deleting disk snapshot. 
  
 I hope my reply to the previous mail helped clear this up a bit. The 
 reason deleting a disk is interesting is because that is what you would 
 do after the backup was finished. 
  
   When we discussed this in previous iterations I suggested a libxl  
   command to tell a VM that it needed to reexamine its disks to see if any  
   of the chains had changed. I'm sure that's not the only potential answer  
   though. 

  About delete disk snapshot in a snapshot chain, whether we need to do 
  extra work to avoid data break, it can be discussed: 
  a). For external snapshots, usually it's based on backing file chain, qemu 
  does this, vhd-util does this. In this 

Re: [Xen-devel] [RFC V9 4/4] domain snapshot design: libxl/libxlu

2014-12-19 Thread Ian Campbell
On Thu, 2014-12-18 at 23:58 -0700, Chun Yan Liu wrote:
 
  On 12/18/2014 at 11:27 PM, in message 
  1418916436.11882.101.ca...@citrix.com,
 Ian Campbell ian.campb...@citrix.com wrote: 
  On Tue, 2014-12-16 at 14:32 +0800, Chunyan Liu wrote: 
   Changes to V8: 
 * remove libxl_domain_snapshot_create/delete/revert API 
 * export disk snapshot functionality for both xl and libvirt usage 

   ===

   Libxl/libxlu Design 

   1. New Structures 

   libxl_disk_snapshot = Struct(disk_snapshot,[ 
   # target disk 
   (disk,libxl_device_disk), 

   # disk snapshot name 
   (name,string), 

   # internal/external disk snapshot? 
   (external,bool), 

   # for external disk snapshot, specify following two field 
   (external_format, string), 
   (external_path,   string), 
   
  Should this be a KeyedUnion over a new LIBXL_DISK_SNAPSHOT_KIND enum 
  (with values INTERNAL and EXTERNAL)?
 The KeyedUnion seems to be unnecessary. Only EXTERNAL has data items,
 INTERNAL doesn't, and no third types.
 
  This would automatically make the 
  binding between external==true and the fields which depend on that. 
   
  external_format should be of type libxl_disk_format, unless it is 
  referring to something else? 
 
 Yes. That's right. I'll update.
 
   
  Is it possible for format to differ from the format of the underlying 
  disk? Perhaps taking a snapshot of a raw disk as a qcow?
 
 This is related to implementation details. As I understand qemu's
 implementation, taking an external disk snapshot is actually a way:
 origin domain disk: a raw disk
 external= true, external_format: qcow2, external_path: test
 a), create a qcow2 file (test.qcow2) with  backing file (the raw disk)
 b), replace domain disk, now domain uses test.qcow2 (the raw disk
  is actually to be the snapshot)
 
 So, I think the external_format can only be those supporting backing file.

Not sure what you mean here.

What about a phy snapshot via lvm snapshotting?

  In any case 
  passing in UNKNOWN and letting libxl choose (probably by picking the 
  same as the underlying disk) should be supported.
 
 If external_format is not passed (NULL), by default, we will use qcow2.

I think you need to base this on the type of the original disk, if it is
e.g. vhd then making a qcow snapshot seems a bit odd.

 
   
   /*  This API might not be used by xl, since xl won't take care of 
   deleting 
*  snapshots. But for libvirt, since libvirt manages snapshots and will 
*  delete snapshot, this API will be used. 
*/ 
   int libxl_disk_snapshot_delete(libxl_ctx *ctx, uint32_t domid, 
  libxl_disk_snapshot *snapshot, int nb); 
   
  The three usecases I mentioned in the previous mail are important here, 
  because depending on which usecases you are considering there maybe a 
  many to one relationship between domains and a given snapshot (gold 
  image case). This interface cannot support that I think.
 
 I'm not quite clear about the three usecases, especially the 3rd usercase,
 so really not sure what's the requirement towards deleting disk snapshot.

I hope my reply to the previous mail helped clear this up a bit. The
reason deleting a disk is interesting is because that is what you would
do after the backup was finished.

  When we discussed this in previous iterations I suggested a libxl 
  command to tell a VM that it needed to reexamine its disks to see if any 
  of the chains had changed. I'm sure that's not the only potential answer 
  though.
  
 About delete disk snapshot in a snapshot chain, whether we need to do
 extra work to avoid data break, it can be discussed:
 a). For external snapshots, usually it's based on backing file chain, qemu
 does this, vhd-util does this. In this case, to delete a domain snapshot,
 one doesn't need to do anything to disk (no need to delete disk snapshot
 at all). Downside is, there might be a long backing chain.

I'm not sure what you mean here I'm afraid. If you are deleting a domain
snapshot why do you not want to delete the disk snapshots associated
with it?

 b). For internal snapshot, like qcow2, lvm too. For lvm, it doesn't support
 snapshot of snapshot, so out of scope. For qcow2, delete any disk snapshot
 won't affect others.

For either internal or external if you are removing a snapshot from the
middle of a chain which ends in one or more active disks, then surely
the disk backend associated with those domains need to get some sort of
notification, otherwise they would need to be written *very* carefully
in order to be able to cope with disk metadata changing under their
feet.

Are you saying that the qemu/qcow implementation has indeed been written
with this in mind and can cope with arbitrary other processes modifying
the qcow metadata under their feet?

e.g. 
BASE---SNAPSHOT A---SNAPSHOT B 

Re: [Xen-devel] [RFC V9 4/4] domain snapshot design: libxl/libxlu

2014-12-18 Thread Ian Campbell
On Tue, 2014-12-16 at 14:32 +0800, Chunyan Liu wrote:
 Changes to V8:
   * remove libxl_domain_snapshot_create/delete/revert API
   * export disk snapshot functionality for both xl and libvirt usage
 
 ===
 Libxl/libxlu Design
 
 1. New Structures
 
 libxl_disk_snapshot = Struct(disk_snapshot,[
 # target disk
 (disk,libxl_device_disk),
 
 # disk snapshot name
 (name,string),
 
 # internal/external disk snapshot?
 (external,bool),
 
 # for external disk snapshot, specify following two field
 (external_format, string),
 (external_path,   string),

Should this be a KeyedUnion over a new LIBXL_DISK_SNAPSHOT_KIND enum
(with values INTERNAL and EXTERNAL)? This would automatically make the
binding between external==true and the fields which depend on that.

external_format should be of type libxl_disk_format, unless it is
referring to something else?

Is it possible for format to differ from the format of the underlying
disk? Perhaps taking a snapshot of a raw disk as a qcow? In any case
passing in UNKNOWN and letting libxl choose (probably by picking the
same as the underlying disk) should be supported.

 /*  This API might not be used by xl, since xl won't take care of deleting
  *  snapshots. But for libvirt, since libvirt manages snapshots and will
  *  delete snapshot, this API will be used.
  */
 int libxl_disk_snapshot_delete(libxl_ctx *ctx, uint32_t domid,
libxl_disk_snapshot *snapshot, int nb);

The three usecases I mentioned in the previous mail are important here,
because depending on which usecases you are considering there maybe a
many to one relationship between domains and a given snapshot (gold
image case). This interface cannot support that I think.

When we discussed this in previous iterations I suggested a libxl
command to tell a VM that it needed to reexamine its disks to see if any
of the chains had changed. I'm sure that's not the only potential answer
though.

 int libxl_disk_to_snapshot(libxl_ctx *ctx, uint32_t domid,
libxl_disk_snapshot **snapshot, int *num);
 
 This is for domain snapshot create. If user doesn't specify disks,
 then by default it will take internal disk snapshot to each domain
 disk. This function will fill libxl_disk_snapshot according to domain
 disks info.

Is this just a helper to produce an array to pass to
libxl_disk_snapshot_create? Or does it actually do stuff?

I think it's the former, but it could be clarified. I *think* this is
just a special case of libxl_device_disk_list which returns plausible
snapshot objects instead of the disks themselves.
 
 For disk snapshot revert, no qmp command for that, it always calls
 external commands to finish the work, so put in libxlu (?):

I think rather than no qmp the issue is that revert is (at least as
far as libxl knows) essentially, destroy, rollback disks, restore from
RAM snapshot. So there is no qemu to speak to during the rollback. Is
that right?

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [RFC V9 4/4] domain snapshot design: libxl/libxlu

2014-12-18 Thread Chun Yan Liu


 On 12/18/2014 at 11:27 PM, in message 
 1418916436.11882.101.ca...@citrix.com,
Ian Campbell ian.campb...@citrix.com wrote: 
 On Tue, 2014-12-16 at 14:32 +0800, Chunyan Liu wrote: 
  Changes to V8: 
* remove libxl_domain_snapshot_create/delete/revert API 
* export disk snapshot functionality for both xl and libvirt usage 
   
  === 
  Libxl/libxlu Design 
   
  1. New Structures 
   
  libxl_disk_snapshot = Struct(disk_snapshot,[ 
  # target disk 
  (disk,libxl_device_disk), 
   
  # disk snapshot name 
  (name,string), 
   
  # internal/external disk snapshot? 
  (external,bool), 
   
  # for external disk snapshot, specify following two field 
  (external_format, string), 
  (external_path,   string), 
  
 Should this be a KeyedUnion over a new LIBXL_DISK_SNAPSHOT_KIND enum 
 (with values INTERNAL and EXTERNAL)?
The KeyedUnion seems to be unnecessary. Only EXTERNAL has data items,
INTERNAL doesn't, and no third types.

 This would automatically make the 
 binding between external==true and the fields which depend on that. 
  
 external_format should be of type libxl_disk_format, unless it is 
 referring to something else? 

Yes. That's right. I'll update.

  
 Is it possible for format to differ from the format of the underlying 
 disk? Perhaps taking a snapshot of a raw disk as a qcow?

This is related to implementation details. As I understand qemu's
implementation, taking an external disk snapshot is actually a way:
origin domain disk: a raw disk
external= true, external_format: qcow2, external_path: test
a), create a qcow2 file (test.qcow2) with  backing file (the raw disk)
b), replace domain disk, now domain uses test.qcow2 (the raw disk
 is actually to be the snapshot)

So, I think the external_format can only be those supporting backing file.

 In any case 
 passing in UNKNOWN and letting libxl choose (probably by picking the 
 same as the underlying disk) should be supported.

If external_format is not passed (NULL), by default, we will use qcow2.

  
  /*  This API might not be used by xl, since xl won't take care of deleting 
   *  snapshots. But for libvirt, since libvirt manages snapshots and will 
   *  delete snapshot, this API will be used. 
   */ 
  int libxl_disk_snapshot_delete(libxl_ctx *ctx, uint32_t domid, 
 libxl_disk_snapshot *snapshot, int nb); 
  
 The three usecases I mentioned in the previous mail are important here, 
 because depending on which usecases you are considering there maybe a 
 many to one relationship between domains and a given snapshot (gold 
 image case). This interface cannot support that I think.

I'm not quite clear about the three usecases, especially the 3rd usercase,
so really not sure what's the requirement towards deleting disk snapshot.
 
  
 When we discussed this in previous iterations I suggested a libxl 
 command to tell a VM that it needed to reexamine its disks to see if any 
 of the chains had changed. I'm sure that's not the only potential answer 
 though.
 
About delete disk snapshot in a snapshot chain, whether we need to do
extra work to avoid data break, it can be discussed:
a). For external snapshots, usually it's based on backing file chain, qemu
does this, vhd-util does this. In this case, to delete a domain snapshot,
one doesn't need to do anything to disk (no need to delete disk snapshot
at all). Downside is, there might be a long backing chain.
b). For internal snapshot, like qcow2, lvm too. For lvm, it doesn't support
snapshot of snapshot, so out of scope. For qcow2, delete any disk snapshot
won't affect others.

  
  int libxl_disk_to_snapshot(libxl_ctx *ctx, uint32_t domid, 
 libxl_disk_snapshot **snapshot, int *num); 
   
  This is for domain snapshot create. If user doesn't specify disks, 
  then by default it will take internal disk snapshot to each domain 
  disk. This function will fill libxl_disk_snapshot according to domain 
  disks info. 
  
 Is this just a helper to produce an array to pass to 
 libxl_disk_snapshot_create? Or does it actually do stuff? 
  
 I think it's the former, but it could be clarified.

Yes, the former.

 I *think* this is 
 just a special case of libxl_device_disk_list which returns plausible 
 snapshot objects instead of the disks themselves. 

So we prefer adding codes to libxl_device_disk_list rather than adding
a new API, right?

   
  For disk snapshot revert, no qmp command for that, it always calls 
  external commands to finish the work, so put in libxlu (?): 
  
 I think rather than no qmp the issue is that revert is (at least as 
 far as libxl knows) essentially, destroy, rollback disks, restore from 
 RAM snapshot. So there is no qemu to speak to during the rollback. Is 
 that right? 

Yes, that's right.

  
 Ian. 
  
  
  



Re: [Xen-devel] [RFC V9 4/4] domain snapshot design: libxl/libxlu

2014-12-17 Thread Wei Liu
On Tue, Dec 16, 2014 at 02:32:57PM +0800, Chunyan Liu wrote:
 Changes to V8:
   * remove libxl_domain_snapshot_create/delete/revert API
   * export disk snapshot functionality for both xl and libvirt usage
 
 ===
 Libxl/libxlu Design
 
 1. New Structures
 
 libxl_disk_snapshot = Struct(disk_snapshot,[
 # target disk
 (disk,libxl_device_disk),
 
 # disk snapshot name
 (name,string),
 
 # internal/external disk snapshot?
 (external,bool),
 
 # for external disk snapshot, specify following two field
 (external_format, string),
 (external_path,   string),
 ])
 

So you don't propose making libxl to have knowledge of the snapshot
chains? And in libvirt (or other toolstack that's interested in snapshot
management) you represent a snapshot as chains (or trees) of
libxl_disk_snapshot? (Not suggesting you do things the other way around,
just to confirm)

 
 2. New Functions
 
 Since there're already APIs for saving memory (libxl_domain_suspend)
 and restoring domain from saved memory (libxl_domain_create_restore), to
 xl domain snapshot tasks, the missing part is disk snapshot functionality.
 And the disk snapshot functionality would be used by libvirt too.
 
 Considering there is qmp handling in creating/deleting disk snapshot,
 will add following new functions to libxl (?):
 
 int libxl_disk_snapshot_create(libxl_ctx *ctx, uint32_t domid,
libxl_disk_snapshot *snapshot, int nb);
 
 Taking disk snapshots to a group of domain disks according to
 configuration. For qcow2 disk backend type, it will call qmp
 transaction command to do the work. For other disk backend types,
 might call other external commands.
 
 Parameters:
ctx (INPUT):
context
domid (INPUT):
domain id
snapshot (INPUT):
array of disk snapshot configuration. Has nb members.
 
libxl_device_disk:
structure to represent which disk.
name:
snapshot name.
external:
internal snapshot or external snapshot.
'false' means internal disk snapshot. external_format and
external_path will be ignored.
'true' means external disk snapshot, then external_format
and external_path should be provided.
external_format:
Should be provided when 'external' is true. If not provided,
will use default format proper to the backend file.
Ignored when 'external' is false.
external_path:
Must be provided when 'external' is true.
Ignored when 'external' is false.
nb (INPUT):
number of disks that need to take disk snapshot.
 
 Return:
0 on success, -1 on error.
 

It should return appropriate libxl error code (ERROR_*) on error.

 
 /*  This API might not be used by xl, since xl won't take care of deleting
  *  snapshots. But for libvirt, since libvirt manages snapshots and will
  *  delete snapshot, this API will be used.
  */
 int libxl_disk_snapshot_delete(libxl_ctx *ctx, uint32_t domid,
libxl_disk_snapshot *snapshot, int nb);
 
 Delete disk snapshot of a group of domain disks according to
 configuration. For qcow2 disk backend type, it will call qmp command
 to delete internal disk snapshot. For other disk backend types, might
 call other external commands.
 
 Parameters:
ctx (INPUT):
context
domid (INPUT):
domain id
snapshot (INPUT):
array of disk snapshot configuration. Has nb members.
nb (INPUT):
number of disks that need to take disk snapshot.
 
 Return:
0 on success, -1 on error.
 
 
 int libxl_disk_to_snapshot(libxl_ctx *ctx, uint32_t domid,
libxl_disk_snapshot **snapshot, int *num);
 
 This is for domain snapshot create. If user doesn't specify disks,
 then by default it will take internal disk snapshot to each domain
 disk. This function will fill libxl_disk_snapshot according to domain
 disks info.
 
 Parameters:
ctx (INPUT):
context
domid (INPUT):
domain id
snapshot (OUTPUT):
array of disk snapshot configuration.
num (OUTPUT):
number of disks.
 
 Return:
0 on success, -1 on error.
 
 
 
 For disk snapshot revert, no qmp command for that, it always calls
 external commands to finish the work, so put in libxlu (?):
 
 int xlu_disk_snapshot_revert(libxl_disk_snapshot *snapshot, int nb);
 

IMHO it's fine for this to be in libxl. Calling out to other programs
is fine.

Wei.

 Apply disk snapshot for a group of disks according to configuration. To
 different