Re: [libvirt] [PATCH v3 00/20] Incremental Backup API additions
On 10/25/2018 03:20 PM, Eric Blake wrote: > The following is the latest version of my API proposal for > incremental backups, and matches the demo I will be presenting > as part of my KVM Forum 2018 talk tomorrow afternoon: > https://kvmforum2018.sched.com/event/FzuB/facilitating-incremental-backup-eric-blake-red-hat > > The patches are also available via https://repo.or.cz/libvirt/ericb.git > at the tag backup-v3. > > The qemu integration is still incomplete (I don't have push > mode backups working, only pull mode; and there are still some > things to fine-tune as I figure out how to better support > external snapshots and block commit/stream), and I still haven't > gotten any opinions on whether it is better to have: > > virDomainSnapshotCreateXML("...") # existing > virDomainBackupBegin("...", "...") # this > series > virDomainSnapshotCheckpointCreateXML("...", > "...") # new > > or to make checkpoint creation part of the snapshot and backup XML, as in: > > virDomainSnapshotCreateXML("...") > # extension of existing > virDomainBackupBegin("...") # > tweak to this series > > It's also obvious that we don't want to check in the qemu implementation > until after Peter's blockdev work has landed AND qemu has gotten rid > of the x- prefix in its QMP interactions, although the state of this > series SHOULD be sufficient to prove that the proposed interface is > at least flexible enough to adapt to whatever qemu tweaks land and > whatever other hypervisors may support. For what it's worth, the x prefixes for bitmap manipulation were only because there was no user for those features; if the overall design of this API proposal looks palatable to the libvirt community, I'll happily drop the prefix for any commands actually used by this patchset, before it's merged. (From the looks of it, it looks like all of them.) --js > Whether to accept the API now> (to make backporting qemu support downstream > across versions without > rebasing to an even later date for the API) is a harder question; > Jirka has already expressed distaste at the idea. > > There are probably still loads of things needing fixing in this version, > I already know I did not address all of Peter's review comments on v2. > My unit test coverage is not very complete on the XML parsing, and there > are definitely some hacks in place to get things to the point of a working > demo that need to be polished for robustness. > > Below is a diff since the v2 posting (patches 1-10 here, for later > patches a comparison against the state of my tree at the time I posted v2): > https://www.redhat.com/archives/libvir-list/2018-October/msg00700.html > > 001/20:[] [--] 'snapshots: Avoid term 'checkpoint' for full system > snapshot' > 002/20:[] [--] 'domain_conf: Expose virDomainStorageNetworkParseHost' > 003/20:[down] 'qemu: Allow optional export name during NBD export' > 004/20:[] [--] 'backup: Document nuances between different state capture > APIs' > 005/20:[] [--] 'backup: Introduce virDomainCheckpointPtr' > 006/20:[0012] [FC] 'backup: Document new XML for backups' > 007/20:[] [--] 'backup: Introduce virDomainCheckpoint APIs' > 008/20:[0024] [FC] 'backup: Introduce virDomainBackup APIs' > 009/20:[] [--] 'backup: Add new domain:checkpoint access control' > 010/20:[0001] [FC] 'backup: Implement backup APIs for remote driver' > 011/20:[0007] [FC] 'wip: backup: Parse and output checkpoint XML' > 012/20:[0199] [FC] 'wip: backup: Parse and output backup XML' > 013/20:[] [--] 'backup: Implement virsh support for checkpoints' > 014/20:[] [--] 'wip: backup: virsh support for backup' > 015/20:[0223] [FC] 'backup: Add new qemu monitor interactions' > 016/20:[0123] [FC] 'backup: qemu: Implement metadata tracking for checkpoint > APIs' > 017/20:[down] 'wip: backup: Wire up qemu checkpoint commands over QMP' > 018/20:[0072] [FC] 'wip: backup: qemu: Implement framework for backup job > APIs' > 019/20:[down] 'backup: Wire up qemu full pull backup commands over QMP' > 020/20:[down] 'backup: implement qemu incremental pull backup' > > No direct dependency on this one, but included in my git tag: > https://www.redhat.com/archives/libvir-list/2018-October/msg00899.html > > Eric Blake (20): > snapshots: Avoid term 'checkpoint' for full system snapshot > domain_conf: Expose virDomainStorageNetworkParseHost > qemu: Allow optional export name during NBD export > backup: Document nuances between different state capture APIs > backup: Introduce virDomainCheckpointPtr > backup: Document new XML for backups > backup: Introduce virDomainCheckpoint APIs > backup: Introduce virDomainBackup APIs > backup: Add new domain:checkpoint access control > backup: Implement backup APIs for remote driver > wip: backup: Parse and output checkpoint XML > wip: backup: Parse and output backup XML > backup: Implement virsh support for checkpoints > wip: backup: virsh support for backup > backup: Add new qemu
[libvirt] [PATCH v3 00/20] Incremental Backup API additions
The following is the latest version of my API proposal for incremental backups, and matches the demo I will be presenting as part of my KVM Forum 2018 talk tomorrow afternoon: https://kvmforum2018.sched.com/event/FzuB/facilitating-incremental-backup-eric-blake-red-hat The patches are also available via https://repo.or.cz/libvirt/ericb.git at the tag backup-v3. The qemu integration is still incomplete (I don't have push mode backups working, only pull mode; and there are still some things to fine-tune as I figure out how to better support external snapshots and block commit/stream), and I still haven't gotten any opinions on whether it is better to have: virDomainSnapshotCreateXML("...") # existing virDomainBackupBegin("...", "...") # this series virDomainSnapshotCheckpointCreateXML("...", "...") # new or to make checkpoint creation part of the snapshot and backup XML, as in: virDomainSnapshotCreateXML("...") # extension of existing virDomainBackupBegin("...") # tweak to this series It's also obvious that we don't want to check in the qemu implementation until after Peter's blockdev work has landed AND qemu has gotten rid of the x- prefix in its QMP interactions, although the state of this series SHOULD be sufficient to prove that the proposed interface is at least flexible enough to adapt to whatever qemu tweaks land and whatever other hypervisors may support. Whether to accept the API now (to make backporting qemu support downstream across versions without rebasing to an even later date for the API) is a harder question; Jirka has already expressed distaste at the idea. There are probably still loads of things needing fixing in this version, I already know I did not address all of Peter's review comments on v2. My unit test coverage is not very complete on the XML parsing, and there are definitely some hacks in place to get things to the point of a working demo that need to be polished for robustness. Below is a diff since the v2 posting (patches 1-10 here, for later patches a comparison against the state of my tree at the time I posted v2): https://www.redhat.com/archives/libvir-list/2018-October/msg00700.html 001/20:[] [--] 'snapshots: Avoid term 'checkpoint' for full system snapshot' 002/20:[] [--] 'domain_conf: Expose virDomainStorageNetworkParseHost' 003/20:[down] 'qemu: Allow optional export name during NBD export' 004/20:[] [--] 'backup: Document nuances between different state capture APIs' 005/20:[] [--] 'backup: Introduce virDomainCheckpointPtr' 006/20:[0012] [FC] 'backup: Document new XML for backups' 007/20:[] [--] 'backup: Introduce virDomainCheckpoint APIs' 008/20:[0024] [FC] 'backup: Introduce virDomainBackup APIs' 009/20:[] [--] 'backup: Add new domain:checkpoint access control' 010/20:[0001] [FC] 'backup: Implement backup APIs for remote driver' 011/20:[0007] [FC] 'wip: backup: Parse and output checkpoint XML' 012/20:[0199] [FC] 'wip: backup: Parse and output backup XML' 013/20:[] [--] 'backup: Implement virsh support for checkpoints' 014/20:[] [--] 'wip: backup: virsh support for backup' 015/20:[0223] [FC] 'backup: Add new qemu monitor interactions' 016/20:[0123] [FC] 'backup: qemu: Implement metadata tracking for checkpoint APIs' 017/20:[down] 'wip: backup: Wire up qemu checkpoint commands over QMP' 018/20:[0072] [FC] 'wip: backup: qemu: Implement framework for backup job APIs' 019/20:[down] 'backup: Wire up qemu full pull backup commands over QMP' 020/20:[down] 'backup: implement qemu incremental pull backup' No direct dependency on this one, but included in my git tag: https://www.redhat.com/archives/libvir-list/2018-October/msg00899.html Eric Blake (20): snapshots: Avoid term 'checkpoint' for full system snapshot domain_conf: Expose virDomainStorageNetworkParseHost qemu: Allow optional export name during NBD export backup: Document nuances between different state capture APIs backup: Introduce virDomainCheckpointPtr backup: Document new XML for backups backup: Introduce virDomainCheckpoint APIs backup: Introduce virDomainBackup APIs backup: Add new domain:checkpoint access control backup: Implement backup APIs for remote driver wip: backup: Parse and output checkpoint XML wip: backup: Parse and output backup XML backup: Implement virsh support for checkpoints wip: backup: virsh support for backup backup: Add new qemu monitor interactions backup: qemu: Implement metadata tracking for checkpoint APIs wip: backup: Wire up qemu checkpoint commands over QMP wip: backup: qemu: Implement framework for backup job APIs backup: Wire up qemu full pull backup commands over QMP backup: implement qemu incremental pull backup include/libvirt/virterror.h |6 +- src/util/virerror.c | 21 +- include/libvirt/libvirt-domain-checkpoint.h | 177 ++ include/libvirt/libvirt-domain-snapshot.h|2 +- include/libvirt/libvirt-domain.h | 17 +-