Re: [Qemu-devel] [Patch v12 00/10] Block replication for continuous checkpoints
On 12/02/2015 09:00 AM, Wen Congyang wrote: > On 12/01/2015 06:40 PM, Dr. David Alan Gilbert wrote: >> * Wen Congyang (we...@cn.fujitsu.com) wrote: >>> Block replication is a very important feature which is used for >>> continuous checkpoints(for example: COLO). >>> >>> You can get the detailed information about block replication from here: >>> http://wiki.qemu.org/Features/BlockReplication >>> >>> Usage: >>> Please refer to docs/block-replication.txt >>> >>> This patch series is based on the following patch series: >>> 1. http://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg04949.html >>> 2. http://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg06043.html >>> >>> You can get the patch here: >>> https://github.com/coloft/qemu/tree/wency/block-replication-v12 >>> >>> You can get the patch with framework here: >>> https://github.com/coloft/qemu/tree/wency/colo_framework_v11.2 >> >> Neither of these links work for me, and I see that only messages 0..7 in the >> series hit the list. > > I forgot to push it to github... > And I also received the messages 0..7, and I don't know what's wrong... The reason is that: git send-email has a bug: http://permalink.gmane.org/gmane.comp.version-control.git/274569 Thanks Wen Congyang > > I will push it to github, and resend them. > > Thanks > Wen Congyang > >> >> Dave >> >>> >>> TODO: >>> 1. Continuous block replication. It will be started after basic functions >>>are accepted. >>> >>> Changs Log: >>> V12: >>> 1. Rebase to the newest codes >>> 2. Use backing reference to replcace 'allow-write-backing-file' >>> V11: >>> 1. Reopen the backing file when starting blcok replication if it is not >>>opened in R/W mode >>> 2. Unblock BLOCK_OP_TYPE_BACKUP_SOURCE and BLOCK_OP_TYPE_BACKUP_TARGET >>>when opening backing file >>> 3. Block the top BDS so there is only one block job for the top BDS and >>>its backing chain. >>> V10: >>> 1. Use blockdev-remove-medium and blockdev-insert-medium to replace backing >>>reference. >>> 2. Address the comments from Eric Blake >>> V9: >>> 1. Update the error messages >>> 2. Rebase to the newest qemu >>> 3. Split child add/delete support. These patches are sent in another >>> patchset. >>> V8: >>> 1. Address Alberto Garcia's comments >>> V7: >>> 1. Implement adding/removing quorum child. Remove the option non-connect. >>> 2. Simplify the backing refrence option according to Stefan Hajnoczi's >>> suggestion >>> V6: >>> 1. Rebase to the newest qemu. >>> V5: >>> 1. Address the comments from Gong Lei >>> 2. Speed the failover up. The secondary vm can take over very quickly even >>>if there are too many I/O requests. >>> V4: >>> 1. Introduce a new driver replication to avoid touch nbd and qcow2. >>> V3: >>> 1: use error_setg() instead of error_set() >>> 2. Add a new block job API >>> 3. Active disk, hidden disk and nbd target uses the same AioContext >>> 4. Add a testcase to test new hbitmap API >>> V2: >>> 1. Redesign the secondary qemu(use image-fleecing) >>> 2. Use Error objects to return error message >>> 3. Address the comments from Max Reitz and Eric Blake >>> >>> Wen Congyang (10): >>> unblock backup operations in backing file >>> Store parent BDS in BdrvChild >>> Backup: clear all bitmap when doing block checkpoint >>> Allow creating backup jobs when opening BDS >>> docs: block replication's description >>> Add new block driver interfaces to control block replication >>> quorum: implement block driver interfaces for block replication >>> Implement new driver for block replication >>> support replication driver in blockdev-add >>> Add a new API to start/stop replication, do checkpoint to all BDSes >>> >>> block.c| 145 >>> block/Makefile.objs| 3 +- >>> block/backup.c | 14 ++ >>> block/quorum.c | 78 +++ >>> block/replication.c| 549 >>> + >>> blockjob.c | 11 + >>> docs/block-replication.txt | 227 +++ >>> include/block/block.h | 9 + >>> include/block/block_int.h | 15 ++ >>> include/block/blockjob.h | 12 + >>> qapi/block-core.json | 34 ++- >>> 11 files changed, 1093 insertions(+), 4 deletions(-) >>> create mode 100644 block/replication.c >>> create mode 100644 docs/block-replication.txt >>> >>> -- >>> 2.5.0 >>> >>> >>> >> -- >> Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK >> >> >> . >> > > > > > . >
Re: [Qemu-devel] [Patch v12 00/10] Block replication for continuous checkpoints
On 12/01/2015 07:58 PM, Hailiang Zhang wrote: > On 2015/12/1 18:40, Dr. David Alan Gilbert wrote: >> * Wen Congyang (we...@cn.fujitsu.com) wrote: >>> Block replication is a very important feature which is used for >>> continuous checkpoints(for example: COLO). >>> >>> You can get the detailed information about block replication from here: >>> http://wiki.qemu.org/Features/BlockReplication >>> >>> Usage: >>> Please refer to docs/block-replication.txt >>> >>> This patch series is based on the following patch series: >>> 1. http://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg04949.html >>> 2. http://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg06043.html >>> >>> You can get the patch here: >>> https://github.com/coloft/qemu/tree/wency/block-replication-v12 >>> >>> You can get the patch with framework here: >>> https://github.com/coloft/qemu/tree/wency/colo_framework_v11.2 >> >> Neither of these links work for me, and I see that only messages 0..7 in the >> series hit the list. >> > > Hi Dave, > > You can refer to https://github.com/coloft/qemu/tree/colo-v2.2-periodic-mode, > The block replication part in this link is also the newest version. No, I remove one patch, and the usage is changed. Thanks Wen Congyang > > Congyang has deleted this confused branch, we will pay attention to this > later in next version. > > Thanks, > Hailiang > >> >>> >>> TODO: >>> 1. Continuous block replication. It will be started after basic functions >>> are accepted. >>> >>> Changs Log: >>> V12: >>> 1. Rebase to the newest codes >>> 2. Use backing reference to replcace 'allow-write-backing-file' >>> V11: >>> 1. Reopen the backing file when starting blcok replication if it is not >>> opened in R/W mode >>> 2. Unblock BLOCK_OP_TYPE_BACKUP_SOURCE and BLOCK_OP_TYPE_BACKUP_TARGET >>> when opening backing file >>> 3. Block the top BDS so there is only one block job for the top BDS and >>> its backing chain. >>> V10: >>> 1. Use blockdev-remove-medium and blockdev-insert-medium to replace backing >>> reference. >>> 2. Address the comments from Eric Blake >>> V9: >>> 1. Update the error messages >>> 2. Rebase to the newest qemu >>> 3. Split child add/delete support. These patches are sent in another >>> patchset. >>> V8: >>> 1. Address Alberto Garcia's comments >>> V7: >>> 1. Implement adding/removing quorum child. Remove the option non-connect. >>> 2. Simplify the backing refrence option according to Stefan Hajnoczi's >>> suggestion >>> V6: >>> 1. Rebase to the newest qemu. >>> V5: >>> 1. Address the comments from Gong Lei >>> 2. Speed the failover up. The secondary vm can take over very quickly even >>> if there are too many I/O requests. >>> V4: >>> 1. Introduce a new driver replication to avoid touch nbd and qcow2. >>> V3: >>> 1: use error_setg() instead of error_set() >>> 2. Add a new block job API >>> 3. Active disk, hidden disk and nbd target uses the same AioContext >>> 4. Add a testcase to test new hbitmap API >>> V2: >>> 1. Redesign the secondary qemu(use image-fleecing) >>> 2. Use Error objects to return error message >>> 3. Address the comments from Max Reitz and Eric Blake >>> >>> Wen Congyang (10): >>>unblock backup operations in backing file >>>Store parent BDS in BdrvChild >>>Backup: clear all bitmap when doing block checkpoint >>>Allow creating backup jobs when opening BDS >>>docs: block replication's description >>>Add new block driver interfaces to control block replication >>>quorum: implement block driver interfaces for block replication >>>Implement new driver for block replication >>>support replication driver in blockdev-add >>>Add a new API to start/stop replication, do checkpoint to all BDSes >>> >>> block.c| 145 >>> block/Makefile.objs| 3 +- >>> block/backup.c | 14 ++ >>> block/quorum.c | 78 +++ >>> block/replication.c| 549 >>> + >>> blockjob.c | 11 + >>> docs/block-replication.txt | 227 +++ >>> include/block/block.h | 9 + >>> include/block/block_int.h | 15 ++ >>> include/block/blockjob.h | 12 + >>> qapi/block-core.json | 34 ++- >>> 11 files changed, 1093 insertions(+), 4 deletions(-) >>> create mode 100644 block/replication.c >>> create mode 100644 docs/block-replication.txt >>> >>> -- >>> 2.5.0 >>> >>> >>> >> -- >> Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK >> >> >> . >> > > > > > . >
Re: [Qemu-devel] [Patch v12 00/10] Block replication for continuous checkpoints
On 12/01/2015 06:40 PM, Dr. David Alan Gilbert wrote: > * Wen Congyang (we...@cn.fujitsu.com) wrote: >> Block replication is a very important feature which is used for >> continuous checkpoints(for example: COLO). >> >> You can get the detailed information about block replication from here: >> http://wiki.qemu.org/Features/BlockReplication >> >> Usage: >> Please refer to docs/block-replication.txt >> >> This patch series is based on the following patch series: >> 1. http://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg04949.html >> 2. http://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg06043.html >> >> You can get the patch here: >> https://github.com/coloft/qemu/tree/wency/block-replication-v12 >> >> You can get the patch with framework here: >> https://github.com/coloft/qemu/tree/wency/colo_framework_v11.2 > > Neither of these links work for me, and I see that only messages 0..7 in the > series hit the list. I forgot to push it to github... And I also received the messages 0..7, and I don't know what's wrong... I will push it to github, and resend them. Thanks Wen Congyang > > Dave > >> >> TODO: >> 1. Continuous block replication. It will be started after basic functions >>are accepted. >> >> Changs Log: >> V12: >> 1. Rebase to the newest codes >> 2. Use backing reference to replcace 'allow-write-backing-file' >> V11: >> 1. Reopen the backing file when starting blcok replication if it is not >>opened in R/W mode >> 2. Unblock BLOCK_OP_TYPE_BACKUP_SOURCE and BLOCK_OP_TYPE_BACKUP_TARGET >>when opening backing file >> 3. Block the top BDS so there is only one block job for the top BDS and >>its backing chain. >> V10: >> 1. Use blockdev-remove-medium and blockdev-insert-medium to replace backing >>reference. >> 2. Address the comments from Eric Blake >> V9: >> 1. Update the error messages >> 2. Rebase to the newest qemu >> 3. Split child add/delete support. These patches are sent in another >> patchset. >> V8: >> 1. Address Alberto Garcia's comments >> V7: >> 1. Implement adding/removing quorum child. Remove the option non-connect. >> 2. Simplify the backing refrence option according to Stefan Hajnoczi's >> suggestion >> V6: >> 1. Rebase to the newest qemu. >> V5: >> 1. Address the comments from Gong Lei >> 2. Speed the failover up. The secondary vm can take over very quickly even >>if there are too many I/O requests. >> V4: >> 1. Introduce a new driver replication to avoid touch nbd and qcow2. >> V3: >> 1: use error_setg() instead of error_set() >> 2. Add a new block job API >> 3. Active disk, hidden disk and nbd target uses the same AioContext >> 4. Add a testcase to test new hbitmap API >> V2: >> 1. Redesign the secondary qemu(use image-fleecing) >> 2. Use Error objects to return error message >> 3. Address the comments from Max Reitz and Eric Blake >> >> Wen Congyang (10): >> unblock backup operations in backing file >> Store parent BDS in BdrvChild >> Backup: clear all bitmap when doing block checkpoint >> Allow creating backup jobs when opening BDS >> docs: block replication's description >> Add new block driver interfaces to control block replication >> quorum: implement block driver interfaces for block replication >> Implement new driver for block replication >> support replication driver in blockdev-add >> Add a new API to start/stop replication, do checkpoint to all BDSes >> >> block.c| 145 >> block/Makefile.objs| 3 +- >> block/backup.c | 14 ++ >> block/quorum.c | 78 +++ >> block/replication.c| 549 >> + >> blockjob.c | 11 + >> docs/block-replication.txt | 227 +++ >> include/block/block.h | 9 + >> include/block/block_int.h | 15 ++ >> include/block/blockjob.h | 12 + >> qapi/block-core.json | 34 ++- >> 11 files changed, 1093 insertions(+), 4 deletions(-) >> create mode 100644 block/replication.c >> create mode 100644 docs/block-replication.txt >> >> -- >> 2.5.0 >> >> >> > -- > Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK > > > . >
Re: [Qemu-devel] [Patch v12 00/10] Block replication for continuous checkpoints
On 2015/12/1 18:40, Dr. David Alan Gilbert wrote: * Wen Congyang (we...@cn.fujitsu.com) wrote: Block replication is a very important feature which is used for continuous checkpoints(for example: COLO). You can get the detailed information about block replication from here: http://wiki.qemu.org/Features/BlockReplication Usage: Please refer to docs/block-replication.txt This patch series is based on the following patch series: 1. http://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg04949.html 2. http://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg06043.html You can get the patch here: https://github.com/coloft/qemu/tree/wency/block-replication-v12 You can get the patch with framework here: https://github.com/coloft/qemu/tree/wency/colo_framework_v11.2 Neither of these links work for me, and I see that only messages 0..7 in the series hit the list. Hi Dave, You can refer to https://github.com/coloft/qemu/tree/colo-v2.2-periodic-mode, The block replication part in this link is also the newest version. Congyang has deleted this confused branch, we will pay attention to this later in next version. Thanks, Hailiang TODO: 1. Continuous block replication. It will be started after basic functions are accepted. Changs Log: V12: 1. Rebase to the newest codes 2. Use backing reference to replcace 'allow-write-backing-file' V11: 1. Reopen the backing file when starting blcok replication if it is not opened in R/W mode 2. Unblock BLOCK_OP_TYPE_BACKUP_SOURCE and BLOCK_OP_TYPE_BACKUP_TARGET when opening backing file 3. Block the top BDS so there is only one block job for the top BDS and its backing chain. V10: 1. Use blockdev-remove-medium and blockdev-insert-medium to replace backing reference. 2. Address the comments from Eric Blake V9: 1. Update the error messages 2. Rebase to the newest qemu 3. Split child add/delete support. These patches are sent in another patchset. V8: 1. Address Alberto Garcia's comments V7: 1. Implement adding/removing quorum child. Remove the option non-connect. 2. Simplify the backing refrence option according to Stefan Hajnoczi's suggestion V6: 1. Rebase to the newest qemu. V5: 1. Address the comments from Gong Lei 2. Speed the failover up. The secondary vm can take over very quickly even if there are too many I/O requests. V4: 1. Introduce a new driver replication to avoid touch nbd and qcow2. V3: 1: use error_setg() instead of error_set() 2. Add a new block job API 3. Active disk, hidden disk and nbd target uses the same AioContext 4. Add a testcase to test new hbitmap API V2: 1. Redesign the secondary qemu(use image-fleecing) 2. Use Error objects to return error message 3. Address the comments from Max Reitz and Eric Blake Wen Congyang (10): unblock backup operations in backing file Store parent BDS in BdrvChild Backup: clear all bitmap when doing block checkpoint Allow creating backup jobs when opening BDS docs: block replication's description Add new block driver interfaces to control block replication quorum: implement block driver interfaces for block replication Implement new driver for block replication support replication driver in blockdev-add Add a new API to start/stop replication, do checkpoint to all BDSes block.c| 145 block/Makefile.objs| 3 +- block/backup.c | 14 ++ block/quorum.c | 78 +++ block/replication.c| 549 + blockjob.c | 11 + docs/block-replication.txt | 227 +++ include/block/block.h | 9 + include/block/block_int.h | 15 ++ include/block/blockjob.h | 12 + qapi/block-core.json | 34 ++- 11 files changed, 1093 insertions(+), 4 deletions(-) create mode 100644 block/replication.c create mode 100644 docs/block-replication.txt -- 2.5.0 -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK .
Re: [Qemu-devel] [Patch v12 00/10] Block replication for continuous checkpoints
* Wen Congyang (we...@cn.fujitsu.com) wrote: > Block replication is a very important feature which is used for > continuous checkpoints(for example: COLO). > > You can get the detailed information about block replication from here: > http://wiki.qemu.org/Features/BlockReplication > > Usage: > Please refer to docs/block-replication.txt > > This patch series is based on the following patch series: > 1. http://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg04949.html > 2. http://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg06043.html > > You can get the patch here: > https://github.com/coloft/qemu/tree/wency/block-replication-v12 > > You can get the patch with framework here: > https://github.com/coloft/qemu/tree/wency/colo_framework_v11.2 Neither of these links work for me, and I see that only messages 0..7 in the series hit the list. Dave > > TODO: > 1. Continuous block replication. It will be started after basic functions >are accepted. > > Changs Log: > V12: > 1. Rebase to the newest codes > 2. Use backing reference to replcace 'allow-write-backing-file' > V11: > 1. Reopen the backing file when starting blcok replication if it is not >opened in R/W mode > 2. Unblock BLOCK_OP_TYPE_BACKUP_SOURCE and BLOCK_OP_TYPE_BACKUP_TARGET >when opening backing file > 3. Block the top BDS so there is only one block job for the top BDS and >its backing chain. > V10: > 1. Use blockdev-remove-medium and blockdev-insert-medium to replace backing >reference. > 2. Address the comments from Eric Blake > V9: > 1. Update the error messages > 2. Rebase to the newest qemu > 3. Split child add/delete support. These patches are sent in another patchset. > V8: > 1. Address Alberto Garcia's comments > V7: > 1. Implement adding/removing quorum child. Remove the option non-connect. > 2. Simplify the backing refrence option according to Stefan Hajnoczi's > suggestion > V6: > 1. Rebase to the newest qemu. > V5: > 1. Address the comments from Gong Lei > 2. Speed the failover up. The secondary vm can take over very quickly even >if there are too many I/O requests. > V4: > 1. Introduce a new driver replication to avoid touch nbd and qcow2. > V3: > 1: use error_setg() instead of error_set() > 2. Add a new block job API > 3. Active disk, hidden disk and nbd target uses the same AioContext > 4. Add a testcase to test new hbitmap API > V2: > 1. Redesign the secondary qemu(use image-fleecing) > 2. Use Error objects to return error message > 3. Address the comments from Max Reitz and Eric Blake > > Wen Congyang (10): > unblock backup operations in backing file > Store parent BDS in BdrvChild > Backup: clear all bitmap when doing block checkpoint > Allow creating backup jobs when opening BDS > docs: block replication's description > Add new block driver interfaces to control block replication > quorum: implement block driver interfaces for block replication > Implement new driver for block replication > support replication driver in blockdev-add > Add a new API to start/stop replication, do checkpoint to all BDSes > > block.c| 145 > block/Makefile.objs| 3 +- > block/backup.c | 14 ++ > block/quorum.c | 78 +++ > block/replication.c| 549 > + > blockjob.c | 11 + > docs/block-replication.txt | 227 +++ > include/block/block.h | 9 + > include/block/block_int.h | 15 ++ > include/block/blockjob.h | 12 + > qapi/block-core.json | 34 ++- > 11 files changed, 1093 insertions(+), 4 deletions(-) > create mode 100644 block/replication.c > create mode 100644 docs/block-replication.txt > > -- > 2.5.0 > > > -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
[Qemu-devel] [Patch v12 00/10] Block replication for continuous checkpoints
Block replication is a very important feature which is used for continuous checkpoints(for example: COLO). You can get the detailed information about block replication from here: http://wiki.qemu.org/Features/BlockReplication Usage: Please refer to docs/block-replication.txt This patch series is based on the following patch series: 1. http://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg04949.html 2. http://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg06043.html You can get the patch here: https://github.com/coloft/qemu/tree/wency/block-replication-v12 You can get the patch with framework here: https://github.com/coloft/qemu/tree/wency/colo_framework_v11.2 TODO: 1. Continuous block replication. It will be started after basic functions are accepted. Changs Log: V12: 1. Rebase to the newest codes 2. Use backing reference to replcace 'allow-write-backing-file' V11: 1. Reopen the backing file when starting blcok replication if it is not opened in R/W mode 2. Unblock BLOCK_OP_TYPE_BACKUP_SOURCE and BLOCK_OP_TYPE_BACKUP_TARGET when opening backing file 3. Block the top BDS so there is only one block job for the top BDS and its backing chain. V10: 1. Use blockdev-remove-medium and blockdev-insert-medium to replace backing reference. 2. Address the comments from Eric Blake V9: 1. Update the error messages 2. Rebase to the newest qemu 3. Split child add/delete support. These patches are sent in another patchset. V8: 1. Address Alberto Garcia's comments V7: 1. Implement adding/removing quorum child. Remove the option non-connect. 2. Simplify the backing refrence option according to Stefan Hajnoczi's suggestion V6: 1. Rebase to the newest qemu. V5: 1. Address the comments from Gong Lei 2. Speed the failover up. The secondary vm can take over very quickly even if there are too many I/O requests. V4: 1. Introduce a new driver replication to avoid touch nbd and qcow2. V3: 1: use error_setg() instead of error_set() 2. Add a new block job API 3. Active disk, hidden disk and nbd target uses the same AioContext 4. Add a testcase to test new hbitmap API V2: 1. Redesign the secondary qemu(use image-fleecing) 2. Use Error objects to return error message 3. Address the comments from Max Reitz and Eric Blake Wen Congyang (10): unblock backup operations in backing file Store parent BDS in BdrvChild Backup: clear all bitmap when doing block checkpoint Allow creating backup jobs when opening BDS docs: block replication's description Add new block driver interfaces to control block replication quorum: implement block driver interfaces for block replication Implement new driver for block replication support replication driver in blockdev-add Add a new API to start/stop replication, do checkpoint to all BDSes block.c| 145 block/Makefile.objs| 3 +- block/backup.c | 14 ++ block/quorum.c | 78 +++ block/replication.c| 549 + blockjob.c | 11 + docs/block-replication.txt | 227 +++ include/block/block.h | 9 + include/block/block_int.h | 15 ++ include/block/blockjob.h | 12 + qapi/block-core.json | 34 ++- 11 files changed, 1093 insertions(+), 4 deletions(-) create mode 100644 block/replication.c create mode 100644 docs/block-replication.txt -- 2.5.0
