Re: [Qemu-block] [PATCH RFC for-2.6 0/3] block: Add meta dirty bitmap for migration/persistence

On 07.12.2015 08:59, Fam Zheng wrote: Vladimir, This is what I propose to implement meta bitmap. It's implemented in the HBitmap level to be more efficient, and the interface slightly varies too. What is the benefit? Hbitmap usage: 1) BdrvDirtyBitmap - need meta 2) BackupBlockJob - doesn't

Re: [Qemu-block] [PATCH RFC for-2.6 1/3] HBitmap: Introduce "meta" bitmap to track bit changes

On 07.12.2015 08:59, Fam Zheng wrote: The meta bitmap will have the same size and granularity as the tracked bitmap, and upon each bit toggle, the corresponding bit in the meta bitmap, at an identical position, will be set. No, meta bitmap should not have same granularity. If we have 16tb

Re: [Qemu-block] [Qemu-devel] [PATCH RFC for-2.6 0/3] block: Add meta dirty bitmap for migration/persistence

On 08.12.2015 04:36, Fam Zheng wrote: On Mon, 12/07 18:47, John Snow wrote: On 12/07/2015 12:59 AM, Fam Zheng wrote: Vladimir, This is what I propose to implement meta bitmap. It's implemented in the HBitmap level to be more efficient, and the interface slightly varies too. I missed it:

Re: [Qemu-block] [PATCH RFC for-2.6 0/3] block: Add meta dirty bitmap for migration/persistence

On 08.12.2015 04:42, Fam Zheng wrote: On Mon, 12/07 17:19, Vladimir Sementsov-Ogievskiy wrote: On 07.12.2015 08:59, Fam Zheng wrote: Vladimir, This is what I propose to implement meta bitmap. It's implemented in the HBitmap level to be more efficient, and the interface slightly varies too

Re: [Qemu-block] [PATCH RFC for-2.6 1/3] HBitmap: Introduce "meta" bitmap to track bit changes

On 08.12.2015 04:31, Fam Zheng wrote: On Mon, 12/07 16:32, Vladimir Sementsov-Ogievskiy wrote: On 07.12.2015 08:59, Fam Zheng wrote: The meta bitmap will have the same size and granularity as the tracked bitmap, and upon each bit toggle, the corresponding bit in the meta bitmap

Re: [Qemu-block] [PATCH RFC for-2.6 1/3] HBitmap: Introduce "meta" bitmap to track bit changes

On 30.12.2015 14:07, Fam Zheng wrote: On Wed, 12/30 13:53, Vladimir Sementsov-Ogievskiy wrote: On 07.12.2015 08:59, Fam Zheng wrote: The meta bitmap will have the same size and granularity as the tracked bitmap, and upon each bit toggle, the corresponding bit in the meta bitmap

Re: [Qemu-block] [PATCH RFC for-2.6 1/3] HBitmap: Introduce "meta" bitmap to track bit changes

On 07.12.2015 08:59, Fam Zheng wrote: The meta bitmap will have the same size and granularity as the tracked bitmap, and upon each bit toggle, the corresponding bit in the meta bitmap, at an identical position, will be set. Signed-off-by: Fam Zheng --- include/qemu/hbitmap.h

Re: [Qemu-block] [PATCH v3 0/4] Bitmap clean-up patches for 2.6

Hmm, stop. Very bad thing (sorry, that I didn't realize it before): This breaks my dirty bitmap migration series with its meta bitmaps. Meta bitmap is an additional HBitmap in BdrvDirtyBitmap, which tracks dirtiness of this BdrvDirtyBitmap. And it (meta bitmap) have its own granularity of

Re: [Qemu-block] [PATCH for 2.6 1/3] backup: Use Bitmap to replace "s->bitmap"

On 20.11.2015 12:59, Fam Zheng wrote: "s->bitmap" tracks done sectors, we only check bit states without using any iterator which HBitmap is good for. Switch to "Bitmap" which is simpler and more memory efficient. Meanwhile, rename it to done_bitmap, to reflect the intention. Signed-off-by: Fam

Re: [Qemu-block] [PATCH for 2.6 3/3] hbitmap: Drop "granularity"

On 20.11.2015 12:59, Fam Zheng wrote: Sometimes confused with the granularity with coarse levels in HBitmap, the granularity in the hbitmap_alloc is not an essential concept of a bitmap. Now that all callers except the test code use zero, it's possible to drop the parameter to make the

Re: [Qemu-block] [PATCH for 2.6 2/3] block: Hide HBitmap in block dirty bitmap interface

On 20.11.2015 12:59, Fam Zheng wrote: HBitmap is an implementation detail of block dirty bitmap that should be hidden from users. Introduce a BdrvDirtyBitmapIter to encapsulate the underlying HBitmapIter. A small difference in the interface is, before, an HBitmapIter is initialized in place,

Re: [Qemu-block] [PATCH for 2.6 0/3] Bitmap clean-up patches for 2.6

Hi Fam! Thanks for it, I really like the idea about dropping granularity from hbitmap. I've waste lots of time understanding the code about bitmaps. Keeping in mind what units are in what granularity and what granularity is in what units is more than inconvenient) On 20.11.2015 12:59, Fam

Re: [Qemu-block] [Qemu-devel] [PATCH for 2.6 2/3] block: Hide HBitmap in block dirty bitmap interface

On 24.11.2015 05:28, Fam Zheng wrote: On Mon, 11/23 16:34, John Snow wrote: Hmm, what's the idea, here? This patch does a lot more than just hide hbitmap details from callers of block_dirty_bitmap functions. So we're changing the backing hbitmap to always be one where g=0 and the number of

Re: [Qemu-block] [Qemu-devel] [PATCH 06/13] HBitmap: Introduce "meta" bitmap to track bit changes

On 11.01.2016 21:56, John Snow wrote: On 01/11/2016 10:40 AM, Vladimir Sementsov-Ogievskiy wrote: On 04.01.2016 13:27, Fam Zheng wrote: Upon each bit toggle, the corresponding bit in the meta bitmap will be set. Signed-off-by: Fam Zheng <f...@redhat.com> --- include/qemu/hbitmap.

Re: [Qemu-block] [PATCH v4 00/15] Dirty bitmap changes for migration/persistence work

p getters block: Assert that bdrv_release_dirty_bitmap succeeded tests: Add test code for hbitmap serialization block: More operations for meta dirty bitmap Vladimir Sementsov-Ogievskiy (2): hbitmap: serialization block: BdrvDirtyBitmap serialization interface block.c

[Qemu-block] [PATCH] mirror: add target-zeroed flag

Add target-zeroed flag to allow user specify that target is already zeroed. With this flag set zeroes which was in source before mirror start will not be copyed. Without this libvirt migration of empty disk takes too long time. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsem

Re: [Qemu-block] [PATCH 1/1] mirror: add target-zeroed flag

It's a duplicate, sorry. On 03.06.2016 17:19, Denis V. Lunev wrote: From: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Add target-zeroed flag to allow user specify that target is already zeroed. With this flag set zeroes which was in source before mirror start will not be

Re: [Qemu-block] [Qemu-devel] [PATCH v4 00/15] Dirty bitmap changes for migration/persistence work

Hi, what are the plans? On 26.05.2016 03:47, Fam Zheng wrote: On Wed, 05/25 17:45, Vladimir Sementsov-Ogievskiy wrote: Hi! Are you going to update the series in the near future? Yes, probably in a couple days. Fam On 08.03.2016 07:44, Fam Zheng wrote: v4: Rebase. Add rev-by from

Re: [Qemu-block] [PATCH 2/3] block: fix race in bdrv_co_discard with drive-mirror

t;d...@openvz.org> CC: Vladimir Sementsov-Ogievskiy<vsement...@virtuozzo.com> CC: Stefan Hajnoczi <stefa...@redhat.com> CC: Fam Zheng <f...@redhat.com> CC: Kevin Wolf <kw...@redhat.com> CC: Max Reitz <mre...@redhat.com> --- block/io.c | 3 ++- 1 file changed, 2 insertions(+

Re: [Qemu-block] [PATCH 06/13] HBitmap: Introduce "meta" bitmap to track bit changes

On 04.01.2016 13:27, Fam Zheng wrote: Upon each bit toggle, the corresponding bit in the meta bitmap will be set. Signed-off-by: Fam Zheng --- include/qemu/hbitmap.h | 8 +++ util/hbitmap.c | 61 +- 2 files

Re: [Qemu-block] [Qemu-devel] [PATCH 11/13] hbitmap: serialization

On 08.01.2016 00:11, John Snow wrote: On 01/04/2016 05:27 AM, Fam Zheng wrote: From: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Functions to serialize / deserialize(restore) HBitmap. HBitmap should be saved to linear sequence of bits independently of endianness and bitmap

Re: [Qemu-block] [PATCH 11/13] hbitmap: serialization

On 04.01.2016 13:27, Fam Zheng wrote: From: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Functions to serialize / deserialize(restore) HBitmap. HBitmap should be saved to linear sequence of bits independently of endianness and bitmap array element (unsigned long) size. The

Re: [Qemu-block] [PATCH 11/13] hbitmap: serialization

On 04.01.2016 13:27, Fam Zheng wrote: From: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Functions to serialize / deserialize(restore) HBitmap. HBitmap should be saved to linear sequence of bits independently of endianness and bitmap array element (unsigned long) size. The

Re: [Qemu-block] [PATCH v2 08/13] block: Support meta dirty bitmap

On 26.01.2016 09:25, Fam Zheng wrote: On Fri, 01/22 15:05, Vladimir Sementsov-Ogievskiy wrote: In my migration series I need iterators, get granularity, and something like hbitmap_count for meta bitmaps. You can add them here if you want, or I can add them in my series. Okay, I can add

Re: [Qemu-block] [Qemu-devel] [PATCH v4 RFC 00/17] qcow2: persistent dirty bitmaps

/repos/qemu/browse On 02/17/2016 10:28 AM, Vladimir Sementsov-Ogievskiy wrote: This series add persistent dirty bitmaps feature to qcow2. Specification is in docs/spec/qcow2.txt (not merged yet, see [PATCH v10] spec: add qcow2 bitmaps extension specification) This series are based on Fam's [PATCH v2

Re: [Qemu-block] [PATCH RFC for-2.6 1/3] HBitmap: Introduce "meta" bitmap to track bit changes

On 30.12.2015 14:07, Fam Zheng wrote: On Wed, 12/30 13:53, Vladimir Sementsov-Ogievskiy wrote: On 07.12.2015 08:59, Fam Zheng wrote: The meta bitmap will have the same size and granularity as the tracked bitmap, and upon each bit toggle, the corresponding bit in the meta bitmap

Re: [Qemu-block] [PATCH v2 00/13] Dirty bitmap changes for migration/persistence work

ges tests: Add test code for meta bitmap block: Support meta dirty bitmap block: Add two dirty bitmap getters block: Assert that bdrv_release_dirty_bitmap succeeded tests: Add test code for hbitmap serialization Vladimir Sementsov-Ogievskiy (2): hbitmap: serialization block

Re: [Qemu-block] [PATCH v2 08/13] block: Support meta dirty bitmap

On 22.01.2016 14:58, Vladimir Sementsov-Ogievskiy wrote: On 20.01.2016 09:11, Fam Zheng wrote: The added group of operations enables tracking of the changed bits in the dirty bitmap. Signed-off-by: Fam Zheng <f...@redhat.com> --- block/dirty-bitmap.c

Re: [Qemu-block] [PATCH v2 05/13] block: Hide HBitmap in block dirty bitmap interface

On 20.01.2016 09:11, Fam Zheng wrote: HBitmap is an implementation detail of block dirty bitmap that should be hidden from users. Introduce a BdrvDirtyBitmapIter to encapsulate the underlying HBitmapIter. A small difference in the interface is, before, an HBitmapIter is initialized in place,

Re: [Qemu-block] [PATCH v2 14/13] block: More operations for meta dirty bitmap

On 26.01.2016 10:26, Fam Zheng wrote: Callers can create an iterator of meta bitmap with bdrv_dirty_meta_iter_new(), then use the bdrv_dirty_iter_* operations on it. Meta iterators are also counted by bitmap->active_iterators. Also add a couple of functions to retrieve granularity and count.

Re: [Qemu-block] [RFC PATCH 01/16] doc: Add QBM format specification

On 26.01.2016 13:38, Fam Zheng wrote: Signed-off-by: Fam Zheng --- docs/specs/qbm.md | 118 ++ 1 file changed, 118 insertions(+) create mode 100644 docs/specs/qbm.md diff --git a/docs/specs/qbm.md b/docs/specs/qbm.md new

Re: [Qemu-block] [RFC PATCH 10/16] qbm: Implement format driver

As I understand, the difference between our driver interfaces: Fam: methods: bdrv_dirty_bitmap_set_persistent all persistent bitmaps are loaded, and they all are enabled Me: methods: bdrv_dirty_bitmap_load \ bitmaps loaded on demand, by name (by cmd line at qemu

[Qemu-block] [PATCH 01/17] hbitmap: load/store

in cell means all-ones-chunk (should not be saved) hbitmap_prepare_store() fills table with 0 for all-zeroes chunks 1 for all-ones chunks 2 for others Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/dirty-bitmap.c | 23 ++ i

[Qemu-block] [PATCH 03/17] qcow2-dirty-bitmap: read dirty bitmap directory

Adds qcow2_read_bitmaps, reading bitmap directory as specified in docs/specs/qcow2.txt Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/qcow2-dirty-bitmap.c | 163 + block/qcow2.h | 10 +++ 2 files c

[Qemu-block] [PATCH 02/17] qcow2: Bitmaps extension: structs and consts

Add data structures and constraints accordingly to docs/specs/qcow2.txt Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/Makefile.objs| 2 +- block/qcow2-dirty-bitmap.c | 47 ++ block/qcow2.h

[Qemu-block] [PATCH v4 RFC 00/17] qcow2: persistent dirty bitmaps

Vladimir Sementsov-Ogievskiy (17): hbitmap: load/store qcow2: Bitmaps extension: structs and consts qcow2-dirty-bitmap: read dirty bitmap directory qcow2-dirty-bitmap: add qcow2_bitmap_load() qcow2-dirty-bitmap: add qcow2_bitmap_store() qcow2: add dirty bitmaps extension qcow2-dirty

[Qemu-block] [PATCH 04/17] qcow2-dirty-bitmap: add qcow2_bitmap_load()

This function loads block dirty bitmap from qcow2. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/qcow2-dirty-bitmap.c | 110 + block/qcow2.c | 2 + block/qcow2.h | 3 ++ include

[Qemu-block] [PATCH 09/17] block: add bdrv_load_dirty_bitmap()

The funcion loads dirty bitmap from file, using underlying driver function. Note: the function doesn't change BdrvDirtyBitmap.file field. This field is only used by bdrv_store_dirty_bitmap() function and is ONLY written by bdrv_dirty_bitmap_set_file() function. Signed-off-by: Vladimir Sementsov

[Qemu-block] [PATCH 05/17] qcow2-dirty-bitmap: add qcow2_bitmap_store()

This function stores block dirty bitmap to qcow2. If the bitmap with the same name, size and granularity already exists, it will be rewritten, if the bitmap with the same name exists but granularity or size does not match, an error will be genrated. Signed-off-by: Vladimir Sementsov-Ogievskiy

[Qemu-block] [PATCH 10/17] qcow2-dirty-bitmap: add autoclear bit

Add autoclear bit for handling rewriting image by old qemu version. If autoclear bit is not set, but bitmaps extension is found it would not be loaded and warning will be generated. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/qcow2-dirty-bitmap.

[Qemu-block] [PATCH 06/17] qcow2: add dirty bitmaps extension

Add dirty bitmap extension as specified in docs/specs/qcow2.txt. Load bitmap headers on open. Handle close and update_header. Handle resize: for now, just block resize if there are dirty bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/qcow2.

[Qemu-block] [PATCH 07/17] qcow2-dirty-bitmap: add qcow2_bitmap_load_check()

The function checks existing of the bitmap without loading it. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/dirty-bitmap.c | 15 +++ block/qcow2-dirty-bitmap.c | 5 + block/qcow2.c| 1 + block/q

[Qemu-block] [PATCH 12/17] qcow2-dirty-bitmap: add IN_USE flag

This flag is set on bitmap load and unset on store. If it is already set when loading the bitmap, the bitmap should not be load (it is in use by other drive or it is inconsistent (was not successfully saved)) Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block

[Qemu-block] [PATCH 11/17] qemu: command line option for dirty bitmaps

will be generated. If create=off and granularity is specified then granularity will be checked for loaded bitmap and if not match an error will be generated. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- blockdev.c| 36 ++ include/sysemu/bloc

[Qemu-block] [PATCH 13/17] qcow2-dirty-bitmaps: disallow stroing bitmap to other bs

Check, that bitmap is stored to the owning bs. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/dirty-bitmap.c | 12 block/qcow2-dirty-bitmap.c | 5 + include/block/dirty-bitmap.h | 2 ++ 3 files changed, 19 insertions(+) diff

[Qemu-block] [PATCH 16/17] qcow2-dirty-bitmap: add AUTO flag

The bitmap should be auto-loaded if auto flag is set. For now, actually, there are no methods to set it. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/qcow2-dirty-bitmap.c | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a

[Qemu-block] [PATCH 15/17] iotests: test internal persistent dirty bitmap

Add simple test cases for testing persistent dirty bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- tests/qemu-iotests/160| 112 ++ tests/qemu-iotests/160.out| 21 tests/qemu-iotests/group

[Qemu-block] [PATCH 14/17] iotests: add VM.test_launcn()

Test vm can launch and print output in case of fail. This function is needed for testing erroneous cases Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- tests/qemu-iotests/iotests.py | 19 +++ 1 file changed, 19 insertions(+) diff --git a/test

[Qemu-block] [PATCH 17/17] qcow2-dirty-bitmap: add EXTRA_DATA_COMPATIBLE flag

If this flag is unset and exta data present the bitmap should be read-only. For now just return error for this case. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/qcow2-dirty-bitmap.c | 8 1 file changed, 8 insertions(+) diff --git a/block/qcow2

[Qemu-block] [PATCH 08/17] block: store persistent dirty bitmaps

in bdrv_dirty_bitmap_set_file() only once. bdrv_ref/bdrv_unref are used for BdrvDirtyBitmap.file to be sure that files will be closed and resources will be freed. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block.c | 3 +++ block/dirty-bitmap.c

Re: [Qemu-block] [RFC PATCH 01/16] doc: Add QBM format specification

On 26.01.2016 13:38, Fam Zheng wrote: Signed-off-by: Fam Zheng --- docs/specs/qbm.md | 118 ++ 1 file changed, 118 insertions(+) create mode 100644 docs/specs/qbm.md diff --git a/docs/specs/qbm.md b/docs/specs/qbm.md new

[Qemu-block] [PATCH 15/22] qcow2-dirty-bitmap: add autoclear bit

Add autoclear bit for handling rewriting image by old qemu version. If autoclear bit is not set, but bitmaps extension is found it would not be loaded and warning will be generated. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/qcow2-dirty-bitmap.

[Qemu-block] [PATCH 03/22] iotests: maintain several vms in test

com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- tests/qemu-iotests/iotests.py | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 0a238ec..2445cf2 100644 --- a/tests/

[Qemu-block] [PATCH v5 00/22] qcow2: persistent dirty bitmaps

er John's comments on v3 Fam Zheng (1): block: Add two dirty bitmap getters Vladimir Sementsov-Ogievskiy (21): block: fix bdrv_dirty_bitmap_granularity signature iotests: maintain several vms in test iotests: add default node-name qapi: add md5 checksum of last dirty bitmap level to que

[Qemu-block] [PATCH 05/22] qapi: add md5 checksum of last dirty bitmap level to query-block

Reviewed-by: John Snow <js...@redhat.com> Reviewed-by: Eric Blake <ebl...@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/dirty-bitmap.c | 1 + include/qemu/hbitmap.h | 8 qapi/block-core.json | 5 - util/hbi

[Qemu-block] [PATCH 22/22] qcow2-dirty-bitmap: add EXTRA_DATA_COMPATIBLE flag

If this flag is unset and exta data present the bitmap should be read-only. For now just return error for this case. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/qcow2-dirty-bitmap.c | 8 1 file changed, 8 insertions(+) diff --git a/block/qcow2

[Qemu-block] [PATCH 16/22] qemu: command line option for dirty bitmaps

will be generated. If create=off and granularity is specified then granularity will be checked for loaded bitmap and if not match an error will be generated. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- blockdev.c| 36 + include/sysemu/bloc

[Qemu-block] [PATCH 04/22] iotests: add default node-name

When testing migration, auto-generated by qemu node-names differs in source and destination qemu and migration fails. After this patch, auto-generated by iotest nodenames will be the same. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- tests/qemu-iotests/iote

[Qemu-block] [PATCH 12/22] qcow2-dirty-bitmap: add qcow2_bitmap_load_check()

The function checks existing of the bitmap without loading it. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/dirty-bitmap.c | 15 +++ block/qcow2-dirty-bitmap.c | 5 + block/qcow2.c| 1 + block/q

[Qemu-block] [PATCH 21/22] qcow2-dirty-bitmap: add AUTO flag

The bitmap should be auto-loaded if auto flag is set. For now, actually, there are no methods to set it. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/qcow2-dirty-bitmap.c | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a

[Qemu-block] [PATCH 20/22] iotests: test internal persistent dirty bitmap

Add simple test cases for testing persistent dirty bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- tests/qemu-iotests/160| 112 ++ tests/qemu-iotests/160.out| 21 tests/qemu-iotests/group

[Qemu-block] [PATCH 07/22] qcow2: Bitmaps extension: structs and consts

Add data structures and constraints accordingly to docs/specs/qcow2.txt Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/Makefile.objs| 2 +- block/qcow2-dirty-bitmap.c | 47 ++ block/qcow2.h

[Qemu-block] [PATCH 11/22] qcow2: add dirty bitmaps extension

Add dirty bitmap extension as specified in docs/specs/qcow2.txt. Load bitmap headers on open. Handle close and update_header. Handle resize: for now, just block resize if there are dirty bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/qcow2.

[Qemu-block] [PATCH 17/22] qcow2-dirty-bitmap: add IN_USE flag

This flag is set on bitmap load and unset on store. If it is already set when loading the bitmap, the bitmap should not be load (it is in use by other drive or it is inconsistent (was not successfully saved)) Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block

[Qemu-block] [PATCH 19/22] iotests: add VM.test_launcn()

Test vm can launch and print output in case of fail. This function is needed for testing erroneous cases Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- tests/qemu-iotests/iotests.py | 20 1 file changed, 20 insertions(+) diff --git a/test

[Qemu-block] [PATCH 14/22] block: add bdrv_load_dirty_bitmap()

The funcion loads dirty bitmap from file, using underlying driver function. Note: the function doesn't change BdrvDirtyBitmap.file field. This field is only used by bdrv_store_dirty_bitmap() function and is ONLY written by bdrv_dirty_bitmap_set_file() function. Signed-off-by: Vladimir Sementsov

[Qemu-block] [PATCH 01/22] block: Add two dirty bitmap getters

From: Fam Zheng <f...@redhat.com> For dirty bitmap users to get the size and the name of a BdrvDirtyBitmap. Signed-off-by: Fam Zheng <f...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> -

[Qemu-block] [PATCH 18/22] qcow2-dirty-bitmaps: disallow stroing bitmap to other bs

Check, that bitmap is stored to the owning bs. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/dirty-bitmap.c | 12 block/qcow2-dirty-bitmap.c | 5 + include/block/dirty-bitmap.h | 2 ++ 3 files changed, 19 insertions(+) diff

[Qemu-block] [PATCH 10/22] qcow2-dirty-bitmap: add qcow2_bitmap_store()

This function stores block dirty bitmap to qcow2. If the bitmap with the same name, size and granularity already exists, it will be rewritten, if the bitmap with the same name exists but granularity or size does not match, an error will be genrated. Signed-off-by: Vladimir Sementsov-Ogievskiy

[Qemu-block] [PATCH 08/22] qcow2-dirty-bitmap: read dirty bitmap directory

Adds qcow2_read_bitmaps, reading bitmap directory as specified in docs/specs/qcow2.txt Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/qcow2-dirty-bitmap.c | 165 + block/qcow2.h | 10 +++ 2 files c

[Qemu-block] [PATCH 06/22] hbitmap: load/store

in cell means all-ones-chunk (should not be saved) hbitmap_prepare_store() fills table with 0 for all-zeroes chunks 1 for all-ones chunks 2 for others Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/dirty-bitmap.c | 23 + i

Re: [Qemu-block] [PATCH 01/22] block: Add two dirty bitmap getters

On 15.03.2016 23:04, Vladimir Sementsov-Ogievskiy wrote: From: Fam Zheng <f...@redhat.com> For dirty bitmap users to get the size and the name of a BdrvDirtyBitmap. Signed-off-by: Fam Zheng <f...@redhat.com> Reviewed-by: John Snow <js...@redhat.com> Signed-off-by: Vladimir S

Re: [Qemu-block] [PATCH v4 12/15] hbitmap: serialization

On 11.03.2016 19:27, Max Reitz wrote: On 08.03.2016 05:45, Fam Zheng wrote: From: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Functions to serialize / deserialize(restore) HBitmap. HBitmap should be saved to linear sequence of bits independently of endianness and bitmap

Re: [Qemu-block] [Qemu-devel] [PATCH 06/22] hbitmap: load/store

On 22.03.2016 01:42, John Snow wrote: On 03/15/2016 04:04 PM, Vladimir Sementsov-Ogievskiy wrote: Add functions for load/store HBitmap to BDS, using clusters table: Last level of the bitmap is splitted into chunks of 'cluster_size' size. Each cell of the table contains offset in bds, to load

Re: [Qemu-block] [Qemu-devel] [PATCH 06/22] hbitmap: load/store

On 23.03.2016 00:49, John Snow wrote: On 03/15/2016 04:04 PM, Vladimir Sementsov-Ogievskiy wrote: Add functions for load/store HBitmap to BDS, using clusters table: Last level of the bitmap is splitted into chunks of 'cluster_size' size. Each cell of the table contains offset in bds, to load

Re: [Qemu-block] [Qemu-devel] [PATCH v4 RFC 00/17] qcow2: persistent dirty bitmaps

On 04.03.2016 21:00, John Snow wrote: On 02/17/2016 10:28 AM, Vladimir Sementsov-Ogievskiy wrote: This series add persistent dirty bitmaps feature to qcow2. Specification is in docs/spec/qcow2.txt (not merged yet, see [PATCH v10] spec: add qcow2 bitmaps extension specification) This series

Re: [Qemu-block] [PATCH 10/22] qcow2-dirty-bitmap: add qcow2_bitmap_store()

On 22.03.2016 21:49, Eric Blake wrote: On 03/15/2016 02:04 PM, Vladimir Sementsov-Ogievskiy wrote: This function stores block dirty bitmap to qcow2. If the bitmap with the same name, size and granularity already exists, it will be rewritten, if the bitmap with the same name exists

Re: [Qemu-block] [PATCH 20/22] iotests: test internal persistent dirty bitmap

On 22.03.2016 21:27, Eric Blake wrote: On 03/15/2016 02:04 PM, Vladimir Sementsov-Ogievskiy wrote: Add simple test cases for testing persistent dirty bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- tests/qemu-iotests/160

[Qemu-block] [PATCH 3/3] blockjob: fix transaction cancel

Prevent a situation, when some jobs from transaction are already finished and user tries to cancel a job from this transaction. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- blockdev.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/blockd

[Qemu-block] [PATCH 1/3] blockjob: fix dead pointer in txn list

Job may be freed in block_job_unref and in this case this would break transaction QLIST. Fix this by removing job from this list before unref. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- blockjob.c | 1 + 1 file changed, 1 insertion(+) diff --git a/blockj

[Qemu-block] [PATCH RFC 0/3] Backup/blockjob fixes

Hi all! There are some fixes, related to previously posted test [PATCH] backup: block-job error BUG I'm not sure that proposed solution is a true way to fix the problem, but it just works for me. Vladimir Sementsov-Ogievskiy (3): blockjob: fix dead pointer in txn list backup: fix

[Qemu-block] [PATCH 2/3] backup: fix transaction fail scenario

When there are several backups in one transaction, successed block job must wait for possible cancelling by other block job (if it fails). Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/backup.c | 12 blockdev.c

[Qemu-block] [PATCH v2] mirror: double performance of the bulk stage if the disc is full

ial constant (1 Mb here), which is not that big or small. This effectively enables back parallelism in mirror code as it was designed. The result is important: the time to migrate 10 Gb disk is reduced from ~350 sec to 170 sec. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuoz

[Qemu-block] [PATCH v2] qcow2: do not allocate extra memory

o, deflate will not write more than avail_out to output buffer. If there is no enough space in output buffer for compressed data (it may be larger than input data) deflate just returns Z_OK. (if all data is compressed and written to output buffer deflate returns Z_STREAM_END). Signed-off-by: Vladimir

Re: [Qemu-block] [PATCH 1/3] blockjob: fix dead pointer in txn list

On 02.08.2016 01:39, John Snow wrote: On 07/27/2016 06:49 AM, Vladimir Sementsov-Ogievskiy wrote: Job may be freed in block_job_unref and in this case this would break transaction QLIST. Fix this by removing job from this list before unref. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsem

Re: [Qemu-block] [PATCH] backup: block-job error BUG

On 02.08.2016 01:36, John Snow wrote: On 07/18/2016 05:22 PM, Vladimir Sementsov-Ogievskiy wrote: Hi all! This is a variant of existing test case which produces test failure. It looks like the reason is: one block job is in backup_complete, in synchronous bdrv_flush (success job) other

Re: [Qemu-block] [PATCH 5/5] iotests: add transactional failure race test

On 08.08.2016 22:09, John Snow wrote: Add a regression test for the case found by Vladimir. Reported-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Signed-off-by: John Snow <js...@redhat.com> --- tests/qemu-iotests/

Re: [Qemu-block] [PATCH 01/29] hbitmap: fix dirty iter

On 10.08.2016 16:41, Kevin Wolf wrote: Am 08.08.2016 um 17:04 hat Vladimir Sementsov-Ogievskiy geschrieben: If dirty bitmap was cleared during iterator life, we can went to zero current in hbitmap_iter_skip_words, starting from saved (and currently wrong hbi->cur[...]). I was going to sugg

Re: [Qemu-block] [PATCH 16/29] block: add bdrv_load_dirty_bitmap()

On 11.08.2016 14:24, Kevin Wolf wrote: Am 08.08.2016 um 17:05 hat Vladimir Sementsov-Ogievskiy geschrieben: The funcion loads dirty bitmap from file, using underlying driver function. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/dirty-bitmap.c

Re: [Qemu-block] [PATCH 06/29] qcow2-bitmap: add qcow2_read_bitmaps()

On 11.08.2016 12:36, Kevin Wolf wrote: Am 08.08.2016 um 17:04 hat Vladimir Sementsov-Ogievskiy geschrieben: Add qcow2_read_bitmaps, reading bitmap directory as specified in docs/specs/qcow2.txt Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- block/qcow2-bi

Re: [Qemu-block] [Qemu-devel] [PATCH v5 01/10] block: Hide HBitmap in block dirty bitmap interface

On 13.07.2016 01:49, John Snow wrote: On 06/03/2016 12:32 AM, Fam Zheng wrote: HBitmap is an implementation detail of block dirty bitmap that should be hidden from users. Introduce a BdrvDirtyBitmapIter to encapsulate the underlying HBitmapIter. A small difference in the interface is, before,

Re: [Qemu-block] [PATCH 1/1] mirror: double performance of the bulk stage if the disc is full

On 12.07.2016 16:51, Kevin Wolf wrote: Am 12.07.2016 um 11:36 hat Denis V. Lunev geschrieben: From: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Mirror can do up to 16 in-flight requests, but actually on full copy (the whole source disk is non-zero) in-flight is al

[Qemu-block] [PATCH] backup: block-job error BUG

../scripts/qmp/qmp.py", line 44, in __init__ +self.__sock.bind(self.__address) + File "/usr/lib64/python2.7/socket.py", line 224, in meth +return getattr(self._sock,name)(*args) +error: [Errno 98] Address already in use +

Re: [Qemu-block] [PATCH] backup: block-job error BUG

forget qemu-de...@nongnu.org.. add it. On 19.07.2016 00:22, Vladimir Sementsov-Ogievskiy wrote: Hi all! This is a variant of existing test case which produces test failure. It looks like the reason is: one block job is in backup_complete, in synchronous bdrv_flush (success job) other (job

Re: [Qemu-block] [Qemu-devel] [PATCH] backup: block-job error BUG

On 19.07.2016 05:12, Fam Zheng wrote: On Tue, 07/19 00:26, Vladimir Sementsov-Ogievskiy wrote: forget qemu-de...@nongnu.org.. add it. On 19.07.2016 00:22, Vladimir Sementsov-Ogievskiy wrote: Hi all! This is a variant of existing test case which produces test failure. It looks like

[Qemu-block] [PATCH] qcow2: do not allocate extra memory

There are no needs to allocate more than one cluster, as we set avail_out for deflate to one cluster. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- Hi all! Please, can anybody say me what I'm missing? I've looked through deflate documentation at http://www.zl

Re: [Qemu-block] [PATCH] qcow2: do not allocate extra memory

On 12.07.2016 21:43, Eric Blake wrote: On 07/12/2016 11:43 AM, Vladimir Sementsov-Ogievskiy wrote: There are no needs to allocate more than one cluster, as we set avail_out for deflate to one cluster. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> --- Hi all!

Re: [Qemu-block] [PATCH v5 01/10] block: Hide HBitmap in block dirty bitmap interface

On 03.06.2016 07:32, Fam Zheng wrote: HBitmap is an implementation detail of block dirty bitmap that should be hidden from users. Introduce a BdrvDirtyBitmapIter to encapsulate the underlying HBitmapIter. A small difference in the interface is, before, an HBitmapIter is initialized in place,

Re: [Qemu-block] [PATCH v5 01/10] block: Hide HBitmap in block dirty bitmap interface

On 28.06.2016 18:36, Vladimir Sementsov-Ogievskiy wrote: On 03.06.2016 07:32, Fam Zheng wrote: HBitmap is an implementation detail of block dirty bitmap that should be hidden from users. Introduce a BdrvDirtyBitmapIter to encapsulate the underlying HBitmapIter. A small difference

Re: [Qemu-block] [PATCH v5 07/10] hbitmap: serialization

On 03.06.2016 07:32, Fam Zheng wrote: From: Vladimir Sementsov-Ogievskiy <vsement...@virtuozzo.com> Functions to serialize / deserialize(restore) HBitmap. HBitmap should be saved to linear sequence of bits independently of endianness and bitmap array element (unsigned long) size. The

  1   2   3   4   5   6   7   8   9   10   >