Re: [Qemu-block] [Qemu-devel] [PATCH v7 00/20] block: Image locking series for 2.8

2016-08-08 Thread Fam Zheng
On Mon, 08/08 06:59, no-re...@ec2-52-6-146-230.compute-1.amazonaws.com wrote: > Checking PATCH 3/20: block: Add and parse "lock-mode" option for image > locking... > ERROR: do not use assignment in if condition > #80: FILE: blockdev.c:548: > +if ((buf = qemu_opt_get(opts, BDRV_OPT_LOCK_MODE))

Re: [Qemu-block] [Qemu-devel] [PATCH v2 1/8] util: Add UUID API

2016-08-08 Thread Fam Zheng
On Mon, 08/08 15:51, Jeff Cody wrote: > > > +typedef unsigned char QemuUUID[16]; > > > > I'm afraid this typedef is problematic. Consider: > > > > void use_uuid(QemuUUID uuid) > > { > > printf("sizeof(uuid) %zd\n", sizeof(uuid)); > > uuid[0]++; > > } > > > >

Re: [Qemu-block] [PATCH v2 5/8] vpc: Use QEMU UUID API

2016-08-08 Thread Fam Zheng
On Mon, 08/08 16:49, Jeff Cody wrote: > On Mon, Aug 08, 2016 at 02:09:25PM +0800, Fam Zheng wrote: > > Previously we conditionally generate if footer->uuid, when libuuid is > > s/generate if/generated/ > > s/is/was/ Fixing, thanks! Fam

Re: [Qemu-block] [PATCH v2 8/8] configure: Remove detection code for UUID

2016-08-08 Thread Fam Zheng
On Mon, 08/08 16:52, Jeff Cody wrote: > > @@ -1096,6 +1091,9 @@ for opt do > >--enable-vhdx|--disable-vhdx) > >echo "$0: $opt is obsolete, VHDX driver is always built" > >;; > > + --enable-uuid|--disable-uuid) > > + echo "$0: $opt is obsolete, UUID support is always built" >

Re: [Qemu-block] [Qemu-devel] [PATCH 3/5] blockjob: refactor backup_start as backup_job_create

2016-08-08 Thread John Snow
On 08/08/2016 03:09 PM, John Snow wrote: Refactor backup_start as backup_job_create, which only creates the job, but does not automatically start it. The old interface, 'backup_start', is not kept in favor of limiting the number of nearly-identical iterfaces that would have to be edited to

Re: [Qemu-block] [PATCH v2 8/8] configure: Remove detection code for UUID

2016-08-08 Thread Jeff Cody
On Mon, Aug 08, 2016 at 02:09:28PM +0800, Fam Zheng wrote: > All code now uses built-in UUID implementation. Remove the code of > libuuid and make --enable-uuid and --disable-uuid only print a message. > > Signed-off-by: Fam Zheng > --- > configure | 43

Re: [Qemu-block] [PATCH v2 5/8] vpc: Use QEMU UUID API

2016-08-08 Thread Jeff Cody
On Mon, Aug 08, 2016 at 02:09:25PM +0800, Fam Zheng wrote: > Previously we conditionally generate if footer->uuid, when libuuid is s/generate if/generated/ s/is/was/ > available. Now that we have a built-in implementation, we can switch to > it. > > Signed-off-by: Fam Zheng >

Re: [Qemu-block] [PATCH v2 3/8] vhdx: Use QEMU UUID API

2016-08-08 Thread Jeff Cody
On Mon, Aug 08, 2016 at 02:09:23PM +0800, Fam Zheng wrote: > This removes our dependency to libuuid, so that the driver can always be > built. > > Similar to how we handled data plane configure options, --enable-vhdx > and --disable-vhdx are also changed to a nop with a message saying it's >

Re: [Qemu-block] [Qemu-devel] [PATCH v2 1/8] util: Add UUID API

2016-08-08 Thread Jeff Cody
On Mon, Aug 08, 2016 at 01:07:33PM +0200, Markus Armbruster wrote: > Fam Zheng writes: > > > A number of different places across the code base use CONFIG_UUID. Some > > of them are soft dependency, some are not built if libuuid is not > > available, some come with dummy

Re: [Qemu-block] [PATCH 0/5] blockjobs: Fix transactional race condition

2016-08-08 Thread John Snow
I should also clarify that this is ambiguously for either 2.7 or 2.8. 2.7: This fixes a real, observable problem with transactional completion that has the capacity to hang QEMU or segfault due to QLIST corruption. 2.8: Incremental backup is not earnestly a supported feature yet as

Re: [Qemu-block] [Qemu-devel] [PATCH 0/5] blockjobs: Fix transactional race condition

2016-08-08 Thread no-reply
Hi, Your series seems to have some coding style problems. See output below for more information: Message-id: 1470683381-16680-1-git-send-email-js...@redhat.com Type: series Subject: [Qemu-devel] [PATCH 0/5] blockjobs: Fix transactional race condition === TEST SCRIPT BEGIN === #!/bin/bash

[Qemu-block] [PATCH 1/5] blockjob: fix dead pointer in txn list

2016-08-08 Thread John Snow
From: Vladimir Sementsov-Ogievskiy Though it is not intended to be reached through normal circumstances, if we do not gracefully deconstruct the transaction QLIST, we may wind up with stale pointers in the list. The rest of this series attempts to address the

[Qemu-block] [PATCH 3/5] blockjob: refactor backup_start as backup_job_create

2016-08-08 Thread John Snow
Refactor backup_start as backup_job_create, which only creates the job, but does not automatically start it. The old interface, 'backup_start', is not kept in favor of limiting the number of nearly-identical iterfaces that would have to be edited to keep up with QAPI changes in the future.

[Qemu-block] [PATCH 5/5] iotests: add transactional failure race test

2016-08-08 Thread John Snow
Add a regression test for the case found by Vladimir. Reported-by: Vladimir Sementsov-Ogievskiy Signed-off-by: John Snow --- tests/qemu-iotests/124 | 91 ++ tests/qemu-iotests/124.out | 4 +- 2 files

[Qemu-block] [PATCH 2/5] blockjob: add block_job_start

2016-08-08 Thread John Snow
Instead of automatically starting jobs at creation time via backup_start et al, we'd like to return a job object pointer that can be started manually at later point in time. For now, add the block_job_start mechanism and start the jobs automatically as we have been doing, with conversions

[Qemu-block] [PATCH 0/5] blockjobs: Fix transactional race condition

2016-08-08 Thread John Snow
There are a few problems with transactional job completion right now. First, if jobs complete so quickly they complete before remaining jobs get a chance to join the transaction, the completion mode can leave well known state and the QLIST can get corrupted and the transactional jobs can complete

[Qemu-block] [PATCH 4/5] blockjob: add .clean property

2016-08-08 Thread John Snow
Cleaning up after we have deferred to the main thread but before the transaction has converged can be dangerous and result in deadlocks if the job cleanup invokes any BH polling loops. A job may attempt to begin cleaning up, but may induce another job to enter its cleanup routine. The second job,

[Qemu-block] [PATCH v7 3/4] blockdev: Add dynamic module loading for block drivers

2016-08-08 Thread Colin Lord
From: Marc Mari Extend the current module interface to allow for block drivers to be loaded dynamically on request. The only block drivers that can be converted into modules are the drivers that don't perform any init operation except for registering themselves. In addition,

[Qemu-block] [PATCH v7 2/4] blockdev: Add dynamic generation of module_block.h

2016-08-08 Thread Colin Lord
From: Marc Mari To simplify the addition of new block modules, add a script that generates module_block.h automatically from the modules' source code. This script assumes that the QEMU coding style rules are followed. Signed-off-by: Marc MarĂ­

[Qemu-block] [PATCH v7 4/4] blockdev: Modularize nfs block driver

2016-08-08 Thread Colin Lord
Modularizes the nfs block driver so that it gets dynamically loaded. Signed-off-by: Colin Lord Reviewed-by: Stefan Hajnoczi --- block/Makefile.objs | 1 + configure | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git

[Qemu-block] [PATCH v7 1/4] blockdev: prepare iSCSI block driver for dynamic loading

2016-08-08 Thread Colin Lord
This commit moves the initialization of the QemuOptsList qemu_iscsi_opts struct out of block/iscsi.c in order to allow the iscsi module to be dynamically loaded. Signed-off-by: Colin Lord Reviewed-by: Fam Zheng --- block/iscsi.c | 36

[Qemu-block] [PATCH v7 0/4] Dynamic module loading for block drivers

2016-08-08 Thread Colin Lord
One more minor revision from v6, no big changes. v7: - Add ifdef around qemu_iscsi_opts in vl.c (first patch) v6: - Fix bug so that users can specify a modularized driver on the cli without qemu exiting - Remove extra lines from Makefile - Add patch to modularize NFS v5: - No format drivers

Re: [Qemu-block] [PATCH v6 1/4] blockdev: prepare iSCSI block driver for dynamic loading

2016-08-08 Thread Colin Lord
On 08/08/2016 11:18 AM, Stefan Hajnoczi wrote: > On Tue, Aug 02, 2016 at 10:12:14AM -0400, Colin Lord wrote: >> diff --git a/vl.c b/vl.c >> index e7c2c62..1a5f807 100644 >> --- a/vl.c >> +++ b/vl.c >> @@ -506,6 +506,41 @@ static QemuOptsList qemu_fw_cfg_opts = { >> }, >> }; >> >> +static

Re: [Qemu-block] [PATCH v6 4/4] blockdev: Modularize nfs block driver

2016-08-08 Thread Stefan Hajnoczi
On Tue, Aug 02, 2016 at 10:12:17AM -0400, Colin Lord wrote: > Modularizes the nfs block driver so that it gets dynamically loaded. > --- > block/Makefile.objs | 1 + > configure | 4 ++-- > 2 files changed, 3 insertions(+), 2 deletions(-) Reviewed-by: Stefan Hajnoczi

[Qemu-block] [PATCH 25/29] qapi: add md5 checksum of last dirty bitmap level to query-block

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow Reviewed-by: Eric Blake Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 1 + include/qemu/hbitmap.h | 8 qapi/block-core.json | 5 - util/hbitmap.c | 8

[Qemu-block] [PATCH 04/29] block/dirty-bitmap: add deserialize_ones func

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Add bdrv_dirty_bitmap_deserialize_ones() function, which is needed for qcow2 bitmap loading, to handle unallocated bitmap parts, marked as all-ones. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 7 +++ include/block/dirty-bitmap.h

[Qemu-block] [PATCH 02/29] tests: add hbitmap iter test

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Test that hbitmap iter is resistant to bitmap resetting. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev --- tests/test-hbitmap.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/tests/test-hbitmap.c

[Qemu-block] [PATCH v6 00/29] qcow2: persistent dirty bitmaps

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
v6: https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=refs%2Ftags%2Fqcow2-bitmap-v6 based on block-next (https://github.com/XanClic/qemu/commits/block-next) There are a lot of changes, reorderings and additions in comparement with v5. One principal thing: now bitmaps are removed from

[Qemu-block] [PATCH 22/29] qmp: add persistent flag to block-dirty-bitmap-add

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Add optional 'persistent' flag to qmp command block-dirty-bitmap-add. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev --- blockdev.c | 12 +++- qapi/block-core.json | 3 ++-

[Qemu-block] [PATCH 29/29] qcow2-dirty-bitmap: refcounts

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Calculate refcounts for dirty bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev --- block/qcow2-bitmap.c | 58 -- block/qcow2-refcount.c | 14 +++-

[Qemu-block] [PATCH 17/29] qcow2-bitmap: add autoclear bit

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Add autoclear bit for handling rewriting image by old qemu version. If autoclear bit is not set, but bitmaps extension is found it would not be loaded and warning will be generated. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 4

Re: [Qemu-block] [PATCH v6 3/4] blockdev: Add dynamic module loading for block drivers

2016-08-08 Thread Stefan Hajnoczi
On Tue, Aug 02, 2016 at 10:12:16AM -0400, Colin Lord wrote: > Extend the current module interface to allow for block drivers to be > loaded dynamically on request. The only block drivers that can be > converted into modules are the drivers that don't perform any init > operation except for

[Qemu-block] [PATCH 24/29] qcow2-bitmap: maintian BlockDirtyBitmap.autoload

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Sync qcow2 dirty bitmap autload flag and BlockDirtyBitmap autoload flag Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev --- block/qcow2-bitmap.c | 11 +-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git

[Qemu-block] [PATCH 20/29] qcow2-bitmap: add AUTO flag

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
The bitmap should be auto-loaded if auto flag is set. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 40 +++- 1 file changed, 39 insertions(+), 1 deletion(-) diff --git a/block/qcow2-bitmap.c

[Qemu-block] [PATCH 08/29] qcow2-bitmap: delete bitmap from qcow2 after load

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
If we load bitmap for r/w bds, it's data in the image should be considered inconsistent from this point. Therefore it is safe to remove it from the image. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 227

[Qemu-block] [PATCH 05/29] qcow2-bitmap: structs and consts

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Create block/qcow2-bitmap.c Add data structures and constraints accordingly to docs/specs/qcow2.txt Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/Makefile.objs | 2 +- block/qcow2-bitmap.c | 47 +++ block/qcow2.h

[Qemu-block] [PATCH 06/29] qcow2-bitmap: add qcow2_read_bitmaps()

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Add qcow2_read_bitmaps, reading bitmap directory as specified in docs/specs/qcow2.txt Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 100 +++ block/qcow2.h| 9 + 2 files changed, 109

[Qemu-block] [PATCH 07/29] qcow2-bitmap: add qcow2_bitmap_load()

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
This function loads block dirty bitmap from qcow2. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 165 ++ block/qcow2.c | 2 + block/qcow2.h | 3 +

[Qemu-block] [PATCH 16/29] block: add bdrv_load_dirty_bitmap()

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
The funcion loads dirty bitmap from file, using underlying driver function. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 16 include/block/dirty-bitmap.h | 2 ++ 2 files changed, 18 insertions(+) diff --git

[Qemu-block] [PATCH 15/29] block/dirty-bitmap: introduce persistent bitmaps

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Add field 'persistent' to BdrvDirtyBitmap. Store all persistent bitmaps of the BDS in bdrv_close(). Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 2 ++ block/dirty-bitmap.c | 42 ++

[Qemu-block] [PATCH 28/29] qcow2-bitmap: do not try reloading bitmaps

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Sometimes image is being reopened without removing bdrv state and therefore bitmaps. So here we allow not loading qcow2 bitmap if it already exists. It may lead to problem, if existing of such bitmap is a mistake - this mistake would be skipped. Signed-off-by: Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [PATCH v6 1/4] blockdev: prepare iSCSI block driver for dynamic loading

2016-08-08 Thread Stefan Hajnoczi
On Tue, Aug 02, 2016 at 10:12:14AM -0400, Colin Lord wrote: > diff --git a/vl.c b/vl.c > index e7c2c62..1a5f807 100644 > --- a/vl.c > +++ b/vl.c > @@ -506,6 +506,41 @@ static QemuOptsList qemu_fw_cfg_opts = { > }, > }; > > +static QemuOptsList qemu_iscsi_opts = { > +.name = "iscsi", >

Re: [Qemu-block] [PATCH v6 2/4] blockdev: Add dynamic generation of module_block.h

2016-08-08 Thread Stefan Hajnoczi
On Tue, Aug 02, 2016 at 10:12:15AM -0400, Colin Lord wrote: > From: Marc Mari > > To simplify the addition of new block modules, add a script that generates > module_block.h automatically from the modules' source code. > > This script assumes that the QEMU coding style rules

[Qemu-block] [PATCH 27/29] qcow2-bitmap: delete in_use bitmaps on image load

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev --- block/qcow2-bitmap.c | 95 1 file changed, 95 insertions(+) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c

[Qemu-block] [PATCH 11/29] qcow2-bitmap: check constraints

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Check bitmap header constraints as specified in docs/specs/qcow2.txt Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 39 +++ 1 file changed, 39 insertions(+) diff --git a/block/qcow2-bitmap.c

[Qemu-block] [PATCH 19/29] block/dirty-bitmap: add autoload field to BdrvDirtyBitmap

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Add a new field. For now it is meaningless. Will be used in future patches. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev --- block/dirty-bitmap.c | 14 ++ include/block/dirty-bitmap.h | 2 ++ 2 files

[Qemu-block] [PATCH 21/29] qcow2-bitmap: add EXTRA_DATA_COMPATIBLE flag

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
If this flag is unset and extra data is present the bitmap should be read-only. For now just return error for this case. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 8 1 file changed, 8 insertions(+) diff --git

[Qemu-block] [PATCH 26/29] iotests: test qcow2 persistent dirty bitmap

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/160| 87 +++ tests/qemu-iotests/160.out| 5 +++ tests/qemu-iotests/group | 1 + tests/qemu-iotests/iotests.py | 6 +++ 4 files changed, 99

[Qemu-block] [PATCH 01/29] hbitmap: fix dirty iter

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
If dirty bitmap was cleared during iterator life, we can went to zero current in hbitmap_iter_skip_words, starting from saved (and currently wrong hbi->cur[...]). Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev ---

[Qemu-block] [PATCH 09/29] qcow2-bitmap: add qcow2_bitmap_store()

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
This function stores block dirty bitmap to qcow2. If the bitmap with the same name, size and granularity already exists, it will be rewritten, if the bitmap with the same name exists but granularity or size does not match, an error will be generated. Signed-off-by: Vladimir Sementsov-Ogievskiy

[Qemu-block] [PATCH 14/29] qcow2-bitmap: add qcow2_bitmap_load_check()

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
The function checks the existance of the bitmap without loading it. Will be used in future patches. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 15 +++ block/qcow2-bitmap.c | 5 + block/qcow2.c

[Qemu-block] [PATCH 03/29] block: fix bdrv_dirty_bitmap_granularity signature

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Make getter signature const-correct. This allows other functions with const dirty bitmap parameter use bdrv_dirty_bitmap_granularity(). Reviewed-by: Eric Blake Reviewed-by: John Snow Signed-off-by: Vladimir Sementsov-Ogievskiy ---

[Qemu-block] [PATCH 23/29] qmp: add autoload parameter to block-dirty-bitmap-add

2016-08-08 Thread Vladimir Sementsov-Ogievskiy
Optional. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev --- blockdev.c | 22 -- qapi/block-core.json | 2 +- qmp-commands.hx | 4 +++- 3 files changed, 24

Re: [Qemu-block] [PULL 0/4] More block layer patches for 2.7.0-rc2

2016-08-08 Thread Peter Maydell
On 8 August 2016 at 12:52, Kevin Wolf wrote: > The following changes since commit cf5198d58088e3b416fe517b3946e5120b342761: > > Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20160805' into > staging (2016-08-08 10:39:18 +0100) > > are available in the git repository

Re: [Qemu-block] [PATCH v24 00/12] Block replication for continuous checkpoints

2016-08-08 Thread Stefan Hajnoczi
On Wed, Jul 27, 2016 at 03:01:41PM +0800, Changlong Xie wrote: > Block replication is a very important feature which is used for > continuous checkpoints(for example: COLO). > > You can get the detailed information about block replication from here: >

Re: [Qemu-block] [PATCH v7 00/16] backup compression

2016-08-08 Thread Kevin Wolf
Am 22.07.2016 um 10:17 hat Denis V. Lunev geschrieben: > The idea is simple - backup is "written-once" data. It is written block > by block and it is large enough. It would be nice to save storage > space and compress it. > > These patches add the ability to compress data during backup. This >

Re: [Qemu-block] [Qemu-devel] [PATCH v7 00/20] block: Image locking series for 2.8

2016-08-08 Thread no-reply
Hi, Your series seems to have some coding style problems. See output below for more information: Message-id: 1470662013-19785-1-git-send-email-f...@redhat.com Type: series Subject: [Qemu-devel] [PATCH v7 00/20] block: Image locking series for 2.8 === TEST SCRIPT BEGIN === #!/bin/bash BASE=base

Re: [Qemu-block] [PATCH v7 13/16] drive-backup: added support for data compression

2016-08-08 Thread Kevin Wolf
Am 08.08.2016 um 16:07 hat Kevin Wolf geschrieben: > Am 22.07.2016 um 10:17 hat Denis V. Lunev geschrieben: > > @@ -3317,7 +3320,7 @@ void do_blockdev_backup(BlockdevBackup *backup, > > BlockJobTxn *txn, Error **errp) > > } > > } > > backup_start(backup->job_id, bs, target_bs,

Re: [Qemu-block] [PATCH v7 13/16] drive-backup: added support for data compression

2016-08-08 Thread Kevin Wolf
Am 22.07.2016 um 10:17 hat Denis V. Lunev geschrieben: > From: Pavel Butsykin > > The idea is simple - backup is "written-once" data. It is written block > by block and it is large enough. It would be nice to save storage > space and compress it. > > The patch adds a

Re: [Qemu-block] [PATCH v6] qemu-img: add the 'dd' subcommand

2016-08-08 Thread Max Reitz
On 25.07.2016 07:58, Reda Sallahi wrote: > This patch adds a basic dd subcommand analogous to dd(1) to qemu-img. > > For the start, this implements the bs, if, of and count options and requires > both if and of to be specified (no stdin/stdout if not specified) and doesn't > support tty, pipes,

Re: [Qemu-block] [PATCH v7 04/16] qcow2: add qcow2_co_pwritev_compressed

2016-08-08 Thread Kevin Wolf
Am 22.07.2016 um 10:17 hat Denis V. Lunev geschrieben: > From: Pavel Butsykin > > Added implementation of the qcow2_co_pwritev_compressed function that > will allow us to safely use compressed writes for the qcow2 from running > VMs. > > Signed-off-by: Pavel Butsykin

[Qemu-block] [PATCH v7 20/20] qemu-iotests: Add test case 153 for image locking

2016-08-08 Thread Fam Zheng
Signed-off-by: Fam Zheng Reviewed-by: Max Reitz --- tests/qemu-iotests/153 | 197 + tests/qemu-iotests/153.out | 426 + tests/qemu-iotests/group | 1 + 3 files changed, 624 insertions(+)

[Qemu-block] [PATCH v7 14/20] qemu-iotests: 030: Disable image locking when checking test image

2016-08-08 Thread Fam Zheng
The VM is running, qemu-io would fail the lock acquisition. Signed-off-by: Fam Zheng Reviewed-by: Max Reitz --- tests/qemu-iotests/030 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030 index

[Qemu-block] [PATCH v7 17/20] iotests: Disable image locking in 085

2016-08-08 Thread Fam Zheng
The cases is about live snapshot features. Disable image locking because otherwise a few tests are going to fail because we reuse the same images at blockdev-add. Signed-off-by: Fam Zheng --- tests/qemu-iotests/085 | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff

[Qemu-block] [PATCH v7 18/20] tests: Use null-co:// instead of /dev/null

2016-08-08 Thread Fam Zheng
With image locking, opening /dev/null can fail when multiple tests run in parallel (make -j2, for example). Use null-co:// as the null protocol doesn't do image locking. While it's arguable we could special-case /dev/null, /dev/zero, /dev/urandom etc in raw-posix driver, it is not really

Re: [Qemu-block] [Qemu-devel] [PATCH v7 00/16] backup compression

2016-08-08 Thread Kevin Wolf
Am 08.08.2016 um 15:02 hat Stefan Hajnoczi geschrieben: > On Fri, Jul 22, 2016 at 11:17:39AM +0300, Denis V. Lunev wrote: > > The idea is simple - backup is "written-once" data. It is written block > > by block and it is large enough. It would be nice to save storage > > space and compress it. > >

[Qemu-block] [PATCH v7 16/20] iotests: 130: Check image info locklessly

2016-08-08 Thread Fam Zheng
By the time _img_info is run, QEMU process's resources may still be on its way being cleaned up, asynchronously, even though the process itself is already gone after the "kill -KILL" and "wait" commands in _cleanup_qemu. Change the last HMP command to 'q' to ensure the locks are released.

[Qemu-block] [PATCH v7 09/20] qemu-img: Update documentation of "-L" option

2016-08-08 Thread Fam Zheng
Signed-off-by: Fam Zheng --- qemu-img-cmds.hx | 44 ++-- qemu-img.c | 1 + qemu-img.texi| 3 +++ 3 files changed, 26 insertions(+), 22 deletions(-) diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx index 7e95b2d..11bfa96 100644

Re: [Qemu-block] [PATCH v5 00/11] block: Accept node-name in all node level QMP commands

2016-08-08 Thread Kevin Wolf
Am 03.08.2016 um 13:21 hat Kevin Wolf geschrieben: > As stated in the RFC I sent two weeks ago: > >* Node level commands: We need to complete the conversion that makes > commands accept node names instead of BlockBackend names. In some places > we intentionally allow only

[Qemu-block] [PATCH v7 10/20] qemu-nbd: Add "--no-lock/-L" option

2016-08-08 Thread Fam Zheng
Signed-off-by: Fam Zheng Reviewed-by: Max Reitz --- qemu-nbd.c| 7 ++- qemu-nbd.texi | 2 ++ 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/qemu-nbd.c b/qemu-nbd.c index e3571c2..145156c 100644 --- a/qemu-nbd.c +++ b/qemu-nbd.c @@

[Qemu-block] [PATCH v7 15/20] iotests: 087: Disable image locking in cases where file is shared

2016-08-08 Thread Fam Zheng
Otherwise the error handling we are expecting will be masked by the preceding image locking check, and is going to be indistinguishable because the error messages are all the same. Signed-off-by: Fam Zheng Reviewed-by: Max Reitz --- tests/qemu-iotests/087 |

[Qemu-block] [PATCH v7 11/20] block: Don't lock drive-backup target image in none mode

2016-08-08 Thread Fam Zheng
As a very special case, in sync=none mode, the source is the backing image of the target, which will be RO opened again. This won't work with image locking because the first open could be exclusive. Signed-off-by: Fam Zheng Reviewed-by: Max Reitz ---

[Qemu-block] [PATCH v7 06/20] raw-posix: Add image locking support

2016-08-08 Thread Fam Zheng
virtlockd in libvirt locks the first byte, we lock byte 1 to avoid the intervene. Both file and host device protocols are covered. The complication is with reopen. We have three different locking states, namely "unlocked", "shared locked" and "exclusively locked". When we reopen, the new fd may

[Qemu-block] [PATCH v7 04/20] block: Introduce image file locking

2016-08-08 Thread Fam Zheng
Block drivers can implement this new operation .bdrv_lockf to actually lock the image in the protocol specific way. Signed-off-by: Fam Zheng --- block.c | 59 +++ include/block/block.h | 11 -

[Qemu-block] [PATCH v7 08/20] qemu-img: Add "-L" option to sub commands

2016-08-08 Thread Fam Zheng
If specified, BDRV_O_NO_LOCK flag will be set when opening the image. Signed-off-by: Fam Zheng --- qemu-img.c | 91 +- 1 file changed, 72 insertions(+), 19 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index

[Qemu-block] [PATCH v7 00/20] block: Image locking series for 2.8

2016-08-08 Thread Fam Zheng
v7: - Rebase. - Address comments from Kevin and Max. - Option rename: "exclusive" -> "auto". [Kevin] - Option placement: "root BDS" -> "node". It's still a bit controversy where the option should go, but so far it seems per node is the safest. - Assert in

[Qemu-block] [PATCH v7 05/20] osdep: Add qemu_lock_fd and qemu_unlock_fd

2016-08-08 Thread Fam Zheng
They are wrappers of POSIX fcntl "file private locking". Signed-off-by: Fam Zheng --- include/qemu/osdep.h | 2 ++ util/osdep.c | 29 + 2 files changed, 31 insertions(+) diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h index

[Qemu-block] [PATCH v7 03/20] block: Add and parse "lock-mode" option for image locking

2016-08-08 Thread Fam Zheng
Respect the locking mode from CLI or QMP, and set the open flags accordingly. Signed-off-by: Fam Zheng --- block.c | 21 + blockdev.c| 9 + include/block/block.h | 1 + qemu-options.hx | 1 + 4 files changed, 32

Re: [Qemu-block] [Qemu-devel] [PATCH v7 00/16] backup compression

2016-08-08 Thread Denis V. Lunev
On 08/08/2016 04:02 PM, Stefan Hajnoczi wrote: On Fri, Jul 22, 2016 at 11:17:39AM +0300, Denis V. Lunev wrote: The idea is simple - backup is "written-once" data. It is written block by block and it is large enough. It would be nice to save storage space and compress it. These patches add the

Re: [Qemu-block] [Qemu-devel] [PATCH v7 00/16] backup compression

2016-08-08 Thread Stefan Hajnoczi
On Fri, Jul 22, 2016 at 11:17:39AM +0300, Denis V. Lunev wrote: > The idea is simple - backup is "written-once" data. It is written block > by block and it is large enough. It would be nice to save storage > space and compress it. > > These patches add the ability to compress data during backup.

[Qemu-block] [PULL 2/4] tests: Test blockjob IDs

2016-08-08 Thread Kevin Wolf
From: Alberto Garcia Since 7f0317cfc8da6 we have API to specify the ID of block jobs and we also guarantee that they are well-formed and unique. This patch adds tests to check some common scenarios. Signed-off-by: Alberto Garcia Signed-off-by: Kevin Wolf

[Qemu-block] [PULL 0/4] More block layer patches for 2.7.0-rc2

2016-08-08 Thread Kevin Wolf
The following changes since commit cf5198d58088e3b416fe517b3946e5120b342761: Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20160805' into staging (2016-08-08 10:39:18 +0100) are available in the git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream for you to

[Qemu-block] [PULL 4/4] iotests: fix 109

2016-08-08 Thread Kevin Wolf
From: Vladimir Sementsov-Ogievskiy 109 iotest is broken for raw after 0965a41e998ab820b5 [mirror: double performance of the bulk stage if the disc is full] The problem is with finishing block-job with error: before specified patch mirror was not very async and it

[Qemu-block] [PATCH v4] qemu-img: add skip option to dd

2016-08-08 Thread Reda Sallahi
This adds the skip option which allows qemu-img dd to skip a number of blocks before copying the input. A test case was added to test the skip option. Signed-off-by: Reda Sallahi --- Depends on: [PATCH v6] qemu-img: add the 'dd' subcommand Changes from v3: * Change write

Re: [Qemu-block] [Qemu-devel] [PATCH v2 1/8] util: Add UUID API

2016-08-08 Thread Markus Armbruster
Fam Zheng writes: > A number of different places across the code base use CONFIG_UUID. Some > of them are soft dependency, some are not built if libuuid is not > available, some come with dummy fallback, some throws runtime error. > > It is hard to maintain, and hard to reason

Re: [Qemu-block] [Qemu-devel] [PATCH 1/7] util: Add UUID API

2016-08-08 Thread Markus Armbruster
Stefan Weil writes: > Am 08.08.2016 um 08:33 schrieb Fam Zheng: >> On Mon, 08/08 08:30, Stefan Weil wrote: >>> Am 02.08.2016 um 11:18 schrieb Fam Zheng: >>> [...] +void qemu_uuid_generate(qemu_uuid_t out) +{ +/* Version 4 UUID, RFC4122 4.4. */ +

Re: [Qemu-block] [PATCH 0/2] fix [mirror: double performance...]

2016-08-08 Thread Kevin Wolf
Am 03.08.2016 um 14:56 hat Vladimir Sementsov-Ogievskiy geschrieben: > Hi all. > > Here are some fixes related to my > [mirror: double performance of the bulk stage if the disc is full] > > As Kevin noted, iotest 109 is broken after [mirror: double perf..]. > I'm sorry about it, 0002 fixes this

Re: [Qemu-block] [PATCH v2 2/8] tests: Add uuid tests

2016-08-08 Thread Daniel P. Berrange
On Mon, Aug 08, 2016 at 02:09:22PM +0800, Fam Zheng wrote: > Signed-off-by: Fam Zheng > --- > tests/Makefile.include | 2 + > tests/test-uuid.c | 110 > + > 2 files changed, 112 insertions(+) > create mode 100644

Re: [Qemu-block] [PATCH 1/7] util: Add UUID API

2016-08-08 Thread Stefan Weil
Am 08.08.2016 um 08:33 schrieb Fam Zheng: > On Mon, 08/08 08:30, Stefan Weil wrote: >> Am 02.08.2016 um 11:18 schrieb Fam Zheng: >> [...] >>> +void qemu_uuid_generate(qemu_uuid_t out) >>> +{ >>> +/* Version 4 UUID, RFC4122 4.4. */ >>> +QEMU_BUILD_BUG_ON(sizeof(qemu_uuid_t) != 16); >>> +

Re: [Qemu-block] [PATCH 1/7] util: Add UUID API

2016-08-08 Thread Fam Zheng
On Mon, 08/08 08:30, Stefan Weil wrote: > Am 02.08.2016 um 11:18 schrieb Fam Zheng: > [...] > > +void qemu_uuid_generate(qemu_uuid_t out) > > +{ > > +/* Version 4 UUID, RFC4122 4.4. */ > > +QEMU_BUILD_BUG_ON(sizeof(qemu_uuid_t) != 16); > > +*((guint32 *)out + 0) = g_random_int(); > > +

Re: [Qemu-block] [PATCH 1/7] util: Add UUID API

2016-08-08 Thread Stefan Weil
Am 02.08.2016 um 11:18 schrieb Fam Zheng: [...] > +void qemu_uuid_generate(qemu_uuid_t out) > +{ > +/* Version 4 UUID, RFC4122 4.4. */ > +QEMU_BUILD_BUG_ON(sizeof(qemu_uuid_t) != 16); > +*((guint32 *)out + 0) = g_random_int(); > +*((guint32 *)out + 1) = g_random_int(); > +

[Qemu-block] [PATCH v2 8/8] configure: Remove detection code for UUID

2016-08-08 Thread Fam Zheng
All code now uses built-in UUID implementation. Remove the code of libuuid and make --enable-uuid and --disable-uuid only print a message. Signed-off-by: Fam Zheng --- configure | 43 --- 1 file changed, 4 insertions(+), 39 deletions(-)

[Qemu-block] [PATCH v2 7/8] tests: No longer dependent on CONFIG_UUID

2016-08-08 Thread Fam Zheng
crypto now uses built-in uuid implementation, so this check is not needed. Signed-off-by: Fam Zheng --- tests/test-crypto-block.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/test-crypto-block.c b/tests/test-crypto-block.c index a38110d..1957a86

[Qemu-block] [PATCH v2 6/8] crypto: Switch to QEMU UUID API

2016-08-08 Thread Fam Zheng
The uuid generation doesn't return error, so update the function signature and calling code accordingly. Signed-off-by: Fam Zheng --- crypto/block-luks.c | 26 +++--- 1 file changed, 7 insertions(+), 19 deletions(-) diff --git a/crypto/block-luks.c

[Qemu-block] [PATCH v2 5/8] vpc: Use QEMU UUID API

2016-08-08 Thread Fam Zheng
Previously we conditionally generate if footer->uuid, when libuuid is available. Now that we have a built-in implementation, we can switch to it. Signed-off-by: Fam Zheng --- block/vpc.c | 8 ++-- 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/block/vpc.c

[Qemu-block] [PATCH v2 3/8] vhdx: Use QEMU UUID API

2016-08-08 Thread Fam Zheng
This removes our dependency to libuuid, so that the driver can always be built. Similar to how we handled data plane configure options, --enable-vhdx and --disable-vhdx are also changed to a nop with a message saying it's obsolete. Signed-off-by: Fam Zheng ---

[Qemu-block] [PATCH v2 4/8] vdi: Use QEMU UUID API

2016-08-08 Thread Fam Zheng
The QEMU UUID api, including the data structure (QemuUUID), is fully compatible with libuuid. Use it, and remove the unused code. Signed-off-by: Fam Zheng Reviewed-by: Stefan Weil --- block/vdi.c | 49 ++--- 1 file

[Qemu-block] [PATCH v2 0/8] UUID clean ups for 2.8

2016-08-08 Thread Fam Zheng
v2: Fix the endianness wrongness. [Jeff] Add tests/test-uuid.c. [Denial] Add r-b of Stefan Weil to the VDI patch. The facts how we use libuuid now are not particularly pleasant. - VHDX driver depends on uuid, but is unconditionally checked in iotests 109. If it is not built, the test

[Qemu-block] [PATCH v2 1/8] util: Add UUID API

2016-08-08 Thread Fam Zheng
A number of different places across the code base use CONFIG_UUID. Some of them are soft dependency, some are not built if libuuid is not available, some come with dummy fallback, some throws runtime error. It is hard to maintain, and hard to reason for users. Since UUID is a simple standard

[Qemu-block] [PATCH v2 2/8] tests: Add uuid tests

2016-08-08 Thread Fam Zheng
Signed-off-by: Fam Zheng --- tests/Makefile.include | 2 + tests/test-uuid.c | 110 + 2 files changed, 112 insertions(+) create mode 100644 tests/test-uuid.c diff --git a/tests/Makefile.include b/tests/Makefile.include

  1   2   >