Re: [Qemu-devel] [PATCH V9 00/11] Quorum block driver
Le Friday 04 Oct 2013 à 16:31:56 (+0200), Max Reitz a écrit : On 2013-10-02 14:39, Benoît Canet wrote: It must be applied on top of block: Add BlockDriver.bdrv_check_ext_snapshot. This patchset create a block driver implementing a quorum using total qemu disk images. Writes are mirrored on the $total files. For the reading part the $total files are read at the same time and a vote is done to determine if a qiov version is present $threshold or more times. It then return this majority version to the upper layers. When i $threshold versions of the data are returned by the lower layer the quorum is broken and the read return -EIO. The goal of this patchset is to be turned in a QEMU block filter living just above raw-*.c and below qcow2/qed when the required infrastructure will be done. Main use of this feature will be people using NFS appliances which can be subjected to bitflip errors. This patchset can be used to replace blkverify and the out of tree blkmirror. v9: integrate latests comments from kevin Benoît Canet (11): quorum: Create quorum.c, add QuorumSingleAIOCB and QuorumAIOCB. quorum: Create BDRVQuorumState and BlkDriver and do init. quorum: Add quorum_aio_writev and its dependencies. blkverify: Extract qemu_iovec_clone() and qemu_iovec_compare() from blkverify. quorum: Add quorum_aio_readv. quorum: Add quorum mechanism. quorum: Add quorum_getlength(). quorum: Add quorum_invalidate_cache(). quorum: Add quorum_co_get_block_status. quorum: Add quorum_co_flush(). quorum: Add quorum_open() and quorum_close(). block/Makefile.objs |1 + block/blkverify.c | 108 + block/quorum.c| 1035 + configure | 36 ++ include/monitor/monitor.h |2 + include/qemu-common.h |2 + monitor.c |2 + util/iov.c| 103 + 8 files changed, 1183 insertions(+), 106 deletions(-) create mode 100644 block/quorum.c Could you add a qemu-iotest if possible? I don't see how to specify quorum to qemu-io for bytes checking. Best regards Benoît Max
Re: [Qemu-devel] [PATCH V9 00/11] Quorum block driver
Am 29.10.2013 08:59, schrieb Benoît Canet: Le Friday 04 Oct 2013 à 16:31:56 (+0200), Max Reitz a écrit : On 2013-10-02 14:39, Benoît Canet wrote: It must be applied on top of block: Add BlockDriver.bdrv_check_ext_snapshot. This patchset create a block driver implementing a quorum using total qemu disk images. Writes are mirrored on the $total files. For the reading part the $total files are read at the same time and a vote is done to determine if a qiov version is present $threshold or more times. It then return this majority version to the upper layers. When i $threshold versions of the data are returned by the lower layer the quorum is broken and the read return -EIO. The goal of this patchset is to be turned in a QEMU block filter living just above raw-*.c and below qcow2/qed when the required infrastructure will be done. Main use of this feature will be people using NFS appliances which can be subjected to bitflip errors. This patchset can be used to replace blkverify and the out of tree blkmirror. v9: integrate latests comments from kevin Benoît Canet (11): quorum: Create quorum.c, add QuorumSingleAIOCB and QuorumAIOCB. quorum: Create BDRVQuorumState and BlkDriver and do init. quorum: Add quorum_aio_writev and its dependencies. blkverify: Extract qemu_iovec_clone() and qemu_iovec_compare() from blkverify. quorum: Add quorum_aio_readv. quorum: Add quorum mechanism. quorum: Add quorum_getlength(). quorum: Add quorum_invalidate_cache(). quorum: Add quorum_co_get_block_status. quorum: Add quorum_co_flush(). quorum: Add quorum_open() and quorum_close(). block/Makefile.objs |1 + block/blkverify.c | 108 + block/quorum.c| 1035 + configure | 36 ++ include/monitor/monitor.h |2 + include/qemu-common.h |2 + monitor.c |2 + util/iov.c| 103 + 8 files changed, 1183 insertions(+), 106 deletions(-) create mode 100644 block/quorum.c Could you add a qemu-iotest if possible? I don't see how to specify quorum to qemu-io for bytes checking. Best regards Benoît As of commit b543c5cd, you can pass options to the block driver through qemu-io -c open. The problem for quorum is that it cannot handle the filename option (which qemu-io will always set). I've attached a diff to change this behavior (which makes the path parameter for open optional and passes NULL to bdrv_open if it is not given). You're free to make use of it. ;-) If this diff is applied, you can invoke qemu-io with quorum for example as following: qemu-io -c 'open -o file.driver=quorum,file.children.0.file.filename=[Image 1],file.children.1.file.filename=[Image 2],file.vote_threshold=[x]' Max
Re: [Qemu-devel] [PATCH V9 00/11] Quorum block driver
Am 29.10.2013 18:55, schrieb Max Reitz:
Am 29.10.2013 08:59, schrieb Benoît Canet:
Le Friday 04 Oct 2013 à 16:31:56 (+0200), Max Reitz a écrit :
On 2013-10-02 14:39, Benoît Canet wrote:
It must be applied on top of block: Add
BlockDriver.bdrv_check_ext_snapshot.
This patchset create a block driver implementing a quorum using total qemu
disk
images. Writes are mirrored on the $total files.
For the reading part the $total files are read at the same time and a vote
is
done to determine if a qiov version is present $threshold or more times.
It then
return this majority version to the upper layers.
When i $threshold versions of the data are returned by the lower layer
the
quorum is broken and the read return -EIO.
The goal of this patchset is to be turned in a QEMU block filter living
just
above raw-*.c and below qcow2/qed when the required infrastructure will be
done.
Main use of this feature will be people using NFS appliances which can be
subjected to bitflip errors.
This patchset can be used to replace blkverify and the out of tree
blkmirror.
v9: integrate latests comments from kevin
Benoît Canet (11):
quorum: Create quorum.c, add QuorumSingleAIOCB and QuorumAIOCB.
quorum: Create BDRVQuorumState and BlkDriver and do init.
quorum: Add quorum_aio_writev and its dependencies.
blkverify: Extract qemu_iovec_clone() and qemu_iovec_compare() from
blkverify.
quorum: Add quorum_aio_readv.
quorum: Add quorum mechanism.
quorum: Add quorum_getlength().
quorum: Add quorum_invalidate_cache().
quorum: Add quorum_co_get_block_status.
quorum: Add quorum_co_flush().
quorum: Add quorum_open() and quorum_close().
block/Makefile.objs |1 +
block/blkverify.c | 108 +
block/quorum.c| 1035
+
configure | 36 ++
include/monitor/monitor.h |2 +
include/qemu-common.h |2 +
monitor.c |2 +
util/iov.c| 103 +
8 files changed, 1183 insertions(+), 106 deletions(-)
create mode 100644 block/quorum.c
Could you add a qemu-iotest if possible?
I don't see how to specify quorum to qemu-io for bytes checking.
Best regards
Benoît
As of commit b543c5cd, you can pass options to the block driver through
qemu-io -c open. The problem for quorum is that it cannot handle the
filename option (which qemu-io will always set). I've attached a diff to
change this behavior (which makes the path parameter for open optional
and passes NULL to bdrv_open if it is not given). You're free to make
use of it. ;-)
If this diff is applied, you can invoke qemu-io with quorum for example
as following:
qemu-io -c 'open -o
file.driver=quorum,file.children.0.file.filename=[Image
1],file.children.1.file.filename=[Image 2],file.vote_threshold=[x]'
Max
Obviously forgot to attach the diff.
Max
diff --git a/qemu-io.c b/qemu-io.c
index 3b3340a..4d0e9f6 100644
--- a/qemu-io.c
+++ b/qemu-io.c
@@ -57,8 +57,13 @@ static int openfile(char *name, int flags, int growable,
QDict *opts)
if (growable) {
if (bdrv_file_open(qemuio_bs, name, opts, flags, local_err)) {
-fprintf(stderr, %s: can't open device %s: %s\n, progname, name,
-error_get_pretty(local_err));
+if (name) {
+fprintf(stderr, %s: can't open device %s: %s\n, progname,
+name, error_get_pretty(local_err));
+} else {
+fprintf(stderr, %s: can't open device: %s\n, progname,
+error_get_pretty(local_err));
+}
error_free(local_err);
return 1;
}
@@ -66,8 +71,13 @@ static int openfile(char *name, int flags, int growable,
QDict *opts)
qemuio_bs = bdrv_new(hda);
if (bdrv_open(qemuio_bs, name, opts, flags, NULL, local_err) 0) {
-fprintf(stderr, %s: can't open device %s: %s\n, progname, name,
-error_get_pretty(local_err));
+if (name) {
+fprintf(stderr, %s: can't open device %s: %s\n, progname,
+name, error_get_pretty(local_err));
+} else {
+fprintf(stderr, %s: can't open device: %s\n, progname,
+error_get_pretty(local_err));
+}
error_free(local_err);
bdrv_unref(qemuio_bs);
qemuio_bs = NULL;
@@ -127,6 +137,7 @@ static int open_f(BlockDriverState *bs, int argc, char
**argv)
int c;
QemuOpts *qopts;
QDict *opts = NULL;
+char *filename = NULL;
while ((c = getopt(argc, argv, snrgo:)) != EOF) {
switch (c) {
@@ -160,11 +171,13 @@ static int open_f(BlockDriverState *bs, int argc, char
**argv)
flags |= BDRV_O_RDWR;
}
-if (optind != argc - 1) {
+if (optind == argc - 1) {
+filename = argv[optind];
+} else if (optind
Re: [Qemu-devel] [PATCH V9 00/11] Quorum block driver
Le Tuesday 29 Oct 2013 à 18:56:34 (+0100), Max Reitz a écrit :
Am 29.10.2013 18:55, schrieb Max Reitz:
Am 29.10.2013 08:59, schrieb Benoît Canet:
Le Friday 04 Oct 2013 à 16:31:56 (+0200), Max Reitz a écrit :
On 2013-10-02 14:39, Benoît Canet wrote:
It must be applied on top of block: Add
BlockDriver.bdrv_check_ext_snapshot.
This patchset create a block driver implementing a quorum using total
qemu disk
images. Writes are mirrored on the $total files.
For the reading part the $total files are read at the same time and a
vote is
done to determine if a qiov version is present $threshold or more times.
It then
return this majority version to the upper layers.
When i $threshold versions of the data are returned by the lower layer
the
quorum is broken and the read return -EIO.
The goal of this patchset is to be turned in a QEMU block filter living
just
above raw-*.c and below qcow2/qed when the required infrastructure will
be done.
Main use of this feature will be people using NFS appliances which can be
subjected to bitflip errors.
This patchset can be used to replace blkverify and the out of tree
blkmirror.
v9: integrate latests comments from kevin
Benoît Canet (11):
quorum: Create quorum.c, add QuorumSingleAIOCB and QuorumAIOCB.
quorum: Create BDRVQuorumState and BlkDriver and do init.
quorum: Add quorum_aio_writev and its dependencies.
blkverify: Extract qemu_iovec_clone() and qemu_iovec_compare() from
blkverify.
quorum: Add quorum_aio_readv.
quorum: Add quorum mechanism.
quorum: Add quorum_getlength().
quorum: Add quorum_invalidate_cache().
quorum: Add quorum_co_get_block_status.
quorum: Add quorum_co_flush().
quorum: Add quorum_open() and quorum_close().
block/Makefile.objs |1 +
block/blkverify.c | 108 +
block/quorum.c| 1035
+
configure | 36 ++
include/monitor/monitor.h |2 +
include/qemu-common.h |2 +
monitor.c |2 +
util/iov.c| 103 +
8 files changed, 1183 insertions(+), 106 deletions(-)
create mode 100644 block/quorum.c
Could you add a qemu-iotest if possible?
I don't see how to specify quorum to qemu-io for bytes checking.
Best regards
Benoît
As of commit b543c5cd, you can pass options to the block driver through
qemu-io -c open. The problem for quorum is that it cannot handle the
filename option (which qemu-io will always set). I've attached a diff to
change this behavior (which makes the path parameter for open optional
and passes NULL to bdrv_open if it is not given). You're free to make
use of it. ;-)
If this diff is applied, you can invoke qemu-io with quorum for example
as following:
qemu-io -c 'open -o
file.driver=quorum,file.children.0.file.filename=[Image
1],file.children.1.file.filename=[Image 2],file.vote_threshold=[x]'
Max
Obviously forgot to attach the diff.
Max
diff --git a/qemu-io.c b/qemu-io.c
index 3b3340a..4d0e9f6 100644
--- a/qemu-io.c
+++ b/qemu-io.c
@@ -57,8 +57,13 @@ static int openfile(char *name, int flags, int growable,
QDict *opts)
if (growable) {
if (bdrv_file_open(qemuio_bs, name, opts, flags, local_err)) {
-fprintf(stderr, %s: can't open device %s: %s\n, progname, name,
-error_get_pretty(local_err));
+if (name) {
+fprintf(stderr, %s: can't open device %s: %s\n, progname,
+name, error_get_pretty(local_err));
+} else {
+fprintf(stderr, %s: can't open device: %s\n, progname,
+error_get_pretty(local_err));
+}
error_free(local_err);
return 1;
}
@@ -66,8 +71,13 @@ static int openfile(char *name, int flags, int growable,
QDict *opts)
qemuio_bs = bdrv_new(hda);
if (bdrv_open(qemuio_bs, name, opts, flags, NULL, local_err) 0) {
-fprintf(stderr, %s: can't open device %s: %s\n, progname, name,
-error_get_pretty(local_err));
+if (name) {
+fprintf(stderr, %s: can't open device %s: %s\n, progname,
+name, error_get_pretty(local_err));
+} else {
+fprintf(stderr, %s: can't open device: %s\n, progname,
+error_get_pretty(local_err));
+}
error_free(local_err);
bdrv_unref(qemuio_bs);
qemuio_bs = NULL;
@@ -127,6 +137,7 @@ static int open_f(BlockDriverState *bs, int argc, char
**argv)
int c;
QemuOpts *qopts;
QDict *opts = NULL;
+char *filename = NULL;
while ((c = getopt(argc, argv, snrgo:)) != EOF) {
switch (c) {
@@ -160,11 +171,13 @@ static int
Re: [Qemu-devel] [PATCH V9 00/11] Quorum block driver
On 2013-10-02 14:39, Benoît Canet wrote: It must be applied on top of block: Add BlockDriver.bdrv_check_ext_snapshot. This patchset create a block driver implementing a quorum using total qemu disk images. Writes are mirrored on the $total files. For the reading part the $total files are read at the same time and a vote is done to determine if a qiov version is present $threshold or more times. It then return this majority version to the upper layers. When i $threshold versions of the data are returned by the lower layer the quorum is broken and the read return -EIO. The goal of this patchset is to be turned in a QEMU block filter living just above raw-*.c and below qcow2/qed when the required infrastructure will be done. Main use of this feature will be people using NFS appliances which can be subjected to bitflip errors. This patchset can be used to replace blkverify and the out of tree blkmirror. v9: integrate latests comments from kevin Benoît Canet (11): quorum: Create quorum.c, add QuorumSingleAIOCB and QuorumAIOCB. quorum: Create BDRVQuorumState and BlkDriver and do init. quorum: Add quorum_aio_writev and its dependencies. blkverify: Extract qemu_iovec_clone() and qemu_iovec_compare() from blkverify. quorum: Add quorum_aio_readv. quorum: Add quorum mechanism. quorum: Add quorum_getlength(). quorum: Add quorum_invalidate_cache(). quorum: Add quorum_co_get_block_status. quorum: Add quorum_co_flush(). quorum: Add quorum_open() and quorum_close(). block/Makefile.objs |1 + block/blkverify.c | 108 + block/quorum.c| 1035 + configure | 36 ++ include/monitor/monitor.h |2 + include/qemu-common.h |2 + monitor.c |2 + util/iov.c| 103 + 8 files changed, 1183 insertions(+), 106 deletions(-) create mode 100644 block/quorum.c Could you add a qemu-iotest if possible? Max
Re: [Qemu-devel] [PATCH V9 00/11] Quorum block driver
Le Friday 04 Oct 2013 à 16:31:56 (+0200), Max Reitz a écrit : On 2013-10-02 14:39, Benoît Canet wrote: It must be applied on top of block: Add BlockDriver.bdrv_check_ext_snapshot. This patchset create a block driver implementing a quorum using total qemu disk images. Writes are mirrored on the $total files. For the reading part the $total files are read at the same time and a vote is done to determine if a qiov version is present $threshold or more times. It then return this majority version to the upper layers. When i $threshold versions of the data are returned by the lower layer the quorum is broken and the read return -EIO. The goal of this patchset is to be turned in a QEMU block filter living just above raw-*.c and below qcow2/qed when the required infrastructure will be done. Main use of this feature will be people using NFS appliances which can be subjected to bitflip errors. This patchset can be used to replace blkverify and the out of tree blkmirror. v9: integrate latests comments from kevin Benoît Canet (11): quorum: Create quorum.c, add QuorumSingleAIOCB and QuorumAIOCB. quorum: Create BDRVQuorumState and BlkDriver and do init. quorum: Add quorum_aio_writev and its dependencies. blkverify: Extract qemu_iovec_clone() and qemu_iovec_compare() from blkverify. quorum: Add quorum_aio_readv. quorum: Add quorum mechanism. quorum: Add quorum_getlength(). quorum: Add quorum_invalidate_cache(). quorum: Add quorum_co_get_block_status. quorum: Add quorum_co_flush(). quorum: Add quorum_open() and quorum_close(). block/Makefile.objs |1 + block/blkverify.c | 108 + block/quorum.c| 1035 + configure | 36 ++ include/monitor/monitor.h |2 + include/qemu-common.h |2 + monitor.c |2 + util/iov.c| 103 + 8 files changed, 1183 insertions(+), 106 deletions(-) create mode 100644 block/quorum.c Could you add a qemu-iotest if possible? Thanks for the review Max. I will iterate. Best regards Benoît Max
[Qemu-devel] [PATCH V9 00/11] Quorum block driver
It must be applied on top of block: Add BlockDriver.bdrv_check_ext_snapshot. This patchset create a block driver implementing a quorum using total qemu disk images. Writes are mirrored on the $total files. For the reading part the $total files are read at the same time and a vote is done to determine if a qiov version is present $threshold or more times. It then return this majority version to the upper layers. When i $threshold versions of the data are returned by the lower layer the quorum is broken and the read return -EIO. The goal of this patchset is to be turned in a QEMU block filter living just above raw-*.c and below qcow2/qed when the required infrastructure will be done. Main use of this feature will be people using NFS appliances which can be subjected to bitflip errors. This patchset can be used to replace blkverify and the out of tree blkmirror. v9: integrate latests comments from kevin Benoît Canet (11): quorum: Create quorum.c, add QuorumSingleAIOCB and QuorumAIOCB. quorum: Create BDRVQuorumState and BlkDriver and do init. quorum: Add quorum_aio_writev and its dependencies. blkverify: Extract qemu_iovec_clone() and qemu_iovec_compare() from blkverify. quorum: Add quorum_aio_readv. quorum: Add quorum mechanism. quorum: Add quorum_getlength(). quorum: Add quorum_invalidate_cache(). quorum: Add quorum_co_get_block_status. quorum: Add quorum_co_flush(). quorum: Add quorum_open() and quorum_close(). block/Makefile.objs |1 + block/blkverify.c | 108 + block/quorum.c| 1035 + configure | 36 ++ include/monitor/monitor.h |2 + include/qemu-common.h |2 + monitor.c |2 + util/iov.c| 103 + 8 files changed, 1183 insertions(+), 106 deletions(-) create mode 100644 block/quorum.c -- 1.8.1.2
