Re: [Qemu-block] [Qemu-devel] [PATCH for-2.6?] qemu-iotests: iotests: fail hard if not run via "check"

2016-04-19 Thread Markus Armbruster
Max Reitz writes: > On 19.04.2016 14:22, Sascha Silbe wrote: >> Dear Max, >> >> Max Reitz writes: >> >>> On 14.04.2016 13:32, Sascha Silbe wrote: >> [tests/qemu-iotests/iotests.py] >> [...] def main(supported_fmts=[], supported_oses=['linux']): '''Run tests''' +

Re: [Qemu-block] [PATCH for-2.6 1/2] block/mirror: Revive dead yielding code

2016-04-19 Thread Fam Zheng
On Wed, 04/20 00:59, Max Reitz wrote: > mirror_iteration() is supposed to wait if the current chunk is subject > to a still in-flight mirroring operation. However, it mixed checking > this conflict situation with checking the dirty status of a chunk. A > simplification for the latter condition (the

Re: [Qemu-block] [PATCH for-2.6 v2 0/3] Bug fixes for gluster

2016-04-19 Thread Raghavendra Gowdappa
- Original Message - > From: "Jeff Cody" > To: "Ric Wheeler" > Cc: qemu-block@nongnu.org, qemu-de...@nongnu.org, kw...@redhat.com, > pkara...@redhat.com, rgowd...@redhat.com, > nde...@redhat.com, "Rik van Riel" > Sent: Tuesday, April 19, 2016 7:39:17 PM > Subject: Re: [PATCH for-2.6 v

Re: [Qemu-block] [PATCH v13 2/3] quorum: implement bdrv_add_child() and bdrv_del_child()

2016-04-19 Thread Changlong Xie
ping... On 04/13/2016 04:33 PM, Changlong Xie wrote: From: Wen Congyang Signed-off-by: Wen Congyang Signed-off-by: zhanghailiang Signed-off-by: Gonglei Signed-off-by: Changlong Xie --- block.c | 8 +++--- block/quorum.c| 78

[Qemu-block] [PATCH for-2.6 v2 3/3] iotests: Test case for drive-mirror with unaligned image size

2016-04-19 Thread Fam Zheng
This is the regression test for the virtual size mismatch issue between target and source images. Reviewed-by: Max Reitz Signed-off-by: Fam Zheng --- tests/qemu-iotests/152 | 51 ++ tests/qemu-iotests/152.out | 5 + tests/qemu-iotests/group

[Qemu-block] [PATCH for-2.6 v2 2/3] iotests: Add iotests.image_size

2016-04-19 Thread Fam Zheng
This retrieves the virtual size of the image out of qemu-img info. Reviewed-by: Max Reitz Signed-off-by: Fam Zheng --- tests/qemu-iotests/iotests.py | 6 ++ 1 file changed, 6 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index d9ef60e..56f988a 100

[Qemu-block] [PATCH for-2.6 v2 1/3] mirror: Don't extend the last sub-chunk

2016-04-19 Thread Fam Zheng
The last sub-chunk is rounded up to the copy granularity in the target image, resulting in a larger size than the source. Add a function to clip the copied sectors to the end. This undoes the "wrong" changes to tests/qemu-iotests/109.out in e5b43573e28. The remaining two offset changes are okay.

[Qemu-block] [PATCH for-2.6 v2 0/3] block: Fix drive-mirror with image size unaligned with granularity

2016-04-19 Thread Fam Zheng
v2: Move the mirror_clip_sectors() to mirror_iteration. [Max] This fixes the bug introduced in e5b43573e28 and lately noticed by Kevin. Fam Zheng (3): mirror: Don't extend the last sub-chunk iotests: Add iotests.image_size iotests: Test case for drive-mirror with unaligned image size bloc

Re: [Qemu-block] [PATCH for-2.6 v2 0/3] Bug fixes for gluster

2016-04-19 Thread Ric Wheeler
On 04/19/2016 10:09 AM, Jeff Cody wrote: On Tue, Apr 19, 2016 at 08:18:39AM -0400, Ric Wheeler wrote: On 04/19/2016 08:07 AM, Jeff Cody wrote: Bug fixes for gluster; third patch is to prevent a potential data loss when trying to recover from a recoverable error (such as ENOSPC). Hi Jeff, Just

Re: [Qemu-block] [PATCH 1/3] mirror: Don't extend the last sub-chunk

2016-04-19 Thread Fam Zheng
On Tue, 04/19 22:33, Max Reitz wrote: > On 19.04.2016 12:09, Fam Zheng wrote: > > The last sub-chunk is rounded up to the copy granularity in the target > > image, resulting in a larger size than the source. > > > > Add a function to clip the copied sectors to the end. > > > > This undoes the "wr

[Qemu-block] [PATCH for-2.6 1/2] block/mirror: Revive dead yielding code

2016-04-19 Thread Max Reitz
mirror_iteration() is supposed to wait if the current chunk is subject to a still in-flight mirroring operation. However, it mixed checking this conflict situation with checking the dirty status of a chunk. A simplification for the latter condition (the first chunk encountered is always dirty) led

[Qemu-block] [PATCH for-2.6 0/2] block/mirror: Fix mirroring with guest I/O load

2016-04-19 Thread Max Reitz
Bug report: https://bugs.launchpad.net/qemu/+bug/1570134 If you are doing a mirror operation (I just tested with block-commit on the active layer, but I guess any mirroring will do) while the guest has rather heavy I/O load (or light I/O also, you just need to be more unlucky) will lead to the cac

[Qemu-block] [PATCH for-2.6 2/2] block/mirror: Refresh stale bitmap iterator cache

2016-04-19 Thread Max Reitz
If the drive's dirty bitmap is dirtied while the mirror operation is running, the cache of the iterator used by the mirror code may become stale and not contain all dirty bits. This only becomes an issue if we are looking for contiguously dirty chunks on the drive. In that case, we can easily dete

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Andy Lutomirski
On Tue, Apr 19, 2016 at 1:54 PM, Michael S. Tsirkin wrote: > On Tue, Apr 19, 2016 at 01:27:29PM -0700, Andy Lutomirski wrote: >> On Tue, Apr 19, 2016 at 1:16 PM, Michael S. Tsirkin wrote: >> > On Tue, Apr 19, 2016 at 11:01:38AM -0700, Andy Lutomirski wrote: >> >> On Tue, Apr 19, 2016 at 10:49 AM,

Re: [Qemu-block] [PATCH 3/3] iotests: Test case for drive-mirror with unaligned image size

2016-04-19 Thread Max Reitz
On 19.04.2016 12:09, Fam Zheng wrote: > This is the regression test for the virtual size mismatch issue between > target and source images. > > Signed-off-by: Fam Zheng > --- > tests/qemu-iotests/152 | 51 > ++ > tests/qemu-iotests/152.out | 5 ++

Re: [Qemu-block] [PATCH 2/3] iotests: Add iotests.image_size

2016-04-19 Thread Max Reitz
On 19.04.2016 12:09, Fam Zheng wrote: > This retrieves the virtual size of the image out of qemu-img info. > > Signed-off-by: Fam Zheng > --- > tests/qemu-iotests/iotests.py | 6 ++ > 1 file changed, 6 insertions(+) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signa

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Michael S. Tsirkin
On Tue, Apr 19, 2016 at 01:27:29PM -0700, Andy Lutomirski wrote: > On Tue, Apr 19, 2016 at 1:16 PM, Michael S. Tsirkin wrote: > > On Tue, Apr 19, 2016 at 11:01:38AM -0700, Andy Lutomirski wrote: > >> On Tue, Apr 19, 2016 at 10:49 AM, Michael S. Tsirkin > >> wrote: > >> > On Tue, Apr 19, 2016 at

Re: [Qemu-block] [PATCH 1/3] mirror: Don't extend the last sub-chunk

2016-04-19 Thread Max Reitz
On 19.04.2016 12:09, Fam Zheng wrote: > The last sub-chunk is rounded up to the copy granularity in the target > image, resulting in a larger size than the source. > > Add a function to clip the copied sectors to the end. > > This undoes the "wrong" changes to tests/qemu-iotests/109.out in > e5b4

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Andy Lutomirski
On Tue, Apr 19, 2016 at 1:16 PM, Michael S. Tsirkin wrote: > On Tue, Apr 19, 2016 at 11:01:38AM -0700, Andy Lutomirski wrote: >> On Tue, Apr 19, 2016 at 10:49 AM, Michael S. Tsirkin wrote: >> > On Tue, Apr 19, 2016 at 12:26:44PM -0400, David Woodhouse wrote: >> >> On Tue, 2016-04-19 at 19:20 +030

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Michael S. Tsirkin
On Tue, Apr 19, 2016 at 11:01:38AM -0700, Andy Lutomirski wrote: > On Tue, Apr 19, 2016 at 10:49 AM, Michael S. Tsirkin wrote: > > On Tue, Apr 19, 2016 at 12:26:44PM -0400, David Woodhouse wrote: > >> On Tue, 2016-04-19 at 19:20 +0300, Michael S. Tsirkin wrote: > >> > > >> > > I thought that PLATF

[Qemu-block] [PATCH for-2.6? v2] qemu-iotests: iotests: fail hard if not run via "check"

2016-04-19 Thread Sascha Silbe
Running an iotests-based Python test directly might appear to work, but may fail in subtle ways and is insecure: - It creates files with predictable file names in a world-writable location (/var/tmp). - Tests expect the environment to be set up by check. E.g. 041 and 055 may take the wrong co

Re: [Qemu-block] [PATCH for-2.6?] qemu-iotests: iotests: fail hard if not run via "check"

2016-04-19 Thread Sascha Silbe
Dear Max, Max Reitz writes: > On 19.04.2016 14:22, Sascha Silbe wrote: [...] >> # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to >> # indicate that we're not being run via "check". There may be >> # other things set up by "check" that individual test cases rely >> #

Re: [Qemu-block] [PATCH for-2.7 v2 04/17] block: Introduce image file locking

2016-04-19 Thread Max Reitz
On 18.04.2016 07:34, Denis V. Lunev wrote: > On 04/18/2016 04:33 AM, Fam Zheng wrote: >> On Sun, 04/17 01:29, Max Reitz wrote: >>> On 15.04.2016 05:27, Fam Zheng wrote: Block drivers can implement this new operation .bdrv_lockf to actually lock the image in the protocol specific way.

Re: [Qemu-block] [PATCH for-2.7 v2 04/17] block: Introduce image file locking

2016-04-19 Thread Max Reitz
On 18.04.2016 03:33, Fam Zheng wrote: > On Sun, 04/17 01:29, Max Reitz wrote: >> On 15.04.2016 05:27, Fam Zheng wrote: >>> Block drivers can implement this new operation .bdrv_lockf to actually lock >>> the >>> image in the protocol specific way. >>> >>> Signed-off-by: Fam Zheng >>> --- >>> bloc

Re: [Qemu-block] [PATCH for-2.6?] qemu-iotests: iotests: fail hard if not run via "check"

2016-04-19 Thread Max Reitz
On 19.04.2016 14:22, Sascha Silbe wrote: > Dear Max, > > Max Reitz writes: > >> On 14.04.2016 13:32, Sascha Silbe wrote: > [tests/qemu-iotests/iotests.py] > [...] >>> def main(supported_fmts=[], supported_oses=['linux']): >>> '''Run tests''' >>> >>> +if test_dir is None or qemu_defau

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Andy Lutomirski
On Tue, Apr 19, 2016 at 10:49 AM, Michael S. Tsirkin wrote: > On Tue, Apr 19, 2016 at 12:26:44PM -0400, David Woodhouse wrote: >> On Tue, 2016-04-19 at 19:20 +0300, Michael S. Tsirkin wrote: >> > >> > > I thought that PLATFORM served that purpose. Woudn't the host >> > > advertise PLATFORM suppor

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Michael S. Tsirkin
On Tue, Apr 19, 2016 at 12:26:44PM -0400, David Woodhouse wrote: > On Tue, 2016-04-19 at 19:20 +0300, Michael S. Tsirkin wrote: > > > > > I thought that PLATFORM served that purpose.  Woudn't the host > > > advertise PLATFORM support and, if the guest doesn't ack it, the host > > > device would sk

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread David Woodhouse
On Tue, 2016-04-19 at 19:20 +0300, Michael S. Tsirkin wrote: > > > I thought that PLATFORM served that purpose.  Woudn't the host > > advertise PLATFORM support and, if the guest doesn't ack it, the host > > device would skip translation?  Or is that problematic for vfio? > > Exactly that's probl

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.6?] qemu-iotests: iotests: fail hard if not run via "check"

2016-04-19 Thread Sascha Silbe
Dear Markus, Markus Armbruster writes: > Say you had an accurate way to find out whether we're running under > "check". You could then reject any attempt to run the test directly. > I'd oppose that. > > It's okay to have test wrapper scripts to configure the tests just so. > It's okay to tell p

[Qemu-block] [PULL 2/3] block/gluster: code movement of qemu_gluster_close()

2016-04-19 Thread Jeff Cody
Move qemu_gluster_close() further up in the file, in preparation for the next patch, to avoid a forward declaration. Signed-off-by: Jeff Cody --- block/gluster.c | 22 +++--- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/block/gluster.c b/block/gluster.c index b

[Qemu-block] [PULL 3/3] block/gluster: prevent data loss after i/o error

2016-04-19 Thread Jeff Cody
Upon receiving an I/O error after an fsync, by default gluster will dump its cache. However, QEMU will retry the fsync, which is especially useful when encountering errors such as ENOSPC when using the werror=stop option. When using caching with gluster, however, the last written data will be los

[Qemu-block] [PULL 0/3] Block patches for 2.6

2016-04-19 Thread Jeff Cody
The following changes since commit d4dffa4a3f51b10cc0b7e6e34431919cac7a318e: Merge remote-tracking branch 'remotes/armbru/tags/pull-fw_cfg-2016-04-19' into staging (2016-04-19 15:25:20 +0100) are available in the git repository at: g...@github.com:codyprime/qemu-kvm-jtc.git tags/block-pull

[Qemu-block] [PULL 1/3] block/gluster: return correct error value

2016-04-19 Thread Jeff Cody
Upon error, gluster will call the aio callback function with a ret value of -1, with errno set to the proper error value. If we set the acb->ret value to the return value in the callback, that results in every error being EPERM (i.e. 1). Instead, set it to the proper error result. Reviewed-by: N

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Michael S. Tsirkin
On Tue, Apr 19, 2016 at 09:12:03AM -0700, Andy Lutomirski wrote: > On Tue, Apr 19, 2016 at 9:09 AM, Michael S. Tsirkin wrote: > > On Tue, Apr 19, 2016 at 09:02:14AM -0700, Andy Lutomirski wrote: > >> On Tue, Apr 19, 2016 at 3:27 AM, Michael S. Tsirkin > >> wrote: > >> > On Mon, Apr 18, 2016 at 1

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Andy Lutomirski
On Tue, Apr 19, 2016 at 9:09 AM, Michael S. Tsirkin wrote: > On Tue, Apr 19, 2016 at 09:02:14AM -0700, Andy Lutomirski wrote: >> On Tue, Apr 19, 2016 at 3:27 AM, Michael S. Tsirkin wrote: >> > On Mon, Apr 18, 2016 at 12:24:15PM -0700, Andy Lutomirski wrote: >> >> On Mon, Apr 18, 2016 at 11:29 AM,

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Michael S. Tsirkin
On Tue, Apr 19, 2016 at 09:02:14AM -0700, Andy Lutomirski wrote: > On Tue, Apr 19, 2016 at 3:27 AM, Michael S. Tsirkin wrote: > > On Mon, Apr 18, 2016 at 12:24:15PM -0700, Andy Lutomirski wrote: > >> On Mon, Apr 18, 2016 at 11:29 AM, David Woodhouse > >> wrote: > >> > For x86, you *can* enable v

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Michael S. Tsirkin
On Tue, Apr 19, 2016 at 09:00:27AM -0700, Andy Lutomirski wrote: > On Apr 19, 2016 2:13 AM, "Michael S. Tsirkin" wrote: > > > > > > I guess you are right in that we should split this part out. > > What I wanted is really the combination > > PASSTHROUGH && !PLATFORM so that we can say "ok we don't

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Andy Lutomirski
On Tue, Apr 19, 2016 at 3:27 AM, Michael S. Tsirkin wrote: > On Mon, Apr 18, 2016 at 12:24:15PM -0700, Andy Lutomirski wrote: >> On Mon, Apr 18, 2016 at 11:29 AM, David Woodhouse >> wrote: >> > For x86, you *can* enable virtio-behind-IOMMU if your DMAR tables tell >> > the truth, and even legacy

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Andy Lutomirski
On Apr 19, 2016 2:13 AM, "Michael S. Tsirkin" wrote: > > > I guess you are right in that we should split this part out. > What I wanted is really the combination > PASSTHROUGH && !PLATFORM so that we can say "ok we don't > need to guess, this device actually bypasses the IOMMU". What happens when

Re: [Qemu-block] [PATCH for-2.6 v3 0/3] Bug fixes for gluster

2016-04-19 Thread Jeff Cody
On Tue, Apr 19, 2016 at 10:16:18AM -0400, Jeff Cody wrote: > Change from v2: > > Look at the error return from the aio callback from > fsync, as well. (thanks Kevin) > > > Bug fixes for gluster; third patch is to prevent > a potential data loss when trying to recover from > a recoverable error (

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Alex Williamson
On Tue, 19 Apr 2016 12:13:29 +0300 "Michael S. Tsirkin" wrote: > On Mon, Apr 18, 2016 at 02:29:33PM -0400, David Woodhouse wrote: > > On Mon, 2016-04-18 at 19:27 +0300, Michael S. Tsirkin wrote: > > > I balk at adding more hacks to a broken system. My goals are > > > merely to > > > - make thin

Re: [Qemu-block] [PATCH v4 0/8] bdrv_flush_io_queue removal, shared LinuxAioState

2016-04-19 Thread Kevin Wolf
Am 19.04.2016 um 11:10 hat Stefan Hajnoczi geschrieben: > On Thu, Apr 07, 2016 at 06:33:28PM +0200, Paolo Bonzini wrote: > > Patch 1 comes from Kevin's series to do BlockBackend throttling. > > > > Patches 2-6 are from my bdrv_drain patches. They apply on top of Fam's > > patch (which will be in

[Qemu-block] [PATCH for-2.6 v3 3/3] block/gluster: prevent data loss after i/o error

2016-04-19 Thread Jeff Cody
Upon receiving an I/O error after an fsync, by default gluster will dump its cache. However, QEMU will retry the fsync, which is especially useful when encountering errors such as ENOSPC when using the werror=stop option. When using caching with gluster, however, the last written data will be los

[Qemu-block] [PATCH for-2.6 v3 2/3] block/gluster: code movement of qemu_gluster_close()

2016-04-19 Thread Jeff Cody
Move qemu_gluster_close() further up in the file, in preparation for the next patch, to avoid a forward declaration. Signed-off-by: Jeff Cody --- block/gluster.c | 22 +++--- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/block/gluster.c b/block/gluster.c index b

[Qemu-block] [PATCH for-2.6 v3 1/3] block/gluster: return correct error value

2016-04-19 Thread Jeff Cody
Upon error, gluster will call the aio callback function with a ret value of -1, with errno set to the proper error value. If we set the acb->ret value to the return value in the callback, that results in every error being EPERM (i.e. 1). Instead, set it to the proper error result. Reviewed-by: N

[Qemu-block] [PATCH for-2.6 v3 0/3] Bug fixes for gluster

2016-04-19 Thread Jeff Cody
Change from v2: Look at the error return from the aio callback from fsync, as well. (thanks Kevin) Bug fixes for gluster; third patch is to prevent a potential data loss when trying to recover from a recoverable error (such as ENOSPC). The final patch closes the gluster fd and sets the protocol

Re: [Qemu-block] [PATCH for-2.6 v2 0/3] Bug fixes for gluster

2016-04-19 Thread Jeff Cody
On Tue, Apr 19, 2016 at 08:18:39AM -0400, Ric Wheeler wrote: > On 04/19/2016 08:07 AM, Jeff Cody wrote: > >Bug fixes for gluster; third patch is to prevent > >a potential data loss when trying to recover from > >a recoverable error (such as ENOSPC). > > Hi Jeff, > > Just a note, I have been talki

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.7 v2 05/17] raw-posix: Implement .bdrv_lockf

2016-04-19 Thread Richard W.M. Jones
On Tue, Apr 19, 2016 at 02:34:30PM +0100, Daniel P. Berrange wrote: > Have you ever considered integration with the QEMU NBD server. We > don't have APIs for enabling it explicitly in libvirt, but it strikes > me that it could be ideally suited for your needs. > > eg a hypothetical libvirt command

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.7 v2 05/17] raw-posix: Implement .bdrv_lockf

2016-04-19 Thread Daniel P. Berrange
On Tue, Apr 19, 2016 at 02:36:05PM +0100, Richard W.M. Jones wrote: > > I'd prefer some kind of no lock / ignore lock. There is a legitimate > case where you want to have the shared lock behaviour, but also a > legitimate one for turning it off. I'm not opposed to the idea -- > there are very re

Re: [Qemu-block] [RFC PATCH 1/8] virtio: convert to use DMA api

2016-04-19 Thread Michael S. Tsirkin
On Fri, Mar 25, 2016 at 10:13:22AM +0800, Jason Wang wrote: > Currently, all virtio devices bypass IOMMU completely. This is because > address_space_memory is assumed and used during DMA emulation. This > patch converts the virtio core API to use DMA API. This idea is > > - introducing a new trans

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.7 v2 05/17] raw-posix: Implement .bdrv_lockf

2016-04-19 Thread Richard W.M. Jones
On Tue, Apr 19, 2016 at 09:19:02PM +0800, Fam Zheng wrote: > On Tue, 04/19 14:07, Richard W.M. Jones wrote: > > We've done this successfully for years, for people monitoring their > > VMs using virt-df, pulling out files using guestfish and so on. We > > allow you to do it while the guest is live

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.7 v2 05/17] raw-posix: Implement .bdrv_lockf

2016-04-19 Thread Daniel P. Berrange
On Mon, Apr 18, 2016 at 09:04:19AM +0100, Richard W.M. Jones wrote: > On Mon, Apr 18, 2016 at 09:10:36AM +0800, Fam Zheng wrote: > > On Sun, 04/17 20:27, Richard W.M. Jones wrote: > > > On Fri, Apr 15, 2016 at 11:27:55AM +0800, Fam Zheng wrote: > > > > virtlockd in libvirt locks the first byte, we

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.7 v2 05/17] raw-posix: Implement .bdrv_lockf

2016-04-19 Thread Fam Zheng
On Tue, 04/19 14:07, Richard W.M. Jones wrote: > On Tue, Apr 19, 2016 at 08:37:14PM +0800, Fam Zheng wrote: > > On Mon, 04/18 09:04, Richard W.M. Jones wrote: > > > On Mon, Apr 18, 2016 at 09:10:36AM +0800, Fam Zheng wrote: > > > > On Sun, 04/17 20:27, Richard W.M. Jones wrote: > > > > > On Fri, Ap

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.7 v2 05/17] raw-posix: Implement .bdrv_lockf

2016-04-19 Thread Richard W.M. Jones
On Tue, Apr 19, 2016 at 08:37:14PM +0800, Fam Zheng wrote: > On Mon, 04/18 09:04, Richard W.M. Jones wrote: > > On Mon, Apr 18, 2016 at 09:10:36AM +0800, Fam Zheng wrote: > > > On Sun, 04/17 20:27, Richard W.M. Jones wrote: > > > > On Fri, Apr 15, 2016 at 11:27:55AM +0800, Fam Zheng wrote: > > > >

Re: [Qemu-block] [PATCH for-2.7 v2 03/17] blockdev: Add and parse "lock-image" option for block devices

2016-04-19 Thread Fam Zheng
On Sat, 04/16 16:15, Denis V. Lunev wrote: > On 04/15/2016 06:27 AM, Fam Zheng wrote: > >Honor the locking switch specified in CLI or QMP, and set the open flags for > >the image accordingly. > > > >Signed-off-by: Fam Zheng > >--- > > blockdev.c | 8 > > 1 file changed, 8 insertions(+) >

Re: [Qemu-block] [PATCH for-2.7 v2 08/17] qemu-io: Add "-L" option for BDRV_O_NO_LOCK

2016-04-19 Thread Fam Zheng
On Sat, 04/16 16:46, Denis V. Lunev wrote: > On 04/15/2016 06:27 AM, Fam Zheng wrote: > >Signed-off-by: Fam Zheng > >--- > > qemu-io.c | 22 -- > > 1 file changed, 20 insertions(+), 2 deletions(-) > > > >diff --git a/qemu-io.c b/qemu-io.c > >index 288bba8..6bb6232 100644 > >--

Re: [Qemu-block] [PATCH for-2.7 v2 09/17] qemu-img: Add "-L" option to sub commands

2016-04-19 Thread Fam Zheng
On Sat, 04/16 17:29, Denis V. Lunev wrote: > On 04/15/2016 06:27 AM, Fam Zheng wrote: > >If specified, BDRV_O_NO_LOCK flag will be set when opening the image. > > > >Signed-off-by: Fam Zheng > >--- > > qemu-img.c | 89 > > ++ > > 1 file

Re: [Qemu-block] [PATCH for-2.7 v2 11/17] qemu-nbd: Add "--no-lock/-L" option

2016-04-19 Thread Fam Zheng
On Sat, 04/16 17:32, Denis V. Lunev wrote: > On 04/15/2016 06:28 AM, Fam Zheng wrote: > >Signed-off-by: Fam Zheng > >--- > > qemu-nbd.c| 6 +- > > qemu-nbd.texi | 2 ++ > > 2 files changed, 7 insertions(+), 1 deletion(-) > > > >diff --git a/qemu-nbd.c b/qemu-nbd.c > >index b5751f8..37da7a

Re: [Qemu-block] [PATCH for-2.7 v2 00/17] block: Lock images when opening

2016-04-19 Thread Fam Zheng
On Mon, 04/18 10:53, Daniel P. Berrange wrote: > If you want todo locks inside of QEMU, you really can't rely > on delegating handling to each individual block driver instance. > You need to have a process global registry of which files you have > already locked, and check against that registry bef

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.7 v2 05/17] raw-posix: Implement .bdrv_lockf

2016-04-19 Thread Fam Zheng
On Mon, 04/18 09:04, Richard W.M. Jones wrote: > On Mon, Apr 18, 2016 at 09:10:36AM +0800, Fam Zheng wrote: > > On Sun, 04/17 20:27, Richard W.M. Jones wrote: > > > On Fri, Apr 15, 2016 at 11:27:55AM +0800, Fam Zheng wrote: > > > > virtlockd in libvirt locks the first byte, we lock byte 1 to avoid

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] block: Invalidate all children

2016-04-19 Thread Fam Zheng
On Tue, 04/19 16:44, Changlong Xie wrote: > On 04/19/2016 09:42 AM, Fam Zheng wrote: > >Currently we only recurse to bs->file, which will miss the children in quorum > >and VMDK. > > > >Recurse into the whole subtree to avoid that. > > > >Signed-off-by: Fam Zheng > >--- > > block.c | 20 +

Re: [Qemu-block] [PATCH for-2.6 v2 3/3] block/gluster: prevent data loss after i/o error

2016-04-19 Thread Jeff Cody
On Tue, Apr 19, 2016 at 02:27:56PM +0200, Kevin Wolf wrote: > Am 19.04.2016 um 14:07 hat Jeff Cody geschrieben: > > Upon receiving an I/O error after an fsync, by default gluster will > > dump its cache. However, QEMU will retry the fsync, which is especially > > useful when encountering errors su

Re: [Qemu-block] [PATCH for-2.6 v2 3/3] block/gluster: prevent data loss after i/o error

2016-04-19 Thread Kevin Wolf
Am 19.04.2016 um 14:07 hat Jeff Cody geschrieben: > Upon receiving an I/O error after an fsync, by default gluster will > dump its cache. However, QEMU will retry the fsync, which is especially > useful when encountering errors such as ENOSPC when using the werror=stop > option. When using cachin

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.6?] qemu-iotests: iotests: fail hard if not run via "check"

2016-04-19 Thread Markus Armbruster
Sascha Silbe writes: > Dear Markus, > > Markus Armbruster writes: > >>> Running an iotests-based Python test directly might appear to work, >>> but may fail in subtle ways and is insecure: >>> >>> - It creates files with predictable file names in a world-writable >>> location (/var/tmp). >>> >

Re: [Qemu-block] [PATCH for-2.6?] qemu-iotests: iotests: fail hard if not run via "check"

2016-04-19 Thread Sascha Silbe
Dear Max, Max Reitz writes: > On 14.04.2016 13:32, Sascha Silbe wrote: [tests/qemu-iotests/iotests.py] [...] >> def main(supported_fmts=[], supported_oses=['linux']): >> '''Run tests''' >> >> +if test_dir is None or qemu_default_machine is None: > > I think checking test_dir would ha

Re: [Qemu-block] [PATCH for-2.6 v2 0/3] Bug fixes for gluster

2016-04-19 Thread Ric Wheeler
On 04/19/2016 08:07 AM, Jeff Cody wrote: Bug fixes for gluster; third patch is to prevent a potential data loss when trying to recover from a recoverable error (such as ENOSPC). Hi Jeff, Just a note, I have been talking to some of the disk drive people here at LSF (the kernel summit for file

[Qemu-block] [PATCH for-2.6 v2 2/3] block/gluster: code movement of qemu_gluster_close()

2016-04-19 Thread Jeff Cody
Move qemu_gluster_close() further up in the file, in preparation for the next patch, to avoid a forward declaration. Signed-off-by: Jeff Cody --- block/gluster.c | 22 +++--- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/block/gluster.c b/block/gluster.c index b

[Qemu-block] [PATCH for-2.6 v2 0/3] Bug fixes for gluster

2016-04-19 Thread Jeff Cody
Bug fixes for gluster; third patch is to prevent a potential data loss when trying to recover from a recoverable error (such as ENOSPC). The final patch closes the gluster fd and sets the protocol drv to NULL on fsync failure in gluster; we have no way of knowing what gluster versions support reta

[Qemu-block] [PATCH for-2.6 v2 3/3] block/gluster: prevent data loss after i/o error

2016-04-19 Thread Jeff Cody
Upon receiving an I/O error after an fsync, by default gluster will dump its cache. However, QEMU will retry the fsync, which is especially useful when encountering errors such as ENOSPC when using the werror=stop option. When using caching with gluster, however, the last written data will be los

[Qemu-block] [PATCH for-2.6 v2 1/3] block/gluster: return correct error value

2016-04-19 Thread Jeff Cody
Upon error, gluster will call the aio callback function with a ret value of -1, with errno set to the proper error value. If we set the acb->ret value to the return value in the callback, that results in every error being EPERM (i.e. 1). Instead, set it to the proper error result. Reviewed-by: N

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.6?] qemu-iotests: iotests: fail hard if not run via "check"

2016-04-19 Thread Sascha Silbe
Dear Markus, Markus Armbruster writes: >> Running an iotests-based Python test directly might appear to work, >> but may fail in subtle ways and is insecure: >> >> - It creates files with predictable file names in a world-writable >> location (/var/tmp). >> >> - Tests expect the environment to

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Michael S. Tsirkin
On Mon, Apr 18, 2016 at 12:24:15PM -0700, Andy Lutomirski wrote: > On Mon, Apr 18, 2016 at 11:29 AM, David Woodhouse wrote: > > For x86, you *can* enable virtio-behind-IOMMU if your DMAR tables tell > > the truth, and even legacy kernels ought to cope with that. > > FSVO 'ought to' where I suspect

[Qemu-block] [PATCH 3/3] iotests: Test case for drive-mirror with unaligned image size

2016-04-19 Thread Fam Zheng
This is the regression test for the virtual size mismatch issue between target and source images. Signed-off-by: Fam Zheng --- tests/qemu-iotests/152 | 51 ++ tests/qemu-iotests/152.out | 5 + tests/qemu-iotests/group | 1 + 3 files changed

[Qemu-block] [PATCH 2/3] iotests: Add iotests.image_size

2016-04-19 Thread Fam Zheng
This retrieves the virtual size of the image out of qemu-img info. Signed-off-by: Fam Zheng --- tests/qemu-iotests/iotests.py | 6 ++ 1 file changed, 6 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index d9ef60e..56f988a 100644 --- a/tests/qemu-iot

[Qemu-block] [PATCH 0/3] block: Fix drive-mirror with image size unaligned with granularity

2016-04-19 Thread Fam Zheng
This fixes the bug introduced in e5b43573e28 and lately noticed by Kevin. Fam Zheng (3): mirror: Don't extend the last sub-chunk iotests: Add iotests.image_size iotests: Test case for drive-mirror with unaligned image size block/mirror.c| 10 + tests/qemu-iotests/10

[Qemu-block] [PATCH 1/3] mirror: Don't extend the last sub-chunk

2016-04-19 Thread Fam Zheng
The last sub-chunk is rounded up to the copy granularity in the target image, resulting in a larger size than the source. Add a function to clip the copied sectors to the end. This undoes the "wrong" changes to tests/qemu-iotests/109.out in e5b43573e28. The remaining two offset changes are okay.

Re: [Qemu-block] [Qemu-devel] [PATCH v2 REPOST 1/2] Add dynamic module loading for block drivers

2016-04-19 Thread Stefan Hajnoczi
On Tue, Apr 12, 2016 at 02:41:13PM +0100, Richard W.M. Jones wrote: > From: Marc Marí > > Extend the current module interface to allow for block drivers to be loaded > dynamically on request. I like this approach to run-time loading QEMU modules because it's not a plugin system that would inevit

Re: [Qemu-block] [PATCH RFC] fixup! virtio: convert to use DMA api

2016-04-19 Thread Michael S. Tsirkin
On Mon, Apr 18, 2016 at 02:29:33PM -0400, David Woodhouse wrote: > On Mon, 2016-04-18 at 19:27 +0300, Michael S. Tsirkin wrote: > > I balk at adding more hacks to a broken system. My goals are > > merely to > > - make things work correctly with an IOMMU and new guests, > >   so people can use users

Re: [Qemu-block] [PATCH v4 0/8] bdrv_flush_io_queue removal, shared LinuxAioState

2016-04-19 Thread Stefan Hajnoczi
On Thu, Apr 07, 2016 at 06:33:28PM +0200, Paolo Bonzini wrote: > Patch 1 comes from Kevin's series to do BlockBackend throttling. > > Patches 2-6 are from my bdrv_drain patches. They apply on top of Fam's > patch (which will be in 2.6) that introduces bdrv_co_drain. Patch 4 > is new in this vers

Re: [Qemu-block] [PATCH v4 8/8] linux-aio: share one LinuxAioState within an AioContext

2016-04-19 Thread Stefan Hajnoczi
On Thu, Apr 07, 2016 at 06:33:36PM +0200, Paolo Bonzini wrote: > This has better performance because it executes fewer system calls > and does not use a bottom half per disk. Each aio_context_t is initialized for 128 in-flight requests in laio_init(). Will it be possible to hit the limit now that

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] block: Invalidate all children

2016-04-19 Thread Changlong Xie
On 04/19/2016 09:42 AM, Fam Zheng wrote: Currently we only recurse to bs->file, which will miss the children in quorum and VMDK. Recurse into the whole subtree to avoid that. Signed-off-by: Fam Zheng --- block.c | 20 ++-- 1 file changed, 14 insertions(+), 6 deletions(-) di