[Qemu-block] [PATCH v5 09/10] block-backend: Add blk_co_copy_range

2018-05-22 Thread Fam Zheng
It's a BlockBackend wrapper of the BDS interface. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/block-backend.c | 18 ++ include/sysemu/block-backend.h | 4 2 files changed, 22 insertions(+) diff --git a/block/block-backend.c b/block/block-backe

[Qemu-block] [PATCH v5 10/10] qemu-img: Convert with copy offloading

2018-05-22 Thread Fam Zheng
The new blk_co_copy_range interface offers a more efficient way in the case of network based storage. Make use of it to allow faster convert operation. Since copy offloading cannot do zero detection ('-S') and compression (-c), only try it when these options are not used. Signed-off-by: Fam Zheng

[Qemu-block] [PATCH v5 08/10] iscsi: Implement copy offloading

2018-05-22 Thread Fam Zheng
Issue EXTENDED COPY (LID1) command to implement the copy_range API. The parameter data construction code is modified from libiscsi's iscsi-dd.c. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/iscsi.c| 219 +++ include/scs

[Qemu-block] [PATCH v5 06/10] iscsi: Query and save device designator when opening

2018-05-22 Thread Fam Zheng
The device designator data returned in INQUIRY command will be useful to fill in source/target fields during copy offloading. Do this when connecting to the target and save the data for later use. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/iscsi.c| 41 ++

[Qemu-block] [PATCH v5 04/10] qcow2: Implement copy offloading

2018-05-22 Thread Fam Zheng
The two callbacks are implemented quite similarly to the read/write functions: bdrv_co_copy_range_from maps for read and calls into bs->file or bs->backing depending on the allocation status; bdrv_co_copy_range_to maps for write and calls into bs->file. Signed-off-by: Fam Zheng --- block/qcow2.c

[Qemu-block] [PATCH v5 05/10] file-posix: Implement bdrv_co_copy_range

2018-05-22 Thread Fam Zheng
With copy_file_range(2), we can implement the bdrv_co_copy_range semantics. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/file-posix.c | 96 +++-- include/block/raw-aio.h | 10 -- 2 files changed, 101 insertions(+), 5 deleti

[Qemu-block] [PATCH v5 07/10] iscsi: Create and use iscsi_co_wait_for_task

2018-05-22 Thread Fam Zheng
This loop is repeated a growing number times. Make a helper. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi Reviewed-by: Eric Blake --- block/iscsi.c | 54 +- 1 file changed, 17 insertions(+), 37 deletions(-) diff --git a/block/iscsi.

[Qemu-block] [PATCH v5 03/10] raw: Implement copy offloading

2018-05-22 Thread Fam Zheng
Just pass down to ->file. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/raw-format.c | 32 1 file changed, 32 insertions(+) diff --git a/block/raw-format.c b/block/raw-format.c index b69a0674b3..f2e468df6f 100644 --- a/block/raw-format.c +++ b

[Qemu-block] [PATCH v5 02/10] raw: Check byte range uniformly

2018-05-22 Thread Fam Zheng
We don't verify the request range against s->size in the I/O callbacks except for raw_co_pwritev. This is wrong (especially for raw_co_pwrite_zeroes and raw_co_pdiscard), so fix them. Signed-off-by: Fam Zheng --- block/raw-format.c | 64 +- 1 f

[Qemu-block] [PATCH v5 00/10] qemu-img convert with copy offloading

2018-05-22 Thread Fam Zheng
v5: - Fix raw offset/bytes check for read. [Eric] - Fix qcow2_handle_l2meta. [Stefan] - Add coroutine_fn whereever appropriate. [Stefan] v4: - Fix raw offset and size. [Eric] - iscsi: Drop unnecessary return values and variables in favor of constants. [Stefan] - qcow2: Handle

[Qemu-block] [PATCH v5 01/10] block: Introduce API for copy offloading

2018-05-22 Thread Fam Zheng
Introduce the bdrv_co_copy_range() API for copy offloading. Block drivers implementing this API support efficient copy operations that avoid reading each block from the source device and writing it to the destination devices. Examples of copy offload primitives are SCSI EXTENDED COPY and Linux co

Re: [Qemu-block] [Qemu-devel] [PATCH v2 2/3] qemu-iotests: Filter NFS paths

2018-05-22 Thread Fam Zheng
On Fri, 05/18 16:26, Kevin Wolf wrote: > NFS paths were only partially filtered in _filter_img_create, _img_info > and _filter_img_info, resulting in "nfs://127.0.0.1TEST_DIR/t.IMGFMT". > This adds another replacement to the sed calls that matches the test > directory not as a host path, but as an

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/3] qemu-iotests: 086 doesn't work with NFS

2018-05-22 Thread Fam Zheng
On Fri, 05/18 16:26, Kevin Wolf wrote: > The reference output file only works for file. 'qemu-img convert -p' > makes a lot more progress updates for NFS than for file, so disable the > test for NFS. > > Signed-off-by: Kevin Wolf Reviewed-by: Fam Zheng

Re: [Qemu-block] [Qemu-devel] [PATCH v2 1/3] qemu-iotests: Fix paths for NFS

2018-05-22 Thread Fam Zheng
On Fri, 05/18 16:26, Kevin Wolf wrote: > Test cases were trying to use nfs:// URLs as local filenames, which made > every test fail for NFS. With TEST_IMG and TEST_IMG_FILE set like for > the other protocols, NFS tests can pass again. > > Signed-off-by: Kevin Wolf Reviewed-by: Fam Zheng

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] sheepdog: cleanup repeated expression

2018-05-22 Thread Fam Zheng
On Tue, 05/22 22:10, Paolo Bonzini wrote: > The expression "SD_INODE_SIZE - sizeof(inode.data_vdi_id)" already has a macro > defined for the same value (though with a nicer definition using offsetof). > Replace it. > > Signed-off-by: Paolo Bonzini Reviewed-by: Fam Zheng

Re: [Qemu-block] [Qemu-devel] storing machine data in qcow images?

2018-05-22 Thread Fam Zheng
On Tue, 05/22 17:02, Kevin Wolf wrote: > Am 22.05.2018 um 16:19 hat Michael S. Tsirkin geschrieben: > > On Tue, May 22, 2018 at 09:35:55AM +0200, Gerd Hoffmann wrote: > > > Hi, > > > > > > > You must /sometimes/ supply the correct machine type. > > > > > > > > It is quite dependent on the guest

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] sheepdog: remove huge BSS object

2018-05-22 Thread Fam Zheng
On Tue, 05/22 22:10, Paolo Bonzini wrote: > block/sheepdog.o has a 4M static variable that is 90% of QEMU's whole .bss > section. Replace it with a heap-allocated block, and make it smaller too > since only the inode header is actually being used. > > bss size goes down from 4464280 to 269976. >

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] sheepdog: cleanup repeated expression

2018-05-22 Thread Philippe Mathieu-Daudé
On 05/22/2018 05:10 PM, Paolo Bonzini wrote: > The expression "SD_INODE_SIZE - sizeof(inode.data_vdi_id)" already has a macro > defined for the same value (though with a nicer definition using offsetof). > Replace it. > > Signed-off-by: Paolo Bonzini Reviewed-by: Philippe Mathieu-Daudé > --- >

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] sheepdog: remove huge BSS object

2018-05-22 Thread Philippe Mathieu-Daudé
On 05/22/2018 05:10 PM, Paolo Bonzini wrote: > block/sheepdog.o has a 4M static variable that is 90% of QEMU's whole .bss > section. Replace it with a heap-allocated block, and make it smaller too > since only the inode header is actually being used. > > bss size goes down from 4464280 to 269976.

[Qemu-block] [PATCH 2/2] sheepdog: remove huge BSS object

2018-05-22 Thread Paolo Bonzini
block/sheepdog.o has a 4M static variable that is 90% of QEMU's whole .bss section. Replace it with a heap-allocated block, and make it smaller too since only the inode header is actually being used. bss size goes down from 4464280 to 269976. Signed-off-by: Paolo Bonzini --- block/sheepdog.c |

[Qemu-block] [PATCH 1/2] sheepdog: cleanup repeated expression

2018-05-22 Thread Paolo Bonzini
The expression "SD_INODE_SIZE - sizeof(inode.data_vdi_id)" already has a macro defined for the same value (though with a nicer definition using offsetof). Replace it. Signed-off-by: Paolo Bonzini --- block/sheepdog.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/block

[Qemu-block] [PATCH 0/2] sheepdog: remove huge BSS object

2018-05-22 Thread Paolo Bonzini
block/sheepdog.o has a 4M static variable that is 90% of QEMU's whole .bss section. Since it doesn't really have to be static, we can just use a heap allocated block. We can actually make it smaller too. :) Patch 1 is a related cleanup since we're touching that area of the code. Paolo Paolo Bo

Re: [Qemu-block] [Qemu-devel] [PATCH v2 00/40] Generic background jobs

2018-05-22 Thread Marc-André Lureau
Hi On Tue, May 22, 2018 at 1:01 PM, Kevin Wolf wrote: > Am 18.05.2018 um 20:41 hat Dr. David Alan Gilbert geschrieben: >> * Kevin Wolf (kw...@redhat.com) wrote: >> > Before we can make x-blockdev-create a background job, we need to >> > generalise the job infrastructure so that it can be used wit

Re: [Qemu-block] [PATCH 3/4] nbd/client: Support requests of additional block sizing info

2018-05-22 Thread Vladimir Sementsov-Ogievskiy
02.05.2018 00:13, Eric Blake wrote: The NBD spec is clarifying [1] that a server may want to advertise different limits for READ/WRITE (in our case, 32M) than for TRIM/ZERO (in our case, nearly 4G). Implement the client side support for these alternate limits, by always requesting the new inform

Re: [Qemu-block] [Qemu-devel] storing machine data in qcow images?

2018-05-22 Thread Eduardo Habkost
On Tue, May 22, 2018 at 05:02:21PM +0200, Kevin Wolf wrote: > Am 22.05.2018 um 16:19 hat Michael S. Tsirkin geschrieben: > > On Tue, May 22, 2018 at 09:35:55AM +0200, Gerd Hoffmann wrote: > > > Hi, > > > > > > > You must /sometimes/ supply the correct machine type. > > > > > > > > It is quite d

Re: [Qemu-block] [Qemu-devel] storing machine data in qcow images?

2018-05-22 Thread Kevin Wolf
Am 22.05.2018 um 16:19 hat Michael S. Tsirkin geschrieben: > On Tue, May 22, 2018 at 09:35:55AM +0200, Gerd Hoffmann wrote: > > Hi, > > > > > You must /sometimes/ supply the correct machine type. > > > > > > It is quite dependent on the guest OS you have installed, and even > > > just how the g

Re: [Qemu-block] [Qemu-devel] storing machine data in qcow images?

2018-05-22 Thread Michael S. Tsirkin
On Tue, May 22, 2018 at 09:35:55AM +0200, Gerd Hoffmann wrote: > Hi, > > > You must /sometimes/ supply the correct machine type. > > > > It is quite dependent on the guest OS you have installed, and even > > just how the guest OS is configured. In general Linux is very > > flexible and can ada

[Qemu-block] [PATCH] nbd/server: fix trace

2018-05-22 Thread Vladimir Sementsov-Ogievskiy
Return code = 1 doesn't mean that we parsed base:allocation. Move trace point to appropriate place. Signed-off-by: Vladimir Sementsov-Ogievskiy --- nbd/server.c | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/nbd/server.c b/nbd/server.c index 9e1f227178..84381baaa8 100

Re: [Qemu-block] [PATCH] sheepdog: Remove unnecessary NULL check in sd_prealloc()

2018-05-22 Thread Kevin Wolf
Am 18.05.2018 um 20:17 hat Peter Maydell geschrieben: > In commit 8b9ad56e9cbfd852a, we removed the code that could result > in our getting to sd_prealloc()'s out_with_err_set label with a > NULL blk pointer. That makes the NULL check in the error-handling > path unnecessary, and Coverity gripes ab

Re: [Qemu-block] [Qemu-devel] [PATCH v2 00/40] Generic background jobs

2018-05-22 Thread Kevin Wolf
Am 18.05.2018 um 20:41 hat Dr. David Alan Gilbert geschrieben: > * Kevin Wolf (kw...@redhat.com) wrote: > > Before we can make x-blockdev-create a background job, we need to > > generalise the job infrastructure so that it can be used without any > > associated block node. > > Is there any relatio

Re: [Qemu-block] [Qemu-devel] storing machine data in qcow images?

2018-05-22 Thread Eduardo Habkost
On Tue, May 22, 2018 at 09:35:55AM +0200, Gerd Hoffmann wrote: > Hi, > > > You must /sometimes/ supply the correct machine type. > > > > It is quite dependent on the guest OS you have installed, and even > > just how the guest OS is configured. In general Linux is very > > flexible and can ada

Re: [Qemu-block] [PATCH v2 37/40] job: Add query-jobs QMP command

2018-05-22 Thread Kevin Wolf
Am 18.05.2018 um 20:22 hat Eric Blake geschrieben: > On 05/18/2018 08:21 AM, Kevin Wolf wrote: > > This adds a minimal query-jobs implementation that shouldn't pose many > > design questions. It can later be extended to expose more information, > > and especially job-specific information. > > > >

Re: [Qemu-block] [PATCH v2 36/40] job: Add lifecycle QMP commands

2018-05-22 Thread Kevin Wolf
Am 18.05.2018 um 20:12 hat Eric Blake geschrieben: > On 05/18/2018 08:21 AM, Kevin Wolf wrote: > > This adds QMP commands that control the transition between states of the > > job lifecycle. > > > > Signed-off-by: Kevin Wolf > > --- > > qapi/job.json | 99 ++

Re: [Qemu-block] [Qemu-devel] storing machine data in qcow images?

2018-05-22 Thread Gerd Hoffmann
Hi, > You must /sometimes/ supply the correct machine type. > > It is quite dependent on the guest OS you have installed, and even > just how the guest OS is configured. In general Linux is very > flexible and can adapt to a wide range of hardware, automatically > detecting things as needed. I