Re: [PATCH v9] virtio_blk: add discard and write zeroes support
On Wed, Dec 05, 2018 at 09:46:16AM +, Liu, Changpeng wrote: > What's the status of this patch ? anybody pulled it for the branch ? I will merge it for next. > > > -Original Message- > > From: Stefan Hajnoczi [mailto:stefa...@redhat.com] > > Sent: Friday, November 2, 2018 12:18 PM > > To: Daniel Verkamp > > Cc: virtualization@lists.linux-foundation.org; linux-bl...@vger.kernel.org; > > Michael S. Tsirkin ; Jason Wang ; > > Jens Axboe ; Paolo Bonzini ; > > Christoph Hellwig ; Liu, Changpeng > > > > Subject: Re: [PATCH v9] virtio_blk: add discard and write zeroes support > > > > On Thu, Nov 01, 2018 at 03:40:35PM -0700, Daniel Verkamp wrote: > > > From: Changpeng Liu > > > > > > In commit 88c85538, "virtio-blk: add discard and write zeroes features > > > to specification" (https://github.com/oasis-tcs/virtio-spec), the virtio > > > block specification has been extended to add VIRTIO_BLK_T_DISCARD and > > > VIRTIO_BLK_T_WRITE_ZEROES commands. This patch enables support for > > > discard and write zeroes in the virtio-blk driver when the device > > > advertises the corresponding features, VIRTIO_BLK_F_DISCARD and > > > VIRTIO_BLK_F_WRITE_ZEROES. > > > > > > Signed-off-by: Changpeng Liu > > > Signed-off-by: Daniel Verkamp > > > --- > > > dverkamp: I've picked up this patch and made a few minor changes (as > > > listed below); most notably, I changed the kmalloc back to GFP_ATOMIC, > > > since it can be called from a context where sleeping is not allowed. > > > To prevent large allocations, I've also clamped the maximum number of > > > discard segments to 256; this results in a 4K allocation and should be > > > plenty of descriptors for most use cases. > > > > > > I also removed most of the description from the commit message, since it > > > was duplicating the comments from virtio_blk.h and quoting parts of the > > > spec without adding any extra information. I have tested this iteration > > > of the patch using crosvm with modifications to enable the new features: > > > https://chromium.googlesource.com/chromiumos/platform/crosvm/ > > > > > > v9 fixes a number of review issues; I didn't attempt to optimize the > > > single-element write zeroes case, so it still does an allocation per > > > request (I did not see any easy place to put the payload that would > > > avoid the allocation). > > > > > > CHANGELOG: > > > v9: [dverkamp] fix LE types in discard struct; cleanups from Ming Lei > > > v8: [dverkamp] replace shifts by 9 with SECTOR_SHIFT constant > > > v7: [dverkamp] use GFP_ATOMIC for allocation that may not sleep; clarify > > > descriptor flags field; comment wording cleanups. > > > v6: don't set T_OUT bit to discard and write zeroes commands. > > > v5: use new block layer API: blk_queue_flag_set. > > > v4: several optimizations based on MST's comments, remove bit field > > > usage for command descriptor. > > > v3: define the virtio-blk protocol to add discard and write zeroes > > > support, first version implementation based on proposed specification. > > > v2: add write zeroes command support. > > > v1: initial proposal implementation for discard command. > > > --- > > > drivers/block/virtio_blk.c | 83 - > > > include/uapi/linux/virtio_blk.h | 54 + > > > 2 files changed, 135 insertions(+), 2 deletions(-) > > > > Reviewed-by: Stefan Hajnoczi ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
RE: [PATCH v9] virtio_blk: add discard and write zeroes support
What's the status of this patch ? anybody pulled it for the branch ? > -Original Message- > From: Stefan Hajnoczi [mailto:stefa...@redhat.com] > Sent: Friday, November 2, 2018 12:18 PM > To: Daniel Verkamp > Cc: virtualization@lists.linux-foundation.org; linux-bl...@vger.kernel.org; > Michael S. Tsirkin ; Jason Wang ; > Jens Axboe ; Paolo Bonzini ; > Christoph Hellwig ; Liu, Changpeng > > Subject: Re: [PATCH v9] virtio_blk: add discard and write zeroes support > > On Thu, Nov 01, 2018 at 03:40:35PM -0700, Daniel Verkamp wrote: > > From: Changpeng Liu > > > > In commit 88c85538, "virtio-blk: add discard and write zeroes features > > to specification" (https://github.com/oasis-tcs/virtio-spec), the virtio > > block specification has been extended to add VIRTIO_BLK_T_DISCARD and > > VIRTIO_BLK_T_WRITE_ZEROES commands. This patch enables support for > > discard and write zeroes in the virtio-blk driver when the device > > advertises the corresponding features, VIRTIO_BLK_F_DISCARD and > > VIRTIO_BLK_F_WRITE_ZEROES. > > > > Signed-off-by: Changpeng Liu > > Signed-off-by: Daniel Verkamp > > --- > > dverkamp: I've picked up this patch and made a few minor changes (as > > listed below); most notably, I changed the kmalloc back to GFP_ATOMIC, > > since it can be called from a context where sleeping is not allowed. > > To prevent large allocations, I've also clamped the maximum number of > > discard segments to 256; this results in a 4K allocation and should be > > plenty of descriptors for most use cases. > > > > I also removed most of the description from the commit message, since it > > was duplicating the comments from virtio_blk.h and quoting parts of the > > spec without adding any extra information. I have tested this iteration > > of the patch using crosvm with modifications to enable the new features: > > https://chromium.googlesource.com/chromiumos/platform/crosvm/ > > > > v9 fixes a number of review issues; I didn't attempt to optimize the > > single-element write zeroes case, so it still does an allocation per > > request (I did not see any easy place to put the payload that would > > avoid the allocation). > > > > CHANGELOG: > > v9: [dverkamp] fix LE types in discard struct; cleanups from Ming Lei > > v8: [dverkamp] replace shifts by 9 with SECTOR_SHIFT constant > > v7: [dverkamp] use GFP_ATOMIC for allocation that may not sleep; clarify > > descriptor flags field; comment wording cleanups. > > v6: don't set T_OUT bit to discard and write zeroes commands. > > v5: use new block layer API: blk_queue_flag_set. > > v4: several optimizations based on MST's comments, remove bit field > > usage for command descriptor. > > v3: define the virtio-blk protocol to add discard and write zeroes > > support, first version implementation based on proposed specification. > > v2: add write zeroes command support. > > v1: initial proposal implementation for discard command. > > --- > > drivers/block/virtio_blk.c | 83 - > > include/uapi/linux/virtio_blk.h | 54 + > > 2 files changed, 135 insertions(+), 2 deletions(-) > > Reviewed-by: Stefan Hajnoczi ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v9] virtio_blk: add discard and write zeroes support
Hi Dongli, Unfortunately, I am not aware of any in-progress implementation of this feature for qemu. It hopefully should not be too difficult to wire up in the qemu virtio-blk model, but I haven't looked into it in detail. Thanks, -- Daniel On Thu, Nov 1, 2018 at 4:42 PM Dongli Zhang wrote: > > Hi Daniel, > > Other than crosvm, is there any version of qemu (e.g., repositories developed > in > progress on github) where I can try with this feature? > > Thank you very much! > > Dongli Zhang > > On 11/02/2018 06:40 AM, Daniel Verkamp wrote: > > From: Changpeng Liu > > > > In commit 88c85538, "virtio-blk: add discard and write zeroes features > > to specification" (https://github.com/oasis-tcs/virtio-spec), the virtio > > block specification has been extended to add VIRTIO_BLK_T_DISCARD and > > VIRTIO_BLK_T_WRITE_ZEROES commands. This patch enables support for > > discard and write zeroes in the virtio-blk driver when the device > > advertises the corresponding features, VIRTIO_BLK_F_DISCARD and > > VIRTIO_BLK_F_WRITE_ZEROES. > > > > Signed-off-by: Changpeng Liu > > Signed-off-by: Daniel Verkamp > > --- > > dverkamp: I've picked up this patch and made a few minor changes (as > > listed below); most notably, I changed the kmalloc back to GFP_ATOMIC, > > since it can be called from a context where sleeping is not allowed. > > To prevent large allocations, I've also clamped the maximum number of > > discard segments to 256; this results in a 4K allocation and should be > > plenty of descriptors for most use cases. > > > > I also removed most of the description from the commit message, since it > > was duplicating the comments from virtio_blk.h and quoting parts of the > > spec without adding any extra information. I have tested this iteration > > of the patch using crosvm with modifications to enable the new features: > > https://chromium.googlesource.com/chromiumos/platform/crosvm/ > > > > v9 fixes a number of review issues; I didn't attempt to optimize the > > single-element write zeroes case, so it still does an allocation per > > request (I did not see any easy place to put the payload that would > > avoid the allocation). > > > > CHANGELOG: > > v9: [dverkamp] fix LE types in discard struct; cleanups from Ming Lei > > v8: [dverkamp] replace shifts by 9 with SECTOR_SHIFT constant > > v7: [dverkamp] use GFP_ATOMIC for allocation that may not sleep; clarify > > descriptor flags field; comment wording cleanups. > > v6: don't set T_OUT bit to discard and write zeroes commands. > > v5: use new block layer API: blk_queue_flag_set. > > v4: several optimizations based on MST's comments, remove bit field > > usage for command descriptor. > > v3: define the virtio-blk protocol to add discard and write zeroes > > support, first version implementation based on proposed specification. > > v2: add write zeroes command support. > > v1: initial proposal implementation for discard command. > > --- > > drivers/block/virtio_blk.c | 83 - > > include/uapi/linux/virtio_blk.h | 54 + > > 2 files changed, 135 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c > > index 086c6bb12baa..0f39efb4b3aa 100644 > > --- a/drivers/block/virtio_blk.c > > +++ b/drivers/block/virtio_blk.c > > @@ -18,6 +18,7 @@ > > > > #define PART_BITS 4 > > #define VQ_NAME_LEN 16 > > +#define MAX_DISCARD_SEGMENTS 256u > > > > static int major; > > static DEFINE_IDA(vd_index_ida); > > @@ -172,10 +173,48 @@ static int virtblk_add_req(struct virtqueue *vq, > > struct virtblk_req *vbr, > > return virtqueue_add_sgs(vq, sgs, num_out, num_in, vbr, GFP_ATOMIC); > > } > > > > +static int virtblk_setup_discard_write_zeroes(struct request *req, bool > > unmap) > > +{ > > + unsigned short segments = blk_rq_nr_discard_segments(req); > > + unsigned short n = 0; > > + struct virtio_blk_discard_write_zeroes *range; > > + struct bio *bio; > > + u32 flags = 0; > > + > > + if (unmap) > > + flags |= VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP; > > + > > + range = kmalloc_array(segments, sizeof(*range), GFP_ATOMIC); > > + if (!range) > > + return -ENOMEM; > > + > > + __rq_for_each_bio(bio, req) { > > + u64 sector = bio->bi_iter.bi_sector; > > + u32 num_sectors = bio->bi_iter.bi_size >> SECTOR_SHIFT; > > + > > + range[n].flags = cpu_to_le32(flags); > > + range[n].num_sectors = cpu_to_le32(num_sectors); > > + range[n].sector = cpu_to_le64(sector); > > + n++; > > + } > > + > > + req->special_vec.bv_page = virt_to_page(range); > > + req->special_vec.bv_offset = offset_in_page(range); > > + req->special_vec.bv_len = sizeof(*range) * segments; > > + req->rq_flags |= RQF_SPECIAL_PAYLOAD; > > + > > + return 0; > > +} > > + > > static inline void virtblk_request_done(struct request *req) > > { > > struct
Re: [PATCH v9] virtio_blk: add discard and write zeroes support
On Thu, Nov 01, 2018 at 03:40:35PM -0700, Daniel Verkamp wrote: > From: Changpeng Liu > > In commit 88c85538, "virtio-blk: add discard and write zeroes features > to specification" (https://github.com/oasis-tcs/virtio-spec), the virtio > block specification has been extended to add VIRTIO_BLK_T_DISCARD and > VIRTIO_BLK_T_WRITE_ZEROES commands. This patch enables support for > discard and write zeroes in the virtio-blk driver when the device > advertises the corresponding features, VIRTIO_BLK_F_DISCARD and > VIRTIO_BLK_F_WRITE_ZEROES. > > Signed-off-by: Changpeng Liu > Signed-off-by: Daniel Verkamp > --- > dverkamp: I've picked up this patch and made a few minor changes (as > listed below); most notably, I changed the kmalloc back to GFP_ATOMIC, > since it can be called from a context where sleeping is not allowed. > To prevent large allocations, I've also clamped the maximum number of > discard segments to 256; this results in a 4K allocation and should be > plenty of descriptors for most use cases. > > I also removed most of the description from the commit message, since it > was duplicating the comments from virtio_blk.h and quoting parts of the > spec without adding any extra information. I have tested this iteration > of the patch using crosvm with modifications to enable the new features: > https://chromium.googlesource.com/chromiumos/platform/crosvm/ > > v9 fixes a number of review issues; I didn't attempt to optimize the > single-element write zeroes case, so it still does an allocation per > request (I did not see any easy place to put the payload that would > avoid the allocation). > > CHANGELOG: > v9: [dverkamp] fix LE types in discard struct; cleanups from Ming Lei > v8: [dverkamp] replace shifts by 9 with SECTOR_SHIFT constant > v7: [dverkamp] use GFP_ATOMIC for allocation that may not sleep; clarify > descriptor flags field; comment wording cleanups. > v6: don't set T_OUT bit to discard and write zeroes commands. > v5: use new block layer API: blk_queue_flag_set. > v4: several optimizations based on MST's comments, remove bit field > usage for command descriptor. > v3: define the virtio-blk protocol to add discard and write zeroes > support, first version implementation based on proposed specification. > v2: add write zeroes command support. > v1: initial proposal implementation for discard command. > --- > drivers/block/virtio_blk.c | 83 - > include/uapi/linux/virtio_blk.h | 54 + > 2 files changed, 135 insertions(+), 2 deletions(-) Reviewed-by: Stefan Hajnoczi signature.asc Description: PGP signature ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v9] virtio_blk: add discard and write zeroes support
Hi Daniel, Other than crosvm, is there any version of qemu (e.g., repositories developed in progress on github) where I can try with this feature? Thank you very much! Dongli Zhang On 11/02/2018 06:40 AM, Daniel Verkamp wrote: > From: Changpeng Liu > > In commit 88c85538, "virtio-blk: add discard and write zeroes features > to specification" (https://github.com/oasis-tcs/virtio-spec), the virtio > block specification has been extended to add VIRTIO_BLK_T_DISCARD and > VIRTIO_BLK_T_WRITE_ZEROES commands. This patch enables support for > discard and write zeroes in the virtio-blk driver when the device > advertises the corresponding features, VIRTIO_BLK_F_DISCARD and > VIRTIO_BLK_F_WRITE_ZEROES. > > Signed-off-by: Changpeng Liu > Signed-off-by: Daniel Verkamp > --- > dverkamp: I've picked up this patch and made a few minor changes (as > listed below); most notably, I changed the kmalloc back to GFP_ATOMIC, > since it can be called from a context where sleeping is not allowed. > To prevent large allocations, I've also clamped the maximum number of > discard segments to 256; this results in a 4K allocation and should be > plenty of descriptors for most use cases. > > I also removed most of the description from the commit message, since it > was duplicating the comments from virtio_blk.h and quoting parts of the > spec without adding any extra information. I have tested this iteration > of the patch using crosvm with modifications to enable the new features: > https://chromium.googlesource.com/chromiumos/platform/crosvm/ > > v9 fixes a number of review issues; I didn't attempt to optimize the > single-element write zeroes case, so it still does an allocation per > request (I did not see any easy place to put the payload that would > avoid the allocation). > > CHANGELOG: > v9: [dverkamp] fix LE types in discard struct; cleanups from Ming Lei > v8: [dverkamp] replace shifts by 9 with SECTOR_SHIFT constant > v7: [dverkamp] use GFP_ATOMIC for allocation that may not sleep; clarify > descriptor flags field; comment wording cleanups. > v6: don't set T_OUT bit to discard and write zeroes commands. > v5: use new block layer API: blk_queue_flag_set. > v4: several optimizations based on MST's comments, remove bit field > usage for command descriptor. > v3: define the virtio-blk protocol to add discard and write zeroes > support, first version implementation based on proposed specification. > v2: add write zeroes command support. > v1: initial proposal implementation for discard command. > --- > drivers/block/virtio_blk.c | 83 - > include/uapi/linux/virtio_blk.h | 54 + > 2 files changed, 135 insertions(+), 2 deletions(-) > > diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c > index 086c6bb12baa..0f39efb4b3aa 100644 > --- a/drivers/block/virtio_blk.c > +++ b/drivers/block/virtio_blk.c > @@ -18,6 +18,7 @@ > > #define PART_BITS 4 > #define VQ_NAME_LEN 16 > +#define MAX_DISCARD_SEGMENTS 256u > > static int major; > static DEFINE_IDA(vd_index_ida); > @@ -172,10 +173,48 @@ static int virtblk_add_req(struct virtqueue *vq, struct > virtblk_req *vbr, > return virtqueue_add_sgs(vq, sgs, num_out, num_in, vbr, GFP_ATOMIC); > } > > +static int virtblk_setup_discard_write_zeroes(struct request *req, bool > unmap) > +{ > + unsigned short segments = blk_rq_nr_discard_segments(req); > + unsigned short n = 0; > + struct virtio_blk_discard_write_zeroes *range; > + struct bio *bio; > + u32 flags = 0; > + > + if (unmap) > + flags |= VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP; > + > + range = kmalloc_array(segments, sizeof(*range), GFP_ATOMIC); > + if (!range) > + return -ENOMEM; > + > + __rq_for_each_bio(bio, req) { > + u64 sector = bio->bi_iter.bi_sector; > + u32 num_sectors = bio->bi_iter.bi_size >> SECTOR_SHIFT; > + > + range[n].flags = cpu_to_le32(flags); > + range[n].num_sectors = cpu_to_le32(num_sectors); > + range[n].sector = cpu_to_le64(sector); > + n++; > + } > + > + req->special_vec.bv_page = virt_to_page(range); > + req->special_vec.bv_offset = offset_in_page(range); > + req->special_vec.bv_len = sizeof(*range) * segments; > + req->rq_flags |= RQF_SPECIAL_PAYLOAD; > + > + return 0; > +} > + > static inline void virtblk_request_done(struct request *req) > { > struct virtblk_req *vbr = blk_mq_rq_to_pdu(req); > > + if (req->rq_flags & RQF_SPECIAL_PAYLOAD) { > + kfree(page_address(req->special_vec.bv_page) + > + req->special_vec.bv_offset); > + } > + > switch (req_op(req)) { > case REQ_OP_SCSI_IN: > case REQ_OP_SCSI_OUT: > @@ -225,6 +264,7 @@ static blk_status_t virtio_queue_rq(struct blk_mq_hw_ctx > *hctx, > int qid = hctx->queue_num; > int err; > bool notify = false; > + bool unmap =
[PATCH v9] virtio_blk: add discard and write zeroes support
From: Changpeng Liu In commit 88c85538, "virtio-blk: add discard and write zeroes features to specification" (https://github.com/oasis-tcs/virtio-spec), the virtio block specification has been extended to add VIRTIO_BLK_T_DISCARD and VIRTIO_BLK_T_WRITE_ZEROES commands. This patch enables support for discard and write zeroes in the virtio-blk driver when the device advertises the corresponding features, VIRTIO_BLK_F_DISCARD and VIRTIO_BLK_F_WRITE_ZEROES. Signed-off-by: Changpeng Liu Signed-off-by: Daniel Verkamp --- dverkamp: I've picked up this patch and made a few minor changes (as listed below); most notably, I changed the kmalloc back to GFP_ATOMIC, since it can be called from a context where sleeping is not allowed. To prevent large allocations, I've also clamped the maximum number of discard segments to 256; this results in a 4K allocation and should be plenty of descriptors for most use cases. I also removed most of the description from the commit message, since it was duplicating the comments from virtio_blk.h and quoting parts of the spec without adding any extra information. I have tested this iteration of the patch using crosvm with modifications to enable the new features: https://chromium.googlesource.com/chromiumos/platform/crosvm/ v9 fixes a number of review issues; I didn't attempt to optimize the single-element write zeroes case, so it still does an allocation per request (I did not see any easy place to put the payload that would avoid the allocation). CHANGELOG: v9: [dverkamp] fix LE types in discard struct; cleanups from Ming Lei v8: [dverkamp] replace shifts by 9 with SECTOR_SHIFT constant v7: [dverkamp] use GFP_ATOMIC for allocation that may not sleep; clarify descriptor flags field; comment wording cleanups. v6: don't set T_OUT bit to discard and write zeroes commands. v5: use new block layer API: blk_queue_flag_set. v4: several optimizations based on MST's comments, remove bit field usage for command descriptor. v3: define the virtio-blk protocol to add discard and write zeroes support, first version implementation based on proposed specification. v2: add write zeroes command support. v1: initial proposal implementation for discard command. --- drivers/block/virtio_blk.c | 83 - include/uapi/linux/virtio_blk.h | 54 + 2 files changed, 135 insertions(+), 2 deletions(-) diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c index 086c6bb12baa..0f39efb4b3aa 100644 --- a/drivers/block/virtio_blk.c +++ b/drivers/block/virtio_blk.c @@ -18,6 +18,7 @@ #define PART_BITS 4 #define VQ_NAME_LEN 16 +#define MAX_DISCARD_SEGMENTS 256u static int major; static DEFINE_IDA(vd_index_ida); @@ -172,10 +173,48 @@ static int virtblk_add_req(struct virtqueue *vq, struct virtblk_req *vbr, return virtqueue_add_sgs(vq, sgs, num_out, num_in, vbr, GFP_ATOMIC); } +static int virtblk_setup_discard_write_zeroes(struct request *req, bool unmap) +{ + unsigned short segments = blk_rq_nr_discard_segments(req); + unsigned short n = 0; + struct virtio_blk_discard_write_zeroes *range; + struct bio *bio; + u32 flags = 0; + + if (unmap) + flags |= VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP; + + range = kmalloc_array(segments, sizeof(*range), GFP_ATOMIC); + if (!range) + return -ENOMEM; + + __rq_for_each_bio(bio, req) { + u64 sector = bio->bi_iter.bi_sector; + u32 num_sectors = bio->bi_iter.bi_size >> SECTOR_SHIFT; + + range[n].flags = cpu_to_le32(flags); + range[n].num_sectors = cpu_to_le32(num_sectors); + range[n].sector = cpu_to_le64(sector); + n++; + } + + req->special_vec.bv_page = virt_to_page(range); + req->special_vec.bv_offset = offset_in_page(range); + req->special_vec.bv_len = sizeof(*range) * segments; + req->rq_flags |= RQF_SPECIAL_PAYLOAD; + + return 0; +} + static inline void virtblk_request_done(struct request *req) { struct virtblk_req *vbr = blk_mq_rq_to_pdu(req); + if (req->rq_flags & RQF_SPECIAL_PAYLOAD) { + kfree(page_address(req->special_vec.bv_page) + + req->special_vec.bv_offset); + } + switch (req_op(req)) { case REQ_OP_SCSI_IN: case REQ_OP_SCSI_OUT: @@ -225,6 +264,7 @@ static blk_status_t virtio_queue_rq(struct blk_mq_hw_ctx *hctx, int qid = hctx->queue_num; int err; bool notify = false; + bool unmap = false; u32 type; BUG_ON(req->nr_phys_segments + 2 > vblk->sg_elems); @@ -237,6 +277,13 @@ static blk_status_t virtio_queue_rq(struct blk_mq_hw_ctx *hctx, case REQ_OP_FLUSH: type = VIRTIO_BLK_T_FLUSH; break; + case REQ_OP_DISCARD: + type = VIRTIO_BLK_T_DISCARD; + break; + case REQ_OP_WRITE_ZEROES: +