Re: [libvirt] [RFC] block I/O throttling: how to enable in libvirt
On Thu, Sep 01, 2011 at 01:05:31PM +0800, Zhi Yong Wu wrote: > On Wed, Aug 31, 2011 at 08:18:19AM +0100, Stefan Hajnoczi wrote: > >On Tue, Aug 30, 2011 at 2:46 PM, Adam Litke wrote: > >> On Tue, Aug 30, 2011 at 09:53:33AM +0100, Stefan Hajnoczi wrote: > >>> I/O throttling can be applied independently to each -drive attached to > >>> a guest and supports throughput/iops limits. For more information on > >>> this QEMU feature and a comparison with blkio-controller, see Ryan > >>> Harper's KVM Forum 2011 presentation: > >> > >>> http://www.linux-kvm.org/wiki/images/7/72/2011-forum-keep-a-limit-on-it-io-throttling-in-qemu.pdf > >> > >> From the presentation, it seems that both the cgroups method the the qemu > >> method > >> offer comparable control (assuming a block device) so it might possible to > >> apply > >> either method from the same API in a transparent manner. Am I correct or > >> are we > >> suggesting that the Qemu throttling approach should always be used for Qemu > >> domains? > > > >QEMU I/O throttling does not provide a proportional share mechanism. > >So you cannot assign weights to VMs and let them receive a fraction of > >the available disk time. That is only supported by cgroups > >blkio-controller because it requires a global view which QEMU does not > >have. > > > >So I think the two are complementary: > > > >If proportional share should be used on a host block device, use > >cgroups blkio-controller. > >Otherwise use QEMU I/O throttling. > Stefan, > > Do you agree with introducing one new libvirt command blkiothrottle now? > If so, i will work on the code draft to make it work. No, I think that the blkiotune command should be extended to support QEMU I/O throttling. This is not new functionality, we already have cgroups blkio-controller support today. Therefore I think it makes sense to keep a unified interface instead of adding a new command. Stefan -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [RFC] block I/O throttling: how to enable in libvirt
On Thu, Sep 01, 2011 at 09:11:49AM +0100, Stefan Hajnoczi wrote: > On Thu, Sep 01, 2011 at 01:05:31PM +0800, Zhi Yong Wu wrote: > > On Wed, Aug 31, 2011 at 08:18:19AM +0100, Stefan Hajnoczi wrote: > > >On Tue, Aug 30, 2011 at 2:46 PM, Adam Litke wrote: > > >> On Tue, Aug 30, 2011 at 09:53:33AM +0100, Stefan Hajnoczi wrote: > > >>> I/O throttling can be applied independently to each -drive attached to > > >>> a guest and supports throughput/iops limits. For more information on > > >>> this QEMU feature and a comparison with blkio-controller, see Ryan > > >>> Harper's KVM Forum 2011 presentation: > > >> > > >>> http://www.linux-kvm.org/wiki/images/7/72/2011-forum-keep-a-limit-on-it-io-throttling-in-qemu.pdf > > >> > > >> From the presentation, it seems that both the cgroups method the the > > >> qemu method > > >> offer comparable control (assuming a block device) so it might possible > > >> to apply > > >> either method from the same API in a transparent manner. Am I correct > > >> or are we > > >> suggesting that the Qemu throttling approach should always be used for > > >> Qemu > > >> domains? > > > > > >QEMU I/O throttling does not provide a proportional share mechanism. > > >So you cannot assign weights to VMs and let them receive a fraction of > > >the available disk time. That is only supported by cgroups > > >blkio-controller because it requires a global view which QEMU does not > > >have. > > > > > >So I think the two are complementary: > > > > > >If proportional share should be used on a host block device, use > > >cgroups blkio-controller. > > >Otherwise use QEMU I/O throttling. > > Stefan, > > > > Do you agree with introducing one new libvirt command blkiothrottle now? > > If so, i will work on the code draft to make it work. > > No, I think that the blkiotune command should be extended to support > QEMU I/O throttling. This is not new functionality, we already have > cgroups blkio-controller support today. Therefore I think it makes > sense to keep a unified interface instead of adding a new command. Agreed, the virDomainGetBlkioParameters/virDomainSetBlkioParameters APIs, and blkio virsh command are intended to be a generic interface for setting any block related tuning parameters, regardless of what the underling implementation is. So any use of QEMU I/O throttling features should be added to those APIs/commands. Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [RFC] block I/O throttling: how to enable in libvirt
On 2011-9-1 16:11, Stefan Hajnoczi wrote: > On Thu, Sep 01, 2011 at 01:05:31PM +0800, Zhi Yong Wu wrote: >> On Wed, Aug 31, 2011 at 08:18:19AM +0100, Stefan Hajnoczi wrote: >>> On Tue, Aug 30, 2011 at 2:46 PM, Adam Litke wrote: On Tue, Aug 30, 2011 at 09:53:33AM +0100, Stefan Hajnoczi wrote: > I/O throttling can be applied independently to each -drive attached to > a guest and supports throughput/iops limits. For more information on > this QEMU feature and a comparison with blkio-controller, see Ryan > Harper's KVM Forum 2011 presentation: > http://www.linux-kvm.org/wiki/images/7/72/2011-forum-keep-a-limit-on-it-io-throttling-in-qemu.pdf From the presentation, it seems that both the cgroups method the the qemu method offer comparable control (assuming a block device) so it might possible to apply either method from the same API in a transparent manner. Am I correct or are we suggesting that the Qemu throttling approach should always be used for Qemu domains? >>> >>> QEMU I/O throttling does not provide a proportional share mechanism. >>> So you cannot assign weights to VMs and let them receive a fraction of >>> the available disk time. That is only supported by cgroups >>> blkio-controller because it requires a global view which QEMU does not >>> have. >>> >>> So I think the two are complementary: >>> >>> If proportional share should be used on a host block device, use >>> cgroups blkio-controller. >>> Otherwise use QEMU I/O throttling. >> Stefan, >> >> Do you agree with introducing one new libvirt command blkiothrottle now? >> If so, i will work on the code draft to make it work. > > No, I think that the blkiotune command should be extended to support > QEMU I/O throttling. This is not new functionality, we already have > cgroups blkio-controller support today. Therefore I think it makes > sense to keep a unified interface instead of adding a new command. Agreed. Proportional controlling interfaces and throttling interfaces are all the same cgroup subsystem. So Just extend blkiotune to add new options to support throttling tuning. Thanks, Gui > > Stefan > > -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [RFC] block I/O throttling: how to enable in libvirt
On Thu, Sep 01, 2011 at 09:49:34AM +0100, Daniel P. Berrange wrote: >From: "Daniel P. Berrange" >To: Stefan Hajnoczi >Cc: Zhi Yong Wu , libvir-list@redhat.com, > a...@us.ibm.com, qemu-de...@nongnu.org, guijianf...@cn.fujitsu.com, > hu...@cn.fujitsu.com, zwu.ker...@gmail.com >Subject: Re: [RFC] block I/O throttling: how to enable in libvirt >Message-ID: <20110901084934.ga14...@redhat.com> >Reply-To: "Daniel P. Berrange" >References: <20110901050531.gb17...@f15.cn.ibm.com> > <20110901081149.GB14245@stefanha-thinkpad.localdomain> >MIME-Version: 1.0 >Content-Type: text/plain; charset=utf-8 >Content-Disposition: inline >Content-Transfer-Encoding: 8bit >In-Reply-To: <20110901081149.GB14245@stefanha-thinkpad.localdomain> >User-Agent: Mutt/1.5.21 (2010-09-15) >X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 >X-Brightmail-Tracker: AA== >X-Xagent-From: berra...@redhat.com >X-Xagent-To: wu...@linux.vnet.ibm.com >X-Xagent-Gateway: emeavsc.vnet.ibm.com (XAGENTU at EMEAVSC) > >On Thu, Sep 01, 2011 at 09:11:49AM +0100, Stefan Hajnoczi wrote: >> On Thu, Sep 01, 2011 at 01:05:31PM +0800, Zhi Yong Wu wrote: >> > On Wed, Aug 31, 2011 at 08:18:19AM +0100, Stefan Hajnoczi wrote: >> > >On Tue, Aug 30, 2011 at 2:46 PM, Adam Litke wrote: >> > >> On Tue, Aug 30, 2011 at 09:53:33AM +0100, Stefan Hajnoczi wrote: >> > >>> I/O throttling can be applied independently to each -drive attached to >> > >>> a guest and supports throughput/iops limits. For more information on >> > >>> this QEMU feature and a comparison with blkio-controller, see Ryan >> > >>> Harper's KVM Forum 2011 presentation: >> > >> >> > >>> http://www.linux-kvm.org/wiki/images/7/72/2011-forum-keep-a-limit-on-it-io-throttling-in-qemu.pdf >> > >> >> > >> From the presentation, it seems that both the cgroups method the the >> > >> qemu method >> > >> offer comparable control (assuming a block device) so it might possible >> > >> to apply >> > >> either method from the same API in a transparent manner. Am I correct >> > >> or are we >> > >> suggesting that the Qemu throttling approach should always be used for >> > >> Qemu >> > >> domains? >> > > >> > >QEMU I/O throttling does not provide a proportional share mechanism. >> > >So you cannot assign weights to VMs and let them receive a fraction of >> > >the available disk time. That is only supported by cgroups >> > >blkio-controller because it requires a global view which QEMU does not >> > >have. >> > > >> > >So I think the two are complementary: >> > > >> > >If proportional share should be used on a host block device, use >> > >cgroups blkio-controller. >> > >Otherwise use QEMU I/O throttling. >> > Stefan, >> > >> > Do you agree with introducing one new libvirt command blkiothrottle now? >> > If so, i will work on the code draft to make it work. >> >> No, I think that the blkiotune command should be extended to support >> QEMU I/O throttling. This is not new functionality, we already have >> cgroups blkio-controller support today. Therefore I think it makes >> sense to keep a unified interface instead of adding a new command. > >Agreed, the virDomainGetBlkioParameters/virDomainSetBlkioParameters >APIs, and blkio virsh command are intended to be a generic interface >for setting any block related tuning parameters, regardless of what >the underling implementation is. So any use of QEMU I/O throttling >features should be added to those APIs/commands. thanks for your suggestions. Regards, Zhi Yong Wu > > >Daniel >-- >|: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| >|: http://libvirt.org -o- http://virt-manager.org :| >|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| >|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [RFC] block I/O throttling: how to enable in libvirt
On Thu, Sep 01, 2011 at 01:05:31PM +0800, Zhi Yong Wu wrote: >- Forwarded message from Zhi Yong Wu - >Date: Thu, 1 Sep 2011 11:55:17 +0800 >From: Zhi Yong Wu >To: Stefan Hajnoczi >Cc: "Daniel P. Berrange" , Stefan Hajnoczi > , Adam Litke , Zhi Yong Wu > , QEMU Developers , > guijianf...@cn.fujitsu.com, hu...@cn.fujitsu.com >Subject: [RFC] block I/O throttling: how to enable in libvirt >Message-ID: <20110901035517.gd16...@f15.cn.ibm.com> >References: > > > <20110830134636.gb29...@aglitke.rchland.ibm.com> > >MIME-Version: 1.0 >Content-Type: text/plain; charset=utf-8 >Content-Disposition: inline >Content-Transfer-Encoding: 8bit >In-Reply-To: > >User-Agent: Mutt/1.5.21 (2010-09-15) >X-Xagent-From: wu...@linux.vnet.ibm.com >X-Xagent-To: wu...@linux.vnet.ibm.com >X-Xagent-Gateway: vmsdvma.vnet.ibm.com (XAGENTU7 at VMSDVMA) > >On Wed, Aug 31, 2011 at 08:18:19AM +0100, Stefan Hajnoczi wrote: >>Subject: Re: The design choice for how to enable block I/O throttling >> function in libvirt >>From: Stefan Hajnoczi >>To: Adam Litke >>Cc: libvir-list@redhat.com, "Daniel P. Berrange" , Zhi >> Yong Wu , Zhi Yong Wu >>Content-Type: text/plain; charset=ISO-8859-1 >>Content-Transfer-Encoding: quoted-printable >>X-Brightmail-Tracker: AA== >>X-Xagent-From: stefa...@gmail.com >>X-Xagent-To: wu...@linux.vnet.ibm.com >>X-Xagent-Gateway: bldgate.vnet.ibm.com (XAGENTU7 at BLDGATE) >> >>On Tue, Aug 30, 2011 at 2:46 PM, Adam Litke wrote: >>> On Tue, Aug 30, 2011 at 09:53:33AM +0100, Stefan Hajnoczi wrote: On Tue, Aug 30, 2011 at 3:55 AM, Zhi Yong Wu wrote: > I am trying to enable block I/O throttling function in libvirt. But > currently i met some design questions, and don't make sure if we > should extend blkiotune to support block I/O throttling or introduce > one new libvirt command "blkiothrottle" to cover it or not. If you > have some better idea, pls don't hesitate to drop your comments. A little bit of context: this discussion is about adding libvirt support for QEMU disk I/O throttling. >>> >>> Thanks for the additional context Stefan. >>> Today libvirt supports the cgroups blkio-controller, which handles proportional shares and throughput/iops limits on host block devices. blkio-controller does not support network file systems (NFS) or other QEMU remote block drivers (curl, Ceph/rbd, sheepdog) since they are not host block devices. QEMU I/O throttling works with all types of -drive and therefore complements blkio-controller. >>> >>> The first question that pops into my mind is: Should a user need to >>> understand >>> when to use the cgroups blkio-controller vs. the QEMU I/O throttling >>> method? In >>> my opinion, it would be nice if libvirt had a single interface for block I/O >>> throttling and libvirt would decide which mechanism to use based on the >>> type of >>> device and the specific limits that need to be set. >> >>Yes, I agree it would be simplest to pick the right mechanism, >>depending on the type of throttling the user wants. More below. >> I/O throttling can be applied independently to each -drive attached to a guest and supports throughput/iops limits. For more information on this QEMU feature and a comparison with blkio-controller, see Ryan Harper's KVM Forum 2011 presentation: >>> http://www.linux-kvm.org/wiki/images/7/72/2011-forum-keep-a-limit-on-it-io-throttling-in-qemu.pdf >>> >>> From the presentation, it seems that both the cgroups method the the qemu >>> method >>> offer comparable control (assuming a block device) so it might possible to >>> apply >>> either method from the same API in a transparent manner. Am I correct or >>> are we >>> suggesting that the Qemu throttling approach should always be used for Qemu >>> domains? >> >>QEMU I/O throttling does not provide a proportional share mechanism. >>So you cannot assign weights to VMs and let them receive a fraction of >>the available disk time. That is only supported by cgroups >>blkio-controller because it requires a global view which QEMU does not >>have. >> >>So I think the two are complementary: >> >>If proportional share should be used on a host block device, use >>cgroups blkio-controller. >>Otherwise use QEMU I/O throttling. >Stefan, > >Do you agree with introducing one new libvirt command blkiothrottle now? >If so, i will work on the code draft to make it work. > >Daniel and other maintainers, > >If you are available, can you make some comments for us?:) HI, Adam, Now stefan, Daniel, and Gui all suggest extending blkiotune to keep libivrt unified interface. What do you think of it? Regards, Zhi Yong Wu > > >Regards, > >Zhi Yong Wu >> >>Stefan > >- End forwarded message - > >-- >libvir-list mailing list >libvir-list@redhat.com >https://www.redhat.com/mailman/listinfo/libvir-list -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/li
Re: [libvirt] [RFC] block I/O throttling: how to enable in libvirt
On Fri, Sep 02, 2011 at 08:34:19AM -0500, Adam Litke wrote: >From: Adam Litke >To: Zhi Yong Wu >Cc: libvir-list@redhat.com >Subject: Re: [libvirt] [RFC] block I/O throttling: how to enable in libvirt >Message-ID: <20110902133419.gw15...@aglitke.rchland.ibm.com> >References: <20110901050531.gb17...@f15.cn.ibm.com> > <20110902021447.gd19...@f15.cn.ibm.com> >MIME-Version: 1.0 >Content-Type: text/plain; charset=us-ascii >Content-Disposition: inline >In-Reply-To: <20110902021447.gd19...@f15.cn.ibm.com> >User-Agent: Mutt/1.5.21 (2010-09-15) >X-Brightmail-Tracker: AA== >X-Xagent-From: a...@us.ibm.com >X-Xagent-To: wu...@linux.vnet.ibm.com >X-Xagent-Gateway: vmsdvma.vnet.ibm.com (XAGENTU8 at VMSDVMA) > >On Fri, Sep 02, 2011 at 10:14:48AM +0800, Zhi Yong Wu wrote: >> On Thu, Sep 01, 2011 at 01:05:31PM +0800, Zhi Yong Wu wrote: >> HI, Adam, >> Now stefan, Daniel, and Gui all suggest extending blkiotune to keep libivrt >> unified interface. What do you think of it? > >It seems like it would be nice to extend the blkiotune API for this use case, >but there is one problem. The blkiotune interface operates at the VM level and >the QEMU throttling is at the disk level. So if you want per-disk throttling >granularity the blkiotune API may not be suitable. I see 3 possible paths >forward: > >1) Accept the blkiotune limitation of a global setting and add a throttle >option >that assigns the same throttle value to all disks. > > >250 >1024 > > > + Nice for simplicity and ease of implementation > - May be too simplistic (can't set different throttling for different disks) > > >2) Extend the blkiotune API to allow per-disk tunings: > > >250 >1024 > > > + Use an existing API > - Unfortunately it doesn't look like this can be done cleanly > > >3) Use a new API virDomainSetBlkDevParameters() to set device specific >tuning parameters: > > > 250 > >... > > > > > >1024 Several I/O throttling tags should be supported, such as thrbps, thrbpsrd, thrbpswr, thriops, etc. > > > > + Throttle can be specified with the disk it is effecting > - Proliferation of tuning APIs > > >Obviously #1 is the best if you can accept the limitations of the API. #2 >would #1 is not accepted by us. If it is adopted, the advantage of QEMU I/O throttling is not used fully. >be nice if we could figure out how to extend virTypedParameter to represent a I guess that #2 is less extendable than #3. >(disk, param) pair. I see #3 as a last resort. Did I miss any other >solutions. To be honest, i prefer to #3. At first, i plan to check how blkiotune works currently. >Did I just manage to completely over-complicate this? :) NO, they are nice. thanks. Regards, Zhiyong Wu > >-- >Adam Litke >IBM Linux Technology Center -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
