John, For the billing, as no one works on billing now, users need to calculate the billing by themselves. They can get the service_offering and disk_offering of a VMs and volumes for calculation. Of course it is better to tell user the exact limitation value of individual volume, and network rate limitation for nics as well. I can work on it later. Do you think it is a part of I/O throttling?
Sorry my misunstand the second the question. Agree with what you said about the two features. -Wei 2013/6/3 John Burwell <jburw...@basho.com> > Wei, > > > On Jun 3, 2013, at 2:13 AM, Wei ZHOU <ustcweiz...@gmail.com> wrote: > > > Hi John, Mike > > > > I hope Mike's aswer helps you. I am trying to adding more. > > > > (1) I think billing should depend on IO statistics rather than IOPS > > limitation. Please review disk_io_stat if you have time. disk_io_stat > can > > get the IO statistics including bytes/iops read/write for an individual > > virtual machine. > > Going by the AWS model, customers are billed more for volumes with > provisioned IOPS, as well as, for those operations ( > http://aws.amazon.com/ebs/). I would imagine our users would like the > option to employ similar cost models. Could an operator implement such a > billing model in the current patch? > > > > > (2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS > > limitation for a running virtual machine through command line. However, > > CloudStack does not support changing the parameters of a created offering > > (computer offering or disk offering). > > I meant at the Java interface level. I apologize for being unclear. Can > we more generalize allocation algorithms with a set of interfaces that > describe the service guarantees provided by a resource? > > > > > (3) It is a good question. Maybe it is better to commit Mike's patch > after > > disk_io_throttling as Mike needs to consider the limitation in hypervisor > > type, I think. > > I will expand on my thoughts in a later response to Mike regarding the > touch points between these two features. I think that disk_io_throttling > will need to be merged before SolidFire, but I think we need closer > coordination between the branches (possibly have have solidfire track > disk_io_throttling) to coordinate on this issue. > > > > > - Wei > > > > > > 2013/6/3 John Burwell <jburw...@basho.com> > > > >> Mike, > >> > >> The things I want to understand are the following: > >> > >> 1) Is there value in capturing IOPS policies be captured in a common > >> data model (e.g. for billing/usage purposes, expressing offerings). > >> 2) Should there be a common interface model for reasoning about IOP > >> provisioning at runtime? > >> 3) How are conflicting provisioned IOPS configurations between a > >> hypervisor and storage device reconciled? In particular, a scenario > where > >> is lead to believe (and billed) for more IOPS configured for a VM than a > >> storage device has been configured to deliver. Another scenario could a > >> consistent configuration between a VM and a storage device at creation > >> time, but a later modification to storage device introduces logical > >> inconsistency. > >> > >> Thanks, > >> -John > >> > >> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski < > mike.tutkow...@solidfire.com> > >> wrote: > >> > >> Hi John, > >> > >> I believe Wei's feature deals with controlling the max number of IOPS > from > >> the hypervisor side. > >> > >> My feature is focused on controlling IOPS from the storage system side. > >> > >> I hope that helps. :) > >> > >> > >> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell <jburw...@basho.com> > wrote: > >> > >>> Wei, > >>> > >>> My opinion is that no features should be merged until all functional > >>> issues have been resolved and it is ready to turn over to test. Until > >> the > >>> total Ops vs discrete read/write ops issue is addressed and re-reviewed > >> by > >>> Wido, I don't think this criteria has been satisfied. > >>> > >>> Also, how does this work intersect/compliment the SolidFire patch ( > >>> https://reviews.apache.org/r/11479/)? As I understand it that work is > >>> also involves provisioned IOPS. I would like to ensure we don't have a > >>> scenario where provisioned IOPS in KVM and SolidFire are unnecessarily > >>> incompatible. > >>> > >>> Thanks, > >>> -John > >>> > >>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU <ustcweiz...@gmail.com> wrote: > >>> > >>> Wido, > >>> > >>> > >>> Sure. I will change it next week. > >>> > >>> > >>> -Wei > >>> > >>> > >>> > >>> 2013/6/1 Wido den Hollander <w...@widodh.nl> > >>> > >>> > >>> Hi Wei, > >>> > >>> > >>> > >>> On 06/01/2013 08:24 AM, Wei ZHOU wrote: > >>> > >>> > >>> Wido, > >>> > >>> > >>> Exactly. I have pushed the features into master. > >>> > >>> > >>> If anyone object thems for technical reason till Monday, I will revert > >>> > >>> them. > >>> > >>> > >>> For the sake of clarity I just want to mention again that we should > >> change > >>> > >>> the total IOps to R/W IOps asap so that we never release a version with > >>> > >>> only total IOps. > >>> > >>> > >>> You laid the groundwork for the I/O throttling and that's great! We > >> should > >>> > >>> however prevent that we create legacy from day #1. > >>> > >>> > >>> Wido > >>> > >>> > >>> -Wei > >>> > >>> > >>> > >>> 2013/5/31 Wido den Hollander <w...@widodh.nl> > >>> > >>> > >>> On 05/31/2013 03:59 PM, John Burwell wrote: > >>> > >>> > >>> Wido, > >>> > >>> > >>> +1 -- this enhancement must to discretely support read and write IOPS. > >>> > >>> I > >>> > >>> don't see how it could be fixed later because I don't see how we > >>> > >>> correctly > >>> > >>> split total IOPS into read and write. Therefore, we would be stuck > >>> > >>> with a > >>> > >>> total unless/until we decided to break backwards compatibility. > >>> > >>> > >>> > >>> What Wei meant was merging it into master now so that it will go in the > >>> > >>> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2 > >>> > >>> will be released with Read and Write instead of Total IOps. > >>> > >>> > >>> This is to make the May 31st feature freeze date. But if the window > moves > >>> > >>> (see other threads) then it won't be necessary to do that. > >>> > >>> > >>> Wido > >>> > >>> > >>> > >>> I also completely agree that there is no association between network > >>> > >>> and > >>> > >>> > >>> disk I/O. > >>> > >>> > >>> Thanks, > >>> > >>> -John > >>> > >>> > >>> On May 31, 2013, at 9:51 AM, Wido den Hollander <w...@widodh.nl> > wrote: > >>> > >>> > >>> Hi Wei, > >>> > >>> > >>> > >>> On 05/31/2013 03:13 PM, Wei ZHOU wrote: > >>> > >>> > >>> Hi Wido, > >>> > >>> > >>> Thanks. Good question. > >>> > >>> > >>> I thought about at the beginning. Finally I decided to ignore the > >>> > >>> difference of read and write mainly because the network throttling did > >>> > >>> not > >>> > >>> care the difference of sent and received bytes as well. > >>> > >>> That reasoning seems odd. Networking and disk I/O completely different. > >>> > >>> > >>> Disk I/O is much more expensive in most situations then network > >>> > >>> bandwith. > >>> > >>> > >>> Implementing it will be some copy-paste work. It could be > >>> > >>> implemented in > >>> > >>> > >>> few days. For the deadline of feature freeze, I will implement it > >>> > >>> after > >>> > >>> that , if needed. > >>> > >>> > >>> > >>> It think it's a feature we can't miss. But if it goes into the 4.2 > >>> > >>> window we have to make sure we don't release with only total IOps and > >>> > >>> fix > >>> > >>> it in 4.3, that will confuse users. > >>> > >>> > >>> Wido > >>> > >>> > >>> -Wei > >>> > >>> > >>> > >>> > >>> > >>> > >>> 2013/5/31 Wido den Hollander <w...@widodh.nl> > >>> > >>> > >>> Hi Wei, > >>> > >>> > >>> > >>> > >>> On 05/30/2013 06:03 PM, Wei ZHOU wrote: > >>> > >>> > >>> Hi, > >>> > >>> > >>> I would like to merge disk_io_throttling branch into master. > >>> > >>> If nobody object, I will merge into master in 48 hours. > >>> > >>> The purpose is : > >>> > >>> > >>> Virtual machines are running on the same storage device (local > >>> > >>> storage or > >>> > >>> share strage). Because of the rate limitation of device (such as > >>> > >>> iops), if > >>> > >>> one VM has large disk operation, it may affect the disk performance > >>> > >>> of > >>> > >>> other VMs running on the same storage device. > >>> > >>> It is neccesary to set the maximum rate and limit the disk I/O of > >>> > >>> VMs. > >>> > >>> > >>> > >>> Looking at the code I see you make no difference between Read and > >>> > >>> Write > >>> > >>> IOps. > >>> > >>> > >>> Qemu and libvirt support setting both a different rate for Read and > >>> > >>> Write > >>> > >>> IOps which could benefit a lot of users. > >>> > >>> > >>> It's also strange, in the polling side you collect both the Read and > >>> > >>> Write > >>> > >>> IOps, but on the throttling side you only go for a global value. > >>> > >>> > >>> Write IOps are usually much more expensive then Read IOps, so it > >>> > >>> seems > >>> > >>> like a valid use-case where that an admin would set a lower value for > >>> > >>> write > >>> > >>> IOps vs Read IOps. > >>> > >>> > >>> Since this only supports KVM at this point I think it would be of > >>> > >>> great > >>> > >>> value to at least have the mechanism in place to support both, > >>> > >>> implementing > >>> > >>> this later would be a lot of work. > >>> > >>> > >>> If a hypervisor doesn't support setting different values for read and > >>> > >>> write you can always sum both up and set that as the total limit. > >>> > >>> > >>> Can you explain why you implemented it this way? > >>> > >>> > >>> Wido > >>> > >>> > >>> The feature includes: > >>> > >>> > >>> > >>> (1) set the maximum rate of VMs (in disk_offering, and global > >>> > >>> configuration) > >>> > >>> (2) change the maximum rate of VMs > >>> > >>> (3) limit the disk rate (total bps and iops) > >>> > >>> JIRA ticket: https://issues.apache.org/**** > >>> > >>> jira/browse/CLOUDSTACK-1192<ht**tps://issues.apache.org/**** > >>> > >>> jira/browse/CLOUDSTACK-1192< > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192> > >>> > >>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1192< > >>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1192> > >>> > >>> <** > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1192< > >> https://issues.apache.org/jira/browse/CLOUDSTACK-1192> > >>> > >>> > >>> > >>> FS (I will update later) : > >>> > >>> > >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******< > >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****> > >>> > >>> < > >>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**< > >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**> > >>> > >>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****< > >>> http://cwiki.apache.org/confluence/**> > >>> > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.** > >>> > >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling< > >>> > >> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling > >>>> > >>> > >>> > >>> > >>> Merge check list :- > >>> > >>> > >>> * Did you check the branch's RAT execution success? > >>> > >>> Yes > >>> > >>> > >>> * Are there new dependencies introduced? > >>> > >>> No > >>> > >>> > >>> * What automated testing (unit and integration) is included in the > >>> > >>> new > >>> > >>> feature? > >>> > >>> Unit tests are added. > >>> > >>> > >>> * What testing has been done to check for potential regressions? > >>> > >>> (1) set the bytes rate and IOPS rate on CloudStack UI. > >>> > >>> (2) VM operations, including > >>> > >>> deploy, stop, start, reboot, destroy, expunge. migrate, restore > >>> > >>> (3) Volume operations, including > >>> > >>> Attach, Detach > >>> > >>> > >>> To review the code, you can try > >>> > >>> git diff c30057635d04a2396f84c588127d7e******be42e503a7 > >>> > >>> f2e5591b710d04cc86815044f5823e******73a4a58944 > >>> > >>> > >>> Best regards, > >>> > >>> Wei > >>> > >>> > >>> [1] > >>> > >>> > >>> https://cwiki.apache.org/******confluence/display/CLOUDSTACK/******< > >> https://cwiki.apache.org/****confluence/display/CLOUDSTACK/****> > >>> > >>> < > >>> https://cwiki.apache.org/****confluence/display/**CLOUDSTACK/**< > >> https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**> > >>> > >>> VM+Disk+IO+Throttling<https://****cwiki.apache.org/confluence/****< > >>> http://cwiki.apache.org/confluence/**> > >>> > >>> display/CLOUDSTACK/VM+Disk+IO+****Throttling<https://cwiki.** > >>> > >>> apache.org/confluence/display/**CLOUDSTACK/VM+Disk+IO+**Throttling< > >>> > >> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/VM+Disk+IO+Throttling > >>>> > >>> > >>> > >>> > >>> [2] refs/heads/disk_io_throttling > >>> > >>> [3] > >>> https://issues.apache.org/******jira/browse/CLOUDSTACK-1301< > >> https://issues.apache.org/****jira/browse/CLOUDSTACK-1301> > >>> > >>> <ht**tps://issues.apache.org/****jira/browse/CLOUDSTACK-1301< > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301> > >>> > >>> <ht**tps://issues.apache.org/**jira/**browse/CLOUDSTACK-1301< > >>> http://issues.apache.org/jira/**browse/CLOUDSTACK-1301> > >>> > >>> <** > >>> https://issues.apache.org/**jira/browse/CLOUDSTACK-1301< > >> https://issues.apache.org/jira/browse/CLOUDSTACK-1301> > >>> > >>> > >>> > >>> <ht**tps://issues.apache.org/****jira/**browse/CLOUDSTACK-2071< > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071> > >>> > >>> **< > >>> http://issues.apache.org/**jira/**browse/CLOUDSTACK-2071< > >> http://issues.apache.org/jira/**browse/CLOUDSTACK-2071> > >>> > >>> <** > >>> https://issues.apache.org/****jira/browse/CLOUDSTACK-2071< > >> https://issues.apache.org/**jira/browse/CLOUDSTACK-2071> > >>> > >>> <h**ttps://issues.apache.org/jira/**browse/CLOUDSTACK-2071< > >>> https://issues.apache.org/jira/browse/CLOUDSTACK-2071> > >>> > >>> > >>> > >>> (**CLOUDSTACK-1301 > >>> > >>> - VM Disk I/O Throttling) > >>> > >>> > >>> > >> > >> > >> -- > >> *Mike Tutkowski* > >> *Senior CloudStack Developer, SolidFire Inc.* > >> e: mike.tutkow...@solidfire.com > >> o: 303.746.7302 > >> Advancing the way the world uses the > >> cloud<http://solidfire.com/solution/overview/?video=play> > >> *™* > >> > >
