On 04/30/2012 11:39 PM, Doug Hellmann wrote: > > > On Mon, Apr 30, 2012 at 3:43 PM, Loic Dachary <l...@enovance.com > <mailto:l...@enovance.com>> wrote: > > On 04/30/2012 08:03 PM, Doug Hellmann wrote: >> >> >> On Mon, Apr 30, 2012 at 11:43 AM, Loic Dachary <l...@enovance.com >> <mailto:l...@enovance.com>> wrote: >> >> On 04/30/2012 03:49 PM, Doug Hellmann wrote: >>> >>> >>> On Mon, Apr 30, 2012 at 6:46 AM, Loic Dachary <l...@enovance.com >>> <mailto:l...@enovance.com>> wrote: >>> >>> On 04/30/2012 12:15 PM, Loic Dachary wrote: >>> > We could start a discussion from the content of the following >>> sections: >>> > >>> > http://wiki.openstack.org/EfficientMetering#Counters >>> I think the rationale of the counter aggregation needs to be >>> explained. My understanding is that the metering system will be able to >>> deliver the following information: 10 floating IPv4 addresses were >>> allocated to the tenant during three months and were leased from provider >>> NNN. From this, the billing system could add a line to the invoice : 10 >>> IPv4, $N each = $10xN because it has been configured to invoice each IPv4 >>> leased from provider NNN for $N. >>> >>> It is not the purpose of the metering system to display each >>> IPv4 used, therefore it only exposes the aggregated information. The >>> counters define how the information should be aggregated. If the idea was >>> to expose each resource usage individually, defining counters would be >>> meaningless as they would duplicate the activity log from each OpenStack >>> component. >>> >>> What do you think ? >>> >>> >>> At DreamHost we are going to want to show each individual resource >>> (the IPv4 address, the instance, etc.) along with the charge information. >>> Having the metering system aggregate that data will make it >>> difficult/impossible to present the bill summary and detail views that we >>> want. It would be much more useful for us if it tracked the usage details >>> for each resource, and let us aggregate the data ourselves. >>> >>> If other vendors want to show the data differently, perhaps we >>> should provide separate APIs for retrieving the detailed and aggregate data. >>> >>> Doug >>> >> Hi, >> >> For the record, here is the unfinished conversation we had on IRC >> >> (04:29:06 PM) dhellmann: dachary, did you see my reply about counter >> definitions on the list today? >> (04:39:05 PM) dachary: It means some counters must not be >> aggregated. Only the amount associated with it is but there is one counter >> per IP. >> (04:55:01 PM) dachary: dhellmann: what about this :the id of the >> ressource controls the agregation of all counters : if it is missing, all >> resources of the same kind and their measures are aggregated. Otherwise only >> the measures are agreggated. >> http://wiki.openstack.org/EfficientMetering?action=diff&rev2=40&rev1=39 >> <http://wiki.openstack.org/EfficientMetering?action=diff&rev2=40&rev1=39> >> (04:55:58 PM) dachary: it makes me a little unconfortable to define >> such an "ad-hoc" grouping >> (04:56:53 PM) dachary: i.e. you actuall control the aggregation by >> chosing which value to put in the id column >> (04:58:43 PM) dachary: s/actuall/actually/ >> (05:05:38 PM) ***dachary reading >> http://www.ogf.org/documents/GFD.98.pdf >> (05:05:54 PM) dachary: I feel like we're trying to resolve a non >> problem here >> (05:08:42 PM) dachary: values need to be aggregated. The raw input >> is a full description of the resource and a value ( gauge ). The question is >> how to control the aggregation in a reasonably flexible way. >> (05:11:34 PM) dachary: The definition of a counter could probably be >> described as : the id of a resource and code to fill each column associated >> with it. >> >> I tried to append the following, but the wiki kept failing. >> >> Propose that the counters are defined by a function instead of being >> fixed. That helps addressing the issue of aggregating the bandwidth >> associated to a given IP into a single counter. >> >> Alternate idea : >> * a counter is defined by >> * a name ( o1, n2, etc. ) that uniquely identifies the nature of >> the measure ( outbound internet transit, amount of RAM, etc. ) >> * the component in which it can be found ( nova, swift etc.) >> * and by columns, each one is set with the result of >> aggregate(find(record),record) where >> * find() looks for the existing column as found by selecting with >> the unique key ( maybe the name and the resource id ) >> * record is a detailed description of the metering event to be >> aggregated ( >> http://wiki.openstack.org/SystemUsageData#compute.instance.exists: ) >> * the aggregate() function returns the updated row. By default it >> just += the counter value with the old row returned by find() >> >> >> Would we want aggregation to occur within the database where we are >> collecting events, or should that move somewhere else? > I assume the events collected by the metering agents will all be archived > for auditing (or re-building the database) > http://wiki.openstack.org/EfficientMetering?action=diff&rev2=45&rev1=44 > <http://wiki.openstack.org/EfficientMetering?action=diff&rev2=45&rev1=44> > > Therefore the aggregation should occur when the database is updated to > account for a new event. > > Does this make sense ? I may have misunderstood part of your question. > > > I guess what I don't understand is why the aggregated data is written back to > the metering database at all. If it's in the same database, it seems like it > should be in a different "table" (or equivalent) so the original data is left > alone. In my view the events are not stored in a database, they are merely appended to a log file. The database is built from the events with aggregated data. I now understand that you (and Joshua Harlow) think it's better to not aggregate the data and let the billing system do this job. > > Maybe it's time to start focusing these discussions on user stories? > I agree. Would you like to go first ?
Cheers -- Loïc Dachary Chief Research Officer // eNovance labs http://labs.enovance.com // ? l...@enovance.com ? +33 1 49 70 99 82
_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
