Hi, I'd like to share my thoughts on metering openstack resources usage. Except those data available from SystemUsageData and those mentioned in blueprints some of the cloud providers charge for I/O on disk drives (to prevent users from dd if=/dev/zero nosense and to teach them properly implementing cache strategies). Those data along with network card usage can be gathered from libvirt using domblkstat and domifstat. My idea is to gather data using agent (or modified nova-compute) and send them to messaging queue using following jsoned data schema:
{'instance': 'instance-0000003b', 'host':'tytan-1','zone':'r4cz1','counters': {'interface': {'vnet0': (80796L, 1212L, 0L, 0L, 53403L, 621L, 0L, 0L)}, 'disk': {'vda': (629L, 11699200L, 58L, 219136L, 0L)}}} interface is a result of: interfaceStats(), disk is a result of: blockStats() Those messages are consumed from queue and stored in mysql tables. I assume that instance is a parent resource for each disk (ephemeral or volume) and for each network interface. So for message mentioned earlier we have three resources: 1) instances resource: instance-0000003b 2) child resource: vnet0 (network) 3) child resource: vda 'ephemeral Mysql table for resources is like below: CREATE TABLE resources ( id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT, parent BIGINT UNSIGNED, type VARCHAR(255) NOT NULL, value VARCHAR(255) NOT NULL, zone VARCHAR(255) NOT NULL, added TIMESTAMP, ) ENGINE=INNODB; Counters are stored also in Mysql using following table: CREATE TABLE counters ( resource BIGINT UNSIGNED NOT NULL, type VARCHAR(255) NOT NULL, value BIGINT UNSIGNED, delta BIGINT UNISGNED, added TIMESTAMP NOT NULL, prev TIMESTAMP NOT NULL, ) ENGINE=INNODB; Where prev is a reference to previous counter value. Process which is reading data from queue is puting raw counter value into the table and if possible (reference to previous entry present) evaluates delta value. By using this model of stroing usage counter's it very easy for billing system to evaluate charges. We just run SUM(delta) on each counter for given time range. This model could be very easy adopted to other counters (IP Traffic external/internal counters from iptables). On Mon, Apr 30, 2012 at 12:15 PM, Loic Dachary <l...@enovance.com> wrote: > Hi, > > To prepare for the next meeting ( thursday 3rd, may 2012 > http://wiki.openstack.org/Meetings/MeteringAgenda ) I cleaned up and > reorganized the Metering blueprint so that it ( hopefully ) incorporates all > the information temporarily stored in the etherpad ( > http://etherpad.openstack.org/EfficientMetering revision 67 in case it is > vandalized ). > > We could start a discussion from the content of the following sections: > > http://wiki.openstack.org/EfficientMetering#Counters > http://wiki.openstack.org/EfficientMetering#Storage > > and come up with a list of the counters that should exist by default and how > they should be stored. > > This morning we had a discussion with Zhongyue Luo on > irc.freenode.net#openstack-metering about how Dough could use the metering > service. Since it already knows about instance creations, counter c1 that > records how long a given instance was up is of no interest. However, other > counters such as the external bandwidth used would be useful. I advocated > that one of the advantages for Dough to rely on metering to collect counters > is that it does not need to know about each OpenStack component and can rely > on metering to figure out how to extract such counters from nova-compute, > nova-network soon to be quantum, nova-volume soon to be cinder, swift, glance > and free it from the burden of tracking structural changes. > > Cheers > > -- > Loïc Dachary Chief Research Officer > // eNovance labs http://labs.enovance.com > // ✉ l...@enovance.com ☎ +33 1 49 70 99 82 > > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp -- Tomasz Paszkowski SS7, Asterisk, SAN, Datacenter, Cloud Computing +48500166299 _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp