Re: [Openstack] [ceilometer] Potential New Use Cases
Let's imagine that the service that launch instances can tag the instance with: a) a common service identifier (constant) b) a uuid unique for each Unit of the service such as constant:uuid If that tag is passed onto the events which ceilometer stores in its entirety as meta, I do not see what the difficulty would be for the rating engine to be able to reconcile the information to handle your 2 use cases. Am I missing something? Nick On 10/25/2012 12:03 AM, Dan Dyer wrote: I don't think its just a matter of adding more meters or events for a couple of reasons: 1. In many cases the metadata I am referring to comes from a different source than the base usage data. Nova is still emitting its normal events, but we get the service/user mapping from a different source. I would not characterize this data as usage metrics but more data about the system relationships. 2. in the multiple VM case, we need to have the relationships specified so that we can ignore the proper VM's. There has also been talk of hybrid billing models that charge for some part of the VM usage as well as other metrics. Once again we need a way to characterize the relationships so that processing can associate and filter correctly. Dan On 10/24/2012 3:35 PM, Julien Danjou wrote: On Wed, Oct 24 2012, Dan Dyer wrote: Use Case 1 Service Owned Instances There are a set of use cases where a service is acting on behalf of a user, the service is the owner of the VM but billing needs to be attributed to the end user of the system.This scenario drives two requirements: 1. Pricing is similar to base VM's but with a premium. So the type of service for a VM needs to be identifiable so that the appropriate pricing can be applied. 2. The actual end user of the VM needs to be identified so usage can be properly attributed I think that for this, you just need to add more meters on top of the existing one with your own user and project id information. As an example, in some of our PAAS use cases, there is a service controller running on top of the base VM that maintains the control and and manages the customer experience. The idea is to expose the service and not have the customer have to (or even be able to) manipulate the virtual machine directly. So in this case, from a Nova perspective, the PAAS service owns the VM and it's tenantID is what is reported back in events. The way we resolve this is to query the service controller for meta data about that instances they own. This is stored off in a separate table and used to determine the real user at aggregation time. This is probably where you should emit the meters you need. Use Case 2 Multple Instances combine to make a billable product/service In this use case, a service might consist of several VM's, but the actual number does not directly drive the billing. An example of this might be a redundant service that has a primary and two backup VM's that make up a deployment. The customer is charged for the service, not the fact that there are 3 VM's running. Once again, we need meta data that is able to describe this relationship so that when the billing records are processed, this relationship can be identified and billed properly. Kind of the same here, if you don't want to really bill the vm, just don't meter them (or ignore the meters) and emit your own meter via your PaaS platform to bill your customer. Or is there a limitation I miss? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Announce][Ceilometer] Release 0.1 (Folsom)
Dear Fellow stackers, We are pleased to announce today the release of ceilometer 0.1 (Folsom). As its version number implies, it is still a project in its infancy, as it has not yet been incubated as an OpenStack project. It should, however, be ready to be used by the most adventurous ones as it offers a fairly nice coverage of OpenStack, a workable database backend, and a clean and sweet admin API to retrieve metering information from. What is ceilometer? == The ceilometer project aims to deliver a unique point of contact for billing systems to acquire all of the measurements they need to establish customer billing, across all current OpenStack core components with work underway to support future OpenStack components. We hope that, once accepted for incubation, it’s official name will become OpenStack Metering. What does Ceilometer cover at the moment? = Ceilometer covers meters collected from Compute (nova), Network (quantum), Image (glance), and Volume (cinder). We’re obviously missing Object (swift) in this first release, but we have this clearly on our roadmap for our Grizzly release, and maybe even before depending on the agility of our contributors. Regarding Nova, most of the Nova meters will only work with libvirt fronted hypervisors at the moment, and our test coverage was mostly done on KVM. Contributors are of course welcome to implement meters for other virtualization backends. A detailed list of meters can be found at: http://ceilometer.readthedocs.org/en/latest/measurements.html Detailed release notes are at: http://ceilometer.readthedocs.org/en/latest/releasenotes/folsom.html Where is ceilometer’s release? == As for all other OpenStack project, it can be found on the tarballs server, and more specifically at http://tarballs.openstack.org/ceilometer/ and we hope you will be able to find packages in most distributions soon. Note that, even though we are not yet an official OpenStack project, the infra team has been supporting us wonderfully and we’ve been using OpenStack infrastructure happily since the beginning of this project. Big thanks to the infra team for their help and wonderful setup. Who has contributed to ceilometer so far? == Doug Hellmann, Julien Danjou, Eoghan Glynn, Loic Dachary, John Tran, Steven Berler, Jay Pipes, Graham Bins, Francis Lacoste, Thierry Carrez, Anne Gentle, Surya Prabhakar, Nick Barcet and the OpenStack Infrastructure team have provided help in making this release possible (we hope we did not miss anyone here...). Why doesn’t ceilometer do this? === We would enjoy hearing about the missing feature that we either never thought about or haven’t had time to write yet. You can tell us about it on the mailing list (see below) or even better, if you are coming to the OpenStack Developer summit, join us in one of our technical workshop sessions at the summit on Monday morning: Monday 9:50am - State of Metering http://openstacksummitfall2012.sched.org/event/bf543d09fcf4de3f5b4a1008341e5524 Monday 11am - Customizing Ceilometer to measure your cloud http://openstacksummitfall2012.sched.org/event/c3bca2fa753590b2429759810b753c19 Monday 11:50am - Beyond metering - Extending Ceilometer http://openstacksummitfall2012.sched.org/event/235e725deb3c224dfd4da88726729faf or come to the conference presentation Doug Hellmann and Nick Barcet will be giving: Monday at 2:40pm - What about billing? An introduction to Ceilometer http://openstacksummitfall2012.sched.org/event/94fb2cd407b143af4c3fbd979d9c8f91 Where to find more about Ceilometer? Project home and bug tracker: http://launchpad.net/ceilometer Documentation: http://ceiilometer.readthedocs.org Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev (prefix subjects with [metering] for faster responses) Wiki: http://wiki.openstack.org/EfficientMetering Code hosting: https://github.com/stackforge/ceilometer Tarballs: http://tarballs.openstack.org/ceilometer/ Code reviews: https://review.openstack.org/#/q/status:open+project:stackforge/ceilometer,n,z Jenkins: https://jenkins.openstack.org/view/Ceilometer/ IRC: #openstack-metering on freenode We hope that you’ll find this contribution valuable. On behalf of the ceilometer team, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] billing for openstack
On 10/09/2012 05:33 PM, Frans Thamura wrote: hi all my member asks related to billing for openstack, so we can bill anyone use openstack's services are one of this? WHMCS, Clientexec, Blesta, Hostbill or anyone can help? Hello Frans, We are just about to release a first version of Ceilometer[1][2] which provides a central point for a billing implementation to fetch its usage data from. Not sure if that fully answers your question though... [1]http://launchpad.net/ceilometer [2]http://ceilometer.readthedocs.org Best, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Bare metal and Metering
On 09/13/2012 04:33 PM, Tim Bell wrote: Reading through the post from Mirantis on bare metal support on OpenStack (http://www.mirantis.com/blog/bare-metal-provisioning-with-openstack-cloud/), I was wondering how this would interact with the metering work in ceilometer. So far, only libvirt driven hypervisors are covered. For bare metal, or other hypervisors, new pollsters would be needed. Contributions are welcome :) Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] *ALT TIME* Metering meeting agenda for Wed at 21:00 UTC (Sept 12th, 2012)
On 09/11/2012 11:22 PM, Nick Barcet wrote: PLEASE NOTE THE ALTERNATIVE MEETING TIME WED 21:00 UTC The metering project team will hold its next meeting at alternate time on *Wednesday* at 9PM UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=21min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions * nijaba to see with ttx how our sessions can be shown on official program * nijaba to update meeting page to note new alternating meeting time * gmb to a plan on the ml with fixed dates * dhellmann make sure flask is listed as a dependency of ceilometer * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, The meeting took place, here is the summary: == #openstack-meeting: Ceilometer == Meeting started by nijaba at 21:00:17 UTC. The full logs are available at http://eavesdrop.openstack.org/meetings/ceilometer/2012/ceilometer.2012-09-12-21.00.log.html . Meeting summary --- * LINK: http://wiki.openstack.org/Meetings/MeteringAgenda (nijaba, 21:00:17) * actions from previous meeting (nijaba, 21:01:27) * nijaba to see with ttx how our sessions can be shown on official program (nijaba, 21:01:40) * nijaba to update meeting page to note new alternating meeting time (nijaba, 21:03:50) * VOTE: Voted on should we maintain alt meet time for the next month? Results are, yes: 1, bleh: 1, no: 2 (nijaba, 21:10:09) * gmb to a plan on the ml with fixed dates (nijaba, 21:10:36) * AGREED: 0.1 feature freeze will take effect on 2012-09-28 (nijaba, 21:12:15) * AGREED: 0.1 release date will be 2012-10-12 (nijaba, 21:12:15) * dhellmann make sure flask is listed as a dependency of ceilometer (nijaba, 21:12:32) * Open Discusssion (nijaba, 21:13:29) * LINK: http://wiki.openstack.org/EfficientMetering/RoadMap (nijaba, 21:22:54) * ACTION: dhellmann write up details of blueprint https://blueprints.launchpad.net/ceilometer/+spec/config-driven-notification-monitoring (dhellmann, 21:23:36) * LINK: http://teambox.com/n/8fb5404848e9f0b9/ceilometer (nijaba, 21:25:13) * AGREED: next meeting thu sept 20 at 15UTC! (nijaba, 21:40:54) Meeting ended at 21:41:15 UTC. Action items, by person --- * dhellmann * dhellmann write up details of blueprint https://blueprints.launchpad.net/ceilometer/+spec/config-driven-notification-monitoring People present (lines said) --- * nijaba (70) * jtran (42) * asalkeld (26) * dhellmann (18) * spn2 (17) * jd___ (17) * openstack (10) * notmyname (2) * uvirtbot (1) Generated by `MeetBot`_ 0.1.4 signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] *ALT TIME* Metering meeting agenda for Wed at 21:00 UTC (Sept 12th, 2012)
PLEASE NOTE THE ALTERNATIVE MEETING TIME WED 21:00 UTC The metering project team will hold its next meeting at alternate time on *Wednesday* at 9PM UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=21min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions * nijaba to see with ttx how our sessions can be shown on official program * nijaba to update meeting page to note new alternating meeting time * gmb to a plan on the ml with fixed dates * dhellmann make sure flask is listed as a dependency of ceilometer * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Weekly irc meetings time change?
On 09/05/2012 11:51 AM, Nick Barcet wrote: Thanks for asking, I was just about to come up with this. So based on the poll, it seems that the 3-4pm UTC time slot received the most favors with 9 yes, 1 if need be and 1 no. So I guess we'll have to do without Angus, unless we want to do alternating meetings to satisfy both sides of the planet. In the later case we could do one week at 3pm UTC, and the other at 9PM. What would the others think about this? In any case, the next meeting will be tomorrow at 3pm UTC. I'll be sending a formal invite later today. Based on yesterday's meeting, we decided to try alternating time. Since 9PM UTC is taken on thursdays, I am proposing to move it on Wednesdays. If Nobody objects, this means that starting next week we'll have our meetings on: Wednesday at 9PM UTC for odd weeks (September 12, 26) Thursday at 3PM UTC for even weeks (September 20 and October 4) Does this work for everyone? Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] *NEW TIME* Metering meeting agenda for Thursday at 15:00 UTC (Sept 5th, 2012)
On 09/05/2012 12:19 PM, Nick Barcet wrote: PLEASE NOTE THE NEW MEETING TIME, ONE HOUR EARLIER The metering project team holds a meeting in #openstack-meeting, Thursdays at 1500 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - dhellmann to move summit proposal outlines to the wiki and email links to the dev mailing list - nijaba to open a bug for cookbook and assign to jaypipes - nijaba to add architecture image to project /doc rather than link from google - nijaba to include a comment in the rst file linking to the google document where you build the image, so we can remember where it is later * Discussion on session proposal for the summit http://wiki.openstack.org/EfficientMetering/GrizzlySummit * Discussion on alternating meeting time to allow presence from down under * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. The meeting just took place and here is the summary: == #openstack-meeting: Ceilometer == Meeting started by nijaba at 15:00:36 UTC. The full logs are available at http://eavesdrop.openstack.org/meetings/ceilometer/2012/ceilometer.2012-09-06-15.00.log.html . Meeting summary --- * LINK: http://wiki.openstack.org/Meetings/MeteringAgenda (nijaba, 15:00:36) * actions from previous meeting (nijaba, 15:01:28) * dhellmann to move summit proposal outlines to the wiki and email links to the dev mailing list (nijaba, 15:01:49) * nijaba to open a bug for cookbook and assign to jaypipes (nijaba, 15:02:08) * nijaba to add architecture image to project /doc rather than link from google to include a comment in the rst file linking to the google document where you build the image, so we can remember where it is later (nijaba, 15:03:04) * Discussion on session proposal for the summit (nijaba, 15:04:04) * LINK: http://wiki.openstack.org/EfficientMetering/GrizzlySummit (nijaba, 15:04:16) * ACTION: nijaba to see with ttx how our sessions can be shown on official program (nijaba, 15:14:04) * Discussion on alternating meeting time to allow presence from down under (nijaba, 15:14:16) * VOTE: Voted on should we hold our meetings at alternating times on even and odd weeks? Results are, Yes: 6 (nijaba, 15:22:48) * ACTION: nijaba to update meeting page to note new alternating meeting time (nijaba, 15:23:58) * Release management (nijaba, 15:24:14) * ACTION: gmb to act as release manager (nijaba, 15:27:32) * LINK: https://launchpad.net/~heut2008/+archive/ppa (dachary, 15:30:15) * ACTION: gmb to a plan on the ml with fixed dates (nijaba, 15:48:39) * VOTE: Voted on use 0.1 as our first release? Results are, yes: 5 (nijaba, 15:50:33) * VOTE: Voted on let's see if voting is case sensitive? Results are, YeS: 1, nO: 4 (nijaba, 15:51:43) * Open Discusssion (nijaba, 15:52:44) * ACTION: dhellmann make sure flask is listed as a dependency of ceilometer (dhellmann, 15:56:26) Meeting ended at 15:59:07 UTC. Action items, by person --- * dhellmann * dhellmann make sure flask is listed as a dependency of ceilometer * gmb * gmb to act as release manager * gmb to a plan on the ml with fixed dates * nijaba * nijaba to see with ttx how our sessions can be shown on official program * nijaba to update meeting page to note new alternating meeting time * ttx * nijaba to see with ttx how our sessions can be shown on official program People present (lines said) --- * nijaba (121) * dhellmann (61) * zigo (31) * spn (21) * openstack (20) * jd___ (18) * gmb (14) * dachary (13) * eglynn__ (9) * ttx (7) * asalkeld (7) * zul (4) * uvirtbot (3) * jaypipes (2) -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Weekly irc meetings time change?
On 09/05/2012 08:55 AM, SPN wrote: On 2012-08-31 21:34, Nick Barcet wrote: On 08/30/2012 09:49 AM, Nick Barcet wrote: I am about to open a poll to pick from 0600, 1200, 1500 and 2100 UTC on Google. Will reply to this thread when it opens. We'll have to go with what fits most. The poll is now open at [1], please submit your preference(s) there. I will close the poll on tuesday. http://doodle.com/srxx5244qg9pz2pm Cheers, Nick Is there a concenses on the timings? Thanks for asking, I was just about to come up with this. So based on the poll, it seems that the 3-4pm UTC time slot received the most favors with 9 yes, 1 if need be and 1 no. So I guess we'll have to do without Angus, unless we want to do alternating meetings to satisfy both sides of the planet. In the later case we could do one week at 3pm UTC, and the other at 9PM. What would the others think about this? In any case, the next meeting will be tomorrow at 3pm UTC. I'll be sending a formal invite later today. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] *NEW TIME* Metering meeting agenda for Thursday at 15:00 UTC (Sept 5th, 2012)
PLEASE NOTE THE NEW MEETING TIME, ONE HOUR EARLIER The metering project team holds a meeting in #openstack-meeting, Thursdays at 1500 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - dhellmann to move summit proposal outlines to the wiki and email links to the dev mailing list - nijaba to open a bug for cookbook and assign to jaypipes - nijaba to add architecture image to project /doc rather than link from google - nijaba to include a comment in the rst file linking to the google document where you build the image, so we can remember where it is later * Discussion on session proposal for the summit http://wiki.openstack.org/EfficientMetering/GrizzlySummit * Discussion on alternating meeting time to allow presence from down under * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] *NEW TIME* Metering meeting agenda for Thursday at 15:00 UTC (Sept 5th, 2012)
On 09/05/2012 01:11 PM, surya_prabha...@dell.com wrote: On 09/05/12 15:49, Nick Barcet wrote: PLEASE NOTE THE NEW MEETING TIME, ONE HOUR EARLIER The metering project team holds a meeting in #openstack-meeting, Thursdays at 1500 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Isn't it 15.00 UTC? http://www.timeanddate.com/worldclock/fixedtime.html?hour=15min=0sec=0 Absolutely, my bad for thinking I had updated the URL while I obviously had not. Thanks, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Weekly irc meetings time change?
On 08/30/2012 09:49 AM, Nick Barcet wrote: I am about to open a poll to pick from 0600, 1200, 1500 and 2100 UTC on Google. Will reply to this thread when it opens. We'll have to go with what fits most. The poll is now open at [1], please submit your preference(s) there. I will close the poll on tuesday. http://doodle.com/srxx5244qg9pz2pm Cheers, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Billing in Openstack
On 08/30/2012 03:40 AM, Emilien Macchi wrote: Hi, You should read Ceilometer Project [1] and try it into DevStack directly in editing localrc before to run devstack and add : enable_service ceilometer-acompute,ceilometer-acentral, ceilometer-collector Enjoy ! [1] http://wiki.openstack.org/EfficientMetering If fully second Emilien's suggestions. As a quick note, I'd like to level set your expectation as Ceilometer is metering tool, not a full billing tool. A full billing solution needs 3 components: - Metering: to know what has happened on the cloud - Rating: which transforms what has happened into price to bill (line items) - Billing: which accumulates line items and create a bill to send to customer and collect money Which means that you will still need component 2 and 3 to do billing after you have deployed Ceilometer. Hope this helps. Nick On Thu, Aug 30, 2012 at 12:21 PM, Rajesh Avula avula.rajes...@gmail.com mailto:avula.rajes...@gmail.com wrote: Greetings...!! I would like to generate billing of used resources (CPU, RAM, Disk) in openstack. Below are the details of my setup (Please ask me for more details if require regarding setup) Mainly I have used 2 system to make my setup, below are the details... 1. *On First Laptop *(Dell - Latitude, with 4 GB RAM, Intel Core i5 vpro Processor, 500 GB Hard disk) I have installed a *virtual machine with CentOS (6.2)* on *Xenserver (6.0.2)* and on that *virtual machine I installed openstack* by following below links and some other links from google https://fedoraproject.org/wiki/Getting_started_with_OpenStack_Nova http://wiki.openstack.org/XenServer 2. *On Second Laptop* (same as above configuration) I have installed *openfiler and made NFS and iSCSI targets* for glance, swift, and for instances. 3. For network I am using BELKIN wireless adapter 5 port L2-switch) I am able to launch Instances from AMI's, instances are getting IP addresses, able to create volumes and attaching to the instances. All I need is, Is there any possibility in openstack where I can define the values / prices for the resources so that users will get the bills accordingly ? Below are the packages installed in my setup for dashboard python-django-horizon-2012.1.1-1.el6.noarch openstack-dashboard-2012.1.1-1.el6.noarch Is there any other packages should be installed to get billing option on dashboard? Any specific configuration required ? Please let me know how to get billing information of the used / using resources. -- Thanks Regards, Rajesh Avula 09831138115 07259024701. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Emilien Macchi *System Engineer* **www.stackops.com* http://www.stackops.com |*emilien.mac...@stackops.com mailto:emilien.mac...@stackops.com *|*skype:emilien.macchi* * http://www.stackops.com * * ADVERTENCIA LEGAL Le informamos, como destinatario de este mensaje, que el correo electrónico y las comunicaciones por medio de Internet no permiten asegurar ni garantizar la confidencialidad de los mensajes transmitidos, así como tampoco su integridad o su correcta recepción, por lo que STACKOPS TECHNOLOGIES S.L. no asume responsabilidad alguna por tales circunstancias. Si no consintiese en la utilización del correo electrónico o de las comunicaciones vía Internet le rogamos nos lo comunique y ponga en nuestro conocimiento de manera inmediata. Este mensaje va dirigido, de manera exclusiva, a su destinatario y contiene información confidencial y sujeta al secreto profesional, cuya divulgación no está permitida por la ley. En caso de haber recibido este mensaje por error, le rogamos que, de forma inmediata, nos lo comunique mediante correo electrónico remitido a nuestra atención y proceda a su eliminación, así como a la de cualquier documento adjunto al mismo. Asimismo, le comunicamos que la distribución, copia o utilización de este mensaje, o de cualquier documento adjunto al mismo, cualquiera que fuera su finalidad, están prohibidas por la ley. * PRIVILEGED AND CONFIDENTIAL We hereby inform you, as addressee of this message, that e-mail and Internet do not guarantee the confidentiality, nor the completeness or proper reception of the messages sent and, thus, STACKOPS TECHNOLOGIES S.L. does not assume any liability for those circumstances. Should you not agree to the use of e-mail or to communications via Internet, you are kindly requested to notify us
Re: [Openstack] [ceilometer] Weekly irc meetings day and time change?
On 08/30/2012 09:11 AM, Thomas Goirand wrote: On 08/30/2012 04:20 PM, Julien Danjou wrote: On Thu, Aug 30 2012, Angus Salkeld wrote: I'd like to attend but am in Australia. I am quite flexible so it might be easier to say what times don't suit. Basically midnight - 6am which in UTC is 14:00 to 20:00 That's gonna be a tough one. :) In the same idea, living in France, I'd exclude UTC 22:00 to 04:00. So that let us with UTC 04:00 to 14:00 and 20:00 to 22:00 Doug being in the US in IIRC UTC-7 that would exclude UTC 07:00 to UTC 13:00. Finally letting us with UTC 04:00 to 06:00 and 20:00 to 22:00. So finally, I would propose to use UTC 21:00: - 23:00 for me (and Nick most of the time I guess) in France - 14:00 for Doug - 7:00 for Angus or UTC 06:00: - 08:00 for me (and Nick most of the time I guess) in France - 23:00 for Doug - 16:00 for Angus (Hope I did not get the math wrong) If it's the former, I wont ever attend, no way I can get up at 5am (Shanghai is GMT+8). This has always been at this time, and never, I can go online. The later is ok though. I am about to open a poll to pick from 0600, 1200, 1500 and 2100 UTC on Google. Will reply to this thread when it opens. We'll have to go with what fits most. Thomas P.S: I'd like to contribute to the ceilometer project. I haven't yet (though I've done quite some work on the Debian packaging of the rest of Openstack), and I'd like to know in which area I could start implementing. I would recommend that you take a look at the roadmap[1], and assign yourself one of the bugs in there if you feel like it. A good recommended read to start is at [2]. [1] http://wiki.openstack.org/EfficientMetering/RoadMap [2] http://ceilometer.readthedocs.org/ Hope this helps, and welcome to the ceilometer team! Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 30rd, 2012)
On 08/29/2012 08:29 PM, Nick Barcet wrote: Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - jaypipes to create ceilometer cookbook - nijaba to link schema in the doc - nijaba to start a thread on meeting time * Discuss session proposal for Grizzly summit * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. The meeting took place today, the 30th, contrarily to what I wrongly titled the previous email, and here is the summary: == #openstack-meeting: Ceilometer == Meeting started by nijaba at 16:01:12 UTC. The full logs are available at http://eavesdrop.openstack.org/meetings/ceilometer/2012/ceilometer.2012-08-30-16.01.log.html . Meeting summary --- * LINK: http://wiki.openstack.org/Meetings/MeteringAgenda (nijaba, 16:01:12) * actions from previous meeting (nijaba, 16:02:11) * jaypipes to create ceilometer cookbook (nijaba, 16:02:26) * ACTION: nijaba to open a bug for cookbook and assign to jaypipes (nijaba, 16:03:42) * nijaba to link schema in the doc (nijaba, 16:03:53) * LINK: https://review.openstack.org/#/c/12171 (nijaba, 16:04:04) * LINK: http://ceilometer.readthedocs.org/en/latest/architecture.html#high-level-description (nijaba, 16:04:04) * ACTION: nijaba to add architecture image to project /doc rather than link from google (nijaba, 16:06:04) * ACTION: nijaba to include a comment in the rst file linking to the google document where you build the image, so we can remember where it is later (nijaba, 16:06:46) * nijaba to start a thread on meeting time (nijaba, 16:09:41) * Discuss sessions proposal for Grizzly summit (nijaba, 16:10:12) * ACTION: dhellmann to move summit proposal outlines to the wiki and email links to the dev mailing list (dhellmann, 16:13:56) * Discuss changing the meeting time (nijaba, 16:16:08) * ACTION: nijaba to open a doodle poll to pick between viable options (nijaba, 16:23:15) * Open Discussion (nijaba, 16:25:23) Meeting ended at 16:35:25 UTC. Action items, by person --- * dhellmann * dhellmann to move summit proposal outlines to the wiki and email links to the dev mailing list * jaypipes * nijaba to open a bug for cookbook and assign to jaypipes * nijaba * nijaba to open a bug for cookbook and assign to jaypipes * nijaba to add architecture image to project /doc rather than link from google * nijaba to include a comment in the rst file linking to the google document where you build the image, so we can remember where it is later * nijaba to open a doodle poll to pick between viable options People present (lines said) --- * nijaba (78) * dhellmann (35) * spn (15) * jd___ (8) * oubiwann (6) * jaypipes (4) * openstack (4) * gmb (1) * DanD (1) -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] Weekly irc meetings day and time change?
Hello, As we are observing that the current meeting time is not very convenient for a few contributors, we would like to propose to change the date and time to something that would suit a larger number. I propose that we do this in a two phase process: 1/ Each project member that care should send a list of days and times that would be convenient for them as a reply to this message. Please try to avoid a day an time already used by another project as listed on [1] 2/ We then open a poll with a compiled list of proposals and get members vote on it. If you are not a contributor to the project, we are happy to receive your suggestions too, but since this meeting is meant to coordinate between contributors, the result of this poll will be heavily skewed toward active members preferences. [1] http://wiki.openstack.org/Meetings/ Thanks, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 29rd, 2012)
Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - jaypipes to create ceilometer cookbook - nijaba to link schema in the doc - nijaba to start a thread on meeting time * Discuss session proposal for Grizzly summit * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 23rd, 2012)
On 08/23/2012 05:18 AM, Nick Barcet wrote: Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - jaypipes to create ceilometer cookbook - nijaba to write description of component responsibility - dhellmann and nijaba to work on sessions for summit via email - dhellmann to ask jtrans about interest in reviewer status - nijaba to give core reviewer rights to gmb * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. The meeting took place, here are the minutes: == #openstack-meeting: Ceilometer == Meeting started by nijaba at 16:00:33 UTC. The full logs are available at http://eavesdrop.openstack.org/meetings/ceilometer/2012/ceilometer.2012-08-23-16.00.log.html . Meeting summary --- * LINK: http://wiki.openstack.org/Meetings/MeteringAgenda (nijaba, 16:00:49) * actions from previous meeting (nijaba, 16:01:55) * jaypipes to create ceilometer cookbook (nijaba, 16:02:13) * ACTION: jaypipes to create ceilometer cookbook (nijaba, 16:02:41) * nijaba to write description of component responsibility (nijaba, 16:03:44) * ACTION: nijaba to link schema in the doc (nijaba, 16:05:44) * LINK: https://docs.google.com/drawings/pub?id=1_cIFir6HS6jSkPw7chrmyu8DGE2ZgXk79Kbj8nw-Hqow=960h=720 (nijaba, 16:06:24) * dhellmann and nijaba to work on sessions for summit via email (nijaba, 16:06:56) * ACTION: dhellmann and nijaba to work on sessions for summit via email (nijaba, 16:07:32) * dhellmann to ask jtrans about interest in reviewer status (nijaba, 16:08:01) * nijaba to give core reviewer rights to gmb (nijaba, 16:08:52) * Open Discussion (nijaba, 16:11:33) * ACTION: nijaba to start a thread on meeting time (nijaba, 16:34:26) Meeting ended at 16:35:40 UTC. Action items, by person --- * dhellmann * dhellmann and nijaba to work on sessions for summit via email * nijaba * nijaba to link schema in the doc * dhellmann and nijaba to work on sessions for summit via email * nijaba to start a thread on meeting time People present (lines said) --- * nijaba (68) * dhellmann (53) * openstack (4) * gmb (3) * heckj (1) * _surya_ (1) -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 23rd, 2012)
Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - jaypipes to create ceilometer cookbook - nijaba to write description of component responsibility - dhellmann and nijaba to work on sessions for summit via email - dhellmann to ask jtrans about interest in reviewer status - nijaba to give core reviewer rights to gmb * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 16th, 2012)
Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - jaypipes to create ceilometer cookbook - nijaba to write description of componet responsibility - nijaba to do a second thouroughness check on API and report next week * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 16th, 2012)
On 08/16/2012 03:54 PM, Nick Barcet wrote: Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - jaypipes to create ceilometer cookbook - nijaba to write description of componet responsibility - nijaba to do a second thouroughness check on API and report next week * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. The meeting took place and here are the minutes: == #openstack-meeting: Ceilometer == Meeting started by nijaba at 16:00:25 UTC. The full logs are available at http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-08-16-16.00.log.html . Meeting summary --- * LINK: http://wiki.openstack.org/Meetings/MeteringAgenda (nijaba, 16:00:25) * actions from previous meeting (nijaba, 16:01:43) * jaypipes to create ceilometer cookbook (nijaba, 16:01:58) * ACTION: jaypipes to create ceilometer cookbook (nijaba, 16:02:38) * nijaba to do a second thouroughness check on API and report next week (nijaba, 16:02:57) * nijaba to write description of component responsibility (nijaba, 16:11:33) * ACTION: nijaba to write description of component responsibility (nijaba, 16:11:43) * Open Discusssion (nijaba, 16:12:19) * ACTION: dhellmann and nijaba to work on sessions for summit via email (nijaba, 16:19:23) * ACTION: nijaba to give core reviewer rights to gmb (nijaba, 16:21:55) * ACTION: dhellmann to ask jtrans about interest in reviewer status (dhellmann, 16:23:28) Meeting ended at 16:27:23 UTC. Action items, by person --- * dhellmann * dhellmann and nijaba to work on sessions for summit via email * dhellmann to ask jtrans about interest in reviewer status * gmb * nijaba to give core reviewer rights to gmb * nijaba * nijaba to write description of component responsibility * dhellmann and nijaba to work on sessions for summit via email * nijaba to give core reviewer rights to gmb * jaypipes * jaypipes to create ceilometer cookbook People present (lines said) --- * nijaba (77) * dhellmann (47) * gmb (5) * openstack (3) * jgriffith (2) * zul (2) Generated by `MeetBot`_ 0.1.4 signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 9th, 2012)
On 08/08/2012 05:54 PM, Nick Barcet wrote: Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - jaypipes to create ceilometer cookbook - jd_ to publish results of PTL election on general ml sometimes tomorrow - jtran to open a ticket for the DB access work - nijaba create a diagram of Ceilometer architecture * Discuss Doug's API change proposal * Discuss priority of maintaining Essex support and find contributor to work on it if we are going to do it * Discuss integration with Heat * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. The meeting took place and here is the summary for it: == #openstack-meeting: Ceilometer == Meeting started by nijaba at 16:00:02 UTC. The full logs are available at http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-08-09-16.00.log.html . Meeting summary --- * LINK: http://wiki.openstack.org/Meetings/MeteringAgenda (nijaba, 16:00:02) * actions from previous meeting (nijaba, 16:01:42) * jaypipes to create ceilometer cookbook (nijaba, 16:01:56) * ACTION: jaypipes to create ceilometer cookbook (nijaba, 16:03:23) * jtran to open a ticket for the DB access work (nijaba, 16:03:41) * LINK: https://bugs.launchpad.net/ceilometer/+bug/1034666 (nijaba, 16:03:50) * nijaba to create a diagram of ceilometer architecture (nijaba, 16:04:03) * LINK: https://docs.google.com/drawings/pub?id=1_cIFir6HS6jSkPw7chrmyu8DGE2ZgXk79Kbj8nw-Hqow=960h=720 (nijaba, 16:04:13) * ACTION: nijaba to write description of componet responsibility (nijaba, 16:06:58) * Discuss dhellmann's API change proposal (nijaba, 16:07:21) * LINK: http://lists.openstack.org/pipermail/openstack-dev/2012-August/000389.html (nijaba, 16:07:21) * AGREED: new api proposal from dhellmann provide a second check that all case are covered (nijaba, 16:11:42) * ACTION: nijaba to do a second thouroughness check on API and report next week (nijaba, 16:12:01) * Discuss priority of maintaining Essex support and find contributor to work on it if we are going to do it (nijaba, 16:12:18) * AGREED: essex support postponed until someone that cares about it signals himself and want to work on it (nijaba, 16:16:57) * Discuss integration with Heat (nijaba, 16:17:19) * AGREED: propose a joint session with heat at ODS regarding cloudwatch (nijaba, 16:26:53) * Open Discusssion (nijaba, 16:28:52) Meeting ended at 16:40:44 UTC. Action items, by person --- * jaypipes * jaypipes to create ceilometer cookbook * nijaba * nijaba to write description of componet responsibility * nijaba to do a second thouroughness check on API and report next week People present (lines said) --- * nijaba (95) * dhellmann (27) * jd___ (24) * ppetit (22) * jaypipes (3) * openstack (3) * mrevell (2) * uvirtbot (1) * gmb (1) * DanD (1) Generated by `MeetBot`_ 0.1.4 -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 9th, 2012)
Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - jaypipes to create ceilometer cookbook - jd_ to publish results of PTL election on general ml sometimes tomorrow - jtran to open a ticket for the DB access work - nijaba create a diagram of Ceilometer architecture * Discuss Doug's API change proposal * Discuss priority of maintaining Essex support and find contributor to work on it if we are going to do it * Discuss integration with Heat * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [metering] General request for feedback on Ceilometer's road-map
Hello everyone, If you have been following this list, you might have seen that the Ceilometer project, while not yet fully functional, has made some great progress over the past three month. The next summit being just a few months away, we thought it would be time to start collecting general feedback on what we should be working on next and how to prioritize this. In order to prime your feedback, we prepared a road-map wiki page [1]. Have a look at it and let us know if you think we should be adding other elements to it. When doing so, please remember that the target of Ceilometer is to collect the metering information for OpenStack components in a single place, not to build a full rating or billing engine. Billing or rating tools should connect to Ceilometer to get their information they need, so feedback on this aspect is welcome (what additional information should we collect? could some evolution of our API[2] make the life of billing or rating engine simpler?) You will certainly notice that we are also tracking, in the same page, some features as bug requests and some of them are targeted for folsom.3: this is what we think needs urgent work _right now_, so if you think you are in a good position to contribute a little code to Ceilometer, feel free to assign yourself to the bug and start contributing patches. If you wonder how to contribute to Ceilometer, we also have put together a little documentation which you can read at [3]. [1] http://wiki.openstack.org/EfficientMetering/RoadMap [2] http://wiki.openstack.org/EfficientMetering/APIProposalv1 [3] http://ceilometer.readthedocs.org Thanks for your help and support, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Ceilometer PTL Election Process - Call for candidate
On 07/11/2012 02:24 PM, Nick Barcet wrote: Candidates should, before July 24th: 1/ declare themselves on this mailing list It has been so much fun and so rewarding to kick-start this project, that I can't resist to propose myself as Candidate. Not being a true developer places me in an awkward position to run for this function, but I think I have shown, as other have done before, that coding is not the only talent one can contribute and lead with. 2/ add their name on the wiki page [1] http://wiki.openstack.org/EfficientMetering/PTLElectionProcess Done, and added a few more words at [1] following the steps of the previous 2 candidates, which are both exceptionally skilled to run this project too. It's absolutely a wonderful feeling that I am not the only one caring enough about this project. Truly, I think this project would be nowhere if we had not met. The road ahead is still quite steep, and we'll all have lots to do, regardless of whom leads next. [1] http://wiki.openstack.org/EfficientMetering/PTLElectionProcess/NickBarcet signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Time for a UK Openstack User Group meeting ?
On 07/05/2012 07:02 PM, Nick Barcet wrote: On 07/04/2012 05:38 PM, Day, Phil wrote: Hi All, I’m thinking it’s about time we had an OpenStack User Group meeting in the UK , and would be interested in hearing from anyone interested in attending, presenting, helping to organise, etc. London would seem the obvious choice, but we could also host here in HP Bristol if that works for people. Reply here or e-mail me directly (phil@hp.com), and if there’s enough interest I’ll pull something together. Canonical would be delighted to host that meeting in our new london office. We have capacity for up to 100 people and are currently checking availability, but are looking for a date in the last week july. We have started the OpenStack-London meetup [1] so anyone interested can subscribe. [1] http://www.meetup.com/Openstack-London/ The room has now been booked: http://www.meetup.com/Openstack-London/events/55354582/ When: Wednesday, July 25, 2012 6:30 PM Where: IPC Media Building (Bluefin) 110 Southwark Street SE1 0SU RSVP limit: 100 Yes RSVPs Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Meeting agenda for Thursday at 16:00 UTC (July 12th, 2012)
On 07/11/2012 02:34 PM, Nick Barcet wrote: Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - nijaba to send an email to the PPB for Incabation application - nijaba to send call for candidate to the general mailing list - jd to setup opa voting system to start on 26th and end on Aug 3rd - nijaba to prime a roadmap page and invite others to populate it - jd handle counter/meter type for real - nijaba to document external API use with a clear warning about the limitation of the sum and duration function * Discuss and define actions from the PPB discussion on Tue regarding our incubation. - http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-10-20.01.html - http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-10-20.01.log.html * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, The meeting took place, here is the summary: == #openstack-meeting: Ceilometer == Meeting started by nijaba at 16:00:01 UTC. The full logs are available at http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-12-16.00.log.html Meeting summary --- * LINK: http://wiki.openstack.org/Meetings/MeteringAgenda (nijaba, 16:00:01) * actions from previous meeting (nijaba, 16:02:08) * nijaba to send an email to the PPB for Incabation application (nijaba, 16:02:20) * nijaba to send call for candidate to the general mailing list (nijaba, 16:02:31) * jd__ to setup opa voting system to start on 26th and end on Aug 3rd (nijaba, 16:04:07) * ACTION: jd to setup opa voting system to start on 26th and end on Aug 3rd (nijaba, 16:04:27) * nijaba to prime a roadmap page and invite others to populate it (nijaba, 16:04:40) * LINK: http://wiki.openstack.org/EfficientMetering/RoadMap (nijaba, 16:04:40) * ACTION: nijaba to post roadmap to mailing list, askingfor feeback and volunteers? (nijaba, 16:05:48) * jd__ handle counter/meter type for real (nijaba, 16:06:12) * ACTION: jd___ adding the type of meters in ceilometer meter code (nijaba, 16:07:57) * nijaba to document external API use with a clear warning about the limitation of the sum and duration function (nijaba, 16:08:13) * LINK: http://wiki.openstack.org/EfficientMetering/APIProposalv1#Limitations (nijaba, 16:08:13) * Discuss and define actions from the PPB discussion on Tue regarding our incubation. (nijaba, 16:10:13) * LINK: http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-10-20.01.html (nijaba, 16:10:13) * LINK: http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-10-20.01.log.html (nijaba, 16:10:13) * ACTION: nijaba to add a table showing the state of integration for each OpenStack project (nijaba, 16:13:53) * ACTION: nijaba to adjust the proposal to reflect a longer incubation period (nijaba, 16:15:45) * ACTION: dhellmann to get some feedback now about the sorts of meters users want from the mailing list (nijaba, 16:19:57) * ACTION: dhellmann to open a bug and work on devstack integration (nijaba, 16:21:39) * Open Discusssion (nijaba, 16:26:42) * ACTION: dhellmann create a diagram of ceilometer architecture (dhellmann, 16:35:35) * ACTION: dhellmann write a walk-through of setting up ceilometer and collecting data (dhellmann, 16:36:02) Meeting ended at 16:47:27 UTC. Action items, by person --- * dhellmann * dhellmann to get some feedback now about the sorts of meters users want from the mailing list * dhellmann to open a bug and work on devstack integration * dhellmann create a diagram of ceilometer architecture * dhellmann write a walk-through of setting up ceilometer and collecting data * jd___ * jd___ adding the type of meters in ceilometer meter code * nijaba * nijaba to post roadmap to mailing list, askingfor feeback and volunteers? * nijaba to add a table showing the state of integration for each OpenStack project * nijaba to adjust the proposal to reflect a longer incubation period People present (lines said) --- * nijaba (114) * dhellmann (65) * jd___ (34) * flacoste (13) * gmb (9) * heckj (8) * DanD (8) * openstack (3) * uvirtbot` (1) signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https
Re: [Openstack] Management tools survey
On 07/11/2012 05:18 AM, Nick Lothian wrote: Hi, I'm trying to understand how people are doing management of servers and storage across multiple clouds (or perhaps it is only me that has this problem!). I've created a short survey I'd appreciate any responses on: http://www.surveymonkey.com/s/8PJCK9H Responses via email are fine too! Hello Nick, I am sure there are others, like me, interested in your findings in this area. Will you share the results of the survey? Thanks, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [metering] Ceilometer PTL Election Process - Call for candidate
As voted on the July 5th meeting the Ceilometer team members will soon vote on electing it's project team lead (PTL). Even though Ceilometer is not yet an official OpenStack project, we are applying for it, so we should be following the standard process as much as possible. The details of the process we will be following is described at [1]. This email is a formal call for candidates, which is step one of the process. Candidates for the PTL role must have contributed either to the source or to the wiki pages for Ceilometer prior to the call for candidates. The list that we assembled can be found on the above mentioned wiki page, but in the event we have missed someone, please shout. Candidates should, before July 24th: 1/ declare themselves on this mailing list 2/ add their name on the wiki page [1] http://wiki.openstack.org/EfficientMetering/PTLElectionProcess Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [metering] Meeting agenda for Thursday at 16:00 UTC (July 12th, 2012)
Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - nijaba to send an email to the PPB for Incabation application - nijaba to send call for candidate to the general mailing list - jd to setup opa voting system to start on 26th and end on Aug 3rd - nijaba to prime a roadmap page and invite others to populate it - jd handle counter/meter type for real - nijaba to document external API use with a clear warning about the limitation of the sum and duration function * Discuss and define actions from the PPB discussion on Tue regarding our incubation. - http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-10-20.01.html - http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-10-20.01.log.html * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Bandwidth limit
On 07/10/2012 03:55 AM, Guillermo Alvarado wrote: Hi Everyone! I want to know if there is a table or something that contains information about the bandwidth of each tenant, to billing purposes. This is not complete yet, but is part of the data that we are trying to collect [1] in the Ceilometer project [2]. We now have a nice guide explaining how to contribute additional plugins [3], help is alsways welcome. [1] http://wiki.openstack.org/EfficientMetering#Meters [2] https://launchpad.net/ceilometer [3] http://wiki.openstack.org/EfficientMetering#Contributing_to_Ceilometer There are a way to limit the bandwith of the tenants?? This would be a request for Quantum, which I don't think has been fully addressed yet, based on this thread [4]. [4] http://www.mail-archive.com/openstack@lists.launchpad.net/msg13611.html Hope this helps, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer application for incubation
On 07/05/2012 09:22 PM, Jonathan Bryce wrote: Thanks, Nick. I've added it to the agenda for next Tuesday's meeting at 20:00 UTC/3:00 PM CDT. If you can join the meeting to field any questions, that would be helpful. Jonathan Great! I'll be there, hopefully joined by other team members. Thanks, Nick On Jul 5, 2012, at 12:52 PM, Nick Barcet wrote: Dear members of the Project Policy Board, After 3 month of work on the Ceilometer project, and great progress being made, our last meeting IRC meeting [1] validated that we should be submitting this project for incubation. Following the OpenStack project rules, we have completed the incubation application form which you will find at [2]. We would be happy to answer any question you may have about our application and hope that you will give a favorable answer to our request. [1] http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-05-16.00.html [2] http://wiki.openstack.org/Governance/Proposed/Ceilometer On behalf of the Ceilometer project team, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Meeting agenda for Thursday at 16:00 UTC (July 5th, 2012)
On 07/04/2012 08:16 PM, Nick Barcet wrote: Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions (there does not seem to be any) * Discuss and vote the incubation application proposal to be submitted to PPB http://wiki.openstack.org/Projects/IncubatorApplicationCeilometer * PTL election process discussion When we had our first project meeting on April 26th, it was agreed that Loic Dachary and I would co-lead this project for the first 3 momths. Time is soon up for those 3 months, so we should discuss what to do to. Some reference: - http://wiki.openstack.org/Governance/Model#Governance%20for%20the%20Individual%20OpenStack%20Projects - http://www.cs.cornell.edu/andru/civs.html - http://www.opavote.org/ * Discuss project's next priorities * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, The meeting just occurred and here is the summary: == #openstack-meeting: Ceilometer == Meeting started by nijaba at 16:00:13 UTC. The full logs are available at http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-05-16.00.log.html Meeting summary --- * LINK: http://wiki.openstack.org/Meetings/MeteringAgenda (nijaba, 16:00:13) * jd__ will unfortunately not be able to join us this week (nijaba, 16:00:13) * actions from previous meetings (nijaba, 16:01:27) * Discuss and vote the incubation application proposal to be submitted to PPB (nijaba, 16:02:19) * LINK: http://wiki.openstack.org/Projects/IncubatorApplicationCeilometer (nijaba, 16:02:19) * VOTE: Voted on Submit the incubation application to the PPB? Results are, Yes: 4 (nijaba, 16:05:14) * ACTION: nijaba to send an email to the PPB (nijaba, 16:05:48) * PTL election process discussion (nijaba, 16:06:07) * LINK: http://wiki.openstack.org/Governance/Model#Governance%20for%20the%20Individual%20OpenStack%20Projects (nijaba, 16:06:35) * LINK: http://www.cs.cornell.edu/andru/civs.html (nijaba, 16:06:35) * LINK: http://www.opavote.org/ (nijaba, 16:06:35) * LINK: http://wiki.openstack.org/Governance/Foundation/TechnicalCommittee (nijaba, 16:06:35) * VOTE: Voted on go ahead with the proposed PTL election process? Results are, Yes: 4 (nijaba, 16:12:04) * ACTION: nijaba to send call for candidate to the general mailing list (nijaba, 16:13:43) * ACTION: jd___ to setup opa voting system to start on 26th and end on Aug 3rd (nijaba, 16:14:15) * Discuss project's next priorities (nijaba, 16:14:36) * ACTION: nijaba to prime a roadmap page and invite others to populate it (nijaba, 16:19:46) * ACTION: jd___ handle counter/meter type for real (jd___, 16:22:13) * Open Discusssion (nijaba, 16:24:05) * LINK: http://openstack.markmail.org/thread/fwkagzdzpleeclj5 (nijaba, 16:24:16) * ACTION: nijaba to document external API use with a clear warning about the limitation of the sum and duration function (nijaba, 16:25:55) Meeting ended at 16:27:40 UTC. Action items, by person --- * jd___ * jd___ to setup opa voting system to start on 26th and end on Aug 3rd * jd___ handle counter/meter type for real * nijaba * nijaba to send an email to the PPB * nijaba to send call for candidate to the general mailing list * nijaba to prime a roadmap page and invite others to populate it * nijaba to document external API use with a clear warning about the limitation of the sum and duration function People present (lines said) --- * nijaba (90) * dhellmann (33) * jd___ (20) * flacoste (16) * openstack (15) signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Time for a UK Openstack User Group meeting ?
On 07/04/2012 05:38 PM, Day, Phil wrote: Hi All, I’m thinking it’s about time we had an OpenStack User Group meeting in the UK , and would be interested in hearing from anyone interested in attending, presenting, helping to organise, etc. London would seem the obvious choice, but we could also host here in HP Bristol if that works for people. Reply here or e-mail me directly (phil@hp.com), and if there’s enough interest I’ll pull something together. Canonical would be delighted to host that meeting in our new london office. We have capacity for up to 100 people and are currently checking availability, but are looking for a date in the last week july. We have started the OpenStack-London meetup [1] so anyone interested can subscribe. [1] http://www.meetup.com/Openstack-London/ Cheers, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Ceilometer application for incubation
Dear members of the Project Policy Board, After 3 month of work on the Ceilometer project, and great progress being made, our last meeting IRC meeting [1] validated that we should be submitting this project for incubation. Following the OpenStack project rules, we have completed the incubation application form which you will find at [2]. We would be happy to answer any question you may have about our application and hope that you will give a favorable answer to our request. [1] http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-05-16.00.html [2] http://wiki.openstack.org/Governance/Proposed/Ceilometer On behalf of the Ceilometer project team, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] resource metadata changes and billing
On 06/29/2012 03:04 PM, Doug Hellmann wrote: [..] My conclusion from all of this (over-)thinking is that the ceilometer API should assume the simple case and ignore the metadata changes when computing the sum or maximum value for a counter over a range of time. More complex processing will be left up to the caller, who can ask for raw metering data in manageable chunks and process them outside of the API. I could be persuaded to do something more complicated if the problems described above can be solved in a relatively simple way, but even then I think we should push that to the v2 API. [..] Sorry for my late reply on this, but... So, if I summarize what you are saying, the problem is that for a given Instance ID, a given meter may have to be interpreted as if the Instance ID was changing over time. Example: t1: Instance A - Has 1 CPU - 64G ram - runs in zone 1 t2: Instance A - Has 2 CPU - 64G ram - runs in zone 1 t3: Instance A - Has 2 CPU - 128G ram - runs in zone 1 t4: Instance A - Has 2 CPU - 128G ram - runs in zone 3 t5: Instance A is stopped From a billing point of view, what is important here is that even though the Instance ID remains the same, we have in fact 4 different segments of time which could lead to 4 different pricing being applied to the same instance: t1-t2: price 1 t2-t3: price 2 t3-t4: price 3 t4-t5: price 4 So we need to be able to inform the rating engine that these events have occurred so that it does not uniformly apply a billing price to from a sum of a given meter volume. But in fact this information is indeed captured and accessible to rating engines via their respective meters. What is interesting here is that, in my mind, the sum and duration function of the API, when I proposed it, were only meant to be able to: * In a simple amazon type billing model where instances cannot change zone, add CPU or add ram, * In a Private cloud scenario where you only need simple usage stats to inform your users, * in a horizon plugin to give a quick summary of use, and would never be used by any serious rating engines that would in each and every case require to have access to the raw list of events so that it can recreate the full time line of the events. This is where we need to draw the line between metering and rating. I therefore propose that we leave the API as is, knowing the side effects of such high level sum and duration calculations. If we agree on this, I take the action to document the limitation of the summary functions of the API. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [metering] Meeting agenda for Thursday at 16:00 UTC (July 5th, 2012)
Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions (there does not seem to be any) * Discuss and vote the incubation application proposal to be submitted to PPB http://wiki.openstack.org/Projects/IncubatorApplicationCeilometer * PTL election process discussion When we had our first project meeting on April 26th, it was agreed that Loic Dachary and I would co-lead this project for the first 3 momths. Time is soon up for those 3 months, so we should discuss what to do to. Some reference: - http://wiki.openstack.org/Governance/Model#Governance%20for%20the%20Individual%20OpenStack%20Projects - http://www.cs.cornell.edu/andru/civs.html - http://www.opavote.org/ * Discuss project's next priorities * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] resource metadata changes and billing
On 07/04/2012 10:55 PM, Julien Danjou wrote: On Wed, Jul 04 2012, Nick Barcet wrote: I therefore propose that we leave the API as is, knowing the side effects of such high level sum and duration calculations. If we agree on this, I take the action to document the limitation of the summary functions of the API. So, if I understand you correctly, that would mean the API is non-usable for anybody wanting to do fine-grained billing and they would have to connect to the database engine directly? Because if that's the case I think we miss an abstraction layer somewhere. No, this is not what I am saying. The API offers two types of calls: * One which gives access to organized raw events, which rating engines are likely to be using, * One which gives access to summary data, but which are only valid for simple use cases. Hopefully, there are no cases here that should justify connecting to the db engine directly unless one really wants to, and that should not be recommended as we may want to change our storage model over time without breaking existing integrations. The possible variants for describing possible billing strategies are indeed one abstraction level above this. The telco industry calls the tool handling this a rating engine (what transforms metering into line items of a bill) and I intentionally proposed since the beginning that Ceilometer should not be a rating engine. We can hope that our current API will be able to evolve to simplify rating engines requests over time, and we should be ready to accept extension of it to do so. However I do think we do not have the necessary experience with rating to propose a universal solution that will effectively work at this time, and therefore am proposing that we postpone that until we do have the experience. Makes sense? -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Metering] First draft of plugin/agent documentation
I've just committed a first of the plugin/agent documentation to stackforge/ceilometer. Reviews welcome: https://review.stackforge.org/#/c/250/ Thanks, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Cinder usage data retrieval
On 06/21/2012 12:44 PM, Thomas, Duncan wrote: John Griffith on 20 June 2012 18:26 wrote: On Wed, Jun 20, 2012 at 10:53 AM, Nick Barcet nick.bar...@canonical.com wrote: What we want is to retrieve the maximum amount of data, so we can meter things, to bill them in the end. For now and for Cinder, this would first include (per user/tenant): - the amount of reserved volume space - the amount of used volume space - the number of volumes but we'll need probably more in a near future. We should chat about how things are shaping up so far and how you're implementing things on the other sides (consistency where practical/possible). Also, it sort of depends on the architecture and use model details of Ceilometer, which I hate to admit but I'm not really up to speed on. My first reaction/thought is the best most appropriate place to tie in is via the python-cinderclient. There would be a number of ways to obtain some of this info, whether deriving it or maybe some extensions to obtain things directly. One thing to watch for here is access control... Don't want one tenant able to find out about another's usage. Probably not important on a private cloud deployment, but certainly important in the public cloud space. Having a separate endpoint to do this kind of admin stuff over also means you can have much tighter IP level access controls... It would seem to me that the easiest way would be for calls that are emitted from the same host as cinder manager on the local interface to be handled as admin calls. We could then place Ceilometer's cinder agent on the same host, which would not be allowed to any user, thus not creating a breach of security. Thoughts? Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Get vcpu hours per instance
On 06/15/2012 10:21 PM, Guillermo Alvarado wrote: Hi everyone! I want to know the vcpu hours per instance, in horizon In nova dashboard I can see the total hours for the tenant, I want to see the hours for each instance, which together must be equal to total. How can I do that? I think the api does not have some method like that, What do you propose? Hello Guillermo, While it is not ready yet for consumption, this is part of the dataset that we are targeting for the ceilometer project. Feel free to join us and help moving this effort forward. https://launchpad.net/ceilometer -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Meeting agenda for Thursday at 16:00 UTC (June 21st, 2012)
On 06/20/2012 08:55 PM, Nick Barcet wrote: Hi, The metering project team holds a weekly meeting in #openstack-meeting, Thursdays at 1600 UTC. Everyone is welcome. http://wiki.openstack.org/Meetings/MeteringAgenda Topics for this week: * Review last week's actions - nijaba: to point to the calculator in the blueprint - dhellmann: start mapping API queries to database engine methods - dhellmann: look at existing plugins and pick one of each for examples in docs - dhellmann: email info on examples to nijaba - nijaba: to prime the doc once info received from dhellmann - nijaba: to prepare incubation application for review at the next meeting - dachary: talk to Dragon about SystemData / ceilometer and try to create cooperation - dhellmann: to talk to Quantum devs about integration with ceilometer - dachary: to talk to swift devs about integration with ceilometer - nijaba: to talk to cinder devs about integration with ceilometer - dachary: talk to Dragon about SystemData / ceilometer and try to create cooperation (i.e. nova) - jd_: to talk to glance devs about integration with ceilometer * Status of the essex compatibility effort that jd is leading Feel free to add other points you would like to discuss to the wiki page. The meeting took place and here is a summary: == #openstack-meeting: Ceilometer == Meeting summary --- * LINK: http://wiki.openstack.org/Meetings/MeteringAgenda (nijaba, 16:00:03) * dachary will unfortunately not be able to join us this week for work related reasons, he asked me to present his excuses. (nijaba, 16:00:23) * actions from previous meetings (nijaba, 16:01:04) * nijaba: to point to the calculator in the blueprint (nijaba, 16:01:17) * LINK: http://wiki.openstack.org/EfficientMetering#Volume_of_data (nijaba, 16:01:17) * dhellmann: start mapping API queries to database engine methods (nijaba, 16:01:46) * dhellmann: look at existing plugins and pick one of each for examples in docs and email info on examples to nijaba (nijaba, 16:04:14) * nijaba: to prime the doc once info received from dhellmann (nijaba, 16:09:09) * ACTION: nijaba: to prime the doc on plugins (nijaba, 16:09:09) * nijaba: to prepare incubation application for review at the next meeting (nijaba, 16:10:27) * LINK: http://wiki.openstack.org/Projects/IncubatorApplicationCeilometer (nijaba, 16:10:27) * ACTION: during next week meeting we will vote on the Incubation application (nijaba, 16:12:44) * dachary: talk to Dragon about SystemData / ceilometer and try to create cooperation (nijaba, 16:13:08) * ACTION: dachary: talk to Dragon about SystemData / ceilometer and try to create cooperation (nijaba, 16:13:22) * dhellmann: to talk to Quantum devs about integration with ceilometer (nijaba, 16:13:32) * ACTION: dhellmann: talk to Quantum devs about integration with ceilometer (dhellmann, 16:14:13) * dachary: to talk to swift devs about integration with ceilometer (nijaba, 16:14:29) * ACTION: dachary: to talk to swift devs about integration with ceilometer (nijaba, 16:14:45) * nijaba: to talk to cinder devs about integration with ceilometer (nijaba, 16:14:57) * jd___: to talk to glance devs about integration with ceilometer (nijaba, 16:16:47) * Status of the essex compatibility effort that jd is leading (nijaba, 16:21:42) * ACTION: jd___ check that openstack-ci runs the tests for Essex (jd___, 16:27:00) * ACTION: jd___ add some tests on daemon launching (jd___, 16:27:11) * discussion about cinder integration (nijaba, 16:27:49) * ACTION: nijaba to question best method to authenticate admin client calls from ceilometer to cinder (and other projects) (nijaba, 16:36:00) * ACTION: nijaba to include adding events to cinder to previous action (nijaba, 16:43:37) * LINK: http://wiki.openstack.org/NovaVolumeMeetings (nijaba, 16:46:01) * open discussion (nijaba, 16:47:55) * ACTION: dhellmann and nijba will be joining and nijaba will initiate a private thread with jgriffith (nijaba, 16:51:26) Meeting ended at 16:52:14 UTC. Action items, by person --- * dhellmann * dhellmann: talk to Quantum devs about integration with ceilometer * dhellmann and nijaba will be joining and nijaba will initiate a private thread with jgriffith * dachary * dachary: talk to Dragon about SystemData / ceilometer and try to create cooperation * dachary: to talk to swift devs about integration with ceilometer * jd___ * jd___ check that openstack-ci runs the tests for Essex * jd___ add some tests on daemon launching * jgriffith * dhellmann and nijba will be joining and nijaba will initiate a private thread with jgriffith * nijaba * nijaba: to prime the doc on plugins * nijaba to question best method to authenticate admin client calls from
[Openstack] [metering] Cinder usage data retrieval
Hello John (or anyone else working on cinder), As part of the ceilometer project¹, we're working on usage data retrieval from various OpenStack components. One of them is Cinder. We're targeting Folsom for the first release, therefore it seems important for both projects to be able to work together, this is why we're bringing ceilometer to your attention and asking for advices. :) What we want is to retrieve the maximum amount of data, so we can meter things, to bill them in the end. For now and for Cinder, this would first include (per user/tenant): - the amount of reserved volume space - the amount of used volume space - the number of volumes but we'll need probably more in a near future. Do you have any advice regarding integration of Ceilometer and Cinder together? What would be a stable interface we could rely on that would be independent of the backend? Thanks in advance, Regards, ¹ http://launchpad.net/ceilometer -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Cinder usage data retrieval
On 06/20/2012 07:26 PM, John Griffith wrote: On Wed, Jun 20, 2012 at 10:53 AM, Nick Barcet nick.bar...@canonical.com wrote: Hello John (or anyone else working on cinder), As part of the ceilometer project¹, we're working on usage data retrieval from various OpenStack components. One of them is Cinder. We're targeting Folsom for the first release, therefore it seems important for both projects to be able to work together, this is why we're bringing ceilometer to your attention and asking for advices. :) What we want is to retrieve the maximum amount of data, so we can meter things, to bill them in the end. For now and for Cinder, this would first include (per user/tenant): - the amount of reserved volume space - the amount of used volume space - the number of volumes but we'll need probably more in a near future. Do you have any advice regarding integration of Ceilometer and Cinder together? What would be a stable interface we could rely on that would be independent of the backend? Thanks in advance, Regards, ¹ http://launchpad.net/ceilometer -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas Hi Nick, We should chat about how things are shaping up so far and how you're implementing things on the other sides (consistency where practical/possible). Also, it sort of depends on the architecture and use model details of Ceilometer, which I hate to admit but I'm not really up to speed on. My first reaction/thought is the best most appropriate place to tie in is via the python-cinderclient. There would be a number of ways to obtain some of this info, whether deriving it or maybe some extensions to obtain things directly. Sounds great. We have our weekly meeting on Thurdays at 4PM UTC [1] which you would be welcome to join, or I can set up a bridge for us a phone chat. Let me know what you would prefer. [1] http://wiki.openstack.org/Meetings/MeteringAgenda Cheers, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Metering] Meeting agenda for Thursday at 16:00 UTC (June 21st, 2012)
Hi, The metering project team holds a weekly meeting in #openstack-meeting, Thursdays at 1600 UTC. Everyone is welcome. http://wiki.openstack.org/Meetings/MeteringAgenda Topics for this week: * Review last week's actions - nijaba: to point to the calculator in the blueprint - dhellmann: start mapping API queries to database engine methods - dhellmann: look at existing plugins and pick one of each for examples in docs - dhellmann: email info on examples to nijaba - nijaba: to prime the doc once info received from dhellmann - nijaba: to prepare incubation application for review at the next meeting - dachary: talk to Dragon about SystemData / ceilometer and try to create cooperation - dhellmann: to talk to Quantum devs about integration with ceilometer - dachary: to talk to swift devs about integration with ceilometer - nijaba: to talk to cinder devs about integration with ceilometer - dachary: talk to Dragon about SystemData / ceilometer and try to create cooperation (i.e. nova) - jd_: to talk to glance devs about integration with ceilometer * Status of the essex compatibility effort that jd is leading Feel free to add other points you would like to discuss to the wiki page. Cheers, -- Nick Barcet nick.bar...@canonical.com aka nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Meeting agenda for Thursday at 16:00 UTC (June 14th, 2012)
On 06/13/2012 12:39 PM, Nick Barcet wrote: Hi, The metering project team holds a weekly meeting in #openstack-meeting, Thursdays at 1600 UTC. Everyone is welcome. http://wiki.openstack.org/Meetings/MeteringAgenda Topics for this week: * Review last week's actions * dhellmann: submit plugin branch for review and merging * dhellmann rename plugin to engine for storage backend ex-plugin-now-engine system * dhellmann: start mapping API queries to database engine methods * nijaba to propose a google spreadsheet calculator to estimate volume of metering message (including nova, swift, cinder, quantum) * Discuss and hopefully agree on meter configuration mechanism [1] * Discuss proposing ceilometer as an incubated project * Prepare documentation and framework for plugin contributors * Establish communication with swift/quantum/cinder on best points of interaction for our agents * Status of the essex compatibility effort that jd___ is leading [1]http://www.mail-archive.com/openstack@lists.launchpad.net/msg12859.html Here is the summary of the meeting that took place: == #openstack-meeting Meeting == Meeting started by nijaba at 16:00:06 UTC. The full logs are available at http://eavesdrop.openstack.org/meetings/openstack-meetin /2012/openstack-meeting.2012-06-14-16.00.log.html Meeting summary --- * LINK: http://wiki.openstack.org/Meetings/MeteringAgenda (nijaba, 16:00:16) * actions from previous meetings (nijaba, 16:00:45) * nijaba: to propose a google spreadsheet calculator to estimate volume of metering message (including nova, swift, cinder, quantum) (nijaba, 16:03:12) * LINK: https://docs.google.com/spreadsheet/ccc?key=0AtziNGvs-uPudDhRbEJJOHFXV3d0ZGc1WE9NLTVPX0E (nijaba, 16:03:29) * ACTION: nijaba to point to the calculator in the blueprint (nijaba, 16:06:21) * dhellmann: submit plugin branch for review and merging (nijaba, 16:06:36) * DONE * dhellmann rename plugin to engine for storage backend ex-plugin-now-engine system (nijaba, 16:07:03) * DONE * dhellmann: start mapping API queries to database engine methods (nijaba, 16:07:22) * ACTION: dhellmann: start mapping API queries to database engine methods (nijaba, 16:08:01) * Discuss and hopefully agree on meter configuration mechanism (nijaba, 16:08:21) * LINK: http://www.mail-archive.com/openstack@lists.launchpad.net/msg12859.html (nijaba, 16:08:35) * AGREED: push meter configuration to a future version (nijaba, 16:21:49) * Discuss proposing ceilometer as an incubated project (nijaba, 16:22:16) * AGREED: propose ceilometer as an incubated project (nijaba, 16:30:13) * ACTION: nijaba to send a proposal to the mailing list (nijaba, 16:30:29) * Prepare documentation and framework for plugin contributors (nijaba, 16:30:45) * ACTION: dhellmann: look at existing plugins and pick one of each for examples in docs (dhellmann, 16:35:12) * ACTION: dhellmann: email info on examples to nijaba (dhellmann, 16:36:04) * ACTION: nijaba to prime the doc once info received from dhellmann (nijaba, 16:36:34) * incubation (nijaba, 16:37:31) * LINK: http://wiki.openstack.org/ProjectTypes (nijaba, 16:37:45) * LINK: http://wiki.openstack.org/Governance/Approved/Incubation (ttx, 16:37:54) * LINK: http://wiki.openstack.org/Governance/Foundation/TechnicalCommittee (ttx, 16:47:13) * ACTION: nijaba to prepare incubation application for review at the next meeting (nijaba, 16:48:02) * Establish communication with swift/quantum/cinder on best points of interaction for our agents (nijaba, 16:49:10) * ACTION: dachary talk to Dragon about SystemData / ceilometer and try to create cooperation (dachary, 16:52:50) * ACTION: dhellmann to talk to Quantum devs about integration with ceilometer (dhellmann, 16:53:26) * ACTION: dachary to talk to swift devs about integration with ceilometer (dachary, 16:54:47) * ACTION: nijaba to talk to cinder devs about integration with ceilometer (nijaba, 16:55:01) * ACTION: dachary talk to Dragon about SystemData / ceilometer and try to create cooperation (i.e. nova) (dachary, 16:55:45) * ACTION: jd___ to talk to glance devs about integration with ceilometer (nijaba, 17:00:03) * Status of the essex compatibility effort that jd is leading (nijaba, 16:57:24) * In progress, to be continued next meeting. Meeting ended at 17:00:51 UTC. Action items, by person --- * dachary * dachary talk to Dragon about SystemData / ceilometer and try to create cooperation * dachary to talk to swift devs about integration with ceilometer * dachary talk to Dragon about SystemData / ceilometer and try to create cooperation (i.e. nova) * dhellmann * dhellmann: start mapping API queries to database engine methods * dhellmann: look at existing plugins
[Openstack] [Metering] Meeting agenda for Thursday at 16:00 UTC (June 14th, 2012)
Hi, The metering project team holds a weekly meeting in #openstack-meeting, Thursdays at 1600 UTC. Everyone is welcome. http://wiki.openstack.org/Meetings/MeteringAgenda Topics for this week: * Review last week's actions * dhellmann: submit plugin branch for review and merging * dhellmann rename plugin to engine for storage backend ex-plugin-now-engine system * dhellmann: start mapping API queries to database engine methods * nijaba to propose a google spreadsheet calculator to estimate volume of metering message (including nova, swift, cinder, quantum) * Discuss and hopefully agree on meter configuration mechanism [1] * Discuss proposing ceilometer as an incubated project * Prepare documentation and framework for plugin contributors * Establish communication with swift/quantum/cinder on best points of interaction for our agents * Status of the essex compatibility effort that jd___ is leading [1]http://www.mail-archive.com/openstack@lists.launchpad.net/msg12859.html Cheers, -- Nick Barcet nick.bar...@canonical.com aka nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] notification metadata
On 06/12/2012 05:51 PM, Doug Hellmann wrote: See https://review.stackforge.org/163 for the code. On Tue, Jun 12, 2012 at 11:05 AM, Doug Hellmann doug.hellm...@dreamhost.com mailto:doug.hellm...@dreamhost.com wrote: I am working on bug 1006120, adding metadata to the instance-related metering events. It's easy to add location data like availability zone to the pollsters because the agent has an object from the database with all of the information about the instance (we will have to change that, but presumably we will be able to get the data via RPC, too). The notification plugins however, only have the data in the incoming message, and those messages don't include all of the data about the instance. My current plan is to have the collector code that processes notifications look up the instance in the database to get the rest of the data. Does anyone have any thoughts on this plan? https://bugs.launchpad.net/ceilometer/+bug/1006120 Well, this seems a bit inelegant to me to have collectors to additional lookup in an external DB where otherwise it's role would be limited to handle and store event. It would definitely add quite a bit of strain to the code on the central collector and what would happen if the nova DB is not available or overloaded? I'd rather have all collection of data done from the agents rather than introduce this lookup on the central collector node. What would prevent us from doing it this way? Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] notification metadata
On 06/13/2012 01:55 PM, Doug Hellmann wrote: On Wed, Jun 13, 2012 at 6:56 AM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com wrote: On 06/12/2012 05:51 PM, Doug Hellmann wrote: See https://review.stackforge.org/163 for the code. On Tue, Jun 12, 2012 at 11:05 AM, Doug Hellmann doug.hellm...@dreamhost.com mailto:doug.hellm...@dreamhost.com mailto:doug.hellm...@dreamhost.com mailto:doug.hellm...@dreamhost.com wrote: I am working on bug 1006120, adding metadata to the instance-related metering events. It's easy to add location data like availability zone to the pollsters because the agent has an object from the database with all of the information about the instance (we will have to change that, but presumably we will be able to get the data via RPC, too). The notification plugins however, only have the data in the incoming message, and those messages don't include all of the data about the instance. My current plan is to have the collector code that processes notifications look up the instance in the database to get the rest of the data. Does anyone have any thoughts on this plan? https://bugs.launchpad.net/ceilometer/+bug/1006120 Well, this seems a bit inelegant to me to have collectors to additional lookup in an external DB where otherwise it's role would be limited to handle and store event. It would definitely add quite a bit of strain to the code on the central collector and what would happen if the nova DB is not available or overloaded? I want to eventually move off of direct DB access and ask for details like that via RPC, so this is just a temporary situation. I'd rather have all collection of data done from the agents rather than introduce this lookup on the central collector node. What would prevent us from doing it this way? That moves the problem even further away from the database. The agent runs on the compute node, and we *know* we won't have DB access there by the time we're done with Folsom (even nova is eliminating direct DB access by the compute node). The agent does not process notification events (it only polls) because of the round-tripping of messages through the message bus. Even if we move the notification handling into the agent, it doesn't have the necessary details so we would still need to ask for them somehow. Yes, I see your point. Bad idea. There are a couple of other alternatives: 1. We could move notification handling into its own daemon to get it out of the collector. This new daemon would still run on a central service, and would need to be set up to support load balancing just as the collector is now. The similarities are why we left the two types of processing in the same process to begin with. RPC seems like a better solution for this, doesn't it? 2. We could modify nova to include more details about an instance when it publishes a notification. This is the best solution from our perspective, and I would like to see all of the properties anyway, but I don't know if there is a performance-related reason for leaving out some details. That would be definitely best. Any nova dev willing to comment here? Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Metering] Meter configuration mechanism v0.2
Following up on our last meeting and the comments collected on the mailing list, here is a second version of the proposal for centrally hosting configuration of meters in ceilometer. The main idea is that all meters of a given type should be sending similarly formatted information in order for the information to be usable, hence the need to ensure that configuration info is centrally stored and retrieved. This would rule out, in my mind, the idea that we could use the global flags object, as distribution of the configuration file is left to the cloud implementor and does not lend for easy and synchronized updates of agent config. This proposal is by design limited to the meter configuration, not to the agent configuration itself which is still pushed using the global object flags. Configuration format and content is left to the agent's implementation, but it is assumed that each meter covered by an agent can be at least enabled or disabled (and eventually, in a future version of the agents, set to send information at a meter specific frequency - it is currently a global value for all meters of an agent). 1/ Configuration is stored for each agent in flat file or DB as follow +--- | Field | Type | Note +--- | meter_type | String | Unique meter type | conf_vers | Integer | Version of the configuration | (simple incremental counter) | config | Text | JSON Configuration info (defined by meter) ++--++ 2/ Config is retreived via the messaging queue upon start of the agent Request is sent by the agent for each meter upon start to retrieve to current config version. If a config is found, it is sent back to the agent, along with conf_vers. Else the collector creates the record based on default value returned by the agent/meter plugin installed on the collector, sets conf_vers to 1 and sends back a normal reply. 3/ When the content of the configuration is modified (conf_vers is updated), a cast is sent to all agents publishing a given meter with the content of the current configuration for immediate update. Comments welcome... Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [metering] Ceilometer volume calculator
Hello, Following up on yesterday's meeting, I have started a first version of a google spreadsheet to estimate volume of events generated by ceilometer [1]. Comments and suggestions for improvement are of course welcome. [1] https://docs.google.com/spreadsheet/ccc?key=0AtziNGvs-uPudDhRbEJJOHFXV3d0ZGc1WE9NLTVPX0E Cheers, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] polling intervals, was: Agent configuration mechanism
On 06/06/2012 05:06 PM, Doug Hellmann wrote: Starting a new thread for this topic... On Wed, Jun 6, 2012 at 5:43 AM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com wrote: On 06/05/2012 09:03 PM, Doug Hellmann wrote: Right now we only have one interval for all polling. Do you think we need to add support for polling different values at different intervals? Do we need other per-agent settings, or are all of the settings the same for all agents? (I had assumed the latter would be all we needed.) I would have thought that we may want to support different intervals per meters, based on the billing rules that one may want to offer. For example, I may want to bill compute by the hour but floating IPs by the day, hence have a different reporting interval for each. I was planning to aggregate the values for items being billed over the longer time frames, but we can make the polling interval configurable. It will take some work, because of the way the scheduled tasks are configured in the service and manager (right now we just schedule one method to run, and it invokes each pollster). How important is it to include this in Folsom? Not crucial. I would classify this as Nice to have. We need a ticket to track this. It's going to require some work on the agent service, because right now the code that loads the plugins doesn't have access to the object that knows how to run jobs at regular intervals. I would rather have a complete tool that works at a single interval, then go back and enhance it to allow other intervals during the next release cycle. Is that going to meet your needs? As I said, this would not be a hard requirement for me, so no issues there. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] signing messages, was Re: Agent configuration mechanism
On 06/06/2012 05:20 PM, Doug Hellmann wrote: Starting a new thread... On Wed, Jun 6, 2012 at 5:43 AM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com wrote: On 06/05/2012 09:03 PM, Doug Hellmann wrote: On Tue, Jun 5, 2012 at 12:59 PM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com mailto:nick.bar...@canonical.com mailto:nick.bar...@canonical.com wrote: On 06/05/2012 04:44 PM, Doug Hellmann wrote: On Tue, Jun 5, 2012 at 10:41 AM, Doug Hellmann doug.hellm...@dreamhost.com mailto:doug.hellm...@dreamhost.com mailto:doug.hellm...@dreamhost.com mailto:doug.hellm...@dreamhost.com mailto:doug.hellm...@dreamhost.com mailto:doug.hellm...@dreamhost.com mailto:doug.hellm...@dreamhost.com mailto:doug.hellm...@dreamhost.com wrote: On Tue, Jun 5, 2012 at 9:56 AM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com mailto:nick.bar...@canonical.com mailto:nick.bar...@canonical.com mailto:nick.bar...@canonical.com mailto:nick.bar...@canonical.com mailto:nick.bar...@canonical.com mailto:nick.bar...@canonical.com wrote: Have you given any thought to distributing the secret value used for signing incoming messages? A central configuration authority does not give us a secure way to deliver secrets like that. If anyone with access to the message queue can retrieve the key by sending RPC requests, we might as well not sign the messages. Actually, the private key used to generate a signature should be unique to each host, if we want them to have any value at all, therefore distributing a common signature should NOT be part of this, or we would fall under the notion of a shared secret, which is, IMHO, not any better than having a global password. I would recommend that, for the time being, we just generate a random key pair per host the first time the agent is run, allowing for someone with further requirement to eventually populate this value by another mean. In any case, if we want to effectively check the signature, the public key does need to be accessible by the collector to check it and have yet to define a way to do so... Proposals welcome, but again, while I think we should lay the ground for a great security experience, we certainly don't need to solve it all in v1. The current implementation uses hmac message signatures, which use a shared secret instead of public/private key pairs. We can have a separate secret for each agent, but we still need the collector(s) to have them all. I thought the point of signing the messages was to prevent an arbitrary agent from signing on to the message queue and sending bogus data. Do we need to be doing more than hmac for security? As stated since the beginning of the project, purpose of the signature is non repudiability, not only authentication. If I understand correctly, hmac signature will only provide authentication through a shared secret, shared secret which then should not be transmited on the wire to the agent, or else it would loose all purpose. Yes, hmac uses a shared secret to produce a SHA-256 (in our case) signature of the contents of the message. How that is interpreted is up to the application. For ceilometer it seemed like hmac would be sufficient for both ensuring that the agent was allowed to send messages (because it knows the secret) and that the message contents had not been modified in transit (because the signature is valid). We could have a separate secret for each agent, but I don't see how a keypair is better for this purpose. I'm no crypto expert, though. :-) To cover the non-repudiability requirement the message includes a UUID1 value, which includes the MAC of the sending host as well as time and counter components. No two agents will produce the same UUID1 value at the same time, and no two messages from the same agent will have the same value. Not a crypto/security expert either, so I think that, for the time being, we could very well live with the scenario you describe. As long at the signature and check function that we put in place are distinct from the rest of the code, we can assume that someone with stronger or different requirements should be able to patch and eventually submit, right? Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack
Re: [Openstack] [Metering] Agent configuration mechanism
On 06/05/2012 09:03 PM, Doug Hellmann wrote:
On Tue, Jun 5, 2012 at 12:59 PM, Nick Barcet nick.bar...@canonical.com
mailto:nick.bar...@canonical.com wrote:
On 06/05/2012 04:44 PM, Doug Hellmann wrote:
On Tue, Jun 5, 2012 at 10:41 AM, Doug Hellmann
doug.hellm...@dreamhost.com mailto:doug.hellm...@dreamhost.com
mailto:doug.hellm...@dreamhost.com
mailto:doug.hellm...@dreamhost.com wrote:
On Tue, Jun 5, 2012 at 9:56 AM, Nick Barcet
nick.bar...@canonical.com mailto:nick.bar...@canonical.com
mailto:nick.bar...@canonical.com
mailto:nick.bar...@canonical.com wrote:
Following up on our last meeting, here is a proposal for
centrally
hosting configuration of agents in ceilometer.
The main idea is that all agents of a given type should be
sending
similarly formatted information in order for the
information to be
usable, hence the need to ensure that configuration info is
centrally
stored and retrieved. This would rule out, in my mind,
the idea
that we
could use the global flags object, as distribution of the
configuration
file is left to the cloud implementor and does not lend
for easy and
synchronized updates of agent config.
Configuration format and content is left to the agent's
implementation,
but it is assumed that each meter covered by an agent can be :
* enabled or disabled
* set to send information at a specified interval.
Right now we only have one interval for all polling. Do you
think we
need to add support for polling different values at different
intervals? Do we need other per-agent settings, or are all of the
settings the same for all agents? (I had assumed the latter
would be
all we needed.)
I would have thought that we may want to support different intervals per
meters, based on the billing rules that one may want to offer. For
example, I may want to bill compute by the hour but floating IPs by the
day, hence have a different reporting interval for each.
I was planning to aggregate the values for items being billed over the
longer time frames, but we can make the polling interval configurable.
It will take some work, because of the way the scheduled tasks are
configured in the service and manager (right now we just schedule one
method to run, and it invokes each pollster).
How important is it to include this in Folsom?
Not crucial. I would classify this as Nice to have.
1/ Configuration is stored for each agent in the database
as follow
+---+
| Field | Type | Note
|
+---+
| AgentType | String | Unique agent type
|
| ConfVers | Integer | Version of the configuration
|
| Config| Text | JSON Configuration info (defined by
agent) |
+---+--++
2/ Config is retreived via the messaging queue upon boot
once a day
(this should be defined in the global flags object) to
check if the
config has changed.
Updating the config once a day is not going to be enough in an
environment with a lot of compute nodes.
Two thoughts merged into one sentence there. Need more caffeine.
What I was trying to say, was that updating the config once a day
might
not be enough and in environments with a lot of compute nodes going
around to manually restart the services each time the config changes
will be a pain. See below for more discussion of pushing config
settings
out.
Agreed, and that's why I proposed that the interval for confguration
refresh should be set in the Global object flag (this is something that
can be shared among all the agents).
Request sent by the agent upon boot and :
'reply_to': 'get_config_data',
'correlation_id': x
'version': '1.0',
'args': {'data': {
'AgentType': agent.type,
'CurrentVersion': agent.version,
'ConfigDefault': agent.default,
},
},
Is this a standard OpenStack RPC call?
Not sure about that, but if it can
Re: [Openstack] [Metering] Agent configuration mechanism
On 06/06/2012 10:39 AM, Julien Danjou wrote: On Tue, Jun 05 2012, Nick Barcet wrote: I would have thought that we may want to support different intervals per meters, based on the billing rules that one may want to offer. For example, I may want to bill compute by the hour but floating IPs by the day, hence have a different reporting interval for each. I don't think you want to poll once a day something you bill per day. If you poll only at noon, and I use a resource from 8:00 to 10:00, you'll miss my usage and I'll use resource for free. :) I don't think I specified which interval we would use for each, but thanks for remind us of the limits of sampling. Yes, there's a minimum interval to be configured, but it's the interval that defined how much grained your metering/billing will be. E.g. if it's 1 hour, you won't charge anything used for less than one hour. You probably wants something like 5 minutes or less, and be sure the agent can keep up with the needed polling speed. :) The only point I am trying to make is that the sampling interval is a function of the billing interval. To insure efficient use of the resources, it would be better (not crucial) to ensure that the polling frequency be settable by meter, not just by agent. The later would just force to set the frequency to the lowest value needed for all meters covered by that agent. Again, this is acceptable, but not optimal. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Agent configuration mechanism
On 06/06/2012 10:35 AM, Julien Danjou wrote: On Tue, Jun 05 2012, Nick Barcet wrote: The main idea is that all agents of a given type should be sending similarly formatted information in order for the information to be usable, hence the need to ensure that configuration info is centrally stored and retrieved. This would rule out, in my mind, the idea that we could use the global flags object, as distribution of the configuration file is left to the cloud implementor and does not lend for easy and synchronized updates of agent config. IMHO this is solving a problem that already exists for all other OpenStack components. A problem that no other OpenStack components tried to resolved, AFAIK. So I don't see why we should try to resolve configuration deployment in ceilometer when it's a much larger issue in the project. Maybe the problem of discrepancy of configuration is more an issue for metering than for other components? At least that's my belief. In fact, I may very well want to have differences in configuration of different compute nodes in a single zone, for very valid reasons (ie use different hypervisor settings, balance usage of components, etc...). However if my metering agents are not all set to report the same things, I think I'll be in trouble to correlate anything, hence the need for metering configuration information to be provided centrally, for anything that sets the behavior of meters and which meters to use. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Agent configuration mechanism
On 06/05/2012 04:44 PM, Doug Hellmann wrote:
On Tue, Jun 5, 2012 at 10:41 AM, Doug Hellmann
doug.hellm...@dreamhost.com mailto:doug.hellm...@dreamhost.com wrote:
On Tue, Jun 5, 2012 at 9:56 AM, Nick Barcet
nick.bar...@canonical.com mailto:nick.bar...@canonical.com wrote:
Following up on our last meeting, here is a proposal for centrally
hosting configuration of agents in ceilometer.
The main idea is that all agents of a given type should be sending
similarly formatted information in order for the information to be
usable, hence the need to ensure that configuration info is
centrally
stored and retrieved. This would rule out, in my mind, the idea
that we
could use the global flags object, as distribution of the
configuration
file is left to the cloud implementor and does not lend for easy and
synchronized updates of agent config.
Configuration format and content is left to the agent's
implementation,
but it is assumed that each meter covered by an agent can be :
* enabled or disabled
* set to send information at a specified interval.
Right now we only have one interval for all polling. Do you think we
need to add support for polling different values at different
intervals? Do we need other per-agent settings, or are all of the
settings the same for all agents? (I had assumed the latter would be
all we needed.)
I would have thought that we may want to support different intervals per
meters, based on the billing rules that one may want to offer. For
example, I may want to bill compute by the hour but floating IPs by the
day, hence have a different reporting interval for each.
1/ Configuration is stored for each agent in the database as follow
+---+
| Field | Type | Note
|
+---+
| AgentType | String | Unique agent type
|
| ConfVers | Integer | Version of the configuration
|
| Config| Text | JSON Configuration info (defined by
agent) |
+---+--++
2/ Config is retreived via the messaging queue upon boot once a day
(this should be defined in the global flags object) to check if the
config has changed.
Updating the config once a day is not going to be enough in an
environment with a lot of compute nodes.
Two thoughts merged into one sentence there. Need more caffeine.
What I was trying to say, was that updating the config once a day might
not be enough and in environments with a lot of compute nodes going
around to manually restart the services each time the config changes
will be a pain. See below for more discussion of pushing config settings
out.
Agreed, and that's why I proposed that the interval for confguration
refresh should be set in the Global object flag (this is something that
can be shared among all the agents).
Request sent by the agent upon boot and :
'reply_to': 'get_config_data',
'correlation_id': x
'version': '1.0',
'args': {'data': {
'AgentType': agent.type,
'CurrentVersion': agent.version,
'ConfigDefault': agent.default,
},
},
Is this a standard OpenStack RPC call?
Not sure about that, but if it can be, it would be easier :)
Where ConfigDefault are the sane default proposed by the agent
authors.
Why is the agent proposing default settings?
So that the first agent of a given type can populate its info with sane
defaults that can then be edited later on?
If no config record is found the collector creates the record, sets
ConfVers to 1 and sends back a normal reply.
Reply sent by the collector:
'correlation_id': x
'version': '1.0',
Do we need minor versions for the config settings, or are those
simple sequence numbers to track which settings are the most current?
Simple sequence was what I was thinking about.
'args': {'data': {
'Result': result.code,
'ConfVers': ConfVers,
'Config': Config,
},
},
}
Result is set as follow:
200 - Config was retrieved successfully
201 - Config was created based on received default (Config
is empty)
304 - Config version is identical
Re: [Openstack] [Glance] Using HTTP PATCH
On 05/23/2012 09:39 PM, Joshua Harlow wrote: Is there anyway u can just make it a configuration option? v2imagemethod = ‘PUT’... This would make sense as rfc 5789 is only 2 years old, which means that there must be numerous firewalls and proxies which may not have been updated yet in production, if an update is available at all, that will just block the patch method... Most of the customers we are talking to are not that keen on updating what works. Nick On 5/23/12 1:22 PM, Brian Waldon brian.wal...@rackspace.com wrote: Hey guys, I'm considering using PATCH rather than PUT for image updates in the v2 Image API, but I wanted to make sure there weren't any major blockers that I might be missing. As far as I can tell, the python libraries we use in Glance support it (httplib2, requests). However, I discovered that Squid doesn't document official support of the method: http://wiki.squid-cache.org/SquidFaq/SquidLogs. I wanted to ask if there are any other key infrastructure pieces that foolishly hard-code request methods like this. It might make us want to rethink using PATCH at all. Brian ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] high-level design proposal
On 05/22/2012 07:15 PM, Doug Hellmann wrote: On Tue, May 22, 2012 at 1:25 PM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com wrote: On 05/22/2012 03:26 PM, Doug Hellmann wrote: - In addition to a signature, I think we would need a sequence number to be embedded by the agent for each message sent, so that loss of messages, or forgery of messages, can be detected by the collector and further audit process. OK. We have a message id, but I assumed those would be used to eliminate duplicates so this sounds like something different or new. It implies that the agent knows its own id (not hard) and keeps up with a sequence counter (more difficult, though not impossible). Did you have something in mind for how to implement that? Actually, this was my intent in the original blueprint when I specified the message_id field then a couple lines bellow: a process may verify that messages were not lost. On the implementation side, I was thinking that each agent would maintain its own sequence count, as a global instance count would be pricier. In my mind, non repudiation was built from the message_signature + message_id which should be unique for each agent. OK. That brings a couple of more specific questions to mind: Does the agent save its sequence counter through a restart? How and where? What about an upgrade? Seems easily stored locally. What would the down-stream consumer of the data do if it discovered there was a missing event? Who should do that detection work? Not sure we need to worry about auditing process yet, just make sure that we provide necessary the necessary information to do proper auditing. In principle, an audit process could then trigger an alert for further investigation of the issue. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] high-level design proposal
On 05/21/2012 10:52 PM, Doug Hellmann wrote: I have written up some of my thoughts on a proposed design for ceilometer in the wiki [1]. I'm sure there are missing details, but I wanted to start getting ideas into writing so they could be discussed here on the list, since I've talked about different parts with a couple of you separately. Let me know what you think, and especially if I am not clear or have left out any details. Thanks, Doug [1] http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1 Thanks a lot for putting this together Doug. A few questions: * The collector runs on one or more central management servers to monitor the message queues (for notifications and for metering data coming from the agent). Notification messages are processed and turned into metering messages and sent back out onto the message bus using the appropriate topic. Metering messages are written to the data store without modification. - Is the reason behind why collectors do not write directly to the database a way to allow db less implementations as Francis suggested earlier? In this case it may be useful to say it explicitly. * Plugins may require configuration options, so when the plugin is loaded it is asked to add options to the global flags object, and the results are made available to the plugin before it is asked to do any work. - I am not sure where the global flags object resides and how option are populated. I think it would make sense for this to be globally controlled, and therefore may require for a simple discovery exchange on the queue to retrieve values and set defaults if it does not exist yet. * Metering messages are signed using the hmac module in Python's standard library. A shared secret value can be provided in the ceilometer configuration settings. The messages are signed by feeding the message key names and values into the signature generator in sorted order. Non-string values are converted to unicode and then encoded as UTF-8. The message signature is included in the message for verification by the collector. - The signature is also kept in the database for future audit processes, maybe worth mentioning it here. - In addition to a signature, I think we would need a sequence number to be embedded by the agent for each message sent, so that loss of messages, or forgery of messages, can be detected by the collector and further audit process. Thanks again, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] high-level design proposal
On 05/22/2012 03:26 PM, Doug Hellmann wrote: - In addition to a signature, I think we would need a sequence number to be embedded by the agent for each message sent, so that loss of messages, or forgery of messages, can be detected by the collector and further audit process. OK. We have a message id, but I assumed those would be used to eliminate duplicates so this sounds like something different or new. It implies that the agent knows its own id (not hard) and keeps up with a sequence counter (more difficult, though not impossible). Did you have something in mind for how to implement that? Actually, this was my intent in the original blueprint when I specified the message_id field then a couple lines bellow: a process may verify that messages were not lost. On the implementation side, I was thinking that each agent would maintain its own sequence count, as a global instance count would be pricier. In my mind, non repudiation was built from the message_signature + message_id which should be unique for each agent. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Meeting agenda for Thursday at 16:00 UTC (May 17th, 2012)
On 05/16/2012 08:44 PM, Nick Barcet wrote: Hi, The metering project team holds a weekly meeting in #openstack-meeting, Thursdays at 1600 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. Since we were not able to conclude on the API discussion last week we continued our dicussions on the list and on the #openstack-metering channel and are now coming with a better proposal (or we are at least hoping it is better). http://wiki.openstack.org/Meetings/MeteringAgenda Topic: external API definition (part 2) * Agree on update to schema to include JSON formated metadata as described at [1] * Agree on API proposal described at [2] * Agree on format for date_time. Suggestion is to use ISO but seeking validation for best practice for REST * Agree on the use of a transparent cache for aggregation * Open discussion (if we have any time left) [1] http://wiki.openstack.org/EfficientMetering#Storage [2] http://wiki.openstack.org/EfficientMetering/APIProposalv1 The meeting took place yesterday, a summary follows. = #openstack-meeting Meeting == Meeting started by nijaba at 16:01:26 UTC. The full logs are available at http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-05-17-16.01.log.html Meeting summary --- * agenda http://wiki.openstack.org/Meetings/MeteringAgenda (nijaba, 16:01:26) * actions from previous meetings (nijaba, 16:02:14) * dachary add info to the wiki on the topic of poll versus push (nijaba, 16:02:26) * dhellmann: reformulate the API proposal as a start point for the dicussion on the ML (nijaba, 16:03:25) * Agree on update to schema to include JSON formated metadata (nijaba, 16:05:55) * LINK: http://wiki.openstack.org/EfficientMetering#Storage (nijaba, 16:05:55) * AGREED: to update to schema to include JSON formated metadata (nijaba, 16:10:40) * Agree on API proposal (nijaba, 16:10:56) * LINK: http://wiki.openstack.org/EfficientMetering/APIProposalv1 (nijaba, 16:10:56) * AGREED: on API proposal http://wiki.openstack.org/EfficientMetering/APIProposalv1 (nijaba, 16:15:01) * ACTION: flacoste to follow on the discussion about a bus only implementation (nijaba, 16:17:43) * Agree on format for date_time (nijaba, 16:15:22) * Suggestion is to use ISO but seeking validation for best practice for REST (nijaba, 16:15:22) * ACTION: nijaba to add the use to UTC for datetime (nijaba, 16:17:03) * AGREED: to use ISO for datetime (nijaba, 16:18:54) * Agree on the use of a transparent cache for aggregation (nijaba, 16:19:11) * AGREED: caching is an implementation detail (nijaba, 16:23:58) * Open discussion (nijaba, 16:24:28) * ACTION: dhellmann to document a counter for discrete event as an example (nijaba, 16:29:02) Meeting ended at 16:31:15 UTC. Action items, by person --- * dhellmann * dhellmann to document a counter for discrete event as an example * flacoste * flacoste to follow on the discussion about a bus only implementation * nijaba * nijaba to add the use to UTC for datetime Next week's meeting will cover the choice of a messaging queue for the project. Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [metering] Choice of a messaging queue
Hello everyone, Next week's irc meeting will have for goal to choose a reference messaging queue service for the ceilometer project. For this meeting to be able to be successful, a discussion on the choices that we have to make need to occur first right here. To open the discussion here are a few requirements that I would consider important for the queue to support: a) the queue must guaranty the delivery of messages. To the contrary of monitoring, loss of events may have important billing impacts, it therefore cannot be an option that message be lost. b) the client should be able to store and forward. As the load of system or traffic increases, or if the client is temporarily disconnected, client element of the queue should be able to hold messages in a local queue to be emitted as soon as condition permit. c) client must authenticate Only client which hold a shared private key should be able to send messages on the queue. d) queue may support client signing of individual messages Each message should be individually signed by the agent that emits it in order to guaranty non repudiability. This function can be done by the queue client or by the agent prior to en-queuing of messages. d) queue must be highly available the queue servers must be able to support multiple instances running in parallel in order to support continuation of operations with the loss of one server. This should be achievable without the need to use complex fail over systems and shared storage. e) queue should be horizontally scalable The scalability of queue servers should be achievable by increasing the number of servers. Not sure this list is exhaustive or viable, feel free to comment on it, but the real question is: which queue should we be using here? Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] example of discrete counter
On 05/17/2012 10:48 PM, Doug Hellmann wrote: I have added a row to the list of counters for discrete events such as uploading an image to glance [1]. Please let me know if you think I need more exposition to explain discrete counters. Doug [1] http://wiki.openstack.org/EfficientMetering?action=diffrev2=89rev1=87 http://wiki.openstack.org/EfficientMetering?action=diffrev2=89rev1=87 Thanks Doug, it looks good to me. Cheers, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Do we need an API and storage?
On 05/17/2012 11:13 AM, Loic Dachary wrote: On 05/16/2012 11:00 PM, Francis J. Lacoste wrote: I'm now of the opinion that we exclude storage and API from the metering project scope. Let's just focus on defining a metering message format, bus, and maybe a client-library to make it easy to write metering consumers. That way we avoid building a lot of things that we only _think will be useful_ for potential billing integration. Only writing/delivering such an integration component would prove that we built at least something that is useful. Hi, I'm a little reluctant to question the whole approach because I'm engaged in it :-) It's scary to contemplate the idea of throwing away some of the work done but I welcome the challenge. Better lose a few days work than keep going in the wrong direction. Are you proposing that such a library would then be integrated in whatever billing system someone has in mind ? For instance Dough https://github.com/lzyeval/dough trystack.org billing https://github.com/trystack/dash_billing nova-billing https://github.com/griddynamics/nova-billing Would depend on this library and rely on its API to collect what they need. The same would be done by proprietary billing systems ? Regarding the relevance of the metrics exposed by the API, I made sure they match the need of the eNovance sales. I'm quite sure Nicolas checked for Canonical and Doug did the same regarding the needs of Dreamhost. I'm confident that the information is actualy useful, at least in these practical cases. Getting rid of the storage imposes a constraint on the billing system : it must make 100% sure that once a message is consumed it will be reliably archived. It also adds a constraint on the chosen bus : it must be able to retain all messages for as long as a consumer needs, which may be days or weeks. Or it adds a constraint on the billing system which must make 100% sure it will consume all relevant messages the bus at all times before they expire. I have no strong feelings about exposing a bus for everyone to use instead of a REST API. I tend to prefer the REST API because it is an established standard for OpenStack. Could you expand on why a bus would be significantly better than a REST API in this specific case ? Thanks for challenging our thought process, Francis, it is a great sanity check :) I do have a few use cases where a REST API is better than a bus: * Private clouds: Users are unlikely to want to activate a complete billing system but still want to be able top provide to their users some metrics of their uses. The REST API plus some scripts would allow them to do this without too much pain. * Metric of usage on GUI: It might be usefull to provide a quick way to assess usage in almost real time to users as part of an extension of Horizon for example. The REST API would allow for such data to be extracted dynamically without having to run a full billing tool in real time. * In house billing tools: about half of the ISPs I surveyed are running some form of a customized ERP system to handle the billing for their customers. They need to be able to produce CSVs on a weekly basis to feed their custom billing solutions. Integrating a bus would be much more complex than the script that would issue request to the rest API. * Auditability/Non repudiability: if the messages go from the queue to some unknown DB, how do you solve, in a generic way, coherent audit check and ensure non-repudiability? I do not mean to totally discard Francis' bus idea though, and think that we should allow for (a not necessarily db-less integration but) direct queue attachment model for billing systems. I do think that for all the above reasons, plus the simplification of testing of the overall system, the REST API must remain part of this project. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] External API definition
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Daniel Dyer dan.dye...@gmail.com wrote: One per installation, at least, since the source field could allow to aggregate informations from multiple installations. Is it your assumption that there will be one metering service per installation or one per service (i.e swift, nova)? My assumption would be a single metering service, so the API would need to handle some additional use cases: -list services supported -list metrics for a service type -get metric details One per installation, at least, since the source field could allow to aggregate information from multiple installations. Can't See any reason why not to offer what you list above, even though one may deduce the component from the counter name. I would also consider separate use cases for accessing raw events vs. aggregated metrics. I think the extension proposal from Loic would cover that and more. Dan Dyer dan.d...@hp.com On Wed, May 9, 2012 at 10:44 AM, Nick Barcet nick.bar...@canonical.comwrote: Doug Hellmann doug.hellm...@dreamhost.com wrote: On Wed, May 9, 2012 at 11:27 AM, Nick Barcet nick.bar...@canonical.comwrote: On 05/08/2012 08:27 AM, Nick Barcet wrote: [..] Thinking about this, I think we need to expend the API a bit to reflect the evolutions of the schema that we decided last week. Here are my proposals: * Requests allow to GET account_id list change to: GET [user_id|project_id|source] list Does the [value|value] syntax mean choose one or combine? I assume choose one and you are using square brackets because parens are used in some of the other queries. You assumed correctly :) GET list of counter_type GET list of events per account optional start and end for counter_datetime optional counter_type change to: GET list of events per [user_id|project_id|source] optional start and end for counter_datetime optional counter_type Users may cross projects, so I'm not sure it makes sense to ask for the events generated by a user without restricting it by the project. At the very least we may need to allow them to specify user_id or project_id or both. Good point. Thanks for catching this. GET sum of (counter_volume, counter_duration) for counter_type and account_id optional start and end for counter_datetime GET sum of (counter_volume, counter_duration) for counter_type and [user_id|project_id|source] optional start and end for counter_datetime Hope this makes sense. Another item that we need to discuss is extensibility of this API. Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp - -- Nick Barcet nick.bar...@canonical.com aka: nicolas, nijaba -BEGIN PGP SIGNATURE- Version: APG v1.0.8 iGsEAREIACsFAk+r0yYkHE5pY29sYXMgQmFyY2V0IDxuaWNvbGFzQGJhcmNldC5j b20+AAoJEFiD3l2iIpt4+w0AmgIBEBQUXHAeOiTko3X5lYcGjqi4AKCQcUC9DyPe FBhL9NxeTMtAv1xsJg== =7Udb -END PGP SIGNATURE- ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Meeting agenda for today 16:00 UTC (May 3rd, 2012)
are recorded in the in the schema too (dachary, 16:33:27) * AGREED: s/counter/meter/ (dachary, 16:37:11) * LINK: http://wiki.openstack.org/EfficientMetering#Counters is agreed on, provided the actions listed above are carried out. ? (dachary, 16:37:38) * AGREED: http://wiki.openstack.org/EfficientMetering#Counters is agreed on, provided the actions listed above are carried out. ? (dachary, 16:38:52) * schema definition (dachary, 16:39:05) * Proposed http://wiki.openstack.org/EfficientMetering#Storage (dachary, 16:39:05) * ACTION: jd___ clarify / document the fact that the secondary field or the resource_id field carries the IP/VIF for the meters related to network so that they can be per IP (dachary, 16:40:42) * ACTION: nijaba describe the source field rationale and use case, initiate a thread to validate its use. (dachary, 16:50:26) * AGREED: the fields should be source (to be discussed), user_id, project_id, resource_id, counter_type, counter_volume, counter_duration, counter_datetime, secondary type / payload, message_signature, message_id ? (dachary, 16:53:20) * ACTION: jd___ update the documentation (dachary, 16:53:48) * ACTION: jd___ document the resource_id for each counter (dachary, 16:54:10) * discuss API assumptions (dachary, 16:54:51) * open discussion (dachary, 16:55:19) Meeting ended at 17:00:05 UTC. Action items, by person --- * dachary * dachary fix the note for net_float still talks about number of floating IPs * dachary add note about the fact that the resource_id for the object count is the container_id * jd___ * jd___ include Number of object in Swift, Number of containers in Swift, Number of GET/HEAD/PUT/POST requests in Swift in the table * jd___ document the resource_id for each counter * jd___ describes the general table schema and then something that says for each counter exactly what goes in the fields of that table and show how secondary field counters are recorded in the in the schema too * jd___ clarify / document the fact that the secondary field or the resource_id field carries the IP/VIF for the meters related to network so that they can be per IP * jd___ update the documentation * jd___ document the resource_id for each counter * nijaba * nijaba describe the source field rationale and use case, initiate a thread to validate its use. -- Loïc Dachary Chief Research Officer // eNovance labs http://labs.enovance.com // ✉ l...@enovance.com ☎ +33 1 49 70 99 82 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp - -- Nick Barcet nick.bar...@canonical.com aka: nicolas, nijaba -BEGIN PGP SIGNATURE- Version: APG v1.0.8 iGsEAREIACsFAk+r1T8kHE5pY29sYXMgQmFyY2V0IDxuaWNvbGFzQGJhcmNldC5j b20+AAoJEFiD3l2iIpt4QXUAoLJwCIl2vnyb9uTOFfJCH63E2qoNAJ0XkRyeBSty +nXCM+8IxnXGB3TAFg== =3gjy -END PGP SIGNATURE- ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] External API definition
On 05/08/2012 08:27 AM, Nick Barcet wrote: [..] Thinking about this, I think we need to expend the API a bit to reflect the evolutions of the schema that we decided last week. Here are my proposals: * Requests allow to GET account_id list change to: GET [user_id|project_id|source] list GET list of counter_type GET list of events per account optional start and end for counter_datetime optional counter_type change to: GET list of events per [user_id|project_id|source] optional start and end for counter_datetime optional counter_type GET sum of (counter_volume, counter_duration) for counter_type and account_id optional start and end for counter_datetime GET sum of (counter_volume, counter_duration) for counter_type and [user_id|project_id|source] optional start and end for counter_datetime Hope this makes sense. Another item that we need to discuss is extensibility of this API. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] External API definition
On 05/09/2012 08:36 AM, Doug Hellmann wrote: On Tue, May 8, 2012 at 8:43 PM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com wrote: On 05/08/2012 11:39 AM, Doug Hellmann wrote: [..] * Requests must be authenticated (separate from keystone, or only linked to accounting type account) What is the motivation for authenticating with a service other than keystone? The only thing I am trying to express here is that that profiles that have access to other OpenStack components should not necessarily have access to metering information. This information should be accessible only a few select users which group may or may not intersect with users stored in Keystone already. I see. Is it enough to say that the API is meant for admin users only, or does that still imply more access than we want to grant? I don't see the point to try to restrict admins from this, as it would be mostly pointless in the end, but I do see the need to define a type of account which only right is to consult this information without any other privilege. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] External API definition
Doug Hellmann doug.hellm...@dreamhost.com wrote: On Wed, May 9, 2012 at 11:27 AM, Nick Barcet nick.bar...@canonical.comwrote: On 05/08/2012 08:27 AM, Nick Barcet wrote: [..] Thinking about this, I think we need to expend the API a bit to reflect the evolutions of the schema that we decided last week. Here are my proposals: * Requests allow to GET account_id list change to: GET [user_id|project_id|source] list Does the [value|value] syntax mean choose one or combine? I assume choose one and you are using square brackets because parens are used in some of the other queries. You assumed correctly :) GET list of counter_type GET list of events per account optional start and end for counter_datetime optional counter_type change to: GET list of events per [user_id|project_id|source] optional start and end for counter_datetime optional counter_type Users may cross projects, so I'm not sure it makes sense to ask for the events generated by a user without restricting it by the project. At the very least we may need to allow them to specify user_id or project_id or both. Good point. Thanks for catching this. GET sum of (counter_volume, counter_duration) for counter_type and account_id optional start and end for counter_datetime GET sum of (counter_volume, counter_duration) for counter_type and [user_id|project_id|source] optional start and end for counter_datetime Hope this makes sense. Another item that we need to discuss is extensibility of this API. Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Metering] External API definition
Hello, The 3rd meeting of the Ceilometer project will be held on May 10th at 1600 UTC in the #openstack-meeting channel on Freenode. The main subject we will be tackling this week is the external API definition and I'd like to gather you opinions on the subject prior to the meeting. The current proposal which can be found on the blueprint [1] states the following: * Database can only be queried via a REST API (i.e. the database schema is not a supported API and can change in a non backward compatible way from one version to the other). * Requests must be authenticated (separate from keystone, or only linked to accounting type account) * API Server must be able to be redundant * Requests allow to GET account_id list GET list of counter_type GET list of events per account optional start and end for counter_datetime optional counter_type GET sum of (counter_volume, counter_duration) for counter_type and account_id optional start and end for counter_datetime Note: the aggregation of values is done by the API and is not stored in the database. It may be cached for performance reasons but the caching strategy is outside of the scope of this blueprint. Any comments, suggestion, etc... are very welcome. [1] http://wiki.openstack.org/EfficientMetering#API Thanks a lot, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] External API definition
On 05/08/2012 11:39 AM, Doug Hellmann wrote: [..] * Requests must be authenticated (separate from keystone, or only linked to accounting type account) What is the motivation for authenticating with a service other than keystone? The only thing I am trying to express here is that that profiles that have access to other OpenStack components should not necessarily have access to metering information. This information should be accessible only a few select users which group may or may not intersect with users stored in Keystone already. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Metering] Adding a source notion to the schema
Following up on the discussion on IRC yesterday during the metering meeting, I'd like to explain my proposal to add the notion of source to the schema of our event. The current goal we have for ceilometer is to provide a common way to accumulate counters/meters from various openstack component in a central repository that can be then used by other tools to produce bills in fine. One thing we can currently assume in Essex is that all components share a common repository for identity management: Keystone. This is the assumption we made when defining the existing schema. However, I think we need to plan a little further ahead here, and allow for this not to necessarily remain the same in some complex deployment cases. For example, it could be envisioned that some software, outside of the OpenStack project, maybe deployed and used on top of an OpenStack deployment. This software may need to be billed to customers, and collecting meters for it maybe as important for the provider than doing it for OpenStack components. It cannot be assumed that the identity management used by the software will be keystone. The software may not provide a metering interface built in, and it would make sense for the provider to be able to implement this using existing tool present in the underlying OpenStack deployment: ceilometer. In order to allow for this I think it would make sense to: * extend the current schema to allow for an additional source field in the event record. This should be a short identifier. * add another record definition that maps source to identity managment URL location * Collecting agent would be in charge to specify the source they map to (keystone by default). I think this would allow for easy extension of ceilometer to support other software, but I'd like to know if you share the usefulness of this extension from your experiences. Thanks, Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] schema and counter definitions
-Original Message- From: Thierry Carrez thie...@openstack.org To: openstack@lists.launchpad.net Sent: Fri, 04 May 2012 2:50 Subject: Re: [Openstack] [Metering] schema and counter definitions Robert Collins wrote: On Fri, May 4, 2012 at 5:27 AM, Turner, Whit (Cloud Services) whit.tur...@hp.com wrote: Hi - I think a flexible aggregation scheme is needed; the levels of aggregation available should be definable in the meter independent of the sources of usage data themselves. If invoices need to be very granular down to the lowest possible level, then this drives higher data requirements all through the processing chain, including the rating engine. Traditional systems tend to pass less granular (more highly aggregated) data into the rating engine so that bill runs and invoices can be generated efficiently. At cloud-scale, this can be problematic. Given some “big data” approaches, though, this could be handled in a more granular and real-time fashion. Has anyone looked at what statsd does? It has very similar requirements (simple to use, no hard a-priori definition of things to count, a few base types to track), and needs to be horizontally scalable. Also Swift has plans to use statsd for instrumentation/monitoring, so it's definitely worth a look to see if it could be used here as well. http://folsomdesignsummit2012.sched.org/event/d9135eabdd775432c74c3f1d32a325d3 http://etherpad.openstack.org/FolsomSwiftStatsd -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] schema and counter definitions
On 05/04/2012 02:50 AM, Thierry Carrez wrote: Robert Collins wrote: On Fri, May 4, 2012 at 5:27 AM, Turner, Whit (Cloud Services) whit.tur...@hp.com wrote: Hi - I think a flexible aggregation scheme is needed; the levels of aggregation available should be definable in the meter independent of the sources of usage data themselves. If invoices need to be very granular down to the lowest possible level, then this drives higher data requirements all through the processing chain, including the rating engine. Traditional systems tend to pass less granular (more highly aggregated) data into the rating engine so that bill runs and invoices can be generated efficiently. At cloud-scale, this can be problematic. Given some “big data” approaches, though, this could be handled in a more granular and real-time fashion. Has anyone looked at what statsd does? It has very similar requirements (simple to use, no hard a-priori definition of things to count, a few base types to track), and needs to be horizontally scalable. Also Swift has plans to use statsd for instrumentation/monitoring, so it's definitely worth a look to see if it could be used here as well. http://folsomdesignsummit2012.sched.org/event/d9135eabdd775432c74c3f1d32a325d3 http://etherpad.openstack.org/FolsomSwiftStatsd I am no Stastd expert, but a quick look at the project shows that it is aimed add data collection for the requirements of monitoring, and uses UDP as a way to aggregate vast quantity of data at short interval. The use of UDP implies that delivery is not guaranteed, which is fines for the objectives of monitoring, but is conflicting with the requirements of metering (as a sub component of a billing system). Stastd does not seem either to allow for message signature and authentication of collectors. Here are the requirements I think we have: * The data is sent from agents to the storage daemon via a trusted messaging system * The messages in queue are signed and non repudiable * The agents collecting data are authenticated to avoid pollution of the metering service Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Adding a source notion to the schema
On 05/04/2012 02:25 PM, Doug Hellmann wrote: On Fri, May 4, 2012 at 12:29 PM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com wrote: Following up on the discussion on IRC yesterday during the metering meeting, I'd like to explain my proposal to add the notion of source to the schema of our event. The current goal we have for ceilometer is to provide a common way to accumulate counters/meters from various openstack component in a central repository that can be then used by other tools to produce bills in fine. One thing we can currently assume in Essex is that all components share a common repository for identity management: Keystone. This is the assumption we made when defining the existing schema. However, I think we need to plan a little further ahead here, and allow for this not to necessarily remain the same in some complex deployment cases. For example, it could be envisioned that some software, outside of the OpenStack project, maybe deployed and used on top of an OpenStack deployment. This software may need to be billed to customers, and collecting meters for it maybe as important for the provider than doing it for OpenStack components. It cannot be assumed that the identity management used by the software will be keystone. The software may not provide a metering interface built in, and it would make sense for the provider to be able to implement this using existing tool present in the underlying OpenStack deployment: ceilometer. In order to allow for this I think it would make sense to: * extend the current schema to allow for an additional source field in the event record. This should be a short identifier. That makes sense. It seems like the identifier(s) used are meant to be defined by the deployer. Is that your intent? Correct. We'll just need a sane default for Keystone. * add another record definition that maps source to identity managment URL location * Collecting agent would be in charge to specify the source they map to (keystone by default). It is not clear why the identity management location needs to be recorded in ceilometer. If the deployer controls the source strings, they know what each value means. The only thing that needs to translate the (source, project, user) values to a billing identity is the end-consumer of all of this data, which is also under the control of the deployer. Couldn't the mapping of the source token to the identity location be handled in that layer? True. It might be over-engineering to try to keep the info in the same place as well, but the impact on the system would be negligible. I'd be happy either ways :) Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Adding a source notion to the schema
On 05/04/2012 03:11 PM, Doug Hellmann wrote: On Fri, May 4, 2012 at 6:03 PM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com wrote: On 05/04/2012 02:25 PM, Doug Hellmann wrote: On Fri, May 4, 2012 at 12:29 PM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com [...] In order to allow for this I think it would make sense to: * extend the current schema to allow for an additional source field in the event record. This should be a short identifier. That makes sense. It seems like the identifier(s) used are meant to be defined by the deployer. Is that your intent? Correct. We'll just need a sane default for Keystone. You're focusing on source as the origin for the identity information, but it seems like a layer sitting on top of OpenStack may well use Keystone for identity but generate different billable events. Should source be the origin of the metric data being sent to ceilometer? As I see it, source should remain the same as long as you can correlate project_id and user_id from the same source. * add another record definition that maps source to identity managment URL location * Collecting agent would be in charge to specify the source they map to (keystone by default). It is not clear why the identity management location needs to be recorded in ceilometer. If the deployer controls the source strings, they know what each value means. The only thing that needs to translate the (source, project, user) values to a billing identity is the end-consumer of all of this data, which is also under the control of the deployer. Couldn't the mapping of the source token to the identity location be handled in that layer? True. It might be over-engineering to try to keep the info in the same place as well, but the impact on the system would be negligible. I'd be happy either ways :) It will be easier to add it later if we really need it than to take it out if we decide we don't. Agreed. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] schema and counter definitions
On 05/01/2012 02:23 AM, Loic Dachary wrote: On 04/30/2012 11:39 PM, Doug Hellmann wrote: On Mon, Apr 30, 2012 at 3:43 PM, Loic Dachary l...@enovance.com mailto:l...@enovance.com wrote: On 04/30/2012 08:03 PM, Doug Hellmann wrote: On Mon, Apr 30, 2012 at 11:43 AM, Loic Dachary l...@enovance.com mailto:l...@enovance.com wrote: On 04/30/2012 03:49 PM, Doug Hellmann wrote: On Mon, Apr 30, 2012 at 6:46 AM, Loic Dachary l...@enovance.com mailto:l...@enovance.com wrote: On 04/30/2012 12:15 PM, Loic Dachary wrote: We could start a discussion from the content of the following sections: http://wiki.openstack.org/EfficientMetering#Counters I think the rationale of the counter aggregation needs to be explained. My understanding is that the metering system will be able to deliver the following information: 10 floating IPv4 addresses were allocated to the tenant during three months and were leased from provider NNN. From this, the billing system could add a line to the invoice : 10 IPv4, $N each = $10xN because it has been configured to invoice each IPv4 leased from provider NNN for $N. It is not the purpose of the metering system to display each IPv4 used, therefore it only exposes the aggregated information. The counters define how the information should be aggregated. If the idea was to expose each resource usage individually, defining counters would be meaningless as they would duplicate the activity log from each OpenStack component. What do you think ? At DreamHost we are going to want to show each individual resource (the IPv4 address, the instance, etc.) along with the charge information. Having the metering system aggregate that data will make it difficult/impossible to present the bill summary and detail views that we want. It would be much more useful for us if it tracked the usage details for each resource, and let us aggregate the data ourselves. If other vendors want to show the data differently, perhaps we should provide separate APIs for retrieving the detailed and aggregate data. Doug Hi, For the record, here is the unfinished conversation we had on IRC (04:29:06 PM) dhellmann: dachary, did you see my reply about counter definitions on the list today? (04:39:05 PM) dachary: It means some counters must not be aggregated. Only the amount associated with it is but there is one counter per IP. (04:55:01 PM) dachary: dhellmann: what about this :the id of the ressource controls the agregation of all counters : if it is missing, all resources of the same kind and their measures are aggregated. Otherwise only the measures are agreggated. http://wiki.openstack.org/EfficientMetering?action=diffrev2=40rev1=39 http://wiki.openstack.org/EfficientMetering?action=diffrev2=40rev1=39 (04:55:58 PM) dachary: it makes me a little unconfortable to define such an ad-hoc grouping (04:56:53 PM) dachary: i.e. you actuall control the aggregation by chosing which value to put in the id column (04:58:43 PM) dachary: s/actuall/actually/ (05:05:38 PM) ***dachary reading http://www.ogf.org/documents/GFD.98.pdf (05:05:54 PM) dachary: I feel like we're trying to resolve a non problem here (05:08:42 PM) dachary: values need to be aggregated. The raw input is a full description of the resource and a value ( gauge ). The question is how to control the aggregation in a reasonably flexible way. (05:11:34 PM) dachary: The definition of a counter could probably be described as : the id of a resource and code to fill each column associated with it. I tried to append the following, but the wiki kept failing. Propose that the counters are defined by a function instead of being fixed. That helps addressing the issue of aggregating the bandwidth associated to a given IP into a single counter. Alternate idea : * a counter is defined by * a name ( o1, n2, etc. ) that uniquely identifies the nature of the measure ( outbound internet transit, amount of RAM, etc. ) * the component in which it can be found ( nova, swift etc.) * and by columns, each one is set with the result of aggregate(find(record),record) where * find() looks for the existing column as found by selecting with the unique key ( maybe the
Re: [Openstack] [Metering] First meeting for the Metering project
On 04/25/2012 09:10 AM, Nick Barcet wrote: Thanks to the 16 persons that took the time to give their preference on Doodle [1]. The poll is now closed and the meeting time that came out of it is: Thursdays at 4pm UTC on #openstack-metering I was not sure if we could use #openstack-meeting as we are not yet an official project, lkm if you have an opinion on this. The proposed agenda can be found at [2]. Anyone is welcome to attend. The meeting occurred as planned, and overran... Nevertheless a lot of good decisions were made, the obviously most important one grin being the project name: ceilometer. For a more complete meeting summary, go visit: http://wiki.openstack.org/Meetings/MeteringAgenda/meeting-0-summary Next week we will have for objective to find an agreement on schema and counter definitions. If anyone does not do it before me I'll fire an introduction email on the subject soon (sometime this we), as we agreed to start the discussion via mailing list and use the irc meeting to finalize the choices. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Configuring Keystone in OpenStack (Essex) white-papers
On 04/27/2012 04:35 PM, Shake Chen wrote:
HI
Canonical provide the keystone document about how to config.
http://www.canonical.com/about-canonical/resources/white-papers/configuring-keystone-openstack-essex
but The document have one mistake.
when I run
NOVA_PUBLIC_URL=”http://$NOVA_IP:8774/v1.1/%(tenant_id)s”
root@node77:~# NOVA_PUBLIC_URL=”http://$NOVA_IP:8774/v1.1/%(tenant_id)s”
bash: syntax error near unexpected token `('
the error was cause by lack for (), I need add ( ) as below:
NOVA_PUBLIC_URL=”http://$NOVA_IP:8774/v1.1/%;(tenant_id)s”
Thanks for reporting this Shake. Putting Joshua (the author of the WP)
in cc to makes sure he gets it.
Nick
signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] First meeting for the Metering project
Thanks to the 16 persons that took the time to give their preference on Doodle [1]. The poll is now closed and the meeting time that came out of it is: Thursdays at 4pm UTC on #openstack-metering I was not sure if we could use #openstack-meeting as we are not yet an official project, lkm if you have an opinion on this. The proposed agenda can be found at [2]. Anyone is welcome to attend. [1] http://doodle.com/zi6b6vxxkiuxyuqk [2] http://wiki.openstack.org/Meetings/MeteringAgenda Thanks, Nick (aka nijaba on irc) signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Monitoring / Billing Architecture proposed
On 04/23/2012 10:45 PM, Doug Hellmann wrote: On Mon, Apr 23, 2012 at 4:14 PM, Brian Schott brian.sch...@nimbisservices.com mailto:brian.sch...@nimbisservices.com wrote: Doug, Do we mirror the table structure of nova, etc. and add created/modified columns? Or do we flatten into an instance event record with everything? I lean towards flattening the data as it is recorded and making a second pass during the bill calculation. You need to record instance modifications separately from the creation, especially if the modification changes the billing rate. So you might have records for: created instance, with UUID, name, size, timestamp, ownership information, etc. resized instance, with UUID, name, new size, timestamp, ownership information, etc. deleted instance, with UUID, name, size, timestamp, ownership information, etc. Maybe some of those values don't need to be reported in some cases, but if you record a complete picture of the state of the instance then the code that aggregates the event records to produce billing information can use it to make decisions about how to record the charges. There is also the case where an instance is still no longer running but nova thinks it is (or the reverse), so some sort of auditing sweep needs to be included (I think that's what Dough called the farmer but I don't have my notes in front of me). When I wrote [1], one of the things that I never assumed was how agents would collect their information. I imagined that the system should allow for multiple implementation of agents that would collect the same counters, assuming that 2 implementations for the same counter should never be running at once. That said, I am not sure an event based collection of what nova is notifying would satisfy the requirements I have heard from many cloud providers: - how do we ensure that event are not forged or lost in the current nova system? - how can I be sure that an instance has not simply crashed and never started? - how can I collect information which is not captured by nova events? Hence the proposal to use a dedicated event queue for billing, allowing for agents to collect and eventually validate data from different sources, including, but not necessarily limiting, collection from the nova events. Moreover, as soon as you generalize the problem to other components than just Nova (swift, glance, quantum, daas, ...) just using the nova event queue is not an option anymore. [1] http://wiki.openstack.org/EfficientMetering Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Monitoring / Billing Architecture proposed
Hello Luis, I presented a blueprint last week [1] which proposes to clearly differentiate metering from the overall billing process. It is my understanding that billing is too complex a beast to be solved for each requirement in a satisfactory way and have therefore proposed that we should first concentrate on collecting the informations necessary for billing systems to pull their information from. The blueprint, and the discussion we had at the summit last week, outlines a proposed architecture but has, so far, left open the component choices to implement it. It is our proposal to start a a weekly meeting from that week to start selecting the components to deliver this. You are more than welcome to join the project if you want. [1] http://wiki.openstack.org/EfficientMetering Best regards, Nick On 04/22/2012 08:50 PM, Luis Gervaso wrote: Hi, I want to share the architecture i am developing in order to perform the monitorig / billing OpenStack support: 1. AMQP Client which listen to RabbitMQ / QPid (this should be interchangeable) (Own Stuff or ServiceMix / Camel) 2. Events should be stored on a NoSQL document oriented database (I think mongodb is perfect, since we can query in a super easy fashion) 3a. The monitoring system can pull/push MongoDB 3b. The billing system can pull to create invoices 4. A mediation EIP should be necessary to integrate a billing/monitoring product. (ServiceMix / Camel) This is to receive your feedback. So please, critics are welcome! Cheers! -- --- Luis Alberto Gervaso Martin Woorea Solutions, S.L CEO CTO mobile: (+34) 627983344 luis@ mailto:luis.gerv...@gmail.comwoorea.es http://woorea.es/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Monitoring / Billing Architecture proposed
On 04/23/2012 11:17 AM, Ahn, Jaesuk wrote: Hi, Although metering and billing always comes together for the deployment, for the sake of clarity, I also think metering should be a separate project from the billing, especially in openstack. (As you mentioned it, billing is complex and has too many different requirements per provider) Anyway, I am really interested in weekly meeting you mentioned in the email. How can I join the meeting? Is it irc-meeting?, mailing-list?, or else? Of course you can join. It will be a weekly irc meeting and I am about to send a doodle with multiple time choices to everyone that mentioned being interested in joining this project at the summit, and have included your email now. The first meeting will have for objective to finalize a project name and validate a road-map of decisions that need to be made. Thanks a lot, Nick -- *Jaesuk Ahn*, Ph.D. Team Leader | Cloud OS Dev. Team Cloud Infrastructure Department KT (Korea Telecom) *T. +82-10-9888-0328 | F. +82-303-0993-5340* *Active member on **OpenStack Korea Community* http://www.openstack.or.kr/ Apr 23, 2012, 4:09 PM, Nick Barcet 작성: Hello Luis, I presented a blueprint last week [1] which proposes to clearly differentiate metering from the overall billing process. It is my understanding that billing is too complex a beast to be solved for each requirement in a satisfactory way and have therefore proposed that we should first concentrate on collecting the informations necessary for billing systems to pull their information from. The blueprint, and the discussion we had at the summit last week, outlines a proposed architecture but has, so far, left open the component choices to implement it. It is our proposal to start a a weekly meeting from that week to start selecting the components to deliver this. You are more than welcome to join the project if you want. [1] http://wiki.openstack.org/EfficientMetering Best regards, Nick On 04/22/2012 08:50 PM, Luis Gervaso wrote: Hi, I want to share the architecture i am developing in order to perform the monitorig / billing OpenStack support: 1. AMQP Client which listen to RabbitMQ / QPid (this should be interchangeable) (Own Stuff or ServiceMix / Camel) 2. Events should be stored on a NoSQL document oriented database (I think mongodb is perfect, since we can query in a super easy fashion) 3a. The monitoring system can pull/push MongoDB 3b. The billing system can pull to create invoices 4. A mediation EIP should be necessary to integrate a billing/monitoring product. (ServiceMix / Camel) This is to receive your feedback. So please, critics are welcome! Cheers! -- --- Luis Alberto Gervaso Martin Woorea Solutions, S.L CEO CTO mobile: (+34) 627983344 luis@ mailto:luis.gerv...@gmail.comwoorea.es http://woorea.es/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Monitoring / Billing Architecture proposed
On 04/23/2012 04:39 PM, Alexey Ababilov wrote: Hi! We have developed Nova Billing v2. Its documentation is currently available at http://aababilov.github.com/nova-billing-doc.github.com/. The documentation includes a glossary and architecture and API descriptions. Nova Billing v2 is a totally new solution. Its API and architecture were rewritten from scratch. The new Nova Billing introduces extensible modularized system with separate components. Nova Billing v2 can charge arbitrary resources according to custom billing schemas and, naturally, with different tariffs. Nova Billing v2 does not depend on any UI for OpenStack Cloud (neither OpenStack Dashboard nor any other solution). It does not require a particular OpenStack release. Provided components allow integration with diablo and essex and this list can be extended. Nova Billing v2 is event-driven and does not consumes system resources for periodical polling. Nova Billing v2 uses a RESTful protocol, so it is easy to integrate it with miscellaneous user clients and to add third-party components notifying about arbitrary events. Alessio Ababilov, Software Engineer Grid Dynamics Excellent! It sounds from the architecture that your work will easily be integrated into a the generic OpenStack metering solution that we are currently defining. The main goal of our design is to dissociate the metering from the billing part and to offer extensibility to all current and future components of OpenStack. Would you care to help us define this architecture? Thanks, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Canonical AWSOME
On 04/23/2012 02:39 PM, Thierry Carrez wrote: Philipp Wollermann wrote: What's the advantage of replacing the native EC2 compatibility layer with AWSOME from a user / operator point of view? One thing that was mentioned is that the proxy could be run on top of a public cloud that chose to only deploy OpenStack API support. Another reason why we started this project is that we believe this architecture would be easier to use and maintain than what currently exists in Nova. This is why we are proposing this work as a longer term solution for EC2 compatibility. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Monitoring / Billing Architecture proposed
On 04/23/2012 07:06 PM, Luis Gervaso wrote: Nick, I want contribute in the discussion group. Luis, I just sent your an invite to the doodle to pick the best time for this irc meeting. Once the date will have been finalized (I'll close the poll tomorrow EOD), I'll announce the date, time and place on this list as well as to everyone that entered the poll. Anyone else wanting to participate in the poll for the meeting time may follow this link: http://doodle.com/zi6b6vxxkiuxyuqk Thanks, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
