Re: [Openstack] Monitoring / Billing Architecture proposed

2012-04-23 Thread Nick Barcet
Hello Luis,

I presented a blueprint last week [1] which proposes to clearly
differentiate metering from the overall billing process.  It is my
understanding that billing is too complex a beast to be solved for each
requirement in a satisfactory way and have therefore proposed that we
should first concentrate on collecting the informations necessary for
billing systems to pull their information from.

The blueprint, and the discussion we had at the summit last week,
outlines a proposed architecture but has, so far, left open the
component choices to implement it.  It is our proposal to start a a
weekly meeting from that week to start selecting the components to
deliver this.  You are more than welcome to join the project if you want.

[1] http://wiki.openstack.org/EfficientMetering

Best regards,
Nick

On 04/22/2012 08:50 PM, Luis Gervaso wrote:
> Hi,
> 
> I want to share the architecture i am developing in order to perform the
> monitorig / billing OpenStack support:
> 
> 1. AMQP Client which listen to RabbitMQ / QPid (this should be
> interchangeable) (Own Stuff or ServiceMix / Camel)
> 
> 2. Events should be stored on a NoSQL document oriented database (I
> think mongodb is perfect, since we can query in a super easy fashion)
> 
> 3a. The monitoring system can pull/push MongoDB
> 
> 3b. The billing system can pull to create invoices 
> 
> 4. A mediation EIP should be necessary to integrate a billing/monitoring
> product. (ServiceMix / Camel)
> 
> This is to receive your feedback. So please, critics are welcome!
> 
> Cheers!
> 
> -- 
> ---
> Luis Alberto Gervaso Martin
> Woorea Solutions, S.L
> CEO & CTO
> mobile: (+34) 627983344
> luis@ woorea.es 
> 
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Monitoring / Billing Architecture proposed

2012-04-23 Thread Nick Barcet
On 04/23/2012 11:17 AM, Ahn, Jaesuk wrote:
> Hi, 
> 
> Although metering and billing always comes together for the deployment,
> for the sake of clarity, I also think metering should be a separate
> project from the billing, especially in openstack.  
> (As you mentioned it, billing is complex and has too many different
> requirements per provider)
> 
> Anyway, I am really interested in weekly meeting you mentioned in the
> email. 
> How can I join the meeting? Is it irc-meeting?, mailing-list?, or else?  

Of course you can join.  It will be a weekly irc meeting and I am about
to send a doodle with multiple time choices to everyone that mentioned
being interested in joining this project at the summit, and have
included your email now.   The first meeting will have for objective to
finalize a project name and validate a road-map of decisions that need
to be made.

Thanks a lot,

Nick

> -- 
> *Jaesuk Ahn*, Ph.D.
> Team Leader | Cloud OS Dev. Team
> Cloud Infrastructure Department
> KT (Korea Telecom)
> *T. +82-10-9888-0328 | F. +82-303-0993-5340*
> *Active member on **OpenStack Korea Community* <http://www.openstack.or.kr/>
> 
> 
> Apr 23, 2012, 4:09 PM, Nick Barcet 작성:
> 
>> Hello Luis,
>>
>> I presented a blueprint last week [1] which proposes to clearly
>> differentiate metering from the overall billing process.  It is my
>> understanding that billing is too complex a beast to be solved for each
>> requirement in a satisfactory way and have therefore proposed that we
>> should first concentrate on collecting the informations necessary for
>> billing systems to pull their information from.
>>
>> The blueprint, and the discussion we had at the summit last week,
>> outlines a proposed architecture but has, so far, left open the
>> component choices to implement it.  It is our proposal to start a a
>> weekly meeting from that week to start selecting the components to
>> deliver this.  You are more than welcome to join the project if you want.
>>
>> [1] http://wiki.openstack.org/EfficientMetering
>>
>> Best regards,
>> Nick
>>
>> On 04/22/2012 08:50 PM, Luis Gervaso wrote:
>>> Hi,
>>>
>>> I want to share the architecture i am developing in order to perform the
>>> monitorig / billing OpenStack support:
>>>
>>> 1. AMQP Client which listen to RabbitMQ / QPid (this should be
>>> interchangeable) (Own Stuff or ServiceMix / Camel)
>>>
>>> 2. Events should be stored on a NoSQL document oriented database (I
>>> think mongodb is perfect, since we can query in a super easy fashion)
>>>
>>> 3a. The monitoring system can pull/push MongoDB
>>>
>>> 3b. The billing system can pull to create invoices
>>>
>>> 4. A mediation EIP should be necessary to integrate a billing/monitoring
>>> product. (ServiceMix / Camel)
>>>
>>> This is to receive your feedback. So please, critics are welcome!
>>>
>>> Cheers!
>>>
>>> -- 
>>> ---
>>> Luis Alberto Gervaso Martin
>>> Woorea Solutions, S.L
>>> CEO & CTO
>>> mobile: (+34) 627983344
>>> luis@ <mailto:luis.gerv...@gmail.com>woorea.es <http://woorea.es/>
>>>
>>>
>>>
>>>
>>> ___
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to : openstack@lists.launchpad.net
>>> <mailto:openstack@lists.launchpad.net>
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> <mailto:openstack@lists.launchpad.net>
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
> 
> 
> 




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Monitoring / Billing Architecture proposed

2012-04-23 Thread Nick Barcet
On 04/23/2012 04:39 PM, Alexey Ababilov wrote:
> Hi!
> 
> We have developed Nova Billing v2. Its documentation is currently
> available at http://aababilov.github.com/nova-billing-doc.github.com/.
> The documentation includes a glossary and architecture and API descriptions.
> 
> Nova Billing v2 is a totally new solution. Its API and architecture were
> rewritten from scratch. The new Nova Billing introduces extensible
> modularized system with separate components.
> 
> Nova Billing v2 can charge arbitrary resources according to custom
> billing schemas and, naturally, with different tariffs.
> 
> Nova Billing v2 does not depend on any UI for OpenStack Cloud (neither
> OpenStack Dashboard nor any other solution). It does not require a
> particular OpenStack release. Provided components allow integration with
> diablo and essex and this list can be extended.
> 
> Nova Billing v2 is event-driven and does not consumes system resources
> for periodical polling.
> 
> Nova Billing v2 uses a RESTful protocol, so it is easy to integrate it
> with miscellaneous user clients and to add third-party components
> notifying about arbitrary events.
> 
> Alessio Ababilov,
> Software Engineer
> Grid Dynamics

Excellent! It sounds from the architecture that your work will easily be
integrated into a the generic OpenStack metering solution that we are
currently defining.  The main goal of our design is to dissociate the
metering from the billing part and to offer extensibility to all current
and future components of OpenStack.  Would you care to help us define
this architecture?

Thanks,
Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Canonical AWSOME

2012-04-23 Thread Nick Barcet
On 04/23/2012 02:39 PM, Thierry Carrez wrote:
> Philipp Wollermann wrote:
>> What's the advantage of replacing the native EC2 compatibility layer with 
>> AWSOME from a user / operator point of view?
> 
> One thing that was mentioned is that the proxy could be run on top of a
> public cloud that chose to only deploy OpenStack API support.

Another reason why we started this project is that we believe this
architecture would be easier to use and maintain than what currently
exists in Nova. This is why we are proposing this work as a longer term
solution for EC2 compatibility.

Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Monitoring / Billing Architecture proposed

2012-04-23 Thread Nick Barcet
On 04/23/2012 07:06 PM, Luis Gervaso wrote:
> Nick, I want contribute in the discussion group.

Luis,
I just sent your an invite to the doodle to pick the best time for this
irc meeting.  Once the date will have been finalized (I'll close the
poll tomorrow EOD), I'll announce the date, time and place on this list
as well as to everyone that entered the poll.

Anyone else wanting to participate in the poll for the meeting time may
follow this link:
http://doodle.com/zi6b6vxxkiuxyuqk

Thanks,
Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Monitoring / Billing Architecture proposed

2012-04-24 Thread Nick Barcet
On 04/23/2012 10:45 PM, Doug Hellmann wrote:
> 
> 
> On Mon, Apr 23, 2012 at 4:14 PM, Brian Schott
>  > wrote:
> 
> Doug,
> 
> Do we mirror the table structure of nova, etc. and add
> created/modified columns? 
> 
> 
> Or do we flatten into an instance event record with everything?  
> 
> 
> I lean towards flattening the data as it is recorded and making a second
> pass during the bill calculation. You need to record instance
> modifications separately from the creation, especially if the
> modification changes the billing rate. So you might have records for:
> 
> created instance, with UUID, name, size, timestamp, ownership
> information, etc.
> resized instance, with UUID, name, new size, timestamp, ownership
> information, etc.
> deleted instance, with UUID, name, size, timestamp, ownership
> information, etc.
> 
> Maybe some of those values don't need to be reported in some cases, but
> if you record a complete picture of the state of the instance then the
> code that aggregates the event records to produce billing information
> can use it to make decisions about how to record the charges.
> 
> There is also the case where an instance is still no longer running but
> nova thinks it is (or the reverse), so some sort of auditing sweep needs
> to be included (I think that's what Dough called the "farmer" but I
> don't have my notes in front of me).

When I wrote [1], one of the things that I never assumed was how agents
would collect their information. I imagined that the system should allow
for multiple implementation of agents that would collect the same
counters, assuming that 2 implementations for the same counter should
never be running at once.

That said, I am not sure an event based collection of what nova is
notifying would satisfy the requirements I have heard from many cloud
providers:
- how do we ensure that event are not forged or lost in the current nova
system?
- how can I be sure that an instance has not simply crashed and never
started?
- how can I collect information which is not captured by nova events?

Hence the proposal to use a dedicated event queue for billing, allowing
for agents to collect and eventually validate data from different
sources, including, but not necessarily limiting, collection from the
nova events.

Moreover, as soon as you generalize the problem to other components than
just Nova (swift, glance, quantum, daas, ...) just using the nova event
queue is not an option anymore.

[1] http://wiki.openstack.org/EfficientMetering

Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] First meeting for the Metering project

2012-04-25 Thread Nick Barcet
Thanks to the 16 persons that took the time to give their preference on
Doodle [1].  The poll is now closed and the meeting time that came out
of it is:

  Thursdays at 4pm UTC on #openstack-metering

I was not sure if we could use #openstack-meeting as we are not yet an
official project, lkm if you have an opinion on this.

The proposed agenda can be found at [2].

Anyone is welcome to attend.

[1] http://doodle.com/zi6b6vxxkiuxyuqk
[2] http://wiki.openstack.org/Meetings/MeteringAgenda

Thanks,
Nick (aka nijaba on irc)



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] First meeting for the Metering project

2012-04-27 Thread Nick Barcet
On 04/25/2012 09:10 AM, Nick Barcet wrote:
> Thanks to the 16 persons that took the time to give their preference on
> Doodle [1].  The poll is now closed and the meeting time that came out
> of it is:
> 
>   Thursdays at 4pm UTC on #openstack-metering
> 
> I was not sure if we could use #openstack-meeting as we are not yet an
> official project, lkm if you have an opinion on this.
> 
> The proposed agenda can be found at [2].
> 
> Anyone is welcome to attend.

The meeting occurred as planned, and overran...  Nevertheless a lot of
good decisions were made, the obviously most important one  being
the project name: ceilometer.

For a more complete meeting summary, go visit:
http://wiki.openstack.org/Meetings/MeteringAgenda/meeting-0-summary

Next week we will have for objective to find an agreement on "schema and
counter definitions". If anyone does not do it before me  I'll fire an
introduction email on the subject soon (sometime this we), as we agreed
to start the discussion via mailing list and use the irc meeting to
finalize the choices.

Nick





signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Configuring Keystone in OpenStack (Essex) white-papers

2012-04-27 Thread Nick Barcet
On 04/27/2012 04:35 PM, Shake Chen wrote:
> HI
> 
>  Canonical provide the keystone document about how to config.
> 
> http://www.canonical.com/about-canonical/resources/white-papers/configuring-keystone-openstack-essex
> 
> 
> 
> but The document have one mistake.
> 
> when I run
> NOVA_PUBLIC_URL=”http://$NOVA_IP:8774/v1.1/%(tenant_id)s”
> 
> root@node77:~# NOVA_PUBLIC_URL=”http://$NOVA_IP:8774/v1.1/%(tenant_id)s”
> bash: syntax error near unexpected token `('
> 
> the error was cause by lack " for (), I need add "(  )" as below:
> 
> 
> NOVA_PUBLIC_URL=”http://$NOVA_IP:8774/v1.1/%";(tenant_id)"s”

Thanks for reporting this Shake.  Putting Joshua (the author of the WP)
in cc to makes sure he gets it.

Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] schema and counter definitions

2012-05-01 Thread Nick Barcet
On 05/01/2012 02:23 AM, Loic Dachary wrote:
> On 04/30/2012 11:39 PM, Doug Hellmann wrote:
>>
>>
>> On Mon, Apr 30, 2012 at 3:43 PM, Loic Dachary > > wrote:
>>
>> On 04/30/2012 08:03 PM, Doug Hellmann wrote:
>>>
>>>
>>> On Mon, Apr 30, 2012 at 11:43 AM, Loic Dachary >> > wrote:
>>>
>>> On 04/30/2012 03:49 PM, Doug Hellmann wrote:


 On Mon, Apr 30, 2012 at 6:46 AM, Loic Dachary
 mailto:l...@enovance.com>> wrote:

 On 04/30/2012 12:15 PM, Loic Dachary wrote:
 > We could start a discussion from the content of the
 following sections:
 >
 > http://wiki.openstack.org/EfficientMetering#Counters
 I think the rationale of the counter aggregation needs
 to be explained. My understanding is that the metering
 system will be able to deliver the following
 information: 10 floating IPv4 addresses were allocated
 to the tenant during three months and were leased from
 provider NNN. From this, the billing system could add a
 line to the invoice : 10 IPv4, $N each = $10xN because
 it has been configured to invoice each IPv4 leased from
 provider NNN for $N.

 It is not the purpose of the metering system to display
 each IPv4 used, therefore it only exposes the aggregated
 information. The counters define how the information
 should be aggregated. If the idea was to expose each
 resource usage individually, defining counters would be
 meaningless as they would duplicate the activity log
 from each OpenStack component.

 What do you think ?


 At DreamHost we are going to want to show each individual
 resource (the IPv4 address, the instance, etc.) along with
 the charge information. Having the metering system aggregate
 that data will make it difficult/impossible to present the
 bill summary and detail views that we want. It would be much
 more useful for us if it tracked the usage details for each
 resource, and let us aggregate the data ourselves.

 If other vendors want to show the data differently, perhaps
 we should provide separate APIs for retrieving the detailed
 and aggregate data.

 Doug

>>> Hi,
>>>
>>> For the record, here is the unfinished conversation we had on IRC
>>>
>>> (04:29:06 PM) dhellmann: dachary, did you see my reply about
>>> counter definitions on the list today?
>>> (04:39:05 PM) dachary: It means some counters must not be
>>> aggregated. Only the amount associated with it is but there
>>> is one counter per IP.
>>> (04:55:01 PM) dachary: dhellmann: what about this :the id of
>>> the ressource controls the agregation of all counters : if it
>>> is missing, all resources of the same kind and their measures
>>> are aggregated. Otherwise only the measures are agreggated.
>>> 
>>> http://wiki.openstack.org/EfficientMetering?action=diff&rev2=40&rev1=39
>>> 
>>> 
>>> (04:55:58 PM) dachary: it makes me a little unconfortable to
>>> define such an "ad-hoc" grouping
>>> (04:56:53 PM) dachary: i.e. you actuall control the
>>> aggregation by chosing which value to put in the id column
>>> (04:58:43 PM) dachary: s/actuall/actually/
>>> (05:05:38 PM) ***dachary reading
>>> http://www.ogf.org/documents/GFD.98.pdf
>>> (05:05:54 PM) dachary: I feel like we're trying to resolve a
>>> non problem here
>>> (05:08:42 PM) dachary: values need to be aggregated. The raw
>>> input is a full description of the resource and a value (
>>> gauge ). The question is how to control the aggregation in a
>>> reasonably flexible way.
>>> (05:11:34 PM) dachary: The definition of a counter could
>>> probably be described as : the id of a resource and code to
>>> fill each column associated with it.
>>>
>>> I tried to append the following, but the wiki kept failing.
>>>
>>> Propose that the counters are defined by a function instead
>>> of being fixed. That helps addressing the issue of
>>> aggregating the bandwidth associated to a given IP into a
>>> single counter.
>>>
>>> Alternate idea :
>>>  * a counter is defined by
>>>   * a name ( o1, n2, etc. ) that uniquely identifies the
>>> nature of the measure ( outbound internet transit, amount of
>>> 

[Openstack] [Metering] Adding a source notion to the schema

2012-05-04 Thread Nick Barcet
Following up on the discussion on IRC yesterday during the metering meeting, 
I'd like to explain my proposal to add the notion of source to the schema of 
our event.  The current goal we have for ceilometer is to provide a common way 
to accumulate counters/meters from various openstack component in a central 
repository that can be then used by other tools to produce bills in fine.   One 
thing we can currently assume in Essex is that all components share a common 
repository for identity management: Keystone.  This is the assumption we made 
when defining the existing schema. However, I think we need to plan a little 
further ahead here, and allow for this not to necessarily remain the same in 
some complex deployment cases.

For example, it could be envisioned that some software, outside of the 
OpenStack project, maybe deployed and used on top of an OpenStack deployment.  
This software may need to be billed to customers, and collecting meters for it 
maybe as important for the provider than doing it for OpenStack components. It 
cannot be assumed that the identity management used by  the software will be 
keystone. The software may not provide a metering interface built in, and it 
would make sense for the provider to be able to implement this using existing 
tool present in the underlying OpenStack deployment: ceilometer.  

In order to allow for this I think it would make sense to:
* extend the current schema to allow for an additional "source" field in the 
event record.  This should be a short identifier.
* add another record definition that maps source to identity managment URL 
location
* Collecting agent would be in charge to specify the source they map to 
(keystone by default).  

I think this would allow for easy extension of ceilometer to support other 
software, but I'd like to know if you share the usefulness of this extension 
from your experiences.

Thanks,

Nick 
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] schema and counter definitions

2012-05-04 Thread nick . barcet
-Original Message-
From: Thierry Carrez 
To: openstack@lists.launchpad.net
Sent: Fri, 04 May 2012 2:50
Subject: Re: [Openstack] [Metering] schema and counter definitions

Robert Collins wrote:
> On Fri, May 4, 2012 at 5:27 AM, Turner, Whit (Cloud Services)
>  wrote:
>> Hi - I think a flexible aggregation scheme is needed; the levels of
>> aggregation available should be definable in the meter independent of the
>> sources of usage data themselves. If invoices need to be very granular down
>> to the lowest possible level, then this drives higher data requirements all
>> through the processing chain, including the rating engine. Traditional
>> systems tend to pass less granular (more highly aggregated) data into the
>> rating engine so that bill runs and invoices can be generated efficiently.
>> At cloud-scale, this can be problematic. Given some “big data” approaches,
>> though, this could be handled in a more granular and real-time fashion.
> 
> Has anyone looked at what statsd does? It has very similar
> requirements (simple to use, no hard a-priori definition of things to
> count, a few base types to track), and needs to be horizontally
> scalable.

Also Swift has plans to use statsd for instrumentation/monitoring, so
it's definitely worth a look to see if it could be used here as well.

http://folsomdesignsummit2012.sched.org/event/d9135eabdd775432c74c3f1d32a325d3
http://etherpad.openstack.org/FolsomSwiftStatsd

-- 
Thierry Carrez (ttx)
Release Manager, OpenStack

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] schema and counter definitions

2012-05-04 Thread Nick Barcet
On 05/04/2012 02:50 AM, Thierry Carrez wrote:
> Robert Collins wrote:
>> On Fri, May 4, 2012 at 5:27 AM, Turner, Whit (Cloud Services)
>>  wrote:
>>> Hi - I think a flexible aggregation scheme is needed; the levels of
>>> aggregation available should be definable in the meter independent of the
>>> sources of usage data themselves. If invoices need to be very granular down
>>> to the lowest possible level, then this drives higher data requirements all
>>> through the processing chain, including the rating engine. Traditional
>>> systems tend to pass less granular (more highly aggregated) data into the
>>> rating engine so that bill runs and invoices can be generated efficiently.
>>> At cloud-scale, this can be problematic. Given some “big data” approaches,
>>> though, this could be handled in a more granular and real-time fashion.
>>
>> Has anyone looked at what statsd does? It has very similar
>> requirements (simple to use, no hard a-priori definition of things to
>> count, a few base types to track), and needs to be horizontally
>> scalable.
> 
> Also Swift has plans to use statsd for instrumentation/monitoring, so
> it's definitely worth a look to see if it could be used here as well.
> 
> http://folsomdesignsummit2012.sched.org/event/d9135eabdd775432c74c3f1d32a325d3
> http://etherpad.openstack.org/FolsomSwiftStatsd

I am no Stastd expert, but a quick look at the project shows that it is
aimed add data collection for the requirements of monitoring, and uses
UDP as a way to aggregate vast quantity of data at short interval.  The
use of UDP implies that delivery is not guaranteed, which is fines for
the objectives of monitoring, but is conflicting with the requirements
of metering (as a sub component of a billing system).

Stastd does not seem either to allow for message signature and
authentication of collectors.

Here are the requirements I think we have:
 * The data is sent from agents to the storage daemon via a trusted
messaging system
 * The messages in queue are signed and non repudiable
 * The agents collecting data are authenticated to avoid pollution of
the metering service

Nick





signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] Adding a source notion to the schema

2012-05-04 Thread Nick Barcet
On 05/04/2012 02:25 PM, Doug Hellmann wrote:
> 
> 
> On Fri, May 4, 2012 at 12:29 PM, Nick Barcet  <mailto:nick.bar...@canonical.com>> wrote:
> 
> Following up on the discussion on IRC yesterday during the metering
> meeting, I'd like to explain my proposal to add the notion of source
> to the schema of our event.  The current goal we have for ceilometer
> is to provide a common way to accumulate counters/meters from
> various openstack component in a central repository that can be then
> used by other tools to produce bills in fine.   One thing we can
> currently assume in Essex is that all components share a common
> repository for identity management: Keystone.  This is the
> assumption we made when defining the existing schema. However, I
> think we need to plan a little further ahead here, and allow for
> this not to necessarily remain the same in some complex deployment
> cases.
> 
> For example, it could be envisioned that some software, outside of
> the OpenStack project, maybe deployed and used on top of an
> OpenStack deployment.  This software may need to be billed to
> customers, and collecting meters for it maybe as important for the
> provider than doing it for OpenStack components. It cannot be
> assumed that the identity management used by  the software will be
> keystone. The software may not provide a metering interface built
> in, and it would make sense for the provider to be able to implement
> this using existing tool present in the underlying OpenStack
> deployment: ceilometer. 
> 
> In order to allow for this I think it would make sense to:
> * extend the current schema to allow for an additional "source"
> field in the event record.  This should be a short identifier.
> 
> 
> That makes sense. It seems like the identifier(s) used are meant to be
> defined by the deployer. Is that your intent?

Correct.  We'll just need a sane default for Keystone.

> 
> * add another record definition that maps source to identity
> managment URL location
> * Collecting agent would be in charge to specify the source they map
> to (keystone by default). 
> 
> 
> It is not clear why the identity management location needs to be
> recorded in ceilometer. If the deployer controls the source strings,
> they know what each value means. The only thing that needs to translate
> the (source, project, user) values to a billing identity is the
> end-consumer of all of this data, which is also under the control of the
> deployer. Couldn't the mapping of the source token to the identity
> location be handled in that layer?

True.  It might be over-engineering to try to keep the info in the same
place as well, but the impact on the system would be negligible. I'd be
happy either ways :)

Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] Adding a source notion to the schema

2012-05-04 Thread Nick Barcet
On 05/04/2012 03:11 PM, Doug Hellmann wrote:
> 
> 
> On Fri, May 4, 2012 at 6:03 PM, Nick Barcet  <mailto:nick.bar...@canonical.com>> wrote:
> 
> On 05/04/2012 02:25 PM, Doug Hellmann wrote:
> >
> >
> > On Fri, May 4, 2012 at 12:29 PM, Nick Barcet
> mailto:nick.bar...@canonical.com>
[...]
> > In order to allow for this I think it would make sense to:
> > * extend the current schema to allow for an additional "source"
> > field in the event record.  This should be a short identifier.
> >
> >
> > That makes sense. It seems like the identifier(s) used are meant to be
> > defined by the deployer. Is that your intent?
> 
> Correct.  We'll just need a sane default for Keystone.
> 
> 
> You're focusing on source as the origin for the identity information,
> but it seems like a layer sitting on top of OpenStack may well use
> Keystone for identity but generate different billable events. Should
> "source" be the origin of the metric data being sent to ceilometer?

As I see it, source should remain the same as long as you can correlate
project_id and user_id from the same source.

> >
> > * add another record definition that maps source to identity
> > managment URL location
> > * Collecting agent would be in charge to specify the source
> they map
> > to (keystone by default).
> >
> >
> > It is not clear why the identity management location needs to be
> > recorded in ceilometer. If the deployer controls the source strings,
> > they know what each value means. The only thing that needs to
> translate
> > the (source, project, user) values to a billing identity is the
> > end-consumer of all of this data, which is also under the control
> of the
> > deployer. Couldn't the mapping of the source token to the identity
> > location be handled in that layer?
> 
> True.  It might be over-engineering to try to keep the info in the same
> place as well, but the impact on the system would be negligible. I'd be
> happy either ways :)
> 
> 
> It will be easier to add it later if we really need it than to take it
> out if we decide we don't.

Agreed.

Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [Metering] External API definition

2012-05-08 Thread Nick Barcet
Hello,

The 3rd meeting of the Ceilometer project will be held on May 10th at
1600 UTC in the #openstack-meeting channel on Freenode. The main subject
we will be tackling this week is the external API definition and I'd
like to gather you opinions on the subject prior to the meeting.

The current proposal which can be found on the blueprint [1] states the
following:

* Database can only be queried via a REST API (i.e. the database schema
is not a supported API and can change in a non backward compatible way
from one version to the other).
* Requests must be authenticated (separate from keystone, or only linked
to accounting type account)
* API Server must be able to be redundant
* Requests allow to
  GET account_id list
  GET list of counter_type
  GET list of events per account
optional start and end for counter_datetime
optional counter_type
  GET sum of (counter_volume, counter_duration) for counter_type and
account_id
optional start and end for counter_datetime

Note: the aggregation of values is done by the API and is not stored in
the database. It may be cached for performance reasons but the caching
strategy is outside of the scope of this blueprint.

Any comments, suggestion, etc... are very welcome.

[1] http://wiki.openstack.org/EfficientMetering#API

Thanks a lot,
Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] External API definition

2012-05-08 Thread Nick Barcet
On 05/08/2012 11:39 AM, Doug Hellmann wrote:
[..]
> * Requests must be authenticated (separate from keystone, or only linked
> to accounting type account)
> 
> 
> What is the motivation for authenticating with a service other than
> keystone?

The only thing I am trying to express here is that that profiles that
have access to other OpenStack components should not necessarily have
access to metering information.  This information should be accessible
only a few select users which group may or may not intersect with users
stored in Keystone already.

Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] External API definition

2012-05-09 Thread Nick Barcet
On 05/08/2012 08:27 AM, Nick Barcet wrote:
[..]

Thinking about this, I think we need to expend the API a bit to reflect
the evolutions of the schema that we decided last week.  Here are my
proposals:

> * Requests allow to
>   GET account_id list

change to: GET [user_id|project_id|source] list

>   GET list of counter_type
>   GET list of events per account
> optional start and end for counter_datetime
> optional counter_type

change to: GET list of events per [user_id|project_id|source]
optional start and end for counter_datetime
optional counter_type

>   GET sum of (counter_volume, counter_duration) for counter_type and
> account_id
> optional start and end for counter_datetime

   GET sum of (counter_volume, counter_duration) for counter_type and
[user_id|project_id|source]
 optional start and end for counter_datetime

Hope this makes sense.

Another item that we need to discuss is extensibility of this API.

Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] External API definition

2012-05-09 Thread Nick Barcet
On 05/09/2012 08:36 AM, Doug Hellmann wrote:
> 
> 
> On Tue, May 8, 2012 at 8:43 PM, Nick Barcet  <mailto:nick.bar...@canonical.com>> wrote:
> 
> On 05/08/2012 11:39 AM, Doug Hellmann wrote:
> [..]
> > * Requests must be authenticated (separate from keystone, or
> only linked
> > to accounting type account)
> >
> >
> > What is the motivation for authenticating with a service other than
> > keystone?
> 
> The only thing I am trying to express here is that that profiles that
> have access to other OpenStack components should not necessarily have
> access to metering information.  This information should be accessible
> only a few select users which group may or may not intersect with users
> stored in Keystone already.
> 
> 
> I see. Is it enough to say that the API is meant for "admin" users only,
> or does that still imply more access than we want to grant?

I don't see the point to try to restrict admins from this, as it would
be mostly pointless in the end, but I do see the need to define a type
of account which only right is to consult this information without any
other privilege.

Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] External API definition

2012-05-09 Thread Nick Barcet


Doug Hellmann  wrote:

>On Wed, May 9, 2012 at 11:27 AM, Nick Barcet
>wrote:
>
>> On 05/08/2012 08:27 AM, Nick Barcet wrote:
>> [..]
>>
>> Thinking about this, I think we need to expend the API a bit to
>reflect
>> the evolutions of the schema that we decided last week.  Here are my
>> proposals:
>>
>> > * Requests allow to
>> >   GET account_id list
>>
>> change to: GET [user_id|project_id|source] list
>>
>
>Does the [value|value] syntax mean "choose one" or "combine"? I assume
>"choose one" and you are using square brackets because parens are used
>in some of the other queries.

You assumed correctly :)

>>
>> >   GET list of counter_type
>> >   GET list of events per account
>> > optional start and end for counter_datetime
>> > optional counter_type
>>
>> change to: GET list of events per [user_id|project_id|source]
>> optional start and end for counter_datetime
>>optional counter_type
>>
>
>Users may cross projects, so I'm not sure it makes sense to ask for the
>events generated by a user without restricting it by the project. At
>the very least we may need to allow them to specify user_id or project_id
>or both.

Good point. Thanks for catching this.

>>
>> >   GET sum of (counter_volume, counter_duration) for counter_type
>and
>> > account_id
>> > optional start and end for counter_datetime
>>
>>   GET sum of (counter_volume, counter_duration) for counter_type and
>> [user_id|project_id|source]
>>  optional start and end for counter_datetime
>>
>> Hope this makes sense.
>>
>> Another item that we need to discuss is extensibility of this API.
>>
>> Nick


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] API Extensibility (was: External API definition)

2012-05-10 Thread Nick Barcet
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Loic Dachary  wrote:

>> Another item that we need to discuss is extensibility of this API.
>
>Hi,
>
>Here is a proposal, which we could discuss further during the meeting.
>
>GET extension=¶m1=foo¶m2=bar
>
>The API looks up /usr/share/ceilometer/extensions/.py and loads it.
>The  module defines a query function that takes the following
>arguments:
>
>* QUERY_STRING (i.e. extension=¶m1=foo¶m2=bar )
>* a handler to the storage
>* a pointer to the configuration (assuming there is a
>/etc/ceilometer.ini file, for instance)
>
>The query function would return the result. For instance { 'in': 20001,
>'out': 489324 } if asked for aggregated network usage.
>
>Multiple extensions directories could be specified and searched,
>allowing a mixture of extensions provided in ceilometer and custom
>extensions to address specific needs or to mature an new extension.
>
>The primary benefit of defining extensions in this way is to avoid
>complex conventions for aggregations or other advanced operations. If
>the API was to impose a syntax or conventions to say "sum this field
>and this one and display the result ordered in this way and grouped by
>this field and this one", it would be redundant with the query language
>of the underlying data. For instance, if using mongodb, it would be
>difficult to expose all the features provided by
>http://www.mongodb.org/display/DOCS/Aggregation or
>http://www.mongodb.org/display/DOCS/MapReduce

I like this approach and the possibilities it provides. It will introduce some 
interesting questions for caching implementation, but we hound not worry about 
that yet, I think.

- --
Nick Barcet 
aka: nicolas, nijaba
-BEGIN PGP SIGNATURE-
Version: APG v1.0.8

iGsEAREIACsFAk+r0LkkHE5pY29sYXMgQmFyY2V0IDxuaWNvbGFzQGJhcmNldC5j
b20+AAoJEFiD3l2iIpt4/zQAmQEqxPvRVlTpndcwNwhl0SeHq9i8AKCXMsGcPI0g
NNaIx8a+3rwi2Dlaeg==
=WkXC
-END PGP SIGNATURE-


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] External API definition

2012-05-10 Thread Nick Barcet
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Daniel Dyer  wrote: One per installation, at least, since 
the source field could allow to aggregate informations from multiple 
installations.

>Is it your assumption that there will be one metering service per
>"installation" or one per service (i.e swift, nova)? My assumption
>would be
>a single metering service, so the API would need to handle some
>additional
>use cases:
>-list services supported
>-list metrics for a service type
>-get metric details

One per installation, at least, since the source field could allow to aggregate 
information from multiple installations. Can't See any reason why not to offer 
what you list above, even though one may deduce the component from the counter 
name.

>I would also consider separate use cases for accessing raw events vs.
>aggregated metrics.

I think the extension proposal from Loic would cover that and more.

>Dan Dyer
>dan.d...@hp.com
>
>On Wed, May 9, 2012 at 10:44 AM, Nick Barcet
>wrote:
>
>>
>>
>> Doug Hellmann  wrote:
>>
>> >On Wed, May 9, 2012 at 11:27 AM, Nick Barcet
>> >wrote:
>> >
>> >> On 05/08/2012 08:27 AM, Nick Barcet wrote:
>> >> [..]
>> >>
>> >> Thinking about this, I think we need to expend the API a bit to
>> >reflect
>> >> the evolutions of the schema that we decided last week.  Here are
>my
>> >> proposals:
>> >>
>> >> > * Requests allow to
>> >> >   GET account_id list
>> >>
>> >> change to: GET [user_id|project_id|source] list
>> >>
>> >
>> >Does the [value|value] syntax mean "choose one" or "combine"? I
>assume
>> >"choose one" and you are using square brackets because parens are
>used
>> >in some of the other queries.
>>
>> You assumed correctly :)
>>
>> >>
>> >> >   GET list of counter_type
>> >> >   GET list of events per account
>> >> > optional start and end for counter_datetime
>> >> > optional counter_type
>> >>
>> >> change to: GET list of events per [user_id|project_id|source]
>> >> optional start and end for counter_datetime
>> >>optional counter_type
>> >>
>> >
>> >Users may cross projects, so I'm not sure it makes sense to ask for
>the
>> >events generated by a user without restricting it by the project. At
>> >the very least we may need to allow them to specify user_id or
>project_id
>> >or both.
>>
>> Good point. Thanks for catching this.
>>
>> >>
>> >> >   GET sum of (counter_volume, counter_duration) for counter_type
>> >and
>> >> > account_id
>> >> > optional start and end for counter_datetime
>> >>
>> >>   GET sum of (counter_volume, counter_duration) for counter_type
>and
>> >> [user_id|project_id|source]
>> >>  optional start and end for counter_datetime
>> >>
>> >> Hope this makes sense.
>> >>
>> >> Another item that we need to discuss is extensibility of this API.
>> >>
>> >> Nick
>>
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>


- --
Nick Barcet 
aka: nicolas, nijaba
-BEGIN PGP SIGNATURE-
Version: APG v1.0.8

iGsEAREIACsFAk+r0yYkHE5pY29sYXMgQmFyY2V0IDxuaWNvbGFzQGJhcmNldC5j
b20+AAoJEFiD3l2iIpt4+w0AmgIBEBQUXHAeOiTko3X5lYcGjqi4AKCQcUC9DyPe
FBhL9NxeTMtAv1xsJg==
=7Udb
-END PGP SIGNATURE-


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] Meeting agenda for today 16:00 UTC (May 3rd, 2012)

2012-05-10 Thread Nick Barcet
; of floating IPs"  (dachary, 16:09:18)
>>   * ACTION: jd___ include Number of object in Swift, Number of
>> containers in Swift, Number of GET/HEAD/PUT/POST requests in
>Swift
>> in the table  (dachary, 16:10:11)
>>   * ACTION: dachary add note about the fact that the resource_id for
>the
>> object count is the container_id  (dachary, 16:21:44)
>>   * LINK: http://wiki.openstack.org/EfficientMetering#Counters is
>agreed
>> on, provided the actions listed above are carried out.  (dachary,
>> 16:25:35)
>>   * ACTION: jd___ document the resource_id for each counter
>(dachary,
>> 16:30:33)
>>   * ACTION: jd___  describes the general table schema and then
>something
>> that says for each counter exactly what goes in the fields of
>that
>> table and show how secondary field counters are recorded in the
>in
>> the schema too  (dachary, 16:33:27)
>>   * AGREED: s/counter/meter/  (dachary, 16:37:11)
>>   * LINK: http://wiki.openstack.org/EfficientMetering#Counters is
>agreed
>> on, provided the actions listed above are carried out. ?
>(dachary,
>> 16:37:38)
>>   * AGREED: http://wiki.openstack.org/EfficientMetering#Counters is
>> agreed on, provided the actions listed above are carried out. ?
>> (dachary, 16:38:52)
>>
>> * schema definition  (dachary, 16:39:05)
>>   * Proposed http://wiki.openstack.org/EfficientMetering#Storage
>> (dachary, 16:39:05)
>>   * ACTION: jd___ clarify / document the fact that the secondary
>field
>> or the resource_id field carries the IP/VIF for the meters
>related
>> to network so that they can be "per IP"  (dachary, 16:40:42)
>>   * ACTION: nijaba describe the source field rationale and use case,
>> initiate a thread to validate its use.  (dachary, 16:50:26)
>>   * AGREED: the fields should be source (to be discussed), user_id,
>> project_id, resource_id, counter_type, counter_volume,
>> counter_duration, counter_datetime, secondary type / payload,
>> message_signature, message_id ?  (dachary, 16:53:20)
>>   * ACTION: jd___ update the documentation  (dachary, 16:53:48)
>>   * ACTION: jd___ document the resource_id for each counter
>(dachary,
>> 16:54:10)
>>
>> * discuss API assumptions  (dachary, 16:54:51)
>>
>> * open discussion  (dachary, 16:55:19)
>>
>>
>>
>> Meeting ended at 17:00:05 UTC.
>>
>>
>>
>> Action items, by person
>> ---
>>
>> * dachary
>>   * dachary fix the note for net_float still talks about "number of
>> floating IPs"
>>   * dachary add note about the fact that the resource_id for the
>object
>> count is the container_id
>> * jd___
>>   * jd___ include Number of object in Swift, Number of containers in
>> Swift, Number of GET/HEAD/PUT/POST requests in Swift in the table
>>   * jd___ document the resource_id for each counter
>>   * jd___  describes the general table schema and then something that
>> says for each counter exactly what goes in the fields of that
>table
>> and show how secondary field counters are recorded in the in the
>> schema too
>>   * jd___ clarify / document the fact that the secondary field or the
>>     resource_id field carries the IP/VIF for the meters related to
>> network so that they can be "per IP"
>>   * jd___ update the documentation
>>   * jd___ document the resource_id for each counter
>> * nijaba
>>   * nijaba describe the source field rationale and use case, initiate
>a
>> thread to validate its use.
>>
>>
>>
>> --
>> Loïc Dachary Chief Research Officer
>> // eNovance labs   http://labs.enovance.com
>> // ✉ l...@enovance.com  ☎ +33 1 49 70 99 82
>>
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
>
>
>___
>Mailing list: https://launchpad.net/~openstack
>Post to : openstack@lists.launchpad.net
>Unsubscribe : https://launchpad.net/~openstack
>More help   : https://help.launchpad.net/ListHelp


- --
Nick Barcet 
aka: nicolas, nijaba
-BEGIN PGP SIGNATURE-
Version: APG v1.0.8

iGsEAREIACsFAk+r1T8kHE5pY29sYXMgQmFyY2V0IDxuaWNvbGFzQGJhcmNldC5j
b20+AAoJEFiD3l2iIpt4QXUAoLJwCIl2vnyb9uTOFfJCH63E2qoNAJ0XkRyeBSty
+nXCM+8IxnXGB3TAFg==
=3gjy
-END PGP SIGNATURE-


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [metering] resources metadata

2012-05-16 Thread Nick Barcet

> On Tue, May 15, 2012 at 10:26 AM, Doug Hellmann
> mailto:doug.hellm...@dreamhost.com>> wrote:
> 
> 
> 
> On Tue, May 15, 2012 at 8:21 AM, Julien Danjou
> mailto:julien.dan...@enovance.com>> wrote:
> 
> On Tue, May 15 2012, Loic Dachary wrote:
> 
> > On 05/15/2012 12:05 PM, Julien Danjou wrote:
> >>
> >> OTOH I find the metadata proposal in another table too much
> >> complicated. Why not storing what metadata in the
> meter.payload field
> >> in the same table (e.g. as a JSON string)?
> > I would be much simpler to store the metadata in the
> resource_id field
> > which could be renamed into resource field.
> 
> That'd be even more radical.
> 
> 
> I like it because it would simplify the messaging. We can leave the
> storage optimization question to the daemon that stores the data.
>
> > Instead of resource_id=134123 we could have resource={ 'id':
> 134123,
> > 'name': 'foobar', 'flavor': 'm1.small' etc.. } There would be
> no need
> > for versioning, format, separate table, etc. etc. The only
> convention
> > would be that it's a hash with at least one field : the id of the
> > resource. The rest is metadata.
> >
> > It will use a lot of disk space with highly redundant information.
> 
> Ok, so the current proposal is just early optimization, as I
> understood.
> 
> If you want to optimize the storage, why not use resource_id as a
> foreign key to the metatable table which would contains unique
> records
> of metadata?
> 
> That would allow to store identical metadata once (and therefore
> optimize space) and will be much simpler. There would not be any
> need of
> version, timestamp, or whatever on metadata.

I fully second Julien's analysis regarding the storage of metadata in a
separate table.  We are overly complicating the schema for the sole
purpose of resource optimization.  In fact, the current metadata table
seems to be defining 5 fields where only one constitutes some valid
content we would like to extract at some point, the rest existing only
for relational or versioning purposes.  A bit of an overkill, it seems.

I would not recommend however to overcharge a field that can be used as
a key value for searches (resource_id) to store additional data, as it
would block us to search on that key easily.  Why mot just extend the
meter schema to add a metadata field, which can use the proposed JSON
format?

Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [Metering] Meeting agenda for Thursday at 16:00 UTC (May 17th, 2012)

2012-05-16 Thread Nick Barcet
Hi,

The metering project team holds a weekly meeting in #openstack-meeting,
Thursdays at 1600 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.
Everyone is welcome.

Since we were not able to conclude on the API discussion last week we
continued our dicussions on the list and on the #openstack-metering
channel and are now coming with a better proposal (or we are at least
hoping it is better).

http://wiki.openstack.org/Meetings/MeteringAgenda
Topic: external API definition (part 2)

  * Agree on update to schema to include JSON formated metadata as
described at [1]
  * Agree on API proposal described at [2]
  * Agree on format for date_time.  Suggestion is to use ISO but seeking
validation for best practice for REST
  * Agree on the use of a transparent cache for aggregation
  * Open discussion (if we have any time left)

[1] http://wiki.openstack.or
<http://wiki.openstack.org/EfficientMetering#Storage>g/EfficientMetering#Storage
<http://wiki.openstack.org/EfficientMetering#Storage>

[2] http://wiki.openstack.or
<http://wiki.openstack.org/EfficientMetering/APIProposalv1>g/EfficientMetering/APIProposalv1
<http://wiki.openstack.org/EfficientMetering/APIProposalv1>

Cheers,
--
Nick Barcet 
aka nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Stackforge server migration

2012-05-16 Thread Nick Barcet
On 05/14/2012 08:57 PM, Doug Hellmann wrote:
> 
> 
> On Mon, May 14, 2012 at 1:35 PM, Andrew Hutchings
> mailto:and...@linuxjedi.co.uk>> wrote:
> 
> Hey Jay,
> 
> On 14/05/12 17:12, Jay Pipes wrote:
> > On 05/14/2012 11:38 AM, Andrew Hutchings wrote:
> >> The first problem is more serious and can't be fixed where Gerrit is
> >> now, so it will require us to migrate Gerrit to a different
> server.  I
> >> also intend to do this later this week and it will cause at least an
> >> hour of downtime.
> >
> > Any more detail on this? What is the reason that mail doesn't work on
> > the original Gerrit server?
> 
> There is something outside the node (it is an HP Cloud node) blocking
> outgoing port 25 which isn't affecting any of our other nodes (we
> presume some bad iptables on the node's host server).
> 
> It needs migrating to the stackforge cloud account anyway, this will
> just speed up the migration.
> 
> 
> +1 to migrate so we can get email notification working, especially if we
> need to do it for other reasons anyway.

+1 here too.  Do not worry about our commits, there aren't that many yet
anyway and you are working toward simplifying our lives, so thanks!

Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [metering] Do we need an API and storage?

2012-05-17 Thread Nick Barcet
On 05/17/2012 11:13 AM, Loic Dachary wrote:
> On 05/16/2012 11:00 PM, Francis J. Lacoste wrote:
>> 
>> I'm now of the opinion that we exclude storage and API from the
>> metering project scope. Let's just focus on defining a metering
>> message format, bus, and maybe a client-library to make it easy to
>> write metering consumers.
>> 
>> That way we avoid building a lot of things that we only _think will
>> be useful_ for potential billing integration. Only
>> writing/delivering such an integration component would prove that
>> we built at least something that is useful.
>> 
>> 
> 
> Hi,
> 
> I'm a little reluctant to question the whole approach because I'm
> engaged in it :-) It's scary to contemplate the idea of throwing away
> some of the work done but I welcome the challenge. Better lose a few
> days work than keep going in the wrong direction.
> 
> Are you proposing that such a library would then be integrated in
> whatever billing system someone has in mind ? For instance
> 
> Dough https://github.com/lzyeval/dough trystack.org billing
> https://github.com/trystack/dash_billing nova-billing
> https://github.com/griddynamics/nova-billing
> 
> Would depend on this library and rely on its API to collect what they
> need. The same would be done by proprietary billing systems ?
> 
> Regarding the relevance of the metrics exposed by the API, I made
> sure they match the need of the eNovance sales. I'm quite sure
> Nicolas checked for Canonical and Doug did the same regarding the
> needs of Dreamhost. I'm confident that the information is actualy
> useful, at least in these practical cases.
> 
> Getting rid of the storage imposes a constraint on the billing system
> : it must make 100% sure that once a message is consumed it will be
> reliably archived. It also adds a constraint on the chosen bus : it
> must be able to retain all messages for as long as a consumer needs,
> which may be days or weeks. Or it adds a constraint on the billing
> system which must make 100% sure it will consume all relevant
> messages the bus at all times before they expire.
> 
> I have no strong feelings about exposing a bus for everyone to use
> instead of a REST API. I tend to prefer the REST API because it is an
> established standard for OpenStack. Could you expand on why a bus
> would be significantly better than a REST API in this specific case
> ?

Thanks for challenging our thought process, Francis, it is a great
sanity check :)

I do have a few use cases where a REST API is better than a bus:

* Private clouds:  Users are unlikely to want to activate a complete
billing system but still want to be able top provide to their users some
metrics of their uses.  The REST API plus some scripts would allow them
to do this without too much pain.

* Metric of usage on GUI: It might be usefull to provide a quick way to
assess usage in almost real time to users as part of an extension of
Horizon for example.  The REST API would allow for such data to be
extracted dynamically without having to run a full billing tool in real
time.

* In house billing tools: about half of the ISPs I surveyed are running
some form of a customized ERP system to handle the billing for their
customers.  They need to be able to produce CSVs on a weekly basis to
feed their custom billing solutions.  Integrating a bus would be much
more complex than the script that would issue request to the rest API.

* Auditability/Non repudiability: if the messages go from the queue to
some unknown DB, how do you solve, in a generic way, coherent audit
check and ensure non-repudiability?

I do not mean to totally discard Francis' bus idea though, and think
that we should allow for (a not necessarily db-less integration but)
direct queue attachment model for billing systems. I do think that for
all the above reasons, plus the simplification of testing of the overall
system, the REST API must remain part of this project.

Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] Meeting agenda for Thursday at 16:00 UTC (May 17th, 2012)

2012-05-18 Thread Nick Barcet
On 05/16/2012 08:44 PM, Nick Barcet wrote:
> Hi,
> 
> The metering project team holds a weekly meeting in #openstack-meeting,
> Thursdays at 1600 UTC
> <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.
> Everyone is welcome.
> 
> Since we were not able to conclude on the API discussion last week we
> continued our dicussions on the list and on the #openstack-metering
> channel and are now coming with a better proposal (or we are at least
> hoping it is better).
> 
> http://wiki.openstack.org/Meetings/MeteringAgenda
> Topic: external API definition (part 2)
> 
>   * Agree on update to schema to include JSON formated metadata as
> described at [1]
>   * Agree on API proposal described at [2]
>   * Agree on format for date_time.  Suggestion is to use ISO but seeking
> validation for best practice for REST
>   * Agree on the use of a transparent cache for aggregation
>   * Open discussion (if we have any time left)
> 
> [1] http://wiki.openstack.org/EfficientMetering#Storage
> [2] http://wiki.openstack.org/EfficientMetering/APIProposalv1

The meeting took place yesterday, a summary follows.

=
#openstack-meeting Meeting
==

Meeting started by nijaba at 16:01:26 UTC.  The full logs are available
at
http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-05-17-16.01.log.html

Meeting summary
---

* agenda http://wiki.openstack.org/Meetings/MeteringAgenda  (nijaba,
  16:01:26)

* actions from previous meetings  (nijaba, 16:02:14)
  * dachary add info to the wiki on the topic of poll versus push
(nijaba, 16:02:26)
  * dhellmann: reformulate the API proposal as a start point for the
dicussion on the ML  (nijaba, 16:03:25)

* Agree on update to schema to include JSON formated metadata  (nijaba,
  16:05:55)
  * LINK: http://wiki.openstack.org/EfficientMetering#Storage  (nijaba,
16:05:55)
  * AGREED: to update to schema to include JSON formated metadata
(nijaba, 16:10:40)

* Agree on API proposal  (nijaba, 16:10:56)
  * LINK: http://wiki.openstack.org/EfficientMetering/APIProposalv1
(nijaba, 16:10:56)
  * AGREED: on API proposal
http://wiki.openstack.org/EfficientMetering/APIProposalv1  (nijaba,
16:15:01)
  * ACTION: flacoste to follow on the discussion about a bus only
implementation  (nijaba, 16:17:43)

* Agree on format for date_time  (nijaba, 16:15:22)
  * Suggestion is to use ISO but seeking validation for best practice
for REST  (nijaba, 16:15:22)
  * ACTION: nijaba to add the use to UTC for datetime  (nijaba,
16:17:03)
  * AGREED: to use ISO for datetime  (nijaba, 16:18:54)

* Agree on the use of a transparent cache for aggregation  (nijaba,
  16:19:11)
  * AGREED: caching is an implementation detail  (nijaba, 16:23:58)

* Open discussion  (nijaba, 16:24:28)
  * ACTION: dhellmann to document a counter for discrete event as an
example  (nijaba, 16:29:02)

Meeting ended at 16:31:15 UTC.

Action items, by person
---

* dhellmann
  * dhellmann to document a counter for discrete event as an example
* flacoste
  * flacoste to follow on the discussion about a bus only implementation
* nijaba
  * nijaba to add the use to UTC for datetime

Next week's meeting will cover the choice of a messaging queue for the
project.

Cheers,
--
Nick Barcet 
aka: nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [metering] Choice of a messaging queue

2012-05-18 Thread Nick Barcet
Hello everyone,

Next week's irc meeting will have for goal to choose a reference
messaging queue service for the ceilometer project.  For this meeting to
be able to be successful, a discussion on the choices that we have to
make need to occur first right here.

To open the discussion here are a few requirements that I would consider
important for the queue to support:

a) the queue must guaranty the delivery of messages.
To the contrary of monitoring, loss of events may have important billing
impacts, it therefore cannot be an option that message be lost.

b) the client should be able to store and forward.
As the load of system or traffic increases, or if the client is
temporarily disconnected, client element of the queue should be able to
hold messages in a local queue to be emitted as soon as condition permit.

c) client must authenticate
Only client which hold a shared private key should be able to send
messages on the queue.

d) queue may support client signing of individual messages
Each message should be individually signed by the agent that emits it in
order to guaranty non repudiability.  This function can be done by the
queue client or by the agent prior to en-queuing of messages.

d) queue must be highly available
the queue servers must be able to support multiple instances running in
parallel in order to support continuation of operations with the loss of
one server.  This should be achievable without the need to use complex
fail over systems and shared storage.

e) queue should be horizontally scalable
The scalability of queue servers should be achievable by increasing the
number of servers.

Not sure this list is exhaustive or viable, feel free to comment on it,
but the real question is: which queue should we be using here?

Cheers,
--
Nick Barcet 
aka: nijaba, nicolas




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [metering] example of discrete counter

2012-05-18 Thread Nick Barcet
On 05/17/2012 10:48 PM, Doug Hellmann wrote:
> I have added a row to the list of counters for discrete events such as
> uploading an image to glance [1]. Please let me know if you think I need
> more exposition to explain discrete counters.
> 
> Doug
> 
> [1] http://wiki.openstack.org/EfficientMetering?action=diff&rev2=89&rev1=87
> 

Thanks Doug, it looks good to me.

Cheers,
Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [metering] high-level design proposal

2012-05-22 Thread Nick Barcet
On 05/21/2012 10:52 PM, Doug Hellmann wrote:
> I have written up some of my thoughts on a proposed design for
> ceilometer in the wiki [1]. I'm sure there are missing details, but I
> wanted to start getting ideas into writing so they could be discussed
> here on the list, since I've talked about different parts with a couple
> of you separately.
> 
> Let me know what you think, and especially if I am not clear or have
> left out any details.
> 
> Thanks,
> Doug
> 
> [1] http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1

Thanks a lot for putting this together Doug.

A few questions:

* "The collector runs on one or more central management servers to
monitor the message queues (for notifications and for metering data
coming from the agent). Notification messages are processed and turned
into metering messages and sent back out onto the message bus using the
appropriate topic. Metering messages are written to the data store
without modification."
-> Is the reason behind why collectors do not write directly to the
database a way to allow db less implementations as Francis suggested
earlier?  In this case it may be useful to say it explicitly.

* "Plugins may require configuration options, so when the plugin is
loaded it is asked to add options to the global flags object, and the
results are made available to the plugin before it is asked to do any work."
-> I am not sure where the "global flags object" resides and how option
are populated.  I think it would make sense for this to be globally
controlled, and therefore may require for a simple discovery exchange on
the queue to retrieve values and set defaults if it does not exist yet.

* "Metering messages are signed using the hmac module in Python's
standard library. A shared secret value can be provided in the
ceilometer configuration settings. The messages are signed by feeding
the message key names and values into the signature generator in sorted
order. Non-string values are converted to unicode and then encoded as
UTF-8. The message signature is included in the message for verification
by the collector."
-> The signature is also kept in the database for future audit
processes, maybe worth mentioning it here.
-> In addition to a signature, I think we would need a sequence number
to be embedded by the agent for each message sent, so that loss of
messages, or forgery of messages, can be detected by the collector and
further audit process.

Thanks again,
Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [metering] high-level design proposal

2012-05-22 Thread Nick Barcet
On 05/22/2012 03:26 PM, Doug Hellmann wrote:
> -> In addition to a signature, I think we would need a sequence number
> to be embedded by the agent for each message sent, so that loss of
> messages, or forgery of messages, can be detected by the collector and
> further audit process.
> 
> 
> OK. We have a message id, but I assumed those would be used to eliminate
> duplicates so this sounds like something different or new. It implies
> that the agent knows its own id (not hard) and keeps up with a sequence
> counter (more difficult, though not impossible). Did you have something
> in mind for how to implement that?

Actually, this was my intent in the original blueprint when I specified
the "message_id" field then a couple lines bellow: "a process may verify
that messages were not lost".  On the implementation side, I was
thinking that each agent would maintain its own sequence count, as a
global instance count would be pricier.  In my mind, non repudiation was
built from the message_signature + message_id which should be unique for
each agent.

Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Glance] Using HTTP PATCH

2012-05-23 Thread Nick Barcet
On 05/23/2012 09:39 PM, Joshua Harlow wrote:
> Is there anyway u can just make it a configuration option?
> 
> v2imagemethod = ‘PUT’...

This would make sense as rfc 5789 is only 2 years old, which means that
there must be numerous firewalls and proxies which may not have been
updated yet in production, if an update is available at all, that will
just block the patch method...  Most of the customers we are talking to
are not that keen on updating what works.

Nick

> On 5/23/12 1:22 PM, "Brian Waldon"  wrote:
> 
> Hey guys,
> 
> I'm considering using PATCH rather than PUT for image updates in the
> v2 Image API, but I wanted to make sure there weren't any major
> blockers that I might be missing. As far as I can tell, the python
> libraries we use in Glance support it (httplib2, requests). However,
> I discovered that Squid doesn't document official support of the
> method: http://wiki.squid-cache.org/SquidFaq/SquidLogs. I wanted to
> ask if there are any other key infrastructure pieces that foolishly
> hard-code request methods like this. It might make us want to
> rethink using PATCH at all.
> 
> Brian
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [metering] high-level design proposal

2012-05-23 Thread Nick Barcet
On 05/22/2012 07:15 PM, Doug Hellmann wrote:
> 
> 
> On Tue, May 22, 2012 at 1:25 PM, Nick Barcet  <mailto:nick.bar...@canonical.com>> wrote:
> 
> On 05/22/2012 03:26 PM, Doug Hellmann wrote:
> > -> In addition to a signature, I think we would need a
> sequence number
> > to be embedded by the agent for each message sent, so that loss of
> > messages, or forgery of messages, can be detected by the
> collector and
> > further audit process.
> >
> >
> > OK. We have a message id, but I assumed those would be used to
> eliminate
> > duplicates so this sounds like something different or new. It implies
> > that the agent knows its own id (not hard) and keeps up with a
> sequence
> > counter (more difficult, though not impossible). Did you have
> something
> > in mind for how to implement that?
> 
> Actually, this was my intent in the original blueprint when I specified
> the "message_id" field then a couple lines bellow: "a process may verify
> that messages were not lost".  On the implementation side, I was
> thinking that each agent would maintain its own sequence count, as a
> global instance count would be pricier.  In my mind, non repudiation was
> built from the message_signature + message_id which should be unique for
> each agent.
> 
> 
> OK. That brings a couple of more specific questions to mind:
> 
> Does the agent save its sequence counter through a restart? How and
> where? What about an upgrade?

Seems easily stored locally.

> What would the down-stream consumer of the data do if it discovered
> there was a missing event? Who should do that detection work?

Not sure we need to worry about auditing process yet, just make sure
that we provide necessary the necessary information to do proper
auditing.  In principle, an audit process could then trigger an alert
for further investigation of the issue.

Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [Metering] Agent configuration mechanism

2012-06-05 Thread Nick Barcet
Following up on our last meeting, here is a proposal for centrally
hosting configuration of agents in ceilometer.

The main idea is that all agents of a given type should be sending
similarly formatted information in order for the information to be
usable, hence the need to ensure that configuration info is centrally
stored and retrieved.  This would rule out, in my mind, the idea that we
could use the global flags object, as distribution of the configuration
file is left to the cloud implementor and does not lend for easy and
synchronized updates of agent config.

Configuration format and content is left to the agent's implementation,
but it is assumed that each meter covered by an agent can be :
 * enabled or disabled
 * set to send information at a specified interval.

1/ Configuration is stored for each agent in the database as follow
+---+
| Field | Type | Note   |
+---+
| AgentType | String   | Unique agent type  |
| ConfVers  | Integer  | Version of the configuration   |
| Config| Text | JSON Configuration info (defined by agent) |
+---+--++

2/ Config is retreived via the messaging queue upon boot once a day
(this should be defined in the global flags object) to check if the
config has changed.

Request sent by the agent upon boot and :

'reply_to': 'get_config_data',
'correlation_id': x
'version': '1.0',
'args': {'data': {
   'AgentType': agent.type,
   'CurrentVersion': agent.version,
   'ConfigDefault': agent.default,
   },
},

Where ConfigDefault are the "sane" default proposed by the agent authors.

If no config record is found the collector creates the record, sets
ConfVers to 1 and sends back a normal reply.

Reply sent by the collector:
'correlation_id': x
'version': '1.0',
'args': {'data': {
   'Result': result.code,
   'ConfVers': ConfVers,
   'Config': Config,
   },
},
}

Result is set as follow:
200  -> Config was retrieved successfully
201  -> Config was created based on received default (Config is empty)
304  -> Config version is identical to CurrentVersion (Config is empty)

This leaves open the question of having some UI to change the config,
but I thing we can live with manual updating of the records for the time
being.

Comments welcome...

Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] Agent configuration mechanism

2012-06-05 Thread Nick Barcet
On 06/05/2012 04:44 PM, Doug Hellmann wrote:
> On Tue, Jun 5, 2012 at 10:41 AM, Doug Hellmann
> mailto:doug.hellm...@dreamhost.com>> wrote:
> On Tue, Jun 5, 2012 at 9:56 AM, Nick Barcet
> mailto:nick.bar...@canonical.com>> wrote:
> 
> Following up on our last meeting, here is a proposal for centrally
> hosting configuration of agents in ceilometer.
> 
> The main idea is that all agents of a given type should be sending
> similarly formatted information in order for the information to be
> usable, hence the need to ensure that configuration info is
> centrally
> stored and retrieved.  This would rule out, in my mind, the idea
> that we
> could use the global flags object, as distribution of the
> configuration
> file is left to the cloud implementor and does not lend for easy and
> synchronized updates of agent config.
> 
> Configuration format and content is left to the agent's
> implementation,
> but it is assumed that each meter covered by an agent can be :
>  * enabled or disabled
>  * set to send information at a specified interval.
> 
> 
> Right now we only have one interval for all polling. Do you think we
> need to add support for polling different values at different
> intervals? Do we need other per-agent settings, or are all of the
> settings the same for all agents? (I had assumed the latter would be
> all we needed.)

I would have thought that we may want to support different intervals per
meters, based on the billing rules that one may want to offer.  For
example, I may want to bill compute by the hour but floating IPs by the
day, hence have a different reporting interval for each.

> 1/ Configuration is stored for each agent in the database as follow
> +---+
> | Field | Type | Note  
> |
> +---+
> | AgentType | String   | Unique agent type  
>|
> | ConfVers  | Integer  | Version of the configuration  
> |
> | Config| Text | JSON Configuration info (defined by
> agent) |
> +---+--++
> 
> 2/ Config is retreived via the messaging queue upon boot once a day
> (this should be defined in the global flags object) to check if the
> config has changed.
> 
> 
> Updating the config once a day is not going to be enough in an
> environment with a lot of compute nodes.
> 
> 
> Two thoughts merged into one sentence there. Need more caffeine. 
> 
> What I was trying to say, was that updating the config once a day might
> not be enough and in environments with a lot of compute nodes going
> around to manually restart the services each time the config changes
> will be a pain. See below for more discussion of pushing config settings
> out.

Agreed, and that's why I proposed that the interval for confguration
refresh should be set in the Global object flag (this is something that
can be shared among all the agents).

> 
> 
> Request sent by the agent upon boot and :
> 
>'reply_to': 'get_config_data',
>'correlation_id': x
>'version': '1.0',
>'args': {'data': {
>   'AgentType': agent.type,
>   'CurrentVersion': agent.version,
>   'ConfigDefault': agent.default,
>   },
>},
> 
> 
> Is this a standard OpenStack RPC call?

Not sure about that, but if it can be, it would be easier :)

> Where ConfigDefault are the "sane" default proposed by the agent
> authors.
> 
> 
> Why is the agent proposing default settings?

So that the first agent of a given type can populate its info with sane
defaults that can then be edited later on?

> If no config record is found the collector creates the record, sets
> ConfVers to 1 and sends back a normal reply.
> 
> Reply sent by the collector:
>'correlation_id': x
>'version': '1.0',
> 
> 
> Do we need minor versions for the config settings, or are those
> simple sequence numbers to track which settings are the "most current"?

Simple sequence was w

Re: [Openstack] [Metering] Agent configuration mechanism

2012-06-06 Thread Nick Barcet
On 06/05/2012 09:03 PM, Doug Hellmann wrote:
> 
> 
> On Tue, Jun 5, 2012 at 12:59 PM, Nick Barcet  <mailto:nick.bar...@canonical.com>> wrote:
> 
> On 06/05/2012 04:44 PM, Doug Hellmann wrote:
> > On Tue, Jun 5, 2012 at 10:41 AM, Doug Hellmann
> > mailto:doug.hellm...@dreamhost.com>
> <mailto:doug.hellm...@dreamhost.com
> <mailto:doug.hellm...@dreamhost.com>>> wrote:
> > On Tue, Jun 5, 2012 at 9:56 AM, Nick Barcet
> > mailto:nick.bar...@canonical.com>
> <mailto:nick.bar...@canonical.com
> <mailto:nick.bar...@canonical.com>>> wrote:
> >
> > Following up on our last meeting, here is a proposal for
> centrally
> > hosting configuration of agents in ceilometer.
> >
> > The main idea is that all agents of a given type should be
> sending
> > similarly formatted information in order for the
> information to be
> > usable, hence the need to ensure that configuration info is
> > centrally
> > stored and retrieved.  This would rule out, in my mind,
> the idea
> > that we
> > could use the global flags object, as distribution of the
> > configuration
> > file is left to the cloud implementor and does not lend
> for easy and
> > synchronized updates of agent config.
> >
> > Configuration format and content is left to the agent's
> > implementation,
> > but it is assumed that each meter covered by an agent can be :
> >  * enabled or disabled
> >  * set to send information at a specified interval.
> >
> >
> > Right now we only have one interval for all polling. Do you
> think we
> > need to add support for polling different values at different
> > intervals? Do we need other per-agent settings, or are all of the
> > settings the same for all agents? (I had assumed the latter
> would be
> > all we needed.)
> 
> I would have thought that we may want to support different intervals per
> meters, based on the billing rules that one may want to offer.  For
> example, I may want to bill compute by the hour but floating IPs by the
> day, hence have a different reporting interval for each.
> 
> 
> I was planning to aggregate the values for items being billed over the
> longer time frames, but we can make the polling interval configurable.
> It will take some work, because of the way the scheduled tasks are
> configured in the service and manager (right now we just schedule one
> method to run, and it invokes each pollster).
> 
> How important is it to include this in Folsom?

Not crucial.  I would classify this as "Nice to have".

> > 1/ Configuration is stored for each agent in the database
> as follow
> >
> +---+
> > | Field | Type | Note
> > |
> >
> +---+
> > | AgentType | String   | Unique agent type
> >|
> > | ConfVers  | Integer  | Version of the configuration
> > |
> > | Config| Text | JSON Configuration info (defined by
> > agent) |
> >
> +---+--++
> >
> > 2/ Config is retreived via the messaging queue upon boot
> once a day
> > (this should be defined in the global flags object) to
> check if the
> > config has changed.
> >
> >
> > Updating the config once a day is not going to be enough in an
> > environment with a lot of compute nodes.
> >
> >
> > Two thoughts merged into one sentence there. Need more caffeine.
> >
> > What I was trying to say, was that updating the config once a day
> might
> > not be enough and in environments with a lot of compute nodes going
> > around to manually restart the services each time the config changes
> > will be a pain. See below for more discussion of pushing config
> settings
> > out.
> 
> Agreed, and that's why I proposed that the interval for confguration
>   

Re: [Openstack] [Metering] Agent configuration mechanism

2012-06-06 Thread Nick Barcet
On 06/06/2012 10:39 AM, Julien Danjou wrote:
> On Tue, Jun 05 2012, Nick Barcet wrote:
> 
>> I would have thought that we may want to support different intervals per
>> meters, based on the billing rules that one may want to offer.  For
>> example, I may want to bill compute by the hour but floating IPs by the
>> day, hence have a different reporting interval for each.
> 
> I don't think you want to poll once a day something you bill per day. If
> you poll only at noon, and I use a resource from 8:00 to 10:00, you'll
> miss my usage and I'll use resource for free. :)

I don't think I specified which interval we would use for each, but
thanks for remind us of the limits of sampling.

> Yes, there's a minimum interval to be configured, but it's the interval
> that defined how much grained your metering/billing will be. E.g. if
> it's 1 hour, you won't charge anything used for less than one hour. You
> probably wants something like 5 minutes or less, and be sure the agent
> can keep up with the needed polling speed. :)

The only point I am trying to make is that the sampling interval is a
function of the billing interval. To insure efficient use of the
resources, it would be better (not crucial) to ensure that the polling
frequency be settable by meter, not just by agent.  The later would just
force to set the frequency to the lowest value needed for all meters
covered by that agent.  Again, this is acceptable, but not optimal.

Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] Agent configuration mechanism

2012-06-06 Thread Nick Barcet
On 06/06/2012 10:35 AM, Julien Danjou wrote:
> On Tue, Jun 05 2012, Nick Barcet wrote:
> 
>> The main idea is that all agents of a given type should be sending
>> similarly formatted information in order for the information to be
>> usable, hence the need to ensure that configuration info is centrally
>> stored and retrieved.  This would rule out, in my mind, the idea that we
>> could use the global flags object, as distribution of the configuration
>> file is left to the cloud implementor and does not lend for easy and
>> synchronized updates of agent config.
> 
> IMHO this is solving a problem that already exists for all other
> OpenStack components. A problem that no other OpenStack components tried
> to resolved, AFAIK. So I don't see why we should try to resolve
> configuration deployment in ceilometer when it's a much larger issue in
> the project.

Maybe the problem of discrepancy of configuration is more an issue for
metering than for other components? At least that's my belief.

In fact, I may very well want to have differences in configuration of
different compute nodes in a single zone, for very valid reasons (ie use
different hypervisor settings, balance usage of components, etc...).
However if my metering agents are not all set to report the same things,
I think I'll be in trouble to correlate anything, hence the need for
metering configuration information to be provided centrally, for
anything that sets the behavior of meters and which meters to use.

Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] polling intervals, was: Agent configuration mechanism

2012-06-07 Thread Nick Barcet
On 06/06/2012 05:06 PM, Doug Hellmann wrote:
> Starting a new thread for this topic...
> 
> On Wed, Jun 6, 2012 at 5:43 AM, Nick Barcet  <mailto:nick.bar...@canonical.com>> wrote:
> 
> On 06/05/2012 09:03 PM, Doug Hellmann wrote:
> 
>  
> 
> > >
> > > Right now we only have one interval for all polling. Do you
> > think we
> > > need to add support for polling different values at
> different
> > > intervals? Do we need other per-agent settings, or are
> all of the
> > > settings the same for all agents? (I had assumed the latter
> > would be
> > > all we needed.)
> >
> > I would have thought that we may want to support different
> intervals per
> > meters, based on the billing rules that one may want to offer.
>  For
> > example, I may want to bill compute by the hour but floating
> IPs by the
> > day, hence have a different reporting interval for each.
> >
> >
> > I was planning to aggregate the values for items being billed over the
> > longer time frames, but we can make the polling interval configurable.
> > It will take some work, because of the way the scheduled tasks are
> > configured in the service and manager (right now we just schedule one
> > method to run, and it invokes each pollster).
> >
> > How important is it to include this in Folsom?
> 
> Not crucial.  I would classify this as "Nice to have".
> 
> 
> We need a ticket to track this. It's going to require some work on the
> agent service, because right now the code that loads the plugins doesn't
> have access to the object that knows how to run jobs at regular
> intervals. I would rather have a complete tool that works at a single
> interval, then go back and enhance it to allow other intervals during
> the next release cycle. Is that going to meet your needs?

As I said, this would not be a hard requirement for me, so no issues there.

Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] signing messages, was Re: Agent configuration mechanism

2012-06-07 Thread Nick Barcet
On 06/06/2012 05:20 PM, Doug Hellmann wrote:
> Starting a new thread...
> 
> On Wed, Jun 6, 2012 at 5:43 AM, Nick Barcet  <mailto:nick.bar...@canonical.com>> wrote:
> 
> On 06/05/2012 09:03 PM, Doug Hellmann wrote:
> >
> >
> > On Tue, Jun 5, 2012 at 12:59 PM, Nick Barcet
> mailto:nick.bar...@canonical.com>
> > <mailto:nick.bar...@canonical.com
> <mailto:nick.bar...@canonical.com>>> wrote:
> >
> > On 06/05/2012 04:44 PM, Doug Hellmann wrote:
> > > On Tue, Jun 5, 2012 at 10:41 AM, Doug Hellmann
> > >  <mailto:doug.hellm...@dreamhost.com>
> <mailto:doug.hellm...@dreamhost.com
> <mailto:doug.hellm...@dreamhost.com>>
> > <mailto:doug.hellm...@dreamhost.com
> <mailto:doug.hellm...@dreamhost.com>
>     > <mailto:doug.hellm...@dreamhost.com
> <mailto:doug.hellm...@dreamhost.com>>>> wrote:
> > > On Tue, Jun 5, 2012 at 9:56 AM, Nick Barcet
> > >  <mailto:nick.bar...@canonical.com> <mailto:nick.bar...@canonical.com
> <mailto:nick.bar...@canonical.com>>
> > <mailto:nick.bar...@canonical.com
> <mailto:nick.bar...@canonical.com>
> > <mailto:nick.bar...@canonical.com
> <mailto:nick.bar...@canonical.com>>>> wrote:
> > >
> > > Have you given any thought to distributing the secret value
> > used for
> > > signing incoming messages? A central configuration
> authority does
> > > not give us a secure way to deliver secrets like that.
> If anyone
> > > with access to the message queue can retrieve the key by
> > sending RPC
> > > requests, we might as well not sign the messages.
> >
> > Actually, the private key used to generate a signature should
> be unique
> > to each host, if we want them to have any value at all, therefore
> > distributing a common signature should NOT be part of this, or
> we would
> > fall under the notion of a shared secret, which is, IMHO, not
> any better
> > than having a global password.
> >
> > I would recommend that, for the time being, we just generate a
> random
> > key pair per host the first time the agent is run, allowing
> for someone
> > with further requirement to eventually populate this value by
> another
> > mean.
> >
> > In any case, if we want to effectively check the signature,
> the public
> > key does need to be accessible by the collector to check it
> and have yet
> > to define a way to do so...  Proposals welcome, but again,
> while I think
> > we should lay the ground for a great security experience, we
> certainly
> > don't need to solve it all in v1.
> >
> >
> > The current implementation uses hmac message signatures, which use a
> > shared secret instead of public/private key pairs. We can have a
> > separate secret for each agent, but we still need the collector(s) to
> > have them all. I thought the point of signing the messages was to
> > prevent an arbitrary agent from signing on to the message queue and
> > sending bogus data. Do we need to be doing more than hmac for
> security?
> 
> As stated since the beginning of the project, purpose of the signature
> is non repudiability, not only authentication.  If I understand
> correctly, hmac signature will only provide authentication through a
> shared secret, shared secret which then should not be transmited on the
> wire to the agent, or else it would loose all purpose. 
> 
> 
> Yes, hmac uses a shared secret to produce a SHA-256 (in our case)
> signature of the contents of the message. How that is interpreted is up
> to the application. For ceilometer it seemed like hmac would be
> sufficient for both ensuring that the agent was allowed to send messages
> (because it knows the secret) and that the message contents had not been
> modified in transit (because the signature is valid). We could have a
> separate secret for each agent, but I don't see how a keypair is better
> for this purpose. I'm no crypto expert, though. :-)
> 
> To cover the non-repudiability requirement the message includes a UUID1
> value, which includes the MAC of the sending host as well as time a

[Openstack] [metering] Ceilometer volume calculator

2012-06-08 Thread Nick Barcet
Hello,

Following up on yesterday's meeting, I have started a first version of a
google spreadsheet to estimate volume of events generated by ceilometer [1].

Comments and suggestions for improvement are of course welcome.

[1]
https://docs.google.com/spreadsheet/ccc?key=0AtziNGvs-uPudDhRbEJJOHFXV3d0ZGc1WE9NLTVPX0E

Cheers,
Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [Metering] Meter configuration mechanism v0.2

2012-06-11 Thread Nick Barcet
Following up on our last meeting and the comments collected on the
mailing list, here is a second version of the proposal for centrally
hosting configuration of meters in ceilometer.

The main idea is that all meters of a given type should be sending
similarly formatted information in order for the information to be
usable, hence the need to ensure that configuration info is centrally
stored and retrieved.  This would rule out, in my mind, the idea that we
could use the global flags object, as distribution of the configuration
file is left to the cloud implementor and does not lend for easy and
synchronized updates of agent config.  This proposal is by design
limited to the meter configuration, not to the agent configuration
itself which is still pushed using the global object flags.

Configuration format and content is left to the agent's implementation,
but it is assumed that each meter covered by an agent can be at least
enabled or disabled (and eventually, in a future version of the agents,
set to send information at a meter specific frequency - it is currently
a global value for all meters of an agent).

1/ Configuration is stored for each agent in flat file or DB as follow
+---
| Field  | Type | Note
+---
| meter_type | String   | Unique meter type
| conf_vers  | Integer  | Version of the configuration
| (simple incremental counter)
| config | Text | JSON Configuration info (defined by meter)
++--++

2/ Config is retreived via the messaging queue upon start of the agent

Request is sent by the agent for each meter upon start to retrieve to
current config version.

If a config is found, it is sent back to the agent, along with conf_vers.
Else the collector creates the record based on default value returned by
the agent/meter plugin installed on the collector, sets conf_vers to 1
and sends back a normal reply.

3/ When the content of the configuration is modified (conf_vers is
updated), a cast is sent to all agents publishing a given meter with the
content of the current configuration for immediate update.

Comments welcome...

Nick






signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [Metering] Meeting agenda for Thursday at 16:00 UTC (June 14th, 2012)

2012-06-13 Thread Nick Barcet
Hi,

The metering project team holds a weekly meeting in #openstack-meeting,
Thursdays at 1600 UTC. Everyone is welcome.

http://wiki.openstack.org/Meetings/MeteringAgenda

Topics for this week:

* Review last week's actions
  * dhellmann: submit plugin branch for review and merging
  * dhellmann rename plugin to engine for storage backend
ex-plugin-now-engine system
  * dhellmann: start mapping API queries to database engine methods
  * nijaba to propose a google spreadsheet calculator to estimate volume
of metering message (including nova, swift, cinder, quantum)

* Discuss and hopefully agree on meter configuration mechanism [1]

* Discuss proposing ceilometer as an incubated project

* Prepare documentation and framework for plugin contributors

* Establish communication with swift/quantum/cinder on best points of
interaction for our agents

* Status of the essex compatibility effort that jd___ is leading

[1]http://www.mail-archive.com/openstack@lists.launchpad.net/msg12859.html

Cheers,
--
Nick Barcet 
aka nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [metering] notification metadata

2012-06-13 Thread Nick Barcet
On 06/12/2012 05:51 PM, Doug Hellmann wrote:
> See https://review.stackforge.org/163 for the code.
> 
> On Tue, Jun 12, 2012 at 11:05 AM, Doug Hellmann
> mailto:doug.hellm...@dreamhost.com>> wrote:
> 
> I am working on bug 1006120, adding metadata to the instance-related
> metering events. It's easy to add location data like availability
> zone to the pollsters because the agent has an object from the
> database with all of the information about the instance (we will
> have to change that, but presumably we will be able to get the data
> via RPC, too). The notification plugins however, only have the data
> in the incoming message, and those messages don't include all of the
> data about the instance. My current plan is to have the collector
> code that processes notifications look up the instance in the
> database to get the rest of the data.
> 
> Does anyone have any thoughts on this plan?
> 
> https://bugs.launchpad.net/ceilometer/+bug/1006120

Well, this seems a bit inelegant to me to have collectors to additional
lookup in an external DB where otherwise it's role would be limited to
handle and store event.  It would definitely add quite a bit of strain
to the code on the central collector and what would happen if the nova
DB is not available or overloaded?

I'd rather have all collection of data done from the agents rather than
introduce this lookup on the central collector node.  What would prevent
us from doing it this way?

Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [metering] notification metadata

2012-06-13 Thread Nick Barcet
On 06/13/2012 01:55 PM, Doug Hellmann wrote:
> 
> 
> On Wed, Jun 13, 2012 at 6:56 AM, Nick Barcet  <mailto:nick.bar...@canonical.com>> wrote:
> 
> On 06/12/2012 05:51 PM, Doug Hellmann wrote:
> > See https://review.stackforge.org/163 for the code.
> >
> > On Tue, Jun 12, 2012 at 11:05 AM, Doug Hellmann
> > mailto:doug.hellm...@dreamhost.com>
> <mailto:doug.hellm...@dreamhost.com
> <mailto:doug.hellm...@dreamhost.com>>> wrote:
> >
> > I am working on bug 1006120, adding metadata to the
> instance-related
> > metering events. It's easy to add location data like availability
> > zone to the pollsters because the agent has an object from the
> > database with all of the information about the instance (we will
> > have to change that, but presumably we will be able to get the
> data
> > via RPC, too). The notification plugins however, only have the
> data
> > in the incoming message, and those messages don't include all
> of the
> > data about the instance. My current plan is to have the collector
> > code that processes notifications look up the instance in the
> > database to get the rest of the data.
> >
> > Does anyone have any thoughts on this plan?
> >
> > https://bugs.launchpad.net/ceilometer/+bug/1006120
> 
> Well, this seems a bit inelegant to me to have collectors to additional
> lookup in an external DB where otherwise it's role would be limited to
> handle and store event.  It would definitely add quite a bit of strain
> to the code on the central collector and what would happen if the nova
> DB is not available or overloaded?
> 
> 
> I want to eventually move off of direct DB access and ask for details
> like that via RPC, so this is just a temporary situation.
>  
> 
> 
> I'd rather have all collection of data done from the agents rather than
> introduce this lookup on the central collector node.  What would prevent
> us from doing it this way?
> 
> 
> That moves the problem even further away from the database. The agent
> runs on the compute node, and we *know* we won't have DB access there by
> the time we're done with Folsom (even nova is eliminating direct DB
> access by the compute node). The agent does not process notification
> events (it only polls) because of the round-tripping of messages through
> the message bus. Even if we move the notification handling into the
> agent, it doesn't have the necessary details so we would still need to
> ask for them somehow.

Yes, I see your point.  Bad idea.

> There are a couple of other alternatives:
> 
> 1. We could move notification handling into its own daemon to get it out
> of the collector. This new daemon would still run on a central service,
> and would need to be set up to support load balancing just as the
> collector is now. The similarities are why we left the two types of
> processing in the same process to begin with.

RPC seems like a better solution for this, doesn't it?

> 2. We could modify nova to include more details about an instance when
> it publishes a notification. This is the best solution from our
> perspective, and I would like to see all of the properties anyway, but I
> don't know if there is a performance-related reason for leaving out some
> details.

That would be definitely best.  Any nova dev willing to comment here?

Nick





signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] Meeting agenda for Thursday at 16:00 UTC (June 14th, 2012)

2012-06-14 Thread Nick Barcet
On 06/13/2012 12:39 PM, Nick Barcet wrote:
> Hi,
> 
> The metering project team holds a weekly meeting in #openstack-meeting,
> Thursdays at 1600 UTC. Everyone is welcome.
> 
> http://wiki.openstack.org/Meetings/MeteringAgenda
> 
> Topics for this week:
> 
> * Review last week's actions
>   * dhellmann: submit plugin branch for review and merging
>   * dhellmann rename plugin to engine for storage backend
> ex-plugin-now-engine system
>   * dhellmann: start mapping API queries to database engine methods
>   * nijaba to propose a google spreadsheet calculator to estimate volume
> of metering message (including nova, swift, cinder, quantum)
> 
> * Discuss and hopefully agree on meter configuration mechanism [1]
> 
> * Discuss proposing ceilometer as an incubated project
> 
> * Prepare documentation and framework for plugin contributors
> 
> * Establish communication with swift/quantum/cinder on best points of
> interaction for our agents
> 
> * Status of the essex compatibility effort that jd___ is leading
> 
> [1]http://www.mail-archive.com/openstack@lists.launchpad.net/msg12859.html

Here is the summary of the meeting that took place:
==
#openstack-meeting Meeting
==


Meeting started by nijaba at 16:00:06 UTC.  The full logs are available
at http://eavesdrop.openstack.org/meetings/openstack-meetin
/2012/openstack-meeting.2012-06-14-16.00.log.html


Meeting summary
---

* LINK: http://wiki.openstack.org/Meetings/MeteringAgenda  (nijaba,
  16:00:16)
* actions from previous meetings  (nijaba, 16:00:45)

* nijaba: to propose a google spreadsheet calculator to estimate volume
  of metering message (including nova, swift, cinder, quantum)  (nijaba,
  16:03:12)
  * LINK:

https://docs.google.com/spreadsheet/ccc?key=0AtziNGvs-uPudDhRbEJJOHFXV3d0ZGc1WE9NLTVPX0E
(nijaba, 16:03:29)
  * ACTION: nijaba to point to the calculator in the blueprint  (nijaba,
16:06:21)

* dhellmann: submit plugin branch for review and merging  (nijaba,
  16:06:36)
   * DONE

* dhellmann rename plugin to engine for storage backend
  ex-plugin-now-engine system  (nijaba, 16:07:03)
   * DONE

* dhellmann: start mapping API queries to database engine methods
  (nijaba, 16:07:22)
  * ACTION: dhellmann: start mapping API queries to database engine
methods  (nijaba, 16:08:01)

* Discuss and hopefully agree on meter configuration mechanism  (nijaba,
  16:08:21)
  * LINK:
http://www.mail-archive.com/openstack@lists.launchpad.net/msg12859.html
(nijaba, 16:08:35)
  * AGREED: push meter configuration to a future version  (nijaba,
16:21:49)

* Discuss proposing ceilometer as an incubated project  (nijaba,
  16:22:16)
  * AGREED: propose ceilometer as an incubated project  (nijaba,
16:30:13)
  * ACTION: nijaba to send a proposal to the mailing list  (nijaba,
16:30:29)

* Prepare documentation and framework for plugin contributors  (nijaba,
  16:30:45)
  * ACTION: dhellmann: look at existing plugins and pick one of each for
examples in docs  (dhellmann, 16:35:12)
  * ACTION: dhellmann: email info on examples to nijaba  (dhellmann,
16:36:04)
  * ACTION: nijaba to prime the doc once info received from dhellmann
(nijaba, 16:36:34)

* incubation  (nijaba, 16:37:31)
  * LINK: http://wiki.openstack.org/ProjectTypes  (nijaba, 16:37:45)
  * LINK: http://wiki.openstack.org/Governance/Approved/Incubation
(ttx, 16:37:54)
  * LINK:
http://wiki.openstack.org/Governance/Foundation/TechnicalCommittee
(ttx, 16:47:13)
  * ACTION: nijaba to prepare incubation application for review at the
next meeting  (nijaba, 16:48:02)

* Establish communication with swift/quantum/cinder on best points of
  interaction for our agents  (nijaba, 16:49:10)
  * ACTION: dachary talk to Dragon about SystemData / ceilometer and try
to create cooperation  (dachary, 16:52:50)
  * ACTION: dhellmann to talk to Quantum devs about integration with
ceilometer  (dhellmann, 16:53:26)
  * ACTION: dachary to talk to swift devs about integration with
ceilometer  (dachary, 16:54:47)
  * ACTION: nijaba to talk to cinder devs about integration with
ceilometer  (nijaba, 16:55:01)
  * ACTION: dachary talk to Dragon about SystemData / ceilometer and try
to create cooperation (i.e. nova)  (dachary, 16:55:45)
  * ACTION: jd___ to talk to glance devs about integration with
ceilometer  (nijaba, 17:00:03)

* Status of the essex compatibility effort that jd is leading  (nijaba,
  16:57:24)
   * In progress, to be continued next meeting.

Meeting ended at 17:00:51 UTC.

Action items, by person
---

* dachary
  * dachary talk to Dragon about SystemData / ceilometer and try to
create cooperation
  * dachary to talk to swift devs about integration with ceilometer
  * dachary talk to Dragon about SystemData / ceilometer and try to
create cooperation

[Openstack] [metering] Cinder usage data retrieval

2012-06-20 Thread Nick Barcet
Hello John (or anyone else working on cinder),

As part of the ceilometer project¹, we're working on usage data
retrieval from various OpenStack components. One of them is Cinder.
We're targeting Folsom for the first release, therefore it seems
important for both projects to be able to work together, this is why
we're bringing ceilometer to your attention and asking for advices. :)

What we want is to retrieve the maximum amount of data, so we can meter
things, to bill them in the end. For now and for Cinder, this would
first include (per user/tenant):
- the amount of reserved volume space
- the amount of used volume space
- the number of volumes
but we'll need probably more in a near future.

Do you have any advice regarding integration of Ceilometer and Cinder
together? What would be a stable interface we could rely on that would
be independent of the backend?

Thanks in advance,

Regards,

¹  http://launchpad.net/ceilometer
--
Nick Barcet 
aka: nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [metering] Cinder usage data retrieval

2012-06-20 Thread Nick Barcet
On 06/20/2012 07:26 PM, John Griffith wrote:
> On Wed, Jun 20, 2012 at 10:53 AM, Nick Barcet  
> wrote:
>> Hello John (or anyone else working on cinder),
>>
>> As part of the ceilometer project¹, we're working on usage data
>> retrieval from various OpenStack components. One of them is Cinder.
>> We're targeting Folsom for the first release, therefore it seems
>> important for both projects to be able to work together, this is why
>> we're bringing ceilometer to your attention and asking for advices. :)
>>
>> What we want is to retrieve the maximum amount of data, so we can meter
>> things, to bill them in the end. For now and for Cinder, this would
>> first include (per user/tenant):
>> - the amount of reserved volume space
>> - the amount of used volume space
>> - the number of volumes
>> but we'll need probably more in a near future.
>>
>> Do you have any advice regarding integration of Ceilometer and Cinder
>> together? What would be a stable interface we could rely on that would
>> be independent of the backend?
>>
>> Thanks in advance,
>>
>> Regards,
>>
>> ¹  http://launchpad.net/ceilometer
>> --
>> Nick Barcet 
>> aka: nijaba, nicolas
>>
> 
> Hi Nick,
> 
> We should chat about how things are shaping up so far and how you're
> implementing things on the other sides (consistency where
> practical/possible).  Also, it sort of depends on the architecture and
> use model details of Ceilometer, which I hate to admit but I'm not
> really up to speed on.
> 
> My first reaction/thought is the best most appropriate place to tie in
> is via the python-cinderclient.  There would be a number of ways to
> obtain some of this info, whether deriving it or maybe some extensions
> to obtain things directly.

Sounds great.  We have our weekly meeting on Thurdays at 4PM UTC [1]
which you would be welcome to join, or I can set up a bridge for us a
phone chat.  Let me know what you would prefer.

[1] http://wiki.openstack.org/Meetings/MeteringAgenda

Cheers,
Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [Metering] Meeting agenda for Thursday at 16:00 UTC (June 21st, 2012)

2012-06-20 Thread Nick Barcet
Hi,

The metering project team holds a weekly meeting in #openstack-meeting,
Thursdays at 1600 UTC. Everyone is welcome.

http://wiki.openstack.org/Meetings/MeteringAgenda

Topics for this week:

* Review last week's actions
  - nijaba: to point to the calculator in the blueprint
  - dhellmann: start mapping API queries to database engine methods
  - dhellmann: look at existing plugins and pick one of each for
examples in docs
  - dhellmann: email info on examples to nijaba
  - nijaba: to prime the doc once info received from dhellmann
  - nijaba: to prepare incubation application for review at the next meeting
  - dachary: talk to Dragon about SystemData / ceilometer and try to
create cooperation
  - dhellmann: to talk to Quantum devs about integration with ceilometer
  - dachary: to talk to swift devs about integration with ceilometer
  - nijaba: to talk to cinder devs about integration with ceilometer
  - dachary: talk to Dragon about SystemData / ceilometer and try to
create cooperation (i.e. nova)
  - jd_: to talk to glance devs about integration with ceilometer

* Status of the essex compatibility effort that jd is leading

Feel free to add other points you would like to discuss to the wiki page.

Cheers,
--
Nick Barcet 
aka nijaba, nicolas





signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Get vcpu hours per instance

2012-06-21 Thread Nick Barcet
On 06/15/2012 10:21 PM, Guillermo Alvarado wrote:
> Hi everyone!
> 
> I want to know the vcpu hours per instance, in horizon In nova
> dashboard I can see the total hours for the tenant, I want to see the
> hours for each instance, which together must be equal to  total. How can
>  I do that? 
> 
> I think the api does not have some method like that, What do you propose?

Hello Guillermo,

While it is not ready yet for consumption, this is part of the dataset
that we are targeting for the ceilometer project.  Feel free to join us
and help moving this effort forward.

https://launchpad.net/ceilometer

--
Nick Barcet 
aka: nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Metering] Meeting agenda for Thursday at 16:00 UTC (June 21st, 2012)

2012-06-21 Thread Nick Barcet
On 06/20/2012 08:55 PM, Nick Barcet wrote:
> Hi,
> 
> The metering project team holds a weekly meeting in #openstack-meeting,
> Thursdays at 1600 UTC. Everyone is welcome.
> 
> http://wiki.openstack.org/Meetings/MeteringAgenda
> 
> Topics for this week:
> 
> * Review last week's actions
>   - nijaba: to point to the calculator in the blueprint
>   - dhellmann: start mapping API queries to database engine methods
>   - dhellmann: look at existing plugins and pick one of each for
> examples in docs
>   - dhellmann: email info on examples to nijaba
>   - nijaba: to prime the doc once info received from dhellmann
>   - nijaba: to prepare incubation application for review at the next meeting
>   - dachary: talk to Dragon about SystemData / ceilometer and try to
> create cooperation
>   - dhellmann: to talk to Quantum devs about integration with ceilometer
>   - dachary: to talk to swift devs about integration with ceilometer
>   - nijaba: to talk to cinder devs about integration with ceilometer
>   - dachary: talk to Dragon about SystemData / ceilometer and try to
> create cooperation (i.e. nova)
>   - jd_: to talk to glance devs about integration with ceilometer
> 
> * Status of the essex compatibility effort that jd is leading
> 
> Feel free to add other points you would like to discuss to the wiki page.

The meeting took place and here is a summary:

==
#openstack-meeting: Ceilometer
==

Meeting summary
---

* LINK: http://wiki.openstack.org/Meetings/MeteringAgenda  (nijaba,
  16:00:03)
* dachary will unfortunately not be able to join us this week for work
  related reasons, he asked me to present his excuses.  (nijaba,
  16:00:23)
* actions from previous meetings  (nijaba, 16:01:04)

* nijaba: to point to the calculator in the blueprint  (nijaba,
  16:01:17)
  * LINK: http://wiki.openstack.org/EfficientMetering#Volume_of_data
(nijaba, 16:01:17)

* dhellmann: start mapping API queries to database engine methods
  (nijaba, 16:01:46)

* dhellmann: look at existing plugins and pick one of each for examples
  in docs and email info on examples to nijaba  (nijaba, 16:04:14)

* nijaba: to prime the doc once info received from dhellmann  (nijaba,
  16:09:09)
  * ACTION: nijaba: to prime the doc on plugins  (nijaba, 16:09:09)

* nijaba: to prepare incubation application for review at the next
  meeting  (nijaba, 16:10:27)
  * LINK:
http://wiki.openstack.org/Projects/IncubatorApplicationCeilometer
(nijaba, 16:10:27)
  * ACTION: during next week meeting we will vote on the Incubation
application  (nijaba, 16:12:44)

* dachary: talk to Dragon about SystemData / ceilometer and try to
  create cooperation  (nijaba, 16:13:08)
  * ACTION: dachary: talk to Dragon about SystemData / ceilometer and
try to create cooperation  (nijaba, 16:13:22)

* dhellmann: to talk to Quantum devs about integration with ceilometer
  (nijaba, 16:13:32)
  * ACTION: dhellmann: talk to Quantum devs about integration with
ceilometer  (dhellmann, 16:14:13)

* dachary: to talk to swift devs about integration with ceilometer
  (nijaba, 16:14:29)
  * ACTION: dachary: to talk to swift devs about integration with
ceilometer  (nijaba, 16:14:45)

* nijaba: to talk to cinder devs about integration with ceilometer
  (nijaba, 16:14:57)

* jd___: to talk to glance devs about integration with ceilometer
  (nijaba, 16:16:47)

* Status of the essex compatibility effort that jd is leading  (nijaba,
  16:21:42)
  * ACTION: jd___ check that openstack-ci runs the tests for Essex
(jd___, 16:27:00)
  * ACTION: jd___ add some tests on daemon launching  (jd___, 16:27:11)

* discussion about cinder integration  (nijaba, 16:27:49)
  * ACTION: nijaba to question best method to authenticate admin client
calls from ceilometer to cinder (and other projects)  (nijaba,
16:36:00)
  * ACTION: nijaba to include adding events to cinder to previous action
(nijaba, 16:43:37)
  * LINK: http://wiki.openstack.org/NovaVolumeMeetings  (nijaba,
16:46:01)

* open discussion  (nijaba, 16:47:55)
  * ACTION: dhellmann and nijba will be joining and nijaba will initiate
a private thread with jgriffith  (nijaba, 16:51:26)



Meeting ended at 16:52:14 UTC.



Action items, by person
---

* dhellmann
  * dhellmann: talk to Quantum devs about integration with ceilometer
  * dhellmann and nijaba will be joining and nijaba will initiate a
private thread with jgriffith
* dachary
  * dachary: talk to Dragon about SystemData / ceilometer and try to
create cooperation
  * dachary: to talk to swift devs about integration with ceilometer
* jd___
  * jd___ check that openstack-ci runs the tests for Essex
  * jd___ add some tests on daemon launching
* jgriffith
  * dhellmann and nijba will be joining and nijaba will initiate a
private thread with jgriffith
* nijab

Re: [Openstack] [metering] Cinder usage data retrieval

2012-06-25 Thread Nick Barcet
On 06/21/2012 12:44 PM, Thomas, Duncan wrote:
> John Griffith on 20 June 2012 18:26 wrote:
> 
> 
>> On Wed, Jun 20, 2012 at 10:53 AM, Nick Barcet
>>  wrote:
>>> What we want is to retrieve the maximum amount of data, so we can
>> meter
>>> things, to bill them in the end. For now and for Cinder, this would
>>> first include (per user/tenant):
>>> - the amount of reserved volume space
>>> - the amount of used volume space
>>> - the number of volumes
>>> but we'll need probably more in a near future.
> 
> 
>> We should chat about how things are shaping up so far and how you're
>> implementing things on the other sides (consistency where
>> practical/possible).  Also, it sort of depends on the architecture and
>> use model details of Ceilometer, which I hate to admit but I'm not
>> really up to speed on.
>>
>> My first reaction/thought is the best most appropriate place to tie in
>> is via the python-cinderclient.  There would be a number of ways to
>> obtain some of this info, whether deriving it or maybe some extensions
>> to obtain things directly.
> 
> One thing to watch for here is access control... Don't want one tenant 
> able to find out about another's usage. Probably not important on a private
> cloud deployment, but certainly important in the public cloud space. Having
> a separate endpoint to do this kind of admin stuff over also means you can
> have much tighter IP level access controls...

It would seem to me that the easiest way would be for calls that are
emitted from the same host as cinder manager on the local interface to
be handled as "admin" calls. We could then place Ceilometer's cinder
agent on the same host, which would not be allowed to any user, thus not
creating a breach of security.

Thoughts?

Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [Metering] First draft of plugin/agent documentation

2012-06-27 Thread Nick Barcet
I've just committed a first of the plugin/agent documentation to
stackforge/ceilometer.  Reviews welcome:

https://review.stackforge.org/#/c/250/

Thanks,
Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [metering] resource metadata changes and billing

2012-07-04 Thread Nick Barcet
On 06/29/2012 03:04 PM, Doug Hellmann wrote:
[..]
> My conclusion from all of this (over-)thinking is that the ceilometer
> API should assume the simple case and ignore the metadata changes when
> computing the sum or maximum value for a counter over a range of time.
> More complex processing will be left up to the caller, who can ask for
> raw metering data in manageable chunks and process them outside of the
> API. I could be persuaded to do something more complicated if the
> problems described above can be solved in a relatively simple way, but
> even then I think we should push that to the v2 API.
[..]

Sorry for my late reply on this, but...

So, if I summarize what you are saying, the problem is that for a given
Instance ID, a given meter may have to be interpreted as if the Instance
ID was changing over time.

Example:
t1: Instance A - Has 1 CPU - 64G ram - runs in zone 1
t2: Instance A - Has 2 CPU - 64G ram - runs in zone 1
t3: Instance A - Has 2 CPU - 128G ram - runs in zone 1
t4: Instance A - Has 2 CPU - 128G ram - runs in zone 3
t5: Instance A is stopped

From a billing point of view, what is important here is that even though
the Instance ID remains the same, we have in fact 4 different segments
of time which could lead to 4 different pricing being applied to the
same instance:

t1->t2: price 1
t2->t3: price 2
t3->t4: price 3
t4->t5: price 4

So we need to be able to inform the rating engine that these events have
occurred so that it does not uniformly apply a billing price to from a
sum of a given meter volume.  But in fact this information is indeed
captured and accessible to rating engines via their respective meters.

What is interesting here is that, in my mind, the sum and duration
function of the API, when I proposed it, were only meant to be able to:
 * In a simple amazon type billing model where instances cannot change
zone, add CPU or add ram,
 * In a Private cloud scenario where you only need simple usage stats to
inform your users,
 * in a horizon plugin to give a quick summary of use,
and would never be used by any serious rating engines that would in each
and every case require to have access to the raw list of events so that
it can recreate the full time line of the events.  This is where we need
to draw the line between metering and rating.

I therefore propose that we leave the API as is, knowing the side
effects of such high level sum and duration calculations. If we agree on
this, I take the action to document the limitation of the summary
functions of the API.

Nick






signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [metering] Meeting agenda for Thursday at 16:00 UTC (July 5th, 2012)

2012-07-04 Thread Nick Barcet
Hi,

The metering project team holds a meeting in #openstack-meeting,
Thursdays at 1600 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.

Everyone is welcome.

Agenda:
http://wiki.openstack.org/Meetings/MeteringAgenda

  * Review last week's actions (there does not seem to be any)

  * Discuss and vote the incubation application proposal to be submitted
to PPB
http://wiki.openstack.org/Projects/IncubatorApplicationCeilometer

  * PTL election process discussion
When we had our first project meeting on April 26th, it was agreed
that Loic Dachary and I would co-lead this project for the first 3
momths.  Time is soon up for those 3 months, so we should discuss
what to do to.  Some reference:
-
http://wiki.openstack.org/Governance/Model#Governance%20for%20the%20Individual%20OpenStack%20Projects
- http://www.cs.cornell.edu/andru/civs.html
- http://www.opavote.org/

  * Discuss project's next priorities

  * Open discussion

If you are not able to attend or have additional topic you would like to
cover, please update the agenda on the wiki.

Cheers,
--
Nick Barcet 
aka: nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [metering] resource metadata changes and billing

2012-07-04 Thread Nick Barcet
On 07/04/2012 10:55 PM, Julien Danjou wrote:
> On Wed, Jul 04 2012, Nick Barcet wrote:
> 
>> I therefore propose that we leave the API as is, knowing the side
>> effects of such high level sum and duration calculations. If we agree on
>> this, I take the action to document the limitation of the summary
>> functions of the API.
> 
> So, if I understand you correctly, that would mean the API is non-usable
> for anybody wanting to do fine-grained billing and they would have to
> connect to the database engine directly?
> Because if that's the case I think we miss an abstraction layer
> somewhere.

No, this is not what I am saying. The API offers two types of calls:
 * One which gives access to organized raw events, which rating engines
are likely to be using,
 * One which gives access to summary data, but which are only valid for
simple use cases.
Hopefully, there are no cases here that should justify connecting to the
db engine directly unless one really wants to, and that should not be
recommended as we may want to change our storage model over time without
breaking existing integrations.

The possible variants for describing possible billing strategies are
indeed one abstraction level above this. The telco industry calls the
tool handling this a rating engine (what transforms metering into line
items of a bill) and I intentionally proposed since the beginning that
Ceilometer should not be a rating engine.

We can hope that our current API will be able to evolve to simplify
rating engines requests over time, and we should be ready to accept
extension of it to do so. However I do think we do not have the
necessary experience with rating to propose a universal solution that
will effectively work at this time, and therefore am proposing that we
postpone that until we do have the experience.

Makes sense?
--
Nick Barcet 
aka: nijaba, nicolas





signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [metering] Meeting agenda for Thursday at 16:00 UTC (July 5th, 2012)

2012-07-05 Thread Nick Barcet
On 07/04/2012 08:16 PM, Nick Barcet wrote:
> Hi,
> 
> The metering project team holds a meeting in #openstack-meeting,
> Thursdays at 1600 UTC
> <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.
> 
> Everyone is welcome.
> 
> Agenda:
> http://wiki.openstack.org/Meetings/MeteringAgenda
> 
>   * Review last week's actions (there does not seem to be any)
> 
>   * Discuss and vote the incubation application proposal to be submitted
> to PPB
> http://wiki.openstack.org/Projects/IncubatorApplicationCeilometer
> 
>   * PTL election process discussion
> When we had our first project meeting on April 26th, it was agreed
> that Loic Dachary and I would co-lead this project for the first 3
> momths.  Time is soon up for those 3 months, so we should discuss
> what to do to.  Some reference:
> -
> http://wiki.openstack.org/Governance/Model#Governance%20for%20the%20Individual%20OpenStack%20Projects
> - http://www.cs.cornell.edu/andru/civs.html
> - http://www.opavote.org/
> 
>   * Discuss project's next priorities
> 
>   * Open discussion
> 
> If you are not able to attend or have additional topic you would like to
> cover, please update the agenda on the wiki.
> 
> Cheers,

The meeting just occurred and here is the summary:
==
#openstack-meeting: Ceilometer
==


Meeting started by nijaba at 16:00:13 UTC.  The full logs are available
at
http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-05-16.00.log.html


Meeting summary
---

* LINK: http://wiki.openstack.org/Meetings/MeteringAgenda  (nijaba,
  16:00:13)
* jd__ will unfortunately not be able to join us this week  (nijaba,
  16:00:13)
* actions from previous meetings  (nijaba, 16:01:27)

* Discuss and vote the incubation application proposal to be submitted
  to PPB  (nijaba, 16:02:19)
  * LINK:
http://wiki.openstack.org/Projects/IncubatorApplicationCeilometer
(nijaba, 16:02:19)
  * VOTE: Voted on "Submit the incubation application to the PPB?"
Results are, Yes: 4  (nijaba, 16:05:14)
  * ACTION: nijaba to send an email to the PPB  (nijaba, 16:05:48)

* PTL election process discussion  (nijaba, 16:06:07)
  * LINK:

http://wiki.openstack.org/Governance/Model#Governance%20for%20the%20Individual%20OpenStack%20Projects
(nijaba, 16:06:35)
  * LINK: http://www.cs.cornell.edu/andru/civs.html  (nijaba, 16:06:35)
  * LINK: http://www.opavote.org/  (nijaba, 16:06:35)
  * LINK:
http://wiki.openstack.org/Governance/Foundation/TechnicalCommittee
(nijaba, 16:06:35)
  * VOTE: Voted on "go ahead with the proposed PTL election process?"
Results are, Yes: 4  (nijaba, 16:12:04)
  * ACTION: nijaba to send call for candidate to the general mailing
list  (nijaba, 16:13:43)
  * ACTION: jd___ to setup opa voting system to start on 26th and end on
Aug 3rd  (nijaba, 16:14:15)

* Discuss project's next priorities  (nijaba, 16:14:36)
  * ACTION: nijaba to prime a roadmap page and invite others to populate
it  (nijaba, 16:19:46)
  * ACTION: jd___ handle counter/meter type for real  (jd___, 16:22:13)

* Open Discusssion  (nijaba, 16:24:05)
  * LINK: http://openstack.markmail.org/thread/fwkagzdzpleeclj5
(nijaba, 16:24:16)
  * ACTION: nijaba to document external API use with a clear warning
about the limitation of the sum and duration function  (nijaba,
16:25:55)



Meeting ended at 16:27:40 UTC.



Action items, by person
---

* jd___
  * jd___ to setup opa voting system to start on 26th and end on Aug 3rd
  * jd___ handle counter/meter type for real
* nijaba
  * nijaba to send an email to the PPB
  * nijaba to send call for candidate to the general mailing list
  * nijaba to prime a roadmap page and invite others to populate it
  * nijaba to document external API use with a clear warning about the
limitation of the sum and duration function



People present (lines said)
---

* nijaba (90)
* dhellmann (33)
* jd___ (20)
* flacoste (16)
* openstack (15)



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Time for a UK Openstack User Group meeting ?

2012-07-05 Thread Nick Barcet
On 07/04/2012 05:38 PM, Day, Phil wrote:
> Hi All,
> 
> I’m thinking it’s about time we had an OpenStack User Group meeting in
> the UK , and would be interested in hearing from anyone interested in
> attending, presenting, helping to organise, etc.
> 
> London would seem the obvious choice, but we could also host here in HP
> Bristol if that works for people.
> 
> Reply here or e-mail me directly (phil@hp.com), and if there’s
> enough interest I’ll pull something together.
> 

Canonical would be delighted to host that meeting in our new london
office.  We have capacity for up to 100 people and are currently
checking availability, but are looking for a date in the last week july.
 We have started the OpenStack-London meetup [1] so anyone interested
can subscribe.

[1] http://www.meetup.com/Openstack-London/

Cheers,
Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] Ceilometer application for incubation

2012-07-05 Thread Nick Barcet
Dear members of the Project Policy Board,

After 3 month of work on the Ceilometer project, and great progress
being made, our last meeting IRC meeting [1] validated that we should be
submitting this project for incubation.  Following the OpenStack project
rules, we have completed the incubation application form which you will
find at [2].

We would be happy to answer any question you may have about our
application and hope that you will give a favorable answer to our request.

[1]
http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-05-16.00.html
[2] http://wiki.openstack.org/Governance/Proposed/Ceilometer

On behalf of the Ceilometer project team,
--
Nick Barcet 
aka: nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Ceilometer application for incubation

2012-07-06 Thread Nick Barcet
On 07/05/2012 09:22 PM, Jonathan Bryce wrote:
> Thanks, Nick. I've added it to the agenda for next Tuesday's meeting
> at 20:00 UTC/3:00 PM CDT. If you can join the meeting to field any
> questions, that would be helpful.
> 
> Jonathan

Great! I'll be there, hopefully joined by other team members.

Thanks,
Nick

> On Jul 5, 2012, at 12:52 PM, Nick Barcet wrote:
> 
>> Dear members of the Project Policy Board,
>> 
>> After 3 month of work on the Ceilometer project, and great
>> progress being made, our last meeting IRC meeting [1] validated
>> that we should be submitting this project for incubation.
>> Following the OpenStack project rules, we have completed the
>> incubation application form which you will find at [2].
>> 
>> We would be happy to answer any question you may have about our 
>> application and hope that you will give a favorable answer to our
>> request.
>> 
>> [1] 
>> http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-05-16.00.html
>>
>> 
[2] http://wiki.openstack.org/Governance/Proposed/Ceilometer
>> 
>> On behalf of the Ceilometer project team, -- Nick Barcet
>>  aka: nijaba, nicolas
>> 
>> ___ Mailing list:
>> https://launchpad.net/~openstack Post to :
>> openstack@lists.launchpad.net Unsubscribe :
>> https://launchpad.net/~openstack More help   :
>> https://help.launchpad.net/ListHelp
> 
> 
> ___ Mailing list:
> https://launchpad.net/~openstack Post to :
> openstack@lists.launchpad.net Unsubscribe :
> https://launchpad.net/~openstack More help   :
> https://help.launchpad.net/ListHelp
> 





signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Bandwidth limit

2012-07-10 Thread Nick Barcet
On 07/10/2012 03:55 AM, Guillermo Alvarado wrote:
> Hi Everyone!
> 
> I want to know if there is a table or something that contains
> information about the bandwidth of each tenant, to billing purposes.

This is not complete yet, but is part of the data that we are trying to
collect [1] in the Ceilometer project [2].  We now have a nice guide
explaining how to contribute additional plugins [3], help is alsways
welcome.

[1] http://wiki.openstack.org/EfficientMetering#Meters
[2] https://launchpad.net/ceilometer
[3] http://wiki.openstack.org/EfficientMetering#Contributing_to_Ceilometer

> There are a way to limit the bandwith of the tenants??

This would be a request for Quantum, which I don't think has been fully
addressed yet, based on this thread [4].

[4] http://www.mail-archive.com/openstack@lists.launchpad.net/msg13611.html

Hope this helps,
--
Nick Barcet 
aka: nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Management tools survey

2012-07-11 Thread Nick Barcet
On 07/11/2012 05:18 AM, Nick Lothian wrote:
> Hi,
> 
> I'm trying to understand how people are doing management of servers and
> storage across multiple clouds (or perhaps it is only me that has this
> problem!).
> 
> I've created a short survey I'd appreciate any responses on:
> http://www.surveymonkey.com/s/8PJCK9H
> 
> Responses via email are fine too!

Hello Nick,

I am sure there are others, like me, interested in your findings in this
area.  Will you share the results of the survey?

Thanks,
Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [metering] Ceilometer PTL Election Process - Call for candidate

2012-07-11 Thread Nick Barcet
As voted on the July 5th meeting the Ceilometer team members will soon
vote on electing it's project team lead (PTL). Even though Ceilometer is
not yet an official OpenStack project, we are applying for it, so we
should be following the standard process as much as possible. The
details of the process we will be following is described at [1].

This email is a formal call for candidates, which is step one of the
process.  Candidates for the PTL role must have contributed either to
the source or to the wiki pages for Ceilometer prior to the call for
candidates. The list that we assembled can be found on the above
mentioned wiki page, but in the event we have missed someone, please shout.

Candidates should, before July 24th:
1/ declare themselves on this mailing list
2/ add their name on the wiki page

[1] http://wiki.openstack.org/EfficientMetering/PTLElectionProcess

Cheers,
--
Nick Barcet 
aka: nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [metering] Meeting agenda for Thursday at 16:00 UTC (July 12th, 2012)

2012-07-11 Thread Nick Barcet
Hi,

The metering project team holds a meeting in #openstack-meeting,
Thursdays at 1600 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.

Everyone is welcome.

Agenda:
http://wiki.openstack.org/Meetings/MeteringAgenda

 * Review last week's actions
   - nijaba to send an email to the PPB for Incabation application
   - nijaba to send call for candidate to the general mailing list
   - jd to setup opa voting system to start on 26th and end on Aug 3rd
   - nijaba to prime a roadmap page and invite others to populate it
   - jd handle counter/meter type for real
   - nijaba to document external API use with a clear warning about the
 limitation of the sum and duration function

 * Discuss and define actions from the PPB discussion on Tue regarding
   our incubation.
   -
http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-10-20.01.html
   -
http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-10-20.01.log.html

 * Open discussion

If you are not able to attend or have additional topic you would like to
cover, please update the agenda on the wiki.

Cheers,
--
Nick Barcet 
aka: nijaba, nicolas





signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Time for a UK Openstack User Group meeting ?

2012-07-12 Thread Nick Barcet
On 07/05/2012 07:02 PM, Nick Barcet wrote:
> On 07/04/2012 05:38 PM, Day, Phil wrote:
>> Hi All,
>>
>> I’m thinking it’s about time we had an OpenStack User Group meeting in
>> the UK , and would be interested in hearing from anyone interested in
>> attending, presenting, helping to organise, etc.
>>
>> London would seem the obvious choice, but we could also host here in HP
>> Bristol if that works for people.
>>
>> Reply here or e-mail me directly (phil@hp.com), and if there’s
>> enough interest I’ll pull something together.
>>
> 
> Canonical would be delighted to host that meeting in our new london
> office.  We have capacity for up to 100 people and are currently
> checking availability, but are looking for a date in the last week july.
>  We have started the OpenStack-London meetup [1] so anyone interested
> can subscribe.
> 
> [1] http://www.meetup.com/Openstack-London/

The room has now been booked:

http://www.meetup.com/Openstack-London/events/55354582/

When: Wednesday, July 25, 2012 6:30 PM

Where: IPC Media Building (Bluefin)
110 Southwark Street
SE1 0SU

RSVP limit: 100 "Yes" RSVPs

Cheers,
--
Nick Barcet 
aka: nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [metering] Meeting agenda for Thursday at 16:00 UTC (July 12th, 2012)

2012-07-12 Thread Nick Barcet
On 07/11/2012 02:34 PM, Nick Barcet wrote:
> Hi,
> 
> The metering project team holds a meeting in #openstack-meeting,
> Thursdays at 1600 UTC
> <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.
> 
> Everyone is welcome.
> 
> Agenda:
> http://wiki.openstack.org/Meetings/MeteringAgenda
> 
>  * Review last week's actions
>- nijaba to send an email to the PPB for Incabation application
>- nijaba to send call for candidate to the general mailing list
>- jd to setup opa voting system to start on 26th and end on Aug 3rd
>- nijaba to prime a roadmap page and invite others to populate it
>- jd handle counter/meter type for real
>- nijaba to document external API use with a clear warning about the
>  limitation of the sum and duration function
> 
>  * Discuss and define actions from the PPB discussion on Tue regarding
>our incubation.
>-
> http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-10-20.01.html
>-
> http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-10-20.01.log.html
> 
>  * Open discussion
> 
> If you are not able to attend or have additional topic you would like to
> cover, please update the agenda on the wiki.
> 
> Cheers,

The meeting took place, here is the summary:

==
#openstack-meeting: Ceilometer
==


Meeting started by nijaba at 16:00:01 UTC.  The full logs are available
at
http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-12-16.00.log.html


Meeting summary
---

* LINK: http://wiki.openstack.org/Meetings/MeteringAgenda  (nijaba,
  16:00:01)
* actions from previous meeting  (nijaba, 16:02:08)

* nijaba to send an email to the PPB for Incabation application
  (nijaba, 16:02:20)

* nijaba to send call for candidate to the general mailing list
  (nijaba, 16:02:31)

* jd__ to setup opa voting system to start on 26th and end on Aug 3rd
  (nijaba, 16:04:07)
  * ACTION: jd to setup opa voting system to start on 26th and end on
Aug 3rd  (nijaba, 16:04:27)

* nijaba to prime a roadmap page and invite others to populate it
  (nijaba, 16:04:40)
  * LINK: http://wiki.openstack.org/EfficientMetering/RoadMap  (nijaba,
16:04:40)
  * ACTION: nijaba to post roadmap to mailing list, askingfor feeback
and volunteers?  (nijaba, 16:05:48)

* jd__ handle counter/meter type for real  (nijaba, 16:06:12)
  * ACTION: jd___ adding the type of meters in ceilometer meter code
(nijaba, 16:07:57)

* nijaba to document external API use with a clear warning about the
  limitation of the sum and duration function  (nijaba, 16:08:13)
  * LINK:
http://wiki.openstack.org/EfficientMetering/APIProposalv1#Limitations
(nijaba, 16:08:13)

* Discuss and define actions from the PPB discussion on Tue regarding
  our incubation.  (nijaba, 16:10:13)
  * LINK:

http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-10-20.01.html
(nijaba, 16:10:13)
  * LINK:

http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-10-20.01.log.html
(nijaba, 16:10:13)
  * ACTION: nijaba to add a table showing the state of integration for
each OpenStack project  (nijaba, 16:13:53)
  * ACTION: nijaba to adjust the proposal to reflect a longer incubation
period  (nijaba, 16:15:45)
  * ACTION: dhellmann to get some feedback now about the sorts of meters
users want from the mailing list  (nijaba, 16:19:57)
  * ACTION: dhellmann to open a bug and work on devstack integration
(nijaba, 16:21:39)

* Open Discusssion  (nijaba, 16:26:42)
  * ACTION: dhellmann create a diagram of ceilometer architecture
(dhellmann, 16:35:35)
  * ACTION: dhellmann write a walk-through of setting up ceilometer and
collecting data  (dhellmann, 16:36:02)



Meeting ended at 16:47:27 UTC.



Action items, by person
---

* dhellmann
  * dhellmann to get some feedback now about the sorts of meters users
want from the mailing list
  * dhellmann to open a bug and work on devstack integration
  * dhellmann create a diagram of ceilometer architecture
  * dhellmann write a walk-through of setting up ceilometer and
collecting data
* jd___
  * jd___ adding the type of meters in ceilometer meter code
* nijaba
  * nijaba to post roadmap to mailing list, askingfor feeback and
volunteers?
  * nijaba to add a table showing the state of integration for each
OpenStack project
  * nijaba to adjust the proposal to reflect a longer incubation period




People present (lines said)
---

* nijaba (114)
* dhellmann (65)
* jd___ (34)
* flacoste (13)
* gmb (9)
* heckj (8)
* DanD (8)
* openstack (3)
* uvirtbot` (1)



signature.asc
Description: OpenPGP digital signature

[Openstack] [metering] General request for feedback on Ceilometer's road-map

2012-07-13 Thread Nick Barcet
Hello everyone,

If you have been following this list, you might have seen that the
Ceilometer project, while not yet fully functional, has made some great
progress over the past three month.  The next summit being just a few
months away, we thought it would be time to start collecting general
feedback on what we should be working on next and how to prioritize this.

In order to prime your feedback, we prepared a road-map wiki page [1].
Have a look at it and let us know if you think we should be adding other
elements to it.

When doing so, please remember that the target of Ceilometer is to
collect the metering information for OpenStack components in a single
place, not to build a full rating or billing engine.  Billing or rating
tools should connect to Ceilometer to get their information they need,
so feedback on this aspect is welcome (what additional information
should we collect? could some evolution of our API[2] make the life of
billing or rating engine simpler?)

You will certainly notice that we are also tracking, in the same page,
some features as bug requests and some of them are targeted for
folsom.3: this is what we think needs urgent work _right now_, so if you
think you are in a good position to contribute a little code to
Ceilometer, feel free to assign yourself to the bug and start
contributing patches.

If you wonder how to contribute to Ceilometer, we also have put together
a little documentation which you can read at [3].

[1] http://wiki.openstack.org/EfficientMetering/RoadMap
[2] http://wiki.openstack.org/EfficientMetering/APIProposalv1
[3] http://ceilometer.readthedocs.org

Thanks for your help and support,
--
Nick Barcet 
aka: nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [metering] Ceilometer PTL Election Process - Call for candidate

2012-07-13 Thread Nick Barcet
On 07/11/2012 02:24 PM, Nick Barcet wrote:
> Candidates should, before July 24th:
> 1/ declare themselves on this mailing list

It has been so much fun and so rewarding to kick-start this project,
that I can't resist to propose myself as Candidate.  Not being a true
developer places me in an awkward position to run for this function, but
I think I have shown, as other have done before, that coding is not the
only talent one can contribute and lead with.

> 2/ add their name on the wiki page
> 
> [1] http://wiki.openstack.org/EfficientMetering/PTLElectionProcess

Done, and added a few more words at [1] following the steps of the
previous 2 candidates, which are both exceptionally skilled to run this
project too. It's absolutely a wonderful feeling that I am not the only
one caring enough about this project. Truly, I think this project would
be nowhere if we had not met.  The road ahead is still quite steep, and
we'll all have lots to do, regardless of whom leads next.

[1]
http://wiki.openstack.org/EfficientMetering/PTLElectionProcess/NickBarcet



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 9th, 2012)

2012-08-08 Thread Nick Barcet
Hi,

The metering project team holds a meeting in #openstack-meeting,
Thursdays at 1600 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.

Everyone is welcome.

Agenda:
http://wiki.openstack.org/Meetings/MeteringAgenda

 * Review last week's actions
   - jaypipes to create ceilometer cookbook
   - jd_ to publish results of PTL election on general ml sometimes tomorrow
   - jtran to open a ticket for the DB access work
   - nijaba create a diagram of Ceilometer architecture

 * Discuss Doug's API change proposal

 * Discuss priority of maintaining Essex support and find contributor to
work on it if we are going to do it

 * Discuss integration with Heat

 * Open discussion

If you are not able to attend or have additional topic you would like to
cover, please update the agenda on the wiki.

Cheers,
--
Nick Barcet 
aka: nijaba, nicolas







signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 9th, 2012)

2012-08-10 Thread Nick Barcet
On 08/08/2012 05:54 PM, Nick Barcet wrote:
> Hi,
> 
> The metering project team holds a meeting in #openstack-meeting,
> Thursdays at 1600 UTC
> <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.
> 
> Everyone is welcome.
> 
> Agenda:
> http://wiki.openstack.org/Meetings/MeteringAgenda
> 
>  * Review last week's actions
>- jaypipes to create ceilometer cookbook
>- jd_ to publish results of PTL election on general ml sometimes tomorrow
>- jtran to open a ticket for the DB access work
>- nijaba create a diagram of Ceilometer architecture
> 
>  * Discuss Doug's API change proposal
> 
>  * Discuss priority of maintaining Essex support and find contributor to
> work on it if we are going to do it
> 
>  * Discuss integration with Heat
> 
>  * Open discussion
> 
> If you are not able to attend or have additional topic you would like to
> cover, please update the agenda on the wiki.

The meeting took place and here is the summary for it:

==
#openstack-meeting: Ceilometer
==


Meeting started by nijaba at 16:00:02 UTC.  The full logs are available
at
http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-08-09-16.00.log.html
.



Meeting summary
---

* LINK: http://wiki.openstack.org/Meetings/MeteringAgenda  (nijaba,
  16:00:02)
* actions from previous meeting  (nijaba, 16:01:42)

* jaypipes to create ceilometer cookbook  (nijaba, 16:01:56)
  * ACTION: jaypipes to create ceilometer cookbook  (nijaba, 16:03:23)

* jtran to open a ticket for the DB access work  (nijaba, 16:03:41)
  * LINK: https://bugs.launchpad.net/ceilometer/+bug/1034666  (nijaba,
16:03:50)

* nijaba to create a diagram of ceilometer architecture  (nijaba,
  16:04:03)
  * LINK:

https://docs.google.com/drawings/pub?id=1_cIFir6HS6jSkPw7chrmyu8DGE2ZgXk79Kbj8nw-Hqo&w=960&h=720
(nijaba, 16:04:13)
  * ACTION: nijaba to write description of componet responsibility
(nijaba, 16:06:58)

* Discuss dhellmann's API change proposal  (nijaba, 16:07:21)
  * LINK:

http://lists.openstack.org/pipermail/openstack-dev/2012-August/000389.html
(nijaba, 16:07:21)
  * AGREED: new api proposal from dhellmann provide a second check that
all case are covered  (nijaba, 16:11:42)
  * ACTION: nijaba to do a second thouroughness check on API and report
next week  (nijaba, 16:12:01)

* Discuss priority of maintaining Essex support and find contributor to
  work on it if we are going to do it  (nijaba, 16:12:18)
  * AGREED: essex support postponed until someone that cares about it
signals himself and want to work on it  (nijaba, 16:16:57)

* Discuss integration with Heat  (nijaba, 16:17:19)
  * AGREED: propose a joint session with heat at ODS regarding
cloudwatch  (nijaba, 16:26:53)

* Open Discusssion  (nijaba, 16:28:52)



Meeting ended at 16:40:44 UTC.



Action items, by person
---

* jaypipes
  * jaypipes to create ceilometer cookbook
* nijaba
  * nijaba to write description of componet responsibility
  * nijaba to do a second thouroughness check on API and report next
week



People present (lines said)
---

* nijaba (95)
* dhellmann (27)
* jd___ (24)
* ppetit (22)
* jaypipes (3)
* openstack (3)
* mrevell (2)
* uvirtbot (1)
* gmb (1)
* DanD (1)



Generated by `MeetBot`_ 0.1.4
--
Nick Barcet 
aka: nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 16th, 2012)

2012-08-16 Thread Nick Barcet
Hi,

The metering project team holds a meeting in #openstack-meeting,
Thursdays at 1600 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.

Everyone is welcome.

Agenda:
http://wiki.openstack.org/Meetings/MeteringAgenda

 * Review last week's actions
   - jaypipes to create ceilometer cookbook
   - nijaba to write description of componet responsibility
   - nijaba to do a second thouroughness check on API and report next week

 * Open discussion

If you are not able to attend or have additional topic you would like to
cover, please update the agenda on the wiki.

Cheers,
--
Nick Barcet 
aka: nijaba, nicolas









signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 16th, 2012)

2012-08-16 Thread Nick Barcet
On 08/16/2012 03:54 PM, Nick Barcet wrote:
> Hi,
> 
> The metering project team holds a meeting in #openstack-meeting,
> Thursdays at 1600 UTC
> <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.
> 
> Everyone is welcome.
> 
> Agenda:
> http://wiki.openstack.org/Meetings/MeteringAgenda
> 
>  * Review last week's actions
>- jaypipes to create ceilometer cookbook
>- nijaba to write description of componet responsibility
>- nijaba to do a second thouroughness check on API and report next week
> 
>  * Open discussion
> 
> If you are not able to attend or have additional topic you would like to
> cover, please update the agenda on the wiki.

The meeting took place and here are the minutes:

==
#openstack-meeting: Ceilometer
==


Meeting started by nijaba at 16:00:25 UTC.  The full logs are available
at
http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-08-16-16.00.log.html
.



Meeting summary
---

* LINK: http://wiki.openstack.org/Meetings/MeteringAgenda  (nijaba,
  16:00:25)
* actions from previous meeting  (nijaba, 16:01:43)

* jaypipes to create ceilometer cookbook  (nijaba, 16:01:58)
  * ACTION: jaypipes to create ceilometer cookbook  (nijaba, 16:02:38)

* nijaba to do a second thouroughness check on API and report next week
  (nijaba, 16:02:57)

* nijaba to write description of component responsibility  (nijaba,
  16:11:33)
  * ACTION: nijaba to write description of component responsibility
(nijaba, 16:11:43)

* Open Discusssion  (nijaba, 16:12:19)
  * ACTION: dhellmann and nijaba to work on sessions for summit via
email  (nijaba, 16:19:23)
  * ACTION: nijaba to give core reviewer rights to gmb  (nijaba,
16:21:55)
  * ACTION: dhellmann to ask jtrans about interest in reviewer status
(dhellmann, 16:23:28)



Meeting ended at 16:27:23 UTC.



Action items, by person
---

* dhellmann
  * dhellmann and nijaba to work on sessions for summit via email
  * dhellmann to ask jtrans about interest in reviewer status
* gmb
  * nijaba to give core reviewer rights to gmb
* nijaba
  * nijaba to write description of component responsibility
  * dhellmann and nijaba to work on sessions for summit via email
  * nijaba to give core reviewer rights to gmb
* jaypipes
  * jaypipes to create ceilometer cookbook



People present (lines said)
---

* nijaba (77)
* dhellmann (47)
* gmb (5)
* openstack (3)
* jgriffith (2)
* zul (2)



Generated by `MeetBot`_ 0.1.4



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 23rd, 2012)

2012-08-22 Thread Nick Barcet
Hi,

The metering project team holds a meeting in #openstack-meeting,
Thursdays at 1600 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.

Everyone is welcome.

Agenda:
http://wiki.openstack.org/Meetings/MeteringAgenda

 * Review last week's actions
   - jaypipes to create ceilometer cookbook
   - nijaba to write description of component responsibility
   - dhellmann and nijaba to work on sessions for summit via email
   - dhellmann to ask jtrans about interest in reviewer status
   - nijaba to give core reviewer rights to gmb
 * Open discussion

If you are not able to attend or have additional topic you would like to
cover, please update the agenda on the wiki.

Cheers,
--
Nick Barcet 
aka: nijaba, nicolas











signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 23rd, 2012)

2012-08-23 Thread Nick Barcet
On 08/23/2012 05:18 AM, Nick Barcet wrote:
> Hi,
> 
> The metering project team holds a meeting in #openstack-meeting,
> Thursdays at 1600 UTC
> <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.
> 
> Everyone is welcome.
> 
> Agenda:
> http://wiki.openstack.org/Meetings/MeteringAgenda
> 
>  * Review last week's actions
>- jaypipes to create ceilometer cookbook
>- nijaba to write description of component responsibility
>- dhellmann and nijaba to work on sessions for summit via email
>- dhellmann to ask jtrans about interest in reviewer status
>- nijaba to give core reviewer rights to gmb
>  * Open discussion
> 
> If you are not able to attend or have additional topic you would like to
> cover, please update the agenda on the wiki.

The meeting took place, here are the minutes:

==
#openstack-meeting: Ceilometer
==


Meeting started by nijaba at 16:00:33 UTC.  The full logs are available
at
http://eavesdrop.openstack.org/meetings/ceilometer/2012/ceilometer.2012-08-23-16.00.log.html
.



Meeting summary
---

* LINK: http://wiki.openstack.org/Meetings/MeteringAgenda  (nijaba,
  16:00:49)
* actions from previous meeting  (nijaba, 16:01:55)

* jaypipes to create ceilometer cookbook  (nijaba, 16:02:13)
  * ACTION: jaypipes to create ceilometer cookbook  (nijaba, 16:02:41)

* nijaba to write description of component responsibility  (nijaba,
  16:03:44)
  * ACTION: nijaba to link schema in the doc  (nijaba, 16:05:44)
  * LINK:

https://docs.google.com/drawings/pub?id=1_cIFir6HS6jSkPw7chrmyu8DGE2ZgXk79Kbj8nw-Hqo&w=960&h=720
(nijaba, 16:06:24)

* dhellmann and nijaba to work on sessions for summit via email
  (nijaba, 16:06:56)
  * ACTION: dhellmann and nijaba to work on sessions for summit via
email  (nijaba, 16:07:32)

* dhellmann to ask jtrans about interest in reviewer status  (nijaba,
  16:08:01)

* nijaba to give core reviewer rights to gmb  (nijaba, 16:08:52)

* Open Discussion  (nijaba, 16:11:33)
  * ACTION: nijaba to start a thread on meeting time  (nijaba, 16:34:26)



Meeting ended at 16:35:40 UTC.



Action items, by person
---

* dhellmann
  * dhellmann and nijaba to work on sessions for summit via email
* nijaba
  * nijaba to link schema in the doc
  * dhellmann and nijaba to work on sessions for summit via email
  * nijaba to start a thread on meeting time



People present (lines said)
---

* nijaba (68)
* dhellmann (53)
* openstack (4)
* gmb (3)
* heckj (1)
* _surya_ (1)


--
Nick Barcet 
aka: nijaba, nicolas




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [ceilometer] Weekly irc meetings day and time change?

2012-08-29 Thread Nick Barcet
Hello,

As we are observing that the current meeting time is not very convenient
for a few contributors, we would like to propose to change the date and
time to something that would suit a larger number.  I propose that we do
this in a two phase process:

1/ Each project member that care should send a list of days and times
that would be convenient for them as a reply to this message.  Please
try to avoid a day an time already used by another project as listed on [1]

2/ We then open a poll with a compiled list of proposals and get members
vote on it.

If you are not a contributor to the project, we are happy to receive
your suggestions too, but since this meeting is meant to coordinate
between contributors, the result of this poll will be heavily skewed
toward active members preferences.

[1] http://wiki.openstack.org/Meetings/

Thanks,
--
Nick Barcet 
aka: nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 29rd, 2012)

2012-08-29 Thread Nick Barcet
Hi,

The metering project team holds a meeting in #openstack-meeting,
Thursdays at 1600 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.

Everyone is welcome.

Agenda:
http://wiki.openstack.org/Meetings/MeteringAgenda

 * Review last week's actions
   - jaypipes to create ceilometer cookbook
   - nijaba to link schema in the doc
   - nijaba to start a thread on meeting time

 * Discuss session proposal for Grizzly summit

 * Open discussion

If you are not able to attend or have additional topic you would like to
cover, please update the agenda on the wiki.

Cheers,
--
Nick Barcet 
aka: nijaba, nicolas













signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Billing in Openstack

2012-08-30 Thread Nick Barcet
On 08/30/2012 03:40 AM, Emilien Macchi wrote:
> Hi,
> 
> You should read Ceilometer Project [1] and try it into DevStack directly
> in editing localrc before to run devstack and add :
> 
> enable_service ceilometer-acompute,ceilometer-acentral, ceilometer-collector
> 
> Enjoy !
> 
> 
> [1] http://wiki.openstack.org/EfficientMetering

If fully second Emilien's suggestions.  As a quick note, I'd like to
level set your expectation as Ceilometer is metering tool, not a full
billing tool.  A full billing solution needs 3 components:

- Metering: to know what has happened on the cloud
- Rating: which transforms what has happened into price to bill (line items)
- Billing: which accumulates line items and create a bill to send to
customer and collect money

Which means that you will still need component 2 and 3 to do billing
after you have deployed Ceilometer.

Hope this helps.
Nick


> On Thu, Aug 30, 2012 at 12:21 PM, Rajesh Avula  > wrote:
> 
> Greetings...!!
> 
> I would like to generate billing of used resources (CPU, RAM, Disk)
> in openstack. Below are the details of my setup (Please ask me for
> more details if require regarding setup)
> 
> Mainly I have used 2 system to make my setup, below are the details...
> 
> 1. *On First Laptop *(Dell - Latitude, with 4 GB RAM, Intel Core i5
> vpro Processor, 500 GB Hard disk) I have installed a *virtual
> machine with CentOS (6.2)* on *Xenserver (6.0.2)* and on that
> *virtual machine I installed openstack* by following below links and
> some other links from google
> 
> https://fedoraproject.org/wiki/Getting_started_with_OpenStack_Nova
> http://wiki.openstack.org/XenServer
> 
> 2. *On Second Laptop* (same as above configuration) I have installed
> *openfiler and made NFS and iSCSI targets* for glance, swift, and
> for instances.
> 
> 3. For network I am using BELKIN wireless adapter 5 port L2-switch)
> 
> I am able to launch Instances from AMI's, instances are getting IP
> addresses, able to create volumes and attaching to the instances.
> 
> All I need is, Is there any possibility in openstack where I can
> define the values / prices for the resources so that users will get
> the bills accordingly ?
> 
> Below are the packages installed in my setup for dashboard
> python-django-horizon-2012.1.1-1.el6.noarch
> openstack-dashboard-2012.1.1-1.el6.noarch
> 
> Is there any other packages should be installed to get billing
> option on dashboard?
> Any specific configuration required ?
> 
> Please let me know how to get billing information of the used /
> using resources.
> 
> 
> -- 
> Thanks & Regards,
> Rajesh Avula
> 09831138115
> 07259024701.
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> 
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 
> 
> 
> 
> -- 
> Emilien Macchi
> *System Engineer*
> **www.stackops.com* 
>  
> |*emilien.mac...@stackops.com  
> *|*skype:emilien.macchi*
> * 
> *
> 
> 
> 
> *
> 
>  ADVERTENCIA LEGAL  
> Le informamos, como destinatario de este mensaje, que el correo
> electrónico y las comunicaciones por medio de Internet no permiten
> asegurar ni garantizar la confidencialidad de los mensajes transmitidos,
> así como tampoco su integridad o su correcta recepción, por lo que
> STACKOPS TECHNOLOGIES S.L. no asume responsabilidad alguna por tales
> circunstancias. Si no consintiese en la utilización del correo
> electrónico o de las comunicaciones vía Internet le rogamos nos lo
> comunique y ponga en nuestro conocimiento de manera inmediata. Este
> mensaje va dirigido, de manera exclusiva, a su destinatario y contiene
> información confidencial y sujeta al secreto profesional, cuya
> divulgación no está permitida por la ley. En caso de haber recibido este
> mensaje por error, le rogamos que, de forma inmediata, nos lo comunique
> mediante correo electrónico remitido a nuestra atención y proceda a su
> eliminación, así como a la de cualquier documento adjunto al mismo.
> Asimismo, le comunicamos que la distribución, copia o utilización de
> este mensaje, o de cualquier documento adjunto al mismo, cualquiera que
> fuera su finalidad, están prohibidas por la ley. 
> 
> * PRIVILEGED AND CONFIDENTIAL  
> We hereby inform you, as addressee of this message, that e-mail and
> Internet do not guarantee the confidentiality, nor the completeness or
> proper reception of the messages sent and, thus, STACKOPS TECHNOLOGIES
> S.L. does not assume any liability for those circumstances. Should you
> not agree to the use

Re: [Openstack] [ceilometer] Weekly irc meetings day and time change?

2012-08-30 Thread Nick Barcet
On 08/30/2012 09:11 AM, Thomas Goirand wrote:
> On 08/30/2012 04:20 PM, Julien Danjou wrote:
>> On Thu, Aug 30 2012, Angus Salkeld wrote:
>>
>>> I'd like to attend but am in Australia. I am quite flexible so it might
>>> be easier to say what times don't suit.
>>> Basically midnight - 6am which in UTC is 14:00 to 20:00
>>
>> That's gonna be a tough one. :)
>>
>> In the same idea, living in France, I'd exclude UTC 22:00 to 04:00.
>>
>> So that let us with UTC 04:00 to 14:00 and 20:00 to 22:00
>>
>> Doug being in the US in IIRC UTC-7 that would exclude
>> UTC 07:00 to UTC 13:00.
>>
>> Finally letting us with UTC 04:00 to 06:00 and 20:00 to 22:00.
>>
>> So finally, I would propose to use UTC 21:00:
>> - 23:00 for me (and Nick most of the time I guess) in France
>> - 14:00 for Doug
>> - 7:00 for Angus
>>
>> or UTC 06:00:
>> - 08:00 for me (and Nick most of the time I guess) in France
>> - 23:00 for Doug
>> - 16:00 for Angus
>>
>> (Hope I did not get the math wrong)
>
> If it's the former, I wont ever attend, no way I can get up at 5am
> (Shanghai is GMT+8). This has always been at this time, and never, I can
> go online.
>
> The later is ok though.

I am about to open a poll to pick from 0600, 1200, 1500 and 2100 UTC on
Google.  Will reply to this thread when it opens.  We'll have to go with
what fits most.

> Thomas
>
> P.S: I'd like to contribute to the ceilometer project. I haven't yet
> (though I've done quite some work on the Debian packaging of the rest of
> Openstack), and I'd like to know in which area I could start implementing.

I would recommend that you take a look at the roadmap[1], and assign
yourself one of the bugs in there if you feel like it.  A good
recommended read to start is at [2].

[1] http://wiki.openstack.org/EfficientMetering/RoadMap
[2] http://ceilometer.readthedocs.org/

Hope this helps, and welcome to the ceilometer team!

Nick



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 30rd, 2012)

2012-08-30 Thread Nick Barcet
On 08/29/2012 08:29 PM, Nick Barcet wrote:
> Hi,
> 
> The metering project team holds a meeting in #openstack-meeting,
> Thursdays at 1600 UTC
> <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.
> 
> Everyone is welcome.
> 
> Agenda:
> http://wiki.openstack.org/Meetings/MeteringAgenda
> 
>  * Review last week's actions
>- jaypipes to create ceilometer cookbook
>- nijaba to link schema in the doc
>- nijaba to start a thread on meeting time
> 
>  * Discuss session proposal for Grizzly summit
> 
>  * Open discussion
> 
> If you are not able to attend or have additional topic you would like to
> cover, please update the agenda on the wiki.

The meeting took place today, the 30th, contrarily to what I wrongly
titled the previous email, and here is the summary:

==
#openstack-meeting: Ceilometer
==


Meeting started by nijaba at 16:01:12 UTC.  The full logs are available
at
http://eavesdrop.openstack.org/meetings/ceilometer/2012/ceilometer.2012-08-30-16.01.log.html
.



Meeting summary
---

* LINK: http://wiki.openstack.org/Meetings/MeteringAgenda  (nijaba,
  16:01:12)
* actions from previous meeting  (nijaba, 16:02:11)

* jaypipes to create ceilometer cookbook  (nijaba, 16:02:26)
  * ACTION: nijaba to open a bug for cookbook and assign to jaypipes
(nijaba, 16:03:42)

* nijaba to link schema in the doc  (nijaba, 16:03:53)
  * LINK: https://review.openstack.org/#/c/12171  (nijaba, 16:04:04)
  * LINK:

http://ceilometer.readthedocs.org/en/latest/architecture.html#high-level-description
(nijaba, 16:04:04)
  * ACTION: nijaba to add architecture image to project /doc rather than
link from google  (nijaba, 16:06:04)
  * ACTION: nijaba to include a comment in the rst file linking to the
google document where you build the image, so we can remember where
it is later  (nijaba, 16:06:46)

* nijaba to start a thread on meeting time  (nijaba, 16:09:41)

* Discuss sessions proposal for Grizzly summit  (nijaba, 16:10:12)
  * ACTION: dhellmann to move summit proposal outlines to the wiki and
email links to the dev mailing list  (dhellmann, 16:13:56)

* Discuss changing the meeting time  (nijaba, 16:16:08)
  * ACTION: nijaba to open a doodle poll to pick between viable options
(nijaba, 16:23:15)

* Open Discussion  (nijaba, 16:25:23)



Meeting ended at 16:35:25 UTC.



Action items, by person
---

* dhellmann
  * dhellmann to move summit proposal outlines to the wiki and email
links to the dev mailing list
* jaypipes
  * nijaba to open a bug for cookbook and assign to jaypipes
* nijaba
  * nijaba to open a bug for cookbook and assign to jaypipes
  * nijaba to add architecture image to project /doc rather than link
from google
  * nijaba to include a comment in the rst file linking to the google
document where you build the image, so we can remember where it is
later
  * nijaba to open a doodle poll to pick between viable options



People present (lines said)
---

* nijaba (78)
* dhellmann (35)
* spn (15)
* jd___ (8)
* oubiwann (6)
* jaypipes (4)
* openstack (4)
* gmb (1)
* DanD (1)

--
Nick Barcet 
aka: nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [ceilometer] Weekly irc meetings time change?

2012-08-31 Thread Nick Barcet
On 08/30/2012 09:49 AM, Nick Barcet wrote:
> I am about to open a poll to pick from 0600, 1200, 1500 and 2100 UTC on
> Google.  Will reply to this thread when it opens.  We'll have to go with
> what fits most.

The poll is now open at [1], please submit your preference(s) there.  I
will close the poll on tuesday.

http://doodle.com/srxx5244qg9pz2pm

Cheers,
Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [ceilometer] Weekly irc meetings time change?

2012-09-05 Thread Nick Barcet
On 09/05/2012 08:55 AM, SPN wrote:
> On 2012-08-31 21:34, Nick Barcet wrote:
>> On 08/30/2012 09:49 AM, Nick Barcet wrote:
>>> I am about to open a poll to pick from 0600, 1200, 1500 and 2100 UTC on
>>> Google.  Will reply to this thread when it opens.  We'll have to go with
>>> what fits most.
>>
>> The poll is now open at [1], please submit your preference(s) there.  I
>> will close the poll on tuesday.
>>
>> http://doodle.com/srxx5244qg9pz2pm
>>
>> Cheers,
>> Nick
> 
> 
> Is there a concenses on the timings?

Thanks for asking, I was just about to come up with this.  So based on
the poll, it seems that the 3-4pm UTC time slot received the most favors
with 9 yes, 1 "if need be" and 1 no.  So I guess we'll have to do
without Angus, unless we want to do alternating meetings to satisfy both
sides of the planet. In the later case we could do one week at 3pm UTC,
and the other at 9PM.  What would the others think about this?

In any case, the next meeting will be tomorrow at 3pm UTC.  I'll be
sending a formal invite later today.

Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [ceilometer] *NEW TIME* Metering meeting agenda for Thursday at 15:00 UTC (Sept 5th, 2012)

2012-09-05 Thread Nick Barcet
 PLEASE NOTE THE NEW MEETING TIME, ONE HOUR EARLIER 

The metering project team holds a meeting in #openstack-meeting,
Thursdays at 1500 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.

Everyone is welcome.

Agenda:
http://wiki.openstack.org/Meetings/MeteringAgenda

* Review last week's actions
  - dhellmann to move summit proposal outlines to the wiki and
email links to the dev mailing list
  - nijaba to open a bug for cookbook and assign to jaypipes
  - nijaba to add architecture image to project /doc rather than link
from google
  - nijaba to include a comment in the rst file linking to the google
document where you build the image, so we can remember where it is
later
* Discussion on session proposal for the summit
  http://wiki.openstack.org/EfficientMetering/GrizzlySummit
* Discussion on alternating meeting time to allow presence from down
  under
* Open discussion

If you are not able to attend or have additional topic you would like to
cover, please update the agenda on the wiki.

Cheers,
--
Nick Barcet 
aka: nijaba, nicolas















signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [ceilometer] *NEW TIME* Metering meeting agenda for Thursday at 15:00 UTC (Sept 5th, 2012)

2012-09-05 Thread Nick Barcet
On 09/05/2012 01:11 PM, surya_prabha...@dell.com wrote:
> 
> On 09/05/12 15:49, Nick Barcet wrote:
> 
>  PLEASE NOTE THE NEW MEETING TIME, ONE HOUR EARLIER 
> 
> The metering project team holds a meeting in #openstack-meeting,
> Thursdays at 1500 UTC
> <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0><http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.
> 
> 
> Isn't it 15.00 UTC?
> 
> http://www.timeanddate.com/worldclock/fixedtime.html?hour=15&min=0&sec=0

Absolutely, my bad for thinking I had updated the URL while I obviously
had not.

Thanks,
Nick






signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [ceilometer] *NEW TIME* Metering meeting agenda for Thursday at 15:00 UTC (Sept 5th, 2012)

2012-09-06 Thread Nick Barcet
On 09/05/2012 12:19 PM, Nick Barcet wrote:
>  PLEASE NOTE THE NEW MEETING TIME, ONE HOUR EARLIER 
> 
> The metering project team holds a meeting in #openstack-meeting,
> Thursdays at 1500 UTC
> <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.
> 
> Everyone is welcome.
> 
> Agenda:
> http://wiki.openstack.org/Meetings/MeteringAgenda
> 
> * Review last week's actions
>   - dhellmann to move summit proposal outlines to the wiki and
> email links to the dev mailing list
>   - nijaba to open a bug for cookbook and assign to jaypipes
>   - nijaba to add architecture image to project /doc rather than link
> from google
>   - nijaba to include a comment in the rst file linking to the google
> document where you build the image, so we can remember where it is
> later
> * Discussion on session proposal for the summit
>   http://wiki.openstack.org/EfficientMetering/GrizzlySummit
> * Discussion on alternating meeting time to allow presence from down
>   under
> * Open discussion
> 
> If you are not able to attend or have additional topic you would like to
> cover, please update the agenda on the wiki.

The meeting just took place and here is the summary:

==
#openstack-meeting: Ceilometer
==


Meeting started by nijaba at 15:00:36 UTC.  The full logs are available
at
http://eavesdrop.openstack.org/meetings/ceilometer/2012/ceilometer.2012-09-06-15.00.log.html
.



Meeting summary
---

* LINK: http://wiki.openstack.org/Meetings/MeteringAgenda  (nijaba,
  15:00:36)
* actions from previous meeting  (nijaba, 15:01:28)

* dhellmann to move summit proposal outlines to the wiki and email links
  to the dev mailing list  (nijaba, 15:01:49)

* nijaba to open a bug for cookbook and assign to jaypipes  (nijaba,
  15:02:08)

* nijaba to add architecture image to project /doc rather than link from
  google & to include a comment in the rst file linking to the google
  document where you build the image, so we can remember where it is
  later  (nijaba, 15:03:04)

* Discussion on session proposal for the summit  (nijaba, 15:04:04)
  * LINK: http://wiki.openstack.org/EfficientMetering/GrizzlySummit
(nijaba, 15:04:16)
  * ACTION: nijaba to see with ttx how our sessions can be shown on
official program  (nijaba, 15:14:04)

* Discussion on alternating meeting time to allow presence from down
  under  (nijaba, 15:14:16)
  * VOTE: Voted on "should we hold our meetings at alternating times on
even and odd weeks?" Results are, Yes: 6  (nijaba, 15:22:48)
  * ACTION: nijaba to update meeting page to note new alternating
meeting time  (nijaba, 15:23:58)

* Release management  (nijaba, 15:24:14)
  * ACTION: gmb to act as release manager  (nijaba, 15:27:32)
  * LINK: https://launchpad.net/~heut2008/+archive/ppa  (dachary,
15:30:15)
  * ACTION: gmb to a plan on the ml with fixed dates  (nijaba, 15:48:39)
  * VOTE: Voted on "use 0.1 as our first release?" Results are, yes: 5
(nijaba, 15:50:33)
  * VOTE: Voted on "let's see if voting is case sensitive?" Results are,
YeS: 1, nO: 4  (nijaba, 15:51:43)

* Open Discusssion  (nijaba, 15:52:44)
  * ACTION: dhellmann make sure flask is listed as a dependency of
ceilometer  (dhellmann, 15:56:26)



Meeting ended at 15:59:07 UTC.



Action items, by person
---

* dhellmann
  * dhellmann make sure flask is listed as a dependency of ceilometer
* gmb
  * gmb to act as release manager
  * gmb to a plan on the ml with fixed dates
* nijaba
  * nijaba to see with ttx how our sessions can be shown on official
program
  * nijaba to update meeting page to note new alternating meeting time
* ttx
  * nijaba to see with ttx how our sessions can be shown on official
program



People present (lines said)
---

* nijaba (121)
* dhellmann (61)
* zigo (31)
* spn (21)
* openstack (20)
* jd___ (18)
* gmb (14)
* dachary (13)
* eglynn__ (9)
* ttx (7)
* asalkeld (7)
* zul (4)
* uvirtbot (3)
* jaypipes (2)

--
Nick Barcet 
aka: nijaba, nicolas





signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [ceilometer] Weekly irc meetings time change?

2012-09-07 Thread Nick Barcet
On 09/05/2012 11:51 AM, Nick Barcet wrote:
> Thanks for asking, I was just about to come up with this.  So based on
> the poll, it seems that the 3-4pm UTC time slot received the most favors
> with 9 yes, 1 "if need be" and 1 no.  So I guess we'll have to do
> without Angus, unless we want to do alternating meetings to satisfy both
> sides of the planet. In the later case we could do one week at 3pm UTC,
> and the other at 9PM.  What would the others think about this?
> 
> In any case, the next meeting will be tomorrow at 3pm UTC.  I'll be
> sending a formal invite later today.

Based on yesterday's meeting, we decided to try alternating time. Since
9PM UTC is taken on thursdays, I am proposing to move it on Wednesdays.
 If Nobody objects, this means that starting next week we'll have our
meetings on:

Wednesday at 9PM UTC for odd weeks (September 12, 26)
Thursday at 3PM UTC for even weeks (September 20 and October 4)

Does this work for everyone?

Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [ceilometer] *ALT TIME* Metering meeting agenda for Wed at 21:00 UTC (Sept 12th, 2012)

2012-09-11 Thread Nick Barcet
 PLEASE NOTE THE ALTERNATIVE MEETING TIME WED 21:00 UTC 

The metering project team will hold its next meeting at alternate time
on *Wednesday* at 9PM UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?hour=21&min=0&sec=0>.

Everyone is welcome.

Agenda:
http://wiki.openstack.org/Meetings/MeteringAgenda

* Review last week's actions
  * nijaba to see with ttx how our sessions can be shown on official program
  * nijaba to update meeting page to note new alternating meeting time
  * gmb to a plan on the ml with fixed dates
  * dhellmann make sure flask is listed as a dependency of ceilometer
* Open discussion

If you are not able to attend or have additional topic you would like to
cover, please update the agenda on the wiki.

Cheers,
--
Nick Barcet 
aka: nijaba, nicolas

















signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [ceilometer] *ALT TIME* Metering meeting agenda for Wed at 21:00 UTC (Sept 12th, 2012)

2012-09-12 Thread Nick Barcet
On 09/11/2012 11:22 PM, Nick Barcet wrote:
>  PLEASE NOTE THE ALTERNATIVE MEETING TIME WED 21:00 UTC 
> 
> The metering project team will hold its next meeting at alternate time
> on *Wednesday* at 9PM UTC
> <http://www.timeanddate.com/worldclock/fixedtime.html?hour=21&min=0&sec=0>.
> 
> Everyone is welcome.
> 
> Agenda:
> http://wiki.openstack.org/Meetings/MeteringAgenda
> 
> * Review last week's actions
>   * nijaba to see with ttx how our sessions can be shown on official program
>   * nijaba to update meeting page to note new alternating meeting time
>   * gmb to a plan on the ml with fixed dates
>   * dhellmann make sure flask is listed as a dependency of ceilometer
> * Open discussion
> 
> If you are not able to attend or have additional topic you would like to
> cover, please update the agenda on the wiki.
> 
> Cheers,

The meeting took place, here is the summary:

==
#openstack-meeting: Ceilometer
==


Meeting started by nijaba at 21:00:17 UTC.  The full logs are available
at
http://eavesdrop.openstack.org/meetings/ceilometer/2012/ceilometer.2012-09-12-21.00.log.html
.



Meeting summary
---

* LINK: http://wiki.openstack.org/Meetings/MeteringAgenda  (nijaba,
  21:00:17)
* actions from previous meeting  (nijaba, 21:01:27)

* nijaba to see with ttx how our sessions can be shown on official
  program  (nijaba, 21:01:40)

* nijaba to update meeting page to note new alternating meeting time
  (nijaba, 21:03:50)
  * VOTE: Voted on "should we maintain alt meet time for the next
month?" Results are, yes: 1, bleh: 1, no: 2  (nijaba, 21:10:09)

* gmb to a plan on the ml with fixed dates  (nijaba, 21:10:36)
  * AGREED: 0.1 feature freeze will take effect on 2012-09-28  (nijaba,
21:12:15)
  * AGREED: 0.1 release date will be  2012-10-12  (nijaba, 21:12:15)

* dhellmann make sure flask is listed as a dependency of ceilometer
  (nijaba, 21:12:32)

* Open Discusssion  (nijaba, 21:13:29)
  * LINK: http://wiki.openstack.org/EfficientMetering/RoadMap  (nijaba,
21:22:54)
  * ACTION: dhellmann write up details of blueprint

https://blueprints.launchpad.net/ceilometer/+spec/config-driven-notification-monitoring
(dhellmann, 21:23:36)
  * LINK: http://teambox.com/n/8fb5404848e9f0b9/ceilometer  (nijaba,
21:25:13)
  * AGREED: next meeting thu sept 20 at 15UTC!  (nijaba, 21:40:54)



Meeting ended at 21:41:15 UTC.



Action items, by person
---

* dhellmann
  * dhellmann write up details of blueprint

https://blueprints.launchpad.net/ceilometer/+spec/config-driven-notification-monitoring



People present (lines said)
---

* nijaba (70)
* jtran (42)
* asalkeld (26)
* dhellmann (18)
* spn2 (17)
* jd___ (17)
* openstack (10)
* notmyname (2)
* uvirtbot (1)



Generated by `MeetBot`_ 0.1.4




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [ceilometer] Bare metal and Metering

2012-09-13 Thread Nick Barcet
On 09/13/2012 04:33 PM, Tim Bell wrote:
>  
> 
> Reading through the post from Mirantis on bare metal support on
> OpenStack
> (http://www.mirantis.com/blog/bare-metal-provisioning-with-openstack-cloud/),
> I was wondering how this would interact with the metering work in
> ceilometer.

So far, only libvirt driven hypervisors are covered.  For bare metal, or
other hypervisors, new pollsters would be needed. Contributions are
welcome :)

Nick




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] billing for openstack

2012-10-09 Thread Nick Barcet
On 10/09/2012 05:33 PM, Frans Thamura wrote:
> hi all
> 
> my member asks related to billing for openstack, so we can bill anyone
> use openstack's services
> 
> are one of this?  WHMCS, Clientexec, Blesta, Hostbill
> 
> or anyone can help?

Hello Frans,

We are just about to release a first version of Ceilometer[1][2] which
provides a central point for a billing implementation to fetch its usage
data from.

Not sure if that fully answers your question though...

[1]http://launchpad.net/ceilometer
[2]http://ceilometer.readthedocs.org

Best,
--
Nick Barcet 
aka: nijaba, nicolas




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [Announce][Ceilometer] Release 0.1 (Folsom)

2012-10-12 Thread Nick Barcet
Dear Fellow stackers,

We are pleased to announce today the release of ceilometer 0.1 (Folsom).
 As its version number implies, it is still a project in its infancy, as
it has not yet been incubated as an OpenStack project.  It should,
however, be ready to be used by the most adventurous ones as it offers a
fairly nice coverage of OpenStack, a workable database backend, and a
clean and sweet admin API to retrieve metering information from.

What is ceilometer?
==
The ceilometer project aims to deliver a unique point of contact for
billing systems to acquire all of the measurements they need to
establish customer billing, across all current OpenStack core components
with work underway to support future OpenStack components. We hope that,
once accepted for incubation, it’s official name will become OpenStack
Metering.

What does Ceilometer cover at the moment?
=
Ceilometer covers meters collected from Compute (nova), Network
(quantum), Image (glance), and Volume (cinder).  We’re obviously missing
Object (swift) in this first release, but we have this clearly on our
roadmap for our Grizzly release, and maybe even before depending on the
agility of our contributors.

Regarding Nova, most of the Nova meters will only work with libvirt
fronted hypervisors at the moment, and our test coverage was mostly done
on KVM. Contributors are of course welcome to implement meters for other
virtualization backends.

A detailed list of meters can be found at:
  http://ceilometer.readthedocs.org/en/latest/measurements.html

Detailed release notes are at:
  http://ceilometer.readthedocs.org/en/latest/releasenotes/folsom.html

Where is ceilometer’s release?
==
As for all other OpenStack project, it can be found on the tarballs
server, and more specifically at
  http://tarballs.openstack.org/ceilometer/
and we hope you will be able to find packages in most distributions soon.

Note that, even though we are not yet an official OpenStack project, the
infra team has been supporting us wonderfully and we’ve been using
OpenStack infrastructure happily since the beginning of this project.
Big thanks to the infra team for their help and wonderful setup.

Who has contributed to ceilometer so far?
==
Doug Hellmann, Julien Danjou, Eoghan Glynn, Loic Dachary, John Tran,
Steven Berler, Jay Pipes, Graham Bins, Francis Lacoste, Thierry Carrez,
Anne Gentle, Surya Prabhakar, Nick Barcet and the OpenStack
Infrastructure team have provided help in making this release possible
(we hope we did not miss anyone here...).

Why doesn’t ceilometer do this?
===
We would enjoy hearing about the missing feature that we either never
thought about or haven’t had time to write yet.  You can tell us about
it on the mailing list (see below) or even better, if you are coming to
the OpenStack Developer summit, join us in one of our technical workshop
sessions at the summit on Monday morning:

Monday 9:50am - State of Metering

http://openstacksummitfall2012.sched.org/event/bf543d09fcf4de3f5b4a1008341e5524
Monday 11am - Customizing Ceilometer to measure your cloud

http://openstacksummitfall2012.sched.org/event/c3bca2fa753590b2429759810b753c19
Monday 11:50am - Beyond metering - Extending Ceilometer

http://openstacksummitfall2012.sched.org/event/235e725deb3c224dfd4da88726729faf

or come to the conference presentation Doug Hellmann and Nick Barcet
will be giving:

Monday at 2:40pm - What about billing? An introduction to Ceilometer

http://openstacksummitfall2012.sched.org/event/94fb2cd407b143af4c3fbd979d9c8f91

Where to find more about Ceilometer?

Project home and bug tracker:
 http://launchpad.net/ceilometer

Documentation:
  http://ceiilometer.readthedocs.org

Mailing list:
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  (prefix subjects with [metering] for faster responses)

Wiki:
  http://wiki.openstack.org/EfficientMetering

Code hosting:
  https://github.com/stackforge/ceilometer

Tarballs:
  http://tarballs.openstack.org/ceilometer/

Code reviews:

https://review.openstack.org/#/q/status:open+project:stackforge/ceilometer,n,z

Jenkins:
  https://jenkins.openstack.org/view/Ceilometer/

IRC:
  #openstack-metering on freenode


We hope that you’ll find this contribution valuable.

On behalf of the ceilometer team,
--
Nick Barcet 
aka: nijaba, nicolas



signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [ceilometer] Potential New Use Cases

2012-10-25 Thread Nick Barcet
Let's imagine that the service that launch instances can tag the
instance with:
a) a common service identifier (constant)
b) a uuid unique for each "Unit" of the service
such as :

If that tag is passed onto the events which ceilometer stores in its
entirety as meta, I do not see what the difficulty would be for the
rating engine to be able to reconcile the information to handle your 2
use cases.  Am I missing something?

Nick

On 10/25/2012 12:03 AM, Dan Dyer wrote:
> I don't think its just a matter of adding more meters or events for a
> couple of reasons:
> 1. In many cases the metadata I am referring to comes from a different
> source than the base usage data. Nova is still emitting its normal
> events, but we get the service/user mapping from a different source. I
> would not characterize this data as usage metrics but more data about
> the system relationships.
> 2. in the multiple VM case, we need to have the relationships specified
> so that we can ignore the proper VM's. There has also been talk of
> hybrid billing models that charge for some part of the VM usage as well
> as other metrics. Once again we need a way to characterize the
> relationships so that processing can associate and filter correctly.
> 
> Dan
> 
> On 10/24/2012 3:35 PM, Julien Danjou wrote:
>> On Wed, Oct 24 2012, Dan Dyer wrote:
>>
>>> Use Case 1
>>> Service Owned Instances
>>> There are a set of use cases where a service is acting on behalf of a
>>> user,
>>> the service is the owner of the VM but billing needs to be attributed
>>> to the
>>> end user of the system.This scenario drives two requirements:
>>> 1. Pricing is similar to base VM's but with a premium. So the type of
>>> service for a VM needs to be identifiable so that the appropriate
>>> pricing
>>> can be applied.
>>> 2. The actual end user of the VM needs to be identified so usage can be
>>> properly attributed
>> I think that for this, you just need to add more meters on top of the
>> existing one with your own user and project id information.
>>
>>> As an example, in some of our PAAS use cases, there is a service
>>> controller
>>> running on top of the base VM that maintains the control and and
>>> manages the
>>> customer experience. The idea is to expose the service and not have the
>>> customer have to (or even be able to) manipulate the virtual machine
>>> directly. So in this case, from a Nova perspective, the PAAS service
>>> owns
>>> the VM and it's tenantID is what is reported back in events. The way we
>>> resolve this is to query the service controller for meta data about that
>>> instances they own. This is stored off in a separate "table" and used to
>>> determine the real user at aggregation time.
>> This is probably where you should emit the meters you need.
>>
>>> Use Case 2
>>> Multple Instances combine to make a billable "product/service"
>>> In this use case, a service might consist of several VM's, but the
>>> actual
>>> number does not directly drive the billing.  An example of this might
>>> be a
>>> redundant service that has a primary and two backup VM's that make up a
>>> deployment. The customer is charged for the service, not the fact
>>> that there
>>> are 3 VM's running. Once again, we need meta data that is able to
>>> describe
>>> this relationship so that when the billing records are processed, this
>>> relationship can be identified and billed properly.
>> Kind of the same here, if you don't want to really bill the vm, just
>> don't meter them (or ignore the meters) and emit your own meter via your
>> PaaS platform to bill your customer.
>>
>> Or is there a limitation I miss?
>>
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp