Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-26 Thread Daniele Venzano
On 09/25/14 17:55, Clint Byrum wrote: Now I use Ceilometer's pipeline to forward events to elasticsearch via udp + logstash and do not use Ceilometer's DB or API at all. Interesting, this almost sounds like what should be the default configuration honestly. Ceilometer generates a lot of data in

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Qiming Teng
On Thu, Sep 25, 2014 at 11:51:23AM -0400, gordon chung wrote: > > mysql> select count(*) from metadata_text; > > +--+ > > | count(*) | > > +--+ > > | 25249913 | > > +--+ > > 1 row in set (3.83 sec)> > > There were 25M records in one table. The deletion time is reaching an

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Clint Byrum
Excerpts from Daniele Venzano's message of 2014-09-25 02:40:11 -0700: > On 09/25/14 10:12, Qiming Teng wrote: > > Yes, just about 3 VMs running on two hosts, for at most 3 weeks. This > > is leading me to another question -- any best practices/tools to > > retire the old data on a regular basis?

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread gordon chung
> mysql> select count(*) from metadata_text; > +--+ > | count(*) | > +--+ > | 25249913 | > +--+ > 1 row in set (3.83 sec)> > There were 25M records in one table. The deletion time is reaching an > unacceptable level (7 minutes for 4M records) and it was not increasing > in

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Qiming Teng
On Thu, Sep 25, 2014 at 11:40:11AM +0200, Daniele Venzano wrote: > On 09/25/14 10:12, Qiming Teng wrote: > >Yes, just about 3 VMs running on two hosts, for at most 3 weeks. > >This is leading me to another question -- any best practices/tools > >to retire the old data on a regular basis? Regards, Q

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Daniele Venzano
On 09/25/14 10:12, Qiming Teng wrote: Yes, just about 3 VMs running on two hosts, for at most 3 weeks. This is leading me to another question -- any best practices/tools to retire the old data on a regular basis? Regards, Qiming There is a tool: ceilometer-expirer I tried to use it on a mysql

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Qiming Teng
On Wed, Sep 24, 2014 at 09:43:54PM -0700, Preston L. Bannister wrote: > Sorry, I am jumping into this without enough context, but ... > > > On Wed, Sep 24, 2014 at 8:37 PM, Qiming Teng > wrote: > > > > mysql> select count(*) from metadata_text; > > +--+ > > | count(*) | > > +--+

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Dina Belova
Qiming, yes - for MongoDB, DB2, HBase and SQL-based, all the backends support events feature for now, this has been merged afair ~month or two ago. Cheers Dina On Thu, Sep 25, 2014 at 11:45 AM, Qiming Teng wrote: > So MongoDB support to events is ready in tree? > > Regards, > Qiming > > On Th

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Qiming Teng
So MongoDB support to events is ready in tree? Regards, Qiming On Thu, Sep 25, 2014 at 10:26:08AM +0300, Igor Degtiarov wrote: > Hi, Qiming Teng. > > Now all backends support events. So you may use MongoDB instead of > MySQL, or if you like you may choose HBase. > > Cheers, Igor. > -- Igor >

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-25 Thread Igor Degtiarov
Hi, Qiming Teng. Now all backends support events. So you may use MongoDB instead of MySQL, or if you like you may choose HBase. Cheers, Igor. -- Igor On Thu, Sep 25, 2014 at 7:43 AM, Preston L. Bannister wrote: > Sorry, I am jumping into this without enough context, but ... > > > On Wed, Sep 2

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-24 Thread Preston L. Bannister
Sorry, I am jumping into this without enough context, but ... On Wed, Sep 24, 2014 at 8:37 PM, Qiming Teng wrote: > > mysql> select count(*) from metadata_text; > +--+ > | count(*) | > +--+ > | 25249913 | > +--+ > 1 row in set (3.83 sec) > There are problems where a sim

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-24 Thread Clint Byrum
Excerpts from Qiming Teng's message of 2014-09-24 20:37:39 -0700: > Hi, > > Some weeks ago, I checked my then latest devstack install and I learned > this: event support in Ceilometer is only available for sqlalchemy > backend; mongodb backend was still under development. I have been using > MySQ

Re: [openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-24 Thread Mark Kirkwood
On 25/09/14 15:37, Qiming Teng wrote: Hi, Some weeks ago, I checked my then latest devstack install and I learned this: event support in Ceilometer is only available for sqlalchemy backend; mongodb backend was still under development. I have been using MySQL during the past weeks and now I thin

[openstack-dev] [Ceilometer] MySQL performance and Mongodb backend maturity question

2014-09-24 Thread Qiming Teng
Hi, Some weeks ago, I checked my then latest devstack install and I learned this: event support in Ceilometer is only available for sqlalchemy backend; mongodb backend was still under development. I have been using MySQL during the past weeks and now I think I'm trapped by a performance problem o