On 8/11/2014 5:29 PM, Eoghan Glynn wrote: > >> On 8/11/2014 4:22 PM, Eoghan Glynn wrote: >>>> Hi Eoghan, >>>> >>>> Thanks for the note below. However, one thing the overview below does not >>>> cover is why InfluxDB ( http://influxdb.com/ ) is not being leveraged. >>>> Many >>>> folks feel that this technology is a viable solution for the problem space >>>> discussed below. >>> Great question Brad! >>> >>> As it happens we've been working closely with Paul Dix (lead >>> developer of InfluxDB) to ensure that this metrics store would be >>> usable as a backend driver. That conversation actually kicked off >>> at the Juno summit in Atlanta, but it really got off the ground >>> at our mid-cycle meet-up in Paris on in early July. >> ... >>> The InfluxDB folks have committed to implementing those features in >>> over July and August, and have made concrete progress on that score. >>> >>> I hope that provides enough detail to answer to your question? >> I guess it begs the question, if influxdb will do what you want and it's >> open source (MIT) as well as commercially supported, how does gnocchi >> differentiate? > Hi Sandy, > > One of the ideas behind gnocchi is to combine resource representation > and timeseries-oriented storage of metric data, providing an efficient > and convenient way to query for metric data associated with individual > resources.
Doesn't InfluxDB do the same? > > Also, having an API layered above the storage driver avoids locking in > directly with a particular metrics-oriented DB, allowing for the > potential to support multiple storage driver options (e.g. to choose > between a canonical implementation based on Swift, an InfluxDB driver, > and an OpenTSDB driver, say). Right, I'm not suggesting to remove the storage abstraction layer. I'm just curious what gnocchi does better/different than InfluxDB? Or, am I missing the objective here and gnocchi is the abstraction layer and not an influxdb alternative? If so, my apologies for the confusion. > A less compelling reason would be to provide a well-defined hook point > to innovate with aggregation/analytic logic not supported natively > in the underlying drivers (e.g. period-spanning statistics such as > exponentially-weighted moving average or even Holt-Winters). > Cheers, > Eoghan > > >>> Cheers, >>> Eoghan >>> >>>> Thanks, >>>> >>>> Brad >>>> >>>> >>>> Brad Topol, Ph.D. >>>> IBM Distinguished Engineer >>>> OpenStack >>>> (919) 543-0646 >>>> Internet: bto...@us.ibm.com >>>> Assistant: Kendra Witherspoon (919) 254-0680 >>>> >>>> >>>> >>>> From: Eoghan Glynn <egl...@redhat.com> >>>> To: "OpenStack Development Mailing List (not for usage questions)" >>>> <openstack-dev@lists.openstack.org>, >>>> Date: 08/06/2014 11:17 AM >>>> Subject: [openstack-dev] [tc][ceilometer] Some background on the gnocchi >>>> project >>>> >>>> >>>> >>>> >>>> >>>> Folks, >>>> >>>> It's come to our attention that some key individuals are not >>>> fully up-to-date on gnocchi activities, so it being a good and >>>> healthy thing to ensure we're as communicative as possible about >>>> our roadmap, I've provided a high-level overview here of our >>>> thinking. This is intended as a precursor to further discussion >>>> with the TC. >>>> >>>> Cheers, >>>> Eoghan >>>> >>>> >>>> What gnocchi is: >>>> =============== >>>> >>>> Gnocchi is a separate, but related, project spun up on stackforge >>>> by Julien Danjou, with the objective of providing efficient >>>> storage and retrieval of timeseries-oriented data and resource >>>> representations. >>>> >>>> The goal is to experiment with a potential approach to addressing >>>> an architectural misstep made in the very earliest days of >>>> ceilometer, specifically the decision to store snapshots of some >>>> resource metadata alongside each metric datapoint. The core idea >>>> is to move to storing datapoints shorn of metadata, and instead >>>> allow the resource-state timeline to be reconstructed more cheaply >>>> from much less frequently occurring events (e.g. instance resizes >>>> or migrations). >>>> >>>> >>>> What gnocchi isn't: >>>> ================== >>>> >>>> Gnocchi is not a large-scale under-the-radar rewrite of a core >>>> OpenStack component along the lines of keystone-lite. >>>> >>>> The change is concentrated on the final data-storage phase of >>>> the ceilometer pipeline, so will have little initial impact on the >>>> data-acquiring agents, or on transformation phase. >>>> >>>> We've been totally open at the Atlanta summit and other forums >>>> about this approach being a multi-cycle effort. >>>> >>>> >>>> Why we decided to do it this way: >>>> ================================ >>>> >>>> The intent behind spinning up a separate project on stackforge >>>> was to allow the work progress at arms-length from ceilometer, >>>> allowing normalcy to be maintained on the core project and a >>>> rapid rate of innovation on gnocchi. >>>> >>>> Note that that the developers primarily contributing to gnocchi >>>> represent a cross-section of the core team, and there's a regular >>>> feedback loop in the form of a recurring agenda item at the >>>> weekly team meeting to avoid the effort becoming silo'd. >>>> >>>> >>>> But isn't re-architecting frowned upon? >>>> ====================================== >>>> >>>> Well, the architecture of other OpenStack projects have also >>>> under-gone change as the community understanding of the >>>> implications of prior design decisions has evolved. >>>> >>>> Take for example the move towards nova no-db-compute & the >>>> unified-object-model in order to address issues in the nova >>>> architecture that made progress towards rolling upgrades >>>> unneccessarily difficult. >>>> >>>> The point, in my understanding, is not to avoid doing the >>>> course-correction where it's deemed necessary. Rather, the >>>> principle is more that these corrections happen in an open >>>> and planned way. >>>> >>>> >>>> The path forward: >>>> ================ >>>> >>>> A subset of the ceilometer community will continue to work on >>>> gnocchi in parallel with the ceilometer core over the remainder >>>> of the Juno cycle and into the Kilo timeframe. The goal is to >>>> have an initial implementation of gnocchi ready for tech preview >>>> by the end of Juno, and to have the integration/migration/ >>>> co-existence questions addressed in Kilo. >>>> >>>> Moving the ceilometer core to using gnocchi will be contingent >>>> on it demonstrating the required performance characteristics and >>>> providing the semantics needed to support a v3 ceilometer API >>>> that's fit-for-purpose. >>>> >>>> _______________________________________________ >>>> OpenStack-dev mailing list >>>> OpenStack-dev@lists.openstack.org >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>>> >>>> _______________________________________________ >>>> OpenStack-dev mailing list >>>> OpenStack-dev@lists.openstack.org >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev