On 11/19/2015 5:52 PM, gord chung wrote:
ceilometer cares. we listen to all notifications and build Measurement
and Event data from (some of) them. to be honest, i don't know if
changes in nova notifications have/are broken in ceilometer because
quite frankly there are far too many notifications to test and track.
as a consumer of data, we don't really care what the format of the
entire payload is. we just care that the select attributes we do care
about are present and in some known position.
in ceilometer, we have two mapping files which define the notifications
we are interested in and how we to normalise them[1][2]. currently, they
are two files contained within ceilometer which we load at runtime. that
said, at the summit we talked about how the projects should own their
own definitions rather than ceilometer[3]. essentially, producer and
consumer schemas are tightly coupled.
the basic premise was that we as consumers have no idea what is getting
produced, but the projects do know and they know exactly what is useful
in what they are producing. so if nova owned their own meters.yaml and
events.yaml, nova devs could easily edit the appropriate file themselves
when they add/modify/remove notification payloads. if a new metric is
added to a notification, it can easily be added to meters.yaml; if a
metric value is moved, meters.yaml can be updated for old and new
locations.
using each projects definition files, ceilometer would load them all in
and as a consumer, not have to worry about what every single change that
happens outside of its domain. it's a different take on it: instead of
having ownership on consumer to interpret accurately, the ownership is
on producer to ensure consumer can understand.
we are working on a POC currently as there some logistics to be thought
out still(ie. how ceilometer knows where to look for each projects
definition files) but using this idea, we wouldn't even need to know
what version of notification we were getting, just that we are told that
we should look "either here or here...". if there's interest in
exploring this idea or if there are any glaring issues please let us know.
[1]
https://github.com/openstack/ceilometer/blob/master/ceilometer/meter/data/meters.yaml
[2]
https://github.com/openstack/ceilometer/blob/master/etc/ceilometer/event_definitions.yaml
[3] https://etherpad.openstack.org/p/mitaka-telemetry-cross-project
On 19/11/2015 5:05 PM, Ryan Rossiter wrote:
Reading through [1] I started getting worries in the back of my head
about versioning these notifications. The main concern being how can
the consumer know about the versions and what's different between
them? Because these versioned notification payloads hold live nova
objects, there can be a lot of rug-pulling going on underneath these
notifications. If the payload doesn't pin itself to a certain level of
the object, a consumer can never be guaranteed the version of the
payload's object they will be receiving. I ran through a few of the
scenarios about irregular versions in the notifications subteam
meeting on Tuesday [2].
My question is.... do we care about the consumer? Or is it a case of
"the consumer is always right" so we need to make sure we hand them
super consistent, well-defined blobs across the wire? Consumers will
have no idea of nova object internals, unless they feel like `python
-c import nova`. I do think we get one piece of help from o.vo though.
When the object is serialized, it hands the version with the object.
So consumers can look at the object and say "oh, I got 1.4 I know what
to do with this". But... they will have to implement their own compat
logic. Everyone will have to implement their own compat logic.
We could expose a new API for getting the schema for a specific
version of a notification, so a consumer will know what they're
getting with their notifications. But I think that made mriedem
nauseous. We could make an oslo library that stuffs a shim in between
o.vo and nova's notifications to help out with compat/versioning, but
that sounds like a lot of work, especially because the end goal is
still not clearly defined.
Thoughts?
[1] https://review.openstack.org/#/c/247024
[2]
http://eavesdrop.openstack.org/irclogs/%23openstack-meeting-alt/%23openstack-meeting-alt.2015-11-17.log.html#t2015-11-17T20:22:29
I don't think nova is going to maintain it's own meters/events yaml
files in the form that ceilometer necessarily expects/uses today, since
there are other things out there that consume nova notifications besides
ceilometer. But I think you're thinking along the same lines of what
Ryan was thinking, i.e. somehow nova providing a schema or some way for
the consumer to digest/interpret what they are getting, but I don't know
what that would look like or how it would be used by the consumer.
--
Thanks,
Matt Riedemann
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
