Re: [openstack-dev] Treating notifications as a contract
From: Doug Hellmann [d...@doughellmann.com] Tuesday, October 14, 2014 7:19 PM > It might be more appropriate to put it on the cross-project session list: > https://etherpad.openstack.org/p/kilo-crossproject-summit-topics Done ... thanks! ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Treating notifications as a contract
On Oct 13, 2014, at 5:49 PM, Chris Dent wrote: > On Tue, 7 Oct 2014, Sandy Walsh wrote: > >> Haven't had any time to get anything written down (pressing deadlines >> with StackTach.v3) but open to suggestions. Perhaps we should just add >> something to the olso.messaging etherpad to find time at the summit to >> talk about it? > > Have you got a link for that? It might be more appropriate to put it on the cross-project session list: https://etherpad.openstack.org/p/kilo-crossproject-summit-topics Doug > > Another topic that I think is at least somewhat related to the > standardizing/contractualizing notifications topic is deprecating > polling (to get metrics/samples). > > In the ceilometer side of the telemetry universe, if samples can't > be gathered via notifications then somebody writes a polling plugin > or agent and sticks it in the ceilometer tree where it is run as > either an independent agent (c.f. the new ipmi-agent) or a plugin > under the compute-agent or a plugin under the central-agent. > > This is problematic in a few ways (at least to me): > > * Those plugins distract from the potential leanness of a core > ceilometer system. > > * The meters created by those plugins are produced for "ceilometer" > rather than for "telemetry". Yes, of course you can re-publish the > samples in all sorts of ways. > > * The services aren't owning the form and publication of information > about themselves. > > There are solid arguments against each of these problems individually > but as a set I find them saying "services should make more > notifications" pretty loud and clear and obviously to make that work > we need tidy notifications with good clean semantics. > > -- > Chris Dent tw:@anticdent freenode:cdent > https://tank.peermore.com/tanks/cdent > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Treating notifications as a contract
On Tue, 7 Oct 2014, Sandy Walsh wrote: Haven't had any time to get anything written down (pressing deadlines with StackTach.v3) but open to suggestions. Perhaps we should just add something to the olso.messaging etherpad to find time at the summit to talk about it? Have you got a link for that? Another topic that I think is at least somewhat related to the standardizing/contractualizing notifications topic is deprecating polling (to get metrics/samples). In the ceilometer side of the telemetry universe, if samples can't be gathered via notifications then somebody writes a polling plugin or agent and sticks it in the ceilometer tree where it is run as either an independent agent (c.f. the new ipmi-agent) or a plugin under the compute-agent or a plugin under the central-agent. This is problematic in a few ways (at least to me): * Those plugins distract from the potential leanness of a core ceilometer system. * The meters created by those plugins are produced for "ceilometer" rather than for "telemetry". Yes, of course you can re-publish the samples in all sorts of ways. * The services aren't owning the form and publication of information about themselves. There are solid arguments against each of these problems individually but as a set I find them saying "services should make more notifications" pretty loud and clear and obviously to make that work we need tidy notifications with good clean semantics. -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Treating notifications as a contract
> >From: Sandy Walsh [sandy.wa...@rackspace.com] Tuesday, October 07, 2014 6:07 PM > >Haven't had any time to get anything written down (pressing deadlines with >StackTach.v3) but open to suggestions. Perhaps we should just add something to >the olso.messaging etherpad to find time at the summit to talk about it? > >-S Actually, that's not really true. The Monasca team has been playing with schema definitions for their wire format (a variation on the kind of notification we ultimately want). And http://apiary.io/ is introducing support for structure schemas soon. Perhaps we can start with some schema proposals there? JSON-Schema based? For green-field installations, CADF is a possibility, but for already established services we will to document what's in place first. At some point we'll need a cross-project effort to identify all the important characteristics of the various services. Also, we've been finding no end of problems with the wild-west payload section. For example, look at all the different places we have to look to find the instance UUID from Nova. https://github.com/SandyWalsh/stacktach-sandbox/blob/verifier/winchester/event_definitions.yaml#L12-L17 Likewise for project_id, flavor_id, deleted_at, etc. Definitely need a solution to this. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Treating notifications as a contract
>From: Chris Dent [chd...@redhat.com] Tuesday, October 07, 2014 12:07 PM
>
>On Wed, 3 Sep 2014, Sandy Walsh wrote:
>
>> Good goals. When Producer and Consumer know what to expect, things are
>> good ... "I know to find the Instance ID ". When the consumer
>> wants to deal with a notification as a generic object, things get tricky
>> ("find the instance ID in the payload", "What is the image type?", "Is
>> this an error notification?")
>>
>> Basically, how do we define the principle artifacts for each service and
>> grant the consumer easy/consistent access to them? (like the 7-W's above)
>>
>> I'd really like to find a way to solve that problem.
>>
>>> Is that a good summary? What did I leave out or get wrong?
>>
>> Great start! Let's keep it simple and do-able.
>
>Has there been any further thinking on these topics? Summit is soon
>and kilo specs are starting so I imagine more people than just me are
>hoping to get rolling on plans.
>
>If there is going to be a discussion at summit I hope people will be
>good about keeping some artifacts for those of us watching from afar.
>
>It seems to me that if the notifications ecosystem becomes
>sufficiently robust and resilient we ought to be able to achieve
>some interesting scale and distributed-ness opporunities throughout
>OpenStack, not just in telemetry/metering/eventing (choose your term
>of art).
Haven't had any time to get anything written down (pressing deadlines with
StackTach.v3) but open to suggestions. Perhaps we should just add something to
the olso.messaging etherpad to find time at the summit to talk about it?
-S
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Treating notifications as a contract
On Wed, 3 Sep 2014, Sandy Walsh wrote:
Good goals. When Producer and Consumer know what to expect, things are
good ... "I know to find the Instance ID ". When the consumer
wants to deal with a notification as a generic object, things get tricky
("find the instance ID in the payload", "What is the image type?", "Is
this an error notification?")
Basically, how do we define the principle artifacts for each service and
grant the consumer easy/consistent access to them? (like the 7-W's above)
I'd really like to find a way to solve that problem.
Is that a good summary? What did I leave out or get wrong?
Great start! Let's keep it simple and do-able.
Has there been any further thinking on these topics? Summit is soon
and kilo specs are starting so I imagine more people than just me are
hoping to get rolling on plans.
If there is going to be a discussion at summit I hope people will be
good about keeping some artifacts for those of us watching from afar.
It seems to me that if the notifications ecosystem becomes
sufficiently robust and resilient we ought to be able to achieve
some interesting scale and distributed-ness opporunities throughout
OpenStack, not just in telemetry/metering/eventing (choose your term
of art).
--
Chris Dent tw:@anticdent freenode:cdent
https://tank.peermore.com/tanks/cdent
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Treating notifications as a contract
On Wed, 10 Sep 2014, Jay Pipes wrote: There would be an Oslo library that would store the codification of the resource classes and actions, along with the mapping of (resource_class, action, version) to the JSONSchema document describing the payload field. This seems reasonable with two caveats: * What Sandy says about making sure the _actual_ codification was in a language neutral format of some kind. * That we don't limit my other concern (the macro one requoted below): Anybody should be able to rock up and dump _new_ notifications on the bus given suitable credentials and configuration with neither writing of local custom code nor centralized reviewing of global code. (I'd really like it to be possible to publish the notifications with _only_ credentials and no config and have a reasonable expectation of them being captured, but I understand that's not very robust and rather pipe dreamy.) I think it ought to be possible to accomplish both by: * Having the library you suggest but it reads schema from rather than embedding in code. * Same library can read additional files which contain 'local' customizations. Endpoints then know how to compose and decompose a vast swath of notifcations. A nice to have would be that the schema are inheritable. * At the macro level standardize a packaging or envelope of all notifications so that they can be consumed by very similar code. That is: constrain the notifications in some way so we can also constrain the consumer code. Or maybe we should just use RDF and produce a super upper ontology and consume all the world's knowledge as events? That's been super successful in other contexts... -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Treating notifications as a contract
> Jay Pipes - Wednesday, September 10, 2014 3:56 PM >On 09/03/2014 11:21 AM, Sandy Walsh wrote: >> On 9/3/2014 11:32 AM, Chris Dent wrote: >>> I took some notes on this a few weeks ago and extracted what seemed >>> to be the two main threads or ideas the were revealed by the >>> conversation that happened in this thread: >>> >>> * At the micro level have versioned schema for notifications such that >>> one end can declare "I am sending version X of notification >>> foo.bar.Y" and the other end can effectively deal. >> >> Yes, that's table-stakes I think. Putting structure around the payload >> section. >> >> Beyond type and version we should be able to attach meta information >> like public/private visibility and perhaps hints for external mapping >> (this trait -> that trait in CADF, for example). > >CADF doesn't address the underlying problem that Chris mentions above: >that our notification events themselves needs to have a version >associated with them. > >Instead of versioning the message payloads themselves, instead CADF >focuses versioning on the CADF spec itself, which is less than useful, >IMO, and a sympton of what I like to call "XML-itis". Well, the spec is the payload, so you can't change the payload without changing the spec. Could be semantics, but I see your point. >Where I *do* see some value in CADF is the primitive string codes it >defines for resource classifications, actions, and outcomes (Sections >A.2.5, A.3.5., and A.4.5 respectively in the CADF spec). I see no value >in the long-form XML-itis fully-qualified URI long-forms of the >primitive string codes. +1 to the xml-itis, but do we really get any value from the resource classifications without them? Other than, "yes, that's a good list to work from."? >For resource classifications, it defines things like "compute", >"storage", "service", etc, as well as a structured hierarchy for >sub-classifications, like "storage/volume" or "service/block". Actions >are string codes for verbs like "create", "configure" or "authenticate". >Outcomes are string codes for "success", "failure", etc. > >What I feel we need is a library that matches a (resource_type, action, >version) tuple to a JSONSchema document that describes the payload for >that combination of resource_type, action, and version. The 7-W's that CADF define are quite useful and we should try to ensure our notification payloads address as many of them as possible. Who, What, When, Where, Why, On-What, To-Whom, To-Where ... not all are applicable for every notification type. Also, we need to define standard units-of-measure for numeric fields: mb vs. gb, bps vs. kbps, image type definitions ... ideally all this should be part of the standard openstack nomenclature. These are the things that really belong in oslo and used by everything from notifications to the scheduler to flavor definitions, etc. >If I were king for a day, I'd have a standardized notification message >format that simply consisted of: > >resource_class (string) <-- From CADF, e.g. "service/block" >occurred_on (timestamp) <-- when the event was published >action (string) <-- From CADF, e.g. "create" >version (int or tuple) <-- version of the (resource_class, action) >payload (json-encoded string) <-- the message itself >outcome (string) <-- Still on fence for this, versus just using payload Yep, not a problem with that, so long as the payload has all the other things we need (versioning, data types, visibility, etc) >There would be an Oslo library that would store the codification of the >resource classes and actions, along with the mapping of (resource_class, >action, version) to the JSONSchema document describing the payload field. > >Producers of messages would consume the oslo lib like so: > >```python >from oslo.notifications import resource_classes >from oslo.notifications import actions >from oslo.notifications import message Not sure how this would look from a packaging perspective, but sure. I'm not sure if I like having to define every resource/action type in code and then having an explosion of types in notification.actions ... perhaps that should just be part of the schema definition 'action_type': [acceptable values: "create, delete, update"] I'd rather see these schemas defined in some machine readable format (yaml or something) vs. code. Other languages are going to want to consume these notifications and should be able to reuse the definitions. >from nova.compute import power_states >from nova.compute import task_states >... > >msg = message.Message(resource_classes.compute.machine, > actions.update, > version=1) > ># msg is now an object that is guarded by the JSONSchema document ># that describes the version 1.0 schema of the UPDATE action ># for the resource class representing a VM (compute.machine) ># This means that if the producer attempts to set an ># attribute
Re: [openstack-dev] Treating notifications as a contract
On 09/03/2014 11:21 AM, Sandy Walsh wrote:
On 9/3/2014 11:32 AM, Chris Dent wrote:
I took some notes on this a few weeks ago and extracted what seemed
to be the two main threads or ideas the were revealed by the
conversation that happened in this thread:
* At the micro level have versioned schema for notifications such that
one end can declare "I am sending version X of notification
foo.bar.Y" and the other end can effectively deal.
Yes, that's table-stakes I think. Putting structure around the payload
section.
Beyond type and version we should be able to attach meta information
like public/private visibility and perhaps hints for external mapping
(this trait -> that trait in CADF, for example).
CADF doesn't address the underlying problem that Chris mentions above:
that our notification events themselves needs to have a version
associated with them.
Instead of versioning the message payloads themselves, instead CADF
focuses versioning on the CADF spec itself, which is less than useful,
IMO, and a sympton of what I like to call "XML-itis".
Where I *do* see some value in CADF is the primitive string codes it
defines for resource classifications, actions, and outcomes (Sections
A.2.5, A.3.5., and A.4.5 respectively in the CADF spec). I see no value
in the long-form XML-itis fully-qualified URI long-forms of the
primitive string codes.
For resource classifications, it defines things like "compute",
"storage", "service", etc, as well as a structured hierarchy for
sub-classifications, like "storage/volume" or "service/block". Actions
are string codes for verbs like "create", "configure" or "authenticate".
Outcomes are string codes for "success", "failure", etc.
What I feel we need is a library that matches a (resource_type, action,
version) tuple to a JSONSchema document that describes the payload for
that combination of resource_type, action, and version.
If I were king for a day, I'd have a standardized notification message
format that simply consisted of:
resource_class (string) <-- From CADF, e.g. "service/block"
occurred_on (timestamp) <-- when the event was published
action (string) <-- From CADF, e.g. "create"
version (int or tuple) <-- version of the (resource_class, action)
payload (json-encoded string) <-- the message itself
outcome (string) <-- Still on fence for this, versus just using payload
There would be an Oslo library that would store the codification of the
resource classes and actions, along with the mapping of (resource_class,
action, version) to the JSONSchema document describing the payload field.
Producers of messages would consume the oslo lib like so:
```python
from oslo.notifications import resource_classes
from oslo.notifications import actions
from oslo.notifications import message
from nova.compute import power_states
from nova.compute import task_states
...
msg = message.Message(resource_classes.compute.machine,
actions.update,
version=1)
# msg is now an object that is guarded by the JSONSchema document
# that describes the version 1.0 schema of the UPDATE action
# for the resource class representing a VM (compute.machine)
# This means that if the producer attempts to set an
# attribute of the msg object that is *not* in that JSONSchema
# document, then an AttributeError would be raised. This essentially
# codifies the message's resource_class and action attributes
# (as constants in the oslo.notifications.resource_classes and
# oslo.notifications.actions module) as well as codifies the
# schema of the (resource_class, action, version) combo.
# Assume the JSONSchema document for a
# (resource_class, action, version) of
# ("compute.machine", "update", 1) looks like this:
# {
#"properties": {
# "state": {
# "type": "string",
# "description": "The new power state of VM"
# },
# "state": {
# "type": "string",
# "description": "The old power state of VM"
# },
# "task_state": {
# "type": "string",
# "description": "The new task state of VM"
# },
# "old_task_state": {
# "type": "string",
# "description": "The old task state of VM"
# }
# "additionalProperties": false
# }
msg.old_state = power_states.RUNNING
msg.state = power_states.SHUTDOWN
msg.old_taskkk_state = None # <--- would blow up with TypeError,
# since taskkk is mispelled.
# Send the message over the wire...
message.send(...)
```
Similarly, on the consumer side, the message for a particular
resource_class, action and version can be constructed from the
oslo.notifications library and the JSONSchema document could be
consulted for payload introspection:
```python
from oslo.notifications imp
Re: [openstack-dev] Treating notifications as a contract (CADF)
> On 9/04/2014 Sandy Walsh wrote: > Yesterday, we had a great conversation with Matt Rutkowski from IBM, > one > of the authors of the CADF spec. > > I was having a disconnect on what CADF offers and got it clarified. > > My assumption was CADF was a set of transformation/extraction rules > for > taking data from existing data structures and defining them as > well-known things. For example, CADF needs to know "who sent this > notification". I thought CADF would give us a means to point at an > existing data structure and say "that's where you find it". > > But I was wrong. CADF is a full-on schema/data structure of its own. It > would be a fork-lift replacement for our existing notifications. This was my "aha" as well, following a similar discussion with Matt and team, but also note that they've articulated an approach for bolt-on changes that would enable CADF content in existing pipelines. (https://wiki.openstack.org/wiki/Ceilometer/blueprints/support-standard-audit-formats) > However, if your service hasn't really adopted notifications yet (green > field) or you can handle a fork-lift replacement, CADF is a good option. > There are a few gotcha's though. If you have required data that is > outside of the CADF spec, it would need to go in the "attachment" > section of the notification and that still needs a separate schema to > define it. Matt's team is very receptive to extending the spec to > include these special cases though. Agreed that Matt's team was very willing to extend, but I still wonder about having to migrate appended data from its "pre-approval" location to its "permanent" location, depending on the speed of the CADF standard update. > > Anyway, I've written up all the options (as I see them) [1] with the > advantages/disadvantages of each approach. It's just a strawman, so > bend/spindle/mutilate. Cool...will add comments there. > > Look forward to feedback! > -S > > > [1] https://wiki.openstack.org/wiki/NotificationsAndCADF > > > > > On 9/3/2014 12:30 PM, Sandy Walsh wrote: > > On 9/3/2014 11:32 AM, Chris Dent wrote: > >> On Wed, 3 Sep 2014, Sandy Walsh wrote: > >> > >>> We're chatting with IBM about CADF and getting down to specifics > on > >>> their applicability to notifications. Once I get StackTach.v3 into > >>> production I'm keen to get started on revisiting the notification > >>> format and olso.messaging support for notifications. > >>> > >>> Perhaps a hangout for those keenly interested in doing something > about this? > >> That seems like a good idea. I'd like to be a part of that. I would , too, and I would suggest that much of the Ceilometer team would > >> Unfortunately I won't be at summit but would like to contribute what > >> I can before and after. > >> > >> I took some notes on this a few weeks ago and extracted what > seemed > >> to be the two main threads or ideas the were revealed by the > >> conversation that happened in this thread: > >> > >> * At the micro level have versioned schema for notifications such > that > >>one end can declare "I am sending version X of notification > >>foo.bar.Y" and the other end can effectively deal. > > Yes, that's table-stakes I think. Putting structure around the payload > > section. > > > > Beyond type and version we should be able to attach meta information > > like public/private visibility and perhaps hints for external mapping > > (this trait -> that trait in CADF, for example). > > > >> * At the macro level standardize a packaging or envelope of all > >>notifications so that they can be consumed by very similar code. > >>That is: constrain the notifications in some way so we can also > >>constrain the consumer code. > > That's the intention of what we have now. The top level traits are > > standard, the payload is open. We really only require: message_id, > > timestamp and event_type. For auditing we need to cover Who, What, > When, > > Where, Why, OnWhat, OnWhere, FromWhere. > > To whit, I think we've made good progress in this by defining the "what is the minimum content" for PaaS service notifications and gotten agreement around https://review.openstack.org/#/c/113396/11/doc/source/format.rst for the Juno release. It's been driven by many of these same questions but is fairly narrow in scope; it defines a minimum set of content, but doesn't tackle the question of structure (beyond trait typing). The timing seems right to dig deeper. > >> These ideas serve two different purposes: One is to ensure that > >> existing notification use cases are satisfied with robustness and > >> provide a contract between two endpoints. The other is to allow a > >> fecund notification environment that allows and enables many > >> participants. > > Good goals. When Producer and Consumer know what to expect, > things are > > good ... "I know to find the Instance ID ". When the consumer > > wants to deal with a notification as a generic object,
Re: [openstack-dev] Treating notifications as a contract (CADF)
Yesterday, we had a great conversation with Matt Rutkowski from IBM, one
of the authors of the CADF spec.
I was having a disconnect on what CADF offers and got it clarified.
My assumption was CADF was a set of transformation/extraction rules for
taking data from existing data structures and defining them as
well-known things. For example, CADF needs to know "who sent this
notification". I thought CADF would give us a means to point at an
existing data structure and say "that's where you find it".
But I was wrong. CADF is a full-on schema/data structure of its own. It
would be a fork-lift replacement for our existing notifications.
However, if your service hasn't really adopted notifications yet (green
field) or you can handle a fork-lift replacement, CADF is a good option.
There are a few gotcha's though. If you have required data that is
outside of the CADF spec, it would need to go in the "attachment"
section of the notification and that still needs a separate schema to
define it. Matt's team is very receptive to extending the spec to
include these special cases though.
Anyway, I've written up all the options (as I see them) [1] with the
advantages/disadvantages of each approach. It's just a strawman, so
bend/spindle/mutilate.
Look forward to feedback!
-S
[1] https://wiki.openstack.org/wiki/NotificationsAndCADF
On 9/3/2014 12:30 PM, Sandy Walsh wrote:
> On 9/3/2014 11:32 AM, Chris Dent wrote:
>> On Wed, 3 Sep 2014, Sandy Walsh wrote:
>>
>>> We're chatting with IBM about CADF and getting down to specifics on
>>> their applicability to notifications. Once I get StackTach.v3 into
>>> production I'm keen to get started on revisiting the notification
>>> format and olso.messaging support for notifications.
>>>
>>> Perhaps a hangout for those keenly interested in doing something about this?
>> That seems like a good idea. I'd like to be a part of that.
>> Unfortunately I won't be at summit but would like to contribute what
>> I can before and after.
>>
>> I took some notes on this a few weeks ago and extracted what seemed
>> to be the two main threads or ideas the were revealed by the
>> conversation that happened in this thread:
>>
>> * At the micro level have versioned schema for notifications such that
>>one end can declare "I am sending version X of notification
>>foo.bar.Y" and the other end can effectively deal.
> Yes, that's table-stakes I think. Putting structure around the payload
> section.
>
> Beyond type and version we should be able to attach meta information
> like public/private visibility and perhaps hints for external mapping
> (this trait -> that trait in CADF, for example).
>
>> * At the macro level standardize a packaging or envelope of all
>>notifications so that they can be consumed by very similar code.
>>That is: constrain the notifications in some way so we can also
>>constrain the consumer code.
> That's the intention of what we have now. The top level traits are
> standard, the payload is open. We really only require: message_id,
> timestamp and event_type. For auditing we need to cover Who, What, When,
> Where, Why, OnWhat, OnWhere, FromWhere.
>
>> These ideas serve two different purposes: One is to ensure that
>> existing notification use cases are satisfied with robustness and
>> provide a contract between two endpoints. The other is to allow a
>> fecund notification environment that allows and enables many
>> participants.
> Good goals. When Producer and Consumer know what to expect, things are
> good ... "I know to find the Instance ID ". When the consumer
> wants to deal with a notification as a generic object, things get tricky
> ("find the instance ID in the payload", "What is the image type?", "Is
> this an error notification?")
>
> Basically, how do we define the principle artifacts for each service and
> grant the consumer easy/consistent access to them? (like the 7-W's above)
>
> I'd really like to find a way to solve that problem.
>
>> Is that a good summary? What did I leave out or get wrong?
>>
> Great start! Let's keep it simple and do-able.
>
> We should also review the oslo.messaging notification api ... I've got
> some concerns we've lost our way there.
>
> -S
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Treating notifications as a contract
On Sep 3, 2014, at 9:50 AM, Sandy Walsh wrote: > Is there anything slated for the Paris summit around this? > > I just spent nearly a week parsing Nova notifications and the pain of no > schema has overtaken me. > > We're chatting with IBM about CADF and getting down to specifics on their > applicability to notifications. Once I get StackTach.v3 into production I'm > keen to get started on revisiting the notification format and olso.messaging > support for notifications. > > Perhaps a hangout for those keenly interested in doing something about this? Julien did start some work on it a while back, but it was shelved for other more pressing things at the time. https://blueprints.launchpad.net/oslo.messaging/+spec/notification-structured https://wiki.openstack.org/wiki/Oslo/blueprints/notification-structured It would be good to start building up a set of requirements in anticipation of a cross-project or Oslo session at the summit. Doug > > Thoughts? > -S > > > From: Eoghan Glynn [egl...@redhat.com] > Sent: Monday, July 14, 2014 8:53 AM > To: OpenStack Development Mailing List (not for usage questions) > Subject: Re: [openstack-dev] [all] Treating notifications as a contract > >>> So what we need to figure out is how exactly this common structure can be >>> accommodated without reverting back to what Sandy called the "wild west" >>> in another post. >> >> I got the impression that "wild west" is what we've already got >> (within the payload)? > > Yeah, exactly, that was my interpretation too. > > So basically just to ensure that the lightweight-schema/common-structure > notion doesn't land us back not too far beyond square one (if there are > too many degrees-of-freedom in that declaration of "a list of dicts with > certain required fields" that you had envisaged in an earlier post). > >>> For example you could write up a brief wiki walking through how an >>> existing widely-consumed notification might look under your vision, >>> say compute.instance.start.end. Then post a link back here as an RFC. >>> >>> Or, possibly better, maybe submit up a strawman spec proposal to one >>> of the relevant *-specs repos and invite folks to review in the usual >>> way? >> >> Would oslo-specs (as in messaging) be the right place for that? > > That's a good question. > > Another approach would be to hone in on the producer-side that's > currently the heaviest user of notifications, i.e. nova, and propose > the strawman to nova-specs given that (a) that's where much of the > change will be needed, and (b) many of the notification patterns > originated in nova and then were subsequently aped by other projects > as they were spun up. > >> My thinking is the right thing to do is bounce around some questions >> here (or perhaps in a new thread if this one has gone far enough off >> track to have dropped people) and catch up on some loose ends. > > Absolutely! > >> For example: It appears that CADF was designed for this sort of thing and >> was considered at some point in the past. It would be useful to know >> more of that story if there are any pointers. >> >> My initial reaction is that CADF has the stank of enterprisey all over >> it rather than "less is more" and "worse is better" but that's a >> completely uninformed and thus unfair opinion. > > TBH I don't know enough about CADF, but I know a man who does ;) > > (gordc, I'm looking at you!) > >> Another question (from elsewhere in the thread) is if it is worth, in >> the Ironic notifications, to try and cook up something generic or to >> just carry on with what's being used. > > Well, my gut instinct is that the content of the Ironic notifications > is perhaps on the outlier end of the spectrum compared to the more > traditional notifications we see emitted by nova, cinder etc. So it > may make better sense to concentrate initially on how contractizing > these more established notifications might play out. > >>> This feels like something that we should be thinking about with an eye >>> to the K* cycle - would you agree? >> >> Yup. >> >> Thanks for helping to tease this all out and provide some direction on >> where to go next. > > Well thank *you* for picking up the baton on this and running with it :) > > Cheers, > Eoghan > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Treating notifications as a contract
> > For example: It appears that CADF was designed for this sort of thing and> > > > was considered at some point in the past. It would be useful to know> > > > more of that story if there are any pointers.> >> > My initial reaction is > > that CADF has the stank of enterprisey all over> > it rather than "less is > > more" and "worse is better" but that's a> > completely uninformed and thus > > unfair opinion.> > TBH I don't know enough about CADF, but I know a man who > > does ;)> > (gordc, I'm looking at you!)** so i was on vacation when this > > thread popped up. i'll just throw a disclaimer, i didn't read the initial > > conversion thread... also, i just read what i typed below and ignore the > > fact it sounds like a sales pitch. ** CADF is definitely a well-defined open standard with contributions from multiple companies so there are a lot of use cases, case and point the daunting 100+ pg spec [1]. the purpose of CADF was to be an auditable event model to describe cloud events (basically what our notifications are in OpenStack). regarding CADF in OpenStack[2], pyCADF has now been moved under the Keystone umbrella to handle auditing. Keystone thus far has done a great job incorporating pyCADF into their notification messages. while the spec is quite verbose, there is a short intro to CADF events and how to define them in the pycadf docs [3]. we also did a talk at the Atlanta summit [4] (apologies for my lack of presentation skills). lastly, i know we previously had a bunch of slides describing/explaining CADF at a highlevel. i'll let ibmers find a copy to post to slideshare or the like. > * At the micro level have versioned schema for notifications such that > one end can declare "I am sending version X of notification > foo.bar.Y" and the other end can effectively deal. the event model has a mandatory typeURI field where you could define a version > These ideas serve two different purposes: One is to ensure that > existing notification use cases are satisfied with robustness and > provide a contract between two endpoints. The other is to allow a > fecund notification environment that allows and enables many > participants. CADF is designed to be extensible so even if a use cases is not specifically defined in spec, the model can be extended to accommodate. additionally, one of the chairs of the CADF spec is also a contributor to pyCADF so there are opportunities to shape the future of the CADF (something we did, while building pyCADF). > Another approach would be to hone in on the producer-side that's > currently the heaviest user of notifications, i.e. nova, and propose > the strawman to nova-specs i'd love for OpenStack to converge on a standard (whether CADF or not). personal experience tells me it'll be difficult, but i think more and more have realised just making the 'wild west' even wilder isn't helping. [1] http://www.dmtf.org/sites/default/files/standards/documents/DSP0262_1.0.0.pdf[2] http://www.dmtf.org/standards/cadf [3] http://docs.openstack.org/developer/pycadf/event_concept.html[4] https://www.openstack.org/summit/openstack-summit-atlanta-2014/session-videos/presentation/an-overview-of-cloud-auditing-support-for-openstack cheers, gord ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Treating notifications as a contract
On 9/3/2014 11:32 AM, Chris Dent wrote:
> On Wed, 3 Sep 2014, Sandy Walsh wrote:
>
>> We're chatting with IBM about CADF and getting down to specifics on
>> their applicability to notifications. Once I get StackTach.v3 into
>> production I'm keen to get started on revisiting the notification
>> format and olso.messaging support for notifications.
>>
>> Perhaps a hangout for those keenly interested in doing something about this?
> That seems like a good idea. I'd like to be a part of that.
> Unfortunately I won't be at summit but would like to contribute what
> I can before and after.
>
> I took some notes on this a few weeks ago and extracted what seemed
> to be the two main threads or ideas the were revealed by the
> conversation that happened in this thread:
>
> * At the micro level have versioned schema for notifications such that
>one end can declare "I am sending version X of notification
>foo.bar.Y" and the other end can effectively deal.
Yes, that's table-stakes I think. Putting structure around the payload
section.
Beyond type and version we should be able to attach meta information
like public/private visibility and perhaps hints for external mapping
(this trait -> that trait in CADF, for example).
> * At the macro level standardize a packaging or envelope of all
>notifications so that they can be consumed by very similar code.
>That is: constrain the notifications in some way so we can also
>constrain the consumer code.
That's the intention of what we have now. The top level traits are
standard, the payload is open. We really only require: message_id,
timestamp and event_type. For auditing we need to cover Who, What, When,
Where, Why, OnWhat, OnWhere, FromWhere.
> These ideas serve two different purposes: One is to ensure that
> existing notification use cases are satisfied with robustness and
> provide a contract between two endpoints. The other is to allow a
> fecund notification environment that allows and enables many
> participants.
Good goals. When Producer and Consumer know what to expect, things are
good ... "I know to find the Instance ID ". When the consumer
wants to deal with a notification as a generic object, things get tricky
("find the instance ID in the payload", "What is the image type?", "Is
this an error notification?")
Basically, how do we define the principle artifacts for each service and
grant the consumer easy/consistent access to them? (like the 7-W's above)
I'd really like to find a way to solve that problem.
> Is that a good summary? What did I leave out or get wrong?
>
Great start! Let's keep it simple and do-able.
We should also review the oslo.messaging notification api ... I've got
some concerns we've lost our way there.
-S
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Treating notifications as a contract
On Wed, 3 Sep 2014, Sandy Walsh wrote: Is there anything slated for the Paris summit around this? There are plans to make plans, but that's about all I know. I just spent nearly a week parsing Nova notifications and the pain of no schema has overtaken me. /me passes the ibuprofen We're chatting with IBM about CADF and getting down to specifics on their applicability to notifications. Once I get StackTach.v3 into production I'm keen to get started on revisiting the notification format and olso.messaging support for notifications. Perhaps a hangout for those keenly interested in doing something about this? That seems like a good idea. I'd like to be a part of that. Unfortunately I won't be at summit but would like to contribute what I can before and after. I took some notes on this a few weeks ago and extracted what seemed to be the two main threads or ideas the were revealed by the conversation that happened in this thread: * At the micro level have versioned schema for notifications such that one end can declare "I am sending version X of notification foo.bar.Y" and the other end can effectively deal. * At the macro level standardize a packaging or envelope of all notifications so that they can be consumed by very similar code. That is: constrain the notifications in some way so we can also constrain the consumer code. These ideas serve two different purposes: One is to ensure that existing notification use cases are satisfied with robustness and provide a contract between two endpoints. The other is to allow a fecund notification environment that allows and enables many participants. Is that a good summary? What did I leave out or get wrong? -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Treating notifications as a contract
Is there anything slated for the Paris summit around this? I just spent nearly a week parsing Nova notifications and the pain of no schema has overtaken me. We're chatting with IBM about CADF and getting down to specifics on their applicability to notifications. Once I get StackTach.v3 into production I'm keen to get started on revisiting the notification format and olso.messaging support for notifications. Perhaps a hangout for those keenly interested in doing something about this? Thoughts? -S From: Eoghan Glynn [egl...@redhat.com] Sent: Monday, July 14, 2014 8:53 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [all] Treating notifications as a contract > > So what we need to figure out is how exactly this common structure can be > > accommodated without reverting back to what Sandy called the "wild west" > > in another post. > > I got the impression that "wild west" is what we've already got > (within the payload)? Yeah, exactly, that was my interpretation too. So basically just to ensure that the lightweight-schema/common-structure notion doesn't land us back not too far beyond square one (if there are too many degrees-of-freedom in that declaration of "a list of dicts with certain required fields" that you had envisaged in an earlier post). > > For example you could write up a brief wiki walking through how an > > existing widely-consumed notification might look under your vision, > > say compute.instance.start.end. Then post a link back here as an RFC. > > > > Or, possibly better, maybe submit up a strawman spec proposal to one > > of the relevant *-specs repos and invite folks to review in the usual > > way? > > Would oslo-specs (as in messaging) be the right place for that? That's a good question. Another approach would be to hone in on the producer-side that's currently the heaviest user of notifications, i.e. nova, and propose the strawman to nova-specs given that (a) that's where much of the change will be needed, and (b) many of the notification patterns originated in nova and then were subsequently aped by other projects as they were spun up. > My thinking is the right thing to do is bounce around some questions > here (or perhaps in a new thread if this one has gone far enough off > track to have dropped people) and catch up on some loose ends. Absolutely! > For example: It appears that CADF was designed for this sort of thing and > was considered at some point in the past. It would be useful to know > more of that story if there are any pointers. > > My initial reaction is that CADF has the stank of enterprisey all over > it rather than "less is more" and "worse is better" but that's a > completely uninformed and thus unfair opinion. TBH I don't know enough about CADF, but I know a man who does ;) (gordc, I'm looking at you!) > Another question (from elsewhere in the thread) is if it is worth, in > the Ironic notifications, to try and cook up something generic or to > just carry on with what's being used. Well, my gut instinct is that the content of the Ironic notifications is perhaps on the outlier end of the spectrum compared to the more traditional notifications we see emitted by nova, cinder etc. So it may make better sense to concentrate initially on how contractizing these more established notifications might play out. > > This feels like something that we should be thinking about with an eye > > to the K* cycle - would you agree? > > Yup. > > Thanks for helping to tease this all out and provide some direction on > where to go next. Well thank *you* for picking up the baton on this and running with it :) Cheers, Eoghan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
