apologies, if the below was mentioned at some point in this thread.
On 04/11/2015 10:42 AM, Sean Dague wrote:
This seems like a fundamental abuse of HTTP honestly. If you find
yourself creating a ton of new headers, you are probably doing it wrong.
if we want to explore the HTTP path, did we consider using ETags[1] to
check whether resources have changed? it's something used by Gnocchi's
API to handle resource changes.
I do think the near term work around is to actually use Searchlight.
They're monitoring the notifications bus for nova, and refreshing
resources when they see a notification which might have changed it. It
still means that Searchlight is hitting our API more than ideal, but at
least only one service is doing so, and if the rest hit that instead
they'll get the resource without any db hits (it's all through an
elastic search cluster).
I think longer term we probably need a dedicated event service in
OpenStack. A few of us actually had an informal conversation about this
during the Nova notifications session to figure out if there was a way
to optimize the Searchlight path. Nearly everyone wants websockets,
which is good. The problem is, that means you've got to anticipate
10,000+ open websockets as soon as we expose this. Which means the stack
to deliver that sanely isn't just a bit of python code, it's also the
highly optimized server underneath.
as part of the StackTach integration efforts, Ceilometer (as of Juno)
listens to all notifications in the OpenStack ecosystem and builds a
normalised event model[2] from it. the normalised event data is stored
in a backend (elasticsearch, sql, mongodb, hbase) and from this you can
query based on required attributes. in addition to storing events, in
Liberty, Aodh (alarming service) added support to take events and create
alarms based on change of state[3] with expanded functionality to be
added. this was added to handle the NFV use case but may also be
relevant here as it seems like we want to have an action based on status
changes.
i should mention that we discussed splitting out the event logic in
Ceilometer to create a generic listener[4] service which could convert
notification data to meters, events, and anything else. this isn't a
high priority item but might be an integration point for those looking
to leverage notifications in OpenStack.
[1] https://en.wikipedia.org/wiki/HTTP_ETag
[2] http://docs.openstack.org/admin-guide-cloud/telemetry-events.html
[3]
http://specs.openstack.org/openstack/ceilometer-specs/specs/liberty/event-alarm-evaluator.html
[4] https://etherpad.openstack.org/p/mitaka-telemetry-split
cheers,
--
gord
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
