i should start by saying i re-read my subject line and it arguably comes
off aggressive -- i should probably have dropped 'explain' :)
On 28/08/15 01:47 PM, Alec Hothan (ahothan) wrote:
On 8/28/15, 10:07 AM, "gord chung" <g...@live.ca> wrote:
On 28/08/15 12:18 PM, Roman Dobosz wrote:
So imagine we have new versions of the schema for the events, alarms or
samples in ceilometer introduced in Mitaka release while you have all
your ceilo services on Liberty release. To upgrade ceilometer you'll
have to stop all services to avoid data corruption. With
versionedobjects you can do this one by one without disrupting
telemetry jobs.
are versions checked for every single message? has anyone considered the
overhead to validating each message? since ceilometer is queue based, we
could technically just publish to a new queue when schema changes... and
the consuming services will listen to the queue it knows of.
ie. our notification service changes schema so it will now publish to a
v2 queue, the existing collector service consumes the v1 queue until
done at which point you can upgrade it and it will listen to v2 queue.
this way there is no need to validate/convert anything and you can still
take services down one at a time. this support doesn't exist currently
(i just randomly thought of it) but assuming there's no flaw in my idea
(which there may be) isn't this more efficient?
If high performance is a concern for ceilometer (and it should) then maybe
there might be better options than JSON?
JSON is great for many applications but can be inappropriate for other
demanding applications.
There are other popular open source encoding options that yield much more
compact wire payload, more efficient encoding/decoding and handle
versioning to a reasonable extent.
i should clarify. we let oslo.messaging serialise our dictionary how it
does... i believe it's JSON. i'd be interested to switch it to something
more efficient. maybe it's time we revive the msgpacks patch[1] or are
there better alternatives? (hoping i didn't just unleash a storm of
'this is better' replies)
Queue based versioning might be less runtime overhead per message but at
the expense of a potentially complex queue version management (which can
become tricky if you have more than 2 versions).
I think Neutron was considering to use versioned queues as well for its
rolling upgrade (along with versioned objects) and I already pointed out
that managing the queues could be tricky.
In general, trying to provide a versioning framework that allows to do
arbitrary changes between versions is quite difficult (and often bound to
fail).
yeah, so that's what a lot of the devs are debating about right now.
performance is our key driver so if we do something we think/know will
negatively impact performance, it better bring a whole lot more of
something else. if queue based versioning offers comparable
functionalities, i'd personally be more interested to explore that route
first. is there a thread/patch/log that we could read to see what
Neutron discovered when they looked into it?
[1] https://review.openstack.org/#/c/151301/
--
gord
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
