On 12/09/2013 04:10 PM, Russell Bryant wrote:
From looking it appears that RabbitMQ's support is via an experimental
plugin. I don't know any more about it. Has anyone looked at it in
detail?
I believe initial support was added in 3.1.0:
http://www.rabbitmq.com/release-notes/README-3.1.0.txt
I have certainly successfully tested basic interaction with RabbitMQ
over 1.0.
https://www.rabbitmq.com/specification.html
As I understand it, "Qpid" supports it, in that it's a completely new
implementation as a library (Proton) under the Qpid project umbrella.
There's also a message router called Dispatch. This is not to be
confused with the existing Qpid libraries, or the existing Qpid broker
(qpidd).
Yes, proton is a library that encapsulates the AMQP 1.0 encoding and
protocol rules. It is used in the existing native broker (i.e. qpidd) to
offer 1.0 support (as well as the qpid::messaging c++ client library).
In addition there is as you mention the 'Dispatch Router'. This is an
alternative architecture for an intermediary to address some of the
issues that can arise with qpidd or other brokers (distributed in nature
to scale better, end-to-end reliability rather than store and forward etc).
So the Qpid project offers both new components as well as 1.0 support
and smooth transition for existing components. (Disclosure, I'm a
developer in the Qpid community also).
(There are of course other implementations also e.g., ActiveMQ,
ApolloMQ, HornetQ, Microsoft ServiceBus, SwiftMQ.)
http://qpid.apache.org/proton/
http://qpid.apache.org/components/dispatch-router/
[...]
All of this sounds fine to me. Surely a single driver for multiple
systems is an improvement. What's not really mentioned though is why
we should care about AMQP 1.0 beyond that. Why is it architecturally
superior? It has been discussed on this list some before, but I
figure it's worth re-visiting if some code is going to show up soon.
Personally I think there is benefit to having a standardised, open
wire-protocol as the basis for communication in systems like OpenStack,
rather than having the driver tied to a particular implementation
throughout (and having more of the key details of the interaction as
details of the implementation of the driver). The bytes over the wire
are another level of interface and having that tightly specified can be
valuable.
Having one driver that still offers choice with regard to intermediaries
used (I avoid the term broker in case it is implies particular
approaches), is I think an advantage. Hypothetically for example it
would have been an advantage had the same driver been usable against
both RabbitMQ and Qpid previously. The (bumpy!) evolution of AMQP meant
that wasn't quite possible since they both spoke different versions
ofthe early protocol. AMQP 1.0 might in the future avoid needing new
drivers in such cases however, making it easier to adopt alternative or
emerging solutions.
AMQP is not the only messaging protocol of course, However its general
purpose nature and the fact that both RabbitMQ and Qpid really came
about through AMQP makes it a reasonable choice.
In the case of Nova (and others that followed Nova's messaging
patterns), I firmly believe that for scaling reasons, we need to move
toward it becoming the norm to use peer-to-peer messaging for most
things. For example, the API and conductor services should be talking
directly to compute nodes instead of through a broker.
Is scale the only reason for preferring direct communication? I don't
think an intermediary based solution _necessarily_ scales less
effectively (providing it is distributed in nature, which for example is
one of the central aims of the dispatch router in Qpid).
That's not to argue that peer-to-peer shouldn't be used, just trying to
understand all the factors.
One other pattern that can benefit from intermediated message flow is in
load balancing. If the processing entities are effectively 'pulling'
messages, this can more naturally balance the load according to capacity
than when the producer of the workload is trying to determine the best
balance.
The exception
to that is cases where we use a publish-subscribe model, and a broker
serves that really well. Notifications and notification consumers
(such as Ceilometer) are the prime example.
The 'fanout' RPC cast would perhaps be another?
In terms of existing messaging drivers, you could accomplish this with
a combination of both RabbitMQ or Qpid for brokered messaging and
ZeroMQ for the direct messaging cases. It would require only a small
amount of additional code to allow you to select a separate driver for
each case.
Based on my understanding, AMQP 1.0 could be used for both of these
patterns. It seems ideal long term to be able to use the same
protocol for everything we need.
That is certainly true. AMQP 1.0 is fully symmetric so it can be used
directly peer-to-peer as well as between intermediaries. In fact, apart
from the establishment of the connection in the first place, a process
need not see any difference in the interaction either way.
We could use only ZeroMQ, as well. It doesn't have the
publish-subscribe stuff we need built in necessarily. Surely that has
been done multiple times by others already, though. We could build it
too, if we had to.
Indeed. However the benefit of choosing a protocol is that you can use
solutions developed outside OpenStack or any other single project.
Can you (or someone) elaborate further on what will make this solution
superior to our existing options?
Superior is a very bold claim to make :-) I do personally think that an
AMQP 1.0 based solution would be worthwhile for the reasons above. Given
a hypothetical choice between say the current qpid driver and one that
could talk to different back-ends, over a standard protocol for which
e.g. semantic monitoring tools could be developed and which would make
reasoning about partial upgrades or migrations easier, I know which I
would lean to. Obviously that is not the choice here, since one already
exists and the other is as yet hypothetical. However, as I say I think
this could be a worthwhile journey and that would justify at least
taking some initial steps.
--Gordon.
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
