Now, to answer your actual questions... See inline below.
On 10/09/2013 02:22 PM, Fraser Adams wrote:
Hey all,
The thread below on the dev list has prompted me to ask something that
I've tentatively mentioned before, but am still a bit embarrassed to
raise 'cause it probably makes me seem a bit stupid :-( here goes
anyway.....
So I've kind of held off going down the AMQP 1.0 path partly due to
lack of time, but also partly due to lack of understanding of how it
"all hangs together", the new website helps a bit - but TBH I'm still
left scratching my head somewhat.
I'll try to explain:
Now I know that Proton is intended to be a component usable beyond
just the Qpid "product set", but there's a "protocol engine" and a
"messenger API" and I'm not even that clear on the relationship
between the two of those - for example could one use the protocol
engine completely independently (is there an engine API?) or is the
messenger API intended to be the lowest "unit of currency", what would
be the benefit using the raw engine?
Proton is a multi-layered thing. There is an "engine" API, but it's
neither documented nor stabilized. The Messenger API is built on top of
the engine API and is better documented. There's also a separate driver
component in Proton for integration into a particular run-time environment.
The benefit of using the raw engine API is that it gives detailed access
to the specific parts of the AMQP protocol. If you need to work
directly with connections, sessions, links and deliveries, the engine
API is what you need.
Dispatch is built on the engine API (the C language one) and bypasses
all the services offered by Messenger. There's a layer within Dispatch
that provides multi-threaded access to engine (which is by itself,
non-thread-safe).
Then beyond that there's the relationship with say qpidd and
qpid::messaging. Now I'm aware that when the Proton libraries are
detected qpidd and qpid::messaging get built with Proton support, I'm
"guessing" that in that case the relationship analogous to that of
qpid::client where qpid::client was the low level AMQP speaking API
and qpid::messaging provides a higher level abstraction, so I *think*
that's the relationship with Proton there - but I'm not sure? Is the
proton API close to the AMQP 1.0 specification in say the way that
qpid::client was?
This is sort of true. The qpid::client API was the original one and it
was tied pretty tightly to the protocol. The qpid::messaging APIs (both
C++ and Python) were created prior to AMQP 1.0 for the purpose of being
forward compatible to the new protocol. It is simply an implementation
detail that qpid::messaging is layered on qpid::client.
Messenger and qpid::messaging are peers. They are alternative APIs for
messaging. Messenger is more abstract in that it deals with messages
and hides connections, sessions, links, etc. Qpid::messaging is closer
to JMS in its design.
But then there's more nuance, so I'm aware that with AMQP 1.0 there's
a more peer-to-peer relationship and indeed the Proton tests seem to
have msgr-recv and msgr-send talking directly to each other without a
broker. So that leads me to ask the question what's the relationship
with the broker - in other words what services are provided in
messenger, what are enhanced in qpid::messaging and what are layered
on top of that via the broker (and how does the addressing and routing
work?).
This is a pretty big question... AMQP 1.0 is a point-to-point,
symmetric protocol. Because of this, there is no requirement for a
broker. It allows for the full semantics of message exchange to occur
between endpoints regardless of whether one (or both) of them is a broker.
Regarding addressing and routing. This is a topic of research. Dispatch
is one research vehicle for these topics. It takes a particular
approach to addressing and routing that may be different from other
approaches. We should explore them all.
Some examples of where I'm befuddled include how does subscription
work at a peer to peer level? For example I think that exchange nodes
are only something I've heard discussed in the context of qpidd and
similarly I think the same is true of message selectors, so does
Proton only provide low level network connectivity and data
serialisation (and possibly single client queue) and all the other
stuff needed for connecting a network of clients are part of the
broker services.
Pub-sub in a brokerless environment is another topic of research.
Dispatch is attempting to solve this problem. I expect others will as well.
I suppose what I'm really asking is what "services" are provided at
each "layer" of the Qpid "stack" - clearly you can do useful stuff
with just Proton - but what stuff and what are the limits? What would
you then get from qpid:messaging and what then does the broker throw
into the mix. Are there any diagrams that illustrate this sort of
relationship?
I think the Qpid "stack" is still being envisioned. I see a network of
routers interconnecting brokers and clients. Communication patterns are
varied and use brokers only when Queueing (capital Q as opposed to
merely buffering) and persistence are required. The brokers and routers
are both optional and can be deployed if needed.
The dispatch router adds yet more nuance into the mix. From my
(limited) understanding it seems to offer at least some of the same
services as the broker - but I'm not quite sure what. In my case I've
got a very large federated topology and I have lots of left hand
systems feeding in to fewer systems towards the right. Given that it's
only on the right hand side broker that I have lots of consumers doing
complex subscriptions and the rest of the brokers are employing fairly
simple queue routes I'm thinking that the dispatch router could
ultimately be something to "tidy up" the left hand side of my system -
but I'm not quite sure.
I think your description is correct. Dispatch is for routing. The
broker is for "brokering". Dispatch will never accept a message itself,
only pass it through to an endpoint or broker that can take ownership of
the message.
Apologies if these seem silly questions, I'm sure that the answers are
obvious to those who've been involved at the architectural stages, but
ultimately from my perspective the overall holistic architecture isn't
totally clear.
Even at a basic level I've not actually noticed anything in the
programming book
http://qpid.apache.org/releases/qpid-0.24/programming/book/index.html
that seems to mention even how to connect via AMQP 1.0 vice 0.10. I
think that it has been mentioned on the mailing list by Gordon so I'm
sure I could dig the info out, but is it missing from the docs (or am
I just not looking hard enough). On a similar note for Proton the
msgr-send and msgr-recv examples are fine as far as it goes, but I'm
thinking that to figure out how to do anything more complex my best
bet is likely to be to "reverse engineer" the qpid::messaging bindings
- I can't see anything obvious for how to send a map for example. I'm
guessing that Proton is just as erm "nuanced" as qpid::client, so
really powerful and flexible, but you have to know what you're doing
to get the best (say performance) out of it, the API documentation
looks pretty decent to be fair but I'm not sure that's enough to help
me drive it really effectively.
If you set the connection-option of "protocol" to "amqp1.0", you will
enable the new protocol.
On top of that there seems to be a growing number of JMS clients,
there's the original AMQP 0.10, there's an AMQP 1.0 one in the main
Qpid tree and there's a separate Proton based AMQP 1.0 one that's a
separate component (in a similar vein to Proton). I can see that the
increased modularisation is a good thing and I assume that at some
point the original AMQP 1.0 JMS client will be deprecated in favour of
the Proton based one, but at the moment it's all a bit confusing
without anything that describes the relationship between then. I'm
gleaning what little knowledge I have out of a range of threads on the
mailing list and I've probably missed something.
Hopefully there will be fewer JMS clients in the future. The 0-*
versions will be legacy and eventually become obsolete. The "prototype"
client for AMQP 1.0 will be deprecated once the proton-based one is
complete.
I'm sorry if this comes across in any way as critical in email form,
it's really not intended to, I'm just keen to finally make a proper
start on my AMQP 1.0 journey and to be honest I feel a little out of
my depth at the moment :-(
Blame Ted for prompting me to write this ;->
Cheers,
Frase
On 09/10/13 17:20, Rob Godfrey wrote:
Hi Ted,
I think before we make this a full sub project, it would be good to have
clarity on exactly the proposed scope of Dispatch, how it is expected to
interact with other components within Qpid, or within wider AMQP
networks.
I think in retrospect we didn't do this clearly enough with Proton (for
example).
Moreover I would personally like to understand which AMQP standards
it will
be looking to implement, and which not. For instance I notice this
line in
the docs for Dispatch:
*Address**Description* /_local/agentThe management agent on the attached
router/container. This address would be used by an endpoint that is a
management client/console/tool wishing to access management data from
the
attached container.
Which doesn't seem to conform with the proposed management specification
for AMQP, nor does the document make any mention of how dispatch is
to be
managed.
Cheers,
Rob
On 9 October 2013 17:22, Ted Ross <tr...@redhat.com> wrote:
The AMQP Router project (Qpid Dispatch, announced previously on the
user
list) is gaining in community interest and is nearing the point where a
first release is appropriate. In preparation for a release, I
proposethat
this sub-project follow the lead of both Proton and the AMQP1.0 JMS
projects. This involves:
1. Moving the code from qpid/extras to
http://svn.apache.org/repos/**asf/qpid/dispatch<http://svn.apache.org/repos/asf/qpid/dispatch>
,
2. Requesting, by vote, the creation of a JIRA project to track its
issues and releases.
Unless there are objections, I will move forward with the above two
tasks.
Regards,
-Ted
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org
