On 18 May 2016 at 14:24, Alex Szczuczko <aszcz...@redhat.com> wrote: > > Adding translation isn't *that* hard - it'd take me a few hours I > guess... > > The issue is more that there don't seem to be any (even de-facto) > standards > > for content type when encoding maps in 0-9-1. A quick google shows that > > people talking from one RabbitMQ focussed client to another have been > told > > to use JSON or similar to serialize their data. > > > > I agree that not having standard content encodings for structured data > was > > a fault in AMQP 0.x versions - and one that we fixed in AMQP 1.0 (and for > > the message translation in the Java Broker, amqp maps will be turned into > > AMQP 1.0 map data and vice versa). > > Ah, ok! I stopped looking into AMQP 1.0 after I read it moved queue and > exchange management outside the protocol (that was already hard enough to > understand in the older versions), but maybe I should revisit. > > > I'll have a think about the best way to implement something here for the > > Java Broker so that it can work out if a) the client is likely to be able > > to understand the message content encoding and then b) how to translate > the > > message into something the receiving client can understand. > > The main difficulty I saw was the potential for undesired broker > interference, if it translated some content that the user was expecting > verbatim. If you think it could work however, that's great. > > Yeah - I'm thinking any intervention should be minimal - basically pick up on content types that are likely to be hard to understand (e.g. amqp 0-10 maps) and if the client is known to not understand them, translate to something they might have a better hope of understanding. I'll need to do a bit of research into how different clients populate their identification strings in the initial connection establishment so that I can properly key off that in order to only do the desired amount of translation (as above, for the Qpid JMS client of 0-9-1/0-10 we wouldn't want to translate the message on the broker as the client can understand them).
We should also put up a page somewhere detailing the known content types clients are using to describe AMQP 0-x encoded data. Cheers, Rob > Alex > > > > > -- Rob > > > > On 18 May 2016 at 13:45, Alex Szczuczko <aszcz...@redhat.com> wrote: > > > > > Rob, > > > > > > I see your reply in the list archive, but since I'm not subscribed (not > > > interested other than for this thread), and a direct reply wasn't > copied to > > > me, I will have to break threading (hopefully not too much) to reply to > > > you. > > > > > > > On 17 May 2016 at 18:59, Alex Szczuczko <aszcz...@redhat.com> wrote: > > > > > Hi, > > > > > > > > > > I'm trying to send a message from a Qpid 0.32 Python client (which > uses > > > > > protocol 0-10), to Qpid Java Broker 6.0.2, and receive it with a > 0-9-1 > > > > > protocol client (amqpy 0.12.4). The message content/body is a > Python > > > > > dictionary, which is transparently encoded as a AMQP 0-10 map[1] > by the > > > > > sending client. Instead of an AMQP 0-9-1 field-table[2], the > receiving > > > > > client gets the untranslated AMQP 0-10 map, which it can only > present > > > as a > > > > > raw byte string. > > > > > > > > > > Is the "translates among all versions of AMQP" feature[3] of the > Java > > > > > Broker supposed to include conversion of message bodies? Looking at > > > > > MessageConverter_0_10_to_0_8.java[4] it seems to only convert > > > > > headers/properties, and the body is just passed through unchanged. > This > > > > > doesn't meet my impression of what is implied by the feature > > > description. > > > > > > > > This is an interesting issue. The problem here is that both AMQP > 0-9-1 > > > > and AMQP 0-10 are silent on the issue of message encoding - both > simply > > > > define a transport for opaque binary messages, with properties (and > > > > particularly the content-type property) being used to indicate to the > > > > ultimate recipient the nature of the message being transmitted. > Since > > > > sending structured data (e.g. maps) is such a natural thing to want > to > > > do, > > > > and since each version of AMQP supports a type system which includes > a > > > > representation of these structured types, it is natural that > > > > clients/applications have defined their own MIME types to represent > these > > > > encodings, and used the native type systems of AMQP to transport the > > > data. > > > > The issue is that these MIME types are not standardised and not > > > necessarily > > > > common between clients. To take a concrete example, the Qpid Java > JMS > > > > client for AMQP 0-9-1 does not use FIeldTable to represent JMS Map > > > > Messages, and would not know how to present a field table encoded > message > > > > through its JMS API. On the other hand it is perfectly able to > decode > > > the > > > > AMQP 0-10 map encoded message content because it understands that > MIME > > > type. > > > > > > I suspected it might be something like that. The lack of standard > content > > > types is disappointing, and takes away a lot of AMQP's value in my > opinion. > > > > > > > > > > > I think what we need here is a way to indicate for each client which > MIME > > > > types it "accepts" and to introduce message content translation > between > > > > MIME types so that we can convert from types not understood by a > > > particular > > > > recipient into types that are understood. This is may even occur > > > between a > > > > sender and receiver of the same protocol (see the example above > where the > > > > Qpid JMS client would require translation of the FieldTable map > message, > > > or > > > > the JMS client may send and 0-10 encoded message over and 0-9-1 > > > connection). > > > > > > > > I'll raise a JIRA for this presently - as a start can you tell me the > > > > content type that the amqpy client is using when it is sending > messages > > > > containing field tables? > > > > > > The example I gave was the opposite: Qpid Python (sender) to amqpy > > > (receiver), but I've attached Wireshark packet dumps for both > directions > > > (which include the dissected content-types). > > > > > > It seems that amqpy as a sender doesn't do anything transparently, but > > > I've used its AMQPWriter to translate the Python dict into a > field-table. > > > The content-type it gives is blank, and I can't see a conventional > > > equivalent to amqp/map for field tables. So, content translation may > not be > > > possible with amqpy as a sender, unless the broker inspected the > content > > > independently. > > > > > > Adding content translation looks very difficult, so maybe all that can > be > > > done is to update the features list and/or documentation to disclaim > that? > > > > > > Alex > > > > > > > > > > > Thanks, > > > > Rob > > >
