On 08/08/2013 05:46 PM, Jakub Scholz wrote:
Are these AMQP 1.0 related changes documented somewhere?
Not properly yet. I've been sending out some rough notes to the user list (updated version attached). I need to spend some time figuring out where and how to make the information more accessible.
Perhaps while I'm trying to find that time, I should just check it in alongside the other READMEs in the cpp tree? Would anyone object?
AMQP 1.0 support for the qpid::messaging API -------------------------------------------- The guiding principle has been to allow applications written to the qpid::messaging API to speak AMQP 1.0 in a clear and natural way, to avoid tying its use to any particular broker. The 0-10 support will of course remain unaltered. The API is itself fairly simple. It is in the address syntax and specifically the more detailed options that much of the complexity of the mapping lies. Reply-To addresses and temporary queues ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ There has been one minor change to the way the API itself works over 1.0. This does not affect existing 0-10 use however. The change involves the creation of temporary queues (or topics), for retrieving replies in a request-response pattern for example. Over 0-10, the Address will convert a node name starting with a '#' character by inserting a UUID. This works well for 0-10 where the name is chose by clients and must be unique. This transformation of the name is done when constructing an Address from a single address string (rather than from its constituent parts). The modified name could then be accessed via Address::getName(). Over 1.0 however the name for such nodes is determined by the server. In this case the name assigned needs to be communicated back to the application when the attach succeeds. To handle that a new accessor - getAddress() - has been added to both Sender and Receiver. In order to keep backward compatibility for 0-10, the Address constructor still does the transformation, but applications that want to be able to switch to 1.0 should use these new accessors to obtain the correct address for setting reply-to on any request messages they send. (This new approach will work for both 0-10 and 1.0). Connections, Session and Links ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The protocol used is selected at runtime via the 'protocol' connection property. The recognised values are 'amqp1.0' and 'amqp0-10'. AMQP 0-10 is still the default and the 1.0 support is only available if the required module is loaded. There is no failover support for 1.0 connections yet[1]. The SASL negotiation is optional in AMQP 1.0. If no SASL layer is desired, the sasl_mechanisms connection option can be set to NONE. AMQP 1.0 can be used over SSL, however the messaging client does not at this stage use an AMQP negotiated security layer for that prupose. Peers must expect SSL on the port being used (either exclusively or by being able to detect an SSL header). The container id that the client advertises when establishing the connection can be set through the connection-id/connection_id property on the connection. If not set a UUID will be used. Transactional sessions are not yet supported[2]. The creation of senders or receivers results in the attaching of a link to the peer. The details of the attach, in particular the source and/or target, are controlled through the address string. Addresses ~~~~~~~~~ The name specified in the address supplied when creating a sender or receiver is used to set the address of the target or source respectively. If the subject is specified for a sender it is used the default subject for messages sent without an explicit subject set. If the subject is specified for a receiver it is interpreted as a filter on the set of messages of interest. If it includes a wildcard (i.e. a '*' or a '#') it is sent as a legacy-amqp-topic-binding, if not it is sent as a legacy-amqp-direct-binding. When the name of the address is (or starts with) '#', the dynamic flag is set on the corresponding source or target and the dynamic-node-properties are populated based on the node properties. Note that when the dynamic flag is set the address should not be specified. However due to PROTON-277[3], I have to set the address to something in order to work at all against another proton-c based peer, such as qpidd (so I set it to '.'). This can be resolved as soon as the proton bug is fixed. As mentioned above in discussing the changes around reply-to addresses, AMQP 1.0 doesn't allow on demand creation of nodes with a client specified name. However, I have defined a special extension capability for the c++ broker that will allow 'create' behaviour that is similar to that supported over 0-10. That is, it will create a node with the name specified by the client if it does not already exist. I see this as a temporary measure to help transition situations that rely on create policy at present. It is non-standard however and the recommendation would be to look for ways to avoid relying on that. Hopefully at some point a standard management mechanism will provide better alternatives. If the addressed node is to be created on demand - either through use of '#' as the name, or through the create policy - the node properties are sent as dynamic-node-properties on the source or target. These can be specified in a nested map within the node. Additionally the durable and type properties in the node map itself are sent if specified. There is at present also a translation from the 0-10 style x-declare in the node whereby all the fields specified in it are included as if they had been listed in properties, the arguments map is flattened into this. The x-bindings property is not currently supported for AMQP 1.0 in nodes or links. This has really been a question of priorities rather than ruling out any mapping. Though I don't think a generic binding solution is appropriate for the 1.0 implementation, I'm eager to here of use cases that would be affected here and see how best to address them. For receivers, where the node is an exchange, the binding can be controlled through the filters for the link. These can be specified through the filter property in the link properties specified in the address. The value of this should be a list of maps, with each map specifying a filter through key-value pairs for name, descriptor (can be specified as numeric or symbolic) and a value. The subject in the address and the selector property provide shorter ways of specifying the most common use cases. Subscriptions can also be shared between receivers. To do this, the 'shared' capability should be requested and the receivers should all specify the same link name. Note that due to fact that link and container id combination must be unique, a subscription cannot be shared between receivers on the same connection (a problem for JMS 2.0 for example). This will likely be rectified in the future by allowing the link to be identified via the link properties instead of the name. The delete policy is not supported over 1.0 at all. There isn't really any obvious way to map this. It was never terribly well thought through even for AMQP 0-10 in my view. I believe what is more important is expanding/revising the scope of nodes. The 1.0 specification defines the following policies in connection with nodes created in response to establishing the link: delete-on-close, delete-on-no-links, delete-on-no-messages or delete-on-no-links-or-messages. For dynamically created nodes - i.e. those with name '#', which will result in the dynamic flag being set on the source or target, the default is delete-on-close but the others can be chosen by specifying a lifetime-policy in the properties map in the node options. The valid values are delete-on-close, delete-if-unused, delete-if-empty and delete-if-empty-and-unused and these are sent as the AMQP 1.0 specified policy types (as above). There is some support for the assert option, though it is not entirely equivalent to the 0-10 based mechanism. Over AMQP 1.0 it uses the source or target capabilities. The client will set the capabilities it desires, the broker (or other peer) will set the capabilities it can offer and if the assert option is on, then the client will ensure all the capabilities it requested are indeed supported. If assert sis specified it will also ensure that any requested filters are actually in place. The specific capabilities are yet to be standardised. The capabilities sent by the client can be controlled through a nested list within the node map. Note that capabilities are simply strings (symbols in 1.0 to be precise), not name value pairs. If durable is set in the node properties, then a capability of 'durable' will be requested, meaning the node will not lose messages if its volatile memory is lost, and requested if the durable property within the node map is present in the address. If the type is set, then that will also be passed a requested capability e.g. 'queue' meaning the node supports queue-like characteristics (stores messages until consumers claim them and allocates messages between competing consumers), 'topic' meaning the node supports classic pub-sub characteristics. The 'browse' mode is not yet supported due to PROTON-139[4] The Message class and the AMQP 1.0 message format ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The message-id, correlation-id, user-id, subject, reply-to and content-type fields in the properties section of a 1.0 message can all be set or retreived via accessors of the same name on the Message instance. Likewise for the durable, priority and ttl fields in the header section. An AMQP 1.0 message has a delivery-count field (within the header section) for which there is no direct accessor yet on the Message class. However if the value is greater than 0, then the Message::getRedelivered() method will return true. If Message::setRedelivered() is called with a value of true, then the delivery count will be set to 1, else it will be set to 0. The delivery count can also be retrieved or set through the 'x-amqp-delivery-count' pseudo-property. The application-properties section of a 1.0 message is made available via the properties map of the Message class. Likewise on sending a message the properties map is used to poplulate the application-properties section. There are other fields defined in the AMQP 1.0 message format that as yet do not have direct accessors on the Message class. These are made available on received messages via special keys into the properties map, and can be controlled for outgoing messages by setting the properties with the same keys. The format for the keys in general is x-amqp-<field-name>. The keys currently in use are: x-amqp-first-acquirer and x-amqp-delivery-count for the header section, and x-amqp-to, x-amqp-absolute-expiry-time, x-amqp-creation-time, x-amqp-group-id, x-amqp-qroup-sequence and x-amqp-reply-to-group-id for the properties section. In addition the delivery- and message- annotations sections are made available via a nested maps with key x-amqp-delivery-annotations and x-amqp-message-annotations respectively. AMQP 1.0 support in qpidd ------------------------- To enable 1.0 support in qpidd, the amqp module must be loaded. This allows the broker to recognise the 1.0 protocol header alongside the 0-10 one. The broker can accept connections with or without and underlying SASL security layer as defined by the 1.0 specification. However if authentication is turned on -- i.e. auth=yes -- then a SASL security layer MUST be used. There is as yet no AMQP 1.0 specific integration with the ACL. This means for example that you cannot prevent an authenticated user for subscribing to a particular queue, or limit the number of 1.0 based connections opened up by such a user. This will be rectified in a future release[5]. The broker allows links in both directions to be attached to queues or exchanges. The address in the source or target is resolved by checking whether it matches the name of a queue or an exchange. If there exists a queue and an exchange of the same name, the queue is used but a warning is logged. If the node is an exchange, then an outgoing link (i.e. messages to travel out from broker) will cause a temporary, link-scoped queue to be created on the broker and bound to the exchange. Outgoing links may have a filter set on their source. The filters currently supported by the broker are 'legacy-amqp-direct-binding', 'legacy-amqp-topic-binding', 'legacy-amqp-headers-binding', 'selector-filter' and 'xquery-filter' as defined in the registered extensions. The 'selector-filter' is supported for all nodes. The 'legacy-amqp-direct-binding' and 'legacy-amqp-topic-binding' filters are supported when the node is a topic, direct or xml exchange - where it is used as the binding key when binding the subscription queue to the exchange - and when the node is a queue, where the subscription will then skip over messages that do not match that subject. In the case where 'legacy-amqp-topic-binding' was requested for a direct- or xml- exchange it will be interpreted as a direct binding (and this will be indicated via the filters set on the attach sent by the broker). The 'legacy-amqp-headers-binding' is supported where the node is a headers exchange and implemented as a binding to the exchange. The 'xquery-filter' is supported where the node is an xml exchange and provides the xquery to use in the binding. This can be used in conjunction with a 'legacy-amqp-direct-binding' in order to control the binding ley used for the binding. If the dynamic flag is set on the source or target, then the dynamic-node-properties are inspected to determine the characteristics of the node to be created. The properties that are recognised are the same as accepted via the create qmf method; the 0-10 defined options durable, auto-delete, alternate-exchange, exchange-type and then any qpidd specific options (e.g. qpid.max-count). The broker supports all standard values for the standard lifetime-policy property also. The supported-dist-modes property as defined by the 1.0 specification is used to determine whether a queue or exchange is desired (where the create method uses the 'type' property). If 'move' is specified a queue will be created, if 'copy' is specified an exchange will be created. If this property is not set, then a queue is assumed. Messages sent over AMQP 0-10 will be converted by the broker for sending over AMQP 1.0 and vice versa. The message-id, correlation-id, userid, content-type and content-encoding all map fairly simply in both directions between the properties section in 1.0 and the message-properties in an 0-10 header. Note however that in 0-10 a message-id must be a UUID, and in translating to 0-10 from 1.0 this field will be skipped unless it is a valid UUID. Likewise the priority directly between the field in the header section of a 1.0 message and the corresponding field in the delivery-properties of an 0-10 message. The durable header in a 1.0 message is equivalent to the delivery-mode in the delivery-properties of an 0-10 message, with a value of true in the former being equivalent to a value of 2 in the latter and a value of false in the former equivalent to 1 in the latter. When converting from 0-10 to 1.0, the reply-to will be simply the routing-key if the exchange is not set, or if it is the reply-to address for 1.0 will be composed of the exchange and if specified the routing key (separated by a forward slash). When converting from 1.0 to 0-10, if the address contains a forward slash it is assumed to be of the form exchange/routing key. If not it is assumed to be a simple node name. If that name matches an existing queue, then the resulting 0-10 reply to will have the exchange empty and the routing key populated with the queue name. If not, but the name matches and exchange, then the reply to will have the exchange populated with the node name and the routing key left empty. If the node refers to neither a known queue nor exchange then the resulting reply to will be empty. When converting from 0-10 to 1.0, if the 0-10 message has a non-empty destination, then the subject field in the properties of the 1.0 message is set to the value of the routing-key from the message-properties of the 0-10 message. In the reverse direction, the subject field of the properties section of the 1.0 message is used to populate the routing-key in the message-properties of the 0-10 message. The destination of a 0-10 message is used to populate the 'to' field of the properties section when converting to 1.0, but the reverse translation is not done (as the destination for messages sent out by the broker is controlled by the subscription in 0-10). The application-properties section of a 1.0 message is converted to the application-headers field in the message-properties of an 0-10 message and vice versa. No translation of the message body is done at this point[6]. The broker recognises particular capabilities for source and targets. It will only include a given capability in the attach it sends if that has been 'requested' in the attach sent by the peer to trigger the establishment of the link. This gives the peer some means of ensuring their expectations will be met. The 'shared' capability allows subscriptions from an exchange to be shared by multiple receivers. The 'durable' capability will be added if the queue or exchange refered to by the source or target is durable. The 'queue' capability will be added if the source or target references a queue. The 'topic' capability will be added if the source or target references an exchange. If the source or target references a queue or a direct exchange the 'legacy-amqp-direct-binding' will be added. If it references a queue or a topic exchange, 'legacy-amqp-topic-binding' will be added. The 'create-on-demand' capability is currently offered as a temporary extension offering some continuity from the commonly used 'create' policy in the messaging client. If set in the client and the named node does not exist, it will be created using the dynamic-node-properties as when the dynamic flag is set (see above). [1] https://issues.apache.org/jira/browse/QPID-4708 [2] https://issues.apache.org/jira/browse/QPID-4710 [3] https://issues.apache.org/jira/browse/PROTON-277 [4] https://issues.apache.org/jira/browse/PROTON-139 [5] https://issues.apache.org/jira/browse/QPID-4712 [6] https://issues.apache.org/jira/browse/QPID-4711
--------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
