Hi Matt, Comments inline.
On Mon, Nov 21, 2016 at 7:02 PM, Matt Pavlovich <[email protected]> wrote: > Martyn- > > I think you nailed it here-- well done =) > > My notes in-line-- > > On 11/21/16 10:45 AM, Martyn Taylor wrote: > >> 1. Ability to route messages to queues with the same address, but >> different >> routing semantics. >> >> The proposal outlined in ARTEMIS-780 outlines a new model that introduces >> an address object at the configuration and management layer. In the >> proposal it is not possible to create 2 addresses with different routing >> types. This causes a problem with existing clients (JMS, STOMP and for >> compatability with other vendors). >> >> Potential Modification: Addresses can have multiple routing type >> “endpoints”, either “multicast” only, “anycast” only or both. The example >> below would be used to represent a JMS Topic called “foo”, with a single >> subscription queue and a JMS Queue called “foo”. N.B. The actual XML is >> just an example, there are multiple ways this could be represented that we >> can define later. >> >> <*addresses*> <*address* *name**="foo"*> <*anycast*> >> <*queues*> <*queue* *name**="**foo”* /> </*queues*> >> </*anycast*> <*mulcast*> <*queues*> >> <*queue* *name**="my.topic.subscription" */> </*queues*> >> </*multicast*> </*address*></*addresses*> >> > I think this solves it. The crux of the issues (for me) boils down to > auto-creation of destinations across protocols. Having this show up in the > configs would give developers and admins more information to troubleshoot > the mixed address type+protocol scenario. > > 2. Sending to “multicast”, “anycast” or “all” >> >> As mentioned earlier JMS (and other clients such as STOMP via prefixing) >> allow the producer to identify the type of end point it would like to send >> to. >> >> If a JMS client creates a producer and passes in a topic with address >> “foo”. Then only the queues associated with the “multicast” section of the >> address. A similar thing happens when the JMS producer sends to a “queue” >> messages should be distributed amongst the queues associated with the >> “anycast” section of the address. >> >> There may also be a case when a producer does not identify the endpoint >> type, and simply sends to “foo”. AMQP or MQTT may want to do this. In this >> scenario both should happen. All the queues under the multicast section >> get >> a copy of the message, and one queue under the anycast section gets the >> message. >> >> Modification: None Needed. Internal APIs would need to be updated to allow >> this functionality. >> > I think the "deliver to all" scenario should be fine. This seems analogous > to a CompositeDestination in ActiveMQ 5.x. I'll map through some scenarios > and report back any gotchas. > > 3. Support for prefixes to identify endpoint types >> >> Many clients, ActiveMQ 5.x, STOMP and potential clients from alternate >> vendors, identify the endpoint type (in producer and consumer) using a >> prefix notation. >> >> e.g. queue:///foo >> >> Which would identify: >> >> <*addresses*> <*address* *name**="foo"*> <*anycast*> >> <*queues*> <*queue* *name**="my.foo.queue" */> >> </*queues*> </*anycast*> </*address*></*addresses*> >> >> Modifications Needed: None to the model. An additional parameter to the >> acceptors should be added to identify the prefix. >> > Just as a check point in the syntax+naming convention in your provided > example... would the name actually be: > > <*queue* *name**="foo" .. vs "my.foo.queue" ? > The queue name can be anything. It's the address that is used by consumer/producer. The protocol handler / broker will decided which queue to connect to. > > 4. Multiple endpoints are defined, but client does not specify “endpoint >> routing type” when consuming >> >> Handling cases where consumers does not pass enough information in their >> address or via protocol specific mechanisms to identify an endpoint. Let’s >> say an AMQP client, requests to subscribe to the address “foo”, but passes >> no extra information. In the cases where there are only a single endpoint >> type defined, the consumer would associated with that endpoint type. >> However, when both endpoint types are defined, the protocol handler does >> not know whether to associate this consumer with a queue under the >> “anycast” section, or whether to create a new queue under the “multicast” >> section. e.g. >> >> Consume: “foo” >> >> <*addresses*> >> >> <*address* *name**="foo"*> <*anycast*> <*queues*> >> <*queue* *name**="**foo”* /> </*queues*> >> </*anycast*> <*multicast*> <*queues*> <*queue* >> *name**="my.topic.subscription" */> </*queues*> >> </*multicast*> </*address*></*addresses*> >> >> In this scenario, we can make the default configurable on the >> protocol/acceptor. Possible options for this could be: >> >> “multicast”: Defaults multicast >> >> “anycast”: Defaults to anycast >> >> “error”: Returns an error to the client >> >> Alternatively each protocol handler could handle this in the most sensible >> way for that protocol. MQTT might default to “multicast”, STOMP “anycast”, >> and AMQP to “error”. >> > > Yep, this works great. I think there are two flags on the acceptors.. one > for auto-create and one for default handling of name collision. The > defaults would most likely be the same. > > Something along the lines of: > auto-create-default = "multicast | anycast" > no-prefix-default = "multicast | anycast | error" > > 5. Fully qualified address names >> >> This feature allows a client to identify a particular address on a >> specific >> broker in a cluster. This could be achieved by the client using some form >> of address as: >> >> queue:///host/broker/address/ >> >> Matt could you elaborate on the drivers behind this requirement please. >> >> I am of the opinion that this is out of the scope of the addressing >> changes, and is more to do with redirecting in cluster scenarios. The >> current model will support this address syntax if we want to use it in the >> future. >> > I agree that tackling the impl of this should be out-of-scope. My > recommendation is to consider it in addressing now, so we can hopefully > avoid any breakage down the road. > > A widely used feature in other EMS brokers (IBM MQ, Tibco EMS, etc) is the > ability to fully address a destination using a format similar to this: > > queue://brokerB/myQueue > The advantage of this is to allow for scaling of the number of destinations > and allows for more dynamic broker networks to be created without > applications having to have connection information for all brokers in a > broker network. Think simple delivery+routing, and not horizontal scaling. > It is very analogous to SMTP mail routing. > > Producer behavior: > > 1. Client X connects to brokerA and sends it a message addressed: > queue://brokerB/myQueue > 2. brokerA accepts the message on behalf of brokerB and handles all > acknowledgement and persistence accordingly > 3. brokerA would then store the message in a "queue" for brokerB. Note: > All messages for brokerB are generally stored in one queue-- this is how it > helps with destination scaling > > Broker to broker behavior: > > There are generally two scenarios: always-on or periodic-check > > In "always-on" > 1. brokerA looks for a brokerB in its list of cluster connections and then > sends all messages for all queues for brokerB (or brokerB pulls all > messages, depending on cluster connection config) > > In "periodic-check" > 1. brokerB connects to brokerA (or vice-versa) on a given time interval > and then receives any messages that have arrived since last check > > TL;DR; > > It would be cool to consider remote broker delivery for messages while > refactoring the address handling code. This would bring Artemis inline with > the rest of the commercial EMS brokers. The impact now, hopefully, is minor > and just thinking about default prefixes. > Understood, from our conversations on IRC I can see why this might be useful. > > Thanks, > -Matt > > >
