Dear Gordon, thanks for great and clarifying input!
We have decided to do a PoC with the broker instead of the router to see how things work out. We'll post our findings (and questions ;) ) afterwards. When you say that there are plans to let the routers works alongside brokers, how would that work in practise? Would the messenger deliver its messages to the broker directly, then forward them to it's local router? It seems like the broker and router has some overlapping functionality --- Would it make sense to merge these to projects into one in the long run? And lastly, as you say that AMQP supports resume by the protocol, do you know of any plans to actually implement it at some level? It would be a quite important feature for low-bandwidth networks, where, due to cost and speed, it is essential to lower the actual number of bytes transferred as far as possible... Best regards, Tor Rune Skoglund, t...@swi.no Den 24. feb. 2014 18:42, skrev Gordon Sim: > On 02/24/2014 12:51 PM, Tor Rune Skoglund wrote: >> Hello list, >> >> we are setting up an AMQP messaging infrastructure using Qpid (currently >> using proton and the dispatch router, not the broker). This is the basic >> set-up: >> >> * We have a server side, with a public, known IP-address and port. >> * We have X number of clients going on/off the net at unpredictable >> times and intervals. These clients may be NAT'ed (on possible separate >> LANs), and might change public and local IP-address at random. >> * All (server and clients) are running the dispatch router locally, and >> connect to the server's dispatch router address on startup >> automatically. (The server's address is currently hard-coded in the >> clients' dispatch routers' configuration) >> >> We have successful in getting clients and server communicate all with >> each other using the server's public address/port as the only known >> connection point in the system. All in all, things look very >> promising! :-) >> >> Now we are to move on to next steps. There are especially these features >> we need to make sure we support: >> >> a) Sending a message _from_ a device that is not presently on-line at >> the moment: >> >> Devices come on/off the net at unpredictable times and intervals. >> However, there might be messages _from_ the device that needs to be >> delivered to the server when it comes on. On the application level on a >> client, we just want to deliver a message to the local "message handler" >> (i.e. the dispatch router presently), and let the application program >> continue working without halt. For this functionality, is it right to >> assume that we need to implement the broker also, locally, as this is >> not a dispatch router feature? > > You need something to periodically retry the connection and when > successful send any queued messages. As you say dispatch router does > not provide this. > > The aim is for the proton messenger client to provide this sort of > capability at some point in the future, but it isn't there yet I'm > afraid. > You could use a broker co-located with your sender, provided it has > the ability to establish outgoing reconnections over which it can > deliver the messages when connected. > > Alternatively you would at present need to build the logic yourself. > You could do this as a specialised intermediary perhaps (rather than > in the sending code itself). I.e. you would be building a very simple > 'broker'like thing, more specialised to this one role. This would be > colocated with your sender and would do the storing of messages (plus > any expiration needed), when they could not be directly forwarded to > the gateway. > >> b) Sending a message _to_ a device that is not present on-line at the >> moment: >> >> In case the device is not presently on-line, there will be no active >> route to it. However, it should still be addressable (aka. a >> "registered" device in the system), and messages for it (from any other >> device - server or other clients) should be delivered when it is (might >> be) back on-line. This relates to the previous point, but as far as we >> can see, for this to work, the router would need to know that the >> address for the device is still "legal" even though not on-line, and be >> able to deliver the message when it actually comes back on-line. As seen >> from the application program, the situation is the same as the previous >> point. Is it right to assume that we broker for this? > > Yes, this is essentially 'store-and-forward' behaviour, which the > router does not itself provide. One question, do senders generally > care about if/when the actual receiver gets the message? or are they > happy as long as it has been accepted into some queue pending delivery > to the receiver when it comes back on line? Or perhaps the > acknowledgement of messages is not that important but you would like > to minimise loss? > >> c) Time to live on messages: >> >> Every message should have a time-to-live parameter. I.e., if the message >> is not delivered before ttl is expired (ref above cases), the message >> can be discarded at the local "message handler" level. Is this AMQP/Qpid >> functionality; which components do we need to support it? > > Both broker implementations support it, so if you do use a broker > queue to store messages pending delivery then the ttl provides one > means to limit the growth of that queue (which is a concern in both > the send and receive case if the client is disconnected for a 'long' > time, where 'long' really depends on the size and rate of sending). > >> d) Resuming large messages: >> >> The payload of messages could be up to (in really worst cases) several >> gigabytes. Is there AMQP/Qpid functionality at some level that makes the >> transfer resume (rather than restart) in case the is a network dropout >> during transfer? Where does this functionality exist? >> Broker/router/messager lib...? > > The AMQP protocol supports this, but it isn't supported in the c++ > broker, the dispatch router or I don't think the messenger client itself. >> Can the qpid broker replace all dispatch router functionality in these >> cases, or is it an addition to the dispatch router functionality? > > The qpid router provides end-to-end acknowledgement, i.e. the sender > gets an acknowledgement from the client, via the router, whereas a > broker will give the sender an acknowledgement independently of > delivering the message to a consumer. > > The qpid router supports redundant routes between router instances, > which helps building fault tolerance. The c++ brokers federation does > not do this. > > Other than that you could replace the router with a broker if needed. > However there is some work planned for the dispatch router that should > allow it to work well alongside brokers. The implementation is not > quite there yet however. > I hope this helps a little, your project sounds very interesting and > I'm sure there will be many of the list willing to help and offer > opinions/advice, so please don't hesitate to ask any further questions > etc! > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
