Dear Arnaud, All
I have recently started looking into AMQP and came across many doubts. Carl mentioned that you will be the best person to ask about transaction management both for AMQP in general as a protocol and Qpid's implementation of it. I have been having some doubts about reliability and transaction support in AMQP. In general, how are transactions handled in AMQP for the various use cases? For example, when the broker receives a persistent-message and then crashes before it stores it on the external DB, how will this affect the recovery? Also, this will make the publisher to believe that the message wasn't received by the broker since he will get a connection exception and will resend it. Alternatively, if the crash occurs after the message is put in persistence or half way in the process. The client will behave in the same way and resend the message after a broker restarts. But since the message has been put into persistence, the broker will send it too after it restarts. I mean, this is just one possible failure that occurred to me and wasn't very clear from the spec. Another case would be, say for pub-sub, when a persistent message is supposed to be published on 3 queues, and the broker fails after putting in on 2. Could you explain a little about such cases (and others off the top of your head) and help me understand how transactions are handled (what is redone, what is undone, etc) and how does the broker internally assure reliability and no data loss during the process starting from last packet received via TCP, (maybe) put into persistence, routed by exchange, queued, taken by consumer, ack received, message (and queue, etc in case of temporary) deleted. (Am I missing some step here?) I mean, as far as a transaction is concerned, failure can occur at any step of the program/code for any type of operation, right? Also, it seems that the boundaries of transactions is not very well defined by the spec. For example, are the declare, property-change (say setting of mode), delete, etc operations for queues, exchanges, bindings all covered inside a transaction and deleted in case of an abort OR only the messages and acknowledgements covered. And if these issues not very clearly stated in the spec, won't this create a lot of uncertainty as far as things like interoperability is concerned? These are some of my initial thoughts. I would be grateful if you could explain these issues (unresolved problems, short-comings, my misunderstanding, etc.). Thanks in advance. Regards, Tanmay ________________________________ From: Carl Trieloff [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 28, 2007 5:02 PM To: Goel, Tanmay; Arnaud Simon Subject: Re: [rabbitmq-discuss] Transactions in AMQP Tanmay, I have copied Arnaud, he is on leave till Monday (moving to Paris). So you should be able correspond with him next week, or set up a call with him. He is our in house TXN specialist, together with Mark Little. Between Arnuad & Mark they have written/ contributed to just about every txn spec in the industry, and worked on most TP monitors. He should be able to answer any question you could dream of on txns. Regards Carl. Goel, Tanmay wrote: Gotcha. No, I don't have any particular interest in UDP. It was just a "quick question" to confirm, with you, what I had understood from the spec. Yes, please go ahead and put me in touch with Arnaud. It would be nice to get his views, too, on transaction management. Thanks, Tanmay ________________________________ From: Carl Trieloff [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 28, 2007 2:45 PM To: Goel, Tanmay Cc: [EMAIL PROTECTED] Subject: Re: [rabbitmq-discuss] Transactions in AMQP In 0-10 AMQP has full DTX and TX. we assume a reliable transport. AMQP does not yet have a UDP transport so the behavior for UDP is undefined, especially with txn. Any UDP L1 AMQP binding would need frame reliability to be able to support txns (we could do this, but it is currently unspecified.) Arnuad Simon, Mark Little, Gordon Sim drove most of the transaction work with Cisco. I can connect you with Arnuad directly if you like. In terms of txns spanning queue.declare etc, the spec supports/makes that possible. dtx for 0-10 is here https://jira.amqp.org/jira/browse/AMQP-4 tx is existing in the spec docs. "2. Is the protocol currently assuming a reliable transport layer (TCP, SCTP, etc) and no unreliable connection (say, UDP)? If yes, will the later releases work on including unreliable transport layer protocols"" -- > we have framework that will allow us to make UDP reliable, this work has not been done in the spec(provision has been made so it can be done), but if done then we 'should' be able to support txns on UDP. Do you have an interest in UDP, and if so why? Hope that helps Carl. Goel, Tanmay wrote: Hi, 1. I have a basic question about transactions in AMQP. What exactly does a transaction include/mean? For example, does it include the administrative features like creation of queues, exchanges, etc that will be auto-deleted if the client aborts the transaction. I'm not entirely clear about this, so I'm just trying to understand and figure out possibilities. The spec says that messages and acknowledgments are covered in a transaction. What would happen if the publisher sends a message to the broker/queue, consumer receives (reads it from the queue) and processes it, sends an ack to the broker and then the publisher decides to abort the transaction? How can this transaction be atomic and/or rolled-back since the message is already gone? Please help me understand this. I'm trying to understand the scope of a transaction. 2. Is the protocol currently assuming a reliable transport layer (TCP, SCTP, etc) and no unreliable connection (say, UDP)? If yes, will the later releases work on including unreliable transport layer protocols? Thanks, Tanmay ________________________________ _______________________________________________ rabbitmq-discuss mailing list [EMAIL PROTECTED] http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
