>> I guess I could load and send the file in chunks, using ByteMessages, >> probably in the same manner the ActiveMQ output stream does it. So I will >> be >> using my own streaming solution, instead of an AMQ feature, but will gain >> the ablility to defer commiting until I have the whole stream. > >The reason I've been suggesting splitting the file up is precisely so >that you can commit/acknowledge the message in chunks - rather than >deferring until you've got all of the messages. If you must defer >until you have all of the messages then using BytesMessage or the JMS >streams will have the same effect - the broker will need to keep them >all around in RAM until you commit/ack.
Ok, so what you're saying is that deferring message acknowledgement is a bad strategy anyway, especially if the transactions are long and the files are big. Ok, that's good advice :-) So let's say I acknowledge file-chunks and/or other messages (that logically belong together) one at a time, as I get them. In this case I can use the ActiveMQ streams and maybe even non-transacted sessions, with auto-ack. But now, if my consumer crashes while it's in the middle of such a multi-part transmission, the messages already acknowldeged will be lost, and I can no longer rely on the broker for redelivering them. Therefore, maybe I should not use the broker's guaranteed delivery for this purpose, and instead, imlpement a consumer with persistent/recoverable cache. Every message that is acknowledged is also persisted on the consumer side somehow (file system, database). Then, when the consumer verifies it has all parts of the "package" (e.g. all chunks of the file and all related messages) - it can grab them from whereever they were persisted, and forward them together to be processed by the business tier. BTW - I think you misunderstood my question about the ZIP file: the intention was not compression, but to be able to pass around several things together... but maybe it's a bad idea anyway :-) Naaman -- View this message in context: http://www.nabble.com/Consuming-a-group-of-messages-in-a-single-transaction-tf1966918.html#a5536994 Sent from the ActiveMQ - User forum at Nabble.com.
