I had thought a replay queue is where the messages would be put I'm no longer convinced.
In another mail John O'Hara uses the the reading email analogy. Continuing this to its logical conclusion could mean that replay is actually an option when you open the queue for reading. It avoids the client having to muck about with another queue. This is not the traditional view of a queue but is more general and I like it. So, how about a queue contains all messages put on it since it was last "purged". When a queue is opened for reading, it can be opened with a selector and one part of that selector is whether or not the message has been commited by a consumer or not. A consumer could then always open the queue and request all messages, committed or not, from a known point, e.g. a trade ID or message ID. The key thing here I suppose is that this type of consumer is writing the message in some other form into a database and so always knows the last thing written. A nice benefit is that if the database is really hosed you could get replay from start of day. This would definitely help implement DR for purely message based systems. So, can I specialise queues and layer this functionality on top of the spec in the APID implementation with some kind of provider specific header property? Colin. > -----Original Message----- > From: Marnie McCormack [mailto:[EMAIL PROTECTED] > Sent: 22 January 2007 19:54 > To: [email protected]; [EMAIL PROTECTED] > Subject: Re: Message history and replay > > I've been trying to imagine the scenarios in which a user > will make use of a ReplayExchange. > > I thought that the most common usage would be the following: > > - the receiving application has lost some messages sent to it > and needs them to be resent from a point in time or a message ID > > (The only other recovery scenario I thought likely was broker > failure, which would be handled by restarted (with > persistence) and the undelivered messages would already > remain in the BDB store.) > > So, the bit I'm not sure about is this - you have no way of > knowing in advance the point at which replay might be needed > from (unless using client acks - which work with ordinary queues ?). > > Thus your recipient app starts up again (or recognises it has > lost messages) and then wants to replay. So, the queue it is > reading from is rebound to a replay exchange (with a marker > ID) ? Does that mean that it is actually reading from a > different queue than normal operation, or that the queue is > bound to a different exchange type ? > > The logic behind this could be quite complex - in that the > balance between normal operation and replay operation in > subtle and would involve the receiving client understanding > the AMQP behaviour at quite a low level ? > > Apologies if I've missed a simple flow for replay :-) > > Regards, > Marnie > > On 1/22/07, Carl Trieloff <[EMAIL PROTECTED]> wrote: > > > > > > This has been asked for before. I would interested to see if an > > exchange could be used. I also think this might bring up > the topic of > > chained exchanges again. > > > > Colin, do you have any interest in creating a prototype > ReplyExchange > > to see how well it works? > > Carl. > > > > Alan Conway wrote: > > > On Thu, 2007-01-18 at 15:52 +0000, Colin Crist wrote: > > > > > >> Hi, > > >> > > >> A quick question... > > >> > > >> A common recovery scenario is requesting replay on a queue from a > > given, > > >> known message, either by its ID or by something in the > header. Its > > saves all > > >> that mucking about with XA and makes recovery the normal way of > > >> system startup rather then some exceptional case. > > >> > > >> This means when a message has reached all its recipients > it should > > never be > > >> marked for removal from the durable message store. > > >> > > >> As a user, I would also like to control when messages > get removed > > >> as > > part of > > >> my end of day or even weekly processs, probably based on header > > property > > >> selectors. > > >> > > >> I don't believe this is supported in AMQP in the wire > protocol (nor > > perhaps > > >> should it be) - is it maybe something that could be done > via some > > >> standardised command messages to a "replay" exchange? > > >> > > >> Any thoughts on implementation difficulty? > > >> > > > > > > You could represent this in AMQP protocol using a custom > exchange type. > > > > > > A ReplayExchange stores all messages sent to it (until > cleaned up by > > > some administrative function.) > > > > > > When binding a queue to a replay exchange you MAY specify an > > > argument "replay-from=N". A queue so bound will be > "pre-filled" with > > > the range of message starting from the Nth message up to the most > > > recent message sent to the exchange. Thereafter the > exchange behaves > > > like a fan-out exchange, each new message being added to > all the bound queues. > > > > > > That gives you replay entirely within the standard > protocol, using a > > > standard extension point to add this as proprietary broker > > > functionality. > > > > > > Note that to combine replay with other exchange types requires > > > exchange chaining which has been discussed several times > but is not > > > yet an agreed part of the protocol - this may be another use case > > > for that discussion > > > > > > Cheers, > > > Alan. > > > > > > > > > > > > >
