Alan Conway wrote:
On Mon, 2008-08-18 at 19:15 +0100, Gordon Sim wrote:
Alan Conway wrote:
At most once: accept-mode=none:
- broker MUST forget messages upon sending the transfer
- client MUST NOT process message on receiving the transfer. It MUST
send an accept and wait for the accept to complete before processing.
The client never needs to send an accept in this case; it can process
the message as soon as it receives it.
Yes, I think that was a cut-paste error in my mail.
- risk: if client A crashes, messages that it has not yet processed may
be lost.
The first case is straightforward in Qpid, but the second case is not.
The trivial implementation (send a synchronous accept for each incoming
message) will perform dreadfully.
You only need to wait for completion of the accept if you are updating
some application state based on the assumption that the accepted message
will never be redelivered.
Yes, that's what I mean by at-most-once delivery.
Then I am confused: above I thought we had now agreed that at-most-once
guarantees could be obtained by not requiring an accept at all?
I was trying to describe an approach an application might use to get
exactly-once processing guarantees; an accept would be used to prevent
loss of messages, and completion of that accept might need to be tracked
to update the state used to detect duplicates.
Is there a case for extending the API
or adding utility classes for this use case? E.g. with a variation of
MessageListener that passes you the messages when accepts complete
rather than when the transfer arrives?
I don't really understand why you would want to do that.
So that you can process messages with an at-most-once guarantee. The
current API makes it tricky to do this without creating synchronous
blocks that are a performance problem.
Again, I thought at-most-once came from accept-mode=none, so there would
be _no_ accepts, synchronous or otherwise.
As above, I was referring to an application enforced exactly-once
guarantee using some sort of idempotence barrier that requires
updating/clean-up when the accept completes.
(I can see some benefit in being able to track completion
asynchronously; I can also see some benefit in extending AckPolicy to do
synchronous accepts if desired.)
Processing messages based on async completion of accept is exactly what
I'm proposing above. Are you thinking of a more generic way of tracking
arbitrary commands? That would work too provided there is an easy way to
figure out the set of accepted messages when the accept completes.
In the scenario I can envisage (as above, an application level
exactly-once policy), the message itself would be processed before
sending the accept (because we don't want lost messages). The
application would track completion of the accept so we could clean up
whatever state was needed to detect duplicate delivery of that message.
Of course ideally it would just use one or more sequence counter for
detecting duplicates which would likely remove the need to track
completion of the accept. In other words while I can see value in
providing the mechanisms needed to do this sort of thing but I suspect
many applications will not require it, its a fairly specialised use-case.
