The only way to have reliable delivery without duplicates is to use an exactly-once QoS based on the AMQP 1.0 3-ack handshake. I don't believe this is currently implemented in the broker (or the client).
This involves de-duplication built into the client. The 3-ack significantly reduces the number of message-IDs that needs to be stored for de-duplication. I do believe that this capability will be appearing soon in the Proton API work that Gordon Sim is doing in a branch currently. -Ted On 10/29/2014 09:38 AM, Nitin Shah wrote: > Hi Ted, Dev team > > Thanks for the response. > > We expect it to be a reliable connection and I have tried the capacity values > set to minimum, but then we still get a single message replay. > I think the penalty of performance on synchronous messages would be > unacceptable. > > Even if we look at the redelivered flag, we may have to keep a buffer of last > set of messages to determine if we have received the messages before, unless > I misunderstand the definition of that flag. Does this flag guarantee that > the message has been delivered to the client before? > > Nitin > > -----Original Message----- > From: Ted Ross [mailto:tr...@redhat.com] > Sent: Tuesday, October 28, 2014 12:37 PM > To: Nitin Shah; dev@qpid.apache.org > Subject: Re: Question On QPID behavior > > Nitin, > > What are your reliability requirements? Is your application > bandwidth-sensitive? > > You are dealing with tradeoffs in reliability, message duplication, and > performance. I assume since you are using at-least-once that you don't want > to lose messages. In that case, you will have to tolerate some amount of > duplication if there is a connection or broker failure. You can reduce the > extent of duplication by using a smaller sender capacity or by using > sender.send(message, sync=true), but this will reduce your throughput. > > Are you concerned about duplication because of wasted network bandwidth or > because your consumer application doesn't handle them efficiently? > You can use the 'redelivered' flag on received messages to determine if > de-duplication is needed (i.e. if you were to add a de-dup layer). > > -Ted > > On 10/27/2014 05:28 PM, Nitin Shah wrote: >> Hi Ted , >> >> Thanks for your prompt response. >> >> I would further like to explore how we can avoid receiving the same messages >> when the broker restarts keeping in mind that the messages have been >> acknowledged by the receiving entity in the first place. We would like to >> avoid getting these messages. We are using the reliable connection option ( >> at least once ). >> >> Thanks >> Nitin Shah >> >> -----Original Message----- >> From: Ted Ross [mailto:tr...@redhat.com] >> Sent: Monday, October 27, 2014 4:26 PM >> To: dev@qpid.apache.org >> Subject: Re: Question On QPID behavior >> >> Nitin, >> >> One thing to keep in mind is that there are actually two separate transfers >> involved in moving a message from sender to receiver. You mentioned the >> sender's capacity is 10, which is interesting since it is very close to the >> number of re-sends you are seeing. >> >> The transfer from sender to broker-queue is one acknowledged transfer >> and the transfer from the queue to the receiver is the other. The >> session.acknowledge() is only concerned with the second transfer (queue to >> receiver). >> >> It is likely that the re-sends are coming from the sender and not the queue >> (i.e. 10 duplicate messages are re-sent and re-queued). If this is the >> case, the way you use session.acknowledge() will have no effect on the >> behavior. >> >> Generally, you will see re-sent messages when connections or brokers are >> shut down and restarted. This is because there are a number of in-doubt >> deliveries after the connection loss. The number of in-doubt deliveries is >> capped by the capacity of both the sender and receiver. Note also that the >> messages that are re-sent are annotated as such with the redelivered header >> flag. >> >> In your case, you can have up to 10 in-doubt deliveries from the sender and >> one in-doubt from the queue (since you acknowledge every message). >> >> -Ted >> >> >> On 10/27/2014 03:57 PM, Nitin Shah wrote: >>> Hi, >>> >>> We are running C++ QPID broker and clients on version 0.26 . running on >>> WindRiver Linux. >>> We acknowledge the session received messages on the client using >>> session.acknowledge() interface on every received message. However, when >>> the broker is restarted ( after it being stopped ), we get the last 11 >>> messages repeated to the client. The senders capacity is set to 10. >>> >>> We would like to understand how the acknowledge works. It does not seem to >>> matter if we use the other forms of acknowledges i.e. >>> session.acknowledge(message &) or session.acknowledgeUpTo(message &). The >>> behavior is the same although the documentation seems to indicate that once >>> acked to the broker, the messages should not be repeated. >>> >>> Can I ask for some feedback and clarify this. >>> >>> Thanks >>> >>> Nitin Shah >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional >> commands, e-mail: dev-h...@qpid.apache.org >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org > For additional commands, e-mail: dev-h...@qpid.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
