Gary, Timothy, thanks a lot for confirming my considerations and seconding them!
I just created this [no path yet, though] https://issues.apache.org/jira/browse/AMQ-5406: AMQ-5406 "Support of jms.consumerExpiryCheck=false to avoid JMS Consumers ignoring some messages in case of out-of-synch clocks" Cheers, F. On Wed, Oct 22, 2014 at 4:30 PM, Timothy Bish <tabish...@gmail.com> wrote: > On 10/22/2014 10:06 AM, Gary Tully wrote: > >> i think you are making a strong case for having an option to disable the >> expiry check client side. Please raise an jira issue to track it and if >> you >> have a moment to make a test and patch even better. >> something like jms.consumerExiryCheck=false (and leave the default as is >> (true) >> > > The ActiveMQ .NET client has this option to solve similar complaints, so > you can look there for inspiration. > > > >> On 22 October 2014 12:04, Fred Moore <fred.moor...@gmail.com> wrote: >> >> I see your point about the need to have fully synchronized clocks and the >>> inherent value of being strictly compliant with JMS specs... ...on the >>> other hand I have some real world scenarios at hand that drive me to be >>> extremely pragmatic: >>> >>> 1\ More often that not clocks are reasonably but not fully in synch ...we >>> need to live with it >>> >>> 2\ Message expiration is an incredibly valuable and popular feature of >>> the >>> JMS API, and more in general of message oriented middleware ...we don't >>> want to renounce to it >>> >>> 3\ ActiveMQ provides pragmatic tools to fix clock problems between the >>> producers and the broker via the TimeStampingBrokerPl​ugin, which BTW >>> happens to cause a breach of the JMS specs ("since the timestamp that the >>> producer sees on the messages after as send() will be different from the >>> timestamp the consumer will observe when he receives the message") >>> >>> 4\ Some APIs, like ActiveMQ Stomp API, are proved to be well behaved on >>> the >>> consumer side even in case of out-of-synch clocks >>> >>> 5\ ActiveMQ JMS API -- the best and most widely used API -- appears not >>> to >>> have ways to inhibit the check client side check on JMSExpiration >>> >>> Any thought? >>> Is there a way to solve the JMS consumer side issue with out-of-synch >>> clocks? >>> >>> Cheers, >>> F. >>> >>> >>> On Tue, Oct 21, 2014 at 7:51 PM, Robbie Gemmell < >>> robbie.gemm...@gmail.com> >>> wrote: >>> >>> Hi Fred, >>>> >>>> If you are using message expiration, you ultimately need to have a >>>> handle >>>> on the degree of time synchronization between your hosts. >>>> >>>> The JMS API says for Message#getJMSExpiration() that "Clients should not >>>> receive messages that have expired; however, the JMS API does not >>>> >>> guarantee >>> >>>> that this will not happen". The escape clause there is effectively to >>>> >>> cover >>> >>>> the race that occurs when you decide 'this hasn't expired', only for the >>>> expiration point to have passed by the time the delivery is completed >>>> and >>>> an application checks the value. The earlier bit is saying if >>>> >>> JMSExpiration >>> >>>> currentTime, client applications shouldnt see the message. >>>>> >>>> If a JMS client receives a message from a broker and possibly examines >>>> >>> its >>> >>>> JMSExpiration value, it may determine that according to its local time >>>> >>> the >>> >>>> message expiry time has already been passed, at which point it becomes >>>> an >>>> interesting question as to whether the message should be given to a >>>> consuming application or not. Some implementations will do so simply >>>> because they dont check this, and others wont because they are trying to >>>> enforce what the spec says above and according to the information at >>>> hand >>>> the message really has actually expired. Different people might prefer >>>> >>> one >>> >>>> or the other behaviour. >>>> >>>> The same issue could reasonably exist between a producer and a broker it >>>> sends to on another host, which might never give a message out to a >>>> consumer because it had 'already expired when it arrived' for similar >>>> reason. >>>> >>>> Going back to the start this is basically to say, if you are using TTL >>>> based expiration you need to ensure an appropriate level of clock >>>> synchronization for your particular needs. >>>> >>>> I'll leave the specifics of the questions to someone familiar with the >>>> code. >>>> >>>> Robbie >>>> >>>> On 21 October 2014 16:14, Fred Moore <fred.moor...@gmail.com> wrote: >>>> >>>> Hi folks, >>>>> >>>>> I found a situation where the handling of timestamps/expirations by >>>>> ActiveMQ 5.10 appears not to be correct, and unfortunately -- being >>>>> >>>> this >>> >>>> a >>>> >>>>> consumer side issue -- it does not appear to be solvable via classic >>>>> TimeStampingBrokerPlugin tweaks. >>>>> >>>>> Scenario: >>>>> >>>>> S1\ Broker B1 runs on HOST1 >>>>> S2\ Producer P1 is a java JMS application running on HOST1 >>>>> S3\ HOST1 is on UTC timezone >>>>> S4\ B1 has BrokerTimestampingPlugin with futureOnly=true configured >>>>> (even if I think it does not play any role in this specific case) >>>>> S4\ Consumer C1 is a java JMS API application running on HOST2 >>>>> S5\ Consumer C2 is a java STOMP API application running on HOST2 >>>>> S6\ HOST2 is on UTC timezone and its clock is 3 mins ahead of HOST1 >>>>> >>>> clock >>> >>>> >>>>> Test case: >>>>> >>>>> T1\ P1 sends message M1 with TTL=1 minute >>>>> T2\ C1 on HOST2 immediately tries to receive a message but it does not >>>>> >>>> find >>>> >>>>> it because >>>>> according to HOST2 clock that message has already expired (see [*] >>>>> trace) >>>>> T3\ C2 on HOST2 tries to receive a message and it succeeds >>>>> >>>>> Questions/congeptures: >>>>> >>>>> Q1\ The behaviour looks client API/technology dependent, is this >>>>> >>>> expected? >>>> >>>>> [I think it's not] >>>>> Q2\ It looks like the JMS layer in C1 is re-checking expiration on >>>>> candidate messages >>>>> "served" to it by the broker, causing the reported issue: is there >>>>> a >>>>> >>>> way >>>> >>>>> to inhibit >>>>> this extra logic in JMS? >>>>> Q3\ Is my assumption about BrokerTimestampingPlugin being useless here >>>>> correct? >>>>> [It can only influence the produced messages and not the consumed >>>>> >>>> ones] >>>> >>>>> If there is no general solution to this it means that messages are >>>>> >>>> wrongly >>>> >>>>> ignored by ActiveMQ JMS consumers every time these [not so uncommon] >>>>> conditions are verified: >>>>> >>>>> F1\ the consumer clock runs ahead of the broker clock by X seconds >>>>> F2\ the TTL of messages is less than X seconds >>>>> F3\ the consumer is a java JMS application >>>>> >>>>> Please help! >>>>> Cheers, >>>>> F. >>>>> >>>>> >>>>> ----- >>>>> [*] Trace showing C1 misperception of M1 expiration >>>>> >>>>> 2014-10-21 15:53:11,987 [TestListener] DEBUG ActiveMQMessageConsumer >>>>> .dequeue - ID:gd-57426-1413899591527-1:1:1:1 received expired message: >>>>> MessageDispatch {commandId = 0, responseRequired = false, consumerId = >>>>> ID:gd-57426-1413899591527-1:1:1:1, destination = queue://TEST, message >>>>> >>>> = >>> >>>> ActiveMQTextMessage {commandId = 9, responseRequired = false, >>>>> >>>> messageId = >>> >>>> ID:narsil104.narsil104-29953-1413889401564-3:3:1:1:5, >>>>> >>>> originalDestination = >>>> >>>>> null, originalTransactionId = null, producerId = >>>>> ID:narsil104.narsil104-29953-1413889401564-3:3:1:1, destination = >>>>> queue://TEST, transactionId = null, expiration = 1413898985466, >>>>> >>>> timestamp = >>>> >>>>> 1413898385466, arrival = 0, brokerInTime = 1413898385466, >>>>> >>>> brokerOutTime = >>> >>>> 1413898389795, correlationId = , replyTo = null, persistent = false, >>>>> >>>> type = >>>> >>>>> , priority = 0, groupID = null, groupSequence = 0, targetConsumerId = >>>>> >>>> null, >>>> >>>>> compressed = false, userID = null, content = >>>>> org.apache.activemq.util.ByteSequence@642a590d, marshalledProperties = >>>>> null, dataStructure = null, redeliveryCounter = 0, size = 0, >>>>> >>>> properties = >>> >>>> null, readOnlyProperties = true, readOnlyBody = true, droppable = >>>>> >>>> false, >>> >>>> jmsXGroupFirstForConsumer = false, text = Exp 5 min}, redeliveryCounter >>>>> >>>> = 0 >>>> >>> > > -- > Tim Bish > Sr Software Engineer | RedHat Inc. > tim.b...@redhat.com | www.redhat.com > skype: tabish121 | twitter: @tabish121 > blog: http://timbish.blogspot.com/ > >
