Is your non-camel subscriber running in the same JVM and using the same connection factory with the same client id?
On 18/06/2014, at 1:59 AM, Jeff Bischoff <[email protected]> wrote: > Not using a queue, just an AMQ topic. Was working fine before I introduced > Camel, so I think I must be misconfiguring something in Camel. > > If I do <rollback/> everything works, but if I do <stop/>, then my > non-Camel subscriber never gets the message. It's almost as if my Camel > consumer is not actually creating a subscription on the topic, but is > instead stealing the message intended for my non-camel subscriber. > > Does it make sense that rollback would even work on this route? It isn't > marked <transacted/> and the AMQConnector has no transaction manager > defined! Yet rollback seems to work. > > <route id="configuration.topic-out"> > <from uri="table-jms:topic:configuration.topic"/> > <log message="Picked up message with id: ${id}, > destinationType: ${header.DestinationEntityType}" loggingLevel="INFO"/> > loggingLevel="INFO"/> > <choice> > <when> > > <simple>${header.DestinationEntityType} not in 'FOO,BAR'</simple> > <log message="Detected new > ${header.Name} configuration.topic message on ${header.JMSDestination} for > destination ${header.DestinationEntityType} with id: ${id}" > loggingLevel="INFO"/> > <to > uri="lc-jms:topic:configuration.topic?requestTimeout=0"/> > <log message="Successfully > processed message with id: ${id}, destinationType: > ${header.DestinationEntityType}" loggingLevel="INFO"/> > </when> > <otherwise> > <log message="(otherwise) ignoring > message with id: ${id}, destinationType: ${header.DestinationEntityType}" > loggingLevel="INFO"/> > <rollback markRollbackOnly="true" > /> > </otherwise> > </choice> > </route> > > > Results in: > > [topic-out] (Camel (camelContext) thread #9 - > JmsConsumer[configuration.topic]) Picked up message with id: > ID:TS-0007-jbischoff-2.local-50312-1403019963746-1:2:3:1:1, > destinationType: TABLE > > [topic-out] (Camel (camelContext) thread #9 - > JmsConsumer[configuration.topic]) (otherwise) ignoring message with id: > ID:TS-0007-jbischoff-2.local-50312-1403019963746-1:2:3:1:1, > destinationType: TABLE > > [EndpointMessageListener] (Camel (camelContext) thread #9 - > JmsConsumer[configuration.topic]) Execution of JMS message listener > failed. Caused by: [org.apache.camel.RuntimeCamelException - > org.apache.camel.RollbackExchangeException: Intended rollback. > Exchange[JmsMessage[JmsMessageID: > ID:TS-0007-jbischoff-2.local-50312-1403019963746-1:2:3:1:1]]] > > > > Best, > > Jeff > > > > On 6/17/14 10:59 AM, "kraythe ." <[email protected]> wrote: > >> A JMS topic will send a copy of each message to every topic. If not then >> the system violates basic JMS spec. If the system is a mainstream system >> like ActiveMQ, then that is not your problem. If you are trying to use a >> queue as a topic it doesn't work that way. >> >> A queue is like throwing candy into the middle of a class of elementary >> kids. Each piece of candy will get consumed by only one kid but they will >> scramble for the pieces of candy as fast as they can. However, if a kid >> eats a piece of candy they can't regurgitate it and put it back into the >> pile. >> >> In your case if a route consumes a message on the queue, then it can't put >> it back on the queue, the only way it gets back there is if the route >> fails >> and rolls back. >> >> Try adding a log line or a DLQ to the otherwise and see what happens. And >> don't rollback in the otherwise. If you are using a queue, create a route >> to sort the queue into 2 other queues based on the criteria and then >> another route to process the queue where the relevant messages end up. >> i.e. >> from(my >> queue).choice().when(predicate).to("jms:queue:camel-processed-queue).other >> wise().to("jms:queue:queue-processed-externally") >> >> >> *Robert Simmons Jr. MSc. - Lead Java Architect @ EA* >> *Author of: Maintainable Java (Kindle >> <http://www.amazon.com/Maintainable-Java-Robert-Simmons-Jr-ebook/dp/B00AKH >> I69K>)(iTunes >> <https://itunes.apple.com/us/book/maintainable-java/id585666097?mt=11>)* >> *LinkedIn: **http://www.linkedin.com/pub/robert-simmons/40/852/a39 >> <http://www.linkedin.com/pub/robert-simmons/40/852/a39>* >> >> >> On Tue, Jun 17, 2014 at 9:27 AM, Jeff Bischoff >> <[email protected] >>> wrote: >> >>> Robert, thanks so much for replying. >>> >>>> Without that ack, the external system thinks it is not consumed and in >>>> good faith tries to redeliver. >>> >>> Okay, that seems like a good explanation for why I am getting external >>> redeliveries! >>> >>> I guess the root of my problem is that it seems like my Camel consumer >>> is >>> competing with my non-Camel topic subscribers. To your question: >>> >>>> So the question you have to decide to solve your problem is, "What is >>> the >>>> route I want my messages to take if they are not in the when >>> condition. >>>> Do >>>> they end up in a dead letter queue? Routed elsewhere? Logged to a >>> file? >>>> But >>>> they have to end up somewhere. >>> >>> I have non-Camel endpoints that will pick up the messages that don't >>> meet >>> the filter requirements. Ideally, I would have an "otherwise" that does >>> nothing with the message (but does send the ack that it has been >>> processed). These particular messages do not need to be routed by Camel >>> in >>> my system. >>> >>> However, it seems like Camel is competing with my non-Camel endpoints to >>> consumer topic messages. I would have thought each subscriber (Camel and >>> the other subscriber) would get their own copy of the topic message, and >>> that they would not compete with each other. It seems like the non-Camel >>> consumer can only process the message if I do a "rollback" in Camel. >>> >>> I'm building JUnit tests to try to figure out what I'm doing wrong, but >>> any further insight would be greatly appreciated. >>> >>> >>> Best, >>> >>> Jeff >>> >>> >>> >>> On 6/17/14 9:16 AM, "kraythe ." <[email protected]> wrote: >>> >>>> Good suggestions but you will need to check the JMS server config if >>> you >>>> are getting external redeliveries. Those happen when an external >>> system to >>>> the camel route performs a redelivery. >>>> >>>> As for your original route, there must always be a place for an >>> exchange >>>> to >>>> go or it will stop routing. In the case of your when without otherwise, >>>> what would be the path of the exchange that fails the when condition? >>> It >>>> is >>>> probably returned to the external system in some manner and not acked >>> as >>>> being delivered so it is sent again. Without that ack, the external >>> system >>>> thinks it is not consumed and in good faith tries to redeliver. >>>> >>>> Furthermore when you rollback, you are telling the host JMS system that >>>> you >>>> did not consume the message, and it will try again of course. >>>> >>>> So the question you have to decide to solve your problem is, "What is >>> the >>>> route I want my messages to take if they are not in the when >>> condition. Do >>>> they end up in a dead letter queue? Routed elsewhere? Logged to a file? >>>> But >>>> they have to end up somewhere. >>>> >>>> *Robert Simmons Jr. MSc. - Lead Java Architect @ EA* >>>> *Author of: Hardcore Java (2003) and Maintainable Java (2012)* >>>> *LinkedIn: **http://www.linkedin.com/pub/robert-simmons/40/852/a39 >>>> <http://www.linkedin.com/pub/robert-simmons/40/852/a39>* >>>> >>>> >>>> On Mon, Jun 16, 2014 at 5:56 PM, Minh Tran <[email protected]> >>>> wrote: >>>> >>>>> I have used topics in camel and they generally work the way I expect >>> it. >>>>> >>>>> I would start by leaving the cache levels at the default and >>> simplifying >>>>> your route further. eg take out the choice and send it direct from >>>>> topic to >>>>> queue. Then add the choice but send it directly to the queue without >>> the >>>>> recipient list. Also enable tracing on the camel context so you can >>> see >>>>> what is happening to your message each step of the way. >>>>> >>>>> On 17/06/2014, at 6:28 AM, Jeff Bischoff >>> <[email protected]> >>>>> wrote: >>>>> >>>>>> Does nobody use Topics with Camel? They don't seem to work as >>>>> expected. >>>>>> >>>>>> JB >>>>>> >>>>>> On 5/30/14 10:48 AM, "Jeff Bischoff" <[email protected]> >>>>> wrote: >>>>>> >>>>>>> Correction: >>>>>>> >>>>>>> 1) Why would *CACHE_CONSUMER* on the ActiveMQComponent cause >>> endless >>>>>>> external >>>>>>> redeliveries of a (filtered out) topic message, when with other >>> cache >>>>>>> settings (like CACHE_SESSION) this does not occur? >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 5/30/14 10:39 AM, "Jeff Bischoff" >>> <[email protected]> >>>>> wrote: >>>>>>> >>>>>>>> Maybe I need to narrow my questions: >>>>>>>> >>>>>>>> 1) Why would CACHE_SESSION on the ActiveMQComponent cause endless >>>>>>>> external >>>>>>>> redeliveries of a (filtered out) topic message, when with other >>>>> cache >>>>>>>> settings this does not occur? >>>>>>>> >>>>>>>> 2) Why would <rollback/> prevent the external redeliveries, but >>>>> <stop/> >>>>>>>> does not prevent them? >>>>>>>> >>>>>>>> >>>>>>>> Did I provide enough info below to answer these questions? Topics >>>>> just >>>>>>>> aren't working in Camel the way that I would expect! >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Jeff Bischoff >>>>>>>> WDTS >>>>>>>> >>>>>>>> On 5/29/14 4:25 PM, "Jeff Bischoff" >>> <[email protected]> >>>>> wrote: >>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> Please forgive me if this is a basic question. >>>>>>>>> >>>>>>>>> Working with Camel and JMS Topics has seemed very finicky for >>> me so >>>>> far. >>>>>>>>> If I make slight configuration changes, I see the same message >>>>> repeated >>>>>>>>> endlessly. Using jconsole, I now know these messages are >>> "external >>>>>>>>> redeliveries", i.e. the same message is being sent from JMS to >>>>> Camel >>>>>>>>> repeatedly without ever getting through. It happens in the case >>> of >>>>>>>>> messages that I am intentionally not processing due to some >>>>> criteria. >>>>>>>>> What I don't understand is why these external redeliveries keep >>>>>>>>> occurring, when I simply want this message to be dropped. >>>>>>>>> >>>>>>>>> For example, if I use CACHE_SESSION or lower cache level, the >>>>> following >>>>>>>>> route works just fine for me: >>>>>>>>> >>>>>>>>> >>>>>>>>> <route id="topics-out"> >>>>>>>>> <from uri="topic-jms:topic:projectx.>"/> >>>>>>>>> <log message="Picked up message with id: ${id}, >>>>>>>>> destinationType: ${header.DestinationEntityType}" >>>>> loggingLevel="INFO"/> >>>>>>>>> <choice> >>>>>>>>> <when> >>>>>>>>> <simple>${header.DestinationEntityType} not in >>>>>>>>> 'FOO,BAR'</simple> >>>>>>>>> <log message="Detected new ${header.Name} >>> topic >>>>>>>>> message on ${header.JMSDestination} with id: ${id}, >>>>> destinationType: >>>>>>>>> ${header.DestinationEntityType}" loggingLevel="INFO"/> >>>>>>>>> <recipientList ignoreInvalidEndpoints="false"> >>>>>>>>> >>>>> <simple>lc-jms:${header.JMSDestination}</simple> >>>>>>>>> </recipientList> >>>>>>>>> </when> >>>>>>>>> </choice> >>>>>>>>> </route> >>>>>>>>> >>>>>>>>> >>>>>>>>> However, if I switch to CACHE_CONSUMER and I have even one >>> message >>>>>>>>> addressed to 'FOO' or 'BAR', I will see the endless external >>>>>>>>> redeliveries. Note that I am using a SingleConnectionFactory >>>>>>>>> specifically >>>>>>>>> for this route. >>>>>>>>> >>>>>>>>> <route id="topics-out"> >>>>>>>>> <from uri="topic-jms:topic:projectx.>"/> >>>>>>>>> <log message="Picked up message with id: ${id}, >>>>>>>>> destinationType: ${header.DestinationEntityType}" >>>>> loggingLevel="INFO"/> >>>>>>>>> <choice> >>>>>>>>> <when> >>>>>>>>> <simple>${header.DestinationEntityType} not in >>>>>>>>> 'FOO,BAR'</simple> >>>>>>>>> <log message="Detected new ${header.Name} >>> topic >>>>>>>>> message on ${header.JMSDestination} with id: ${id}, >>>>> destinationType: >>>>>>>>> ${header.DestinationEntityType}" loggingLevel="INFO"/> >>>>>>>>> <recipientList ignoreInvalidEndpoints="false"> >>>>>>>>> >>>>> <simple>lc-jms:${header.JMSDestination}</simple> >>>>>>>>> </recipientList> >>>>>>>>> </when> >>>>>>>>> </choice> >>>>>>>>> </route> >>>>>>>>> >>>>>>>>> Now, if I add an Otherwise clause with a Rollback, everything >>> works >>>>> fine >>>>>>>>> again (albeit with a handful of log messages due to the >>> rollbacks): >>>>>>>>> >>>>>>>>> <route id="topics-out"> >>>>>>>>> <from uri="topic-jms:topic:projectx.>"/> >>>>>>>>> <log message="Picked up message with id: ${id}, >>>>>>>>> destinationType: ${header.DestinationEntityType}" >>>>> loggingLevel="INFO"/> >>>>>>>>> <choice> >>>>>>>>> <when> >>>>>>>>> <simple>${header.DestinationEntityType} not in >>>>>>>>> 'FOO,BAR'</simple> >>>>>>>>> <log message="Detected new ${header.Name} >>> topic >>>>>>>>> message on ${header.JMSDestination} with id: ${id}, >>>>> destinationType: >>>>>>>>> ${header.DestinationEntityType}" loggingLevel="INFO"/> >>>>>>>>> <recipientList ignoreInvalidEndpoints="false"> >>>>>>>>> >>>>> <simple>lc-jms:${header.JMSDestination}</simple> >>>>>>>>> </recipientList> >>>>>>>>> </when> >>>>>>>>> <otherwise> >>>>>>>>> <rollback markRollbackOnly="true" /> >>>>>>>>> </otherwise> >>>>>>>>> </choice> >>>>>>>>> </route> >>>>>>>>> >>>>>>>>> But if I change that Rollback to a Stop instead, I get the >>> endless >>>>>>>>> external redeliveries again: >>>>>>>>> >>>>>>>>> <route id="topics-out"> >>>>>>>>> <from uri="topic-jms:topic:projectx.>"/> >>>>>>>>> <log message="Picked up message with id: ${id}, >>>>>>>>> destinationType: ${header.DestinationEntityType}" >>>>> loggingLevel="INFO"/> >>>>>>>>> <choice> >>>>>>>>> <when> >>>>>>>>> <simple>${header.DestinationEntityType} not in >>>>>>>>> 'FOO,BAR'</simple> >>>>>>>>> <log message="Detected new ${header.Name} >>> topic >>>>>>>>> message on ${header.JMSDestination} with id: ${id}, >>>>> destinationType: >>>>>>>>> ${header.DestinationEntityType}" loggingLevel="INFO"/> >>>>>>>>> <recipientList ignoreInvalidEndpoints="false"> >>>>>>>>> >>>>> <simple>lc-jms:${header.JMSDestination}</simple> >>>>>>>>> </recipientList> >>>>>>>>> </when> >>>>>>>>> <otherwise> >>>>>>>>> <stop/> >>>>>>>>> </otherwise> >>>>>>>>> </choice> >>>>>>>>> </route> >>>>>>>>> >>>>>>>>> Why does rolling back prevent the unwanted message from being >>>>>>>>> redelivered, while using "stop" causes the redelivery to occur? >>>>>>>>> Intuitively, I would have thought it would work the opposite >>> way. >>>>>>>>> Rolling >>>>>>>>> back should put the item back on the "from" endpoint, allowing >>> it >>>>> to >>>>> be >>>>>>>>> processed again. "Stop" should just drop the message, preventing >>>>> further >>>>>>>>> redelivery. What I'm actually seeing is the opposite of this. >>> Am I >>>>>>>>> misunderstanding? >>>>>>>>> >>>>>>>>> I also get the same exact symptoms if I leave the cache level at >>>>>>>>> CACHE_SESSION, but I make the subscription durable. In that case >>>>> the >>>>>>>>> rollbacks prevent the problem, but otherwise I get the eternal >>>>> external >>>>>>>>> redeliveries. >>>>>>>>> >>>>>>>>> Using a Filter element also produces the same results as using >>> the >>>>>>>>> Choice >>>>>>>>> element above: >>>>>>>>> >>>>>>>>> <route id="topics-out"> >>>>>>>>> <from >>>>>>>>> >>>>> uri="topic-jms:topic:projectx.>?durableSubscriptionName=testdurasub"/> >>>>>>>>> <log message="Picked up message with id: ${id}, >>>>>>>>> destinationType: ${header.DestinationEntityType}" >>>>> loggingLevel="INFO"/> >>>>>>>>> <filter> >>>>>>>>> <simple>${header.DestinationEntityType} not in >>>>>>>>> 'FOO,BAR'</simple> >>>>>>>>> <recipientList ignoreInvalidEndpoints="false"> >>>>>>>>> >>> <simple>lc-jms:${header.JMSDestination}</simple> >>>>>>>>> </recipientList> >>>>>>>>> </filter> >>>>>>>>> </route> >>>>>>>>> >>>>>>>>> When I look at the topic in JMS, I see the topic "projectx.>" >>>>> appearing >>>>>>>>> and disappearing in the list rapidly. I assume that's caused by >>> the >>>>>>>>> Camel >>>>>>>>> polling frequency. It seems a little disturbing to me; is that >>>>> normal? >>>>>>>>> >>>>>>>>> Thanks so much for taking the time to help! >>>>>>>>> >>>>>>>>> Jeff Bischoff >>>>>>>>> WDTS >>>>>>>> >>>>>>> >>>>>> >>>>>> ________________________________ >>>>>> >>>>>> Please take note: This email, including attachments, contains >>>>> information which may be confidential or legally privileged and is >>> only >>>>> for >>>>> the use of the individual or entity to whom it is properly addressed. >>>>> Any >>>>> unauthorized review, use, disclosure, copying, or distribution is >>>>> prohibited. If you have reason to believe that you have received this >>>>> communication in error, or that it may be misaddressed or not >>> intended >>>>> for >>>>> you, please destroy it and notify the sender immediately. Thank you. >>>>> >>>>> >>> >>> ________________________________ >>> >>> Please take note: This email, including attachments, contains >>> information >>> which may be confidential or legally privileged and is only for the use >>> of >>> the individual or entity to whom it is properly addressed. Any >>> unauthorized >>> review, use, disclosure, copying, or distribution is prohibited. If you >>> have reason to believe that you have received this communication in >>> error, >>> or that it may be misaddressed or not intended for you, please destroy >>> it >>> and notify the sender immediately. Thank you. >>> > > ________________________________ > > Please take note: This email, including attachments, contains information > which may be confidential or legally privileged and is only for the use of > the individual or entity to whom it is properly addressed. Any unauthorized > review, use, disclosure, copying, or distribution is prohibited. If you have > reason to believe that you have received this communication in error, or that > it may be misaddressed or not intended for you, please destroy it and notify > the sender immediately. Thank you.
