[
https://issues.apache.org/jira/browse/NIFI-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16562370#comment-16562370
]
Michael Moser commented on NIFI-3531:
-------------------------------------
I have another scenario where the session.recover() causes problems. If the
consumer has prefetch enabled, then the call to session.recover() causes
messages sent but not acknowledged to be marked as "redelivered". This can
cause JMS providers to retransmit these messages to the client. Also, some
providers limit the number of times a message can be redelivered, so a message
could actually be lost.
I'm having a really hard time justifying the call to session.recover() before
each call to consumer.receive(). It seems to cause known problems rather than
limit possible message loss.
> modify session.recover behaviour in nifi-jms-processors to cope with
> high-traffic JMS
> -------------------------------------------------------------------------------------
>
> Key: NIFI-3531
> URL: https://issues.apache.org/jira/browse/NIFI-3531
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Dominik Benz
> Priority: Major
>
> As described in this mailing list post
> http://apache-nifi-developer-list.39713.n7.nabble.com/session-recover-behaviour-in-nifi-jms-processor-td14940.html
> the current implementation of nifi-jms-processor, especially of JMSConsumer
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSConsumer.java
> causes frequent re-delivery of JMS messages due to the following behaviour:
> 1) several threads perform the JMS session callback in parallel
> 2) each callback performs a session.recover
> 3) during high traffic, the situation arises that the ACKs from another
> thread may not (yet) have arrived at the JMS server
> 4) this implies that the pointer of session.recover will reconsume the
> not-yet-acked message from another thread
> I understood Nifi prefers message duplication over message loss - however, in
> our case the number of re-delivered messages is "piling up" over time,
> leading to a growing number of un-acked messages in JMS and finally to
> storage problems on the JMS server side.
> It would be great to modify the nifi-jms-processor package to reliably cope
> with higher-traffic JMS sources. Ideas from my side would be / include:
> 1) make synchronous / asynchronous delivery configurable
> 2) add configuration option for custom ACKing modes (i.e. non-standard JMS
> modes like individual ACKing or NO_ACK)
> 3) add configuration option for JMS message selectors
> From other projects, I found the JMS communication options in this
> spark-jms-receiver package
> https://github.com/tbfenet/spark-jms-receiver
> very helpful (in which situations are synchronous / asynchronous approaches
> reliable, how to buffer messages before acking them, how/when to ack, ...).
> Here's also a java port of a subset:
> https://github.com/bernhardschaefer/spark-jms-receiver-java
> For any mentioned option I'm also happy to help!
> Best & thanks for a great piece of software,
> Dominik
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
