[ https://issues.apache.org/jira/browse/NIFI-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565962#comment-16565962 ]
ASF GitHub Bot commented on NIFI-3531: -------------------------------------- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2931 ok @mosermw sounds fair to me. > modify session.recover behaviour in nifi-jms-processors to cope with > high-traffic JMS > ------------------------------------------------------------------------------------- > > Key: NIFI-3531 > URL: https://issues.apache.org/jira/browse/NIFI-3531 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions > Reporter: Dominik Benz > Assignee: Michael Moser > Priority: Major > > As described in this mailing list post > http://apache-nifi-developer-list.39713.n7.nabble.com/session-recover-behaviour-in-nifi-jms-processor-td14940.html > the current implementation of nifi-jms-processor, especially of JMSConsumer > https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSConsumer.java > causes frequent re-delivery of JMS messages due to the following behaviour: > 1) several threads perform the JMS session callback in parallel > 2) each callback performs a session.recover > 3) during high traffic, the situation arises that the ACKs from another > thread may not (yet) have arrived at the JMS server > 4) this implies that the pointer of session.recover will reconsume the > not-yet-acked message from another thread > I understood Nifi prefers message duplication over message loss - however, in > our case the number of re-delivered messages is "piling up" over time, > leading to a growing number of un-acked messages in JMS and finally to > storage problems on the JMS server side. > It would be great to modify the nifi-jms-processor package to reliably cope > with higher-traffic JMS sources. Ideas from my side would be / include: > 1) make synchronous / asynchronous delivery configurable > 2) add configuration option for custom ACKing modes (i.e. non-standard JMS > modes like individual ACKing or NO_ACK) > 3) add configuration option for JMS message selectors > From other projects, I found the JMS communication options in this > spark-jms-receiver package > https://github.com/tbfenet/spark-jms-receiver > very helpful (in which situations are synchronous / asynchronous approaches > reliable, how to buffer messages before acking them, how/when to ack, ...). > Here's also a java port of a subset: > https://github.com/bernhardschaefer/spark-jms-receiver-java > For any mentioned option I'm also happy to help! > Best & thanks for a great piece of software, > Dominik -- This message was sent by Atlassian JIRA (v7.6.3#76005)