[
https://issues.apache.org/jira/browse/NIFI-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Moser updated NIFI-3531:
--------------------------------
Assignee: Michael Moser
Status: Patch Available (was: Open)
> modify session.recover behaviour in nifi-jms-processors to cope with
> high-traffic JMS
> -------------------------------------------------------------------------------------
>
> Key: NIFI-3531
> URL: https://issues.apache.org/jira/browse/NIFI-3531
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Dominik Benz
> Assignee: Michael Moser
> Priority: Major
>
> As described in this mailing list post
> http://apache-nifi-developer-list.39713.n7.nabble.com/session-recover-behaviour-in-nifi-jms-processor-td14940.html
> the current implementation of nifi-jms-processor, especially of JMSConsumer
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSConsumer.java
> causes frequent re-delivery of JMS messages due to the following behaviour:
> 1) several threads perform the JMS session callback in parallel
> 2) each callback performs a session.recover
> 3) during high traffic, the situation arises that the ACKs from another
> thread may not (yet) have arrived at the JMS server
> 4) this implies that the pointer of session.recover will reconsume the
> not-yet-acked message from another thread
> I understood Nifi prefers message duplication over message loss - however, in
> our case the number of re-delivered messages is "piling up" over time,
> leading to a growing number of un-acked messages in JMS and finally to
> storage problems on the JMS server side.
> It would be great to modify the nifi-jms-processor package to reliably cope
> with higher-traffic JMS sources. Ideas from my side would be / include:
> 1) make synchronous / asynchronous delivery configurable
> 2) add configuration option for custom ACKing modes (i.e. non-standard JMS
> modes like individual ACKing or NO_ACK)
> 3) add configuration option for JMS message selectors
> From other projects, I found the JMS communication options in this
> spark-jms-receiver package
> https://github.com/tbfenet/spark-jms-receiver
> very helpful (in which situations are synchronous / asynchronous approaches
> reliable, how to buffer messages before acking them, how/when to ack, ...).
> Here's also a java port of a subset:
> https://github.com/bernhardschaefer/spark-jms-receiver-java
> For any mentioned option I'm also happy to help!
> Best & thanks for a great piece of software,
> Dominik
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
