[jira] [Updated] (NIFI-7563) Optimize the usage of JMS sessions and message producers
[ https://issues.apache.org/jira/browse/NIFI-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-7563: - Fix Version/s: 1.12.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Optimize the usage of JMS sessions and message producers > > > Key: NIFI-7563 > URL: https://issues.apache.org/jira/browse/NIFI-7563 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.6.0, 1.8.0, 1.7.1, 1.10.0, 1.9.2, 1.11.4 >Reporter: Gardella Juan Pablo >Assignee: Gardella Juan Pablo >Priority: Minor > Fix For: 1.12.0 > > Time Spent: 27h 20m > Remaining Estimate: 44h 50m > > Below an scenario to reproduce the non optimize usage of JMS resources. > Suppose it is required to publish 1 message to the destination {{D}} using > [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. > The message is a flow file in the processor input queue. > It is important to know that internally the processor is using > [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] > to reuse objects and a > [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] > to be able to use in thread safe manner. For JMS publishers, the default > configuration is to cache connections, sessions (only 1) and message > producers. > *Preconditions* > # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute > defined. Due to NIFI-7561, it should contain the word {{queue}} or {{topic}}. > Also notice {{jms_destination}} should be ignored, as suggested at NIFI-7564. > That will limit the scenario only when {{jms_replyTo}} attribute is defined. > # For simplicity, the processor is the first time it processes messages. > *Scenario* > # Processor picks the message. The > [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] > is created. > # Connection {{C1}} and session {{S1}} are created. The > [Message|https://docs.oracle.com/javaee/7/api/javax/jms/Message.html] > {{M1_S1}} is created and > [MessageProducer|https://docs.oracle.com/javaee/7/api/javax/jms/MessageProducer.html] > {{MP_S1}} created too. Required to deliver first message at > [JMSPublisher#publish|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L65]. > # S1 and C1 are stored in {{CachingConnectionFactory}}. The caching > connection factory is created at > [AbstractJMSProcessor.java#L208|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L208]. > # An attempt to create a new connection and a new session are requested to > the connection factory to create destination defined in the header > {{jms_destination}} at > [JMSPublisher.java#L131|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L131]. > Notice the connection {{C1}} is reused although *{{S1}} is not reused* (it > is required to check internal logic in CachingConnectionFactory to understand > why not). A new session {{S2}} is created and stored in the > {{CachingConnectionFactory}} as the new cached session. > # Message is published and {{S1}} and {{MP_S1}} are closed. As {{S1}} is not > in the cache, it is physically closed and {{MP_S1}}. > # At this point of time, the cached objects are {{C1}}, {{S2}}. *Ideally*, > all resources should be reused. > The scenario if it is applied to N consecutive messages create a lot of > sessions and message producers. > We found this issue by adding an > [Interceptor|https://activemq.apache.org/interceptors] to an [Apache ActiveMQ > v5.x|http://activemq.apache.org/components/classic/] broker to detect the > optimal usage of resources. For example, only one message producer per > connection. In below scenario we will be created N producers for the same > connection. Also in a Nifi flow that connects a > [ConsumeJMS|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.ConsumeJMS/]
[jira] [Updated] (NIFI-7563) Optimize the usage of JMS sessions and message producers
[ https://issues.apache.org/jira/browse/NIFI-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gardella Juan Pablo updated NIFI-7563: -- Latest PR at https://github.com/apache/nifi/pull/4378 > Optimize the usage of JMS sessions and message producers > > > Key: NIFI-7563 > URL: https://issues.apache.org/jira/browse/NIFI-7563 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.6.0, 1.8.0, 1.7.1, 1.10.0, 1.9.2, 1.11.4 >Reporter: Gardella Juan Pablo >Assignee: Gardella Juan Pablo >Priority: Minor > Time Spent: 25h 10m > Remaining Estimate: 47h > > Below an scenario to reproduce the non optimize usage of JMS resources. > Suppose it is required to publish 1 message to the destination {{D}} using > [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. > The message is a flow file in the processor input queue. > It is important to know that internally the processor is using > [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] > to reuse objects and a > [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] > to be able to use in thread safe manner. For JMS publishers, the default > configuration is to cache connections, sessions (only 1) and message > producers. > *Preconditions* > # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute > defined. Due to NIFI-7561, it should contain the word {{queue}} or {{topic}}. > Also notice {{jms_destination}} should be ignored, as suggested at NIFI-7564. > That will limit the scenario only when {{jms_replyTo}} attribute is defined. > # For simplicity, the processor is the first time it processes messages. > *Scenario* > # Processor picks the message. The > [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] > is created. > # Connection {{C1}} and session {{S1}} are created. The > [Message|https://docs.oracle.com/javaee/7/api/javax/jms/Message.html] > {{M1_S1}} is created and > [MessageProducer|https://docs.oracle.com/javaee/7/api/javax/jms/MessageProducer.html] > {{MP_S1}} created too. Required to deliver first message at > [JMSPublisher#publish|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L65]. > # S1 and C1 are stored in {{CachingConnectionFactory}}. The caching > connection factory is created at > [AbstractJMSProcessor.java#L208|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L208]. > # An attempt to create a new connection and a new session are requested to > the connection factory to create destination defined in the header > {{jms_destination}} at > [JMSPublisher.java#L131|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L131]. > Notice the connection {{C1}} is reused although *{{S1}} is not reused* (it > is required to check internal logic in CachingConnectionFactory to understand > why not). A new session {{S2}} is created and stored in the > {{CachingConnectionFactory}} as the new cached session. > # Message is published and {{S1}} and {{MP_S1}} are closed. As {{S1}} is not > in the cache, it is physically closed and {{MP_S1}}. > # At this point of time, the cached objects are {{C1}}, {{S2}}. *Ideally*, > all resources should be reused. > The scenario if it is applied to N consecutive messages create a lot of > sessions and message producers. > We found this issue by adding an > [Interceptor|https://activemq.apache.org/interceptors] to an [Apache ActiveMQ > v5.x|http://activemq.apache.org/components/classic/] broker to detect the > optimal usage of resources. For example, only one message producer per > connection. In below scenario we will be created N producers for the same > connection. Also in a Nifi flow that connects a > [ConsumeJMS|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.ConsumeJMS/] > with a PublishJMS. Notice {{ConsumeJMS}} populates by default
[jira] [Updated] (NIFI-7563) Optimize the usage of JMS sessions and message producers
[ https://issues.apache.org/jira/browse/NIFI-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gardella Juan Pablo updated NIFI-7563: -- Status: Patch Available (was: In Progress) > Optimize the usage of JMS sessions and message producers > > > Key: NIFI-7563 > URL: https://issues.apache.org/jira/browse/NIFI-7563 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.11.4, 1.9.2, 1.10.0, 1.7.1, 1.8.0, 1.6.0 >Reporter: Gardella Juan Pablo >Assignee: Gardella Juan Pablo >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Below an scenario to reproduce the non optimize usage of JMS resources. > Suppose it is required to publish 1 message to the destination {{D}} using > [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. > The message is a flow file in the processor input queue. > It is important to know that internally the processor is using > [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] > to reuse objects and a > [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] > to be able to use in thread safe manner. For JMS publishers, the default > configuration is to cache connections, sessions (only 1) and message > producers. > *Preconditions* > # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute > defined. Due to NIFI-7561, it should contain the word {{queue}} or {{topic}}. > Also notice {{jms_destination}} should be ignored, as suggested at NIFI-7564. > That will limit the scenario only when {{jms_replyTo}} attribute is defined. > # For simplicity, the processor is the first time it processes messages. > *Scenario* > # Processor picks the message. The > [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] > is created. > # Connection {{C1}} and session {{S1}} are created. The > [Message|https://docs.oracle.com/javaee/7/api/javax/jms/Message.html] > {{M1_S1}} is created and > [MessageProducer|https://docs.oracle.com/javaee/7/api/javax/jms/MessageProducer.html] > {{MP_S1}} created too. Required to deliver first message at > [JMSPublisher#publish|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L65]. > # S1 and C1 are stored in {{CachingConnectionFactory}}. The caching > connection factory is created at > [AbstractJMSProcessor.java#L208|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L208]. > # An attempt to create a new connection and a new session are requested to > the connection factory to create destination defined in the header > {{jms_destination}} at > [JMSPublisher.java#L131|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L131]. > Notice the connection {{C1}} is reused although *{{S1}} is not reused* (it > is required to check internal logic in CachingConnectionFactory to understand > why not). A new session {{S2}} is created and stored in the > {{CachingConnectionFactory}} as the new cached session. > # Message is published and {{S1}} and {{MP_S1}} are closed. As {{S1}} is not > in the cache, it is physically closed and {{MP_S1}}. > # At this point of time, the cached objects are {{C1}}, {{S2}}. *Ideally*, > all resources should be reused. > The scenario if it is applied to N consecutive messages create a lot of > sessions and message producers. > We found this issue by adding an > [Interceptor|https://activemq.apache.org/interceptors] to an [Apache ActiveMQ > v5.x|http://activemq.apache.org/components/classic/] broker to detect the > optimal usage of resources. For example, only one message producer per > connection. In below scenario we will be created N producers for the same > connection. Also in a Nifi flow that connects a > [ConsumeJMS|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.ConsumeJMS/] > with a PublishJMS. Notice {{ConsumeJMS}} populates by default > {{jms_des
[jira] [Updated] (NIFI-7563) Optimize the usage of JMS sessions and message producers
[ https://issues.apache.org/jira/browse/NIFI-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gardella Juan Pablo updated NIFI-7563: -- Description: Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is required to publish 1 message to the destination {{D}} using [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. The message is a flow file in the processor input queue. It is important to know that internally the processor is using [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] to reuse objects and a [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] to be able to use in thread safe manner. For JMS publishers, the default configuration is to cache connections, sessions (only 1) and message producers. *Preconditions* # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute defined. Due to NIFI-7561, it should contain the word {{queue}} or {{topic}}. Also notice {{jms_destination}} should be ignored, as suggested at NIFI-7564. That will limit the scenario only when {{jms_replyTo}} attribute is defined. # For simplicity, the processor is the first time it processes messages. *Scenario* # Processor picks the message. The [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] is created. # Connection {{C1}} and session {{S1}} are created. The [Message|https://docs.oracle.com/javaee/7/api/javax/jms/Message.html] {{M1_S1}} is created and [MessageProducer|https://docs.oracle.com/javaee/7/api/javax/jms/MessageProducer.html] {{MP_S1}} created too. Required to deliver first message at [JMSPublisher#publish|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L65]. # S1 and C1 are stored in {{CachingConnectionFactory}}. The caching connection factory is created at [AbstractJMSProcessor.java#L208|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L208]. # An attempt to create a new connection and a new session are requested to the connection factory to create destination defined in the header {{jms_destination}} at [JMSPublisher.java#L131|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L131]. Notice the connection {{C1}} is reused although *{{S1}} is not reused* (it is required to check internal logic in CachingConnectionFactory to understand why not). A new session {{S2}} is created and stored in the {{CachingConnectionFactory}} as the new cached session. # Message is published and {{S1}} and {{MP_S1}} are closed. As {{S1}} is not in the cache, it is physically closed and {{MP_S1}}. # At this point of time, the cached objects are {{C1}}, {{S2}}. *Ideally*, all resources should be reused. The scenario if it is applied to N consecutive messages create a lot of sessions and message producers. We found this issue by adding an [Interceptor|https://activemq.apache.org/interceptors] to an [Apache ActiveMQ v5.x|http://activemq.apache.org/components/classic/] broker to detect the optimal usage of resources. For example, only one message producer per connection. In below scenario we will be created N producers for the same connection. Also in a Nifi flow that connects a [ConsumeJMS|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.ConsumeJMS/] with a PublishJMS. Notice {{ConsumeJMS}} populates by default {{jms_destination}} flowfile attribute which, if it is not removed, it is processed by {{PublishJMS}} processor (by solving NIFI-7564 should not happen any more). was: Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is required to publish 1 message to the destination {{D}} using [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. The message is a flow file in the processor input queue. It is important to know that internally the processor is using [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-j
[jira] [Updated] (NIFI-7563) Optimize the usage of JMS sessions and message producers
[ https://issues.apache.org/jira/browse/NIFI-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gardella Juan Pablo updated NIFI-7563: -- Description: Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is required to publish 1 message to the destination {{D}} using [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. The message is a flow file in the processor input queue. It is important to know that internally the processor is using [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] to reuse objects and a [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] to be able to use in thread safe manner. For JMS publishers, the default configuration is to cache connections, sessions (only 1) and message producers. *Preconditions* # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute defined. Due to NIFI-7561, it should contain the word {{queue}} or {{topic}}. Also notice {{jms_destination}} should be ignored, as suggested at NIFI-7564. That will limit the scenario only when {{jms_replyTo}} attribute is defined. # For simplicity, the processor is the first time it processes messages. *Scenario* # Processor picks the message. The [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] is created. # Connection {{C1}} and session {{S1}} are created. The [Message|https://docs.oracle.com/javaee/7/api/javax/jms/Message.html] {{M1_S1}} is created and [MessageProducer|https://docs.oracle.com/javaee/7/api/javax/jms/MessageProducer.html] {{MP_S1}} created too. Required to deliver first message at [JMSPublisher#publish|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L65]. # S1 and C1 are stored in {{CachingConnectionFactory}}. The caching connection factory is created at [AbstractJMSProcessor.java#L208|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L208]. # An attempt to create a new connection and a new session are requested to the connection factory to create destination defined in the header {{jms_destination}} at [JMSPublisher.java#L131|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L131]. Notice the connection {{C1}} is reused although *{{S1}} is not reused* (it is required to check internal logic in CachingConnectionFactory to understand why not). A new session {{S2}} is created and stored in the {{CachingConnectionFactory}} as the new cached session. # Message is published and {{S1}} and {{MP_S1}} are closed. As {{S1}} is not in the cache, it is physically closed and {{MP_S1}}. # At this point of time, the cached objects are {{C1}}, {{S2}}. *Ideally*, all resources should be reused. The scenario if it is applied to N consecutive messages create a lot of sessions and message producers. We found this issue by adding an [Interceptor|https://activemq.apache.org/interceptors] to an [Apache ActiveMQ v5.x|http://activemq.apache.org/components/classic/] broker to detect the optimal usage of resources. For example, only one message producer per connection. In below scenario we will be created N producers for the same connection. Also in a Nifi flow that connects a [ConsumeJMS|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.ConsumeJMS/] with a PublishJMS. Notice {{ConsumeJMS}} populates by default {{jms_destination}} flowfile attribute. was: Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is required to publish 1 message to the destination {{D}} using [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. The message is a flow file in the processor input queue. It is important to know that internally the processor is using [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] to reuse objects and a [worker|https://g
[jira] [Updated] (NIFI-7563) Optimize the usage of JMS sessions and message producers
[ https://issues.apache.org/jira/browse/NIFI-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gardella Juan Pablo updated NIFI-7563: -- Description: Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is required to publish 1 message to the destination {{D}} using [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. The message is a flow file in the processor input queue. It is important to know that internally the processor is using [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] to reuse objects and a [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] to be able to use in thread safe manner. For JMS publishers, the default configuration is to cache connections, sessions (only 1) and message producers. *Preconditions* # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute defined. Due to NIFI-7561, it should contain the word {{queue}} or {{topic}}. Also notice {{jms_destination}} should be ignored, as suggested at NIFI-7564. That will limit the scenario only when {{jms_replyTo}} attribute is defined. # For simplicity, the processor is the first time it processes messages. *Scenario* # Processor picks the message. The [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] is created. # Connection {{C1}} and session {{S1}} are created. The [Message|https://docs.oracle.com/javaee/7/api/javax/jms/Message.html] {{M1_S1}} is created and [MessageProducer|https://docs.oracle.com/javaee/7/api/javax/jms/MessageProducer.html] {{MP_S1}} created too. Required to deliver first message at [JMSPublisher#publish|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L65]. # S1 and C1 are stored in {{CachingConnectionFactory}}. The caching connection factory is created at [AbstractJMSProcessor.java#L208|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L208]. # An attempt to create a new connection and a new session are requested to the connection factory to create destination defined in the header {{jms_destination}} at [JMSPublisher.java#L131|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L131]. Notice the connection {{C1}} is reused although *{{S1}} is not reused* (it is required to check internal logic in CachingConnectionFactory to understand why not). A new session {{S2}} is created and stored in the {{CachingConnectionFactory}} as the new cached session. # Message is published and {{S1}} and {{MP_S1}} are closed. As {{S1}} is not in the cache, it is physically closed and {{MP_S1}}. # At this point of time, the cached objects are {{C1}}, {{S2}}. *Ideally*, all resources should be reused. The scenario if it is applied to N consecutive messages create a lot of sessions and message producers. We found this issue by adding an [Interceptor|https://activemq.apache.org/interceptors] to an [Apache ActiveMQ v5.x|http://activemq.apache.org/components/classic/] broker to detect the optimal usage of resources. For example, only one message producer per connection. In below scenario we will be created N producers for the same connection. was: Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is required to publish 1 message to the destination {{D}} using [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. The message is a flow file in the processor input queue. It is important to know that internally the processor is using [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] to reuse objects and a [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] to be able to use in thread safe manner. For JMS publishers, the default configuration is to cac
[jira] [Updated] (NIFI-7563) Optimize the usage of JMS sessions and message producers
[ https://issues.apache.org/jira/browse/NIFI-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gardella Juan Pablo updated NIFI-7563: -- Description: Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is required to publish 1 message to the destination {{D}} using [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. The message is a flow file in the processor input queue. It is important to know that internally the processor is using [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] to reuse objects and a [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] to be able to use in thread safe manner. For JMS publishers, the default configuration is to cache connections, sessions (only 1) and message producers. *Preconditions* # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute defined. Due to NIFI-7561, it should contain the word {{queue}} or {{topic}}. Also notice jms_destination should be ignored, as suggested at NIFI-7564. # For simplicity, the processor is the first time it processes messages. *Scenario* # Processor picks the message. The [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] is created. # Connection {{C1}} and session {{S1}} are created. The [Message|https://docs.oracle.com/javaee/7/api/javax/jms/Message.html] {{M1_S1}} is created and [MessageProducer|https://docs.oracle.com/javaee/7/api/javax/jms/MessageProducer.html] {{MP_S1}} created too. Required to deliver first message at [JMSPublisher#publish|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L65]. # S1 and C1 are stored in {{CachingConnectionFactory}}. The caching connection factory is created at [AbstractJMSProcessor.java#L208|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L208]. # An attempt to create a new connection and a new session are requested to the connection factory to create destination defined in the header {{jms_destination}} at [JMSPublisher.java#L131|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L131]. Notice the connection {{C1}} is reused although *{{S1}} is not reused* (it is required to check internal logic in CachingConnectionFactory to understand why not). A new session {{S2}} is created and stored in the {{CachingConnectionFactory}} as the new cached session. # Message is published and {{S1}} and {{MP_S1}} are closed. As {{S1}} is not in the cache, it is physically closed and {{MP_S1}}. # At this point of time, the cached objects are {{C1}}, {{S2}}. *Ideally*, all resources should be reused. The scenario if it is applied to N consecutive messages create a lot of sessions and message producers. We found this issue by adding an [Interceptor|https://activemq.apache.org/interceptors] to an [Apache ActiveMQ v5.x|http://activemq.apache.org/components/classic/] broker to detect the optimal usage of resources. For example, only one message producer per connection. In below scenario we will be created N producers for the same connection. was: Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is required to publish 1 message to the destination {{D}} using [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. The message is a flow file in the processor input queue. It is important to know that internally the processor is using [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] to reuse objects and a [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] to be able to use in thread safe manner. For JMS publishers, the default configuration is to cache connections, sessions (only 1) and message producers. *Preconditions* # Flowf
[jira] [Updated] (NIFI-7563) Optimize the usage of JMS sessions and message producers
[ https://issues.apache.org/jira/browse/NIFI-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gardella Juan Pablo updated NIFI-7563: -- Description: Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is required to publish 1 message to the destination {{D}} using [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. The message is a flow file in the processor input queue. It is important to know that internally the processor is using [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] to reuse objects and a [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] to be able to use in thread safe manner. For JMS publishers, the default configuration is to cache connections, sessions (only 1) and message producers. *Preconditions* # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute defined. Due to NIFI-7561, it should contain the word {{queue}} or {{topic}}. Also notice jms_destination should be ignored, as suggested at NIFI-7564. # For simplicity, the processor is the first time it processes messages. *Scenario* # Processor picks the message. The [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] is created. # Connection {{C1}} and session {{S1}} are created. The [Message|https://docs.oracle.com/javaee/7/api/javax/jms/Message.html] {{M1_S1}} is created and [MessageProducer|https://docs.oracle.com/javaee/7/api/javax/jms/MessageProducer.html] {{MP_S1}} created too. Required to deliver first message at [JMSPublisher#publish|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L65]. # S1 and C1 are stored in {{CachingConnectionFactory}}. The caching connection factory is created at [AbstractJMSProcessor.java#L208|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L208]. # An attempt to create a new connection and a new session are requested to the connection factory to create destination defined in the header {{jms_destination}} at [JMSPublisher.java#L131|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L131]. Notice the connection {{C1}} is reused although *{{S1}} is not reused* (it is required to check internal logic in CachingConnectionFactory to understand why not). A new session {{S2}} is created and stored in the {{CachingConnectionFactory}} as the new cached session. # Message is published and {{S1}} and {{MP_S1}} are closed. As {{S1}} is not in the cache, it is physically closed and {{MP_S1}}. # At this point of time, the cached objects are {{C1}}, {{S2}}. *Ideally*, all resources should be reused. The scenario if it is applied to N consecutive messages create a lot of sessions and message producers. We found this issue by adding an [Interceptor|https://activemq.apache.org/interceptors] to an Apache ActiveMQ broker to detect the optimal usage of resources. For example, only one message producer per connection. In below scenario we will be created N producers for the same connection. was: Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is required to publish 1 message to the destination {{D}} using [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. The message is a flow file in the processor input queue. It is important to know that internally the processor is using [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] to reuse objects and a [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] to be able to use in thread safe manner. For JMS publishers, the default configuration is to cache connections, sessions (only 1) and message producers. *Preconditions* # Flowfile has either {{jms_destination}} or {{jms_replyTo}} a
[jira] [Updated] (NIFI-7563) Optimize the usage of JMS sessions and message producers
[ https://issues.apache.org/jira/browse/NIFI-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gardella Juan Pablo updated NIFI-7563: -- Summary: Optimize the usage of JMS sessions and message producers (was: Optimize the usage of JMS sessions) > Optimize the usage of JMS sessions and message producers > > > Key: NIFI-7563 > URL: https://issues.apache.org/jira/browse/NIFI-7563 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.6.0, 1.8.0, 1.7.1, 1.10.0, 1.9.2, 1.11.4 >Reporter: Gardella Juan Pablo >Assignee: Gardella Juan Pablo >Priority: Minor > > Below an scenario to reproduce the non optimize usage of JMS resources. > Suppose it is required to publish 1 message to the destination {{D}} using > [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. > The message is a flow file in the processor input queue. > It is important to know that internally the processor is using > [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] > to reuse objects and a > [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] > to be able to use in thread safe manner. For JMS publishers, the default > configuration is to cache connections, sessions (only 1) and message > producers. > *Preconditions* > # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute > defined. Due to NIFI-7561, it should contain the word {{queue}} or {{topic}}. > Also notice jms_destination should be ignored, as suggested at NIFI-7564. > # For simplicity, the processor is the first time it processes messages. > *Scenario* > # Processor picks the message. The > [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] > is created. > # Connection {{C1}} and session {{S1}} are created. The > [Message|https://docs.oracle.com/javaee/7/api/javax/jms/Message.html] > {{M1_S1}} is created and > [MessageProducer|https://docs.oracle.com/javaee/7/api/javax/jms/MessageProducer.html] > {{MP_S1}} created too. Required to deliver first message at > [JMSPublisher#publish|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L65]. > # S1 and C1 are stored in {{CachingConnectionFactory}}. The caching > connection factory is created at > [AbstractJMSProcessor.java#L208|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L208]. > # An attempt to create a new connection and a new session are requested to > the connection factory to create destination defined in the header > {{jms_destination}} at > [JMSPublisher.java#L131|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L131]. > Notice the connection {{C1}} is reused although *{{S1}} is not reused* (it > is required to check internal logic in CachingConnectionFactory to understand > why not). A new session {{S2}} is created and stored in the > {{CachingConnectionFactory}} as the new cached session. > # Message is published and {{S1}} and {{MP_S1}} are closed. As {{S1}} is not > in the cache, it is physically closed and {{MP_S1}}. > # At this point of time, the cached objects are {{C1}}, {{S2}}. *Ideally*, > all resources should be reused. > The scenario if it is applied to N consecutive messages create a lot of > sessions and message producers. We found this issue by adding an > [Interceptor|https://activemq.apache.org/interceptors] to detect the optimal > usage of resources. For example, only one message producer per connection. In > below scenario we will be created N producers for the same connection. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7563) Optimize the usage of JMS sessions
[ https://issues.apache.org/jira/browse/NIFI-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gardella Juan Pablo updated NIFI-7563: -- Description: Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is required to publish 1 message to the destination {{D}} using [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. The message is a flow file in the processor input queue. It is important to know that internally the processor is using [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] to reuse objects and a [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] to be able to use in thread safe manner. For JMS publishers, the default configuration is to cache connections, sessions (only 1) and message producers. *Preconditions* # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute defined. Due to NIFI-7561, it should contain the word {{queue}} or {{topic}}. Also notice jms_destination should be ignored, as suggested at NIFI-7564. # For simplicity, the processor is the first time it processes messages. *Scenario* # Processor picks the message. The [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] is created. # Connection {{C1}} and session {{S1}} are created. The [Message|https://docs.oracle.com/javaee/7/api/javax/jms/Message.html] {{M1_S1}} is created and [MessageProducer|https://docs.oracle.com/javaee/7/api/javax/jms/MessageProducer.html] {{MP_S1}} created too. Required to deliver first message at [JMSPublisher#publish|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L65]. # S1 and C1 are stored in {{CachingConnectionFactory}}. The caching connection factory is created at [AbstractJMSProcessor.java#L208|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L208]. # An attempt to create a new connection and a new session are requested to the connection factory to create destination defined in the header {{jms_destination}} at [JMSPublisher.java#L131|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L131]. Notice the connection {{C1}} is reused although *{{S1}} is not reused* (it is required to check internal logic in CachingConnectionFactory to understand why not). A new session {{S2}} is created and stored in the {{CachingConnectionFactory}} as the new cached session. # Message is published and {{S1}} and {{MP_S1}} are closed. As {{S1}} is not in the cache, it is physically closed and {{MP_S1}}. # At this point of time, the cached objects are {{C1}}, {{S2}}. *Ideally*, all resources should be reused. The scenario if it is applied to N consecutive messages create a lot of sessions and message producers. We found this issue by adding an [Interceptor|https://activemq.apache.org/interceptors] to detect the optimal usage of resources. For example, only one message producer per connection. In below scenario we will be created N producers for the same connection. was: Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is required to publish 1 message to the destination {{D}} using [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. The message is a flow file in the processor input queue. It is important to know that internally the processor is using [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] to reuse objects and a [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] to be able to use in thread safe manner. For JMS publishers, the default configuration is to cache connections, sessions (only 1) and message producers. *Preconditions* # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute defined. Due to NIFI
[jira] [Updated] (NIFI-7563) Optimize the usage of JMS sessions
[ https://issues.apache.org/jira/browse/NIFI-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gardella Juan Pablo updated NIFI-7563: -- Description: Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is required to publish 1 message to the destination {{D}} using [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. The message is a flow file in the processor input queue. It is important to know that internally the processor is using [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] to reuse objects and a [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] to be able to use in thread safe manner. For JMS publishers, the default configuration is to cache connections, sessions (only 1) and message producers. *Preconditions* # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute defined. Due to NIFI-7561, it should contain the word {{queue}} or {{topic}}. # For simplicity, the processor is the first time it processes messages. *Scenario* # Processor picks the message. The [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] is created. # Connection {{C1}} and session {{S1}} are created. The [Message|https://docs.oracle.com/javaee/7/api/javax/jms/Message.html] {{M1_S1}} is created and [MessageProducer|https://docs.oracle.com/javaee/7/api/javax/jms/MessageProducer.html] {{MP_S1}} created too. Required to deliver first message at [JMSPublisher#publish|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L65]. # S1 and C1 are stored in {{CachingConnectionFactory}}. The caching connection factory is created at [AbstractJMSProcessor.java#L208|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L208]. # An attempt to create a new connection and a new session are requested to the connection factory to create destination defined in the header {{jms_destination}} at [JMSPublisher.java#L131|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSPublisher.java#L131]. Notice the connection {{C1}} is reused although *{{S1}} is not reused* (it is required to check internal logic in CachingConnectionFactory to understand why not). A new session {{S2}} is created and stored in the {{CachingConnectionFactory}} as the new cached session. # Message is published and {{S1}} and {{MP_S1}} are closed. As {{S1}} is not in the cache, it is physically closed and {{MP_S1}}. # At this point of time, the cached objects are {{C1}}, {{S2}}. *Ideally*, all resources should be reused. The scenario if it is applied to N consecutive messages create a lot of sessions and message producers. We found this issue by adding an [Interceptor|https://activemq.apache.org/interceptors] to detect the optimal usage of resources. For example, only one message producer per connection. In below scenario we will be created N producers for the same connection. was: [MessageProducer|https://docs.oracle.com/javaee/7/api/javax/jms/MessageProducer.html] objects are created by [Session|https://docs.oracle.com/javaee/7/api/javax/jms/Session.html]. Below an scenario to reproduce the problem. Suppose it is required to publish 1 message to the destination {{D}} using [PublishJMS|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-jms-processors-nar/1.11.4/org.apache.nifi.jms.processors.PublishJMS/index.html]. The message is a flow file in the processor input queue. It is important to know that internally the processor is using [CachingConnectionFactory|https://github.com/spring-projects/spring-framework/blob/master/spring-jms/src/main/java/org/springframework/jms/connection/CachingConnectionFactory.java] to reuse objects and a [worker|https://github.com/apache/nifi/blob/a1b245e051245bb6c65e7b5ffc6ee982669b7ab7/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/AbstractJMSProcessor.java#L180] to be able to use in thread safe manner. For JMS publishers, the default configuration is to cache connections, sessions (only 1) and message producers. *Preconditions* # Flowfile h