[jira] [Commented] (CAMEL-9143) Producers that implement the ServicePoolAware interface cause memory leak due to JMX references
[
https://issues.apache.org/jira/browse/CAMEL-9143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791696#comment-14791696
]
Claus Ibsen commented on CAMEL-9143:
Thanks for the very detailed report.
> Producers that implement the ServicePoolAware interface cause memory leak due
> to JMX references
> ---
>
> Key: CAMEL-9143
> URL: https://issues.apache.org/jira/browse/CAMEL-9143
> Project: Camel
> Issue Type: Bug
> Components: camel-core
>Affects Versions: 2.14.1, 2.14.2, 2.15.0, 2.15.1
>Reporter: Bob Browning
> Fix For: 2.16.0, 2.15.4
>
>
> h4. Description
> Producer instances that implement the ServicePoolAware interface will leak
> memory if their route is stopped, with new producers being leaked every time
> the route is started/stopped.
> Known implementations that are affected are RemoteFileProducer (ftp, sftp)
> and Mina2Producer.
> This is due to the behaviour that the SendProcessor which when the route is
> stopped it shuts down it's `producerCache` instance.
> {code}
> protected void doStop() throws Exception {
> ServiceHelper.stopServices(producerCache, producer);
> }
> {code}
> this in turn calls `stopAndShutdownService(pool)` which will call stop on the
> SharedProducerServicePool instance which is a NOOP however it also calls
> shutdown which effects a stop of the global pool (this stops all the
> registered services and then clears the pool.
> {code}
> protected void doStop() throws Exception {
> // when stopping we intend to shutdown
> ServiceHelper.stopAndShutdownService(pool);
> try {
> ServiceHelper.stopAndShutdownServices(producers.values());
> } finally {
> // ensure producers are removed, and also from JMX
> for (Producer producer : producers.values()) {
> getCamelContext().removeService(producer);
> }
> }
> producers.clear();
> }
> {code}
> However no call to `context.removeService(Producer) is called for the entries
> from the pool only those singleton instances that were in the `producers` map
> hence the JMX `ManagedProducer` that is created when `doGetProducer` invokes
> {code}getCamelContext().addService(answer, false);
> {code} is never removed.
> Since the global pool is empty when the next request to get a producer is
> called a new producer is created, jmx wrapper and all, whilst the old
> instance remains orphaned retaining any objects that pertain to that instance.
> One workaround is for the producer to call
> {code}getEndpoint().getCamelContext().removeService(this){code} in it's stop
> method, however this is fairly obscure and it would probably be better to
> invoke removal of the producer when it is removed from the shared pool.
> Another issue of note is that when a route is shutdown that contains a
> SendProcessor due to the shutdown invocation on the
> SharedProcessorServicePool the global pool is cleared of `everything` and
> remains in `Stopped` state until another route starts it (although it is
> still accessed and used whilst in the `Stopped` state).
> h4. Impact
> For general use where there is no dynamic creation or passivation of routes
> this issue should be minimal, however in our use case where the routes are
> not static, there is a certain amount of recreation of routes as customer
> endpoints change and there is a need to passivate idle routes this causes a
> considerable memory leak (via SFTP in particular).
> h4. Test Case
> {code}
> package org.apache.camel.component;
> import com.google.common.util.concurrent.AtomicLongMap;
> import org.apache.camel.CamelContext;
> import org.apache.camel.Consumer;
> import org.apache.camel.Endpoint;
> import org.apache.camel.Exchange;
> import org.apache.camel.Processor;
> import org.apache.camel.Producer;
> import org.apache.camel.Route;
> import org.apache.camel.Service;
> import org.apache.camel.ServicePoolAware;
> import org.apache.camel.ServiceStatus;
> import org.apache.camel.builder.RouteBuilder;
> import org.apache.camel.impl.DefaultComponent;
> import org.apache.camel.impl.DefaultEndpoint;
> import org.apache.camel.impl.DefaultProducer;
> import org.apache.camel.support.LifecycleStrategySupport;
> import org.apache.camel.support.ServiceSupport;
> import org.apache.camel.test.junit4.CamelTestSupport;
> import org.junit.Test;
> import java.util.Map;
> import static com.google.common.base.Preconditions.checkNotNull;
> /**
> * Test memory behaviour of producers using {@link ServicePoolAware} when
> using JMX.
> */
> public class ServicePoolAwareLeakyTest extends CamelTestSupport {
> private static final String LEAKY_SIEVE_STABLE =
> "leaky://sieve-stable?plugged=true";
>
[jira] [Commented] (CAMEL-9143) Producers that implement the ServicePoolAware interface cause memory leak due to JMX references
[
https://issues.apache.org/jira/browse/CAMEL-9143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791767#comment-14791767
]
Claus Ibsen commented on CAMEL-9143:
Your endpoint should not be singleton, or the producer should override
isSingleton and return false.
> Producers that implement the ServicePoolAware interface cause memory leak due
> to JMX references
> ---
>
> Key: CAMEL-9143
> URL: https://issues.apache.org/jira/browse/CAMEL-9143
> Project: Camel
> Issue Type: Bug
> Components: camel-core
>Affects Versions: 2.14.1, 2.14.2, 2.15.0, 2.15.1
>Reporter: Bob Browning
>Assignee: Claus Ibsen
> Fix For: 2.16.0, 2.15.4
>
>
> h4. Description
> Producer instances that implement the ServicePoolAware interface will leak
> memory if their route is stopped, with new producers being leaked every time
> the route is started/stopped.
> Known implementations that are affected are RemoteFileProducer (ftp, sftp)
> and Mina2Producer.
> This is due to the behaviour that the SendProcessor which when the route is
> stopped it shuts down it's `producerCache` instance.
> {code}
> protected void doStop() throws Exception {
> ServiceHelper.stopServices(producerCache, producer);
> }
> {code}
> this in turn calls `stopAndShutdownService(pool)` which will call stop on the
> SharedProducerServicePool instance which is a NOOP however it also calls
> shutdown which effects a stop of the global pool (this stops all the
> registered services and then clears the pool.
> {code}
> protected void doStop() throws Exception {
> // when stopping we intend to shutdown
> ServiceHelper.stopAndShutdownService(pool);
> try {
> ServiceHelper.stopAndShutdownServices(producers.values());
> } finally {
> // ensure producers are removed, and also from JMX
> for (Producer producer : producers.values()) {
> getCamelContext().removeService(producer);
> }
> }
> producers.clear();
> }
> {code}
> However no call to `context.removeService(Producer) is called for the entries
> from the pool only those singleton instances that were in the `producers` map
> hence the JMX `ManagedProducer` that is created when `doGetProducer` invokes
> {code}getCamelContext().addService(answer, false);
> {code} is never removed.
> Since the global pool is empty when the next request to get a producer is
> called a new producer is created, jmx wrapper and all, whilst the old
> instance remains orphaned retaining any objects that pertain to that instance.
> One workaround is for the producer to call
> {code}getEndpoint().getCamelContext().removeService(this){code} in it's stop
> method, however this is fairly obscure and it would probably be better to
> invoke removal of the producer when it is removed from the shared pool.
> Another issue of note is that when a route is shutdown that contains a
> SendProcessor due to the shutdown invocation on the
> SharedProcessorServicePool the global pool is cleared of `everything` and
> remains in `Stopped` state until another route starts it (although it is
> still accessed and used whilst in the `Stopped` state).
> h4. Impact
> For general use where there is no dynamic creation or passivation of routes
> this issue should be minimal, however in our use case where the routes are
> not static, there is a certain amount of recreation of routes as customer
> endpoints change and there is a need to passivate idle routes this causes a
> considerable memory leak (via SFTP in particular).
> h4. Test Case
> {code}
> package org.apache.camel.component;
> import com.google.common.util.concurrent.AtomicLongMap;
> import org.apache.camel.CamelContext;
> import org.apache.camel.Consumer;
> import org.apache.camel.Endpoint;
> import org.apache.camel.Exchange;
> import org.apache.camel.Processor;
> import org.apache.camel.Producer;
> import org.apache.camel.Route;
> import org.apache.camel.Service;
> import org.apache.camel.ServicePoolAware;
> import org.apache.camel.ServiceStatus;
> import org.apache.camel.builder.RouteBuilder;
> import org.apache.camel.impl.DefaultComponent;
> import org.apache.camel.impl.DefaultEndpoint;
> import org.apache.camel.impl.DefaultProducer;
> import org.apache.camel.support.LifecycleStrategySupport;
> import org.apache.camel.support.ServiceSupport;
> import org.apache.camel.test.junit4.CamelTestSupport;
> import org.junit.Test;
> import java.util.Map;
> import static com.google.common.base.Preconditions.checkNotNull;
> /**
> * Test memory behaviour of producers using {@link ServicePoolAware} when
> using JMX.
> */
> public class ServicePoolAwareLeakyTest extends CamelTestSupport {
>
[jira] [Commented] (CAMEL-9143) Producers that implement the ServicePoolAware interface cause memory leak due to JMX references
[
https://issues.apache.org/jira/browse/CAMEL-9143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791797#comment-14791797
]
Claus Ibsen commented on CAMEL-9143:
Though singleton endpoints/producer is possible with servicepoolaware also
though its not so common.
I got a fix and with the test case there is no mbean leaks anymore.
> Producers that implement the ServicePoolAware interface cause memory leak due
> to JMX references
> ---
>
> Key: CAMEL-9143
> URL: https://issues.apache.org/jira/browse/CAMEL-9143
> Project: Camel
> Issue Type: Bug
> Components: camel-core
>Affects Versions: 2.14.1, 2.14.2, 2.15.0, 2.15.1
>Reporter: Bob Browning
>Assignee: Claus Ibsen
> Fix For: 2.16.0, 2.15.4
>
>
> h4. Description
> Producer instances that implement the ServicePoolAware interface will leak
> memory if their route is stopped, with new producers being leaked every time
> the route is started/stopped.
> Known implementations that are affected are RemoteFileProducer (ftp, sftp)
> and Mina2Producer.
> This is due to the behaviour that the SendProcessor which when the route is
> stopped it shuts down it's `producerCache` instance.
> {code}
> protected void doStop() throws Exception {
> ServiceHelper.stopServices(producerCache, producer);
> }
> {code}
> this in turn calls `stopAndShutdownService(pool)` which will call stop on the
> SharedProducerServicePool instance which is a NOOP however it also calls
> shutdown which effects a stop of the global pool (this stops all the
> registered services and then clears the pool.
> {code}
> protected void doStop() throws Exception {
> // when stopping we intend to shutdown
> ServiceHelper.stopAndShutdownService(pool);
> try {
> ServiceHelper.stopAndShutdownServices(producers.values());
> } finally {
> // ensure producers are removed, and also from JMX
> for (Producer producer : producers.values()) {
> getCamelContext().removeService(producer);
> }
> }
> producers.clear();
> }
> {code}
> However no call to `context.removeService(Producer) is called for the entries
> from the pool only those singleton instances that were in the `producers` map
> hence the JMX `ManagedProducer` that is created when `doGetProducer` invokes
> {code}getCamelContext().addService(answer, false);
> {code} is never removed.
> Since the global pool is empty when the next request to get a producer is
> called a new producer is created, jmx wrapper and all, whilst the old
> instance remains orphaned retaining any objects that pertain to that instance.
> One workaround is for the producer to call
> {code}getEndpoint().getCamelContext().removeService(this){code} in it's stop
> method, however this is fairly obscure and it would probably be better to
> invoke removal of the producer when it is removed from the shared pool.
> Another issue of note is that when a route is shutdown that contains a
> SendProcessor due to the shutdown invocation on the
> SharedProcessorServicePool the global pool is cleared of `everything` and
> remains in `Stopped` state until another route starts it (although it is
> still accessed and used whilst in the `Stopped` state).
> h4. Impact
> For general use where there is no dynamic creation or passivation of routes
> this issue should be minimal, however in our use case where the routes are
> not static, there is a certain amount of recreation of routes as customer
> endpoints change and there is a need to passivate idle routes this causes a
> considerable memory leak (via SFTP in particular).
> h4. Test Case
> {code}
> package org.apache.camel.component;
> import com.google.common.util.concurrent.AtomicLongMap;
> import org.apache.camel.CamelContext;
> import org.apache.camel.Consumer;
> import org.apache.camel.Endpoint;
> import org.apache.camel.Exchange;
> import org.apache.camel.Processor;
> import org.apache.camel.Producer;
> import org.apache.camel.Route;
> import org.apache.camel.Service;
> import org.apache.camel.ServicePoolAware;
> import org.apache.camel.ServiceStatus;
> import org.apache.camel.builder.RouteBuilder;
> import org.apache.camel.impl.DefaultComponent;
> import org.apache.camel.impl.DefaultEndpoint;
> import org.apache.camel.impl.DefaultProducer;
> import org.apache.camel.support.LifecycleStrategySupport;
> import org.apache.camel.support.ServiceSupport;
> import org.apache.camel.test.junit4.CamelTestSupport;
> import org.junit.Test;
> import java.util.Map;
> import static com.google.common.base.Preconditions.checkNotNull;
> /**
> * Test memory behaviour of producers using {@link ServicePoolAware} when
> using JMX.
> */
>
[jira] [Commented] (CAMEL-9143) Producers that implement the ServicePoolAware interface cause memory leak due to JMX references
[
https://issues.apache.org/jira/browse/CAMEL-9143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791806#comment-14791806
]
Bob Browning commented on CAMEL-9143:
-
Yes in the TC as the intent was to simulate RemoteFileXxx it should override
singleton as RemoteFileProducer does, but it still causes the leak.
Prospect of the fix sounds good, cheers :)
> Producers that implement the ServicePoolAware interface cause memory leak due
> to JMX references
> ---
>
> Key: CAMEL-9143
> URL: https://issues.apache.org/jira/browse/CAMEL-9143
> Project: Camel
> Issue Type: Bug
> Components: camel-core
>Affects Versions: 2.14.1, 2.14.2, 2.15.0, 2.15.1
>Reporter: Bob Browning
>Assignee: Claus Ibsen
> Fix For: 2.16.0, 2.15.4
>
>
> h4. Description
> Producer instances that implement the ServicePoolAware interface will leak
> memory if their route is stopped, with new producers being leaked every time
> the route is started/stopped.
> Known implementations that are affected are RemoteFileProducer (ftp, sftp)
> and Mina2Producer.
> This is due to the behaviour that the SendProcessor which when the route is
> stopped it shuts down it's `producerCache` instance.
> {code}
> protected void doStop() throws Exception {
> ServiceHelper.stopServices(producerCache, producer);
> }
> {code}
> this in turn calls `stopAndShutdownService(pool)` which will call stop on the
> SharedProducerServicePool instance which is a NOOP however it also calls
> shutdown which effects a stop of the global pool (this stops all the
> registered services and then clears the pool.
> {code}
> protected void doStop() throws Exception {
> // when stopping we intend to shutdown
> ServiceHelper.stopAndShutdownService(pool);
> try {
> ServiceHelper.stopAndShutdownServices(producers.values());
> } finally {
> // ensure producers are removed, and also from JMX
> for (Producer producer : producers.values()) {
> getCamelContext().removeService(producer);
> }
> }
> producers.clear();
> }
> {code}
> However no call to `context.removeService(Producer) is called for the entries
> from the pool only those singleton instances that were in the `producers` map
> hence the JMX `ManagedProducer` that is created when `doGetProducer` invokes
> {code}getCamelContext().addService(answer, false);
> {code} is never removed.
> Since the global pool is empty when the next request to get a producer is
> called a new producer is created, jmx wrapper and all, whilst the old
> instance remains orphaned retaining any objects that pertain to that instance.
> One workaround is for the producer to call
> {code}getEndpoint().getCamelContext().removeService(this){code} in it's stop
> method, however this is fairly obscure and it would probably be better to
> invoke removal of the producer when it is removed from the shared pool.
> Another issue of note is that when a route is shutdown that contains a
> SendProcessor due to the shutdown invocation on the
> SharedProcessorServicePool the global pool is cleared of `everything` and
> remains in `Stopped` state until another route starts it (although it is
> still accessed and used whilst in the `Stopped` state).
> h4. Impact
> For general use where there is no dynamic creation or passivation of routes
> this issue should be minimal, however in our use case where the routes are
> not static, there is a certain amount of recreation of routes as customer
> endpoints change and there is a need to passivate idle routes this causes a
> considerable memory leak (via SFTP in particular).
> h4. Test Case
> {code}
> package org.apache.camel.component;
> import com.google.common.util.concurrent.AtomicLongMap;
> import org.apache.camel.CamelContext;
> import org.apache.camel.Consumer;
> import org.apache.camel.Endpoint;
> import org.apache.camel.Exchange;
> import org.apache.camel.Processor;
> import org.apache.camel.Producer;
> import org.apache.camel.Route;
> import org.apache.camel.Service;
> import org.apache.camel.ServicePoolAware;
> import org.apache.camel.ServiceStatus;
> import org.apache.camel.builder.RouteBuilder;
> import org.apache.camel.impl.DefaultComponent;
> import org.apache.camel.impl.DefaultEndpoint;
> import org.apache.camel.impl.DefaultProducer;
> import org.apache.camel.support.LifecycleStrategySupport;
> import org.apache.camel.support.ServiceSupport;
> import org.apache.camel.test.junit4.CamelTestSupport;
> import org.junit.Test;
> import java.util.Map;
> import static com.google.common.base.Preconditions.checkNotNull;
> /**
> * Test memory behaviour of producers using {@link ServicePoolAware} when
