[jira] [Updated] (CAMEL-9143) Producers that implement the ServicePoolAware interface cause memory leak due to JMX references
[
https://issues.apache.org/jira/browse/CAMEL-9143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-9143:
---
Fix Version/s: 2.15.4
2.16.0
> Producers that implement the ServicePoolAware interface cause memory leak due
> to JMX references
> ---
>
> Key: CAMEL-9143
> URL: https://issues.apache.org/jira/browse/CAMEL-9143
> Project: Camel
> Issue Type: Bug
> Components: camel-core
>Affects Versions: 2.14.1, 2.14.2, 2.15.0, 2.15.1
>Reporter: Bob Browning
> Fix For: 2.16.0, 2.15.4
>
>
> h4. Description
> Producer instances that implement the ServicePoolAware interface will leak
> memory if their route is stopped, with new producers being leaked every time
> the route is started/stopped.
> Known implementations that are affected are RemoteFileProducer (ftp, sftp)
> and Mina2Producer.
> This is due to the behaviour that the SendProcessor which when the route is
> stopped it shuts down it's `producerCache` instance.
> {code}
> protected void doStop() throws Exception {
> ServiceHelper.stopServices(producerCache, producer);
> }
> {code}
> this in turn calls `stopAndShutdownService(pool)` which will call stop on the
> SharedProducerServicePool instance which is a NOOP however it also calls
> shutdown which effects a stop of the global pool (this stops all the
> registered services and then clears the pool.
> {code}
> protected void doStop() throws Exception {
> // when stopping we intend to shutdown
> ServiceHelper.stopAndShutdownService(pool);
> try {
> ServiceHelper.stopAndShutdownServices(producers.values());
> } finally {
> // ensure producers are removed, and also from JMX
> for (Producer producer : producers.values()) {
> getCamelContext().removeService(producer);
> }
> }
> producers.clear();
> }
> {code}
> However no call to `context.removeService(Producer) is called for the entries
> from the pool only those singleton instances that were in the `producers` map
> hence the JMX `ManagedProducer` that is created when `doGetProducer` invokes
> {code}getCamelContext().addService(answer, false);
> {code} is never removed.
> Since the global pool is empty when the next request to get a producer is
> called a new producer is created, jmx wrapper and all, whilst the old
> instance remains orphaned retaining any objects that pertain to that instance.
> One workaround is for the producer to call
> {code}getEndpoint().getCamelContext().removeService(this){code} in it's stop
> method, however this is fairly obscure and it would probably be better to
> invoke removal of the producer when it is removed from the shared pool.
> Another issue of note is that when a route is shutdown that contains a
> SendProcessor due to the shutdown invocation on the
> SharedProcessorServicePool the global pool is cleared of `everything` and
> remains in `Stopped` state until another route starts it (although it is
> still accessed and used whilst in the `Stopped` state).
> h4. Impact
> For general use where there is no dynamic creation or passivation of routes
> this issue should be minimal, however in our use case where the routes are
> not static, there is a certain amount of recreation of routes as customer
> endpoints change and there is a need to passivate idle routes this causes a
> considerable memory leak (via SFTP in particular).
> h4. Test Case
> {code}
> package org.apache.camel.component;
> import com.google.common.util.concurrent.AtomicLongMap;
> import org.apache.camel.CamelContext;
> import org.apache.camel.Consumer;
> import org.apache.camel.Endpoint;
> import org.apache.camel.Exchange;
> import org.apache.camel.Processor;
> import org.apache.camel.Producer;
> import org.apache.camel.Route;
> import org.apache.camel.Service;
> import org.apache.camel.ServicePoolAware;
> import org.apache.camel.ServiceStatus;
> import org.apache.camel.builder.RouteBuilder;
> import org.apache.camel.impl.DefaultComponent;
> import org.apache.camel.impl.DefaultEndpoint;
> import org.apache.camel.impl.DefaultProducer;
> import org.apache.camel.support.LifecycleStrategySupport;
> import org.apache.camel.support.ServiceSupport;
> import org.apache.camel.test.junit4.CamelTestSupport;
> import org.junit.Test;
> import java.util.Map;
> import static com.google.common.base.Preconditions.checkNotNull;
> /**
> * Test memory behaviour of producers using {@link ServicePoolAware} when
> using JMX.
> */
> public class ServicePoolAwareLeakyTest extends CamelTestSupport {
> private static final String LEAKY_SIEVE_STABLE =
> "leaky://sieve-stable?plugged=true";
> private static final String
[jira] [Updated] (CAMEL-9143) Producers that implement the ServicePoolAware interface cause memory leak due to JMX references
[
https://issues.apache.org/jira/browse/CAMEL-9143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bob Browning updated CAMEL-9143:
Description:
h4. Description
Producer instances that implement the ServicePoolAware interface will leak
memory if their route is stopped, with new producers being leaked every time
the route is started/stopped.
Known implementations that are affected are RemoteFileProducer (ftp, sftp) and
Mina2Producer.
This is due to the behaviour that the SendProcessor which when the route is
stopped it shuts down it's `producerCache` instance.
{code}
protected void doStop() throws Exception {
ServiceHelper.stopServices(producerCache, producer);
}
{code}
this in turn calls `stopAndShutdownService(pool)` which will call stop on the
SharedProducerServicePool instance which is a NOOP however it also calls
shutdown which effects a stop of the global pool (this stops all the registered
services and then clears the pool.
{code}
protected void doStop() throws Exception {
// when stopping we intend to shutdown
ServiceHelper.stopAndShutdownService(pool);
try {
ServiceHelper.stopAndShutdownServices(producers.values());
} finally {
// ensure producers are removed, and also from JMX
for (Producer producer : producers.values()) {
getCamelContext().removeService(producer);
}
}
producers.clear();
}
{code}
However no call to `context.removeService(Producer) is called for the entries
from the pool only those singleton instances that were in the `producers` map
hence the JMX `ManagedProducer` that is created when `doGetProducer` invokes
{code}getCamelContext().addService(answer, false);
{code} is never removed.
Since the global pool is empty when the next request to get a producer is
called a new producer is created, jmx wrapper and all, whilst the old instance
remains orphaned retaining any objects that pertain to that instance.
One workaround is for the producer to call
{code}getEndpoint().getCamelContext().removeService(this){code} in it's stop
method, however this is fairly obscure and it would probably be better to
invoke removal of the producer when it is removed from the shared pool.
Another issue of note is that when a route is shutdown that contains a
SendProcessor due to the shutdown invocation on the SharedProcessorServicePool
the global pool is cleared of `everything` and remains in `Stopped` state until
another route starts it (although it is still accessed and used whilst in the
`Stopped` state).
h4. Impact
For general use where there is no dynamic creation or passivation of routes
this issue should be minimal, however in our use case where the routes are not
static, there is a certain amount of recreation of routes as customer endpoints
change and there is a need to passivate idle routes this causes a considerable
memory leak (via SFTP in particular).
h4. Test Case
{code}
package org.apache.camel.component;
import com.google.common.util.concurrent.AtomicLongMap;
import org.apache.camel.CamelContext;
import org.apache.camel.Consumer;
import org.apache.camel.Endpoint;
import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.apache.camel.Producer;
import org.apache.camel.Route;
import org.apache.camel.Service;
import org.apache.camel.ServicePoolAware;
import org.apache.camel.ServiceStatus;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultComponent;
import org.apache.camel.impl.DefaultEndpoint;
import org.apache.camel.impl.DefaultProducer;
import org.apache.camel.support.LifecycleStrategySupport;
import org.apache.camel.support.ServiceSupport;
import org.apache.camel.test.junit4.CamelTestSupport;
import org.junit.Test;
import java.util.Map;
import static com.google.common.base.Preconditions.checkNotNull;
/**
* Test memory behaviour of producers using {@link ServicePoolAware} when using
JMX.
*/
public class ServicePoolAwareLeakyTest extends CamelTestSupport {
private static final String LEAKY_SIEVE_STABLE =
"leaky://sieve-stable?plugged=true";
private static final String LEAKY_SIEVE_TRANSIENT =
"leaky://sieve-transient?plugged=true";
private static boolean isPatchApplied() {
return Boolean.parseBoolean(System.getProperty("patchApplied", "false"));
}
/**
* Component that provides leaks producers.
*/
private static class LeakySieveComponent extends DefaultComponent {
@Override
protected Endpoint createEndpoint(String uri, String remaining, Map parameters) throws Exception {
boolean plugged = "true".equalsIgnoreCase((String)
parameters.remove("plugged"));
return new LeakySieveEndpoint(uri, isPatchApplied() && plugged);
}
}
/**
* Endpoint that provides leaky producers.
*/
private static class
[jira] [Updated] (CAMEL-9143) Producers that implement the ServicePoolAware interface cause memory leak due to JMX references
[
https://issues.apache.org/jira/browse/CAMEL-9143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bob Browning updated CAMEL-9143:
Affects Version/s: (was: 2.14.3)
> Producers that implement the ServicePoolAware interface cause memory leak due
> to JMX references
> ---
>
> Key: CAMEL-9143
> URL: https://issues.apache.org/jira/browse/CAMEL-9143
> Project: Camel
> Issue Type: Bug
> Components: camel-core
>Affects Versions: 2.14.1, 2.14.2, 2.15.0, 2.15.1
>Reporter: Bob Browning
>
> h4. Description
> Producer instances that implement the ServicePoolAware interface will leak
> memory if their route is stopped, with new producers being leaked every time
> the route is started/stopped.
> Known implementations that are affected are RemoteFileProducer (ftp, sftp)
> and Mina2Producer.
> This is due to the behaviour that the SendProcessor which when the route is
> stopped it shuts down it's `producerCache` instance.
> {code}
> protected void doStop() throws Exception {
> ServiceHelper.stopServices(producerCache, producer);
> }
> {code}
> this in turn calls `stopAndShutdownService(pool)` which will call stop on the
> SharedProducerServicePool instance which is a NOOP however it also calls
> shutdown which effects a stop of the global pool (this stops all the
> registered services and then clears the pool.
> {code}
> protected void doStop() throws Exception {
> // when stopping we intend to shutdown
> ServiceHelper.stopAndShutdownService(pool);
> try {
> ServiceHelper.stopAndShutdownServices(producers.values());
> } finally {
> // ensure producers are removed, and also from JMX
> for (Producer producer : producers.values()) {
> getCamelContext().removeService(producer);
> }
> }
> producers.clear();
> }
> {code}
> However no call to `context.removeService(Producer) is called for the entries
> from the pool only those singleton instances that were in the `producers` map
> hence the JMX `ManagedProducer` that is created when `doGetProducer` invokes
> {code}getCamelContext().addService(answer, false);
> {code} is never removed.
> Since the global pool is empty when the next request to get a producer is
> called a new producer is created, jmx wrapper and all, whilst the old
> instance remains orphaned retaining any objects that pertain to that instance.
> One workaround is for the producer to call
> {code}getEndpoint().getCamelContext().removeService(this){code} in it's stop
> method, however this is fairly obscure and it would probably be better to
> invoke removal of the producer when it is removed from the shared pool.
> Another issue of note is that when a route is shutdown that contains a
> SendProcessor due to the shutdown invocation on the
> SharedProcessorServicePool the global pool is cleared of `everything` and
> remains in `Stopped` state until another route starts it (although it is
> still accessed and used whilst in the `Stopped` state).
> h4. Impact
> For general use where there is no dynamic creation or passivation of routes
> this issue should be minimal, however in our use case where the routes are
> not static, there is a certain amount of recreation of routes as customer
> endpoints change and there is a need to passivate idle routes this causes a
> considerable memory leak (via SFTP in particular).
> h4. Test Case
> {code}
> package org.apache.camel.component;
> import com.google.common.util.concurrent.AtomicLongMap;
> import org.apache.camel.CamelContext;
> import org.apache.camel.Consumer;
> import org.apache.camel.Endpoint;
> import org.apache.camel.Exchange;
> import org.apache.camel.Processor;
> import org.apache.camel.Producer;
> import org.apache.camel.Route;
> import org.apache.camel.Service;
> import org.apache.camel.ServicePoolAware;
> import org.apache.camel.ServiceStatus;
> import org.apache.camel.builder.RouteBuilder;
> import org.apache.camel.impl.DefaultComponent;
> import org.apache.camel.impl.DefaultEndpoint;
> import org.apache.camel.impl.DefaultProducer;
> import org.apache.camel.support.LifecycleStrategySupport;
> import org.apache.camel.support.ServiceSupport;
> import org.apache.camel.test.junit4.CamelTestSupport;
> import org.junit.Test;
> import java.util.Map;
> import static com.google.common.base.Preconditions.checkNotNull;
> /**
> * Test memory behaviour of producers using {@link ServicePoolAware} when
> using JMX.
> */
> public class ServicePoolAwareLeakyTest extends CamelTestSupport {
> private static final String LEAKY_SIEVE_STABLE =
> "leaky://sieve-stable?plugged=true";
> private static final String LEAKY_SIEVE_TRANSIENT =
>
