Re: Camel Cxf Failover causes leak memory on periodic http requests
Hi,
In my point of view, the problem is not in camel : a dump of JVM memory
shows that created objects (org.apache.cxf.phase.
PhaseInterceptor*) are retained by the ConcurrentHashMap that is
instanciated by a class in CXF code
(org.apache.cxf.clustering.FailoverTargetSelector).
Those objects are not collected and remain in this map for a long term and
cause the OOM error.
Any idea
thanks
On Tue, Dec 17, 2013 at 1:29 PM, Willem Jiang willem.ji...@gmail.comwrote:
Hi,
It not easy to upgrade the camel version in the service.
You can using Main[1] or camel:run maven plugin[2] to start your route.
It could more easy to verify if the memory leak issue still there.
[1]
http://camel.apache.org/running-camel-standalone-and-have-it-keep-running.html
[2]http://camel.apache.org/camel-run-maven-goal.html
--
Willem Jiang
Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com(http://willemjiang.blogspot.com/)
(English)
http://jnn.iteye.com(http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On December 17, 2013 at 5:39:03 PM, LORENZA Adnan (adnan.lore...@gmail.com)
wrote:
Hi Willem,
Thanks for this quick answer.
My camel component is deployed on a jbi servicemix-fuse-3.5-00.
If I upgrade my component to Camel 2.12 do you think that I can deploy
it
on servicemix 3.4 or servicemix-fuse-3.5-00 ?
Thanks
On Tue, Dec 17, 2013 at 10:27 AM, Willem Jiang wrote:
Hi,
Your camel version is quite old (it is about two years old), and
we don’t
provide community support for that version.
Can you try to run the test with some latest released Camel ?
--
Willem Jiang
Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com(http://willemjiang.blogspot.com/
)
(English)
http://jnn.iteye.com(http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On December 17, 2013 at 5:03:01 PM, LORENZA Adnan (
adnan.lore...@gmail.com)
wrote:
Hi,
I developed a camel cxf component that is configured to failover
to
alternate addresses in case of connections /availability
failures.
The camel component is a simple timer that sends every second
a request to
a webservice.
The camel route is the following :
public void configure() throws Exception {
from(timer:timerRetrieveAvailableJobs?fixedRate=trueperiod=1000)
.setBody(constant(getRequestMessage())
.setHeader(CxfConstants.OPERATION_NAME,
constant(getAvailableJobs))
.to(cxf:bean:cxfJobsWsEndpoint?synchronous=trueloggingFeatureEnabled=true);
}
private String getRequestMessage(){
//Build the right request
...
}
The camel-context.xml configures the clustering failover
as below :
...
serviceClass=net.jobs.ws.myPTServiceClass address=
http://localhost/myservice/JobWS;
class=org.apache.cxf.clustering.SequentialStrategy
http://server1/JobWS
http://server2/JobWS
...
After a few hours, a dump of JVM memory shows that some objects
of CXF
(org.apage.cxf.) are created and remain in the memory.
They are not
accessible by the GC. Their number continues to grow and causes
a memory
leak after a long term :
command : jmap -histo:live 25698 | grep cxf
8: 90660 9428640
[Lorg.apache.cxf.phase.PhaseInterceptorChain$InterceptorHolder;
20: 135989 3263736
org.apache.cxf.phase.PhaseInterceptorChain$InterceptorHolder
22: 45328 2900992 org.apache.cxf.message.MessageImpl
24: 45330 2175840
org.apache.cxf.phase.PhaseInterceptorChain
25: 22664 1994432 org.apache.cxf.message.ExchangeImpl
33: 45328 1087872
org.apache.cxf.phase.PhaseInterceptorChain$PhaseInterceptorIterator
36: 22664 906560
org.apache.cxf.clustering.FailoverTargetSelector$InvocationContext
40: 22664 725248
org.apache.cxf.helpers.LoadingByteArrayOutputStream$1
41: 45328 725248 org.apache.cxf.binding.soap.SoapMessage
46: 22664 543936
org.apache.cxf.message.MessageContentsList
51: 22664 362624
org.apache.cxf.helpers.LoadingByteArrayOutputStream
52: 22664 362624
org.apache.cxf.clustering.FailoverTargetSelector$InvocationKey
I tried the same test without failover configuration and I
was
surprised :
the component continues to work without memory problem. The
objects above
are collected and removed by the GC.
It seems that the classes that implement the clustering failover
retain
created objects and prevent their removal.
The test has been done on the following configurations :
* apache-servicemix-3.4.0, cxf 2.4.4, camel 2.8.3
* apache-servicemix-3.5.0-fuse-00-00, cxf 2.2.11, camel
2.5
Any help will be appreciated.
Thanks
Regards
Adnan
Re: Camel Cxf Failover causes leak memory on periodic http requests
The CXF fail-over target selector appears to be cleaning up its state
properly, which also implies the actual clients properly complete the
exchanges, I've tried WS RS failover tests on the trunk.
Indeed, try upgrading somehow and see if it persists
Sergey
On 18/12/13 09:43, LORENZA Adnan wrote:
Hi,
In my point of view, the problem is not in camel : a dump of JVM memory
shows that created objects (org.apache.cxf.phase.
PhaseInterceptor*) are retained by the ConcurrentHashMap that is
instanciated by a class in CXF code
(org.apache.cxf.clustering.FailoverTargetSelector).
Those objects are not collected and remain in this map for a long term and
cause the OOM error.
Any idea
thanks
On Tue, Dec 17, 2013 at 1:29 PM, Willem Jiang willem.ji...@gmail.comwrote:
Hi,
It not easy to upgrade the camel version in the service.
You can using Main[1] or camel:run maven plugin[2] to start your route.
It could more easy to verify if the memory leak issue still there.
[1]
http://camel.apache.org/running-camel-standalone-and-have-it-keep-running.html
[2]http://camel.apache.org/camel-run-maven-goal.html
--
Willem Jiang
Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com(http://willemjiang.blogspot.com/)
(English)
http://jnn.iteye.com(http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On December 17, 2013 at 5:39:03 PM, LORENZA Adnan (adnan.lore...@gmail.com)
wrote:
Hi Willem,
Thanks for this quick answer.
My camel component is deployed on a jbi servicemix-fuse-3.5-00.
If I upgrade my component to Camel 2.12 do you think that I can deploy
it
on servicemix 3.4 or servicemix-fuse-3.5-00 ?
Thanks
On Tue, Dec 17, 2013 at 10:27 AM, Willem Jiang wrote:
Hi,
Your camel version is quite old (it is about two years old), and
we don’t
provide community support for that version.
Can you try to run the test with some latest released Camel ?
--
Willem Jiang
Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com(http://willemjiang.blogspot.com/
)
(English)
http://jnn.iteye.com(http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On December 17, 2013 at 5:03:01 PM, LORENZA Adnan (
adnan.lore...@gmail.com)
wrote:
Hi,
I developed a camel cxf component that is configured to failover
to
alternate addresses in case of connections /availability
failures.
The camel component is a simple timer that sends every second
a request to
a webservice.
The camel route is the following :
public void configure() throws Exception {
from(timer:timerRetrieveAvailableJobs?fixedRate=trueperiod=1000)
.setBody(constant(getRequestMessage())
.setHeader(CxfConstants.OPERATION_NAME,
constant(getAvailableJobs))
.to(cxf:bean:cxfJobsWsEndpoint?synchronous=trueloggingFeatureEnabled=true);
}
private String getRequestMessage(){
//Build the right request
...
}
The camel-context.xml configures the clustering failover
as below :
...
serviceClass=net.jobs.ws.myPTServiceClass address=
http://localhost/myservice/JobWS;
class=org.apache.cxf.clustering.SequentialStrategy
http://server1/JobWS
http://server2/JobWS
...
After a few hours, a dump of JVM memory shows that some objects
of CXF
(org.apage.cxf.) are created and remain in the memory.
They are not
accessible by the GC. Their number continues to grow and causes
a memory
leak after a long term :
command : jmap -histo:live 25698 | grep cxf
8: 90660 9428640
[Lorg.apache.cxf.phase.PhaseInterceptorChain$InterceptorHolder;
20: 135989 3263736
org.apache.cxf.phase.PhaseInterceptorChain$InterceptorHolder
22: 45328 2900992 org.apache.cxf.message.MessageImpl
24: 45330 2175840
org.apache.cxf.phase.PhaseInterceptorChain
25: 22664 1994432 org.apache.cxf.message.ExchangeImpl
33: 45328 1087872
org.apache.cxf.phase.PhaseInterceptorChain$PhaseInterceptorIterator
36: 22664 906560
org.apache.cxf.clustering.FailoverTargetSelector$InvocationContext
40: 22664 725248
org.apache.cxf.helpers.LoadingByteArrayOutputStream$1
41: 45328 725248 org.apache.cxf.binding.soap.SoapMessage
46: 22664 543936
org.apache.cxf.message.MessageContentsList
51: 22664 362624
org.apache.cxf.helpers.LoadingByteArrayOutputStream
52: 22664 362624
org.apache.cxf.clustering.FailoverTargetSelector$InvocationKey
I tried the same test without failover configuration and I
was
surprised :
the component continues to work without memory problem. The
objects above
are collected and removed by the GC.
It seems that the classes that implement the clustering failover
retain
created objects and prevent their removal.
The test has been done on the following configurations :
* apache-servicemix-3.4.0, cxf 2.4.4, camel 2.8.3
* apache-servicemix-3.5.0-fuse-00-00, cxf 2.2.11, camel
2.5
Any help will be appreciated.
Thanks
Regards
Adnan
Camel Cxf Failover causes leak memory on periodic http requests
Hi,
I developed a camel cxf component that is configured to failover to
alternate addresses in case of connections /availability failures.
The camel component is a simple timer that sends every second a request to
a webservice.
The camel route is the following :
public void configure() throws Exception {
from(timer:timerRetrieveAvailableJobs?fixedRate=trueperiod=1000)
.setBody(constant(getRequestMessage())
.setHeader(CxfConstants.OPERATION_NAME,
constant(getAvailableJobs))
.to(cxf:bean:cxfJobsWsEndpoint?synchronous=trueloggingFeatureEnabled=true);
}
private String getRequestMessage(){
//Build the right request
...
}
The camel-context.xml configures the clustering failover as below :
...
cxf:cxfEndpoint id=cxfJobsWsEndpoint
serviceClass=net.jobs.ws.myPTServiceClass address=
http://localhost/myservice/JobWS;
cxf:features
clustering:failover
clustering:strategy
bean
class=org.apache.cxf.clustering.SequentialStrategy
property name=alternateAddresses
util:list
valuehttp://server1/JobWS/value
valuehttp://server2/JobWS/value
/util:list
/property
/bean
/clustering:strategy
/clustering:failover
/cxf:features
cxf:properties
entry key=dataFormat value=MESSAGE /
/cxf:properties
/cxf:cxfEndpoint
...
After a few hours, a dump of JVM memory shows that some objects of CXF
(org.apage.cxf.) are created and remain in the memory. They are not
accessible by the GC. Their number continues to grow and causes a memory
leak after a long term :
command : jmap -histo:live 25698 | grep cxf
8: 90660 9428640
[Lorg.apache.cxf.phase.PhaseInterceptorChain$InterceptorHolder;
20:135989 3263736
org.apache.cxf.phase.PhaseInterceptorChain$InterceptorHolder
22: 453282900992 org.apache.cxf.message.MessageImpl
24: 453302175840
org.apache.cxf.phase.PhaseInterceptorChain
25: 226641994432 org.apache.cxf.message.ExchangeImpl
33: 453281087872
org.apache.cxf.phase.PhaseInterceptorChain$PhaseInterceptorIterator
36: 22664 906560
org.apache.cxf.clustering.FailoverTargetSelector$InvocationContext
40: 22664 725248
org.apache.cxf.helpers.LoadingByteArrayOutputStream$1
41: 45328 725248 org.apache.cxf.binding.soap.SoapMessage
46: 22664 543936
org.apache.cxf.message.MessageContentsList
51: 22664 362624
org.apache.cxf.helpers.LoadingByteArrayOutputStream
52: 22664 362624
org.apache.cxf.clustering.FailoverTargetSelector$InvocationKey
I tried the same test without failover configuration and I was surprised :
the component continues to work without memory problem. The objects above
are collected and removed by the GC.
It seems that the classes that implement the clustering failover retain
created objects and prevent their removal.
The test has been done on the following configurations :
* apache-servicemix-3.4.0, cxf 2.4.4, camel 2.8.3
* apache-servicemix-3.5.0-fuse-00-00, cxf 2.2.11, camel 2.5
Any help will be appreciated.
Thanks
Regards
Adnan
Re: Camel Cxf Failover causes leak memory on periodic http requests
Hi,
Your camel version is quite old (it is about two years old), and we don’t
provide community support for that version.
Can you try to run the test with some latest released Camel ?
--
Willem Jiang
Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com(http://willemjiang.blogspot.com/)
(English)
http://jnn.iteye.com(http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On December 17, 2013 at 5:03:01 PM, LORENZA Adnan (adnan.lore...@gmail.com)
wrote:
Hi,
I developed a camel cxf component that is configured to failover
to
alternate addresses in case of connections /availability failures.
The camel component is a simple timer that sends every second
a request to
a webservice.
The camel route is the following :
public void configure() throws Exception {
from(timer:timerRetrieveAvailableJobs?fixedRate=trueperiod=1000)
.setBody(constant(getRequestMessage())
.setHeader(CxfConstants.OPERATION_NAME,
constant(getAvailableJobs))
.to(cxf:bean:cxfJobsWsEndpoint?synchronous=trueloggingFeatureEnabled=true);
}
private String getRequestMessage(){
//Build the right request
...
}
The camel-context.xml configures the clustering failover
as below :
...
serviceClass=net.jobs.ws.myPTServiceClass address=
http://localhost/myservice/JobWS;
class=org.apache.cxf.clustering.SequentialStrategy
http://server1/JobWS
http://server2/JobWS
...
After a few hours, a dump of JVM memory shows that some objects
of CXF
(org.apage.cxf.) are created and remain in the memory.
They are not
accessible by the GC. Their number continues to grow and causes
a memory
leak after a long term :
command : jmap -histo:live 25698 | grep cxf
8: 90660 9428640
[Lorg.apache.cxf.phase.PhaseInterceptorChain$InterceptorHolder;
20: 135989 3263736
org.apache.cxf.phase.PhaseInterceptorChain$InterceptorHolder
22: 45328 2900992 org.apache.cxf.message.MessageImpl
24: 45330 2175840
org.apache.cxf.phase.PhaseInterceptorChain
25: 22664 1994432 org.apache.cxf.message.ExchangeImpl
33: 45328 1087872
org.apache.cxf.phase.PhaseInterceptorChain$PhaseInterceptorIterator
36: 22664 906560
org.apache.cxf.clustering.FailoverTargetSelector$InvocationContext
40: 22664 725248
org.apache.cxf.helpers.LoadingByteArrayOutputStream$1
41: 45328 725248 org.apache.cxf.binding.soap.SoapMessage
46: 22664 543936
org.apache.cxf.message.MessageContentsList
51: 22664 362624
org.apache.cxf.helpers.LoadingByteArrayOutputStream
52: 22664 362624
org.apache.cxf.clustering.FailoverTargetSelector$InvocationKey
I tried the same test without failover configuration and I was
surprised :
the component continues to work without memory problem. The
objects above
are collected and removed by the GC.
It seems that the classes that implement the clustering failover
retain
created objects and prevent their removal.
The test has been done on the following configurations :
* apache-servicemix-3.4.0, cxf 2.4.4, camel 2.8.3
* apache-servicemix-3.5.0-fuse-00-00, cxf 2.2.11, camel
2.5
Any help will be appreciated.
Thanks
Regards
Adnan
Re: Camel Cxf Failover causes leak memory on periodic http requests
Hi Willem,
Thanks for this quick answer.
My camel component is deployed on a jbi servicemix-fuse-3.5-00.
If I upgrade my component to Camel 2.12 do you think that I can deploy it
on servicemix 3.4 or servicemix-fuse-3.5-00 ?
Thanks
On Tue, Dec 17, 2013 at 10:27 AM, Willem Jiang willem.ji...@gmail.comwrote:
Hi,
Your camel version is quite old (it is about two years old), and we don’t
provide community support for that version.
Can you try to run the test with some latest released Camel ?
--
Willem Jiang
Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com(http://willemjiang.blogspot.com/)
(English)
http://jnn.iteye.com(http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On December 17, 2013 at 5:03:01 PM, LORENZA Adnan (adnan.lore...@gmail.com)
wrote:
Hi,
I developed a camel cxf component that is configured to failover
to
alternate addresses in case of connections /availability failures.
The camel component is a simple timer that sends every second
a request to
a webservice.
The camel route is the following :
public void configure() throws Exception {
from(timer:timerRetrieveAvailableJobs?fixedRate=trueperiod=1000)
.setBody(constant(getRequestMessage())
.setHeader(CxfConstants.OPERATION_NAME,
constant(getAvailableJobs))
.to(cxf:bean:cxfJobsWsEndpoint?synchronous=trueloggingFeatureEnabled=true);
}
private String getRequestMessage(){
//Build the right request
...
}
The camel-context.xml configures the clustering failover
as below :
...
serviceClass=net.jobs.ws.myPTServiceClass address=
http://localhost/myservice/JobWS;
class=org.apache.cxf.clustering.SequentialStrategy
http://server1/JobWS
http://server2/JobWS
...
After a few hours, a dump of JVM memory shows that some objects
of CXF
(org.apage.cxf.) are created and remain in the memory.
They are not
accessible by the GC. Their number continues to grow and causes
a memory
leak after a long term :
command : jmap -histo:live 25698 | grep cxf
8: 90660 9428640
[Lorg.apache.cxf.phase.PhaseInterceptorChain$InterceptorHolder;
20: 135989 3263736
org.apache.cxf.phase.PhaseInterceptorChain$InterceptorHolder
22: 45328 2900992 org.apache.cxf.message.MessageImpl
24: 45330 2175840
org.apache.cxf.phase.PhaseInterceptorChain
25: 22664 1994432 org.apache.cxf.message.ExchangeImpl
33: 45328 1087872
org.apache.cxf.phase.PhaseInterceptorChain$PhaseInterceptorIterator
36: 22664 906560
org.apache.cxf.clustering.FailoverTargetSelector$InvocationContext
40: 22664 725248
org.apache.cxf.helpers.LoadingByteArrayOutputStream$1
41: 45328 725248 org.apache.cxf.binding.soap.SoapMessage
46: 22664 543936
org.apache.cxf.message.MessageContentsList
51: 22664 362624
org.apache.cxf.helpers.LoadingByteArrayOutputStream
52: 22664 362624
org.apache.cxf.clustering.FailoverTargetSelector$InvocationKey
I tried the same test without failover configuration and I was
surprised :
the component continues to work without memory problem. The
objects above
are collected and removed by the GC.
It seems that the classes that implement the clustering failover
retain
created objects and prevent their removal.
The test has been done on the following configurations :
* apache-servicemix-3.4.0, cxf 2.4.4, camel 2.8.3
* apache-servicemix-3.5.0-fuse-00-00, cxf 2.2.11, camel
2.5
Any help will be appreciated.
Thanks
Regards
Adnan
Re: Camel Cxf Failover causes leak memory on periodic http requests
Hi,
It not easy to upgrade the camel version in the service.
You can using Main[1] or camel:run maven plugin[2] to start your route.
It could more easy to verify if the memory leak issue still there.
[1]http://camel.apache.org/running-camel-standalone-and-have-it-keep-running.html
[2]http://camel.apache.org/camel-run-maven-goal.html
--
Willem Jiang
Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com(http://willemjiang.blogspot.com/)
(English)
http://jnn.iteye.com(http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On December 17, 2013 at 5:39:03 PM, LORENZA Adnan (adnan.lore...@gmail.com)
wrote:
Hi Willem,
Thanks for this quick answer.
My camel component is deployed on a jbi servicemix-fuse-3.5-00.
If I upgrade my component to Camel 2.12 do you think that I can deploy
it
on servicemix 3.4 or servicemix-fuse-3.5-00 ?
Thanks
On Tue, Dec 17, 2013 at 10:27 AM, Willem Jiang wrote:
Hi,
Your camel version is quite old (it is about two years old), and
we don’t
provide community support for that version.
Can you try to run the test with some latest released Camel ?
--
Willem Jiang
Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com(http://willemjiang.blogspot.com/)
(English)
http://jnn.iteye.com(http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On December 17, 2013 at 5:03:01 PM, LORENZA Adnan (adnan.lore...@gmail.com)
wrote:
Hi,
I developed a camel cxf component that is configured to failover
to
alternate addresses in case of connections /availability
failures.
The camel component is a simple timer that sends every second
a request to
a webservice.
The camel route is the following :
public void configure() throws Exception {
from(timer:timerRetrieveAvailableJobs?fixedRate=trueperiod=1000)
.setBody(constant(getRequestMessage())
.setHeader(CxfConstants.OPERATION_NAME,
constant(getAvailableJobs))
.to(cxf:bean:cxfJobsWsEndpoint?synchronous=trueloggingFeatureEnabled=true);
}
private String getRequestMessage(){
//Build the right request
...
}
The camel-context.xml configures the clustering failover
as below :
...
serviceClass=net.jobs.ws.myPTServiceClass address=
http://localhost/myservice/JobWS;
class=org.apache.cxf.clustering.SequentialStrategy
http://server1/JobWS
http://server2/JobWS
...
After a few hours, a dump of JVM memory shows that some objects
of CXF
(org.apage.cxf.) are created and remain in the memory.
They are not
accessible by the GC. Their number continues to grow and causes
a memory
leak after a long term :
command : jmap -histo:live 25698 | grep cxf
8: 90660 9428640
[Lorg.apache.cxf.phase.PhaseInterceptorChain$InterceptorHolder;
20: 135989 3263736
org.apache.cxf.phase.PhaseInterceptorChain$InterceptorHolder
22: 45328 2900992 org.apache.cxf.message.MessageImpl
24: 45330 2175840
org.apache.cxf.phase.PhaseInterceptorChain
25: 22664 1994432 org.apache.cxf.message.ExchangeImpl
33: 45328 1087872
org.apache.cxf.phase.PhaseInterceptorChain$PhaseInterceptorIterator
36: 22664 906560
org.apache.cxf.clustering.FailoverTargetSelector$InvocationContext
40: 22664 725248
org.apache.cxf.helpers.LoadingByteArrayOutputStream$1
41: 45328 725248 org.apache.cxf.binding.soap.SoapMessage
46: 22664 543936
org.apache.cxf.message.MessageContentsList
51: 22664 362624
org.apache.cxf.helpers.LoadingByteArrayOutputStream
52: 22664 362624
org.apache.cxf.clustering.FailoverTargetSelector$InvocationKey
I tried the same test without failover configuration and I
was
surprised :
the component continues to work without memory problem. The
objects above
are collected and removed by the GC.
It seems that the classes that implement the clustering failover
retain
created objects and prevent their removal.
The test has been done on the following configurations :
* apache-servicemix-3.4.0, cxf 2.4.4, camel 2.8.3
* apache-servicemix-3.5.0-fuse-00-00, cxf 2.2.11, camel
2.5
Any help will be appreciated.
Thanks
Regards
Adnan
