Re: Load test on camel-netty

2013-08-20 Thread Claus Ibsen 
I would suggest to remove the 100 thread pool sizes as the out of the box
settings from Netty is better. Netty is asynchronous and it creates thread
pools based on the number of CPU cores etc.

Netty creates the threads up-front so having 100+ threads take up some
unnecessary memory.


On Tue, Aug 13, 2013 at 5:02 PM, Flavio Magacho - M4U 
flavio.maga...@m4u.com.br wrote:

 Hi, 

 I'm expecting some problems while running a load test on camel-netty
 component.

 ** **

 This is the route configuration:

 route id=rtRead

 from
 uri=netty:tcp://pos-tcp-server.host:9000?decoder=#myDecoderamp;encoder=#myEncoderamp;sync=trueamp;reuseAddress=trueamp;synchronous=falseamp;workerCount=100amp;maximumPoolSize=100”/
 

 log message=Lendo ${body} /

 to uri=ejb:local/GatewayBean?method=process /

 /route

 ** **

 After some test cycles, we receive some exceptions like this:

 2013-08-13 11:25:33,398 WARNING
 [org.jboss.netty.channel.DefaultChannelPipeline] [New I/O  worker #32] An
 exception was thrown by a user handler while handling an exception event
 ([id: 0x8c7d41bd, /10.10.0.28:53469 = /10.11.234.38:9000] EXCEPTION:
 java.lang.OutOfMemoryError: unable to create new native thread)

 java.lang.OutOfMemoryError: unable to create new native thread

 at java.lang.Thread.start0(Native Method)

 at java.lang.Thread.start(Thread.java:640)

 at
 java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:703)
 

 at
 java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:652)
 

 at
 org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor.doUnorderedExecute(MemoryAwareThreadPoolExecutor.java:452)
 

 at
 org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.execute(OrderedMemoryAwareThreadPoolExecutor.java:292)
 

 at
 org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor.doExecute(OrderedMemoryAwareThreadPoolExecutor.java:242)
 

 at
 org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor.execute(MemoryAwareThreadPoolExecutor.java:437)
 

 at
 org.jboss.netty.handler.execution.ExecutionHandler.handleUpstream(ExecutionHandler.java:172)
 

 at
 org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:533)***
 *

 at org.jboss.netty.channel.Channels$7.run(Channels.java:507)

 at
 org.jboss.netty.channel.socket.ChannelRunnableWrapper.run(ChannelRunnableWrapper.java:41)
 

 at
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.processEventQueue(AbstractNioWorker.java:453)
 

 at
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:330)
 

 at
 org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)

 at
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 

 at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 

 at java.lang.Thread.run(Thread.java:662)

 ** **

 ** **

 It sounds strange, our test case has only 5 simultaneous opened
 connections.

 For me we are heaving some leak of threads or we are having some problems
 to close the SocketChannel correctly.

 ** **

 Thanks,

 *Flavio Magacho*

 *Gerente de Desenvolvimento*

 Diretoria de Tecnologia da Informação

 [image: cid:image001.gif@01CB90D9.07FBDA40]  *M4U*
 +55 (21) 2546-4050 ▪ Ramal: 4082

 +55 (21) 8889-1572

 *þ** *Antes de imprimir, pense em sua responsabilidade e compromisso com
 o Meio Ambiente.

 *O conteúdo desta mensagem é confidencial e pode ser privilegiado. **É
 vedada a sua cópia ou divulgação.*

 *The contents of this message are confidential and may be privileged.
 Copying or disclosing is prohibited.*

 ** **




-- 
Claus Ibsen
-
Red Hat, Inc.
Email: cib...@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen

RE: Load test on camel-netty

2013-08-20 Thread Flavio Magacho - M4U 
Claus, 
Thanks for your answer, we have done that and nothing changes.


Flavio Magacho
Gerente de Desenvolvimento
Diretoria de Tecnologia da Informação
  M4U
+55 (21) 2546-4050 ▪ Ramal: 4082
+55 (21) 8889-1572
 Antes de imprimir, pense em sua responsabilidade e compromisso com o Meio 
Ambiente.
O conteúdo desta mensagem é confidencial e pode ser privilegiado. É vedada a 
sua cópia ou divulgação.
The contents of this message are confidential and may be privileged. Copying or 
disclosing is prohibited.


-Original Message-
From: Claus Ibsen [mailto:claus.ib...@gmail.com] 
Sent: terça-feira, 20 de agosto de 2013 12:00
To: users@camel.apache.org
Subject: Re: Load test on camel-netty

I would suggest to remove the 100 thread pool sizes as the out of the box 
settings from Netty is better. Netty is asynchronous and it creates thread 
pools based on the number of CPU cores etc.

Netty creates the threads up-front so having 100+ threads take up some 
unnecessary memory.


On Tue, Aug 13, 2013 at 5:02 PM, Flavio Magacho - M4U  
flavio.maga...@m4u.com.br wrote:

 Hi, 

 I'm expecting some problems while running a load test on camel-netty
 component.

 ** **

 This is the route configuration:

 route id=rtRead

 from
 uri=netty:tcp://pos-tcp-server.host:9000?decoder=#myDecoderamp;encod
 er=#myEncoderamp;sync=trueamp;reuseAddress=trueamp;synchronous=fals
 eamp;workerCount=100amp;maximumPoolSize=100”/
 

 log message=Lendo ${body} /

 to uri=ejb:local/GatewayBean?method=process /

 /route

 ** **

 After some test cycles, we receive some exceptions like this:

 2013-08-13 11:25:33,398 WARNING
 [org.jboss.netty.channel.DefaultChannelPipeline] [New I/O  worker #32] 
 An exception was thrown by a user handler while handling an exception 
 event
 ([id: 0x8c7d41bd, /10.10.0.28:53469 = /10.11.234.38:9000] EXCEPTION:
 java.lang.OutOfMemoryError: unable to create new native thread)

 java.lang.OutOfMemoryError: unable to create new native thread

 at java.lang.Thread.start0(Native Method)

 at java.lang.Thread.start(Thread.java:640)

 at
 java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadP
 oolExecutor.java:703)
 

 at
 java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.jav
 a:652)
 

 at
 org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor.doUnor
 deredExecute(MemoryAwareThreadPoolExecutor.java:452)
 

 at
 org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor
 $ChildExecutor.execute(OrderedMemoryAwareThreadPoolExecutor.java:292)
 

 at
 org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor
 .doExecute(OrderedMemoryAwareThreadPoolExecutor.java:242)
 

 at
 org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor.execut
 e(MemoryAwareThreadPoolExecutor.java:437)
 

 at
 org.jboss.netty.handler.execution.ExecutionHandler.handleUpstream(Exec
 utionHandler.java:172)
 

 at
 org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:533
 )***
 *

 at 
 org.jboss.netty.channel.Channels$7.run(Channels.java:507)

 at
 org.jboss.netty.channel.socket.ChannelRunnableWrapper.run(ChannelRunna
 bleWrapper.java:41)
 

 at
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.processEventQueue
 (AbstractNioWorker.java:453)
 

 at
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWo
 rker.java:330)
 

 at
 org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)***
 *

 at
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecu
 tor.java:886)
 

 at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
 java:908)
 

 at java.lang.Thread.run(Thread.java:662)

 ** **

 ** **

 It sounds strange, our test case has only 5 simultaneous opened
 connections.

 For me we are heaving some leak of threads or we are having some 
 problems to close the SocketChannel correctly.

 ** **

 Thanks,

 *Flavio Magacho*

 *Gerente de Desenvolvimento*

 Diretoria de Tecnologia da Informação

 [image: cid:image001.gif@01CB90D9.07FBDA40]  *M4U*
 +55 (21) 2546-4050 ▪ Ramal: 4082

 +55 (21) 8889-1572

 *þ** *Antes de imprimir, pense em sua responsabilidade e compromisso 
 com o Meio Ambiente.

 *O conteúdo desta mensagem é confidencial e pode ser privilegiado. **É 
 vedada a sua cópia ou divulgação.*

 *The contents of this message are confidential and may be privileged.
 Copying or disclosing is prohibited.*

 ** **




--
Claus Ibsen
-
Red Hat, Inc.
Email: cib...@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen

RE: Load test on camel-netty

2013-08-20 Thread Babak Vahdat 
If Oracle/Sun JDK is in use then maybe ask for a heap dump as the
OutOfMemoryError happens:

http://docs.oracle.com/cd/E15289_01/doc.40/e15062/optionxx.htm#BABBBEAJ

Then you could analyze the dump (e.g. using http://www.eclipse.org/mat/) to
see where that huge memory consumption or the leak (if any) comes from.

Babak


flavio.magacho wrote
 Claus, 
 Thanks for your answer, we have done that and nothing changes.
 
 
 Flavio Magacho
 Gerente de Desenvolvimento
 Diretoria de Tecnologia da Informação
   M4U
 +55 (21) 2546-4050 ▪ Ramal: 4082
 +55 (21) 8889-1572
  Antes de imprimir, pense em sua responsabilidade e compromisso com o
 Meio Ambiente.
 O conteúdo desta mensagem é confidencial e pode ser privilegiado. É vedada
 a sua cópia ou divulgação.
 The contents of this message are confidential and may be privileged.
 Copying or disclosing is prohibited.
 
 
 -Original Message-
 From: Claus Ibsen [mailto:

 claus.ibsen@

 ] 
 Sent: terça-feira, 20 de agosto de 2013 12:00
 To: 

 users@.apache

 Subject: Re: Load test on camel-netty
 
 I would suggest to remove the 100 thread pool sizes as the out of the box
 settings from Netty is better. Netty is asynchronous and it creates thread
 pools based on the number of CPU cores etc.
 
 Netty creates the threads up-front so having 100+ threads take up some
 unnecessary memory.
 
 
 On Tue, Aug 13, 2013 at 5:02 PM, Flavio Magacho - M4U  

 flavio.magacho@.com

 wrote:
 
 Hi, 

 I'm expecting some problems while running a load test on camel-netty
 component.

 ** **

 This is the route configuration:

 
 route id=rtRead
 

 
 from

  uri=netty:tcp://pos-tcp-server.host:9000?decoder=#myDecoderamp;encod
 er=#myEncoderamp;sync=trueamp;reuseAddress=trueamp;synchronous=fals
 eamp;workerCount=100amp;maximumPoolSize=100”/
 

 
 log message=Lendo ${body} /
 

 
 to uri=ejb:local/GatewayBean?method=process /
 

 
 /route
 

 ** **

 After some test cycles, we receive some exceptions like this:

 2013-08-13 11:25:33,398 WARNING
 [org.jboss.netty.channel.DefaultChannelPipeline] [New I/O  worker #32] 
 An exception was thrown by a user handler while handling an exception 
 event
 ([id: 0x8c7d41bd, /10.10.0.28:53469 = /10.11.234.38:9000] EXCEPTION:
 java.lang.OutOfMemoryError: unable to create new native thread)

 java.lang.OutOfMemoryError: unable to create new native thread

 at java.lang.Thread.start0(Native Method)

 at java.lang.Thread.start(Thread.java:640)

 at
 java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadP
 oolExecutor.java:703)
 

 at
 java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.jav
 a:652)
 

 at
 org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor.doUnor
 deredExecute(MemoryAwareThreadPoolExecutor.java:452)
 

 at
 org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor
 $ChildExecutor.execute(OrderedMemoryAwareThreadPoolExecutor.java:292)
 

 at
 org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor
 .doExecute(OrderedMemoryAwareThreadPoolExecutor.java:242)
 

 at
 org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor.execut
 e(MemoryAwareThreadPoolExecutor.java:437)
 

 at
 org.jboss.netty.handler.execution.ExecutionHandler.handleUpstream(Exec
 utionHandler.java:172)
 

 at
 org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:533
 )***
 *

 at 
 org.jboss.netty.channel.Channels$7.run(Channels.java:507)

 at
 org.jboss.netty.channel.socket.ChannelRunnableWrapper.run(ChannelRunna
 bleWrapper.java:41)
 

 at
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.processEventQueue
 (AbstractNioWorker.java:453)
 

 at
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWo
 rker.java:330)
 

 at
 org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)***
 *

 at
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecu
 tor.java:886)
 

 at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
 java:908)
 

 at java.lang.Thread.run(Thread.java:662)

 ** **

 ** **

 It sounds strange, our test case has only 5 simultaneous opened
 connections.

 For me we are heaving some leak of threads or we are having some 
 problems to close the SocketChannel correctly.

 ** **

 Thanks,

 *Flavio Magacho*

 *Gerente de Desenvolvimento*

 Diretoria de Tecnologia da Informação

 [image: cid:image001.gif@01CB90D9.07FBDA40]  *M4U*
 +55 (21) 2546-4050 ▪ Ramal: 4082

 +55 (21) 8889-1572

 *þ** *Antes de imprimir, pense em sua responsabilidade e compromisso 
 com o Meio Ambiente.

 *O conteúdo desta mensagem é confidencial e pode ser privilegiado. **É 
 vedada a sua cópia ou divulgação

RE: Load test on camel-netty

2013-08-20 Thread Flavio Magacho - M4U 
Thank you Vahdat, we will check this out!

Flavio Magacho
Gerente de Desenvolvimento
Diretoria de Tecnologia da Informação
  M4U
+55 (21) 2546-4050 ▪ Ramal: 4082
+55 (21) 8889-1572
 Antes de imprimir, pense em sua responsabilidade e compromisso com o Meio 
Ambiente.
O conteúdo desta mensagem é confidencial e pode ser privilegiado. É vedada a 
sua cópia ou divulgação.
The contents of this message are confidential and may be privileged. Copying or 
disclosing is prohibited.


-Original Message-
From: Babak Vahdat [mailto:babak.vah...@swissonline.ch] 
Sent: terça-feira, 20 de agosto de 2013 15:32
To: users@camel.apache.org
Subject: RE: Load test on camel-netty

If Oracle/Sun JDK is in use then maybe ask for a heap dump as the 
OutOfMemoryError happens:

http://docs.oracle.com/cd/E15289_01/doc.40/e15062/optionxx.htm#BABBBEAJ

Then you could analyze the dump (e.g. using http://www.eclipse.org/mat/) to see 
where that huge memory consumption or the leak (if any) comes from.

Babak


flavio.magacho wrote
 Claus,
 Thanks for your answer, we have done that and nothing changes.
 
 
 Flavio Magacho
 Gerente de Desenvolvimento
 Diretoria de Tecnologia da Informação
   M4U
 +55 (21) 2546-4050 ▪ Ramal: 4082
 +55 (21) 8889-1572
 þ Antes de imprimir, pense em sua responsabilidade e compromisso com o 
 Meio Ambiente.
 O conteúdo desta mensagem é confidencial e pode ser privilegiado. É 
 vedada a sua cópia ou divulgação.
 The contents of this message are confidential and may be privileged.
 Copying or disclosing is prohibited.
 
 
 -Original Message-
 From: Claus Ibsen [mailto:

 claus.ibsen@

 ]
 Sent: terça-feira, 20 de agosto de 2013 12:00
 To: 

 users@.apache

 Subject: Re: Load test on camel-netty
 
 I would suggest to remove the 100 thread pool sizes as the out of the 
 box settings from Netty is better. Netty is asynchronous and it 
 creates thread pools based on the number of CPU cores etc.
 
 Netty creates the threads up-front so having 100+ threads take up some 
 unnecessary memory.
 
 
 On Tue, Aug 13, 2013 at 5:02 PM, Flavio Magacho - M4U 

 flavio.magacho@.com

 wrote:
 
 Hi, 

 I'm expecting some problems while running a load test on camel-netty
 component.

 ** **

 This is the route configuration:

 
 route id=rtRead
 

 
 from

  
 uri=netty:tcp://pos-tcp-server.host:9000?decoder=#myDecoderamp;encod
 er=#myEncoderamp;sync=trueamp;reuseAddress=trueamp;synchronous=fal
 s eamp;workerCount=100amp;maximumPoolSize=100”/
 

 
 log message=Lendo ${body} /
 

 
 to uri=ejb:local/GatewayBean?method=process /
 

 
 /route
 

 ** **

 After some test cycles, we receive some exceptions like this:

 2013-08-13 11:25:33,398 WARNING
 [org.jboss.netty.channel.DefaultChannelPipeline] [New I/O  worker 
 #32] An exception was thrown by a user handler while handling an 
 exception event
 ([id: 0x8c7d41bd, /10.10.0.28:53469 = /10.11.234.38:9000] EXCEPTION:
 java.lang.OutOfMemoryError: unable to create new native thread)

 java.lang.OutOfMemoryError: unable to create new native thread

 at java.lang.Thread.start0(Native Method)

 at java.lang.Thread.start(Thread.java:640)

 at
 java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(Thread
 P
 oolExecutor.java:703)
 

 at
 java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.ja
 v
 a:652)
 

 at
 org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor.doUno
 r
 deredExecute(MemoryAwareThreadPoolExecutor.java:452)
 

 at
 org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecuto
 r
 $ChildExecutor.execute(OrderedMemoryAwareThreadPoolExecutor.java:292)
 

 at
 org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecuto
 r
 .doExecute(OrderedMemoryAwareThreadPoolExecutor.java:242)
 

 at
 org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor.execu
 t
 e(MemoryAwareThreadPoolExecutor.java:437)
 

 at
 org.jboss.netty.handler.execution.ExecutionHandler.handleUpstream(Exe
 c
 utionHandler.java:172)
 

 at
 org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:53
 3
 )***
 *

 at
 org.jboss.netty.channel.Channels$7.run(Channels.java:507)

 at
 org.jboss.netty.channel.socket.ChannelRunnableWrapper.run(ChannelRunn
 a
 bleWrapper.java:41)
 

 at
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.processEventQueu
 e
 (AbstractNioWorker.java:453)
 

 at
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioW
 o
 rker.java:330)
 

 at
 org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)**
 *
 *

 at
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
 u
 tor.java:886)
 

 at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
 java:908

Re: Load test on camel-netty

2013-08-14 Thread Babak Vahdat 
Hi

The maximumPoolSize option is *actually* the value for the corePoolSize of
the ThreadPool being in charge, which is set to 100 in your case (the option
naming seems a bit misleading to me). Can you try with smaller values for
this option to see if you can still reproduce the problem.

Other than that depending on which Camel version you make use of the Netty
OrderedThreadPoolExecutor is *not* in use, see
https://issues.apache.org/jira/browse/CAMEL-5689 which could cause
OutOfMemoryError, search for OutOfMemoryError here:

http://netty.io/3.6/api/org/jboss/netty/handler/execution/ExecutionHandler.html

Babak


flavio.magacho wrote
 Hi,
 I'm expecting some problems while running a load test on camel-netty
 component.
 
 This is the route configuration:
 route id=rtRead
 
 from
 uri=netty:tcp://pos-tcp-server.host:9000?decoder=#myDecoderamp;encoder=#myEncoderamp;sync=trueamp;reuseAddress=trueamp;synchronous=falseamp;workerCount=100amp;maximumPoolSize=100”/
 
 log message=Lendo ${body} /
 
 to uri=ejb:local/GatewayBean?method=process /
 /route
 After some test cycles, we receive some exceptions like this:
 2013-08-13 11:25:33,398 WARNING
 [org.jboss.netty.channel.DefaultChannelPipeline] [New I/O  worker #32] An
 exception was thrown by a user handler while handling an exception event
 ([id: 0x8c7d41bd, /10.10.0.28:53469 = /10.11.234.38:9000] EXCEPTION:
 java.lang.OutOfMemoryError: unable to create new native thread)
 java.lang.OutOfMemoryError: unable to create new native thread
 at java.lang.Thread.start0(Native Method)
 at java.lang.Thread.start(Thread.java:640)
 at
 java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:703)
 at
 java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:652)
 at
 org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor.doUnorderedExecute(MemoryAwareThreadPoolExecutor.java:452)
 at
 org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.execute(OrderedMemoryAwareThreadPoolExecutor.java:292)
 at
 org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor.doExecute(OrderedMemoryAwareThreadPoolExecutor.java:242)
 at
 org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor.execute(MemoryAwareThreadPoolExecutor.java:437)
 at
 org.jboss.netty.handler.execution.ExecutionHandler.handleUpstream(ExecutionHandler.java:172)
 at
 org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:533)
 at org.jboss.netty.channel.Channels$7.run(Channels.java:507)
 at
 org.jboss.netty.channel.socket.ChannelRunnableWrapper.run(ChannelRunnableWrapper.java:41)
 at
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.processEventQueue(AbstractNioWorker.java:453)
 at
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:330)
 at
 org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
 at
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 
 
 It sounds strange, our test case has only 5 simultaneous opened
 connections.
 For me we are heaving some leak of threads or we are having some problems
 to close the SocketChannel correctly.
 
 Thanks,
 Flavio Magacho
 Gerente de Desenvolvimento
 Diretoria de Tecnologia da Informação
 [cid:image001.gif@01CE981C.A891D980]  M4U
 +55 (21) 2546-4050 ▪ Ramal: 4082
 +55 (21) 8889-1572
 þ Antes de imprimir, pense em sua responsabilidade e compromisso com o
 Meio Ambiente.
 O conteúdo desta mensagem é confidencial e pode ser privilegiado. É vedada
 a sua cópia ou divulgação.
 The contents of this message are confidential and may be privileged.
 Copying or disclosing is prohibited.





--
View this message in context: 
http://camel.465427.n5.nabble.com/Load-test-on-camel-netty-tp5737221p5737237.html
Sent from the Camel - Users mailing list archive at Nabble.com.

RE: Load test on camel-netty

2013-08-14 Thread Flavio Magacho - M4U 
Vahdat, 
We have already tested with the default value of the maximumPoolSize option, 
and we got the same problem. The version 2.10.2.

Flavio Magacho
Gerente de Desenvolvimento
Diretoria de Tecnologia da Informação
  M4U
+55 (21) 2546-4050 ▪ Ramal: 4082
+55 (21) 8889-1572
 Antes de imprimir, pense em sua responsabilidade e compromisso com o Meio 
Ambiente.
O conteúdo desta mensagem é confidencial e pode ser privilegiado. É vedada a 
sua cópia ou divulgação.
The contents of this message are confidential and may be privileged. Copying or 
disclosing is prohibited.


-Original Message-
From: Babak Vahdat [mailto:babak.vah...@swissonline.ch] 
Sent: quarta-feira, 14 de agosto de 2013 06:08
To: users@camel.apache.org
Subject: Re: Load test on camel-netty

Hi

The maximumPoolSize option is *actually* the value for the corePoolSize of the 
ThreadPool being in charge, which is set to 100 in your case (the option naming 
seems a bit misleading to me). Can you try with smaller values for this option 
to see if you can still reproduce the problem.

Other than that depending on which Camel version you make use of the Netty 
OrderedThreadPoolExecutor is *not* in use, see
https://issues.apache.org/jira/browse/CAMEL-5689 which could cause 
OutOfMemoryError, search for OutOfMemoryError here:

http://netty.io/3.6/api/org/jboss/netty/handler/execution/ExecutionHandler.html

Babak


flavio.magacho wrote
 Hi,
 I'm expecting some problems while running a load test on camel-netty 
 component.
 
 This is the route configuration:
 route id=rtRead
 
 from
 uri=netty:tcp://pos-tcp-server.host:9000?decoder=#myDecoderamp;encod
 er=#myEncoderamp;sync=trueamp;reuseAddress=trueamp;synchronous=fals
 eamp;workerCount=100amp;maximumPoolSize=100”/
 
 log message=Lendo ${body} /
 
 to uri=ejb:local/GatewayBean?method=process / /route After some 
 test cycles, we receive some exceptions like this:
 2013-08-13 11:25:33,398 WARNING
 [org.jboss.netty.channel.DefaultChannelPipeline] [New I/O  worker #32] 
 An exception was thrown by a user handler while handling an exception 
 event
 ([id: 0x8c7d41bd, /10.10.0.28:53469 = /10.11.234.38:9000] EXCEPTION:
 java.lang.OutOfMemoryError: unable to create new native thread)
 java.lang.OutOfMemoryError: unable to create new native thread
 at java.lang.Thread.start0(Native Method)
 at java.lang.Thread.start(Thread.java:640)
 at
 java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:703)
 at
 java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:652)
 at
 org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor.doUnorderedExecute(MemoryAwareThreadPoolExecutor.java:452)
 at
 org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.execute(OrderedMemoryAwareThreadPoolExecutor.java:292)
 at
 org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor.doExecute(OrderedMemoryAwareThreadPoolExecutor.java:242)
 at
 org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor.execute(MemoryAwareThreadPoolExecutor.java:437)
 at
 org.jboss.netty.handler.execution.ExecutionHandler.handleUpstream(ExecutionHandler.java:172)
 at
 org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:533)
 at org.jboss.netty.channel.Channels$7.run(Channels.java:507)
 at
 org.jboss.netty.channel.socket.ChannelRunnableWrapper.run(ChannelRunnableWrapper.java:41)
 at
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.processEventQueue(AbstractNioWorker.java:453)
 at
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:330)
 at
 org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
 at
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 
 
 It sounds strange, our test case has only 5 simultaneous opened 
 connections.
 For me we are heaving some leak of threads or we are having some 
 problems to close the SocketChannel correctly.
 
 Thanks,
 Flavio Magacho
 Gerente de Desenvolvimento
 Diretoria de Tecnologia da Informação
 [cid:image001.gif@01CE981C.A891D980]  M4U
 +55 (21) 2546-4050 ▪ Ramal: 4082
 +55 (21) 8889-1572
 þ Antes de imprimir, pense em sua responsabilidade e compromisso com o 
 Meio Ambiente.
 O conteúdo desta mensagem é confidencial e pode ser privilegiado. É 
 vedada a sua cópia ou divulgação.
 The contents of this message are confidential and may be privileged.
 Copying or disclosing is prohibited.





--
View this message in context: 
http://camel.465427.n5.nabble.com/Load-test-on-camel-netty-tp5737221p5737237.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Load test on camel-netty

2013-08-14 Thread Babak Vahdat 
Any chance to test on the latest version of the 2.10.x branch as 2.10.2 is
about 10 months old, or maybe even 2.11.1 which relies on a newer Netty
version.

Babak

Am 14.08.13 21:20 schrieb Flavio Magacho - M4U unter
flavio.maga...@m4u.com.br:

Vahdat, 
We have already tested with the default value of the maximumPoolSize
option, and we got the same problem. The version 2.10.2.

Flavio Magacho
Gerente de Desenvolvimento
Diretoria de Tecnologia da Informação
  M4U
+55 (21) 2546-4050 ▪ Ramal: 4082
+55 (21) 8889-1572
 Antes de imprimir, pense em sua responsabilidade e compromisso com o
Meio Ambiente.
O conteúdo desta mensagem é confidencial e pode ser privilegiado. É
vedada a sua cópia ou divulgação.
The contents of this message are confidential and may be privileged.
Copying or disclosing is prohibited.


-Original Message-
From: Babak Vahdat [mailto:babak.vah...@swissonline.ch]
Sent: quarta-feira, 14 de agosto de 2013 06:08
To: users@camel.apache.org
Subject: Re: Load test on camel-netty

Hi

The maximumPoolSize option is *actually* the value for the corePoolSize
of the ThreadPool being in charge, which is set to 100 in your case (the
option naming seems a bit misleading to me). Can you try with smaller
values for this option to see if you can still reproduce the problem.

Other than that depending on which Camel version you make use of the
Netty OrderedThreadPoolExecutor is *not* in use, see
https://issues.apache.org/jira/browse/CAMEL-5689 which could cause
OutOfMemoryError, search for OutOfMemoryError here:

http://netty.io/3.6/api/org/jboss/netty/handler/execution/ExecutionHandler
.html

Babak


flavio.magacho wrote
 Hi,
 I'm expecting some problems while running a load test on camel-netty
 component.
 
 This is the route configuration:
 route id=rtRead
 
 from
 uri=netty:tcp://pos-tcp-server.host:9000?decoder=#myDecoderamp;encod
 er=#myEncoderamp;sync=trueamp;reuseAddress=trueamp;synchronous=fals
 eamp;workerCount=100amp;maximumPoolSize=100”/
 
 log message=Lendo ${body} /
 
 to uri=ejb:local/GatewayBean?method=process / /route After some
 test cycles, we receive some exceptions like this:
 2013-08-13 11:25:33,398 WARNING
 [org.jboss.netty.channel.DefaultChannelPipeline] [New I/O  worker #32]
 An exception was thrown by a user handler while handling an exception
 event
 ([id: 0x8c7d41bd, /10.10.0.28:53469 = /10.11.234.38:9000] EXCEPTION:
 java.lang.OutOfMemoryError: unable to create new native thread)
 java.lang.OutOfMemoryError: unable to create new native thread
 at java.lang.Thread.start0(Native Method)
 at java.lang.Thread.start(Thread.java:640)
 at
 
java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPool
Executor.java:703)
 at
 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:6
52)
 at
 
org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor.doUnorder
edExecute(MemoryAwareThreadPoolExecutor.java:452)
 at
 
org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$Ch
ildExecutor.execute(OrderedMemoryAwareThreadPoolExecutor.java:292)
 at
 
org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor.do
Execute(OrderedMemoryAwareThreadPoolExecutor.java:242)
 at
 
org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor.execute(M
emoryAwareThreadPoolExecutor.java:437)
 at
 
org.jboss.netty.handler.execution.ExecutionHandler.handleUpstream(Executi
onHandler.java:172)
 at
 org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:533)
 at org.jboss.netty.channel.Channels$7.run(Channels.java:507)
 at
 
org.jboss.netty.channel.socket.ChannelRunnableWrapper.run(ChannelRunnable
Wrapper.java:41)
 at
 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.processEventQueue(Ab
stractNioWorker.java:453)
 at
 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorke
r.java:330)
 at
 org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
 at
 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor
.java:886)
 at
 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
a:908)
 at java.lang.Thread.run(Thread.java:662)
 
 
 It sounds strange, our test case has only 5 simultaneous opened
 connections.
 For me we are heaving some leak of threads or we are having some
 problems to close the SocketChannel correctly.
 
 Thanks,
 Flavio Magacho
 Gerente de Desenvolvimento
 Diretoria de Tecnologia da Informação
 [cid:image001.gif@01CE981C.A891D980]  M4U
 +55 (21) 2546-4050 ▪ Ramal: 4082
 +55 (21) 8889-1572
 þ Antes de imprimir, pense em sua responsabilidade e compromisso com o
 Meio Ambiente.
 O conteúdo desta mensagem é confidencial e pode ser privilegiado. É
 vedada a sua cópia ou divulgação.
 The contents of this message are confidential and may be privileged.
 Copying