[jira] [Commented] (DIRMINA-1173) Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly() for ever
[
https://issues.apache.org/jira/browse/DIRMINA-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721051#comment-17721051
]
Jonathan Valliere commented on DIRMINA-1173:
Think of it like this. All of the Processor Pool threads share the same
FilterChain. So if you have 8 cores there are 16 Processor Threads. If you
add a ExecutorFilter with 4 threads then then all the tasks produced by
those 16 threads will then be executed on the 4 shared threads and it will
bottleneck.
> Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly()
> for ever
> ---
>
> Key: DIRMINA-1173
> URL: https://issues.apache.org/jira/browse/DIRMINA-1173
> Project: MINA
> Issue Type: Bug
> Components: Core
>Affects Versions: 2.2.1
>Reporter: KMVS
>Priority: Critical
> Attachments: dumpLatest-1.log
>
>
> Hi All,
> I have attached thread dump too for analysis.
> I have migrated from 2.0.21 to 2.023 for solving CVE, i have seen thread
> blocking issue, So used latest mina verstion 2.2.1 but still threads were
> blocked here is the sample code.Thread hungs at {*}awaitUninterruptibly{*}.
> Once this issue comes in sub sequest launches nothing will work all threads
> will be blocked forever,i have to restart the process to make it work. For
> single thread working fine,if i start 50-100 threads this thread blocking
> issue will surface.I am using the same NioSocketConnector across the threads.
> Here is an scenario, I have one gate way IP which i connect initially ,this
> gate way will return list of ips.
> now i will close the previous IOsession and will create a nio connection with
> the first ip in the list.if it did n't work then again i will try to
> connect to next ip etc..
> All these are in the critical section. i.e i will acquire a lock and then
> after successful connection i will release the lock.
> But problem i have noticed is with the awaitUninterruptibly().
> I have a state machine too. So connection set up is in one thread and
> response processing is in another thread.
>
> *Thread 1:*
> Public class g10CaptureService
> {
>
> private static final ProtocolCodecFilter probeCodecFilter = new
> ProtocolCodecFilter(new ProbeCodecFactory(G10Message.class));
> *private static final ExecutorFilter executorFilter = new
> ExecutorFilter(16,32);*
> private static final G10GPBMessageIoFilter gpbMessageFilter = new
> G10GPBMessageIoFilter(G10ParserContextFactory.getG10ParsingAndEncodingInstance());
> static
> {
> initConnectors()
> }
> protected static void initConnectors()
> {
> StateMachine stateMachine =
> StateMachineFactory.getInstance(IoHandlerTransition.class).create(
> G10MinaClient.CONNECTED, new G10MinaClient(processor));
> IoHandler ioHandler = new
> StateMachineProxyBuilder().setStateContextLookup(
> new IoSessionStateContextLookup(new StateContextFactory() {
> @Override
> public StateContext create() {
> final G10StateContext stateContext = new
> G10StateContext();
> stateContext.setStartedTime(new Date());
> return stateContext;
> }
> })).create(IoHandler.class, stateMachine);
>
>
> //Global connector across the system, i.e multiple threads uses the same
> connector.
> NioSocketConnector connector = new NioSocketConnector();
> connector.getFilterChain().addLast("LoggingFilter",
> G10CaptureService.loggingFilter);
> connector.getFilterChain().addLast("codecFilter",
> G10CaptureService.probeCodecFilter);
> connector.getFilterChain().addLast("executorFilter",
> G10CaptureService.executorFilter);
> connector.getFilterChain().addLast("gpbMessageFilter",
> G10CaptureService.gpbMessageFilter);
> connector.getFilterChain().addLast("keepAliveFilter",
> G10CaptureService.keepAliveFilter);
> connector.setHandler(ioHandler);
> }
>
> public void StartRecordCapture()
> {
> connectionLock.lock();
> try
> {
> ConnectFuture primaryConnectFuture = connector.connect(primaryAddress,
> initializer);
> //hungs forever if the no. of threads are more than 30
> primaryConnectFuture.awaitUninterruptibly();//no time out specified
> if (!primaryConnectFuture.isConnected())
> {
>
> if (handleIOException(searchExpression, captureHandler)) {
> return;
> }
> LOG.info("{} Apache mina connection setup time out
> happend.",
> handleConnectionFailed(primaryAddress, captureHandler,
> "Primary IP connection timeout");
>
Re: [jira] [Commented] (DIRMINA-1173) Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly() for ever
Think of it like this. All of the Processor Pool threads share the same
FilterChain. So if you have 8 cores there are 16 Processor Threads. If you
add a ExecutorFilter with 4 threads then then all the tasks produced by
those 16 threads will then be executed on the 4 shared threads and it will
bottleneck.
On Tue, May 9, 2023 at 2:39 PM KMVS (Jira) wrote:
>
> [
> https://issues.apache.org/jira/browse/DIRMINA-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721040#comment-17721040
> ]
>
> KMVS commented on DIRMINA-1173:
> ---
>
> What is the effect of using our own thread pool in the execution filter ?
>
> *private static final ExecutorFilter executorFilter = new
> ExecutorFilter(16,32);*
>
> vs
>
> *private static final ExecutorFilter executorFilter = new
> ExecutorFilter(Executors.newCachedThreadPool());*
>
>
> > Apache mina 2.2.1 threads blocking on
> ConnectFuture.awaitUninterruptibly() for ever
> >
> ---
> >
> > Key: DIRMINA-1173
> > URL: https://issues.apache.org/jira/browse/DIRMINA-1173
> > Project: MINA
> > Issue Type: Bug
> > Components: Core
> >Affects Versions: 2.2.1
> >Reporter: KMVS
> >Priority: Critical
> > Attachments: dumpLatest-1.log
> >
> >
> > Hi All,
> > I have attached thread dump too for analysis.
> > I have migrated from 2.0.21 to 2.023 for solving CVE, i have seen thread
> blocking issue, So used latest mina verstion 2.2.1 but still threads were
> blocked here is the sample code.Thread hungs at
> {*}awaitUninterruptibly{*}. Once this issue comes in sub sequest launches
> nothing will work all threads will be blocked forever,i have to restart the
> process to make it work. For single thread working fine,if i start 50-100
> threads this thread blocking issue will surface.I am using the same
> NioSocketConnector across the threads.
> > Here is an scenario, I have one gate way IP which i connect initially
> ,this gate way will return list of ips.
> > now i will close the previous IOsession and will create a nio connection
> with the first ip in the list.if it did n't work then again i will try to
> > connect to next ip etc..
> > All these are in the critical section. i.e i will acquire a lock and
> then after successful connection i will release the lock.
> > But problem i have noticed is with the awaitUninterruptibly().
> > I have a state machine too. So connection set up is in one thread and
> response processing is in another thread.
> >
> > *Thread 1:*
> > Public class g10CaptureService
> > {
> >
> > private static final ProtocolCodecFilter probeCodecFilter = new
> ProtocolCodecFilter(new ProbeCodecFactory(G10Message.class));
> > *private static final ExecutorFilter executorFilter = new
> ExecutorFilter(16,32);*
> > private static final G10GPBMessageIoFilter gpbMessageFilter = new
> G10GPBMessageIoFilter(G10ParserContextFactory.getG10ParsingAndEncodingInstance());
> > static
> > {
> > initConnectors()
> > }
> > protected static void initConnectors()
> > {
> > StateMachine stateMachine =
> StateMachineFactory.getInstance(IoHandlerTransition.class).create(
> > G10MinaClient.CONNECTED, new G10MinaClient(processor));
> > IoHandler ioHandler = new
> StateMachineProxyBuilder().setStateContextLookup(
> > new IoSessionStateContextLookup(new
> StateContextFactory() {
> > @Override
> > public StateContext create() {
> > final G10StateContext stateContext = new
> G10StateContext();
> > stateContext.setStartedTime(new Date());
> > return stateContext;
> > }
> > })).create(IoHandler.class, stateMachine);
> >
> >
> > //Global connector across the system, i.e multiple threads uses the same
> connector.
> > NioSocketConnector connector = new NioSocketConnector();
> > connector.getFilterChain().addLast("LoggingFilter",
> G10CaptureService.loggingFilter);
> > connector.getFilterChain().addLast("codecFilter",
> G10CaptureService.probeCodecFilter);
> > connector.getFilterChain().addLast("executorFilter",
> G10CaptureService.executorFilter);
> > connector.getFilterChain().addLast("gpbMessageFilter",
> G10CaptureService.gpbMessageFilter);
> > connector.getFilterChain().addLast("keepAliveFilter",
> G10CaptureService.keepAliveFilter);
> > connector.setHandler(ioHandler);
> > }
> >
> > public void StartRecordCapture()
> > {
> > connectionLock.lock();
> > try
> > {
> > ConnectFuture primaryConnectFuture = connector.connect(primaryAddress,
> initializer);
> > //hungs forever if the no. of threads are more than 30
> > primaryConnectFuture.awaitUninterruptibly();//no time out specified
> > if (!primaryConnectFuture.
[jira] [Commented] (DIRMINA-1173) Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly() for ever
[
https://issues.apache.org/jira/browse/DIRMINA-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721040#comment-17721040
]
KMVS commented on DIRMINA-1173:
---
What is the effect of using our own thread pool in the execution filter ?
*private static final ExecutorFilter executorFilter = new
ExecutorFilter(16,32);*
vs
*private static final ExecutorFilter executorFilter = new
ExecutorFilter(Executors.newCachedThreadPool());*
> Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly()
> for ever
> ---
>
> Key: DIRMINA-1173
> URL: https://issues.apache.org/jira/browse/DIRMINA-1173
> Project: MINA
> Issue Type: Bug
> Components: Core
>Affects Versions: 2.2.1
>Reporter: KMVS
>Priority: Critical
> Attachments: dumpLatest-1.log
>
>
> Hi All,
> I have attached thread dump too for analysis.
> I have migrated from 2.0.21 to 2.023 for solving CVE, i have seen thread
> blocking issue, So used latest mina verstion 2.2.1 but still threads were
> blocked here is the sample code.Thread hungs at {*}awaitUninterruptibly{*}.
> Once this issue comes in sub sequest launches nothing will work all threads
> will be blocked forever,i have to restart the process to make it work. For
> single thread working fine,if i start 50-100 threads this thread blocking
> issue will surface.I am using the same NioSocketConnector across the threads.
> Here is an scenario, I have one gate way IP which i connect initially ,this
> gate way will return list of ips.
> now i will close the previous IOsession and will create a nio connection with
> the first ip in the list.if it did n't work then again i will try to
> connect to next ip etc..
> All these are in the critical section. i.e i will acquire a lock and then
> after successful connection i will release the lock.
> But problem i have noticed is with the awaitUninterruptibly().
> I have a state machine too. So connection set up is in one thread and
> response processing is in another thread.
>
> *Thread 1:*
> Public class g10CaptureService
> {
>
> private static final ProtocolCodecFilter probeCodecFilter = new
> ProtocolCodecFilter(new ProbeCodecFactory(G10Message.class));
> *private static final ExecutorFilter executorFilter = new
> ExecutorFilter(16,32);*
> private static final G10GPBMessageIoFilter gpbMessageFilter = new
> G10GPBMessageIoFilter(G10ParserContextFactory.getG10ParsingAndEncodingInstance());
> static
> {
> initConnectors()
> }
> protected static void initConnectors()
> {
> StateMachine stateMachine =
> StateMachineFactory.getInstance(IoHandlerTransition.class).create(
> G10MinaClient.CONNECTED, new G10MinaClient(processor));
> IoHandler ioHandler = new
> StateMachineProxyBuilder().setStateContextLookup(
> new IoSessionStateContextLookup(new StateContextFactory() {
> @Override
> public StateContext create() {
> final G10StateContext stateContext = new
> G10StateContext();
> stateContext.setStartedTime(new Date());
> return stateContext;
> }
> })).create(IoHandler.class, stateMachine);
>
>
> //Global connector across the system, i.e multiple threads uses the same
> connector.
> NioSocketConnector connector = new NioSocketConnector();
> connector.getFilterChain().addLast("LoggingFilter",
> G10CaptureService.loggingFilter);
> connector.getFilterChain().addLast("codecFilter",
> G10CaptureService.probeCodecFilter);
> connector.getFilterChain().addLast("executorFilter",
> G10CaptureService.executorFilter);
> connector.getFilterChain().addLast("gpbMessageFilter",
> G10CaptureService.gpbMessageFilter);
> connector.getFilterChain().addLast("keepAliveFilter",
> G10CaptureService.keepAliveFilter);
> connector.setHandler(ioHandler);
> }
>
> public void StartRecordCapture()
> {
> connectionLock.lock();
> try
> {
> ConnectFuture primaryConnectFuture = connector.connect(primaryAddress,
> initializer);
> //hungs forever if the no. of threads are more than 30
> primaryConnectFuture.awaitUninterruptibly();//no time out specified
> if (!primaryConnectFuture.isConnected())
> {
>
> if (handleIOException(searchExpression, captureHandler)) {
> return;
> }
> LOG.info("{} Apache mina connection setup time out
> happend.",
> handleConnectionFailed(primaryAddress, captureHandler,
> "Primary IP connection timeout");
> return;
> }
> }catch(Exception e)
> {
[jira] [Commented] (DIRMINA-1173) Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly() for ever
[
https://issues.apache.org/jira/browse/DIRMINA-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720971#comment-17720971
]
Jonathan Valliere commented on DIRMINA-1173:
Yes, all sockets
CONFIDENTIALITY NOTICE: The contents of this email message and any
attachments are intended solely for the addressee(s) and may contain
confidential and/or privileged information and may be legally protected
from disclosure.
> Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly()
> for ever
> ---
>
> Key: DIRMINA-1173
> URL: https://issues.apache.org/jira/browse/DIRMINA-1173
> Project: MINA
> Issue Type: Bug
> Components: Core
>Affects Versions: 2.2.1
>Reporter: KMVS
>Priority: Critical
> Attachments: dumpLatest-1.log
>
>
> Hi All,
> I have attached thread dump too for analysis.
> I have migrated from 2.0.21 to 2.023 for solving CVE, i have seen thread
> blocking issue, So used latest mina verstion 2.2.1 but still threads were
> blocked here is the sample code.Thread hungs at {*}awaitUninterruptibly{*}.
> Once this issue comes in sub sequest launches nothing will work all threads
> will be blocked forever,i have to restart the process to make it work. For
> single thread working fine,if i start 50-100 threads this thread blocking
> issue will surface.I am using the same NioSocketConnector across the threads.
> Here is an scenario, I have one gate way IP which i connect initially ,this
> gate way will return list of ips.
> now i will close the previous IOsession and will create a nio connection with
> the first ip in the list.if it did n't work then again i will try to
> connect to next ip etc..
> All these are in the critical section. i.e i will acquire a lock and then
> after successful connection i will release the lock.
> But problem i have noticed is with the awaitUninterruptibly().
> I have a state machine too. So connection set up is in one thread and
> response processing is in another thread.
>
> *Thread 1:*
> Public class g10CaptureService
> {
>
> private static final ProtocolCodecFilter probeCodecFilter = new
> ProtocolCodecFilter(new ProbeCodecFactory(G10Message.class));
> *private static final ExecutorFilter executorFilter = new
> ExecutorFilter(16,32);*
> private static final G10GPBMessageIoFilter gpbMessageFilter = new
> G10GPBMessageIoFilter(G10ParserContextFactory.getG10ParsingAndEncodingInstance());
> static
> {
> initConnectors()
> }
> protected static void initConnectors()
> {
> StateMachine stateMachine =
> StateMachineFactory.getInstance(IoHandlerTransition.class).create(
> G10MinaClient.CONNECTED, new G10MinaClient(processor));
> IoHandler ioHandler = new
> StateMachineProxyBuilder().setStateContextLookup(
> new IoSessionStateContextLookup(new StateContextFactory() {
> @Override
> public StateContext create() {
> final G10StateContext stateContext = new
> G10StateContext();
> stateContext.setStartedTime(new Date());
> return stateContext;
> }
> })).create(IoHandler.class, stateMachine);
>
>
> //Global connector across the system, i.e multiple threads uses the same
> connector.
> NioSocketConnector connector = new NioSocketConnector();
> connector.getFilterChain().addLast("LoggingFilter",
> G10CaptureService.loggingFilter);
> connector.getFilterChain().addLast("codecFilter",
> G10CaptureService.probeCodecFilter);
> connector.getFilterChain().addLast("executorFilter",
> G10CaptureService.executorFilter);
> connector.getFilterChain().addLast("gpbMessageFilter",
> G10CaptureService.gpbMessageFilter);
> connector.getFilterChain().addLast("keepAliveFilter",
> G10CaptureService.keepAliveFilter);
> connector.setHandler(ioHandler);
> }
>
> public void StartRecordCapture()
> {
> connectionLock.lock();
> try
> {
> ConnectFuture primaryConnectFuture = connector.connect(primaryAddress,
> initializer);
> //hungs forever if the no. of threads are more than 30
> primaryConnectFuture.awaitUninterruptibly();//no time out specified
> if (!primaryConnectFuture.isConnected())
> {
>
> if (handleIOException(searchExpression, captureHandler)) {
> return;
> }
> LOG.info("{} Apache mina connection setup time out
> happend.",
> handleConnectionFailed(primaryAddress, captureHandler,
> "Primary IP connection timeout");
> return;
> }
> }catch(Exception e)
> {
> }final
[jira] [Commented] (DIRMINA-1173) Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly() for ever
[
https://issues.apache.org/jira/browse/DIRMINA-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720884#comment-17720884
]
KMVS commented on DIRMINA-1173:
---
How the association between NIO Connector and executorFilter ?
*private static final ExecutorFilter executorFilter = new
ExecutorFilter(16,32);*
If I have a static executorFilter then will the worker threads will be shared
across all NIO sockets ?
> Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly()
> for ever
> ---
>
> Key: DIRMINA-1173
> URL: https://issues.apache.org/jira/browse/DIRMINA-1173
> Project: MINA
> Issue Type: Bug
> Components: Core
>Affects Versions: 2.2.1
>Reporter: KMVS
>Priority: Critical
> Attachments: dumpLatest-1.log
>
>
> Hi All,
> I have attached thread dump too for analysis.
> I have migrated from 2.0.21 to 2.023 for solving CVE, i have seen thread
> blocking issue, So used latest mina verstion 2.2.1 but still threads were
> blocked here is the sample code.Thread hungs at {*}awaitUninterruptibly{*}.
> Once this issue comes in sub sequest launches nothing will work all threads
> will be blocked forever,i have to restart the process to make it work. For
> single thread working fine,if i start 50-100 threads this thread blocking
> issue will surface.I am using the same NioSocketConnector across the threads.
> Here is an scenario, I have one gate way IP which i connect initially ,this
> gate way will return list of ips.
> now i will close the previous IOsession and will create a nio connection with
> the first ip in the list.if it did n't work then again i will try to
> connect to next ip etc..
> All these are in the critical section. i.e i will acquire a lock and then
> after successful connection i will release the lock.
> But problem i have noticed is with the awaitUninterruptibly().
> I have a state machine too. So connection set up is in one thread and
> response processing is in another thread.
>
> *Thread 1:*
> Public class g10CaptureService
> {
>
> private static final ProtocolCodecFilter probeCodecFilter = new
> ProtocolCodecFilter(new ProbeCodecFactory(G10Message.class));
> *private static final ExecutorFilter executorFilter = new
> ExecutorFilter(16,32);*
> private static final G10GPBMessageIoFilter gpbMessageFilter = new
> G10GPBMessageIoFilter(G10ParserContextFactory.getG10ParsingAndEncodingInstance());
> static
> {
> initConnectors()
> }
> protected static void initConnectors()
> {
> StateMachine stateMachine =
> StateMachineFactory.getInstance(IoHandlerTransition.class).create(
> G10MinaClient.CONNECTED, new G10MinaClient(processor));
> IoHandler ioHandler = new
> StateMachineProxyBuilder().setStateContextLookup(
> new IoSessionStateContextLookup(new StateContextFactory() {
> @Override
> public StateContext create() {
> final G10StateContext stateContext = new
> G10StateContext();
> stateContext.setStartedTime(new Date());
> return stateContext;
> }
> })).create(IoHandler.class, stateMachine);
>
>
> //Global connector across the system, i.e multiple threads uses the same
> connector.
> NioSocketConnector connector = new NioSocketConnector();
> connector.getFilterChain().addLast("LoggingFilter",
> G10CaptureService.loggingFilter);
> connector.getFilterChain().addLast("codecFilter",
> G10CaptureService.probeCodecFilter);
> connector.getFilterChain().addLast("executorFilter",
> G10CaptureService.executorFilter);
> connector.getFilterChain().addLast("gpbMessageFilter",
> G10CaptureService.gpbMessageFilter);
> connector.getFilterChain().addLast("keepAliveFilter",
> G10CaptureService.keepAliveFilter);
> connector.setHandler(ioHandler);
> }
>
> public void StartRecordCapture()
> {
> connectionLock.lock();
> try
> {
> ConnectFuture primaryConnectFuture = connector.connect(primaryAddress,
> initializer);
> //hungs forever if the no. of threads are more than 30
> primaryConnectFuture.awaitUninterruptibly();//no time out specified
> if (!primaryConnectFuture.isConnected())
> {
>
> if (handleIOException(searchExpression, captureHandler)) {
> return;
> }
> LOG.info("{} Apache mina connection setup time out
> happend.",
> handleConnectionFailed(primaryAddress, captureHandler,
> "Primary IP connection timeout");
> return;
> }
> }catch(Exception e)
> {
> }finally
> {
> conne
[jira] [Commented] (DIRMINA-1173) Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly() for ever
[
https://issues.apache.org/jira/browse/DIRMINA-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720882#comment-17720882
]
KMVS commented on DIRMINA-1173:
---
How to scale Apache mina ?
my execution filter has min 16 threads and max 32 threads, these threads are
not sufficient to
go parallelly on 135 sniffers for polling the data.
*Here is the execution filter looks like:*
*private static final ExecutorFilter executorFilter = new
ExecutorFilter(16,32);*
Is it make sense if i increase the pool size to 100 ?
where all i can improve scaling in mina ?
> Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly()
> for ever
> ---
>
> Key: DIRMINA-1173
> URL: https://issues.apache.org/jira/browse/DIRMINA-1173
> Project: MINA
> Issue Type: Bug
> Components: Core
>Affects Versions: 2.2.1
>Reporter: KMVS
>Priority: Critical
> Attachments: dumpLatest-1.log
>
>
> Hi All,
> I have attached thread dump too for analysis.
> I have migrated from 2.0.21 to 2.023 for solving CVE, i have seen thread
> blocking issue, So used latest mina verstion 2.2.1 but still threads were
> blocked here is the sample code.Thread hungs at {*}awaitUninterruptibly{*}.
> Once this issue comes in sub sequest launches nothing will work all threads
> will be blocked forever,i have to restart the process to make it work. For
> single thread working fine,if i start 50-100 threads this thread blocking
> issue will surface.I am using the same NioSocketConnector across the threads.
> Here is an scenario, I have one gate way IP which i connect initially ,this
> gate way will return list of ips.
> now i will close the previous IOsession and will create a nio connection with
> the first ip in the list.if it did n't work then again i will try to
> connect to next ip etc..
> All these are in the critical section. i.e i will acquire a lock and then
> after successful connection i will release the lock.
> But problem i have noticed is with the awaitUninterruptibly().
> I have a state machine too. So connection set up is in one thread and
> response processing is in another thread.
>
> *Thread 1:*
> Public class g10CaptureService
> {
>
> private static final ProtocolCodecFilter probeCodecFilter = new
> ProtocolCodecFilter(new ProbeCodecFactory(G10Message.class));
> *private static final ExecutorFilter executorFilter = new
> ExecutorFilter(16,32);*
> private static final G10GPBMessageIoFilter gpbMessageFilter = new
> G10GPBMessageIoFilter(G10ParserContextFactory.getG10ParsingAndEncodingInstance());
> static
> {
> initConnectors()
> }
> protected static void initConnectors()
> {
> StateMachine stateMachine =
> StateMachineFactory.getInstance(IoHandlerTransition.class).create(
> G10MinaClient.CONNECTED, new G10MinaClient(processor));
> IoHandler ioHandler = new
> StateMachineProxyBuilder().setStateContextLookup(
> new IoSessionStateContextLookup(new StateContextFactory() {
> @Override
> public StateContext create() {
> final G10StateContext stateContext = new
> G10StateContext();
> stateContext.setStartedTime(new Date());
> return stateContext;
> }
> })).create(IoHandler.class, stateMachine);
>
>
> //Global connector across the system, i.e multiple threads uses the same
> connector.
> NioSocketConnector connector = new NioSocketConnector();
> connector.getFilterChain().addLast("LoggingFilter",
> G10CaptureService.loggingFilter);
> connector.getFilterChain().addLast("codecFilter",
> G10CaptureService.probeCodecFilter);
> connector.getFilterChain().addLast("executorFilter",
> G10CaptureService.executorFilter);
> connector.getFilterChain().addLast("gpbMessageFilter",
> G10CaptureService.gpbMessageFilter);
> connector.getFilterChain().addLast("keepAliveFilter",
> G10CaptureService.keepAliveFilter);
> connector.setHandler(ioHandler);
> }
>
> public void StartRecordCapture()
> {
> connectionLock.lock();
> try
> {
> ConnectFuture primaryConnectFuture = connector.connect(primaryAddress,
> initializer);
> //hungs forever if the no. of threads are more than 30
> primaryConnectFuture.awaitUninterruptibly();//no time out specified
> if (!primaryConnectFuture.isConnected())
> {
>
> if (handleIOException(searchExpression, captureHandler)) {
> return;
> }
> LOG.info("{} Apache mina connection setup time out
> happend.",
> handleConnectionFailed(
[jira] [Commented] (DIRMINA-1173) Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly() for ever
[
https://issues.apache.org/jira/browse/DIRMINA-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17716990#comment-17716990
]
KMVS commented on DIRMINA-1173:
---
I have threads min 16 and max 32 in the execution filter.
Surprisingly all are waiting at await0(DefaultIoFuture.java:218).
But with 20.21 it is working fine.
if i give a time out of 40 sec. all are timing out at 40 seconds.
> Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly()
> for ever
> ---
>
> Key: DIRMINA-1173
> URL: https://issues.apache.org/jira/browse/DIRMINA-1173
> Project: MINA
> Issue Type: Bug
> Components: Core
>Affects Versions: 2.2.1
>Reporter: KMVS
>Priority: Critical
> Attachments: dumpLatest-1.log
>
>
> Hi All,
> I have attached thread dump too for analysis.
> I have migrated from 2.0.21 to 2.023 for solving CVE, i have seen thread
> blocking issue, So used latest mina verstion 2.2.1 but still threads were
> blocked here is the sample code.Thread hungs at {*}awaitUninterruptibly{*}.
> Once this issue comes in sub sequest launches nothing will work all threads
> will be blocked forever,i have to restart the process to make it work. For
> single thread working fine,if i start 50-100 threads this thread blocking
> issue will surface.I am using the same NioSocketConnector across the threads.
> Here is an scenario, I have one gate way IP which i connect initially ,this
> gate way will return list of ips.
> now i will close the previous IOsession and will create a nio connection with
> the first ip in the list.if it did n't work then again i will try to
> connect to next ip etc..
> All these are in the critical section. i.e i will acquire a lock and then
> after successful connection i will release the lock.
> But problem i have noticed is with the awaitUninterruptibly().
> I have a state machine too. So connection set up is in one thread and
> response processing is in another thread.
>
> *Thread 1:*
> Public class g10CaptureService
> {
>
> private static final ProtocolCodecFilter probeCodecFilter = new
> ProtocolCodecFilter(new ProbeCodecFactory(G10Message.class));
> *private static final ExecutorFilter executorFilter = new
> ExecutorFilter(16,32);*
> private static final G10GPBMessageIoFilter gpbMessageFilter = new
> G10GPBMessageIoFilter(G10ParserContextFactory.getG10ParsingAndEncodingInstance());
> static
> {
> initConnectors()
> }
> protected static void initConnectors()
> {
> StateMachine stateMachine =
> StateMachineFactory.getInstance(IoHandlerTransition.class).create(
> G10MinaClient.CONNECTED, new G10MinaClient(processor));
> IoHandler ioHandler = new
> StateMachineProxyBuilder().setStateContextLookup(
> new IoSessionStateContextLookup(new StateContextFactory() {
> @Override
> public StateContext create() {
> final G10StateContext stateContext = new
> G10StateContext();
> stateContext.setStartedTime(new Date());
> return stateContext;
> }
> })).create(IoHandler.class, stateMachine);
>
>
> //Global connector across the system, i.e multiple threads uses the same
> connector.
> NioSocketConnector connector = new NioSocketConnector();
> connector.getFilterChain().addLast("LoggingFilter",
> G10CaptureService.loggingFilter);
> connector.getFilterChain().addLast("codecFilter",
> G10CaptureService.probeCodecFilter);
> connector.getFilterChain().addLast("executorFilter",
> G10CaptureService.executorFilter);
> connector.getFilterChain().addLast("gpbMessageFilter",
> G10CaptureService.gpbMessageFilter);
> connector.getFilterChain().addLast("keepAliveFilter",
> G10CaptureService.keepAliveFilter);
> connector.setHandler(ioHandler);
> }
>
> public void StartRecordCapture()
> {
> connectionLock.lock();
> try
> {
> ConnectFuture primaryConnectFuture = connector.connect(primaryAddress,
> initializer);
> //hungs forever if the no. of threads are more than 30
> primaryConnectFuture.awaitUninterruptibly();//no time out specified
> if (!primaryConnectFuture.isConnected())
> {
>
> if (handleIOException(searchExpression, captureHandler)) {
> return;
> }
> LOG.info("{} Apache mina connection setup time out
> happend.",
> handleConnectionFailed(primaryAddress, captureHandler,
> "Primary IP connection timeout");
> return;
> }
> }catch(Exception e)
> {
> }finally
> {
> connectionLock.unLock();
> }
> }
[jira] [Commented] (DIRMINA-1173) Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly() for ever
[
https://issues.apache.org/jira/browse/DIRMINA-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17716870#comment-17716870
]
Emmanuel Lécharny commented on DIRMINA-1173:
There are no dead locks, just threads waiting for something. The typical MINA
thread is like this:
{code:java}
pool-116-thread-22priority : 5
thread Id : 0x7ed6e0001800native
Id : 0x127anative Id (decimal) : 4730
state : TIMED_WAITING
stackTrace:
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(java.base@11.0.14.1/Native Method)
- waiting on
at org.apache.mina.core.future.DefaultIoFuture.await0(DefaultIoFuture.java:218)
- waiting to re-lock in wait() <0x7ee3a961f8d0> (a
org.apache.mina.core.polling.AbstractPollingIoConnector$ConnectionRequest)
at
org.apache.mina.core.future.DefaultIoFuture.awaitUninterruptibly(DefaultIoFuture.java:148)
at
org.apache.mina.core.future.DefaultConnectFuture.awaitUninterruptibly(DefaultConnectFuture.java:149)
at
com.netscout.nsaapp.geo.g10Plugin.g10.service.G10CaptureService.startRecordCapture(G10CaptureService.java:622)
at
com.netscout.nsaapp.geo.g10Plugin.geoblade.service.GeoBladeCaptureService.startRecordCapture(GeoBladeCaptureService.java:67)
at
com.netscout.nsaapp.geo.g10Plugin.g10.processor.G10PluginCaptureProcessor.processGatewaySrQueryResponseSuccess(G10PluginCaptureProcessor.java:2156)
at
com.netscout.nsaapp.geo.minaG10Proto.server.G10MinaClient.doHandleGatewaySrQueryResponse(G10MinaClient.java:283)
at
com.netscout.nsaapp.geo.minaG10Proto.server.G10MinaClient.handleGatewaySrQueryResponse(G10MinaClient.java:268)
at jdk.internal.reflect.GeneratedMethodAccessor281.invoke(Unknown Source)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.14.1/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@11.0.14.1/Method.java:566)
at
org.apache.mina.statemachine.transition.MethodTransition.invokeMethod(MethodTransition.java:281)
at
org.apache.mina.statemachine.transition.MethodTransition.doExecute(MethodTransition.java:232)
at
org.apache.mina.statemachine.transition.AbstractTransition.execute(AbstractTransition.java:100)
at org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:183)
at org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:273)
at
org.apache.mina.statemachine.StateMachine.processEvents(StateMachine.java:170)
at org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:158)
- locked <0x7ee3aac530b0> (a
com.netscout.nsaapp.geo.minaG10Proto.server.G10StateContext)
at
org.apache.mina.statemachine.StateMachineProxyBuilder$MethodInvocationHandler.invoke(StateMachineProxyBuilder.java:261)
at com.sun.proxy.$Proxy85.messageReceived(Unknown Source)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:1015)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
at
org.apache.mina.filter.keepalive.KeepAliveFilter.messageReceived(KeepAliveFilter.java:414)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
at
com.netscout.nsaapp.geo.minaG10Proto.server.G10GPBMessageIoFilter.messageReceived(G10GPBMessageIoFilter.java:100)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:106)
at org.apache.mina.core.session.IoEvent.run(IoEvent.java:89)
at
org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTask(OrderedThreadPoolExecutor.java:763)
at
org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTasks(OrderedThreadPoolExecutor.java:755)
at
org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.run(OrderedThreadPoolExecutor.java:695)
at java.lang.Thread.run(java.base@11.0.14.1/Thread.java:829) {code}
Basically, it's waiting for a connection to succeed.
I wonder if you have'nt exhausted the Thread pool max number of thread (which
is 16 by default)
> Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly()
> for ever
> -
[jira] [Commented] (DIRMINA-1173) Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly() for ever
[
https://issues.apache.org/jira/browse/DIRMINA-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17716862#comment-17716862
]
Emmanuel Lécharny commented on DIRMINA-1173:
Pfew... 779 threads. Isn't it a bit too much?
> Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly()
> for ever
> ---
>
> Key: DIRMINA-1173
> URL: https://issues.apache.org/jira/browse/DIRMINA-1173
> Project: MINA
> Issue Type: Bug
> Components: Core
>Affects Versions: 2.2.1
>Reporter: KMVS
>Priority: Critical
> Attachments: dumpLatest-1.log
>
>
> Hi All,
> I have attached thread dump too for analysis.
> I have migrated from 2.0.21 to 2.023 for solving CVE, i have seen thread
> blocking issue, So used latest mina verstion 2.2.1 but still threads were
> blocked here is the sample code.Thread hungs at {*}awaitUninterruptibly{*}.
> Once this issue comes in sub sequest launches nothing will work all threads
> will be blocked forever,i have to restart the process to make it work. For
> single thread working fine,if i start 50-100 threads this thread blocking
> issue will surface.I am using the same NioSocketConnector across the threads.
> Here is an scenario, I have one gate way IP which i connect initially ,this
> gate way will return list of ips.
> now i will close the previous IOsession and will create a nio connection with
> the first ip in the list.if it did n't work then again i will try to
> connect to next ip etc..
> All these are in the critical section. i.e i will acquire a lock and then
> after successful connection i will release the lock.
> But problem i have noticed is with the awaitUninterruptibly().
> I have a state machine too. So connection set up is in one thread and
> response processing is in another thread.
>
> *Thread 1:*
> Public class g10CaptureService
> {
>
> private static final ProtocolCodecFilter probeCodecFilter = new
> ProtocolCodecFilter(new ProbeCodecFactory(G10Message.class));
> *private static final ExecutorFilter executorFilter = new
> ExecutorFilter(16,32);*
> private static final G10GPBMessageIoFilter gpbMessageFilter = new
> G10GPBMessageIoFilter(G10ParserContextFactory.getG10ParsingAndEncodingInstance());
> static
> {
> initConnectors()
> }
> protected static void initConnectors()
> {
> StateMachine stateMachine =
> StateMachineFactory.getInstance(IoHandlerTransition.class).create(
> G10MinaClient.CONNECTED, new G10MinaClient(processor));
> IoHandler ioHandler = new
> StateMachineProxyBuilder().setStateContextLookup(
> new IoSessionStateContextLookup(new StateContextFactory() {
> @Override
> public StateContext create() {
> final G10StateContext stateContext = new
> G10StateContext();
> stateContext.setStartedTime(new Date());
> return stateContext;
> }
> })).create(IoHandler.class, stateMachine);
>
>
> //Global connector across the system, i.e multiple threads uses the same
> connector.
> NioSocketConnector connector = new NioSocketConnector();
> connector.getFilterChain().addLast("LoggingFilter",
> G10CaptureService.loggingFilter);
> connector.getFilterChain().addLast("codecFilter",
> G10CaptureService.probeCodecFilter);
> connector.getFilterChain().addLast("executorFilter",
> G10CaptureService.executorFilter);
> connector.getFilterChain().addLast("gpbMessageFilter",
> G10CaptureService.gpbMessageFilter);
> connector.getFilterChain().addLast("keepAliveFilter",
> G10CaptureService.keepAliveFilter);
> connector.setHandler(ioHandler);
> }
>
> public void StartRecordCapture()
> {
> connectionLock.lock();
> try
> {
> ConnectFuture primaryConnectFuture = connector.connect(primaryAddress,
> initializer);
> //hungs forever if the no. of threads are more than 30
> primaryConnectFuture.awaitUninterruptibly();//no time out specified
> if (!primaryConnectFuture.isConnected())
> {
>
> if (handleIOException(searchExpression, captureHandler)) {
> return;
> }
> LOG.info("{} Apache mina connection setup time out
> happend.",
> handleConnectionFailed(primaryAddress, captureHandler,
> "Primary IP connection timeout");
> return;
> }
> }catch(Exception e)
> {
> }finally
> {
> connectionLock.unLock();
> }
> }
> *Thread 2:*
> public void processGatewaySrQueryResponseSuccess(G10StateContext context,
> IoSession session, GatewaySrQueryResponse response) {
>
