[
https://issues.apache.org/jira/browse/DIRMINA-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17716870#comment-17716870
]
Emmanuel Lécharny commented on DIRMINA-1173:
--------------------------------------------
There are no dead locks, just threads waiting for something. The typical MINA
thread is like this:
{code:java}
pool-116-thread-22priority : 5
thread Id : 0x00007ed6e0001800native
Id : 0x127anative Id (decimal) : 4730
state : TIMED_WAITING
stackTrace:
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait([email protected]/Native Method)
- waiting on <no object reference available>
at org.apache.mina.core.future.DefaultIoFuture.await0(DefaultIoFuture.java:218)
- waiting to re-lock in wait() <0x00007ee3a961f8d0> (a
org.apache.mina.core.polling.AbstractPollingIoConnector$ConnectionRequest)
at
org.apache.mina.core.future.DefaultIoFuture.awaitUninterruptibly(DefaultIoFuture.java:148)
at
org.apache.mina.core.future.DefaultConnectFuture.awaitUninterruptibly(DefaultConnectFuture.java:149)
at
com.netscout.nsaapp.geo.g10Plugin.g10.service.G10CaptureService.startRecordCapture(G10CaptureService.java:622)
at
com.netscout.nsaapp.geo.g10Plugin.geoblade.service.GeoBladeCaptureService.startRecordCapture(GeoBladeCaptureService.java:67)
at
com.netscout.nsaapp.geo.g10Plugin.g10.processor.G10PluginCaptureProcessor.processGatewaySrQueryResponseSuccess(G10PluginCaptureProcessor.java:2156)
at
com.netscout.nsaapp.geo.minaG10Proto.server.G10MinaClient.doHandleGatewaySrQueryResponse(G10MinaClient.java:283)
at
com.netscout.nsaapp.geo.minaG10Proto.server.G10MinaClient.handleGatewaySrQueryResponse(G10MinaClient.java:268)
at jdk.internal.reflect.GeneratedMethodAccessor281.invoke(Unknown Source)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke([email protected]/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke([email protected]/Method.java:566)
at
org.apache.mina.statemachine.transition.MethodTransition.invokeMethod(MethodTransition.java:281)
at
org.apache.mina.statemachine.transition.MethodTransition.doExecute(MethodTransition.java:232)
at
org.apache.mina.statemachine.transition.AbstractTransition.execute(AbstractTransition.java:100)
at org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:183)
at org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:273)
at
org.apache.mina.statemachine.StateMachine.processEvents(StateMachine.java:170)
at org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:158)
- locked <0x00007ee3aac530b0> (a
com.netscout.nsaapp.geo.minaG10Proto.server.G10StateContext)
at
org.apache.mina.statemachine.StateMachineProxyBuilder$MethodInvocationHandler.invoke(StateMachineProxyBuilder.java:261)
at com.sun.proxy.$Proxy85.messageReceived(Unknown Source)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:1015)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
at
org.apache.mina.filter.keepalive.KeepAliveFilter.messageReceived(KeepAliveFilter.java:414)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
at
com.netscout.nsaapp.geo.minaG10Proto.server.G10GPBMessageIoFilter.messageReceived(G10GPBMessageIoFilter.java:100)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:106)
at org.apache.mina.core.session.IoEvent.run(IoEvent.java:89)
at
org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTask(OrderedThreadPoolExecutor.java:763)
at
org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTasks(OrderedThreadPoolExecutor.java:755)
at
org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.run(OrderedThreadPoolExecutor.java:695)
at java.lang.Thread.run([email protected]/Thread.java:829) {code}
Basically, it's waiting for a connection to succeed.
I wonder if you have'nt exhausted the Thread pool max number of thread (which
is 16 by default)
> Apache mina 2.2.1 threads blocking on ConnectFuture.awaitUninterruptibly()
> for ever
> -----------------------------------------------------------------------------------
>
> Key: DIRMINA-1173
> URL: https://issues.apache.org/jira/browse/DIRMINA-1173
> Project: MINA
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.2.1
> Reporter: KMVS
> Priority: Critical
> Attachments: dumpLatest-1.log
>
>
> Hi All,
> I have attached thread dump too for analysis.
> I have migrated from 2.0.21 to 2.023 for solving CVE, i have seen thread
> blocking issue, So used latest mina verstion 2.2.1 but still threads were
> blocked here is the sample code.Thread hungs at {*}awaitUninterruptibly{*}.
> Once this issue comes in sub sequest launches nothing will work all threads
> will be blocked forever,i have to restart the process to make it work. For
> single thread working fine,if i start 50-100 threads this thread blocking
> issue will surface.I am using the same NioSocketConnector across the threads.
> Here is an scenario, I have one gate way IP which i connect initially ,this
> gate way will return list of ips.
> now i will close the previous IOsession and will create a nio connection with
> the first ip in the list.if it did n't work then again i will try to
> connect to next ip etc..
> All these are in the critical section. i.e i will acquire a lock and then
> after successful connection i will release the lock.
> But problem i have noticed is with the awaitUninterruptibly().
> I have a state machine too. So connection set up is in one thread and
> response processing is in another thread.
>
> *Thread 1:*
> Public class g10CaptureService
> {
>
> private static final ProtocolCodecFilter probeCodecFilter = new
> ProtocolCodecFilter(new ProbeCodecFactory(G10Message.class));
> *private static final ExecutorFilter executorFilter = new
> ExecutorFilter(16,32);*
> private static final G10GPBMessageIoFilter gpbMessageFilter = new
> G10GPBMessageIoFilter(G10ParserContextFactory.getG10ParsingAndEncodingInstance());
> static
> {
> initConnectors()
> }
> protected static void initConnectors()
> {
> StateMachine stateMachine =
> StateMachineFactory.getInstance(IoHandlerTransition.class).create(
> G10MinaClient.CONNECTED, new G10MinaClient(processor));
> IoHandler ioHandler = new
> StateMachineProxyBuilder().setStateContextLookup(
> new IoSessionStateContextLookup(new StateContextFactory() {
> @Override
> public StateContext create() {
> final G10StateContext stateContext = new
> G10StateContext();
> stateContext.setStartedTime(new Date());
> return stateContext;
> }
> })).create(IoHandler.class, stateMachine);
>
>
> //Global connector across the system, i.e multiple threads uses the same
> connector.
> NioSocketConnector connector = new NioSocketConnector();
> connector.getFilterChain().addLast("LoggingFilter",
> G10CaptureService.loggingFilter);
> connector.getFilterChain().addLast("codecFilter",
> G10CaptureService.probeCodecFilter);
> connector.getFilterChain().addLast("executorFilter",
> G10CaptureService.executorFilter);
> connector.getFilterChain().addLast("gpbMessageFilter",
> G10CaptureService.gpbMessageFilter);
> connector.getFilterChain().addLast("keepAliveFilter",
> G10CaptureService.keepAliveFilter);
> connector.setHandler(ioHandler);
> }
>
> public void StartRecordCapture()
> {
> connectionLock.lock();
> try
> {
> ConnectFuture primaryConnectFuture = connector.connect(primaryAddress,
> initializer);
> //hungs forever if the no. of threads are more than 30
> primaryConnectFuture.awaitUninterruptibly();//no time out specified
> if (!primaryConnectFuture.isConnected())
> {
>
> if (handleIOException(searchExpression, captureHandler)) {
> return;
> }
> LOG.info("{} Apache mina connection setup time out
> happend.",
> handleConnectionFailed(primaryAddress, captureHandler,
> "Primary IP connection timeout");
> return;
> }
> }catch(Exception e)
> {
> }finally
> {
> connectionLock.unLock();
> }
> }
> *Thread 2:*
> public void processGatewaySrQueryResponseSuccess(G10StateContext context,
> IoSession session, GatewaySrQueryResponse response) {
> //get the g10CaptureService from the Thread1 IOsession
> if(response.getIpPortCount() > 0) {
> try {
> List<IpAddrPort> msgIpPorts = response.getIpPortList();
> List<InetSocketAddress> probeIpPorts = new
> ArrayList<InetSocketAddress>();
> for (IpAddrPort ipp : msgIpPorts) {
> InetSocketAddress inetSockAddr = new
> InetSocketAddress(InetAddress.getByAddress(ipp.getIp().toByteArray()),
> ipp.getPort());
> probeIpPorts.add(inetSockAddr);
> }
> g10CaptureService.setProbeIps(probeIpPorts);
> } catch (Exception ex) {
> LOG.error("{} Exception occured while creating probe
> IP/Port.Exception:{}", getLogStr(session), ex);
> }
> context.addGeoBladeGwSession(session);
>
> g10CaptureService.closeConnectionForSession(session); //cose thread1
> io session
> g10CaptureService.setIsGeoBladeIdlSession(true);
> g10CaptureService.startRecordCapture(searchExpression,
> captureHandler, captureId); //connect to the probe again using
> same thread 1 instance with different ip
> }
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
