Thanks Jonathan,
Ok, but I did not see any message like "acceptor was sleeping".
I noticed that my open files count for my Java application is keep increasing
day by day as shown below. Why is it not decreasing again?
What configuration am I missing? What can be the reason?
Date
Time
Open file count
30-Nov
20:00 PM
2273
30-Nov
22:00 PM
2544
1-Dec
12:00 AM
2790
1-Dec
2:00 AM
2987
1-Dec
4:00 AM
3283
1-Dec
12:00 PM
4828
1-Dec
14:00 PM
5200
1-Dec
16:00 PM
5635
2-Dec
10:17 AM
6992
2-Dec
12:00 PM
7008
2-Dec
14:02 PM
7024
2-Dec
16:08 PM
7047
In my RequestHandler, I have following settings, means setting session timeout
to 1800 seconds (30 Mins)
@Override
public void messageReceived(IoSession session, Object message) throws
Exception {
if (session != null) {
this.session = session;
this.session.getConfig().setBothIdleTime(1800);
LOGGER.info("Session Idle time in sec: " +
this.session.getConfig().getBothIdleTime());
}
this.message = message;
}
Yes, I can simulate multiple devices with lowering nlimit and test. What would
be the next?
Thanks & Regards,
Krishan Babbar
============================================================================================================================
Disclaimer: This message and the information contained herein is proprietary
and confidential and subject to the Tech Mahindra policy statement, you may
review the policy at http://www.techmahindra.com/Disclaimer.html externally
http://tim.techmahindra.com/tim/disclaimer.html internally within TechMahindra.
===========================================================================================================================
-----Original Message-----
From: Jonathan Valliere <[email protected]>
Sent: Friday, November 30, 2018 7:20 PM
To: [email protected]
Cc: Atul Kandhari <[email protected]>
Subject: Re: Getting "Too many open files" warnings11
Mina is patched to prevent that exception from occurring in a cycle. That
was a long time ago, your version is patched. You probably noticed a
message saying the acceptor was sleeping.
Do you have any kind of testing utility to create fake users for your
application? I would recommend lowering the nlimit and trying to reproduce
entirely artificially. There shouldn’t be any correlation to
On Fri, Nov 30, 2018 at 8:39 AM Krishan Babbar
<[email protected]<mailto:[email protected]>>
wrote:
> Hi Jonathan,
>
> Version - 2.0.16
> Java Application process was using around 1% of CPU and 3.4% of Memory.
>
> After mitigate, were you able to provide the solution?
> I mean what would be the solution if you know?
>
> Thanks & Regards,
> Krishan Babbar
>
> -----Original Message-----
> From: Jonathan Valliere
> <[email protected]<mailto:[email protected]>>
> Sent: Friday, November 30, 2018 6:56 PM
> To: [email protected]<mailto:[email protected]>
> Cc: Atul Kandhari
> <[email protected]<mailto:[email protected]>>
> Subject: Re: Getting "Too many open files" warnings11
>
> What version of Mina are you using? We added code to mitigate this
> specific exemption a while back.
>
> Do you know how much cpu Java was consuming at that point in time?
>
> On Fri, Nov 30, 2018 at 8:24 AM Krishan Babbar <
> [email protected]<mailto:[email protected]>>
> wrote:
>
> > Thanks Stefan,
> >
> > It would be great if you could answers other questions too.
> > -What do you think, why application stops receiving packets from devices?
> > Is it only due to exception "Too many open files" or it can be due
> > to any other reasons as well? Please guide.
> > -Is my MINA configuration fine shown in below Java code? Please
> > suggest if I need to fine tune it more.
> >
> > Thanks & Regards,
> > Krishan Babbar
> >
> >
> >
> > ====================================================================
> > == ======================================================
> > Disclaimer: This message and the information contained herein is
> > proprietary and confidential and subject to the Tech Mahindra policy
> > statement, you may review the policy at
> > http://www.techmahindra.com/Disclaimer.html externally
> > http://tim.techmahindra.com/tim/disclaimer.html internally within
> > TechMahindra.
> >
> > ====================================================================
> > == =====================================================
> >
> > -----Original Message-----
> > From: Stefan Magnus Landrø
> > <[email protected]<mailto:[email protected]>>
> > Sent: Friday, November 30, 2018 5:59 PM
> > To: [email protected]<mailto:[email protected]>
> > Cc: Atul Kandhari
> > <[email protected]<mailto:[email protected]>>
> > Subject: Re: Getting "Too many open files" warnings11
> >
> >
> > https://unix.stackexchange.com/questions/108603/do-changes-in-etc-se
> > cu
> > rity-limits-conf-require-a-reboot
> >
> >
> > Sendt fra min iPhone
> >
> > > 30. nov. 2018 kl. 11:17 skrev Krishan Babbar
> > ><[email protected]
> > >:
> > >
> > > Dear All,
> > >
> > > I am working on an IoT project. I have created a Java application
> > > using
> > MINA library. All the devices are connecting to this Java
> > application using TCP protocol. I am getting around 3000 packets per
> > hour from about 50 devices as of now and it will increase soon.
> > >
> > > I am facing one issue on production environment. After running say
> > > 1-2
> > days, my application stops receiving packets from all the devices
> > but application process keep running. To make it working again, I
> > need to restart my Java application. I noticed following warnings in logs.
> > > Warnings
> > > [NioSocketAcceptor-3] WARN
> > > org.apache.mina.util.DefaultExceptionMonitor
> > 30/11/2018 05:25:40 - Unexpected exception.
> > > java.io.IOException: Too many open files
> > > at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> > > at sun.nio.ch
> > .ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
> > > at sun.nio.ch
> > .ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
> > > at
> > org.apache.mina.transport.socket.nio.NioSocketAcceptor.accept(NioSoc
> > ke
> > tAcceptor.java:194)
> > > at
> > org.apache.mina.transport.socket.nio.NioSocketAcceptor.accept(NioSoc
> > ke
> > tAcceptor.java:51)
> > > at
> > org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.proc
> > es
> > sHandles(AbstractPollingIoAcceptor.java:544)
> > > at
> > org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(
> > Ab
> > stractPollingIoAcceptor.java:484)
> > > at
> > org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnab
> > le
> > .java:64)
> > > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor
> > .j
> > ava:1142)
> > > at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
> > java:617)
> > > at java.lang.Thread.run(Thread.java:745)
> > >
> > > Due to above warnings in logs, yesterday, after doing some Google,
> > > I
> > changed file limits in "/etc/security/limits.conf" file to 100000 as
> > shown below, but I did not restart my Java application.
> > > * hard nofile 100000
> > > * soft nofile 100000
> > >
> > > Today. I got the same warnings again and I was not getting any
> > > packet
> > from any of the devices. So I restarted my Java application and
> > devices started sending packets again.
> > > Now, my questions are
> > >
> > > - After application stopped receiving packets, I restarted my
> > Java application today. Will it consider new file limits i.e. 100000
> > and not give Exceptions again? Or do I need to reboot machine as well?
> > >
> > > - What do you think, why application stops receiving packets
> > from devices? Is it only due to exception "Too many open files" or
> > it can be due to any other reasons as well? Please guide.
> > >
> > > - Is my MINA configuration fine shown in below Java code?
> > Please suggest if I need to fine tune it more.
> > >
> > >
> > > Below is given our code for MINA configuration.
> > > import java.io.IOException;
> > > import java.net.InetSocketAddress; import org.apache.log4j.Logger;
> > > import org.apache.mina.core.service.AbstractIoAcceptor;
> > > import org.apache.mina.core.session.IdleStatus;
> > > import org.apache.mina.filter.logging.LoggingFilter;
> > > import org.apache.mina.transport.socket.nio.NioDatagramAcceptor;
> > >
> > > import com.techm.tnt.protocol.gateway.util.ServerConfigLoader;
> > >
> > > /**
> > > * The abstract class for tcp/ip and udp listeners.
> > > */
> > > public abstract class AbstractListener {
> > >
> > > /** The logger instance.*/
> > > private static Logger LOGGER =
> > > Logger.getLogger(AbstractListener.class);
> > >
> > > /** The buffer size constant. */
> > > protected static final int BUFFER_SIZE = 2048;
> > >
> > > /**
> > > * The method to initialize udp and tcp/ip listeners.
> > > *
> > > * @param portKey
> > > * the port of the udp or tcp/ip listener.
> > > * @throws IOException
> > > * the io exception to be thrown in case of
> > error.
> > > */
> > > public void init(String portKey) throws IOException {
> > > LOGGER.info("Entering init()");
> > > AbstractIoAcceptor acceptor =
> > initInternal();
> > > acceptor.setHandler(new
> > > RequestHandler());
> > >
> > acceptor.getFilterChain().addLast("logger", new LoggingFilter());
> > >
> > acceptor.getSessionConfig().setReadBufferSize(BUFFER_SIZE);
> > >
> > acceptor.getSessionConfig().setIdleTime(IdleStatus.BOTH_IDLE, 10);
> > > if (acceptor instanceof
> > NioDatagramAcceptor) {
> > >
> > > ((NioDatagramAcceptor)
> > acceptor).getSessionConfig().setReuseAddress(true);
> > > }
> > > acceptor.bind(new
> > InetSocketAddress(Integer.parseInt(ServerConfigLoader.getInstance().
> > ge
> > tProperty(portKey))));
> > > LOGGER.info("Exiting init()");
> > > }
> > >
> > > /**
> > > * The abstract method to be over ridden by
> > > appropriate
> > listener (udp /
> > > * tcp/ip) for invoking appropriate listener.
> > > *
> > > * @return the appropriate (udp/ tcp/ip) io acceptor.
> > > */
> > > public abstract AbstractIoAcceptor initInternal();
> > > }
> > >
> > > Appreciate all tour help.
> > >
> > > Thanks & Regards,
> > > Krishan Babbar
> > >
> > >
> > >
> > > ==================================================================
> > > == == ======================================================
> > > Disclaimer: This message and the information contained herein is
> > proprietary and confidential and subject to the Tech Mahindra policy
> > statement, you may review the policy at
> > http://www.techmahindra.com/Disclaimer.html externally
> > http://tim.techmahindra.com/tim/disclaimer.html internally within
> > TechMahindra.
> > > ==================================================================
> > > == == =====================================================
> > >
> > > ==================================================================
> > > == == ======================================================
> > >
> > > Disclaimer: This message and the information contained herein is
> > proprietary and confidential and subject to the Tech Mahindra policy
> > statement, you may review the policy at
> > http://www.techmahindra.com/Disclaimer.html <
> > http://www.techmahindra.com/Disclaimer.html> externally
> > http://tim.techmahindra.com/tim/disclaimer.html <
> > http://tim.techmahindra.com/tim/disclaimer.html> internally within
> > TechMahindra.
> > >
> > > ==================================================================
> > > == == ======================================================
> >
> --
>
> CONFIDENTIALITY NOTICE: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally
> protected from disclosure.
>
> ======================================================================
> ======================================================
>
> Disclaimer: This message and the information contained herein is
> proprietary and confidential and subject to the Tech Mahindra policy
> statement, you may review the policy at
> http://www.techmahindra.com/Disclaimer.html <
> http://www.techmahindra.com/Disclaimer.html> externally
> http://tim.techmahindra.com/tim/disclaimer.html <
> http://tim.techmahindra.com/tim/disclaimer.html> internally within
> TechMahindra.
>
>
> ======================================================================
> ======================================================
>
--
CONFIDENTIALITY NOTICE: The contents of this email message and any attachments
are intended solely for the addressee(s) and may contain confidential and/or
privileged information and may be legally protected from disclosure.
============================================================================================================================
Disclaimer: This message and the information contained herein is proprietary
and confidential and subject to the Tech Mahindra policy statement, you may
review the policy at http://www.techmahindra.com/Disclaimer.html
<http://www.techmahindra.com/Disclaimer.html> externally
http://tim.techmahindra.com/tim/disclaimer.html
<http://tim.techmahindra.com/tim/disclaimer.html> internally within
TechMahindra.
============================================================================================================================
