Generally, you'll see a message when you try to log in indicating the no more descriptors condition. I would think something gets written to /var/log/messages.
I would recommend starting at the system level and determine the PIDs associated with the heaviest FDs (or the highest rate of growth), and as Bill indicated, the nature of the FD. lsof is fairly useful, as it indicates the type of FD (IPv4=socket, REG=file, etc). If the heavy hitters are IPv4, you'll probably notice one port with the majority of the sockets - so you can troubleshoot from there.
Mike
-----Original Message-----
From: Peter Luttrell [mailto:[EMAIL PROTECTED]]
Sent: Friday, December 12, 2003 3:50 PM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: Re: [JBoss-user] JBoss On Linux
Is there a way to tell if i've run out of file descriptors?
thanks.
.peter
David Ward wrote:
> Peter,
>
> I've read others' responses to your problems, and they all seem to be
> addressing the wrong thing: performance. What you describe does not
> sound like a performance problem (neither of speed nor memory
> management). You never said the app was slow, so why do you care
> about "SPECjAppServer2002 benchmarks" and the like?
>
> It sounds more like a resource leak to me, my first guess being of
> file descriptors. On Linux/UNIX based systems, socket connections
> (like to your mailserver and database) - not just files - use file
> descriptors. If that's the case, something - or a growing number of
> "things" - are holding onto sockets or files (same thing to UNIX) and
> not letting go.
>
> If I were you I would do 2 things:
>
> 1) In the short term, increase your system setting for max # of file
> descriptors (in my experience, OS defaults are stunningly low for
> production systems). This will require a system reboot to take effect.
>
> 2) Find out where your resource leak is. It may be there's one in
> your application (that runs in jboss), and then the cron runs that
> might require a lot of descriptors too. Together, they could use up
> to your limit. To diagnose this you might have to employ more than
> just one tool. netstat (comes with Linux/UNIX), filemon
> (http://www.sysinternals.com/linux/utilities/filemon.shtml) ,
> Optimizeit (http://www.borland.com/optimizeit/optimizeit_profiler/), etc.
>
> Last, maybe there really isn't a leak, but the nature of your
> application, plus the crons that run at specific times, just simply
> require more file descriptors than what your system has configured.
> Upping that value might prove to be all you need to do. If it never
> grows past that max, you're set - no leak. Unfortunately, without
> doing some surfing, I can't tell you where to set it for your system,
> or what the best value should be. I'll leave that as an exercise for
> you. ;)
>
> Anyway, hope this helps. And sorry in advance if it ends up leading
> you down the wrong path. I just wanted to share what your problem
> "smelled like" to me.
>
> David
>
>
> Peter Luttrell escribió:
>
>> We're using JBoss3.2.1 with Jetty on RedHat 9 with Suns 1.4.2_01 vm.
>> We have a pretty heavy load.
>>
>> After roughly a week many of the boxes start to experience weird
>> problems where JBoss is unable to get what looks to be socket
>> connections. In some cases, we cannot contact our mailserver, in
>> other cases we cannot contact our database; in the latest case we're
>> unable to get a connection to the local jndi server (localhost:1099).
>> Sometimes a simple restart of jboss will sometimes solves the
>> problems, othertimes we have to restart linux. The times of the
>> crashes are roughly 4am and sometimes 6am, so it's likely caused by a
>> cron job running at those times, which we're currently looking into.
>> Has anyone experienced similar problems?
>> .peter
>>
>>
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: IBM Linux Tutorials.
> Become an expert in LINUX or just sharpen your skills. Sign up for IBM's
> Free Linux Tutorials. Learn everything from the bash shell to sys admin.
> Click now! http://ads.osdn.com/?ad_id78&alloc_id371&opÌk
> _______________________________________________
> JBoss-user mailing list
> [EMAIL PROTECTED]
> https://lists.sourceforge.net/lists/listinfo/jboss-user
----------
CONFIDENTIALITY NOTICE: This e-mail message, including any attachments,is for the sole use of the intended recipient(s), even if addressed incorrectly, and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy or delete all copies of the original message and all attachments, including deletion from the trash or equivalent folder. Thank you.
