You are right!
The code does implement a mutex() part. But the problem lasted for 7 days
untill the server was restarted manually.
There were about 15 threads which were found to be in locked state.And for
all these days tomcat was not accepting any requests.
Do you think Dev that this could have happened under the conditions that you
have mentioned .
And the following message was found in the catalina.out log file:
Dec 7, 2004 3:44:37 PM
org.apache.tomcat.util.threads.ThreadPool logFull
SEVERE: All threads are busy, waiting. Please increase
maxThreads or check the servlet status75 75
-----Original Message-----
From: David Delbecq [mailto:[EMAIL PROTECTED]
Sent: Thursday, May 18, 2006 2:35 PM
To: Tomcat Users List
Subject: Re: FW: tomcat hangs with error msg-"All Threads are Busy....."
ok, i bet then you problem is in this :D:
au.com.AdmTech.incontrol.secureproxy.SecureProxy.doGet(SecureProxy.java:298)
- locked <0xf227ea88> (a
au.com.alcatel.incontrol.secureproxy.SecureProxy)
at
au.com.AdMtech.incontrol.secureproxy.SecureProxy.doPost(SecureProxy.java:435
)
it uses a lock to a ressource or there is a synchronized method in the way.
As such, until one process has finished, others can't start working.
This is bad :)
As a consequence, if one of clients is not responding but not sending data
to the wire, it locks everything.
In normal course of operation, you end up with a connection closed exception
in socketinputstream.read() but it takes time to get them. and the delay is
platform dependant. if client abruptly close, TCP/IP protocol has no
information about this except for a very long delay.
Maybe some protocol level tunings like keep alives might help :/
Anyway, this has nothing to do with GC :)
Ajit Narayanan wrote:
Thanx for ur interest Dav.I am fairly new to java and tomcat
programming so I am bit low on debugging these.
Yes,I do have some of the data from the dumps, I don't know whether
these will be of any help or not but this what I have.This happened on
live about
7 months ago and it has been reopened on cux demand ,a workaround has
been patched where in the socket gets realesed after some 90- secs.The
following dump has been taken before this workaround:
"Thread-540" daemon prio=5 tid=0x0048b9e0 nid=0x33a runnable
[ee47f000..ee4819a8]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(Unknown Source)
at
org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.
java:7
77)
at
org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.do
Read(I
nternalInputBuffer.java:807)
at
org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityIn
putFil
ter.java:158)
at
org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffe
r.java
:742)
at org.apache.coyote.Request.doRead(Request.java:431)
at
org.apache.coyote.tomcat4.CoyoteInputStream.readBytes(CoyoteInputStrea
m.java
:199)
at
org.apache.coyote.tomcat4.CoyoteInputStream.read(CoyoteInputStream.java:156)
at
org.apache.coyote.tomcat4.CoyoteRequest.readPostBody(CoyoteRequest.jav
a:1983
)
at
org.apache.coyote.tomcat4.CoyoteRequest.parseRequestParameters(CoyoteR
equest
.java:1965)
at
org.apache.coyote.tomcat4.CoyoteRequest.getParameter(CoyoteRequest.java:911)
at
org.apache.coyote.tomcat4.CoyoteRequestFacade.getParameter(CoyoteReque
stFaca
de.java:178)
at
au.com.AdmTech.incontrol.secureproxy.SecureProxy.doGet(SecureProxy.java:298)
- locked <0xf227ea88> (a
au.com.alcatel.incontrol.secureproxy.SecureProxy)
at
au.com.AdMtech.incontrol.secureproxy.SecureProxy.doPost(SecureProxy.ja
va:435
)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
And there are 71 threads waiting to lock <0xf227ea88>:
"- waiting to lock <0xf227ea88>
Apart from what you suggested can this be a problem of compatibility
between tomcat-JDK and the POSIX lib of solaris?
Or can it be that the gc is not able to execute as generally the load
on this proxy is very high leading to such a problem.
Thanks and regards,
Ajit.
-----Original Message-----
From: David Delbecq [mailto:[EMAIL PROTECTED]
Sent: Thursday, May 18, 2006 1:32 PM
To: Tomcat Users List
Subject: Re: FW: tomcat hangs with error msg-"All Threads are Busy....."
This most probably means there is deadlock somewhere in your
webapplication, leading to HTTP threads not being released because
they are stuck waiting for something before sending response to a user
that dropped hours agos.
To get clues on where your problem is when this happens, send signal
SIGQUIT to tomcat process, it will force the sun JVM to dump all threads
stacks.
Regards,
David Delbecq
Ajit Narayanan wrote:
hi,
My tomcat 4.1.27 is running on sun Solaris 8 box but ocassionally it
hangs out giving the error " All threads are busy, waiting. Please
increase maxThreads or check the servlet status75". I have tried
increasing the no of threads but its not working I guess. The
application code looks clean but can there be any compatibility
issues (of tomcat and JDK) with the solaris POSIX thread libraries??
If there is any problem then what can be done to resolve this
problem. One more thing restarting the tomcat manually resolves the
problem.
regards,
Ajit.
=====================================================================
= ======================================================
Tech Mahindra, formerly Mahindra-British Telecom.
Disclaimer:
The contents of this E-mail (including the contents of the
enclosure(s) or
attachment(s) if any) are privileged and confidential material of Tech
Mahindra and should not be disclosed to, used by or copied in any
manner by anyone other than the intended addressee(s). In case you are
not the desired addressee, you should delete this message and/or re-direct
it to the sender.
The views expressed in this E-mail message (including the enclosure(s)
or
attachment(s) if any) are those of the individual sender, except where
the sender expressly, and with authority, states them to be the views
of Tech Mahindra.
This e-mail message including attachment/(s), if any, is believed to
be
free of any virus. However, it is the responsibility of the recipient
to ensure that it is virus free and Tech Mahindra is not responsible
for any loss or damage arising in any way from its use.
=====================================================================
= ======================================================
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
======================================================================
======================================================
Tech Mahindra, formerly Mahindra-British Telecom.
Disclaimer:
The contents of this E-mail (including the contents of the enclosure(s) or
attachment(s) if any) are privileged and confidential material of Tech
Mahindra and should not be disclosed to, used by or copied in any manner by
anyone other than the intended addressee(s). In case you are not the desired
addressee, you should delete this message and/or re-direct it to the sender.
The views expressed in this E-mail message (including the enclosure(s) or
attachment(s) if any) are those of the individual sender, except where the
sender expressly, and with authority, states them to be the views of Tech
Mahindra.
This e-mail message including attachment/(s), if any, is believed to be
free of any virus. However, it is the responsibility of the recipient to
ensure that it is virus free and Tech Mahindra is not responsible for any
loss or damage arising in any way from its use.
======================================================================
======================================================
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
============================================================================================================================
Tech Mahindra, formerly Mahindra-British Telecom.
Disclaimer:
The contents of this E-mail (including the contents of the enclosure(s) or
attachment(s) if any) are privileged and confidential material of Tech Mahindra
and should not be disclosed to, used by or copied in any manner by anyone other
than the intended addressee(s). In case you are not the desired addressee, you
should delete this message and/or re-direct it to the sender. The views
expressed in this E-mail message (including the enclosure(s) or attachment(s)
if any) are those of the individual sender, except where the sender expressly,
and with authority, states them to be the views of Tech Mahindra.
This e-mail message including attachment/(s), if any, is believed to be free of
any virus. However, it is the responsibility of the recipient to ensure that it
is virus free and Tech Mahindra is not responsible for any loss or damage
arising in any way from its use.
============================================================================================================================
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
