Hi Antonio, your suggestion helped me. Thank you. But I expected bug in our code. Instead of it, it is somewhere else - java.net.SocketOutputStream.socketWrite0. And I don't know why.
Some background in the beginning: Our application has one instance "MainServlet". Every request (except of pictures, css, js and so on) goes to its "processRequest" method. This method is synchronized. Hence, only one request can be served in one moment. I know it is strange. But it is. I wrote some code to watch behavior of this method. I log start time, end time, requested url. We wrote script on Linux that tries to connect this site. In case of the problem it makes snapshot of system and downloads server status from manager app, sends kill -QUIT. Then it restarts catalina. So, I have four samples. You can download them at http://info.vsb.cz/test/problem.zip. There are three files for every sample. In catalina_critical.out is result of kill -QUIT, in portal_vsb...txt are starts and ends and url from the "processRequest" method, in the server_status...html is snapshot of server status from manager app. In catalina_critical.out you can find that a few threads are waiting for the monitor of "processRequest" method. One thread is runnable. In all four cases the last method in the stack trace is java.net.SocketOutputStream.socketWrite0(). You can see example below. As the server status say, this thread is executed up 100 seconds. I think, this is the reason of problems. But I don't know why it occurs and how to solve it. -------------------------------------------------------------------- "http8080-Processor45" daemon prio=1 tid=0x62b429b0 nid=0x2a07 runnable [b71fe000..b71ff908] at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92) at java.net.SocketOutputStream.write(SocketOutputStream.java:136) at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(InternalOut putBuffer.java:757) at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:436) at org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer .java:347) at org.apache.coyote.http11.Http11Processor.action(Http11Processor.java:959 ) at org.apache.coyote.Response.action(Response.java:226) at org.apache.coyote.tomcat5.OutputBuffer.doFlush(OutputBuffer.java:369) at org.apache.coyote.tomcat5.OutputBuffer.flush(OutputBuffer.java:340) at org.apache.coyote.tomcat5.CoyoteWriter.flush(CoyoteWriter.java:117) at publisher.web.servlets.MainServlet.processRequest(MainServlet.java:213) - locked <0x471933c8> (a publisher.web.servlets.MainServlet) at publisher.web.servlets.MainServlet.doGet(MainServlet.java:226) at javax.servlet.http.HttpServlet.service(HttpServlet.java:743) at javax.servlet.http.HttpServlet.service(HttpServlet.java:856) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica tionFilterChain.java:284) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilt erChain.java:204) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValv e.java:257) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveCo ntext.java:151) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:5 67) at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardCon textValve.java:245) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValv e.java:199) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveCo ntext.java:151) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(Authenticator Base.java:509) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveCo ntext.java:149) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:5 67) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java :184) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveCo ntext.java:151) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java :164) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveCo ntext.java:149) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:578 ) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveCo ntext.java:149) at org.apache.catalina.authenticator.SingleSignOn.invoke(SingleSignOn.java: 463) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveCo ntext.java:149) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:5 67) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve. java:156) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveCo ntext.java:151) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:5 67) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:972) at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:206) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:83 3) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processC onnection(Http11Protocol.java:732) at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:61 9) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool .java:688) at java.lang.Thread.run(Thread.java:534) ------------------------------------------------------------------------ - Maybe it is important - yet another thread is always runnable: "http8080-Processor44" daemon prio=1 tid=0x62bc9b88 nid=0x2a06 runnable [b73ff000..b73ff908] at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353) - locked <0x467ea0f0> (a java.net.PlainSocketImpl) at java.net.ServerSocket.implAccept(ServerSocket.java:448) at java.net.ServerSocket.accept(ServerSocket.java:419) at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(Defau ltServerSocketFactory.java:107) at org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint. java:410) at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:59 1) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool .java:688) at java.lang.Thread.run(Thread.java:534) ------------------------------------------------------------------------ ---- Thank you for any suggestions, Lipi > -----Original Message----- > From: Antonio Fiol Bonnín [mailto:[EMAIL PROTECTED] > Sent: Thursday, May 06, 2004 8:42 PM > To: Tomcat Users List > Subject: Re: How to limit time for Connector threads? > > Hi Yoav, > > What the original poster meant is not that there is a race condition > in Tomcat, but in his own web application. > > Maybe we should try to point him into the direction of fixing the > problem. > > So there I go: > > - When it "hangs", if on Unix/Linux, kill -QUIT your Tomcat process, > and you will get some stack traces. > - Maybe they will tell you something. If they do, you're lucky. > Otherwise, post them: maybe they will tell us something. > > At least, you will see how many threads there are, and their state > (e.g. waiting for a lock). > > Good luck! > > > Antonio Fiol > > > > Shapira, Yoav wrote: > > >Hi, > >I highly doubt you've stumbled on a race condition in the tomcat > connector implementation. Nonetheless, if you can supply a small WAR > that leads to the problem you've described, we'll be glad to test it, > and if your problem is reproducible rest assured it will be fixed > immediately, as that would be more important than the other stuff > currently open in bugzilla. > > > >Yoav Shapira > >Millennium Research Informatics > > > > > > > > > >>-----Original Message----- > >>From: rlipi [mailto:[EMAIL PROTECTED] > >>Sent: Thursday, May 06, 2004 3:26 AM > >>To: 'Tomcat Users List' > >>Subject: RE: How to limit time for Connector threads? > >> > >>Yes, I did it. > >> > >>But it is not solution. Server doesn't answer slowly or for only > >>some requests. It doesn't answer at all. It means that treads are > >>not terminated and resources are not released. Sometimes, server > >>doesn't answer without "All threads are currently busy" exception. > >>That is why I think that the problem is some death lock. > >> > >>Lipi > >> > >> > >> > >> > >>>-----Original Message----- > >>>From: STOCKHOLM, Raymond [mailto:[EMAIL PROTECTED] > >>>Sent: Thursday, May 06, 2004 8:56 AM > >>>To: Tomcat Users List > >>>Subject: RE: How to limit time for Connector threads? > >>> > >>>Maybe you should increase the number of threads in your connector. > >>> > >>>check attribut maxProcessors in your server.xml > >>>In this example, I set it to 500 instead of 75 (default value) > >>> > >>> <Connector className="org.apache.coyote.tomcat4.CoyoteConnector" > >>> port="80" minProcessors="30" maxProcessors="500" > >>> enableLookups="true" redirectPort="8443" > >>> acceptCount="100" debug="0" connectionTimeout="20000" > >>> useURIValidationHack="false" > >>> > >>> > >>disableUploadTimeout="true" /> > >> > >> > >>>-----Message d'origine----- > >>>De : rlipi [mailto:[EMAIL PROTECTED] > >>>Envoyé : jeudi 6 mai 2004 08:40 > >>>À : 'Tomcat Users List' > >>>Objet : How to limit time for Connector threads? > >>> > >>> > >>>Hallo, > >>>is it possible in any way to limit time for threads that realize > >>>user requests? > >>> > >>>Sometimes (once a day in average), the Tomcat 5.0.19 server doesn't > >>>answers. The reason is in error message "All threads are currently > >>>busy". Probably there is a bug in the web application. Some kind of > >>>synchronization death lock, I think. But localization of this bug > >>>is very hard. So, termination (and > >>> > >>> > >>release > >> > >> > >>>of all resources) of long time threads can be a work around. > >>> > >>>Thank you for any suggestions, > >>>Lipi. > >>> > >>> > >>> > >>> > >>>------------------------------------------------------------------- > >>>-- > >>>To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>For additional commands, e-mail: > >>>[EMAIL PROTECTED] > >>> > >>> > >>>------------------------------------------------------------------- > >>>-- > >>>To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>For additional commands, e-mail: > >>>[EMAIL PROTECTED] > >>> > >>> > >>> > >> > >> > >> > >>-------------------------------------------------------------------- > >>- > >>To unsubscribe, e-mail: [EMAIL PROTECTED] > >>For additional commands, e-mail: [EMAIL PROTECTED] > >> > >> > > > > > > > > > >This e-mail, including any attachments, is a confidential business > communication, and may contain information that is confidential, > proprietary and/or privileged. This e-mail is intended only for the > individual(s) to whom it is addressed, and may not be saved, copied, > printed, disclosed or used by anyone else. If you are not the(an) > intended recipient, please immediately delete this e-mail from your > computer system and notify the sender. Thank you. > > > > > >--------------------------------------------------------------------- > >To unsubscribe, e-mail: [EMAIL PROTECTED] > >For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
