Am 24. April 2015 09:08:08 MESZ, schrieb Subhro Paul <subhro.p...@tcs.com>: > > >-----Subhro Paul <subhro.p...@tcs.com> wrote: ----- >To: users@tomcat.apache.org >From: Subhro Paul <subhro.p...@tcs.com> >Date: 04/23/2015 06:20PM >Subject: Re: Tomcat Thread issue > >-----Daniel Mikusa <dmik...@pivotal.io> wrote: ----- >To: Tomcat Users List <users@tomcat.apache.org> >From: Daniel Mikusa <dmik...@pivotal.io> >Date: 04/23/2015 05:01PM >Subject: Re: Tomcat Thread issue > >On Thu, Apr 23, 2015 at 7:15 AM, Subhro Paul <subhro.p...@tcs.com> >wrote: > >> Dear Team, >> >> One of our client's website stopped working yesterday. We observed >that >> Tomcat servers were not working properly during that time. We have >checked >> the memory usage of the server was fine but in the Catalina.out log >we >> found it was already reached to max thread which is 512 though the >number >> of connections to the server was normal. We took a thread dump from >the >> server using VisualVM and we got the below message from threaddump: >> > >Since a thread dump is a point in time snapshot, you should always take >multiple thread dumps, with a few seconds in between each one. This >gives >you additional perspective as to what's happening with the threads over >a >period of time. > > >> >> "http-8080-1" - Thread t@22 >> >> java.lang.Thread.State: BLOCKED >> >> at java.util.Vector$1.nextElement(Vector.java:320) >> >> - waiting to lock <37749687> (a java.util.Vector) owned >by >> "http-8080-116" t@161 >> >> at >> org.apache.jsp.includes.header_jsp.isExcludePath(header_jsp.java:116) >> >> at >> org.apache.jsp.includes.header_jsp._jspService(header_jsp.java:314) >> > >Look at what header.jsp is doing. It seems to be doing something with >the >Vector class which is causing the thread to block. > > >> >> at >> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) >> >> at >javax.servlet.http.HttpServlet.service(HttpServlet.java:717) >> >> at >> >org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:377) >> >> at >> >org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313) >> >> at >> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260) >> >> at >javax.servlet.http.HttpServlet.service(HttpServlet.java:717) >> >> at >> >org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) >> >> at >> >org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >> >> at >> >org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:646) >> >> at >> >org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:551) >> >> at >> >org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:488) >> >> at >> >org.apache.jasper.runtime.JspRuntimeLibrary.include(JspRuntimeLibrary.java:968) >> >> at >> >org.apache.jsp.home.customer_005fservice.bill.my_005fbill_jsp._jspService(my_005fbill_jsp.java:126) >> >> at >> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) >> >> at >javax.servlet.http.HttpServlet.service(HttpServlet.java:717) >> >> at >> >org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:377) >> >> at >> >org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313) >> >> at >> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260) >> >> at >javax.servlet.http.HttpServlet.service(HttpServlet.java:717) >> >> at >> >org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) >> >> at >> >org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >> >> at >> >org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) >> >> at >> >org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) >> >> at >> >org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) >> >> at >> >org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) >> >> at >> >org.apache.catalina.valves.RequestFilterValve.process(RequestFilterValve.java:269) >> >> at >> >org.apache.catalina.valves.RemoteHostValve.invoke(RemoteHostValve.java:81) >> >> at >> >org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555) >> >> at >> >org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) >> >> at >> >org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) >> >> at >> >org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857) >> >> at >> >org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) >> >> at >> >org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) >> >> at java.lang.Thread.run(Thread.java:701) >> >> >> >> Locked ownable synchronizers: >> >> - None >> >> >> >> This was coming for different threads. Once we restarted the servers, >the >> website back to normal again but we got the below exception in the >log : >> >> >> >> Apr 22, 2015 11:15:28 AM org.apache.catalina.loader.WebappClassLoader >> clearReferencesThreads >> >> SEVERE: A web application appears to have started a thread named >> [http-8080-1] but has failed to stop it. This is very likely to >create a >> memory leak. >> >> >This means your app started a thread and failed to properly stop it. >ÿIf >you stopped the process completely, this is not going to cause any >problems >because the entire process would have exited and so this thread would >go >away any way. ˜If you're doing hot redeploy's (i.e. deplying apps >without >restarting the process), this could cause problems. ˜Either way, >when you >have a moment you should try to investigate what is creating that >thread >and setup something to properly close it. > > >> >> >> So, we want to know while the thread is blocked is it like deadlock >> condition for which all threads were unavailable? > > >The thread dump simply lists the thread as "blocked". ˜This >generally means >that at some point it will complete, so it's not technically a >deadlock. >If you had multiple thread dumps, you could watch the thread across all >of >them and see how it progresses. > > >> Current thread count I about 190 but after few days this will reach >to >> 500+ again even if the concurrent users are not high. Memory usage of >the >> server was normal during this issue. This problem is happening from >last 2 >> 3 months. >> > >Look at the header.jsp, especially anything that has changed in the >last 2 >- 3 months. > >Dan > > >> >> >> >> Thanks & Regards, >> >> Subhro Paul > > >Dear Team, > >Thanks for the reply. > >Next thing I want to know, creating a vector and adding objects to the >same in header.jsp which is loaded >and˜referred˜by˜almost˜all the JSP's(more than >1000) in an application, will that leads to memory leak. > >Thanks & Regards, >Subhro Paul > > > >Hi all, > >Is there any way to understand why the vector object was locked by >thread "http-8080-116" for long? From the thread dump we can see there >is around 140 threads were blocked because of the thread >"http-8080-116" acquired locked on the vector object.ÿ
Look at the full thread dump. There you will see where the lock is held. We can't tell you just from looking at the stacktrace. Regards Felix > >Thanks & Regards, >Subhro Paul > >=====-----=====-----===== >Notice: The information contained in this e-mail >message and/or attachments to it may contain >confidential or privileged information. If you are >not the intended recipient, any dissemination, use, >review, distribution, printing or copying of the >information contained in this e-mail message >and/or attachments to it are strictly prohibited. If >you have received this communication in error, >please notify us by reply e-mail or telephone and >immediately and permanently delete the message >and any attachments. Thank you --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
