Am 29. April 2015 14:54:36 MESZ, schrieb Subhro Paul <subhro.p...@tcs.com>:
>-----Christopher Schultz <ch...@christopherschultz.net> wrote: -----
>To: Tomcat Users List <users@tomcat.apache.org>
>From: Christopher Schultz <ch...@christopherschultz.net>
>Date: 04/24/2015 07:14PM
>Subject: Re: Tomcat Thread issue
>
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA256
>
>Felix,
>
>On 4/24/15 3:19 AM, Felix Schumacher wrote:
>> Am 24. April 2015 09:08:08 MESZ, schrieb Subhro Paul
>> <subhro.p...@tcs.com>:
>>>
>>>
>>> -----Subhro Paul <subhro.p...@tcs.com> wrote: ----- To:
>>> users@tomcat.apache.org From: Subhro Paul <subhro.p...@tcs.com>
>>> Date: 04/23/2015 06:20PM Subject: Re: Tomcat Thread issue
>>>
>>> -----Daniel Mikusa <dmik...@pivotal.io> wrote: ----- To: Tomcat
>>> Users List <users@tomcat.apache.org> From: Daniel Mikusa
>>> <dmik...@pivotal.io> Date: 04/23/2015 05:01PM Subject: Re: Tomcat
>>> Thread issue
>>>
>>> On Thu, Apr 23, 2015 at 7:15 AM, Subhro Paul
>>> <subhro.p...@tcs.com> wrote:
>>>
>>>> Dear Team,
>>>>
>>>> One of our client's website stopped working yesterday. We
>>>> observed
>>> that
>>>> Tomcat servers were not working properly during that time. We
>>>> have
>>> checked
>>>> the memory usage of the server was fine but in the Catalina.out
>>>> log
>>> we
>>>> found it was already reached to max thread which is 512 though
>>>> the
>>> number
>>>> of connections to the server was normal. We took a thread dump
>>>> from
>>> the
>>>> server using VisualVM and we got the below message from
>>>> threaddump:
>>>>
>>>
>>> Since a thread dump is a point in time snapshot, you should
>>> always take multiple thread dumps, with a few seconds in between
>>> each one. This gives you additional perspective as to what's
>>> happening with the threads over a period of time.
>>>
>>>
>>>>
>>>> "http-8080-1" - Thread t@22
>>>>
>>>> java.lang.Thread.State: BLOCKED
>>>>
>>>> at java.util.Vector$1.nextElement(Vector.java:320)
>>>>
>>>> - waiting to lock <37749687> (a java.util.Vector) owned
>>> by
>>>> "http-8080-116" t@161
>>>>
>>>> at
>>>>
>org.apache.jsp.includes.header_jsp.isExcludePath(header_jsp.java:116
>)
>>>>
>>>>
>>>>
>at
>>>> org.apache.jsp.includes.header_jsp._jspService(header_jsp.java:314)
>>>>
>>>
>>>
>>>>
>Look at what header.jsp is doing. It seems to be doing something with
>>> the Vector class which is causing the thread to block.
>>>
>>>
>>>>
>>>> at
>>>> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
>>>>
>>>>
>>>>
>at
>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>>>>
>>>> at
>>>>
>>>
>org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper
>.java:377)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:3
>13)
>>>>
>>>>
>>>
>at
>>>> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
>>>>
>>>>
>>>>
>at
>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>>>>
>>>> at
>>>>
>>>
>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appl
>icationFilterChain.java:290)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationF
>ilterChain.java:206)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDisp
>atcher.java:646)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationD
>ispatcher.java:551)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDis
>patcher.java:488)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.jasper.runtime.JspRuntimeLibrary.include(JspRuntimeLibrary
>.java:968)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.jsp.home.customer_005fservice.bill.my_005fbill_jsp._jspSer
>vice(my_005fbill_jsp.java:126)
>>>>
>>>>
>>>
>at
>>>> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
>>>>
>>>>
>>>>
>at
>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>>>>
>>>> at
>>>>
>>>
>org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper
>.java:377)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:3
>13)
>>>>
>>>>
>>>
>at
>>>> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
>>>>
>>>>
>>>>
>at
>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>>>>
>>>> at
>>>>
>>>
>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appl
>icationFilterChain.java:290)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationF
>ilterChain.java:206)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperV
>alve.java:233)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.catalina.core.StandardContextValve.invoke(StandardContextV
>alve.java:191)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.j
>ava:127)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.j
>ava:102)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.catalina.valves.RequestFilterValve.process(RequestFilterVa
>lve.java:269)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.catalina.valves.RemoteHostValve.invoke(RemoteHostValve.jav
>a:81)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:
>555)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineVal
>ve.java:109)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.jav
>a:298)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.coyote.http11.Http11Processor.process(Http11Processor.java
>:857)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.proce
>ss(Http11Protocol.java:588)
>>>>
>>>>
>>>
>at
>>>>
>>>
>org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:48
>9)
>>>>
>>>>
>>>
>at java.lang.Thread.run(Thread.java:701)
>>>>
>>>>
>>>>
>>>> Locked ownable synchronizers:
>>>>
>>>> - ÿ ÿ ÿ ÿ ÿNone
>>>>
>>>>
>>>>
>>>> This was coming for different threads. Once we restarted the
>>>> servers,
>>> the
>>>> website back to normal again but we got the below exception in
>>>> the
>>> log :
>>>>
>>>>
>>>>
>>>> Apr 22, 2015 11:15:28 AM
>>>> org.apache.catalina.loader.WebappClassLoader
>>>> clearReferencesThreads
>>>>
>>>> SEVERE: A web application appears to have started a thread
>>>> named [http-8080-1] but has failed to stop it. This is very
>>>> likely to
>>> create a
>>>> memory leak.
>>>>
>>>>
>>> This means your app started a thread and failed to properly stop
>>> it. ˜If you stopped the process completely, this is not going
>to
>>> cause any problems because the entire process would have exited
>>> and so this thread would go away any way. ˜If you're doing
>>> hot redeploy's (i.e. deplying apps without restarting the
>>> process), this could cause problems. ˜Either way, when you
>>> have a moment you should try to investigate what is creating
>>> that thread and setup something to properly close it.
>>>
>>>
>>>>
>>>>
>>>> So, we want to know while the thread is blocked is it like
>>>> deadlock condition for which all threads were unavailable?
>>>
>>>
>>> The thread dump simply lists the thread as "blocked". ˜This
>>> generally means that at some point it will complete, so it's not
>>> technically a deadlock. If you had multiple thread dumps, you
>>> could watch the thread across all of them and see how it
>>> progresses.
>>>
>>>
>>>> Current thread count I about 190 but after few days this will
>>>> reach
>>> to
>>>> 500+ again even if the concurrent users are not high. Memory
>>>> usage of
>>> the
>>>> server was normal during this issue. This problem is happening
>>>> from
>>> last 2
>>>> 3 months.
>>>>
>>>
>>> Look at the header.jsp, especially anything that has changed in
>>> the last 2 - 3 months.
>>>
>>> Dan
>>>
>>>
>>>>
>>>>
>>>>
>>>> Thanks & Regards,
>>>>
>>>> Subhro Paul
>>>
>>>
>>> Dear Team,
>>>
>>> Thanks for the reply.
>>>
>>> Next thing I want to know, creating a vector and adding objects
>>> to the same in header.jsp which is loaded
>>> and˜referred˜by˜almost˜all the JSP's(more
>>> than 1000) in an application, will that leads to memory leak.
>>>
>>> Thanks & Regards, Subhro Paul
>>>
>>>
>>>
>>> Hi all,
>>>
>>> Is there any way to understand why the vector object was locked
>>> by thread "http-8080-116" for long? From the thread dump we can
>>> see there is around 140 threads were blocked because of the
>>> thread "http-8080-116" acquired locked on the vector object.˜
>>
>> Look at the full thread dump. There you will see where the lock is
>> held. We can't tell you just from looking at the stacktrace.
>
>+1
>
>My question would be what that Vector is being used for. If all
>threads have to consult that Vector, and its synchronized (which is
>it: it's a java.util.Vector), then every thread will be waiting on
>every other thread.
>
>If this Vector is filled with read-only data, and the contents don't
>need to be changed, then consider using another concrete List
>implementation there. If you use ArrayList or LinkedList or whatever,
>your page accesses will speed-up significantly.
>
>If I were you, I'd investigate the possibility of switching
>implementations. I would also wrap the unsynchronized, read-only list
>in a formally-read-only list implementation like this:
>
>public class Startup implements ServletContextListener {
>ÿÿpublic void contextInitialized(...) {
>ÿÿ ÿ List<Thing> listOfThings = new ArrayList<>(...);
>ÿÿ ÿ // fill listOfThings with things
>ÿÿ ÿ listOfThings =
>java.util.Collections.unmodifiableList(listOfThings)
>;
>ÿÿ ÿ getServletContext().setAttribute("listOfThings");
>ÿÿ}
>}
>
>This will give you a structure that will throw an error if you try to
>modify it.
>
>////
>
>If you ARE trying to modify it, you have to ask yourself why you are
>using a single structure that every request coming into the
>application is going to have to fight-over.
>
>In either case, can you explain the use-case here?
>
>- -chris
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v2
>Comment: GPGTools - http://gpgtools.org
>
>
>Hi Team,
>ÿ
>While staring tomcat we can observeÿÿthrough Jconsole thatÿsome threads
>are automatically created by Tomcat. Once we put load on the server,
>depending on the load Tomcat increases the thread count but if we stop
>putting load on the server we can see theÿstillÿThread count on the
>server remain same even after long time. My assumption was it would
>destroy some threads once there is a little load or no load which is
>not the actual scenario. We can see that the unused threads are in
>sleep mode. Is it a normal feature of Tomcat or some configuration is
>required in Tomcat to destroy inactive threads. FYI we are using
>“HTTP 1.1” connector.
Have a look at https://tomcat.apache.org/tomcat-7.0-doc/config/executor.html
Regards
Felix
>
>Thanks & Regards,
>Subhro Paul
>
>=====-----=====-----=====
>Notice: The information contained in this e-mail
>message and/or attachments to it may contain
>confidential or privileged information. If you are
>not the intended recipient, any dissemination, use,
>review, distribution, printing or copying of the
>information contained in this e-mail message
>and/or attachments to it are strictly prohibited. If
>you have received this communication in error,
>please notify us by reply e-mail or telephone and
>immediately and permanently delete the message
>and any attachments. Thank you
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
