Thanks for the response Chris.
Actually, we had a bit of a situation yesterday, it seems the threads
were all locked up and/or the db connection pool was not handing out
connections. I am not sure what happened and have been monitoring the
threads ever since.
Below are some threads from the dump when the above problem happened. I
have no idea what caused the problem, I do know that the mail server I
was trying to connect to was flaky, that would explain thread
"http-443-Processor75" but I have no clue why the dbcp connection
pooling failed.
Chris, are you familiar with Spring at all? Would you be able to answer
a few questions for me on this subject (we can take the topic offline as
it is not relevant here).
Thanks for your help. Appreciate it.
-Riz.
"http-443-Processor75" daemon prio=1 tid=0x09fbc620 nid=0x330a runnable
[0x07d76000..0x07d790c0]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at com.sun.mail.util.TraceInputStream.read(TraceInputStream.java:79)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
- locked <0x5cd7e218> (a java.io.BufferedInputStream)
at com.sun.mail.util.LineInputStream.readLine(LineInputStream.java:57)
at
com.sun.mail.smtp.SMTPTransport.readServerResponse(SMTPTransport.java:1327)
at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1191)
at
com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:311)
at javax.mail.Service.connect(Service.java:233)
at
org.springframework.mail.javamail.JavaMailSenderImpl.doSend(JavaMailSenderImpl.java:373)
at
org.springframework.mail.javamail.JavaMailSenderImpl.send(JavaMailSenderImpl.java:326)
at
org.springframework.mail.javamail.JavaMailSenderImpl.send(JavaMailSenderImpl.java:322)
at
com.cwsi.eshipper.service.impl.EmailManagerImpl.sendMessageWithAttachments(EmailManagerImpl.java:209)
at
com.cwsi.eshipper.service.impl.EmailManagerImpl.onChargeCard(EmailManagerImpl.java:277)
at
com.cwsi.eshipper.service.impl.CCTransactionManagerImpl.emailReceipt(CCTransactionManagerImpl.java:435)
at
com.cwsi.eshipper.service.impl.CCTransactionManagerImpl.processPostAuth(CCTransactionManagerImpl.java:246)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:292)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:155)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:122)
at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:144)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:174)
at $Proxy10.processPostAuth(Unknown Source)
at
com.cwsi.eshipper.service.impl.CarrierServiceManagerImpl.shipOrder(CarrierServiceManagerImpl.java:262)
at sun.reflect.GeneratedMethodAccessor462.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:292)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:155)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:122)
at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:144)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:174)
at $Proxy19.shipOrder(Unknown Source)
at
com.cwsi.eshipper.web.OrderManagerAction.save_internal(OrderManagerAction.java:456)
at
com.cwsi.eshipper.web.OrderManagerAction.save(OrderManagerAction.java:1439)
at sun.reflect.GeneratedMethodAccessor2710.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:274)
at
org.apache.struts.actions.DispatchAction.execute(DispatchAction.java:194)
at
org.springframework.web.struts.DelegatingActionProxy.execute(DelegatingActionProxy.java:106)
at
org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:419)
at
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:224)
at
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1194)
at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
org.displaytag.filter.ResponseOverrideFilter.doFilter(ResponseOverrideFilter.java:125)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
net.sf.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:292)
at
net.sf.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:108)
at
net.sf.acegisecurity.intercept.web.SecurityEnforcementFilter.doFilter(SecurityEnforcementFilter.java:197)
at
net.sf.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:303)
at
net.sf.acegisecurity.wrapper.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:50)
at
net.sf.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:303)
at
net.sf.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:324)
at
net.sf.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:303)
at
net.sf.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:324)
at
net.sf.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:303)
at
net.sf.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:220)
at
net.sf.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:303)
at
net.sf.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:173)
at
net.sf.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:120)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:142)
at
com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:58)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:172)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:432)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:595)
"http-443-Processor74" daemon prio=1 tid=0x09fbb720 nid=0x3309 in
Object.wait() [0x07cf7000..0x07cf8040]
at java.lang.Object.wait(Native Method)
- waiting on <0x55ed65c0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Object.wait(Object.java:474)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:656)
- locked <0x55ed65c0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Thread.run(Thread.java:595)
"http-443-Processor73" daemon prio=1 tid=0xb2ac23d0 nid=0x3308 in
Object.wait() [0x07b96000..0x07b97fc0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at
org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:748)
- locked <0x539f5a70> (a org.apache.commons.pool.impl.GenericObjectPool)
at
org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:95)
at
org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:540)
at
org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConnection(LocalDataSourceConnectionProvider.java:80)
at
org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:417)
at
org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:144)
at
org.hibernate.jdbc.BorrowedConnectionProxy.invoke(BorrowedConnectionProxy.java:40)
at $Proxy18.isReadOnly(Unknown Source)
at
org.springframework.jdbc.datasource.DataSourceUtils.resetConnectionAfterTransaction(DataSourceUtils.java:186)
at
org.springframework.orm.hibernate3.HibernateTransactionManager.doCleanupAfterCompletion(HibernateTransactionManager.java:551)
at
org.springframework.transaction.support.AbstractPlatformTransactionManager.cleanupAfterCompletion(AbstractPlatformTransactionManager.java:749)
at
org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:536)
at
org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:468)
at
org.springframework.transaction.interceptor.TransactionAspectSupport.doCommitTransactionAfterReturning(TransactionAspectSupport.java:258)
at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:144)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:174)
at $Proxy15.search(Unknown Source)
at
com.cwsi.eshipper.web.OrderManagerAction.perform_search(OrderManagerAction.java:568)
at sun.reflect.GeneratedMethodAccessor2674.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:274)
at
org.apache.struts.actions.DispatchAction.execute(DispatchAction.java:194)
at
org.springframework.web.struts.DelegatingActionProxy.execute(DelegatingActionProxy.java:106)
at
org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:419)
at
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:224)
at
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1194)
at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
org.displaytag.filter.ResponseOverrideFilter.doFilter(ResponseOverrideFilter.java:125)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
net.sf.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:292)
at
net.sf.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:108)
at
net.sf.acegisecurity.intercept.web.SecurityEnforcementFilter.doFilter(SecurityEnforcementFilter.java:197)
at
net.sf.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:303)
at
net.sf.acegisecurity.wrapper.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:50)
at
net.sf.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:303)
at
net.sf.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:324)
at
net.sf.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:303)
at
net.sf.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:324)
at
net.sf.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:303)
at
net.sf.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:220)
at
net.sf.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:303)
at
net.sf.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:173)
at
net.sf.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:120)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:142)
at
com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:58)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:172)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:432)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:595)
Christopher Schultz wrote:
Riz,
When I perform a thread dump I am seeing some threads are locked on a
socketRead0 (like http-443-Processor49 in the dump below).
Let's take a look:
"http-443-Processor49" daemon prio=1 tid=0xb208fe80 nid=0x40c5 runnable
[0x06fdd000..0x06fddf40]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
What's wrong with that? Your thread is runnable (good), and it's reading
data from an InputStream. Lots of the time, your request processor
threads will be reading from sockets because they are accepting request
information from the client. Several of your threads are in this state.
Can someone PLEASE help me understand what this means? Also, if you can
see any other threads that seem to be a threat, please feel free to let
me know...
I don't see any "threatening"-looking threads in there at all.
"http-443-Processor50" daemon prio=1 tid=0xb2090c40 nid=0x40c6 in
Object.wait() [0x0705e000..0x0705efc0]
at java.lang.Object.wait(Native Method)
- waiting on <0x54f20ee8> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
Note that this request processor is blocking, waiting for a request to
come in in order to be handled. This thread is basically idle. Many of
your threads are in this state.
Are you posting this question in response to any kind of observed
problem, or are you just freaking out at the number of threads that are
running in your JVM?
-chris
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
