We are running separate tomcat and apache servers, the http requests are being proxied from the apache server to the tomcat server. Tomcat connects to an informix database; our application is a webpage that allows users to make queries against the dB and report the results in html.
Here is the box we are running the tomcat on: [EMAIL PROTECTED] javaapps]# uname -a Linux serv3.nmcourts.com 2.6.9-22.ELsmp #1 SMP Sat Oct 8 19:11:43 CDT 2005 i686 i686 i386 GNU/Linux [EMAIL PROTECTED] javaapps]# cat /etc/redhat-release CentOS release 4.3 (Final) [EMAIL PROTECTED] javaapps]# We are using the following vers. of tomcat & apache: [EMAIL PROTECTED] javaapps]# ls apache-ant-1.6.5 apache-tomcat-5.5.16 jakarta-tomcat-connectors-1.2.15-src jdk1.5.0_06 apache-ant-1.6.5-bin.tar apache-tomcat-5.5.16.tar.gz jakarta-tomcat-connectors-1.2.15-src.tar.gz jdk-1_5_0_06-linux-i586.bin At least once a day the tomcat process "hangs". Loading the default page brings an error saying tomcat cannot connect to the dB. Running queries on the tomcat server itself we get the same error. We run another tomcat app (on a separate box) that queries the same dB and it works fine at the same time as this problem on the other tomcat box. Only restarting tomcat fixes this problem, a restart of the informix server does not fix it. Looking at logs, I do not see anything logged to indicate that the process is hung. Can someone suggest what I can do to get better information on what is happening and why ? We are some logs, we had the restart this morning at 8:06:13 AM and as you can see there was nothing in the logs (catalina.out) prior to that other that a restart not quite 24 hrs prior. Apr 23, 2006 10:38:50 AM org.apache.catalina.core.ApplicationContext log INFO: ContextListener: contextInitialized() Apr 23, 2006 10:38:50 AM org.apache.catalina.core.ApplicationContext log INFO: SessionListener: contextInitialized() Apr 23, 2006 10:38:50 AM org.apache.catalina.core.ApplicationContext log INFO: ContextListener: contextInitialized() Apr 23, 2006 10:38:50 AM org.apache.catalina.core.ApplicationContext log INFO: SessionListener: contextInitialized() Apr 23, 2006 10:38:50 AM org.apache.coyote.http11.Http11BaseProtocol start INFO: Starting Coyote HTTP/1.1 on http-8080 Apr 23, 2006 10:38:51 AM org.apache.jk.common.ChannelSocket init INFO: JK: ajp13 listening on /0.0.0.0:8009 Apr 23, 2006 10:38:51 AM org.apache.jk.server.JkMain start INFO: Jk running ID=0 time=0/24 config=null Apr 23, 2006 10:38:51 AM org.apache.catalina.storeconfig.StoreLoader load INFO: Find registry server-registry.xml at classpath resource Apr 23, 2006 10:38:51 AM org.apache.catalina.startup.Catalina start INFO: Server startup in 4596 ms Apr 24, 2006 8:06:13 AM org.apache.coyote.http11.Http11BaseProtocol pause INFO: Pausing Coyote HTTP/1.1 on http-8080 Apr 24, 2006 8:06:14 AM org.apache.catalina.core.StandardService stop INFO: Stopping service Catalina Apr 24, 2006 8:06:14 AM org.apache.catalina.core.ApplicationContext log INFO: SessionListener: contextDestroyed() Apr 24, 2006 8:06:14 AM org.apache.catalina.core.ApplicationContext log INFO: ContextListener: contextDestroyed() Apr 24, 2006 8:06:14 AM org.apache.catalina.core.ApplicationContext log INFO: SessionListener: contextDestroyed() Apr 24, 2006 8:06:14 AM org.apache.catalina.core.ApplicationContext log INFO: ContextListener: contextDestroyed() - Closing Spring root WebApplicationContext - Closing application context [ org.springframework.web.context.support.XmlWebApplicationContext ;hashCode=32278793] - Destroying singletons in factory { org.springframework.beans.factory.support.DefaultListableBeanFactorydefining beans [dataSource,jdbcTemplate,searchService,partySearchService,searchServiceDaoBase,searchServiceDaoImpl,caselookupDataAccess,searchServiceUtils,searchController,caselookupSearchController,searchDetailController,dwiSearchController,dwiSearchService,searchDetailService,partySearchResults,dwiSearchResults,searchDetailWrapperBean,caseDetail,criminalChargeDetail,judgeAssignmentHistory,roaActivity,caseParties,civilComplaint,searchValidator,urlMapping,viewResolver]; root of BeanFactory hierarchy} - Closing WebApplicationContext of Spring FrameworkServlet 'caselookup' - Closing application context [WebApplicationContext for namespace 'caselookup-servlet'] - Destroying singletons in factory { org.springframework.beans.factory.support.DefaultListableBeanFactorydefining beans [searchController,caselookupSearchController,searchDetailController,dwiSearchController,searchServiceUtils,partySearchService,dwiSearchService,searchDetailService,partySearchResults,dwiSearchResults,searchDetailWrapperBean,caseDetail,criminalChargeDetail,judgeAssignmentHistory,roaActivity,caseParties,civilComplaint,searchValidator,urlMapping,viewResolver]; parent: org.springframework.beans.factory.support.DefaultListableBeanFactorydefining beans [dataSource,jdbcTemplate,searchService,partySearchService,searchServiceDaoBase,searchServiceDaoImpl,caselookupDataAccess,searchServiceUtils,searchController,caselookupSearchController,searchDetailController,dwiSearchController,dwiSearchService,searchDetailService,partySearchResults,dwiSearchResults,searchDetailWrapperBean,caseDetail,criminalChargeDetail,judgeAssignmentHistory,roaActivity,caseParties,civilComplaint,searchValidator,urlMapping,viewResolver]; root of BeanFactory hierarchy} Apr 24, 2006 8:06:14 AM org.apache.coyote.http11.Http11BaseProtocol destroy INFO: Stopping Coyote HTTP/1.1 on http-8080 Apr 24, 2006 8:06:22 AM org.apache.coyote.http11.Http11BaseProtocol init INFO: Initializing Coyote HTTP/1.1 on http-8080 Apr 24, 2006 8:06:22 AM org.apache.catalina.startup.Catalina load INFO: Initialization processed in 870 ms Apr 24, 2006 8:06:22 AM org.apache.catalina.core.StandardService start INFO: Starting service Catalina Apr 24, 2006 8:06:22 AM org.apache.catalina.core.StandardEngine start INFO: Starting Servlet Engine: Apache Tomcat/5.5.16 Apr 24, 2006 8:06:22 AM org.apache.catalina.core.StandardHost start INFO: XML validation disabled Apr 24, 2006 8:06:22 AM org.apache.catalina.loader.WebappClassLoadervalidateJarFile INFO: validateJarFile(/usr/local/javaapps/apache-tomcat-5.5.16 /webapps/caselookup/WEB-INF/lib/servlet-api.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class Apr 24, 2006 8:06:22 AM org.apache.catalina.loader.WebappClassLoadervalidateJarFile INFO: validateJarFile(/usr/local/javaapps/apache-tomcat-5.5.16 /webapps/caselookup/WEB-INF/lib/servletapi-2.3.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class - Root WebApplicationContext: initialization started - Loading Spring root WebApplicationContext - Loading XML bean definitions from ServletContext resource [/WEB-INF/applicationContext.xml] - Loading XML bean definitions from ServletContext resource [/WEB-INF/caselookup-servlet.xml] - Overriding bean definition for bean 'searchServiceUtils': replacing [Root bean with class [com.nmcourts.caselookup.util.SearchServiceUtils] defined in ServletContext resource [/WEB-INF/applicationContext.xml]] with [Root bean with class [com.nmcourts.caselookup.util.SearchServiceUtils] defined in ServletContext resource [/WEB-INF/caselookup-servlet.xml]] - Overriding bean definition for bean 'partySearchService': replacing [Root bean with class [com.nmcourts.caselookup.service.PartySearchService] defined in ServletContext resource [/WEB-INF/applicationContext.xml]] with [Root bean with class [com.nmcourts.caselookup.service.PartySearchService] defined in ServletContext resource [/WEB-INF/caselookup-servlet.xml]] - Bean factory for application context [ org.springframework.web.context.support.XmlWebApplicationContext;hashCode=32278793]: org.springframework.beans.factory.support.DefaultListableBeanFactorydefining beans [dataSource,jdbcTemplate,searchService,partySearchService,searchServiceDaoBase,searchServiceDaoImpl,caselookupDataAccess,searchServiceUtils,searchController,caselookupSearchController,searchDetailController,dwiSearchController,dwiSearchService,searchDetailService,partySearchResults,dwiSearchResults,searchDetailWrapperBean,caseDetail,criminalChargeDetail,judgeAssignmentHistory,roaActivity,caseParties,civilComplaint,searchValidator,urlMapping,viewResolver]; root of BeanFactory hierarchy - 26 beans defined in application context [ org.springframework.web.context.support.XmlWebApplicationContext ;hashCode=32278793] - JDK 1.4+ collections available - Commons Collections 3.x available - Unable to locate MessageSource with name 'messageSource': using default [ [EMAIL PROTECTED] - Unable to locate ApplicationEventMulticaster with name 'applicationEventMulticaster': using default [ [EMAIL PROTECTED] - No ThemeSource found for [ org.springframework.web.context.support.XmlWebApplicationContext;hashCode=32278793]: using ResourceBundleThemeSource - Pre-instantiating singletons in factory [ org.springframework.beans.factory.support.DefaultListableBeanFactorydefining beans [dataSource,jdbcTemplate,searchService,partySearchService,searchServiceDaoBase,searchServiceDaoImpl,caselookupDataAccess,searchServiceUtils,searchController,caselookupSearchController,searchDetailController,dwiSearchController,dwiSearchService,searchDetailService,partySearchResults,dwiSearchResults,searchDetailWrapperBean,caseDetail,criminalChargeDetail,judgeAssignmentHistory,roaActivity,caseParties,civilComplaint,searchValidator,urlMapping,viewResolver]; root of BeanFactory hierarchy] - Creating shared instance of singleton bean 'dataSource' - Creating shared instance of singleton bean 'jdbcTemplate' - Loading XML bean definitions from class path resource [org/springframework/jdbc/support/sql-error-codes.xml] - Creating shared instance of singleton bean 'DB2' - Creating shared instance of singleton bean 'HSQL' - Creating shared instance of singleton bean 'MS-SQL' - Creating shared instance of singleton bean 'MySQL' - Creating shared instance of singleton bean 'Oracle' - Creating shared instance of singleton bean 'Informix' - Creating shared instance of singleton bean 'PostgreSQL' - Creating shared instance of singleton bean 'Sybase' - SQLErrorCodes loaded: [DB2, HSQL, MS-SQL, MySQL, Oracle, Informix, PostgreSQL, Sybase] - Creating shared instance of singleton bean 'searchService' - Creating shared instance of singleton bean 'searchServiceDaoImpl' - Creating shared instance of singleton bean 'partySearchService' - Creating shared instance of singleton bean 'partySearchResults' - Creating shared instance of singleton bean 'searchServiceDaoBase' - Creating shared instance of singleton bean 'caselookupDataAccess' - Creating shared instance of singleton bean 'searchServiceUtils' - Creating shared instance of singleton bean 'searchController' - Creating shared instance of singleton bean 'caselookupSearchController' - Creating shared instance of singleton bean 'searchDetailService' - Creating shared instance of singleton bean 'searchDetailWrapperBean' - Creating shared instance of singleton bean 'caseDetail' - Creating shared instance of singleton bean 'criminalChargeDetail' - Creating shared instance of singleton bean 'judgeAssignmentHistory' - Creating shared instance of singleton bean 'roaActivity' - Creating shared instance of singleton bean 'caseParties' - Creating shared instance of singleton bean 'civilComplaint' - Creating shared instance of singleton bean 'searchValidator' - Creating shared instance of singleton bean 'searchDetailController' - Creating shared instance of singleton bean 'dwiSearchController' - Creating shared instance of singleton bean 'dwiSearchService' - Creating shared instance of singleton bean 'dwiSearchResults' - Creating shared instance of singleton bean 'urlMapping' - Mapped URL path [/search.htm] onto handler [ [EMAIL PROTECTED] - Mapped URL path [/searchDetail.htm] onto handler [ [EMAIL PROTECTED] - Mapped URL path [/dwiSearchDetail.htm] onto handler [ [EMAIL PROTECTED] - Mapped URL path [/partySearchResults.htm] onto handler [ [EMAIL PROTECTED] - Mapped URL path [/dwiSearchResults.htm] onto handler [ [EMAIL PROTECTED] - Creating shared instance of singleton bean 'viewResolver' - Using context class [ org.springframework.web.context.support.XmlWebApplicationContext] for root WebApplicationContext - Root WebApplicationContext: initialization completed in 3155 ms - Initializing servlet 'caselookup' - FrameworkServlet 'caselookup': initialization started - Loading WebApplicationContext for Spring FrameworkServlet 'caselookup' - Loading XML bean definitions from ServletContext resource [/WEB-INF/caselookup-servlet.xml] - Bean factory for application context [WebApplicationContext for namespace 'caselookup-servlet']: org.springframework.beans.factory.support.DefaultListableBeanFactorydefining beans [searchController,caselookupSearchController,searchDetailController,dwiSearchController,searchServiceUtils,partySearchService,dwiSearchService,searchDetailService,partySearchResults,dwiSearchResults,searchDetailWrapperBean,caseDetail,criminalChargeDetail,judgeAssignmentHistory,roaActivity,caseParties,civilComplaint,searchValidator,urlMapping,viewResolver]; parent: org.springframework.beans.factory.support.DefaultListableBeanFactorydefining beans [dataSource,jdbcTemplate,searchService,partySearchService,searchServiceDaoBase,searchServiceDaoImpl,caselookupDataAccess,searchServiceUtils,searchController,caselookupSearchController,searchDetailController,dwiSearchController,dwiSearchService,searchDetailService,partySearchResults,dwiSearchResults,searchDetailWrapperBean,caseDetail,criminalChargeDetail,judgeAssignmentHistory,roaActivity,caseParties,civilComplaint,searchValidator,urlMapping,viewResolver]; root of BeanFactory hierarchy - 20 beans defined in application context [WebApplicationContext for namespace 'caselookup-servlet'] - Unable to locate MessageSource with name 'messageSource': using default [ [EMAIL PROTECTED] - Unable to locate ApplicationEventMulticaster with name 'applicationEventMulticaster': using default [ [EMAIL PROTECTED] - No ThemeSource found for [WebApplicationContext for namespace 'caselookup-servlet']: using ResourceBundleThemeSource - Pre-instantiating singletons in factory [ org.springframework.beans.factory.support.DefaultListableBeanFactorydefining beans [searchController,caselookupSearchController,searchDetailController,dwiSearchController,searchServiceUtils,partySearchService,dwiSearchService,searchDetailService,partySearchResults,dwiSearchResults,searchDetailWrapperBean,caseDetail,criminalChargeDetail,judgeAssignmentHistory,roaActivity,caseParties,civilComplaint,searchValidator,urlMapping,viewResolver]; parent: org.springframework.beans.factory.support.DefaultListableBeanFactorydefining beans [dataSource,jdbcTemplate,searchService,partySearchService,searchServiceDaoBase,searchServiceDaoImpl,caselookupDataAccess,searchServiceUtils,searchController,caselookupSearchController,searchDetailController,dwiSearchController,dwiSearchService,searchDetailService,partySearchResults,dwiSearchResults,searchDetailWrapperBean,caseDetail,criminalChargeDetail,judgeAssignmentHistory,roaActivity,caseParties,civilComplaint,searchValidator,urlMapping,viewResolver]; root of BeanFactory hierarchy] - Creating shared instance of singleton bean 'searchController' - Creating shared instance of singleton bean 'caselookupSearchController' - Creating shared instance of singleton bean 'searchServiceUtils' - Creating shared instance of singleton bean 'searchDetailService' - Creating shared instance of singleton bean 'searchDetailWrapperBean' - Creating shared instance of singleton bean 'caseDetail' - Creating shared instance of singleton bean 'criminalChargeDetail' - Creating shared instance of singleton bean 'judgeAssignmentHistory' - Creating shared instance of singleton bean 'roaActivity' - Creating shared instance of singleton bean 'caseParties' - Creating shared instance of singleton bean 'civilComplaint' - Creating shared instance of singleton bean 'partySearchService' - Creating shared instance of singleton bean 'partySearchResults' - Creating shared instance of singleton bean 'searchValidator' - Creating shared instance of singleton bean 'searchDetailController' - Creating shared instance of singleton bean 'dwiSearchController' - Creating shared instance of singleton bean 'dwiSearchService' - Creating shared instance of singleton bean 'dwiSearchResults' - Creating shared instance of singleton bean 'urlMapping' - Mapped URL path [/search.htm] onto handler [ [EMAIL PROTECTED] - Mapped URL path [/searchDetail.htm] onto handler [ [EMAIL PROTECTED] - Mapped URL path [/dwiSearchDetail.htm] onto handler [ [EMAIL PROTECTED] - Mapped URL path [/partySearchResults.htm] onto handler [ [EMAIL PROTECTED] - Mapped URL path [/dwiSearchResults.htm] onto handler [ [EMAIL PROTECTED] - Creating shared instance of singleton bean 'viewResolver' - Using context class [ org.springframework.web.context.support.XmlWebApplicationContext] for servlet 'caselookup' - Unable to locate MultipartResolver with name 'multipartResolver': no multipart request handling provided - Unable to locate LocaleResolver with name 'localeResolver': using default [EMAIL PROTECTED] - Unable to locate ThemeResolver with name 'themeResolver': using default [ [EMAIL PROTECTED] - No HandlerAdapters found in servlet 'caselookup': using default - FrameworkServlet 'caselookup': initialization completed in 55 ms - Servlet 'caselookup' configured successfully Apr 24, 2006 8:06:26 AM org.apache.catalina.core.ApplicationContext log INFO: org.apache.webapp.balancer.BalancerFilter: init(): ruleChain: [ org.apache.webapp.balancer.RuleChain: [ org.apache.webapp.balancer.rules.URLStringMatchRule: Target string: News / Redirect URL: http://www.cnn.com], [ org.apache.webapp.balancer.rules.RequestParameterRule: Target param name: paramName / Target param value: paramValue / Redirect URL: http://www.yahoo.com], [ org.apache.webapp.balancer.rules.AcceptEverythingRule: Redirect URL: http://jakarta.apache.org]] Apr 24, 2006 8:06:26 AM org.apache.catalina.core.ApplicationContext log INFO: ContextListener: contextInitialized() Apr 24, 2006 8:06:26 AM org.apache.catalina.core.ApplicationContext log INFO: SessionListener: contextInitialized() Apr 24, 2006 8:06:27 AM org.apache.catalina.core.ApplicationContext log INFO: ContextListener: contextInitialized() Apr 24, 2006 8:06:27 AM org.apache.catalina.core.ApplicationContext log INFO: SessionListener: contextInitialized() Apr 24, 2006 8:06:27 AM org.apache.coyote.http11.Http11BaseProtocol start INFO: Starting Coyote HTTP/1.1 on http-8080 Apr 24, 2006 8:06:27 AM org.apache.jk.common.ChannelSocket init INFO: JK: ajp13 listening on /0.0.0.0:8009 Apr 24, 2006 8:06:27 AM org.apache.jk.server.JkMain start INFO: Jk running ID=0 time=0/21 config=null Apr 24, 2006 8:06:27 AM org.apache.catalina.storeconfig.StoreLoader load INFO: Find registry server-registry.xml at classpath resource Apr 24, 2006 8:06:27 AM org.apache.catalina.startup.Catalina start INFO: Server startup in 5213 ms