Hi, Today the problem happened again, but this time we were monitoring the logs:
13/12/2012 15:49:46 org.apache.catalina.core.ApplicationContext log GRAVE: java.sql.SQLException java.lang.RuntimeException: java.sql.SQLException at com.sun.data.provider.impl.CachedRowSetDataProvider.setCursorRow(CachedRowSetDataProvider.java:416) at com.sun.data.provider.impl.CachedRowSetDataProvider.setCursorIndex(CachedRowSetDataProvider.java:373) at com.sun.data.provider.impl.CachedRowSetDataProvider.getRowCount(CachedRowSetDataProvider.java:713) at com.sun.webui.jsf.faces.DataProviderELResolver$SelectItemsData.getSelectItems(DataProviderELResolver.java:702) at com.sun.webui.jsf.faces.DataProviderELResolver.getValue(DataProviderELResolver.java:166) at javax.el.CompositeELResolver.getValue(CompositeELResolver.java:54) at com.sun.faces.el.FacesCompositeELResolver.getValue(FacesCompositeELResolver.java:64) at org.apache.el.parser.AstValue.getValue(AstValue.java:123) at org.apache.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:186) at org.apache.jasper.el.JspValueExpression.getValue(JspValueExpression.java:101) at com.sun.webui.jsf.component.Selector.getItems(Selector.java:577) at com.sun.webui.jsf.component.ListSelector.getOptions(ListSelector.java:212) at com.sun.webui.jsf.component.ListSelector.getListItems(ListSelector.java:140) at com.sun.webui.jsf.renderkit.widget.ListRendererBase.getListProperties(ListRendererBase.java:269) at com.sun.webui.jsf.renderkit.widget.ListRendererBase.getProperties(ListRendererBase.java:220) at com.sun.webui.jsf.renderkit.widget.DropDownRenderer.getProperties(DropDownRenderer.java:74) at com.sun.webui.jsf.renderkit.widget.RendererBase.encodeChildren(RendererBase.java:134) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:812) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:886) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:892) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:892) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:892) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:892) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:892) at com.sun.faces.application.ViewHandlerImpl.doRenderView(ViewHandlerImpl.java:245) at com.sun.faces.application.ViewHandlerImpl.renderView(ViewHandlerImpl.java:176) at com.sun.rave.web.ui.appbase.faces.ViewHandlerImpl.renderView(ViewHandlerImpl.java:320) at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:106) at com.sun.faces.lifecycle.LifecycleImpl.phase(LifecycleImpl.java:251) at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:144) at com.sun.faces.extensions.avatar.lifecycle.PartialTraversalLifecycle.render(PartialTraversalLifecycle.java:106) at javax.faces.webapp.FacesServlet.service(FacesServlet.java:245) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.sun.webui.jsf.util.UploadFilter.doFilter(UploadFilter.java:267) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at com.googlecode.psiprobe.Tomcat60AgentValve.invoke(Tomcat60AgentValve.java:30) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:429) at org.apache.coyote.ajp.AjpAprProtocol$AjpConnectionHandler.process(AjpAprProtocol.java:384) at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1665) at java.lang.Thread.run(Thread.java:662) Caused by: java.sql.SQLException at com.sun.sql.rowset.internal.CachedRowSetXReader.connect(CachedRowSetXReader.java:381) at com.sun.sql.rowset.internal.CachedRowSetXReader.readData(CachedRowSetXReader.java:160) at com.sun.sql.rowset.CachedRowSetXImpl5.execute(CachedRowSetXImpl5.java:865) at com.sun.sql.rowset.CachedRowSetXImpl5.execute(CachedRowSetXImpl5.java:1325) at com.sun.data.provider.impl.CachedRowSetDataProvider.checkExecute(CachedRowSetDataProvider.java:1351) at com.sun.data.provider.impl.CachedRowSetDataProvider.setCursorRow(CachedRowSetDataProvider.java:402) ... 47 more Caused by: javax.naming.NoInitialContextException: Cannot instantiate class: com.sun.jndi.fscontext.RefFSContextFactory [Root exception is java.lang.ClassNotFoundException: com.sun.jndi.fscontext.RefFSContextFactory] at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:657) at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:288) at javax.naming.InitialContext.init(InitialContext.java:223) at javax.naming.InitialContext.<init>(InitialContext.java:175) at com.sun.sql.rowset.internal.CachedRowSetXReader.connect(CachedRowSetXReader.java:365) ... 52 more Caused by: java.lang.ClassNotFoundException: com.sun.jndi.fscontext.RefFSContextFactory at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1680) at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1526) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:247) at com.sun.naming.internal.VersionHelper12.loadClass(VersionHelper12.java:46) at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:654) ... 56 more 13/12/2012 15:49:46 org.apache.catalina.core.StandardWrapperValve invoke GRAVE: Servlet.service() for servlet Faces Servlet threw exception java.lang.ClassNotFoundException: com.sun.jndi.fscontext.RefFSContextFactory at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1680) at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1526) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:247) at com.sun.naming.internal.VersionHelper12.loadClass(VersionHelper12.java:46) at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:654) at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:288) at javax.naming.InitialContext.init(InitialContext.java:223) at javax.naming.InitialContext.<init>(InitialContext.java:175) at com.sun.sql.rowset.internal.CachedRowSetXReader.connect(CachedRowSetXReader.java:365) at com.sun.sql.rowset.internal.CachedRowSetXReader.readData(CachedRowSetXReader.java:160) at com.sun.sql.rowset.CachedRowSetXImpl5.execute(CachedRowSetXImpl5.java:865) at com.sun.sql.rowset.CachedRowSetXImpl5.execute(CachedRowSetXImpl5.java:1325) at com.sun.data.provider.impl.CachedRowSetDataProvider.checkExecute(CachedRowSetDataProvider.java:1351) at com.sun.data.provider.impl.CachedRowSetDataProvider.setCursorRow(CachedRowSetDataProvider.java:402) at com.sun.data.provider.impl.CachedRowSetDataProvider.setCursorIndex(CachedRowSetDataProvider.java:373) at com.sun.data.provider.impl.CachedRowSetDataProvider.getRowCount(CachedRowSetDataProvider.java:713) at com.sun.webui.jsf.faces.DataProviderELResolver$SelectItemsData.getSelectItems(DataProviderELResolver.java:702) at com.sun.webui.jsf.faces.DataProviderELResolver.getValue(DataProviderELResolver.java:166) at javax.el.CompositeELResolver.getValue(CompositeELResolver.java:54) at com.sun.faces.el.FacesCompositeELResolver.getValue(FacesCompositeELResolver.java:64) at org.apache.el.parser.AstValue.getValue(AstValue.java:123) at org.apache.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:186) at org.apache.jasper.el.JspValueExpression.getValue(JspValueExpression.java:101) at com.sun.webui.jsf.component.Selector.getItems(Selector.java:577) at com.sun.webui.jsf.component.ListSelector.getOptions(ListSelector.java:212) at com.sun.webui.jsf.component.ListSelector.getListItems(ListSelector.java:140) at com.sun.webui.jsf.renderkit.widget.ListRendererBase.getListProperties(ListRendererBase.java:269) at com.sun.webui.jsf.renderkit.widget.ListRendererBase.getProperties(ListRendererBase.java:220) at com.sun.webui.jsf.renderkit.widget.DropDownRenderer.getProperties(DropDownRenderer.java:74) at com.sun.webui.jsf.renderkit.widget.RendererBase.encodeChildren(RendererBase.java:134) at javax.faces.component.UIComponentBase.encodeChildren(UIComponentBase.java:812) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:886) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:892) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:892) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:892) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:892) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:892) at com.sun.faces.application.ViewHandlerImpl.doRenderView(ViewHandlerImpl.java:245) at com.sun.faces.application.ViewHandlerImpl.renderView(ViewHandlerImpl.java:176) at com.sun.rave.web.ui.appbase.faces.ViewHandlerImpl.renderView(ViewHandlerImpl.java:320) at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:106) at com.sun.faces.lifecycle.LifecycleImpl.phase(LifecycleImpl.java:251) at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:144) at com.sun.faces.extensions.avatar.lifecycle.PartialTraversalLifecycle.render(PartialTraversalLifecycle.java:106) at javax.faces.webapp.FacesServlet.service(FacesServlet.java:245) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.sun.webui.jsf.util.UploadFilter.doFilter(UploadFilter.java:267) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at com.googlecode.psiprobe.Tomcat60AgentValve.invoke(Tomcat60AgentValve.java:30) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:429) at org.apache.coyote.ajp.AjpAprProtocol$AjpConnectionHandler.process(AjpAprProtocol.java:384) at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1665) at java.lang.Thread.run(Thread.java:662) Any idea? Best regards, Robert On Wed, Dec 5, 2012 at 8:14 AM, Robert Anderson <ranom...@gmail.com> wrote: > Hi Konstantin, > > 1. This NullPointerException happens when an application starts up? > > (Do the apps perform the lookup once and cache the result, or they > perform multiple lookups?) > > No, it happens after application starts up. The applications perform > multiple lookups. > > 2. What is seen in JNDI context depends on the current classloader > (Thread.getContextClassLoader()). > > Does the issue happen in a request processing thread, or somewhere else? > > Request processing thread. > > 3. Does it affect specific web applications, or it is more random? > > When it happens, it affects all datasources (globals and applications > specific) of all applications. > > 4. Anything interesting in the logs and in catalina.out? E.g. > OutOfMemoryError. > > No, but we're still looking for some lead. > > 5. What connector implementations are you using? Bio, Nio, APR? > > APR. > > The last time that this issue happened was on 11/22. > > Thanks, > > Robert > > > On Tue, Dec 4, 2012 at 6:01 PM, Konstantin Kolinko <knst.koli...@gmail.com > > wrote: > >> 2012/11/28 Robert Anderson <ranom...@gmail.com>: >> > Hi, >> > >> > >> > We've some data sources defined in server.xml as following: >> > >> > ... >> > <GlobalNamingResources> >> > ... >> > <Resource name="jdbc/proddb1" auth="Container" >> > type="javax.sql.DataSource" removeAbandoned="true" >> > removeAbandonedTimeout="300" >> > maxActive="400" maxIdle="30" >> > maxWait="10000" >> > validationQuery="select 1 from dual" >> > testOnBorrow="true" >> > username="xxxx" password="xxxx" >> > driverClassName="com.intersys.jdbc.CacheDriver" >> > url="jdbc:Cache://server:1972/DB"/> >> > ... >> > >> > </GlobalNamingResources> >> > ... >> > >> > conf/context.xml >> > >> > <Context> >> > ... >> > <ResourceLink name="jdbc/proddb1" global="jdbc/proddb1"/> >> > ... >> > </Context> >> > >> > >> > Everything has worked normal during many months...until now. >> > >> > At least once a day, since 11/21, webapplications throw >> > NullPointerException because they cannot find jndi data sources. The >> Data >> > sources tab in psi-probe (http://code.google.com/p/psi-probe/) says >> that >> > there aren't jndi data sources in server. We are thinking that some >> > application may have added some jar or class in classpath that is >> causing >> > the problem. >> > >> > After restart, without changes in conf files, everything backs to >> normal. >> > >> > Any idea? >> > >> > Environment: >> > >> > java version "1.6.0_35" >> > Java(TM) SE Runtime Environment (build 1.6.0_35-b10) >> > Java HotSpot(TM) 64-Bit Server VM (build 20.10-b01, mixed mode) >> > >> > Server version: Apache Tomcat/6.0.32 >> > Server built: February 2 2011 2003 >> > Server number: 6.0.32.0 >> > OS Name: Linux >> > OS Version: 2.6.18-194.17.1.el5 >> > Architecture: amd64 >> > JVM Version: 1.6.0_35-b10 >> > JVM Vendor: Sun Microsystems Inc. >> > >> >> 1. This NullPointerException happens when an application starts up? >> >> (Do the apps perform the lookup once and cache the result, or they >> perform multiple lookups?) >> >> 2. What is seen in JNDI context depends on the current classloader >> (Thread.getContextClassLoader()). >> >> Does the issue happen in a request processing thread, or somewhere else? >> >> 3. Does it affect specific web applications, or it is more random? >> >> 4. Anything interesting in the logs and in catalina.out? E.g. >> OutOfMemoryError. >> >> 5. What connector implementations are you using? Bio, Nio, APR? >> >> Best regards, >> Konstantin Kolinko >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >> >