We have a web application that was consistently getting a VerifyError
whenever a jsp invoked a custom tagx. The jsp is a SiteMesh decorator that
uses a custom tagx to display a Spring Security Authentication object. The
issue was resolved by restarting the war using the Tomcat Manager, but I am
concerned the issue may happen again. Does anyone know what might have
caused this to happen and/or how to prevent it from happening again? I have
included a full stacktrace [1] and details about our environment [2] below.
Any feedback would be greatly appreciated.
Additional Information:
1) Since restarting the war the VerifyError cannot be reproduced. Note that
only the war was restarted; the Tomcat instance was NOT restarted.
2) I have tried searching for answers on the forums/Internet. Most of the
results I have seen stated that their problem was that the wrong version of
Java was used or some sort of byte-code manipulation was being done.
a. Since the problem was fixed by cycling the war (and not the JVM) I doubt
using the wrong JVM was our problem.
b. We are using spring-aop, but only with interface based proxies (not
aspjectj compilation). Additionally I do not think this should impact the
compiled jsp's byte code validity.
c. We use hibernate which is using javasist to create proxies of our domain
objects at load time, but again I do not think this should impact the
compiled jsp's byte code validity.
3) I have looked for any jars included in the war that might contain the
wrong JspTag or PageContext. I tried to do an open type in Eclipse on both
classes and found jsp-api and servlet-api both contain these classes.
However, these jars are provided maven dependencies. I also validated that
they were not packaged in the war's WEB-INF/lib/ directory.
4) The code was compiled and is ran using a Sun 1.6 JDK
5) Unfortunately at the time I did not think to save the generated java or
class for the JSP page or the tag lib.
6) The last modified date on the jsp and the jsp tag java/class files in
Tomcat's work directory both have a time stamp that is much (over two
weeks) older than when the war was restarted to resolve the issue. This
seems to imply that neither were recompiled.
7) I have included a full stack trace of the error [1], and details about
the environment [2] below.
[1]
java.lang.VerifyError: (class:
org/apache/jsp/WEB_002dINF/decorators/main_jsp, method:
_jspx_meth_tags_005fusername_005f0 signature:
(Ljavax/servlet/jsp/tagext/JspTag;Ljavax/servlet/jsp/PageContext;)Z)
Illegal constant pool index
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Unknown Source)
at java.lang.Class.getConstructor0(Unknown Source)
at java.lang.Class.newInstance0(Unknown Source)
at java.lang.Class.newInstance(Unknown Source)
at
org.apache.catalina.core.DefaultInstanceManager.newInstance(DefaultInstanceManager.java:125)
at
org.apache.jasper.servlet.JspServletWrapper.getServlet(JspServletWrapper.java:162)
at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:356)
at
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:389)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:333)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:59)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:684)
at
org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:593)
at
org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:530)
at
com.opensymphony.sitemesh.compatability.OldDecorator2NewDecorator.render(OldDecorator2NewDecorator.java:46)
at
com.opensymphony.sitemesh.webapp.decorator.BaseWebAppDecorator.render(BaseWebAppDecorator.java:33)
at
com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:84)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.springframework.web.filter.ShallowEtagHeaderFilter.doFilterInternal(ShallowEtagHeaderFilter.java:58)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:369)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:109)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381)
at
com.cerner.cloud.idm.springsecurity.web.access.CsrfFilter.doFilterInternal(CsrfFilter.java:45)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:97)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381)
at
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:100)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381)
at
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:78)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381)
at
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381)
at
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:35)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381)
at
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:187)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381)
at
com.cerner.cloud.idm.system.netoauth.jaas.j2ee.filter.OAuthValidatorFilter.doFilter(OAuthValidatorFilter.java:186)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381)
at
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381)
at
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:79)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381)
at
com.cerner.cloud.idm.system.web.filter.UserPreferencesFilter.doFilterInternal(UserPreferencesFilter.java:37)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381)
at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:168)
at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:198)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:164)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:462)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:563)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:403)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:301)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:162)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:140)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:309)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
[2] Environment Information:
Tomcat 7.0.16
JVM Version: 1.6.0_27-b07
JVM Vendor: Sun Microsystems Inc.
Red Hat Enterprise Linux Server release 5.5 (Tikanga)
Linux <host> 2.6.18-194.32.1.el5 #1 SMP Mon Dec 20 10:52:42 EST 2010 x86_64
x86_64 x86_64 GNU/Linux
Regards,
Rob Winch
