On Feb 22, 2013, at 12:09 AM, Kirill Ilyukhin wrote: > Hi! > > I have 5 servers with the following configuration: > Windows Server 2008 R2 64-bit, > Oracle JDK 1.7.0_11-b21 64-bit, > Tomcat 7.0.35 running as a service. > A few webapps are running perfectly well under the Tomcat until one of > them is reloaded. > > Reload of a webapp (either by pressing [reload] on manager page or by > changing web.xml file) makes Tomcat extremely unstable. > The first (pre-production) server sporadically fails with absolutely > no traces in log files or Windows event logs:
These three symptoms can often mean different things. For example, if it's not accepting connections then you can't really get a 404 error. Can you be more specific about when you see them and the status of your Tomcat server when you see them. Some troubleshooting thoughts... > does not accept connections, 1.) Is the process still running? Did the JVM crash? Is there a crash file? hs_err_podXXXX.log? 2.) Is the socket still listening? Have you changed any firewall rules? 3.) Try "telnet <port>". What happens? 4.) Try connecting to Tomcat with jconsole or jvisualvm? Can you? If so look at the connector mbean. What is its state? 5.) Take some thread dumps. > responds with HTTP 404 or Assuming the URL that you entered is valid, this sounds like your application may have failed to reload. Can you turn up the log level in your application to see what it is doing? > does not respond at all. Assuming that you mean it accepts your connection and request, but never processes and returns a response. In this case, take some thread dumps after you have sent the response. That should show you what is going on inside the JVM. > Other 4 (production) servers also become unstable but in a different > manner - a lot of HTTP 500 responses (logs are also clean). It would be very odd to see nothing in the logs after a 500 error. Have you looked at all of the log files, in particular "localhost-YYYY-MM-DD.log"? That file should contain a stack trace after a 500 error. Also, what do the see in your browser when you get the 500 error? The default error page should print the stack trace for the 500 error. > Plus on > some of these servers I have a lot (less than HTTP 500 responses > though) of NPEs: > ------------- > SEVERE: Exception Processing /path/here > java.lang.NullPointerException > at > org.apache.catalina.core.StandardWrapper.servletSecurityAnnotationScan(StandardWrapper.java:1216) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:461) > at > org.apache.catalina.valves.RequestFilterValve.process(RequestFilterValve.java:305) > at > org.apache.catalina.valves.RemoteAddrValve.invoke(RemoteAddrValve.java:83) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) > at > org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1822) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:722) > ------------- > These exception occurs for any webapp, not only the one being reloaded. What's the specific version of Tomcat where you saw this error occur? Is it 7.0.37? or was this on an older version? > > All the above happens irregularly, quite often on the pre-production > server, only once on production servers. Tomcat service restart fixes > the problem. Updating Tomcat to 7.0.37 did not help. > I cannot reproduce this on any of developers hosts with the same > versions of JDK and Tomcat. > > Any ideas? 1.) An easy temporary fix would be to not reload the application in production. If you need to redeploy or reload the application, stop the server, undeploy, redeploy and start your server. 2.) Include your Tomcat configuration, minus comments. 3.) See troubleshooting steps above. Dan > > > Thanks a lot in advance, > Kirill > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
