Re: Help in diagnosing server unresponsiveness
On 01/02/2013 20:08, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Zoran, On 1/31/13 8:36 PM, Zoran Avtarovski wrote: We have a application running on the latest Tomcat7 and we are getting a server crash or becoming unresponsive. This occur every few days at no fixed intervals or time of day and they certainly don't correlate to any app function at least not according to the logs. Can you describe the "crash" in more detail? OOME? IF so, what kind (heap or PermGen)? Lock-up (deadlock, etc)? Actual JVM crash (produces a core dump or native stack dump)? I would go in that direction too. Enable logs and core or stack dumps and analyze them. Be sure you are not restarting Tomcat in your crontab (i had a server which was restarted once a week and masked some memory starvation). In my case I can tell you I had to end up disabling JaveMelody (it was provoking some side effects in our webapp as not managing international chars in the right way). If reports from Javamelody are not giving you any clue, beware that Javamelody has its own memory overhead (not much in your case but in my case it was around 200 Mb of heap in a 1 Gb virtual server). I followed Chris directions, I got stack dumps after a server crash and analyzed it with eclipse analyzer. I realized our programmer decided to load too many objects in memory than the server could cope with. So no memory leak but bad memory management was the root cause. Regards, Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 7.x and Internet Explorer Adobe Reader plugin
We are having what sounds like a similar problem (although 7.0.26 works for us) and can provide our details. We are using Solaris 10, Tomcat 7.0.26, Apache/2.2.16, mod_jk/1.2.35 and Java(TM) SE Runtime Environment (build 1.6.0_30-b12) in our production environment. We are using the same except for the latest Tomcat 7.0.29 in our test environment. Recently one of our testers found that when using IE8 against our test environment she could not download PDFs (available as links on web pages) and would instead see the error "A network error occurred while accessing this document on the Internet" but it worked fine on the older Tomcat in production. Indeed if I roll our testing environment back to Tomcat 7.0.26, this problem does not occur. It also doesn't occur in any other browser we tried (IE9, FireFox 14.0.1 and Safari 5.1.7). I had remembered that something similar had been discussed on this list, but the archived thread looks like it went stale when it couldn't be reproduced: PDF Download problem tomcat >= 7.0.27 (http://mail-archives.apache.org/mod_mbox/tomcat-users/201207.mbox/%3CCAC=HunuF5zqKf5s_D9syTZ1J2nFXdCjEWKj=zq+hc8bgnn1...@mail.gmail.com%3E) A good portion of our visitors use IE8 with the 9.5.1 Adobe Reader and we have a number of PDF documents available for download so I'm hoping perhaps our different configuration to the ones previously reported might eventually lead to a fix. As I have answered to Chris, it came down that the downgrade to 7.0.26 fixed it. Only a person reports problems that with a particular PDF file so we have assumed it's an issue with his setup. I couldn't reproduce the issue as I did with 7.0.29, so it seems there is something wrong with that version. Thanks! Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 7.x and Internet Explorer Adobe Reader plugin
On 22/08/2012 18:06, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miguel, On 8/21/12 10:24 AM, Miguel Gonzalez wrote: Our webapp worked fine under Tomcat 5.5. We decided to migrate to Tomcat 7 to get better performance and get support from the community in case something goes wrong as suggested in this mailing list. I installed tomcat 7.0.29 from the Apache website. I had issues downloading PDFs from IE, but not from Firefox or Chrome. I googled a little bit and some people suggested to downgrade to 7.0.26. But still having issues with it under IE while it works under Firefox and Chrome. Any ideas? http://www.catb.org/esr/faqs/smart-questions.html Your question is "I had an app that was working and then I upgraded and it doesn't work. What might be the problem?". Not exactly, I said problems downloading PDFs in IE. It reported an error when tried to download like it couldn't finish off the downloading (it takes forever). Well, the problem might be that the power isn't connected to your server. But you haven't provided any information that suggests what "had issues downloading PDFs from IE" means. What issues? What did you try? What happened when you did that? What have you attempted to do so far to "fix" the "issue" and what have you discovered? In the end downgrading to 7.0.26 has solved the issue. The person that reported that he had also issues with 7.0.26 and IE8 recognized it might be his install since it happened just with a particular file, not all PDF files. I couldn't reproduced the issue with IE8 Thanks, Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: redirecting people to maintenance mode
On 21/08/2012 17:00, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miguel, On 8/21/12 6:25 AM, Miguel González Castaños wrote: Dear all, I have a Tomcat web server. From time to time, I need to do some maintenance and want people not to interact with the Tomcat server while I'm doing it. The key thing here is that the Tomcat server is up and running. This is what I want to achieve: - I want all people to be redirected from 80/443 port at server1 to a different URL http://server2/maintenance.html except my IP address (so I can check Tomcat myself). - If possible, I want this to be transparent to the user, so they get a http redirect showing the server1 in the client's browser, not server2 (but this is not that important). I assume I can use iptables to redirect people to a different web server, but how can I know that I need to redirect them to the maintenance.html if that server is serving other web pages too? I don't know if you want to add this kind of complication to your environment, but this is pretty much what mod_jk was built for: distributing requests to one or more back-end Tomcat servers. With mod_jk's management interface, you can take (load-balanced) workers out of a pool, upgrade them, then add them back to the pool. Before you add them back, you can always access them directly if you have set up an HTTP and you can feel free to use any technique (iptables, bind to 127.0.0.1, etc.) to prevent direct access to the HTTP port from normal users. The use of Apache httpd out in front of your app server(s) also means that you can detect error conditions in the Tomcat instance (like it's crashed or whatever) and give a semi-nice error message instead of a connection failure. This is one of those situations where using Apache httpd out in front of Tomcat makes some sense: not for performance, but for availability. See Mladen's article with explains some of the why, what and how of setting up a mod_jk-based cluster: http://people.apache.org/~mturk/docs/article/ftwai.html It makes sense. I will give it a thought. Unfortunately until we migrate to a bigger server, I'm not sure the overhead of Apache is too much. Although I know Apache is faster for static content. Many thanks for your thoughts Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: redirecting people to maintenance mode
On 20/08/2012 13:50, André Warnier wrote: Miguel González Castaños wrote: Dear all, I have a Tomcat web server. Apparently, you also have an email program that sends messages dated from the future. Or, maybe you should say then : I will have a Tomcat web server. Either way, it makes it a bit harder to follow the gist of the conversation here. Sorry, for any reason the date was changed in my system inadvertently Corrected. Miguel This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: redirecting people to maintenance mode
On 20/08/2012 12:37, Jose María Zaragoza wrote: I assume I can use iptables to redirect people to a different web server, but how can I know that I need to redirect them to the maintenance.html if that server is serving other web pages too? You can try Valve filters . With Valve, you have access to Request and Response objects I didn't do ever , but it's an idea :-) maybe I haven't worded my idea correctly. I can't assume the Tomcat service in server1 is running all the time (since it's under maintenance, tomcat might be restarted a few times during the maintenance). That's why I thought about iptables. The issue is that iptables run at the IP level while redirects work at application level. Miguel This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
redirecting people to maintenance mode
Dear all, I have a Tomcat web server. From time to time, I need to do some maintenance and want people not to interact with the Tomcat server while I'm doing it. The key thing here is that the Tomcat server is up and running. This is what I want to achieve: - I want all people to be redirected from 80/443 port at server1 to a different URL http://server2/maintenance.html except my IP address (so I can check Tomcat myself). - If possible, I want this to be transparent to the user, so they get a http redirect showing the server1 in the client's browser, not server2 (but this is not that important). I assume I can use iptables to redirect people to a different web server, but how can I know that I need to redirect them to the maintenance.html if that server is serving other web pages too? Regards, Miguel This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: heap memory issues
On 19/06/2012 02:47, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miguel, On 6/18/12 6:34 PM, Miguel González Castaños wrote: [H]ow do I know how much ram to assign in a 64 bits jdk? We are planning to move to a bigger server (hopefully 16 or 32 gb). Nobody can tell you how big your heap needs to be: it all depends upon the needs of your app and its users. We ran for 5 years with a 64 MiB heap and never had any problems. We started getting OOMEs because we started supporting more users and their (small) sessions were piling up and busting our heap, so we had to increase the heap size. Now that you have your email issue handled, you might even decide that "bigger" hardware isn't necessary. You are completely right. I won't move to a bigger server until I make sure the webapp is stable and we can measure the load it can handle. I just wanted to know how I could measure how much heap size I need. Your approach seems reasonable to me Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: heap memory issues
On 19/06/2012 06:04, Terence M. Bandoian wrote: On 1:59 PM, Miguel González Castaños wrote: I know, but how do I know how much ram to assign in a 64 bits jdk? We are planning to move to a bigger server (hopefully 16 or 32 gb). Miguel 16 or 32 GB? Is this a medical image processing system? A control system for a space flight? Sorry for the sarcasm but that just seems like an enormous amount of memory. It depends on how many concurrent users you have. If you have plans of having more than 1000 people accessing at the same time your server you have to be ready and I'm not talking about short sessions, but long sessions and intense database access. Anyway, before we move to a bigger server, we need to get a stable webapp and check which load can handle the current server. Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: heap memory issues
On 18/06/2012 09:52, Konstantin Kolinko wrote: 2012/6/16 Miguel González Castaños: Dear all, We have a webapp under Tomcat 5.5 and jdk 1.5 32 bits under a virtual machine of 4 Gb and a 64 bits OS. 4 Gb is memory size? Yes Some of you have helped me suggesting me to install Javamelody and sar to monitor the system. Setting up heap dumps I have managed to see what was going on with the eclipse memory analyzer. The webapp sends emails that have been previously entered into the database. It loads the emails into memory, so if you are sending 10k emails, it loads 10k objects containing those emails. That makes the app to use around 200-300 mb of the heap memory. Do you need to keep 10k of them in the memory at the same time? Usually one would send mails one by one and release their memory immediately. I don't. Fortunately the developer has corrected this, so probably other issues will arise but not this one. The settings we have now for the JDK are: CATALINA_OPTS="$CATALINA_OPTS -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/tomcat5/logs/ -Xms128m -Xmx512m -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails" Facts that I have witnessed: If I try to increase the Xmx to 740m or 1024m the GC starts working very much and Tomcat shuts down. GC? Or OS memory swapping? GC, no swapping reported by sar. I have also tried downloading a 64 bits 1.7 jdk but I get the typical: Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError occurred dispatching signal SIGTERM to handler- the VM may need to be forcibly terminated 64-bit JVM usually needs more memory than 32-bit one, because memory address range is wider and pointers need more memory. I know, but how do I know how much ram to assign in a 64 bits jdk? We are planning to move to a bigger server (hopefully 16 or 32 gb). Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: heap memory issues
On 18/06/2012 02:45, Perry, Andrew wrote: Try setting -XX:MaxPermSize=256m it helps with our applications, it might do the trick here. What is the effect of this? Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: heap memory issues
On 16/06/2012 12:49, Miguel González Castaños wrote: Dear all, We have a webapp under Tomcat 5.5 and jdk 1.5 32 bits under a virtual machine of 4 Gb and a 64 bits OS. Some of you have helped me suggesting me to install Javamelody and sar to monitor the system. Setting up heap dumps I have managed to see what was going on with the eclipse memory analyzer. The webapp sends emails that have been previously entered into the database. It loads the emails into memory, so if you are sending 10k emails, it loads 10k objects containing those emails. That makes the app to use around 200-300 mb of the heap memory. The settings we have now for the JDK are: CATALINA_OPTS="$CATALINA_OPTS -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/tomcat5/logs/ -Xms128m -Xmx512m -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails" Facts that I have witnessed: If I try to increase the Xmx to 740m or 1024m the GC starts working very much and Tomcat shuts down. I have also tried downloading a 64 bits 1.7 jdk but I get the typical: Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError occurred dispatching signal SIGTERM to handler- the VM may need to be forcibly terminated It doesn't matter if i set the maxperm memory to 512m or I increase the Xms and Xmx beyond 1024m, it breaks after 5-10 minutes without doing anything. Also from the heap dumps I have realized that javamelody objects are consuming around 100Mb, so I have undeployed it to get at least more heap memory. Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
heap memory issues
Dear all, We have a webapp under Tomcat 5.5 and jdk 1.5 32 bits under a virtual machine of 4 Gb and a 64 bits OS. Some of you have helped me suggesting me to install Javamelody and sar to monitor the system. Setting up heap dumps I have managed to see what was going on with the eclipse memory analyzer. The webapp sends emails that have been previously entered into the database. It loads the emails into memory, so if you are sending 10k emails, it loads 10k objects containing those emails. That makes the app to use around 200-300 mb of the heap memory. The settings we have now for the JDK are: CATALINA_OPTS="$CATALINA_OPTS -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/tomcat5/logs/ -Xms128m -Xmx512m -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails" Facts that I have witnessed: If I try to increase the Xmx to 740m or 1024m the GC starts working very much and Tomcat shuts down. I have also tried downloading a 64 bits 1.7 jdk but I get the typical: Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError occurred dispatching signal SIGTERM to handler- the VM may need to be forcibly terminated It doesn't matter if i set the maxperm memory to 512m or I increase the Xms and Xmx beyond 1024m, it breaks after 5-10 minutes without doing anything. Regards, Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
wget and Tomcat resources
Dear all, Our developer has set a cronjob similar to this: wget -T 0 http://ourserver.com/email_sender which calls a javabean to check pending emails to send in a database and actually send them. I'm concerned about this, since I have realized that we have peaks of 500 http connections per minute exactly about the same time this cronjob runs. I'm just wondering if this cronjob is performing several retries (since it sets a 0 timeout) and count as http hits. This increase of http hits (not related to people connecting since people connect in office hours and this happens early morning) is happening together with an increase of use of memory. Regards, Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat pausing and no java process
On 07/05/2012 23:13, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miguel, On 5/5/12 5:41 AM, Miguel González Castaños wrote: Yes, I have pinpointed several queries that took quite long and specially subqueries. I forwarded this info to the developer but she said it was alright. More than a second to run a query seems to be a lot to me. That depends on what the query does. If it checks permissions to login, that's insane and you should get a new programmer. If it's a reporting query that gathers lots of data then 1 second might not be quite so far-fetched. I have checked directly the slow query log to pinpoint the real queries. I found many subqueries, which I believe is not a good practice, am I right? There's nothing wrong with subqueries in general, it's just that the MySQL query optimizer appears to be particularly stupid when it comes to them, and you often end up with *DEPENDENT* subqueries which essentially issue N queries instead of only 2 queries (if the subquery for instance were to be independent, it could be evaluated first and then essentially substituted in the outer query). Only EXPLAIN can tell you what will happen with each query. Remember to use live data with EXPLAIN because the query optimizer is very sensitive to the data it will be scanning... if you have a test table with 5 records in it, you're going to get worthless EXPLAIN results. Sorry I didn't reply to your email, but somehow it got lost in my inbox. Well, in the end my client managed to get the developer onboard again and to rewrite the subqueries since they were not using any index at all. Let's see how that goes. Thread dumps can be very helpful as well. You might also want to look at running 'sar' all the time to get CPU/IO/etc sampling to see if you have any other kind of resource shortcoming. I have just installed sar yesterday and I'm downloading the reports through ksar remotely through SSH. Nice tool BTW. Tomorrow we are probably getting many clients connecting to the webapp and we can get some conclusions. I have just realized the server is running a 32 bits 1.5 JDK and the OS is a 64 bits linux machine. I'm wondering if using a JVM 1.7 for 64 bits could give us more room to have more people connected simultaneously. Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: isHexDigit error problems and upgrading Tomcat and jdk
That depends upon how you have configured your access log: you can configure an access log to log only requests that failed and the details of that request like this: You will probably want to put "%t" and "%r" in your pattern. This looks interesting. In my test environment (with Tomcat 7 and java 1.7) it shows by default directory="logs" prefix="localhost_access_log." suffix=".txt" pattern="%h %l %u %t "%r" %s %b" /> this should keep track of the errors just adding just the entry: conditionIf="org.apache.catalina.parameter_parse_failed" ? Unfortunately, this is a POST that is being processed and it's hard to use the RequestDumperFilter to dump the requests of only certain requests that might be the only way to get the offending parameter name or value that is causing this problem. Many thanks for your hints :) Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: isHexDigit error problems and upgrading Tomcat and jdk
On 07/06/2012 00:54, Konstantin Kolinko wrote: 2012/6/7 Miguel González Castaños: Hi, We are getting isHexDigit errors again, although there is no malformed URL requests It is parameter parsing code. It cannot claim the request as malformed (API does not allow it). It can only skip malformed parameters. Does that mean that it won't show up in the access log? It is a problem with a parameter used by the parsing code? How can I log any info that could show where is the problem in the code or when is this happening? It's making Tomcat to pause and then to stop. It sets some error flag that can be examined. FailedRequestFilter is example of a filter that checks that flag. I understand that function has to be added in the parsing code, isn't it? We are using Tomcat 5.5 and jdk 1.5 (from Sun). As some of you have suggested me in the past, I'm considering to upgrade to a more up-to-date Tomcat and/or jdk so I can get more support and help from the community since probably the stability issues are solved in newer versions. I have set up a test environment and Tomcat 7.0.27 and jdk 1.7 seems to work fine. Do you suggest me to upgrade to Tomcat 6 or 7? What about jdk? 1.6 or 1.7? 1.6 is more widely tested (many years), but for a new system I would go with 1.7. It's not a new system, it's been running for 3 years already. I don't want to risk that any library has changed its behavior with tomcat 7 or something similar. I heard that Oracle plans to stop provide free updates for 1.6 this summer. So they will be available only as "Java for Business". Does it mean that it is probably better to go for 1.7 instead? Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
isHexDigit error problems and upgrading Tomcat and jdk
Hi, We are getting isHexDigit errors again, although there is no malformed URL requests We are using Tomcat 5.5 and jdk 1.5 (from Sun). As some of you have suggested me in the past, I'm considering to upgrade to a more up-to-date Tomcat and/or jdk so I can get more support and help from the community since probably the stability issues are solved in newer versions. I have set up a test environment and Tomcat 7.0.27 and jdk 1.7 seems to work fine. Do you suggest me to upgrade to Tomcat 6 or 7? What about jdk? 1.6 or 1.7? Thanks Miguel - Jun 6, 2012 9:05:59 AM org.apache.tomcat.util.http.Parameters processParameters WARNING: Parameters: Character decoding failed. Parameter skipped. java.io.CharConversionException: isHexDigit at org.apache.tomcat.util.buf.UDecoder.convert(UDecoder.java:87) at org.apache.tomcat.util.buf.UDecoder.convert(UDecoder.java:48) at org.apache.tomcat.util.http.Parameters.urlDecode(Parameters.java:411) at org.apache.tomcat.util.http.Parameters.processParameters(Parameters.java:393) at org.apache.tomcat.util.http.Parameters.processParameters(Parameters.java:344) at org.apache.catalina.connector.Request.parseParameters(Request.java:2401) at org.apache.catalina.connector.Request.getParameterNames(Request.java:1047) at org.apache.catalina.connector.RequestFacade.getParameterNames(RequestFacade.java:369) at javax.servlet.ServletRequestWrapper.getParameterNames(ServletRequestWrapper.java:178) at org.apache.struts.util.RequestUtils.populate(RequestUtils.java:1225) at org.apache.struts.action.RequestProcessor.processPopulate(RequestProcessor.java:821) at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:254) at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1482) at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:525) at javax.servlet.http.HttpServlet.service(HttpServlet.java:647) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:185) at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:159) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689) at java.lang.Thread.run(Thread.java:595) Jun 6, 2012 10:15:22 AM org.apache.coyote.http11.Http11BaseProtocol pause INFO: Pausing Coyote HTTP/1.1 on http-80 Jun 6, 2012 10:15:22 AM org.apache.coyote.http11.Http11BaseProtocol pause INFO: Pausing Coyote HTTP/1.1 on http-443 - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Multiple tomcat Instances on Same Server and One Apache Instance
On 15/05/2012 17:22, André Warnier wrote: Vickie Troy-McKoy wrote: ... Thank you Andre for your help. After I configured it as you specified, the behavior was the same as before I inquired on this forum. Well, it doesn't sound as if we really helped a lot then, or am I misunderstanding this too ? :-) Note also for the record (and maybe a later enquiry) that it should not really be necessary to have 2 separate Apache httpd instances. At least if we now understand properly what you are really trying to achieve.. Just wondering. With a 64 bits OS, how much RAM can you handle in a JVM and therefore with each instance of Tomcat? For a 12 Gb of RAM machine, it would be wise if you want to have this kind of setup (Apache and two Tomcat instances in the same machine) if you need more than 400-500 concurrent users? Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: tomcat pausing and process killed
On 09/05/2012 21:47, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miguel, On 5/9/12 11:29 AM, Miguel Gonzalez wrote: I will check it, however I don't see in the access logs something relevant. Any string I can check Anything with %-codes that are invalid. You'll have to write your own regular parser or regex to find them. or status code? Uh, 400 or 500? I don't find anything weird in the access log, I can't relate any of the Invalid chunk ignored to any GET or POST. In fact checking at a particular time I don't find any weird entry Parsing 400 or 500 errors don't show anything either. Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
munin tomcat and mysql
Dear all, I'm thinking about installing munin to monitor mysql, tomcat and system resources (specially memory and network). I'm going to install it under CentOS and Tomcat 5.5. Since it's a virtual machine, is munin going to take many resources of my system or is it a light tool? Does anyone know of any nice tutorial of installing in CentOS and Tomcat? I find many articles about installing in CentOS and Apache, but not Tomcat Regards, Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat pausing and no java process
Yes, I have pinpointed several queries that took quite long and specially subqueries. I forwarded this info to the developer but she said it was alright. More than a second to run a query seems to be a lot to me. That depends on what the query does. If it checks permissions to login, that's insane and you should get a new programmer. If it's a reporting query that gathers lots of data then 1 second might not be quite so far-fetched. I have checked directly the slow query log to pinpoint the real queries. I found many subqueries, which I believe is not a good practice, am I right? Many queries take around 2 seconds and I found two queries that took 16 seconds and 64 seconds to run (with many subqueries nested). Is there any way I can log if any query is causing any trouble in the Tomcat resources? The problem is that since the database E/R is not documented, it's going to take a while to improve the performance of the queries. Look into SchemaSpy. Nice The developer is quitting so I hope a new developer is more cooperative and we can work out these things. So you're saying that you're not going to get a lot of good information out of him/her? Not much. In the meantime I will check what I can do. I have also downloaded mysqltuner script and mysqlfragfinder a long time ago and follow the output of these tuning scripts. Just rememeber to vet all the recommendations. Don't just read "increase your query cache size" as a recommendation and blindly apply that rule. Many times the query cache is less useful that you might think. Ok, thanks for the hint. Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat pausing and no java process
Just the -XX:+HeapDumpOnOutOfMemoryError you already have. Tomcat usually doesn't shut down nicely (as evidenced by the "pausing" of connectors in your log file) when you get an OOME. Are you sure that weekly Tomcat-restart process isn't still running? Yes, it's still running. Should i disable it? Are you using a database? Are you using pooled connections? If so, enable "abandoned" tracing if that's a possibility for you (Tomcat's two built-in choices of dbcp can do abandoned tracing). Yes, a mysql database and pooled connections. How can I enable that abandoned tracing? Check the documentation for DBCP (which Tomcat uses) here: http://commons.apache.org/dbcp/configuration.html I have set logAbandoned=true. I haven't enabled the removeAbandoned and the removeAbandonedTimeout just in case. Will I get the info in the catalina log? I enabled slow queries for mysql, but the developer claims the queries are fine (although some report more than a second to be run). Well, are you actually seeing slow queries in MySQL? It's not good enough to just enable logging... you have to read the log ;) I can revert the GC strategy. the verbose options for GC are fine? Sure: that's just logging. You will have a miniscule performance degradation due to the logging, but I'm sure it's worth it to see what is happening. I'll retake that path of checking mysql indexes and slow queries. Just use mysqldumpslow to be sure. Also, if you suspect a query, use EXPLAIN on it. Mysqldumpslow be default "anonymizes" queries so you can see a "class" of equivalent queries that are failing. MySQL determines its query plan based not just upon the structure of the query, but also upon the data and the indexes that are available at the time of the query. You might want to disable this anonymizing of the dat to get an /actual/ query that was slow. Then run EXPLAIN on that to see what the plan was: maybe MySQL is making a bad decision, or maybe you need an additional index. Or maybe the query is just written poorly in the first place. MySQL is notorious for using DEPENDENT SUBQUERIES (awful performance) when such a query is not necessary. I'll check about the anonymizing thing. Right now I get the following cronjob forwarded to my email /usr/bin/mysqldumpslow -s -t /var/log/mysql-slow.log | mail -s Yes, I have pinpointed several queries that took quite long and specially subqueries. I forwarded this info to the developer but she said it was alright. More than a second to run a query seems to be a lot to me. The problem is that since the database E/R is not documented, it's going to take a while to improve the performance of the queries. The developer is quitting so I hope a new developer is more cooperative and we can work out these things. In the meantime I will check what I can do. I have also downloaded mysqltuner script and mysqlfragfinder a long time ago and follow the output of these tuning scripts. That we have outgrown our hardware, that's probably for sure. This a virtual private server. I have said the owners to move to a dedicated server but I guess they want me to squeeze this VPS as much as I can. Sheesh, if it's virtualized, just have them provision a larger instance for you and move the virtual machine. I'd recommend a 64-bit OS/JVM so you can get heaps bigger than about 1.5GiB. :-) I'm trying to convince them. Many thanks for your inputs, you have been very helpful Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat pausing and no java process
On 03/05/2012 18:21, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miguel, On 5/3/12 11:37 AM, Miguel González Castaños wrote: We had several crashes today: First - 1 outofmemory error Second- 3 Tomcat pausing Do you mean that Tomcat actually paused its connectors, or do you mean that your site became unresponsive? Were you able to capture a thread dump during this unresponsive period? Did the JVM recover or did you have to kill it and restart. If you had to kill it, that's called beign "locked-up" and not just pausing for a while. Site became unresponsive. When I saw in catalina.out that the connector was pausing and the website was not responsive, I checked the java process and was dead, so I couldn't get a thread dump If you can't take a thread dump during these "pauses", you'll never know what is happening. Any setting that can generate a dump automatically as for out of memory errors? Are you using a database? Are you using pooled connections? If so, enable "abandoned" tracing if that's a possibility for you (Tomcat's two built-in choices of dbcp can do abandoned tracing). Yes, a mysql database and pooled connections. How can I enable that abandoned tracing? I enabled slow queries for mysql, but the developer claims the queries are fine (although some report more than a second to be run). Third - 2 or 3 times where catalina.out didn't show any error and tomcat died, only a big increase of memory use right before the java process died. That's why I assumed that maybe I have set the heap memory too high (I set it up because of the out of memory errors). That's why I googled to see if anyone had seen issues and that's why I reached that URL I have set up this: -XX:+HeapDumpOnOutOfMemoryError -XX:+UseConcMarkSweepGC -Xms128m -Xmx768m -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails Great, so you've changed heap size *and* GC strategy and you still have no idea what the problem is. Well, before the heap size was 512m so I have increased it just a little (to overcome the outofmemory errors). Is that increase that much? I can revert the GC strategy. the verbose options for GC are fine? We had a performance problem last week that was very severe. One of my friends said "well, if you're not using concurrent mark-and-sweep, then that's probably the problem". (I'm not sure of the nuances, but the default GC on my system is "ConcurrentMarkSweep", so I don't know if UseConcMarkSweepGC actually gets you anything). I was skeptical. I did my research. The problem was a missing index on a database table. A simple ALTER statement changed 20-second response times back to more reasonable sub-100ms response times. No mucking-around with the JVM required. I'll retake that path of checking mysql indexes and slow queries. That's hy I suggest that you do some research *in your own environment* instead of just guessing at random things and changing every setting that you've found via Google. I hope this way I can trace if there is any memory leak in the webapp. If you have a memory leak in your webapp and you have javamelody, I would argue that you should set your heap to the /largest/ you can possibly get away with and then watch it over time. A memory leak is easy to see over time with a big heap. You are limited by a 32-bit JVM and only ~1.5GiB heap limit, so it might be hard. Maybe you've just outgrown your hardware. That we have outgrown our hardware, that's probably for sure. This a virtual private server. I have said the owners to move to a dedicated server but I guess they want me to squeeze this VPS as much as I can. Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat pausing and no java process
Unlikely. If you are using SSL or serving large static files, APR/tcnative should be something that you look into using. I do use SSL, but all is dynamic jsps Honestly, I would put more effort into upgrading to a more recent version of Tomcat than looking into using APR/tcnative. No errors before in catalina.out. The only error found today (It crashed 3 times in 3 hours) was an outofmemory error. I changed from 1024Mb to 1536Mb in catalina.sh: CATALINA_OPTS="$CATALINA_OPTS -Xms1536m -Xmx1536m" Do you actually have 2Gb of free RAM to assign to the Tomcat process? Well, as someone asked: ]# cat /proc/meminfo MemTotal: 4194304 kB MemFree: 3700308 kB Tomcat 5.5 doesn't really care what heap size it's got as long as it's more than 4MiB or so. The page you referenced in here doesn't even mention Tomcat 5.5. Usually the GC does do its job properly, and that includes pausing the whole JVM in order to reclaim memory if necessary. The above thread from ServerFault is about unresponsiveness, not about OOME: the GC caused unresponsiveness to avoid OOME (and it worked). Stop making crazy claims and actually look at your own environment to figure out what's going on. We had several crashes today: First - 1 outofmemory error Second- 3 Tomcat pausing Third - 2 or 3 times where catalina.out didn't show any error and tomcat died, only a big increase of memory use right before the java process died. That's why I assumed that maybe I have set the heap memory too high (I set it up because of the out of memory errors). That's why I googled to see if anyone had seen issues and that's why I reached that URL I have set up this: -XX:+HeapDumpOnOutOfMemoryError -XX:+UseConcMarkSweepGC -Xms128m -Xmx768m -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails I hope this way I can trace if there is any memory leak in the webapp. Considering that the developer decided to restart Tomcat once a week because the webapp didn't have enough resources, I find this very likely. Thanks for your inputs, Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat pausing and no java process
On 03/05/2012 15:29, Pid wrote: On 03/05/2012 11:35, Miguel González Castaños wrote: Dear all, My Tomcat 5.5 server reports in catalina.out under heavy load that Tomcat is pausing. I checked and no java process was running and I had to start Tomcat manually. Can you post an example of the message from catalina.out that says Tomcat is pausing? May 3, 2012 8:40:10 AM org.apache.catalina.storeconfig.StoreLoader load INFO: Find registry server-registry.xml at classpath resource May 3, 2012 8:40:10 AM org.apache.catalina.startup.Catalina start INFO: Server startup in 5771 ms May 3, 2012 9:56:54 AM org.apache.coyote.http11.Http11BaseProtocol pause INFO: Pausing Coyote HTTP/1.1 on http-80 May 3, 2012 9:56:54 AM org.apache.coyote.http11.Http11BaseProtocol pause INFO: Pausing Coyote HTTP/1.1 on http-443 May 3, 2012 1:35:03 PM org.apache.catalina.core.AprLifecycleListener lifecycleEvent INFO: The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/bin/jdk1.5.0_17/jre/lib/i386/server:/usr/bin/jdk1.5.0_17/jre/lib/i386:/usr/bin/jdk1.5.0_17/jre/../lib/i386 May 3, 2012 1:35:03 PM org.apache.coyote.http11.Http11BaseProtocol init I have just realized about that APR is missing, can be affecting to this issue? No errors before in catalina.out. The only error found today (It crashed 3 times in 3 hours) was an outofmemory error. I changed from 1024Mb to 1536Mb in catalina.sh: CATALINA_OPTS="$CATALINA_OPTS -Xms1536m -Xmx1536m" Do you actually have 2Gb of free RAM to assign to the Tomcat process? Well, the machine has 4 Gb and only runs MySQL and Tomcat. After installing Javamelody I saw that memory use was reaching 1056 Mb (probably that triggered the outofmemory error). Anything I can check or any setting I can set to get more information in the next crash? Enable JMX& connect to Tomcat using VisualVM (with the MBean& Memory Pool plugins). Monitor thread counts& memory usage. I have installed javamelody an the thread count is around 120, reaching a max of 180. Memory usage grows very fast until Tomcat hangs. I suspect is an issue of the garbage collector and setting the heap memory too high. I have reset it to 768 Mb and now the memory seems to keep quite low, some people say that Tomcat 5.x doesn't like much more than 1 Gb of heap memory and that GC is not doing his job: http://serverfault.com/questions/281515/periodic-unresponsiveness-in-tomcat Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Tomcat pausing and no java process
Dear all, My Tomcat 5.5 server reports in catalina.out under heavy load that Tomcat is pausing. I checked and no java process was running and I had to start Tomcat manually. No errors before in catalina.out. The only error found today (It crashed 3 times in 3 hours) was an outofmemory error. I changed from 1024Mb to 1536Mb in catalina.sh: CATALINA_OPTS="$CATALINA_OPTS -Xms1536m -Xmx1536m" After installing Javamelody I saw that memory use was reaching 1056 Mb (probably that triggered the outofmemory error). Anything I can check or any setting I can set to get more information in the next crash? Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Javamelody and Struts
You asked the Tomcat users' group, not the Javamelody users' group: did you expect to find many Javamelody experts around here? The author is around and some other users too, since the recommendation of using this software actually came from this user group. Also in the beginning I thought It was a Tomcat or Struts issue Sorry if my questions bothered you Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Javamelody and Struts
On 28/04/2012 22:50, evernat wrote: Hi Miguel, First, javamelody.war is *optional*. I suggest to try javamelody without using it. You may read the user guide for help with this: http://code.google.com/p/javamelody/wiki/UserGuide (if you have an exception, please copy the complete stack-trace) Then, there is an issue with Tomcat 5.5, only if you use javamelody v1.36. See http://code.google.com/p/javamelody/issues/detail?id=207 It will be fixed in v1.37 soon. That was the issue, we could have started with that :-) Using v1.35 worked great. I'm even using the collector of version 1.36 and it works together with the javamelody.jar from v1.35. Thanks for the answer, Miguel This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Javamelody and Struts
On 23/04/2012 22:00, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miguel, On 4/23/12 3:57 PM, Miguel González Castaños wrote: On 16/04/2012 12:43, Darryl Lewis wrote: Make sure the monitoring code is before the struts call. Start over again from scratch. Same application running in Tomcat 6 but HTTP works fine using a javamelody.war deployed war that I access as a central container of the monitoring. I can't get access, however, to http://myweb/mycontext/monitoring still in this Tomcat 6 server. Same webapp in Tomcat 5.5, I can't add the listener lines without breaking the Struts redirects configured in struts-config.xml Where should I add the monitoring code and where? Thanks, Miguel This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Javamelody and Struts
On 26/04/2012 15:51, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miguel, On 4/26/12 5:58 AM, Miguel González Castaños wrote: On 26/04/2012 03:58, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miguel, On 4/25/12 6:24 PM, Miguel González Castaños wrote: Please post your SSLconfiguration (cleansed of any passwords). By the way, double checking the info from my web browser I get this is a verisign class 3 secure server G3 It looks like you were using the "EV" intermediate certificates before. This page[1] says that C3G3 certs are not frequently used except for client certificates... is that what you've got? [1] http://www.verisign.com/support/roots.html I'm sorry but I come from the Apache world and I'm pretty new to Tomcat. Also I have inherited this server and the configuration is messy. When you use Java, you generally have to work with keystores. It's just a file full of keys and certificates. Think of a Java keystore as all of the following httpd directives mashed together into a single binary entity: SSLCertificateKeyFile SSLCertificateFile SSLCertificateChainFile SSLCACertificateFile Also, you have to use an "alias" that Tomcat uses (it's "tomcat") as the alias for the certificate to actually use for the server (as opposed to any other certificates you might have in the keystore). Maybe I'm wrong but should I add the CAcert somewhere in the SSL connector? There's no place to do that: the whole chain must be in the keystore, including the CA root all the way down to your own certificate. You may be able to get away with not having the very top-root CA certificate... I haven't worked too much with Java keystores so it's possible that there is a set of root, trusted certificates that are inherited by all keystores, but there are many ways to create/configure a ServerSocketFactory, so it's probably possible to set one up both with or without that globally-recognized set of root CA certs (i.e. those trusted by the JVM implicitly). If you are getting this error in Javamelody, then you need to configure Javamelody properly -- this isn't a Tomcat thing if web browsers can connect properly to Tomcat via HTTPS. Thanks for your answers, they have lead me to (partly) the solution. I found some clues here: http://h2.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&taskId=110&prodSeriesId=4164840&prodTypeId=18964&objectID=c03023432 <http://h2.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&taskId=110&prodSeriesId=4164840&prodTypeId=18964&objectID=c03023432> I didn't have to add the certificate to the tomcat keystore, but to the java keystore of the JRE Now I get a javamelody error reporting the app hasn't been configured to use javamelody, so no more SSL handshake errors Many thanks, Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Javamelody and Struts
On 26/04/2012 03:58, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miguel, On 4/25/12 6:24 PM, Miguel González Castaños wrote: Please post your SSL configuration (cleansed of any passwords). By the way, double checking the info from my web browser I get this is a verisign class 3 secure server G3 I'm sorry but I come from the Apache world and I'm pretty new to Tomcat. Also I have inherited this server and the configuration is messy. Maybe I'm wrong but should I add the CAcert somewhere in the SSL connector? Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Javamelody and Struts
Please post your SSL configuration (cleansed of any passwords). acceptCount="100" scheme="https" secure="true" keystorePass="***" keystoreFile="/opt/tomcat5/certs/tcc" keyAlias="importkey" clientAuth="false" sslProtocol="TLS" compression="500" compressableMimeType="text/html,text/xml,text/plain,application/octet-stream" /> - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Javamelody and Struts
If no web browser reports the server certificate as not valid, then what's the problem? From my original email: "Now that I'm trying to configure javamelody in Tomcat 5 with HTTPS I get the following error when I try to add the context and the URL: javamelody avax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target." I'm trying to add my context and the URL for my webapp in the javamelody application: http://myweb.com/javamelody/? I get two fields and an Add button to do that. In my tests I was able to do add a monitor for my webapp if it uses the HTTP protocol but not in the production system where it uses HTTPS. The question is, how do I know that I have added the right intermediate certs from verisign? Regards, Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Javamelody and Struts
Do those certificates match the type of certificate that you have? Last time I checked, VeriSign had a whole set of intermediate certificates and you need the ones that match the type of certificate you have (EV versus whatever the other flavors they have are). Which other flavors? How do I know? keytool -import -trustcacerts -alias EV_root -keystore /opt/tomcat5/certs/tcc -file veriCA1.cer Is /opt/tomcat5/certs/tcc the file you have configured in Tomcat? It's the keystore and yes, configured in tomcat Please post your SSL configuration (cleansed of any passwords). Finally, you didn't post your original stack trace. Since this is happening on the server-side, it's either a problem during startup or during client-certificate verification. Since you mentioned the "site certificate", I assume you are having problems with your server's SSL certificate and not a client certificate being presented by a remote client, right? I'm not sure what you mean with the server and client certificates. No web browser reports the server certificate as no valid. Can you show me what this command returns: $ keytool -list -keystore /opt/tomcat5/certs/tcc -v Your keystore contains 3 entries Alias name: evintermediate Creation date: Apr 24, 2012 Entry type: trustedCertEntry Owner: CN=VeriSign Class 3 Secure Server CA - G3, OU=Terms of use at https://www.verisign.com/rpa (c)10, OU=VeriSign Trust Network, O="VeriSign, Inc.", C=US Issuer: CN=VeriSign Class 3 Public Primary Certification Authority - G5, OU="(c) 2006 VeriSign, Inc. - For authorized use only", OU=VeriSign Trust Network, O="VeriSign, Inc.", C=US Serial number: 6ecc7aa5a7032009b8cebcf4e952d491 Valid from: Mon Feb 08 01:00:00 CET 2010 until: Sat Feb 08 00:59:59 CET 2020 Certificate fingerprints: MD5: 3C:48:42:0D:FF:58:1A:38:86:BC:FD:41:D4:8A:41:DE SHA1: 5D:EB:8F:33:9E:26:4C:19:F6:68:6F:5F:8F:32:B5:4A:4C:46:B4:76 *** *** Alias name: ev_root Creation date: Apr 24, 2012 Entry type: trustedCertEntry Owner: CN=VeriSign Class 3 Public Primary Certification Authority - G5, OU="(c) 2006 VeriSign, Inc. - For authorized use only", OU=VeriSign Trust Network, O="VeriSign, Inc.", C=US Issuer: OU=Class 3 Public Primary Certification Authority, O="VeriSign, Inc.", C=US Serial number: 250ce8e030612e9f2b89f7054d7cf8fd Valid from: Wed Nov 08 01:00:00 CET 2006 until: Mon Nov 08 00:59:59 CET 2021 Certificate fingerprints: MD5: F9:1F:FE:E6:A3:6B:99:88:41:D4:67:DD:E5:F8:97:7A SHA1: 32:F3:08:82:62:2B:87:CF:88:56:C6:3D:B8:73:DF:08:53:B4:DD:27 *** *** Alias name: Creation date: Feb 24, 2012 Entry type: keyEntry Certificate chain length: 1 Certificate[1]: Owner: CN=www.mycompany.com, OU=Terms of use at www.verisign.com/rpa (c)05, OU=Comercial, O="My OU S.L.", L=My city, ST=Madrid, C=ES Issuer: CN=VeriSign Class 3 Secure Server CA - G3, OU=Terms of use at https://www.verisign.com/rpa (c)10, OU=VeriSign Trust Network, O="VeriSign, Inc.", C=US Serial number: 7afc6539f4e816f7fe6b65f47af0 Valid from: Sat Feb 11 01:00:00 CET 2012 until: Fri Apr 12 01:59:59 CEST 2013 Certificate fingerprints: *** *** - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Javamelody and Struts
On 23/04/2012 23:35, Miguel González Castaños wrote: On 23/04/2012 23:00, Miguel González Castaños wrote: Have you imported all of the "intermediate" certificates from VeriSign. I think that for some SSL certificates, you need *two* intermediate certificates from VeriSign. No, I didn't know that I had to do that. Where can I download those intermediate certificates? Many thanks for the hint I found them here I believe http://www.verisign.com/support/verisign-intermediate-ca/secure-site-intermediate/index.html I will give it a try I've downloaded the certificates as veriCA1.cer and veriCA2.cer and run: keytool -import -trustcacerts -alias EV_root -keystore /opt/tomcat5/certs/tcc -file veriCA1.cer keytool -import -trustcacerts -alias EVIntermediate -keystore /opt/tomcat5/certs/tcc -file veriCA2.cer after that I restarted Tomcat. Unfortunately I'm getting the same error Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Javamelody and Struts
On 23/04/2012 23:00, Miguel González Castaños wrote: Have you imported all of the "intermediate" certificates from VeriSign. I think that for some SSL certificates, you need *two* intermediate certificates from VeriSign. No, I didn't know that I had to do that. Where can I download those intermediate certificates? Many thanks for the hint I found them here I believe http://www.verisign.com/support/verisign-intermediate-ca/secure-site-intermediate/index.html I will give it a try Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Javamelody and Struts
Have you imported all of the "intermediate" certificates from VeriSign. I think that for some SSL certificates, you need *two* intermediate certificates from VeriSign. No, I didn't know that I had to do that. Where can I download those intermediate certificates? Many thanks for the hint Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Javamelody and Struts
On 16/04/2012 12:43, Darryl Lewis wrote: Make sure the monitoring code is before the struts call. On 16/04/12 5:38 PM, "Miguel González Castaños" wrote: On 16/04/2012 09:34, Debraj Mallick wrote: hi there, to access use :*http:monitoring* link: *http://code.google.com/p/javamelody/wiki/UserGuide#3._First_results* Don't get me wrong, but what does it matter the access URL if my webapp doesn't even work? Hi, I was able to configure javamelody in Tomcat 6 with a HTTP protocol in a virtual machine. Now that I'm trying to configure javamelody in Tomcat 5 with HTTPS I get the following error when I try to add the context and the URL: javamelody avax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. However the site certificate is signed by Verisign. What we are doing wrong? Regards, Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Javamelody and Struts
On 16/04/2012 09:34, Debraj Mallick wrote: hi there, to access use :*http:monitoring* link: *http://code.google.com/p/javamelody/wiki/UserGuide#3._First_results* Don't get me wrong, but what does it matter the access URL if my webapp doesn't even work? Miguel
Re: Javamelody and Struts
Dear Debraj, I have followed these instructions and I can't figure out where to access the monitoring tool. On 16/04/2012 07:02, Debraj Mallick wrote: hi there, in your web.xml include these * monitoring net.bull.javamelody.MonitoringFilter monitoring /* net.bull.javamelody.SessionListener * If I add this alone, my strut webapp doesn't work and * * This is supposed to go in the struts config xml file, not in web.xml. Anyway It doesn't work either. also put the jar files in your class path. Which classpath and which jar files and where? help link: * http://code.google.com/p/javamelody/wiki/UserGuide#2._web.xml_file* and * http://code.google.com/p/javamelody/wiki/UserGuide#12._Struts_2_Actions* Of course I have followed these instructions but they are not very detailed. Also there is no error log or anything that shows me what I'm doing wrong. In catalina.out there is no reference at all. Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Javamelody and Struts
Dear all, Javamelody is supposedly very easy to install for Tomcat but We do have a webapp using the Struts framework and I can't figure out how to configure Javamelody or get any error log of what I'm doing wrong. The web.xml example in the Javamelody is blocking my webapp to work (we use forwarders). Regards, Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: determining cause of MaxThreads exhausted
The app sends massive emails to users with some info that requires users to log on the webapp and fill in some forms. It seems (without any logs or metrics) that people tend to fill the forms right away after they got the emails and together with the mailing process (that requires DB use to get email addresses) is exhausting the maxthreads connections (300 as of yesterday which I have increased to 400). Are you running out of request processors ( maxThreads) or are you running out of database connections ( maxActive)? MaxThreads Yes: log the timestamp of each access, then use one of the many fine web server analysis tools to get a picture (literally) of your volume. I have an access log with timestamps, which tool do you recommend to get an analysis? Regards, Miguel This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: determining cause of MaxThreads exhausted
On 13/04/2012 10:14, Pid * wrote: On 13 Apr 2012, at 09:09, "Miguel González Castaños" wrote: Dear all, A server that I manage has a Struts webapp with a Tomcat 5.5.x standalone server. No JMX or whatsoever has been configured. The app sends massive emails to users with some info that requires users to log on the webapp and fill in some forms. It seems (without any logs or metrics) that people tend to fill the forms right away after they got the emails and together with the mailing process (that requires DB use to get email addresses) is exhausting the maxthreads connections (300 as of yesterday which I have increased to 400). How many emails does it send? Around 1000 a minute I assume it's not going to be easy to determine what is the cause of the issue but which tools can I use? It might just be simple maths. If you send more than 400 emails and most of those people immediately respond, what would you expect to happen? I guess I need to log in some way how the server is behaving. Maybe enabling JMX or Javamelody? Any way of getting where the webapp is hanged or where in the code of the webapp is hanging? Access log? Enable JMX, use VisualVM/JConsole. Any way to use access log to get a view of how many visitors (not connections) we are getting? And which tool can help me to simulate the load of so many people connecting to the webapp remotely? JMeter. Any howto or article that explains of how to use JMeter in an environment like this? Miguel This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
determining cause of MaxThreads exhausted
Dear all, A server that I manage has a Struts webapp with a Tomcat 5.5.x standalone server. No JMX or whatsoever has been configured. The app sends massive emails to users with some info that requires users to log on the webapp and fill in some forms. It seems (without any logs or metrics) that people tend to fill the forms right away after they got the emails and together with the mailing process (that requires DB use to get email addresses) is exhausting the maxthreads connections (300 as of yesterday which I have increased to 400). I assume it's not going to be easy to determine what is the cause of the issue but which tools can I use? I guess I need to log in some way how the server is behaving. Maybe enabling JMX or Javamelody? Any way of getting where the webapp is hanged or where in the code of the webapp is hanging? And which tool can help me to simulate the load of so many people connecting to the webapp remotely? Regards, Miguel This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Bash script for monitoring status of the Tomcat server
Dear all, Recently we had to restart our standalone Tomcat server because apparently the 150 max threads setting was not enough. I have been googling a bit and it seems it's possible to monitor the performance of the server and also get the numbers when the connection pool or the threads are about to be exhausted. I would like to run a bash script to monitor these values and warn me if the reach a certain level. Also a tool that generates some graphs would be great. I haven't found anything googling. Any ideas of how to build my own script? Regards, Miguel This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender. Please consider the environment before printing this email. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
