Chaitanya, This will be long and somewhat speculative.
On 7/20/2017 9:00 AM, Christopher Schultz wrote: > Chaitanya, > > On 7/20/17 11:03 AM, Chaitanya Sabbineni wrote: >> Stop script in the sense it's Catalina script only but we usually >> stop tomcat using the command Catalina.sh stop. But in our case we >> are not manually executing this script to stop tomcat and tomcat is >> stopping on its own. > >> our main problem here us tomcat is stopping on its own and it needs >> a restart. > > Right. > >> If I understand you correct you are telling TimerThread that does >> not stop when the application is shut down. Can you let me know >> what actually the timer thread mean. And moreover if the timer >> thread didn't stop ideally tomcat shouldn't stop but in our case >> it's stopping. > > Tomcat is stopping but the JVM is not. If your application were to > shut-down cleanly, then the JVM would exit as well. This is unrelated > to your real problem (unexpected Tomcat shutdown), but you might want > to look into fixing that, because it makes your application impossible > to reload without risking serious heap space problems. > >> Yes my question is why Tomcat is being shut down at all. > >> Yes when ever tomcat is stopping on own(not daily) it stops at >> 02:00 . You mentioned that your guess is that we are using a >> service runner that is configured to bounce your services at >> 02:00.Can let me know what this service runner is and how to check >> it. > > I know nothing about your environment. Until you mentioned > "catalina.sh stop" above, I didn't even know you were on a UNIX-like > environment. Honestly, I assumed you were on Windows because > "mysterious service stoppage" has Microsoft Windows behavior written > all over it. > > There are two ways to trigger a Tomcat shut down: > > 1. Send a TERM signal to the process > 2. Connect to Tomcat's shutdown listener (default: port 8005) and give > the shutdown command (default: "SHUTDOWN") > > You can eliminate one of those possibilities by setting the shutdown > port in server.xml to "" (empty) which will disable this type of shutdow > n: > > <Server port="" ... > > You cannot disable the other type of shutdown... any user on the > system who can send a TERM signal to your process could terminate Tomcat > . > > As for catching whoever is shutting down your Tomcat, you may want to > look at who has administrative access to your server, and who has > access to the user running your Tomcat server. > > Check your syslog to find sudo and cron events that might be > automatically shutting-down Tomcat. > > If you want to catch a TCP connection, you will likely have to enable > tcpwrappers, iptables, ipfw, etc. to log connections to port 8005. > Those logs will only tell you that the command is being sent, not who > is sending it. > > -chris I am going to go out on a limb here and try to explain things. Please note that this is all based upon reading between the lines, and may not at all reflect what is actually going on. Overview -------- I suspect the following: 1. Logrotate of catalina.out at 2 AM 2. Tomcat JVM fails to exit, then restart Detail ------ 1. Logrotate (or other log rotation utility) There are several ways that one can use to rotate catalina.out. See the following: https://wiki.apache.org/tomcat/FAQ/Logging#Q10 Some system admins actually stop Tomcat, rotate the logs, and then start Tomcat. This has the advantage over the logrotate's copytruncate option in that there is no possibility of partial log entries. 2. JVM fails to exit From your error log, you have a TimeerTask thread that is not shutting down. This prevents the JVM from exiting (see Chris's comments). I suspect that this then prevents the start script from starting Tomcat again (depending on the script). Solutions --------- 1. Fix your application so that the Tomcat JVM exits cleanly Use a thread pool. Manage the thread pool in a servlet context listener (creation, destruction). This should be done in addition to anything else. 2. Talk to your system admin to see if log rotation is being used Use copytruncate with logrotate rather than stopping and starting the Tomca service. This is assuming that you're using logrotate, and that there is a logrotate process that kicks off every morning at 2 AM. 3. Use another method for rotating catalina.out There are other methods for rotating catalina.out mentioned in the link above. Again, this is assuming that your system admin has implemented some log rotation which is causing the problem. 4. See Chris's comments above concerning potential security issues Finally ------- Your catalina.out file should be small, and consist of startup / shutdown messages from Tomcat. Other (application) information should go into application-specific log files. This means that you should implement some sort of logging for your applications. In other words, there should be little need to periodically rotate catalina.out. . . . just my two cents /mde/
signature.asc
Description: OpenPGP digital signature
