This is really getting frustrating. We have 75 instances on tomcat6 running on the same server. Each instance runs from its own discrete directory, where the tomcat-env.sh script and catalina.properties file set environment vars to point to that unique instance's folders. Every time we reboot the server, we find that certain XML files are getting deleted from the Catalina/localhost folder. This causes the web app to be unable to find certain files at runtime. The main file that gets deteled is called "Attachment.xml". It's contents look like this:
<Context docBase="/ha_ftp.nfs/site001/mobiledoc" debug="1" reloadable="false"> </Context> The siteXXX part differs from instance to instance to instance. It helps the application locate certain files that users need at runtime. We can stop and start any instance of tomcat whenever we wish by using 'service tomcat6_XXX stop' or 'service tomcat6_XXX start'. (These are all symlinks in /etc/init.d that point to /opt/tomcat/bin/tomcat). That works fine. But if we actually reboot the whole server, then when it comes up, we find that about 80% of the instances have deleted the Attachment.xml file and we have to manually put it back in. This really puzzles me because the same script gets run whether at boot time or from a shell prompt. Someone suggested that it is caused by autoDeploy, so we changed all instances to autoDeploy="false" but when we rebooted we got the same thing. Most of the sites had lost their Attachment.xml file. If anyone cares, here is the startup script... #!/bin/bash # # Linux init script for the Apache Tomcat servlet container. # # chkconfig: 2345 96 14 # description: The Apache Tomcat servlet container. # processname: tomcat # config: /opt/tomcat/conf/tomcat-env.sh # # Copyright (c) 2007, 2008 Jason Brittain <jason.britt...@gmail.com> # # ---------------------------------------------------------------------- # $Id: init.linux 112 2008-08-05 10:04:53Z jasonb $ # # Set the site ID; everything else derives from it SID=`echo $0|cut -d"_" -f2` # Set the multisite root folder MULTISITE_ROOT=/alley # Source function library. if [ -x /etc/rc.d/init.d/functions ]; then /etc/rc.d/init.d/functions fi APP_ENV="$MULTISITE_ROOT/site$SID/tomcat6/conf/tomcat-env.sh" # Source the app config file, if it exists. [ -r "$APP_ENV" ] && . "${APP_ENV}" # The path to the Tomcat start/stop script. TOMCAT_SCRIPT=$CATALINA_HOME/bin/catalina.sh # The name of this program. PROG="$0" # Resolve links - $0 may be a soft link. while [ -h "$PROG" ]; do ls=`ls -ld "$PROG"` link=`expr "$ls" : '.*-> \(.*\)$'` if expr "$link" : '.*/.*' > /dev/null; then PROG="$link" else PROG=`dirname "$PROG"`/"$link" fi done PROG="`basename $PROG`" # If TOMCAT_USER is not set, use "tomcat". if [ -z "$TOMCAT_USER" ]; then TOMCAT_USER="tomcat" fi # Since the daemon function will run $TOMCAT_SCRIPT, no environment # stuff should be defined here anymore. Please use the # $MULTISITE_ROOT/site$SID/tomcati6/conf/tomcat-env.sh file instead. let RETVAL=0 JVM_PID="0" JVM_RUNNING="false" start() { echo -n "Starting $PROG: " checkJvmRunning if [ "$JVM_RUNNING" == "true" ]; then echo -n "\"$JVM_ID\" JVM process already running. " else # Raise the process's maximum number of file descriptors to 4096. ulimit -n 4096 # Exit with an explanation if our JAVA_HOME isn't found. if [ ! -d "$JAVA_HOME" ]; then echo "JAVA_HOME of $JAVA_HOME not found." echo "See ${APP_ENV}" if [ -x /etc/rc.d/init.d/functions ]; then echo -n "Starting $PROG: " echo_failure echo fi return 1 fi # Start Tomcat, running as the $TOMCAT_USER. if [ "$USER" == "$TOMCAT_USER" ]; then # We're already the $TOMCAT_USER so just exec the script. exec bash -c "set -a; . $APP_ENV; $TOMCAT_SCRIPT start" \ >/dev/null 2>&1 else if [ -x /etc/rc.d/init.d/functions -a -x /sbin/runuser ]; then echo "APP_ENV='$APP_ENV'" echo "TOMCAT_SCRIPT='$TOMCAT_SCRIPT'" echo "TOMCAT_USER='$TOMCAT_USER'" #runuser -s /bin/bash - $TOMCAT_USER -c "set -a; . $APP_ENV; $TOMCAT_SCRIPT start" &>/dev/null runuser -s /bin/bash - $TOMCAT_USER -c "set -a; . $APP_ENV; $TOMCAT_SCRIPT run" & else su - $TOMCAT_USER -c "/bin/bash -c \ \"set -a; . $APP_ENV; $TOMCAT_SCRIPT start\"" \ >/dev/null 2>&1 fi let RETVAL=$? # If the return value is zero, then the attempt to start it is # good so far. if [ $RETVAL -eq 0 ]; then # Sleep some seconds while Tomcat begins to start, then check it. sleep 7 checkJvmRunning if [ "$JVM_RUNNING" == "false" ]; then let RETVAL=1 fi fi fi # Output "[ OK ]" or "[ FAILED ]" if [ $RETVAL -eq 0 ]; then if [ -x /etc/rc.d/init.d/functions ]; then echo_success echo else echo "[ OK ]" fi else if [ -x /etc/rc.d/init.d/functions ]; then echo_failure echo else echo "[ FAILED ]" fi fi return $RETVAL } stop() { echo -n "Stopping $PROG: " checkJvmRunning if [ "$JVM_RUNNING" == "true" ]; then # Exit with an explanation if our JAVA_HOME isn't found. if [ ! -d "$JAVA_HOME" ]; then echo "JAVA_HOME of $JAVA_HOME not found." echo "See ${APP_ENV}" echo -n "Stopping $PROG: " if [ -x /etc/rc.d/init.d/functions ]; then echo_failure echo else echo "[ FAILED ]" fi return 1 fi # Stop Tomcat, running as the $TOMCAT_USER. We also unset any # JVM memory switches -- the stop client shouldn't start with those. if [ "$USER" == "$TOMCAT_USER" ]; then # We're already the $TOMCAT_USER so just exec the script. exec bash -c "set -a; . $APP_ENV; shopt -s extglob; \ export JAVA_OPTS=\"\${JAVA_OPTS//-Xm[sx]+([0-9])[mM]}\"; \ shopt -u extglob; $TOMCAT_SCRIPT stop" &>/dev/null else if [ -x /etc/rc.d/init.d/functions -a -x /sbin/runuser ]; then runuser -s /bin/bash - $TOMCAT_USER \ -c "set -a; . $APP_ENV; shopt -s extglob; \ export JAVA_OPTS=\"\${JAVA_OPTS//-Xm[sx]+([0-9])[mM]}\"; \ shopt -u extglob; $TOMCAT_SCRIPT stop" &>/dev/null else su - $TOMCAT_USER -c "/bin/bash -c \ \"set -a; . $APP_ENV; shopt -s extglob; \ export JAVA_OPTS=\"\${JAVA_OPTS//-Xm[sx]+([0-9])[mM]}\"; \ shopt -u extglob; $TOMCAT_SCRIPT stop\"" &>/dev/null fi fi let RETVAL=$? if [ $RETVAL -eq 0 ]; then checkJvmRunning if [ "$JVM_RUNNING" == "true" ]; then # Loop here until either Tomcat shuts down on its own, or # until we've waited longer than SHUTDOWN_WAIT seconds. let count=0 until [ "`ps --pid $JVM_PID | grep -c $JVM_PID`" == "0" ] || [ $count -gt $SHUTDOWN_WAIT ] do if [ $count -eq 0 ]; then echo fi echo "Waiting for processes to exit.." sleep 1 let count=$count+1 done if [ $count -gt $SHUTDOWN_WAIT ]; then # Tomcat is still running, so we'll send the JVM a # SIGTERM signal and wait again. echo "Sending the Tomcat processes a SIGTERM asking them" \ "to shut down gracefully.." /bin/kill -s SIGTERM $JVM_PID &>/dev/null # Loop here until either Tomcat shuts down on its own, or # until we've waited longer than SHUTDOWN_WAIT seconds. let count=0 until [ "`ps --pid $JVM_PID | grep -c $JVM_PID`" \ == "0" ] || [ $count -gt $SHUTDOWN_WAIT ] do echo "Waiting for processes to exit.." sleep 1 let count=$count+1 done if [ $count -gt $SHUTDOWN_WAIT ]; then # Tomcat is still running, and just won't shut down. # We'll kill the JVM process by sending it a SIGKILL # signal and wait again for the JVM process to die. echo "Killing processes which didn't stop after" \ "$SHUTDOWN_WAIT seconds." /bin/kill -s SIGKILL $JVM_PID &>/dev/null # Loop here until either Tomcat shuts down on its own, # or until we've waited longer than SHUTDOWN_WAIT # seconds. let count=0 until [ "`ps --pid $JVM_PID | grep -c $JVM_PID`" \ == "0" ] || [ $count -gt $SHUTDOWN_WAIT ] do echo "Waiting for processes to exit.." sleep 1 let count=$count+1 done if [ $count -gt $SHUTDOWN_WAIT ]; then # The JVM process won't shut down even with a # SIGKILL, so there is something really wrong. echo "The \"$JVM_ID\" JVM process is wedged and" \ "won't shut down even when it is" echo "sent a SIGKILL." echo "Process ID $JVM_PID." # Right here we may want to email an administrator. let RETVAL=1 fi fi # We need to sleep here to make sure the JVM process dies. sleep 2 fi fi fi fi # Output "[ OK ]" or "[ FAILED ]" if [ $RETVAL -eq 0 ]; then if [ -x /etc/rc.d/init.d/functions ]; then echo_success echo else echo "[ OK ]" fi else if [ -x /etc/rc.d/init.d/functions ]; then echo_failure echo else echo "[ FAILED ]" fi fi return $RETVAL } getJvmPid() { JVM_PID="`ps awwx | grep \"jvm=$JVM_ID \" | grep -v grep | head -n 1 | \ cut -c -5`" } checkJvmRunning() { getJvmPid if [ "$JVM_PID" != "" ]; then JVM_RUNNING="true" else JVM_RUNNING="false" fi } # See how we were called. case "$1" in start) start ;; stop) stop ;; restart) stop if [ $RETVAL -eq 0 ]; then start fi ;; status) SERVICE_NAME="$PROG" checkJvmRunning if [ "$JVM_RUNNING" == "true" ]; then echo "$SERVICE_NAME (pid $JVM_PID) is running." let RETVAL=0 else echo "$SERVICE_NAME is not running." let RETVAL=1 fi exit $RETVAL ;; condrestart) # If it's already running, restart it, otherwise don't start it. checkJvmRunning if [ "$JVM_RUNNING" == "true" ]; then if [ "$JVM_RUNNING" == "true" ]; then stop if [ $RETVAL -eq 0 ]; then start fi fi ;; *) echo "Usage: $PROG {start|stop|restart|status|condrestart}" let RETVAL=1 exit $RETVAL esac exit $RETVAL And here is a sample tomcat-env.sh... #!/bin/bash APP_ENV="/alley/site001/tomcat6/conf/tomcat-env.sh" JAVA_HOME="/usr/java/jdk1.6.0_21" # Where your Tomcat installation lives. CATALINA_HOME="/alley/site001/tomcat6" JASPER_HOME="/alley/site001/tomcat6" CATALINA_TMPDIR="/alley/site001/tomcat6/temp" # The path to this application's writeable runtime Tomcat tree. CATALINA_BASE="/alley/site001/tomcat6" # The ID of this package's JVM. JVM_ID="tomcat6_001" # You can pass extra JVM startup parameters to java here if you wish. JAVA_OPTS="-Djvm=$JVM_ID -Xms64M -Xmx512M -Djava.awt.headless=true \ -Djava.net.preferIPv4Stack=true $JPDA_OPTS $JMX_OPTS" # What user should run tomcat. TOMCAT_USER="site001" # Time to wait in seconds before sending signals to stop the JVM process. # The total maximum wait time is three times the number you set here! let SHUTDOWN_WAIT=2 -- Eric Robinson Disclaimer - March 22, 2011 This email and any files transmitted with it are confidential and intended solely for users@tomcat.apache.org. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physicians' Managed Care or Physician Select Management. Warning: Although Physicians' Managed Care or Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. This disclaimer was added by Policy Patrol: http://www.policypatrol.com/