Rainer, Thank you so much for your kind reply! I have increased the java heap size to 512MB (-Xms512m -Xmx512m). I am hoping that would fix the issue. I had configured our webserver to use Jasig's Central Authentication System (CAS). Recently I increased the session timeout from 30 minutes to 4 hours. I am guessing that must have had an impact on the number of tickets that the CAS could store in the Java's memory space.
I did run the kill -QUIT command against the tomcat process. It did generate a huge output in the catalina.out file. I am unable to decipher it. I do not want to post it to the mailing list because its very long. Would you be able to please tell me what should I be looking for within this long thread dump? Thanks again, Rainer :) Joe On Sat, Oct 3, 2009 at 12:24 PM, Rainer Jung <rainer.j...@kippdata.de> wrote: > On 03.10.2009 20:07, Joe Hansen wrote: >> Hey All, >> >> I get this error (java.lang.OutOfMemoryError: Java heap space) after >> my Apache 2.0/Tomcat 5.5/mod_jk installation has been up and running >> for a few hours. This problem started just since two days. Never had >> this issue before! >> >> I have also noticed that as soon as I startup the server, 9 httpd >> processes start. Number of httpd processes keep on increasing until I >> get the OutOfMemoryError. >> $ps -aef | grep httpd >> root 31984 1 0 11:23 ? 00:00:00 /usr/sbin/httpd >> apache 31987 31984 0 11:23 ? 00:00:00 /usr/sbin/httpd >> apache 31988 31984 0 11:23 ? 00:00:00 /usr/sbin/httpd >> apache 31989 31984 0 11:23 ? 00:00:00 /usr/sbin/httpd >> apache 31990 31984 0 11:23 ? 00:00:00 /usr/sbin/httpd >> apache 31991 31984 0 11:23 ? 00:00:00 /usr/sbin/httpd >> apache 31992 31984 0 11:23 ? 00:00:00 /usr/sbin/httpd >> apache 31993 31984 0 11:23 ? 00:00:00 /usr/sbin/httpd >> apache 31994 31984 0 11:23 ? 00:00:00 /usr/sbin/httpd > > Sounds like requests get stuck or responses are only returned very slowly. > > I would take thread dumps during the time requests pile up (e.g. httpd > process count increases). Thread dumps are generated by "kil -QUIT" > against the Tomcat process. Result is written to catalina.out. Always > take afew thread dumps shortly after each other, e.g. 3 dumps each 3 > seconds apart from the previous one, so that you can find out, if a > status in a dump is pure coincidence or lasts for somewhat longer. > >> $ps -aef | grep tomcat >> root 31949 1 43 11:23 pts/0 00:00:58 /usr/java/jdk/bin/java >> -Djava.u >> il.logging.manager=org.apache.juli.ClassLoaderLogManager >> -Djava.util.logging.co >> fig.file=/usr/lib/apache-tomcat/conf/logging.properties >> -Djava.endorsed.dirs=/u >> r/lib/apache-tomcat/common/endorsed -classpath >> :/usr/lib/apache-tomcat/bin/boot >> trap.jar:/usr/lib/apache-tomcat/bin/commons-logging-api.jar >> -Dcatalina.base=/us >> /lib/apache-tomcat -Dcatalina.home=/usr/lib/apache-tomcat >> -Djava.io.tmpdir=/usr >> lib/apache-tomcat/temp org.apache.catalina.startup.Bootstrap start > > There is no Java memory configuration included above (i.e. al defaults). > It might well be, that you have to explicitely set heap size, perm size > and if you like also eden and semi spaces. > >> Can someone on this list please help me resolve this issue. >> >> Thanks you, >> Joe > > Regards, > > Rainer > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
