-----Original Message----- From: Daniel Mikusa [mailto:dmik...@gopivotal.com] Sent: Tuesday, March 04, 2014 12:42 PM To: Tomcat Users List Subject: Re: tomcat 6 refuses mod_jk connections after server runs for a couple of days
On Mar 4, 2014, at 1:55 PM, Isaac Gonzalez <igonza...@autoreturn.com> wrote: > Dan, > > ________________________________________ > From: Daniel Mikusa [dmik...@gopivotal.com] > Sent: Tuesday, March 04, 2014 6:20 AM > To: Tomcat Users List > Subject: Re: tomcat 6 refuses mod_jk connections after server runs for > a couple of days > > On Mar 4, 2014, at 6:32 AM, Rainer Jung <rainer.j...@kippdata.de> wrote: > >> On 27.02.2014 23:06, Isaac Gonzalez wrote: >>> Hi Christopher(and Konstantin), attached is a couple of thread dumps of >>> when we experienced the issue again today. I also noticed we get this >>> message right before the problem occurs: >>> Feb 27, 2014 12:47:15 PM >>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run >>> SEVERE: Caught exception (java.lang.OutOfMemoryError: unable to >>> create new native thread) executing >>> org.apache.jk.common.ChannelSocket$SocketAcceptor@177ddea, >>> terminating thread >> >> Is it a 32Bit system? You have 2GB of heap plus Perm plus native >> memory needed by the process plus thread stacks. Not unlikely, that >> you ran out of memory address space for a 32 bit process. >> >> The only fixes would then be: >> >> - switch to a 64 bit system >> >> - reduce heap if the app can work with less >> >> - improve performance or eliminate bottlenecks so that the app works >> with less threads >> >> - limit you connector thread pool size. That will still mean that if >> requests begin to queue because of performance problems, the web >> server can't create additional connections, but you won't get in an >> irregular situation as you experience now. In that case you would >> need to configure a low idle timeout for the connections on the JK and TC >> side. > > It may also be possible to lower the thread stack size with the -Xss option. > > Ok so we are 64 bit Linux with 1024k in the 64-bit VM....would lowering it to > 64k be a bit too low? What sort of repercussions would we run into? > Very helpful information by the way. It depends on your apps, so you'll need to test and see. If you go too low, you'll get StackOverflow exceptions. If you see those, just gradually increase until they go away. Dan > > -Isaac > > > http://www.oracle.com/technetwork/java/hotspotfaq-138619.html#threads_ > oom > > Might buy you some room for a few additional threads. > > Dan > >> >> Regards, >> >> Rainer >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org Ok so the problem just happened again just now. Dan, Can you elaborate on how to configure limiting the connector thread pool size. I am also going to lower the thread stack size as you recommended. It seems like this problem creeps up when we have a hiccup in connectivity at our data center. Perhaps I also need to lower the idle timeout some more between tomcat and mod_jk. They are also between a firewall by the way, so I can configure a timeout between the two there as well. We aren't experiencing too many idle disconnects there. -Isaac --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
