On Mar 5, 2014, at 4:51 PM, Isaac Gonzalez <igonza...@autoreturn.com> wrote:
> > > -----Original Message----- > From: Daniel Mikusa [mailto:dmik...@gopivotal.com] > Sent: Tuesday, March 04, 2014 12:42 PM > To: Tomcat Users List > Subject: Re: tomcat 6 refuses mod_jk connections after server runs for a > couple of days > > On Mar 4, 2014, at 1:55 PM, Isaac Gonzalez <igonza...@autoreturn.com> wrote: > >> Dan, >> >> ________________________________________ >> From: Daniel Mikusa [dmik...@gopivotal.com] >> Sent: Tuesday, March 04, 2014 6:20 AM >> To: Tomcat Users List >> Subject: Re: tomcat 6 refuses mod_jk connections after server runs for >> a couple of days >> >> On Mar 4, 2014, at 6:32 AM, Rainer Jung <rainer.j...@kippdata.de> wrote: >> >>> On 27.02.2014 23:06, Isaac Gonzalez wrote: >>>> Hi Christopher(and Konstantin), attached is a couple of thread dumps of >>>> when we experienced the issue again today. I also noticed we get this >>>> message right before the problem occurs: >>>> Feb 27, 2014 12:47:15 PM >>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run >>>> SEVERE: Caught exception (java.lang.OutOfMemoryError: unable to >>>> create new native thread) executing >>>> org.apache.jk.common.ChannelSocket$SocketAcceptor@177ddea, >>>> terminating thread >>> >>> Is it a 32Bit system? You have 2GB of heap plus Perm plus native >>> memory needed by the process plus thread stacks. Not unlikely, that >>> you ran out of memory address space for a 32 bit process. >>> >>> The only fixes would then be: >>> >>> - switch to a 64 bit system >>> >>> - reduce heap if the app can work with less >>> >>> - improve performance or eliminate bottlenecks so that the app works >>> with less threads >>> >>> - limit you connector thread pool size. That will still mean that if >>> requests begin to queue because of performance problems, the web >>> server can't create additional connections, but you won't get in an >>> irregular situation as you experience now. In that case you would >>> need to configure a low idle timeout for the connections on the JK and TC >>> side. >> >> It may also be possible to lower the thread stack size with the -Xss option. >> >> Ok so we are 64 bit Linux with 1024k in the 64-bit VM....would lowering it >> to 64k be a bit too low? What sort of repercussions would we run into? >> Very helpful information by the way. > > It depends on your apps, so you'll need to test and see. If you go too low, > you'll get StackOverflow exceptions. If you see those, just gradually > increase until they go away. > > Dan > > >> >> -Isaac >> >> >> http://www.oracle.com/technetwork/java/hotspotfaq-138619.html#threads_ >> oom >> >> Might buy you some room for a few additional threads. >> >> Dan >> >>> >>> Regards, >>> >>> Rainer >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>> For additional commands, e-mail: users-h...@tomcat.apache.org >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > > Ok so the problem just happened again just now. Dan, Can you elaborate on how > to configure limiting the connector thread pool size. I am also going to > lower the thread stack size as you recommended. It seems like this problem > creeps up when we have a hiccup in connectivity at our data center. Perhaps I > also need to lower the idle timeout some more between tomcat and mod_jk. They > are also between a firewall by the way, so I can configure a timeout between > the two there as well. We aren't experiencing too many idle disconnects there. See maxConnections / maxThreads on the Connector tag. http://tomcat.apache.org/tomcat-7.0-doc/config/http.html#Standard_Implementation or Executor if you’re using an executor. http://tomcat.apache.org/tomcat-7.0-doc/config/executor.html Dan > > -Isaac > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
