Re: OutOfMemoryError - 100 thread limit?
There are no limits in tomcat. http://jakarta.apache.org/tomcat/faq/memory.html -Tim LeeAnn Pultz wrote: I have a tomcat server with multiple copies of our web application running. We are running into what seems to be a hard-coded limit on the number of threads available to the Tomcat application. I have added code to our servlet class which prints out the number of active threads whenever I initialize the servlet. On Red Hat Linux boxes, whenever we start up enough copies of the servlet to hit 100 active threads, we reach an OutOfMemoryError - regardless of the Xms/Xmx settings (memory profiling shows that we have plenty of memory available). On Windows XP, 50 seems to be the magic number. I have tried tweaking Xss parameters, my ulimit command in Linux shows "unlimited" - and I have reached the same results with Tomcat 3.3.1, 4.0.28 and 5.0.28 using Java 1.4.1 and 1.4.2. When I use WebLogic 8.1 on the same Linux box, I reach 241 active threads with no problems whatsoever (stopped my testing at that number) so it does not seem to be an o/s limitation, or a java limitation - which leads me to believe there is something in Tomcat? Is the number of threads available to tomcat hard coded somewhere? Is there a parameter or configuration setting that I can change to increase this? If this question is better off posted to the developers list, please someone, let me know :) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
I'm able to go up to 647 threads and 500 concurrent connections with 5.5.4 without any problems. chances are, it's a memory leak in the webapp. only way to know for sure is to profile the application. peter lin On 4/20/05, Tim Funk <[EMAIL PROTECTED]> wrote: > There are no limits in tomcat. > http://jakarta.apache.org/tomcat/faq/memory.html > > -Tim > > LeeAnn Pultz wrote: > > > I have a tomcat server with multiple copies of our web application > > running. We are running into what seems to be a hard-coded limit on the > > number of threads available to the Tomcat application. > > > > I have added code to our servlet class which prints out the number of > > active threads whenever I initialize the servlet. On Red Hat Linux > > boxes, whenever we start up enough copies of the servlet to hit 100 > > active threads, we reach an OutOfMemoryError - regardless of the Xms/Xmx > > settings (memory profiling shows that we have plenty of memory > > available). On Windows XP, 50 seems to be the magic number. > > > > I have tried tweaking Xss parameters, my ulimit command in Linux shows > > "unlimited" - and I have reached the same results with Tomcat 3.3.1, > > 4.0.28 and 5.0.28 using Java 1.4.1 and 1.4.2. When I use WebLogic 8.1 > > on the same Linux box, I reach 241 active threads with no problems > > whatsoever (stopped my testing at that number) so it does not seem to be > > an o/s limitation, or a java limitation - which leads me to believe > > there is something in Tomcat? > > > > Is the number of threads available to tomcat hard coded somewhere? Is > > there a parameter or configuration setting that I can change to increase > > this? > > > > If this question is better off posted to the developers list, please > > someone, let me know :) > > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
This is not a memory leak in the application - we have hooked up a profiler to the application and watched the actual memory usage when causing this issue to happen. We have lots of memory available, are nowhere near the Xmx limit, and the machine has lots of memory available over and above the Xmx limit. In fact, I can cause the error to happen every single time, simply by hitting the first web page (login page) of the application - I don't have to log in, or do any "work". All I have to do to get the error to happen is hit 18-19 different instances of the web application, watch the Active thread count go up to 100 and tip over the tomcat. We've gone through all of the suggested infomation out on the web - we've tweaked the Xms, Xmx and Xss parameters - none of these changes made any difference, since we're not actually running out of memory. I've reproduced this on 3 different machines, all running Red Hat Linux ES 3.0 - both with LD_ASSUME_KERNEL 2.4.19 and without, so the Linux threading model doesn't seem to affect the problem. At 04:16 AM 4/20/2005, you wrote: I'm able to go up to 647 threads and 500 concurrent connections with 5.5.4 without any problems. chances are, it's a memory leak in the webapp. only way to know for sure is to profile the application. peter lin On 4/20/05, Tim Funk <[EMAIL PROTECTED]> wrote: > There are no limits in tomcat. > http://jakarta.apache.org/tomcat/faq/memory.html > > -Tim > > LeeAnn Pultz wrote: > > > I have a tomcat server with multiple copies of our web application > > running. We are running into what seems to be a hard-coded limit on the > > number of threads available to the Tomcat application. > > > > I have added code to our servlet class which prints out the number of > > active threads whenever I initialize the servlet. On Red Hat Linux > > boxes, whenever we start up enough copies of the servlet to hit 100 > > active threads, we reach an OutOfMemoryError - regardless of the Xms/Xmx > > settings (memory profiling shows that we have plenty of memory > > available). On Windows XP, 50 seems to be the magic number. > > > > I have tried tweaking Xss parameters, my ulimit command in Linux shows > > "unlimited" - and I have reached the same results with Tomcat 3.3.1, > > 4.0.28 and 5.0.28 using Java 1.4.1 and 1.4.2. When I use WebLogic 8.1 > > on the same Linux box, I reach 241 active threads with no problems > > whatsoever (stopped my testing at that number) so it does not seem to be > > an o/s limitation, or a java limitation - which leads me to believe > > there is something in Tomcat? > > > > Is the number of threads available to tomcat hard coded somewhere? Is > > there a parameter or configuration setting that I can change to increase > > this? > > > > If this question is better off posted to the developers list, please > > someone, let me know :) > > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
that sounds very odd. so if I understand correctly. this happens when you hit the login page. I assume this don't happen with a static html page? by any chance are you using jsp tags or some web framework? peter On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > This is not a memory leak in the application - we have hooked up a profiler > to the application and watched the actual memory usage when causing this > issue to happen. We have lots of memory available, are nowhere near the > Xmx limit, and the machine has lots of memory available over and above the > Xmx limit. > > In fact, I can cause the error to happen every single time, simply by > hitting the first web page (login page) of the application - I don't have > to log in, or do any "work". All I have to do to get the error to happen > is hit 18-19 different instances of the web application, watch the Active > thread count go up to 100 and tip over the tomcat. > > We've gone through all of the suggested infomation out on the web - we've > tweaked the Xms, Xmx and Xss parameters - none of these changes made any > difference, since we're not actually running out of memory. I've > reproduced this on 3 different machines, all running Red Hat Linux ES 3.0 - > both with LD_ASSUME_KERNEL 2.4.19 and without, so the Linux threading model > doesn't seem to affect the problem. > > > At 04:16 AM 4/20/2005, you wrote: > >I'm able to go up to 647 threads and 500 concurrent connections with > >5.5.4 without any problems. chances are, it's a memory leak in the > >webapp. only way to know for sure is to profile the application. > > > >peter lin > > > > > >On 4/20/05, Tim Funk <[EMAIL PROTECTED]> wrote: > > > There are no limits in tomcat. > > > http://jakarta.apache.org/tomcat/faq/memory.html > > > > > > -Tim > > > > > > LeeAnn Pultz wrote: > > > > > > > I have a tomcat server with multiple copies of our web application > > > > running. We are running into what seems to be a hard-coded limit on the > > > > number of threads available to the Tomcat application. > > > > > > > > I have added code to our servlet class which prints out the number of > > > > active threads whenever I initialize the servlet. On Red Hat Linux > > > > boxes, whenever we start up enough copies of the servlet to hit 100 > > > > active threads, we reach an OutOfMemoryError - regardless of the Xms/Xmx > > > > settings (memory profiling shows that we have plenty of memory > > > > available). On Windows XP, 50 seems to be the magic number. > > > > > > > > I have tried tweaking Xss parameters, my ulimit command in Linux shows > > > > "unlimited" - and I have reached the same results with Tomcat 3.3.1, > > > > 4.0.28 and 5.0.28 using Java 1.4.1 and 1.4.2. When I use WebLogic 8.1 > > > > on the same Linux box, I reach 241 active threads with no problems > > > > whatsoever (stopped my testing at that number) so it does not seem to be > > > > an o/s limitation, or a java limitation - which leads me to believe > > > > there is something in Tomcat? > > > > > > > > Is the number of threads available to tomcat hard coded somewhere? Is > > > > there a parameter or configuration setting that I can change to increase > > > > this? > > > > > > > > If this question is better off posted to the developers list, please > > > > someone, let me know :) > > > > > > > > > > - > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > >- > >To unsubscribe, e-mail: [EMAIL PROTECTED] > >For additional commands, e-mail: [EMAIL PROTECTED] > > LeeAnn Pultz > ExtraView Corporation > [EMAIL PROTECTED] > 831-461-7100 x115 > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > This is not a memory leak in the application - we have hooked up a profiler > to the application and watched the actual memory usage when causing this > issue to happen. We have lots of memory available, are nowhere near the > Xmx limit, and the machine has lots of memory available over and above the > Xmx limit. > > In fact, I can cause the error to happen every single time, simply by > hitting the first web page (login page) of the application - I don't have > to log in, or do any "work". All I have to do to get the error to happen > is hit 18-19 different instances of the web application, watch the Active > thread count go up to 100 and tip over the tomcat. This isn't related to the minProcessors, maxProcessors & acceptCount settings for your connector in your server.xml by any chance? Regards, -- Jason Bainbridge http://kde.org - [EMAIL PROTECTED] Personal Site - http://jasonbainbridge.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
We have no jsps in the application. The index.html page loads our servlet class, and I've put logging of active number of threads at the beginning of the servlet ini() and the very end of the service() methods in that servlet class. The application initializes, but we aren't doing any of the real "work" of rendering dynamic pages or talking to the database at this point. I get the same results whether my Xmx is 84 megs or 512 megs - we're just not using that much memory. At 08:48 AM 4/20/2005, Peter Lin wrote: that sounds very odd. so if I understand correctly. this happens when you hit the login page. I assume this don't happen with a static html page? by any chance are you using jsp tags or some web framework? peter On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > This is not a memory leak in the application - we have hooked up a profiler > to the application and watched the actual memory usage when causing this > issue to happen. We have lots of memory available, are nowhere near the > Xmx limit, and the machine has lots of memory available over and above the > Xmx limit. > > In fact, I can cause the error to happen every single time, simply by > hitting the first web page (login page) of the application - I don't have > to log in, or do any "work". All I have to do to get the error to happen > is hit 18-19 different instances of the web application, watch the Active > thread count go up to 100 and tip over the tomcat. > > We've gone through all of the suggested infomation out on the web - we've > tweaked the Xms, Xmx and Xss parameters - none of these changes made any > difference, since we're not actually running out of memory. I've > reproduced this on 3 different machines, all running Red Hat Linux ES 3.0 - > both with LD_ASSUME_KERNEL 2.4.19 and without, so the Linux threading model > doesn't seem to affect the problem. > > > At 04:16 AM 4/20/2005, you wrote: > >I'm able to go up to 647 threads and 500 concurrent connections with > >5.5.4 without any problems. chances are, it's a memory leak in the > >webapp. only way to know for sure is to profile the application. > > > >peter lin > > > > > >On 4/20/05, Tim Funk <[EMAIL PROTECTED]> wrote: > > > There are no limits in tomcat. > > > http://jakarta.apache.org/tomcat/faq/memory.html > > > > > > -Tim > > > > > > LeeAnn Pultz wrote: > > > > > > > I have a tomcat server with multiple copies of our web application > > > > running. We are running into what seems to be a hard-coded limit on the > > > > number of threads available to the Tomcat application. > > > > > > > > I have added code to our servlet class which prints out the number of > > > > active threads whenever I initialize the servlet. On Red Hat Linux > > > > boxes, whenever we start up enough copies of the servlet to hit 100 > > > > active threads, we reach an OutOfMemoryError - regardless of the Xms/Xmx > > > > settings (memory profiling shows that we have plenty of memory > > > > available). On Windows XP, 50 seems to be the magic number. > > > > > > > > I have tried tweaking Xss parameters, my ulimit command in Linux shows > > > > "unlimited" - and I have reached the same results with Tomcat 3.3.1, > > > > 4.0.28 and 5.0.28 using Java 1.4.1 and 1.4.2. When I use WebLogic 8.1 > > > > on the same Linux box, I reach 241 active threads with no problems > > > > whatsoever (stopped my testing at that number) so it does not seem to be > > > > an o/s limitation, or a java limitation - which leads me to believe > > > > there is something in Tomcat? > > > > > > > > Is the number of threads available to tomcat hard coded somewhere? Is > > > > there a parameter or configuration setting that I can change to increase > > > > this? > > > > > > > > If this question is better off posted to the developers list, please > > > > someone, let me know :) > > > > > > > > > > - > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > >- > >To unsubscribe, e-mail: [EMAIL PROTECTED] > >For additional commands, e-mail: [EMAIL PROTECTED] > > LeeAnn Pultz > ExtraView Corporation > [EMAIL PROTECTED] > 831-461-7100 x115 > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
No, unfortunately, changing those options doesn't seem to make any difference either. On tomcat 3, I even tried manipulating the thread pool settings (since in 4 and 5 those are handled inside tomcat). But I'm seeing the same problems on 3, 4 and 5.0 > In fact, I can cause the error to happen every single time, simply by > hitting the first web page (login page) of the application - I don't have > to log in, or do any "work". All I have to do to get the error to happen > is hit 18-19 different instances of the web application, watch the Active > thread count go up to 100 and tip over the tomcat. This isn't related to the minProcessors, maxProcessors & acceptCount settings for your connector in your server.xml by any chance? Regards, -- Jason Bainbridge http://kde.org - [EMAIL PROTECTED] Personal Site - http://jasonbainbridge.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
that is some really odd behavior coming from Tomcat. I've probably run over 1K benchmarks on Tomcat over the last 3 years and I've yet to see that kind of behavior. Here are some random ideas to try, which I've used in the past. They may not help, but I figure it wouldn't hurt to suggest them. use jmeter to send 250 requests with 25 threads against the index page. then view the status in tomcat 5.5.4. Over time, you should see the number of threads drop to the minimum. Under normal conditions, tomcat will create enough threads to service those threads. after a few minutes, the thread count should drop. If it doesn't, it means something is keeping the connection open. if you see that, it is most likley an application bug. Another idea is to do netstat and see what the status of the connections are. It might be some of the connections are not closing. peter On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > We have no jsps in the application. The index.html page loads our servlet > class, and I've put logging of active number of threads at the beginning of > the servlet ini() and the very end of the service() methods in that servlet > class. The application initializes, but we aren't doing any of the real > "work" of rendering dynamic pages or talking to the database at this point. > > I get the same results whether my Xmx is 84 megs or 512 megs - we're just > not using that much memory. > > > At 08:48 AM 4/20/2005, Peter Lin wrote: > >that sounds very odd. so if I understand correctly. this happens when > >you hit the login page. I assume this don't happen with a static html > >page? by any chance are you using jsp tags or some web framework? > > > >peter > > > > > >On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > > > This is not a memory leak in the application - we have hooked up a > > > profiler > > > to the application and watched the actual memory usage when causing this > > > issue to happen. We have lots of memory available, are nowhere near the > > > Xmx limit, and the machine has lots of memory available over and above the > > > Xmx limit. > > > > > > In fact, I can cause the error to happen every single time, simply by > > > hitting the first web page (login page) of the application - I don't have > > > to log in, or do any "work". All I have to do to get the error to happen > > > is hit 18-19 different instances of the web application, watch the Active > > > thread count go up to 100 and tip over the tomcat. > > > > > > We've gone through all of the suggested infomation out on the web - we've > > > tweaked the Xms, Xmx and Xss parameters - none of these changes made any > > > difference, since we're not actually running out of memory. I've > > > reproduced this on 3 different machines, all running Red Hat Linux ES 3.0 > > > - > > > both with LD_ASSUME_KERNEL 2.4.19 and without, so the Linux threading > > > model > > > doesn't seem to affect the problem. > > > > > > > > > At 04:16 AM 4/20/2005, you wrote: > > > >I'm able to go up to 647 threads and 500 concurrent connections with > > > >5.5.4 without any problems. chances are, it's a memory leak in the > > > >webapp. only way to know for sure is to profile the application. > > > > > > > >peter lin > > > > > > > > > > > >On 4/20/05, Tim Funk <[EMAIL PROTECTED]> wrote: > > > > > There are no limits in tomcat. > > > > > http://jakarta.apache.org/tomcat/faq/memory.html > > > > > > > > > > -Tim > > > > > > > > > > LeeAnn Pultz wrote: > > > > > > > > > > > I have a tomcat server with multiple copies of our web application > > > > > > running. We are running into what seems to be a hard-coded limit > > on the > > > > > > number of threads available to the Tomcat application. > > > > > > > > > > > > I have added code to our servlet class which prints out the number > > > > > > of > > > > > > active threads whenever I initialize the servlet. On Red Hat Linux > > > > > > boxes, whenever we start up enough copies of the servlet to hit 100 > > > > > > active threads, we reach an OutOfMemoryError - regardless of the > > Xms/Xmx > > > > > > settings (memory profiling shows that we have plenty of memory > > > > > > available). On Windows XP, 50 seems to be the magic number. > > > > > > > > > > > > I have tried tweaking Xss parameters, my ulimit command in Linux > > shows > > > > > > "unlimited" - and I have reached the same results with Tomcat 3.3.1, > > > > > > 4.0.28 and 5.0.28 using Java 1.4.1 and 1.4.2. When I use > > WebLogic 8.1 > > > > > > on the same Linux box, I reach 241 active threads with no problems > > > > > > whatsoever (stopped my testing at that number) so it does not > > seem to be > > > > > > an o/s limitation, or a java limitation - which leads me to believe > > > > > > there is something in Tomcat? > > > > > > > > > > > > Is the number of threads available to tomcat hard coded somewhere? > > > > > > Is > > > > > > there a parameter or configuration setting that I can change to > > increase > > > > > > this?
Re: OutOfMemoryError - 100 thread limit?
We did run some code that just makes threads, on the machine and it had no effect - I'll incorporate that code inside a servlet, so we can do this test against tomcat and see if it makes a difference. We just tried the same test against tomcat 5.0 using port 8080 and not using any of the mod_jk connectors, to see if that made a difference - we found that we started with more active threads by uncommenting that section of server.xml (started with 67 active threads rather than 44) and got a bit higher (died at 118 threads rather than 100 threads) so I'm taking that result to show that the actual number of threads allocated to the application remained about the same, with or without the usage of mod_jk. At 09:08 AM 4/20/2005, you wrote: that is some really odd behavior coming from Tomcat. I've probably run over 1K benchmarks on Tomcat over the last 3 years and I've yet to see that kind of behavior. Here are some random ideas to try, which I've used in the past. They may not help, but I figure it wouldn't hurt to suggest them. use jmeter to send 250 requests with 25 threads against the index page. then view the status in tomcat 5.5.4. Over time, you should see the number of threads drop to the minimum. Under normal conditions, tomcat will create enough threads to service those threads. after a few minutes, the thread count should drop. If it doesn't, it means something is keeping the connection open. if you see that, it is most likley an application bug. Another idea is to do netstat and see what the status of the connections are. It might be some of the connections are not closing. peter On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > We have no jsps in the application. The index.html page loads our servlet > class, and I've put logging of active number of threads at the beginning of > the servlet ini() and the very end of the service() methods in that servlet > class. The application initializes, but we aren't doing any of the real > "work" of rendering dynamic pages or talking to the database at this point. > > I get the same results whether my Xmx is 84 megs or 512 megs - we're just > not using that much memory. > > > At 08:48 AM 4/20/2005, Peter Lin wrote: > >that sounds very odd. so if I understand correctly. this happens when > >you hit the login page. I assume this don't happen with a static html > >page? by any chance are you using jsp tags or some web framework? > > > >peter > > > > > >On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > > > This is not a memory leak in the application - we have hooked up a profiler > > > to the application and watched the actual memory usage when causing this > > > issue to happen. We have lots of memory available, are nowhere near the > > > Xmx limit, and the machine has lots of memory available over and above the > > > Xmx limit. > > > > > > In fact, I can cause the error to happen every single time, simply by > > > hitting the first web page (login page) of the application - I don't have > > > to log in, or do any "work". All I have to do to get the error to happen > > > is hit 18-19 different instances of the web application, watch the Active > > > thread count go up to 100 and tip over the tomcat. > > > > > > We've gone through all of the suggested infomation out on the web - we've > > > tweaked the Xms, Xmx and Xss parameters - none of these changes made any > > > difference, since we're not actually running out of memory. I've > > > reproduced this on 3 different machines, all running Red Hat Linux ES 3.0 - > > > both with LD_ASSUME_KERNEL 2.4.19 and without, so the Linux threading model > > > doesn't seem to affect the problem. > > > > > > > > > At 04:16 AM 4/20/2005, you wrote: > > > >I'm able to go up to 647 threads and 500 concurrent connections with > > > >5.5.4 without any problems. chances are, it's a memory leak in the > > > >webapp. only way to know for sure is to profile the application. > > > > > > > >peter lin > > > > > > > > > > > >On 4/20/05, Tim Funk <[EMAIL PROTECTED]> wrote: > > > > > There are no limits in tomcat. > > > > > http://jakarta.apache.org/tomcat/faq/memory.html > > > > > > > > > > -Tim > > > > > > > > > > LeeAnn Pultz wrote: > > > > > > > > > > > I have a tomcat server with multiple copies of our web application > > > > > > running. We are running into what seems to be a hard-coded limit > > on the > > > > > > number of threads available to the Tomcat application. > > > > > > > > > > > > I have added code to our servlet class which prints out the number of > > > > > > active threads whenever I initialize the servlet. On Red Hat Linux > > > > > > boxes, whenever we start up enough copies of the servlet to hit 100 > > > > > > active threads, we reach an OutOfMemoryError - regardless of the > > Xms/Xmx > > > > > > settings (memory profiling shows that we have plenty of memory > > > > > > available). On Windows XP, 50 seems to be the magic number. > > > > > > > > > > > > I have tried tweaki
Re: OutOfMemoryError - 100 thread limit?
Perhaps this will trigger some ideas? The stack trace we get when we get the OutOfMemoryError is : 2005-04-20 09:55:33 StandardWrapperValve[ExtraView]: Allocate exception for servlet ExtraView javax.servlet.ServletException: Error allocating a servlet instance at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:691) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:144) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929) at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705) at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683) at java.lang.Thread.run(Thread.java:536) - Root Cause - java.lang.OutOfMemoryError LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
that looks really bizzare to me. From the exception, it looks like tomcat can't create a new thread to process the request. It makes me think the previous threads that are done are being held by something for some odd reason. Normally the threads should be available again. What is the ramp up time for your tests? in other words, are you throwing 100 concurrent requests at it simultaneously, or ramping it up over a few seconds? peter On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > Perhaps this will trigger some ideas? The stack trace we get when we get > the OutOfMemoryError is : > > 2005-04-20 09:55:33 StandardWrapperValve[ExtraView]: Allocate exception for > servlet ExtraView > javax.servlet.ServletException: Error allocating a servlet instance > at > org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:691) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:144) > at > org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) > at > org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) > at > org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152) > at > org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) > at > org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137) > at > org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118) > at > org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102) > at > org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at > org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) > at > org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) > at > org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929) > at > org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160) > at > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799) > at > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705) > at > org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577) > at > org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683) > at java.lang.Thread.run(Thread.java:536) > - Root Cause - > java.lang.OutOfMemoryError > > LeeAnn Pultz > ExtraView Corporation > [EMAIL PROTECTED] > 831-461-7100 x115 > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
> From: LeeAnn Pultz [mailto:[EMAIL PROTECTED] > Subject: Re: OutOfMemoryError - 100 thread limit? > > Perhaps this will trigger some ideas? The stack trace we get > when we get the OutOfMemoryError is : > > > - Root Cause - > java.lang.OutOfMemoryError It's what comes after this spot in the trace that might be really interesting... Unfortunately, OOME is somewhat of a catch-all - the JVM issues it whenever any OS-based resource is exhausted, as well as when out of heap space. Could be something simple like the limit on the number of open file descriptors being reached. It's also possible you're running out of perm gen space, which is not unusual if you have a lot of classes. Might want to turn on -verbose:gc just to see. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
This was first reported when we had multiple web applications running on one tomcat, and a user would be logged into the site and be using the application fine, and then if another user started working in a new site, they would get the OutOfMemory error when the first tried to load the page of their site. The first user would then start getting exceptions once the OutOfMemory error happened. When we reproduce the error, it doesn't seem to make a difference whether we simply click through a list of links, opening each site in a new window quickly, or whether we pause a few seconds in between opening new sites. So it doesn't necessarily *seem* to be dependent on multiple people firing up their sites at the same time, or over the course of several hours as different people use the system. We've identified that each instance of our application starts up with 3 active threads - we have a monitoring thread, the application thread and the user's current thread. When I start up tomcat, and hit the first site, I see about 44 active threads to start with - this seems to be affected by whether we have the Http10 connector commented out, whether the management port is set up etc. If I run the application through Jbuilder + tomcat, the active count doesn't count the "tomcat' threads, and I start with 3 threads and increase by 3 threads for each new application I open in my browser, the first time. At 10:38 AM 4/20/2005, you wrote: that looks really bizzare to me. From the exception, it looks like tomcat can't create a new thread to process the request. It makes me think the previous threads that are done are being held by something for some odd reason. Normally the threads should be available again. What is the ramp up time for your tests? in other words, are you throwing 100 concurrent requests at it simultaneously, or ramping it up over a few seconds? peter On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > Perhaps this will trigger some ideas? The stack trace we get when we get > the OutOfMemoryError is : > > 2005-04-20 09:55:33 StandardWrapperValve[ExtraView]: Allocate exception for > servlet ExtraView > javax.servlet.ServletException: Error allocating a servlet instance > at > org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:691) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:144) > at > org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) > at > org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) > at > org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152) > at > org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) > at > org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137) > at > org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118) > at > org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102) > at > org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at > org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104) > at > org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520) > at > org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929) > at > org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160) > at > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799) > at > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705) > at > org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577) > at > org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683) > at java.lang.Thread.run(Thread.java:536) > - Root Cause - > java.lang.OutOfMemoryError > > LeeAnn Pultz > ExtraView Corporation > [EMAIL PROTECTED] > 831-461-7100 x115 > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 --
RE: OutOfMemoryError - 100 thread limit?
At 10:43 AM 4/20/2005, you wrote: > From: LeeAnn Pultz [mailto:[EMAIL PROTECTED] > Subject: Re: OutOfMemoryError - 100 thread limit? > > Perhaps this will trigger some ideas? The stack trace we get > when we get the OutOfMemoryError is : > > > - Root Cause - > java.lang.OutOfMemoryError It's what comes after this spot in the trace that might be really interesting... Unfortunately, there's nothing after this :) Unfortunately, OOME is somewhat of a catch-all - the JVM issues it whenever any OS-based resource is exhausted, as well as when out of heap space. Could be something simple like the limit on the number of open file descriptors being reached. It's also possible you're running out of perm gen space, which is not unusual if you have a lot of classes. Might want to turn on -verbose:gc just to see. We did run ulimit and the system says everything is "unlimited". I'm not familiar with "perm gen space" - if I turn on -verbose gc could you please tell me what I might be looking for? thanks! LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
Might want to turn on -verbose:gc just to see. I turned on verbose:gc and ran the test again: INFO: Server startup in 14608 ms Start Init Method. The CONFIG_FILE from web.xml is : WEB-INF/configuration/Configuration.properties Using Configuration FILE: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site1/WEB-INF/configuration/Configuration.properties Successfully created a SesameConfig object from FILE: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site1/WEB-INF/configuration/Configuration.properties SesameProjectDir: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site1/WEB-INF Attempting to Open Log File: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site1/WEB-INF/logs/EVJ.log Successfully finished init in SesameServlet. [GC 20785K->9728K(129728K), 0.0274860 secs] Threads Active : 39 [Full GC 15980K->10340K(129728K), 0.1632550 secs] [GC 22085K->11723K(129728K), 0.0286990 secs] Threads Active : 41 Threads Active : 41 Threads Active : 41 Start Init Method. The CONFIG_FILE from web.xml is : WEB-INF/configuration/Configuration.properties Using Configuration FILE: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site2/WEB-INF/configuration/Configuration.properties Successfully created a SesameConfig object from FILE: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site2/WEB-INF/configuration/Configuration.properties SesameProjectDir: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site2/WEB-INF Attempting to Open Log File: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site2/WEB-INF/logs/EVJ.log Successfully finished init in SesameServlet. [Full GC 23440K->12707K(129728K), 0.2036440 secs] [GC 24483K->13950K(129728K), 0.0181640 secs] Start Init Method. ... more like this Successfully finished init in SesameServlet. [GC 54635K->43725K(129792K), 0.0247320 secs] [GC 55562K->44967K(129792K), 0.0167680 secs] [Full GC 45254K->44666K(129792K), 0.6926680 secs] Threads Active : 93 Threads Active : 93 Start Init Method. The CONFIG_FILE from web.xml is : WEB-INF/configuration/Configuration.properties Using Configuration FILE: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site18/WEB-INF/configuration/Configuration.properties Successfully created a SesameConfig object from FILE: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site18/WEB-INF/configuration/Configuration.properties SesameProjectDir: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site18/WEB-INF Attempting to Open Log File: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site18/WEB-INF/logs/EVJ.log Successfully finished init in SesameServlet. [GC 56506K->45573K(129792K), 0.0164210 secs] [GC 57412K->46857K(129792K), 0.0237680 secs] Threads Active : 96 Threads Active : 96 Start Init Method. The CONFIG_FILE from web.xml is : WEB-INF/configuration/Configuration.properties Using Configuration FILE: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site19/WEB-INF/configuration/Configuration.properties Successfully created a SesameConfig object from FILE: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site19/WEB-INF/configuration/Configuration.properties SesameProjectDir: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site19/WEB-INF Attempting to Open Log File: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site19/WEB-INF/logs/EVJ.log Successfully finished init in SesameServlet. [Full GC 52758K->47299K(129792K), 0.6463800 secs] [Full GC 47308K->47304K(129792K), 0.6400790 secs] [Full GC 47304K->47304K(129792K), 0.6315800 secs] [Full GC 47304K->47042K(129792K), 0.6941910 secs] java.lang.OutOfMemoryError java.lang.OutOfMemoryError Threads Active : 94 Threads Active : 94 [Full GC 48357K->47096K(129792K), 0.6358540 secs] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > This was first reported when we had multiple web applications running on > one tomcat, and a user would be logged into the site and be using the > application fine, and then if another user started working in a new site, > they would get the OutOfMemory error when the first tried to load the page > of their site. The first user would then start getting exceptions once the > OutOfMemory error happened. Okay silly question time... How are you setting -Xms and -Xmx ? Are you sure they are being picked up by Tomcat? Regards, -- Jason Bainbridge http://kde.org - [EMAIL PROTECTED] Personal Site - http://jasonbainbridge.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
the number of threads seems odd. unless your webapp creates a bunch of threads using ServletContextListener, the total thread count should be between 10-15. I have a webapp which creates 10 threads on startup. When tomcat is done and my webapp is loaded, I have a total of 38. if your webapp is creating threads at start, what are they doing? peter lin > Threads Active : 94 > Threads Active : 94 > [Full GC 48357K->47096K(129792K), 0.6358540 secs] > > > LeeAnn Pultz > ExtraView Corporation > [EMAIL PROTECTED] > 831-461-7100 x115 > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
> From: LeeAnn Pultz [mailto:[EMAIL PROTECTED] > Subject: RE: OutOfMemoryError - 100 thread limit? > > I turned on verbose:gc and ran the test again: > [Full GC 48357K->47096K(129792K), 0.6358540 secs] This shows you've given the JVM only 128 MB to work with; looks like you really are out of heap space. You probably want to increase the maximum (-Xmx) considerably. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
We tried a new test :) We added code that spins up 8 threads inside just a plain servlet that doesn't do anything else. When I fire up 10 different "instances" of that servlet, I get hundreds of threads printing out as being active - and no out of memory errors. So we added the same code to our (slightly more complex) application servlet, and did the same thing. Now the reported number of threads is much higher, but I still get an OutOfMemory error at the same number of new instances fired up (between 17-18 sites started up). Active Threads : WHEN STARTING INIT() 617 [GC 57193K->46096K(129792K), 0.0353880 secs] [GC 57936K->47554K(129792K), 0.0238210 secs] [Full GC 49312K->47441K(129792K), 0.6546550 secs] [Full GC 47732K->47129K(129792K), 0.7706500 secs] [Full GC 48310K->47256K(129792K), 0.6425030 secs] [Full GC 47276K->47263K(129792K), 0.6445770 secs] [Full GC 47263K->47263K(129792K), 0.6368020 secs] [Full GC 47263K->47241K(129792K), 0.7775600 secs] java.lang.OutOfMemoryError [Full GC 47295K->47242K(129792K), 0.6301470 secs] [Full GC 47248K->47245K(129792K), 0.6451800 secs] [Full GC 47257K->47250K(129792K), 0.6454320 secs] [Full GC 47271K->47262K(129792K), 0.7056980 secs] java.lang.OutOfMemoryError java.lang.OutOfMemoryError So it looks like it isn't just a thread issue. If I fire up the simple servlet 18 times, I don't get the error. In fact, I can then go on to fire up our main servlet another 17-18 times before we get the OutOfMemory error - this time at 844 threads before it dies. There must be something else going on within our application that is consuming something that the simple servlet doesn't. If anyone has any suggestions on other things that I could check for ? Someone said OOM errors can mean out of file descriptors as well? thanks for all your help so far - I'm really hoping we can figure this out! At 11:20 AM 4/20/2005, you wrote: the number of threads seems odd. unless your webapp creates a bunch of threads using ServletContextListener, the total thread count should be between 10-15. I have a webapp which creates 10 threads on startup. When tomcat is done and my webapp is loaded, I have a total of 38. if your webapp is creating threads at start, what are they doing? peter lin > Threads Active : 94 > Threads Active : 94 > [Full GC 48357K->47096K(129792K), 0.6358540 secs] > > > LeeAnn Pultz > ExtraView Corporation > [EMAIL PROTECTED] > 831-461-7100 x115 > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
> From: LeeAnn Pultz [mailto:[EMAIL PROTECTED] > Subject: Re: OutOfMemoryError - 100 thread limit? > > [Full GC 47271K->47262K(129792K), 0.7056980 secs] > > If anyone has any suggestions on other things that I could > check for ? It still looks like your heap size is too small - 128MB isn't much these days. Assuming you're using a Sun 1.4 JVM or later, try specifying -XX:+PrintGCDetails on the command line to see data about the individual generations of the heap to see which one is becoming exhausted. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
Thanks - I'll try that - we do have CATALINA_OPTS set in catalina.sh of -Xms128m and -Xmx512m. At 02:12 PM 4/20/2005, you wrote: > From: LeeAnn Pultz [mailto:[EMAIL PROTECTED] > Subject: Re: OutOfMemoryError - 100 thread limit? > > [Full GC 47271K->47262K(129792K), 0.7056980 secs] > > If anyone has any suggestions on other things that I could > check for ? It still looks like your heap size is too small - 128MB isn't much these days. Assuming you're using a Sun 1.4 JVM or later, try specifying -XX:+PrintGCDetails on the command line to see data about the individual generations of the heap to see which one is becoming exhausted. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
After you've started your application, do a "ps ax|grep java" and see if that value is really making it into the command line; I'm doubting it is. To set these options, I added a the following the the .bash_profile of my tomcat user: export JAVA_OPTS="-Xmx1000m" -ryan -Original Message- From: LeeAnn Pultz [mailto:[EMAIL PROTECTED] Sent: Wednesday, April 20, 2005 4:35 PM To: Tomcat Users List Subject: RE: OutOfMemoryError - 100 thread limit? Thanks - I'll try that - we do have CATALINA_OPTS set in catalina.sh of -Xms128m and -Xmx512m. At 02:12 PM 4/20/2005, you wrote: > > From: LeeAnn Pultz [mailto:[EMAIL PROTECTED] > > Subject: Re: OutOfMemoryError - 100 thread limit? > > > > [Full GC 47271K->47262K(129792K), 0.7056980 secs] > > > > If anyone has any suggestions on other things that I could > > check for ? > >It still looks like your heap size is too small - 128MB isn't much these >days. Assuming you're using a Sun 1.4 JVM or later, try specifying >-XX:+PrintGCDetails on the command line to see data about the individual >generations of the heap to see which one is becoming exhausted. > > - Chuck > > >THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY >MATERIAL and is thus for use only by the intended recipient. If you >received this in error, please contact the sender and delete the e-mail >and its attachments from all computers. > >- >To unsubscribe, e-mail: [EMAIL PROTECTED] >For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
The memory parameters as well as the other CATALINA_OPTS that we set do appear on the process: [EMAIL PROTECTED] webapps]$ ps ax|grep stage2_java 866 pts/1R 0:10 /home/sesame/stage2_java/bin/java -verbose:gc -server -Xms128m -Xmx512m -Djava.awt.headless=true -Djava.endorsed.dirs=/usr/local/extraview/stage2/tomcat/common/endorsed -classpath /home/sesame/stage2_java/lib/tools.jar:/usr/local/extraview/stage2/tomcat/bin/bootstrap.jar:/usr/local/extraview/ At 03:00 PM 4/20/2005, you wrote: After you've started your application, do a "ps ax|grep java" and see if that value is really making it into the command line; I'm doubting it is. To set these options, I added a the following the the .bash_profile of my tomcat user: export JAVA_OPTS="-Xmx1000m" -ryan -Original Message- From: LeeAnn Pultz [mailto:[EMAIL PROTECTED] Sent: Wednesday, April 20, 2005 4:35 PM To: Tomcat Users List Subject: RE: OutOfMemoryError - 100 thread limit? Thanks - I'll try that - we do have CATALINA_OPTS set in catalina.sh of -Xms128m and -Xmx512m. At 02:12 PM 4/20/2005, you wrote: > > From: LeeAnn Pultz [mailto:[EMAIL PROTECTED] > > Subject: Re: OutOfMemoryError - 100 thread limit? > > > > [Full GC 47271K->47262K(129792K), 0.7056980 secs] > > > > If anyone has any suggestions on other things that I could > > check for ? > >It still looks like your heap size is too small - 128MB isn't much these >days. Assuming you're using a Sun 1.4 JVM or later, try specifying >-XX:+PrintGCDetails on the command line to see data about the individual >generations of the heap to see which one is becoming exhausted. > > - Chuck > > >THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY >MATERIAL and is thus for use only by the intended recipient. If you >received this in error, please contact the sender and delete the e-mail >and its attachments from all computers. > >- >To unsubscribe, e-mail: [EMAIL PROTECTED] >For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
if your application is thread heavy, I would recommend changing it so that one thread can manage several processes. creating 800+ threads is going to hit a scalability and reliability wall very quickly. In general, you want to make sure the number of threads remain constant under constant load. If it doesn't the server will eventually crash as you currently see. I'm guessing the application is using threads to handle async processes, which is good, but spawning new threads with every request isn't going to work. hope that helps peter On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > We tried a new test :) > > We added code that spins up 8 threads inside just a plain servlet that > doesn't do anything else. When I fire up 10 different "instances" of that > servlet, I get hundreds of threads printing out as being active - and no > out of memory errors. > > So we added the same code to our (slightly more complex) application > servlet, and did the same thing. Now the reported number of threads is > much higher, but I still get an OutOfMemory error at the same number of new > instances fired up (between 17-18 sites started up). > > Active Threads : WHEN STARTING INIT() 617 > [GC 57193K->46096K(129792K), 0.0353880 secs] > [GC 57936K->47554K(129792K), 0.0238210 secs] > [Full GC 49312K->47441K(129792K), 0.6546550 secs] > [Full GC 47732K->47129K(129792K), 0.7706500 secs] > [Full GC 48310K->47256K(129792K), 0.6425030 secs] > [Full GC 47276K->47263K(129792K), 0.6445770 secs] > [Full GC 47263K->47263K(129792K), 0.6368020 secs] > [Full GC 47263K->47241K(129792K), 0.7775600 secs] > java.lang.OutOfMemoryError > [Full GC 47295K->47242K(129792K), 0.6301470 secs] > [Full GC 47248K->47245K(129792K), 0.6451800 secs] > [Full GC 47257K->47250K(129792K), 0.6454320 secs] > [Full GC 47271K->47262K(129792K), 0.7056980 secs] > java.lang.OutOfMemoryError > java.lang.OutOfMemoryError > > So it looks like it isn't just a thread issue. > > If I fire up the simple servlet 18 times, I don't get the error. In fact, > I can then go on to fire up our main servlet another 17-18 times before we > get the OutOfMemory error - this time at 844 threads before it dies. > > There must be something else going on within our application that is > consuming something that the simple servlet doesn't. > > If anyone has any suggestions on other things that I could check for ? > Someone said OOM errors can mean out of file descriptors as well? > > thanks for all your help so far - I'm really hoping we can figure this out! > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
I think we've gotten things a bit confused :) The application itself is not thread intensive - our original tests show that each separate copy of the web application started up 3 active threads. Further testing, when we added in debugging code that would spin up dozens of threads just to see if we could cause the tomcat to get the error if we started one application up and had tons of threads, showed that we could get 800+ threads with no problems (no outofmemory errors) and that the problem does not in fact seem to be specifically related to the number of threads the application creates. In fact, regardless of whether each instance of the application spins up 3 threads (real situation) or 10+ (debug code only) threads, we still get the out of memory error not when we hit a certain number of threads, but actually when we hit a certain number of servlets having been initialized. Testing a very simple servlet, I can start up >25 instances of the servlet with no OOM error. Testing our application's servlet, I can only start up 17-18 instances of the servlet before getting an OOM error. At this point I am going through our standard application servlet, trying to isolate what work it does that may cause a problem when we do it 17 or 18 times on one installation of tomcat. At 05:16 PM 4/20/2005, you wrote: if your application is thread heavy, I would recommend changing it so that one thread can manage several processes. creating 800+ threads is going to hit a scalability and reliability wall very quickly. In general, you want to make sure the number of threads remain constant under constant load. If it doesn't the server will eventually crash as you currently see. I'm guessing the application is using threads to handle async processes, which is good, but spawning new threads with every request isn't going to work. hope that helps peter On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > We tried a new test :) > > We added code that spins up 8 threads inside just a plain servlet that > doesn't do anything else. When I fire up 10 different "instances" of that > servlet, I get hundreds of threads printing out as being active - and no > out of memory errors. > > So we added the same code to our (slightly more complex) application > servlet, and did the same thing. Now the reported number of threads is > much higher, but I still get an OutOfMemory error at the same number of new > instances fired up (between 17-18 sites started up). > > Active Threads : WHEN STARTING INIT() 617 > [GC 57193K->46096K(129792K), 0.0353880 secs] > [GC 57936K->47554K(129792K), 0.0238210 secs] > [Full GC 49312K->47441K(129792K), 0.6546550 secs] > [Full GC 47732K->47129K(129792K), 0.7706500 secs] > [Full GC 48310K->47256K(129792K), 0.6425030 secs] > [Full GC 47276K->47263K(129792K), 0.6445770 secs] > [Full GC 47263K->47263K(129792K), 0.6368020 secs] > [Full GC 47263K->47241K(129792K), 0.7775600 secs] > java.lang.OutOfMemoryError > [Full GC 47295K->47242K(129792K), 0.6301470 secs] > [Full GC 47248K->47245K(129792K), 0.6451800 secs] > [Full GC 47257K->47250K(129792K), 0.6454320 secs] > [Full GC 47271K->47262K(129792K), 0.7056980 secs] > java.lang.OutOfMemoryError > java.lang.OutOfMemoryError > > So it looks like it isn't just a thread issue. > > If I fire up the simple servlet 18 times, I don't get the error. In fact, > I can then go on to fire up our main servlet another 17-18 times before we > get the OutOfMemory error - this time at 844 threads before it dies. > > There must be something else going on within our application that is > consuming something that the simple servlet doesn't. > > If anyone has any suggestions on other things that I could check for ? > Someone said OOM errors can mean out of file descriptors as well? > > thanks for all your help so far - I'm really hoping we can figure this out! > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
ahh ok. .. my confusion. back to the problem you see. I've tested tomcat with 30-40 threads without any problems in the past. Even with heavy weight JSTL tags in JSP's, I'm able to go up to 50 threads with tomcat4.1. I'll try hitting tomcat's status servlet with 50 threads tonight and see what happens. peter On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > I think we've gotten things a bit confused :) > > The application itself is not thread intensive - our original tests show > that each separate copy of the web application started up 3 active threads. > > Further testing, when we added in debugging code that would spin up dozens > of threads just to see if we could cause the tomcat to get the error if we > started one application up and had tons of threads, showed that we could > get 800+ threads with no problems (no outofmemory errors) and that the > problem does not in fact seem to be specifically related to the number of > threads the application creates. > > In fact, regardless of whether each instance of the application spins up 3 > threads (real situation) or 10+ (debug code only) threads, we still get the > out of memory error not when we hit a certain number of threads, but > actually when we hit a certain number of servlets having been initialized. > > Testing a very simple servlet, I can start up >25 instances of the servlet > with no OOM error. > Testing our application's servlet, I can only start up 17-18 instances of > the servlet before getting an OOM error. > > At this point I am going through our standard application servlet, trying > to isolate what work it does that may cause a problem when we do it 17 or > 18 times on one installation of tomcat. > > > At 05:16 PM 4/20/2005, you wrote: > >if your application is thread heavy, I would recommend changing it so > >that one thread can manage several processes. creating 800+ threads is > >going to hit a scalability and reliability wall very quickly. In > >general, you want to make sure the number of threads remain constant > >under constant load. If it doesn't the server will eventually crash as > >you currently see. I'm guessing the application is using threads to > >handle async processes, which is good, but spawning new threads with > >every request isn't going to work. > > > >hope that helps > > > >peter > > > > > >On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > > > We tried a new test :) > > > > > > We added code that spins up 8 threads inside just a plain servlet that > > > doesn't do anything else. When I fire up 10 different "instances" of that > > > servlet, I get hundreds of threads printing out as being active - and no > > > out of memory errors. > > > > > > So we added the same code to our (slightly more complex) application > > > servlet, and did the same thing. Now the reported number of threads is > > > much higher, but I still get an OutOfMemory error at the same number of > > > new > > > instances fired up (between 17-18 sites started up). > > > > > > Active Threads : WHEN STARTING INIT() 617 > > > [GC 57193K->46096K(129792K), 0.0353880 secs] > > > [GC 57936K->47554K(129792K), 0.0238210 secs] > > > [Full GC 49312K->47441K(129792K), 0.6546550 secs] > > > [Full GC 47732K->47129K(129792K), 0.7706500 secs] > > > [Full GC 48310K->47256K(129792K), 0.6425030 secs] > > > [Full GC 47276K->47263K(129792K), 0.6445770 secs] > > > [Full GC 47263K->47263K(129792K), 0.6368020 secs] > > > [Full GC 47263K->47241K(129792K), 0.7775600 secs] > > > java.lang.OutOfMemoryError > > > [Full GC 47295K->47242K(129792K), 0.6301470 secs] > > > [Full GC 47248K->47245K(129792K), 0.6451800 secs] > > > [Full GC 47257K->47250K(129792K), 0.6454320 secs] > > > [Full GC 47271K->47262K(129792K), 0.7056980 secs] > > > java.lang.OutOfMemoryError > > > java.lang.OutOfMemoryError > > > > > > So it looks like it isn't just a thread issue. > > > > > > If I fire up the simple servlet 18 times, I don't get the error. In fact, > > > I can then go on to fire up our main servlet another 17-18 times before we > > > get the OutOfMemory error - this time at 844 threads before it dies. > > > > > > There must be something else going on within our application that is > > > consuming something that the simple servlet doesn't. > > > > > > If anyone has any suggestions on other things that I could check for ? > > > Someone said OOM errors can mean out of file descriptors as well? > > > > > > thanks for all your help so far - I'm really hoping we can figure this > > > out! > > > > > > > > > >- > >To unsubscribe, e-mail: [EMAIL PROTECTED] > >For additional commands, e-mail: [EMAIL PROTECTED] > > LeeAnn Pultz > ExtraView Corporation > [EMAIL PROTECTED] > 831-461-7100 x115 > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -
Re: OutOfMemoryError - 100 thread limit?
when you say 50 threads, do you mean 50 separate web applications? My concern right now is that we seem to be limited to 17-18 copies of our web app on a particular installation of tomcat. And that is just hitting the login page - not doing any work. Practically, for applications with many users, we may be limited to 7-10 copies of our application on a tomcat. Maybe I'm expecting too much? Perhaps that's a reasonable number of concurrent webapps that can be run on a tomcat? But if that is the case, it would be good to know what the limiting factor is, if it's not number of threads or amount of memory. thanks for much for your help! At 05:48 PM 4/20/2005, you wrote: ahh ok. .. my confusion. back to the problem you see. I've tested tomcat with 30-40 threads without any problems in the past. Even with heavy weight JSTL tags in JSP's, I'm able to go up to 50 threads with tomcat4.1. I'll try hitting tomcat's status servlet with 50 threads tonight and see what happens. peter On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > I think we've gotten things a bit confused :) > > The application itself is not thread intensive - our original tests show > that each separate copy of the web application started up 3 active threads. > > Further testing, when we added in debugging code that would spin up dozens > of threads just to see if we could cause the tomcat to get the error if we > started one application up and had tons of threads, showed that we could > get 800+ threads with no problems (no outofmemory errors) and that the > problem does not in fact seem to be specifically related to the number of > threads the application creates. > > In fact, regardless of whether each instance of the application spins up 3 > threads (real situation) or 10+ (debug code only) threads, we still get the > out of memory error not when we hit a certain number of threads, but > actually when we hit a certain number of servlets having been initialized. > > Testing a very simple servlet, I can start up >25 instances of the servlet > with no OOM error. > Testing our application's servlet, I can only start up 17-18 instances of > the servlet before getting an OOM error. > > At this point I am going through our standard application servlet, trying > to isolate what work it does that may cause a problem when we do it 17 or > 18 times on one installation of tomcat. > > > At 05:16 PM 4/20/2005, you wrote: > >if your application is thread heavy, I would recommend changing it so > >that one thread can manage several processes. creating 800+ threads is > >going to hit a scalability and reliability wall very quickly. In > >general, you want to make sure the number of threads remain constant > >under constant load. If it doesn't the server will eventually crash as > >you currently see. I'm guessing the application is using threads to > >handle async processes, which is good, but spawning new threads with > >every request isn't going to work. > > > >hope that helps > > > >peter > > > > > >On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > > > We tried a new test :) > > > > > > We added code that spins up 8 threads inside just a plain servlet that > > > doesn't do anything else. When I fire up 10 different "instances" of that > > > servlet, I get hundreds of threads printing out as being active - and no > > > out of memory errors. > > > > > > So we added the same code to our (slightly more complex) application > > > servlet, and did the same thing. Now the reported number of threads is > > > much higher, but I still get an OutOfMemory error at the same number of new > > > instances fired up (between 17-18 sites started up). > > > > > > Active Threads : WHEN STARTING INIT() 617 > > > [GC 57193K->46096K(129792K), 0.0353880 secs] > > > [GC 57936K->47554K(129792K), 0.0238210 secs] > > > [Full GC 49312K->47441K(129792K), 0.6546550 secs] > > > [Full GC 47732K->47129K(129792K), 0.7706500 secs] > > > [Full GC 48310K->47256K(129792K), 0.6425030 secs] > > > [Full GC 47276K->47263K(129792K), 0.6445770 secs] > > > [Full GC 47263K->47263K(129792K), 0.6368020 secs] > > > [Full GC 47263K->47241K(129792K), 0.7775600 secs] > > > java.lang.OutOfMemoryError > > > [Full GC 47295K->47242K(129792K), 0.6301470 secs] > > > [Full GC 47248K->47245K(129792K), 0.6451800 secs] > > > [Full GC 47257K->47250K(129792K), 0.6454320 secs] > > > [Full GC 47271K->47262K(129792K), 0.7056980 secs] > > > java.lang.OutOfMemoryError > > > java.lang.OutOfMemoryError > > > > > > So it looks like it isn't just a thread issue. > > > > > > If I fire up the simple servlet 18 times, I don't get the error. In fact, > > > I can then go on to fire up our main servlet another 17-18 times before we > > > get the OutOfMemory error - this time at 844 threads before it dies. > > > > > > There must be something else going on within our application that is > > > consuming something that the simple servlet doesn't. > > > > > > If anyone has any suggestions on other things that I coul
Re: OutOfMemoryError - 100 thread limit?
I'll try loading up 18 version of my webapp and see if tomcat blows up. should know in a hour or so peter On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > when you say 50 threads, do you mean 50 separate web applications? > > My concern right now is that we seem to be limited to 17-18 copies of our > web app on a particular installation of tomcat. And that > is just hitting the login page - not doing any work. Practically, for > applications with many users, we may be limited to 7-10 copies of our > application on a tomcat. > > Maybe I'm expecting too much? Perhaps that's a reasonable number of > concurrent webapps that can be run on a tomcat? But if that is the case, > it would be good to know what the limiting factor is, if it's not number of > threads or amount of memory. > > thanks for much for your help! > > > At 05:48 PM 4/20/2005, you wrote: > >ahh ok. .. my confusion. > > > >back to the problem you see. I've tested tomcat with 30-40 threads > >without any problems in the past. Even with heavy weight JSTL tags in > >JSP's, I'm able to go up to 50 threads with tomcat4.1. > > > >I'll try hitting tomcat's status servlet with 50 threads tonight and > >see what happens. > > > > > >peter > > > > > >On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > > > I think we've gotten things a bit confused :) > > > > > > The application itself is not thread intensive - our original tests show > > > that each separate copy of the web application started up 3 active > > > threads. > > > > > > Further testing, when we added in debugging code that would spin up dozens > > > of threads just to see if we could cause the tomcat to get the error if we > > > started one application up and had tons of threads, showed that we could > > > get 800+ threads with no problems (no outofmemory errors) and that the > > > problem does not in fact seem to be specifically related to the number of > > > threads the application creates. > > > > > > In fact, regardless of whether each instance of the application spins up 3 > > > threads (real situation) or 10+ (debug code only) threads, we still get > > > the > > > out of memory error not when we hit a certain number of threads, but > > > actually when we hit a certain number of servlets having been initialized. > > > > > > Testing a very simple servlet, I can start up >25 instances of the servlet > > > with no OOM error. > > > Testing our application's servlet, I can only start up 17-18 instances of > > > the servlet before getting an OOM error. > > > > > > At this point I am going through our standard application servlet, trying > > > to isolate what work it does that may cause a problem when we do it 17 or > > > 18 times on one installation of tomcat. > > > > > > > > > At 05:16 PM 4/20/2005, you wrote: > > > >if your application is thread heavy, I would recommend changing it so > > > >that one thread can manage several processes. creating 800+ threads is > > > >going to hit a scalability and reliability wall very quickly. In > > > >general, you want to make sure the number of threads remain constant > > > >under constant load. If it doesn't the server will eventually crash as > > > >you currently see. I'm guessing the application is using threads to > > > >handle async processes, which is good, but spawning new threads with > > > >every request isn't going to work. > > > > > > > >hope that helps > > > > > > > >peter > > > > > > > > > > > >On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > > > > > We tried a new test :) > > > > > > > > > > We added code that spins up 8 threads inside just a plain servlet that > > > > > doesn't do anything else. When I fire up 10 different "instances" > > of that > > > > > servlet, I get hundreds of threads printing out as being active - > > and no > > > > > out of memory errors. > > > > > > > > > > So we added the same code to our (slightly more complex) application > > > > > servlet, and did the same thing. Now the reported number of threads > > > > > is > > > > > much higher, but I still get an OutOfMemory error at the same > > number of new > > > > > instances fired up (between 17-18 sites started up). > > > > > > > > > > Active Threads : WHEN STARTING INIT() 617 > > > > > [GC 57193K->46096K(129792K), 0.0353880 secs] > > > > > [GC 57936K->47554K(129792K), 0.0238210 secs] > > > > > [Full GC 49312K->47441K(129792K), 0.6546550 secs] > > > > > [Full GC 47732K->47129K(129792K), 0.7706500 secs] > > > > > [Full GC 48310K->47256K(129792K), 0.6425030 secs] > > > > > [Full GC 47276K->47263K(129792K), 0.6445770 secs] > > > > > [Full GC 47263K->47263K(129792K), 0.6368020 secs] > > > > > [Full GC 47263K->47241K(129792K), 0.7775600 secs] > > > > > java.lang.OutOfMemoryError > > > > > [Full GC 47295K->47242K(129792K), 0.6301470 secs] > > > > > [Full GC 47248K->47245K(129792K), 0.6451800 secs] > > > > > [Full GC 47257K->47250K(129792K), 0.6454320 secs] > > > > > [Full GC 47271K->47262K(129792K), 0.7056980 secs] > > > > > java.lang.OutOfMemo
Re: OutOfMemoryError - 100 thread limit?
ok, I just tried starting tomcat with the default heap size and 18 instances of my webapp and it got OOME just as you see. If I set my heap to -Xms256m -Xmx512m I'm able to load the 18 webapps just fine. the total memory used after all the webapps are loaded is 152Megs. the total number of webapps is 25 webapps including the default tomcat webapps and a few more. so basically you'll need to set the heap larger enough and it should work. If i do some math, we can calculate a rough estimate for the min heap (154 - 64)/25 = 3.6megs so basically, you'll need to multiply the number of webapps by 4 megs to figure out the heap setting. peter lin On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > when you say 50 threads, do you mean 50 separate web applications? > > My concern right now is that we seem to be limited to 17-18 copies of our > web app on a particular installation of tomcat. And that > is just hitting the login page - not doing any work. Practically, for > applications with many users, we may be limited to 7-10 copies of our > application on a tomcat. > > Maybe I'm expecting too much? Perhaps that's a reasonable number of > concurrent webapps that can be run on a tomcat? But if that is the case, > it would be good to know what the limiting factor is, if it's not number of > threads or amount of memory. > > thanks for much for your help! > > > At 05:48 PM 4/20/2005, you wrote: > >ahh ok. .. my confusion. > > > >back to the problem you see. I've tested tomcat with 30-40 threads > >without any problems in the past. Even with heavy weight JSTL tags in > >JSP's, I'm able to go up to 50 threads with tomcat4.1. > > > >I'll try hitting tomcat's status servlet with 50 threads tonight and > >see what happens. > > > > > >peter > > > > > >On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > > > I think we've gotten things a bit confused :) > > > > > > The application itself is not thread intensive - our original tests show > > > that each separate copy of the web application started up 3 active > > > threads. > > > > > > Further testing, when we added in debugging code that would spin up dozens > > > of threads just to see if we could cause the tomcat to get the error if we > > > started one application up and had tons of threads, showed that we could > > > get 800+ threads with no problems (no outofmemory errors) and that the > > > problem does not in fact seem to be specifically related to the number of > > > threads the application creates. > > > > > > In fact, regardless of whether each instance of the application spins up 3 > > > threads (real situation) or 10+ (debug code only) threads, we still get > > > the > > > out of memory error not when we hit a certain number of threads, but > > > actually when we hit a certain number of servlets having been initialized. > > > > > > Testing a very simple servlet, I can start up >25 instances of the servlet > > > with no OOM error. > > > Testing our application's servlet, I can only start up 17-18 instances of > > > the servlet before getting an OOM error. > > > > > > At this point I am going through our standard application servlet, trying > > > to isolate what work it does that may cause a problem when we do it 17 or > > > 18 times on one installation of tomcat. > > > > > > > > > At 05:16 PM 4/20/2005, you wrote: > > > >if your application is thread heavy, I would recommend changing it so > > > >that one thread can manage several processes. creating 800+ threads is > > > >going to hit a scalability and reliability wall very quickly. In > > > >general, you want to make sure the number of threads remain constant > > > >under constant load. If it doesn't the server will eventually crash as > > > >you currently see. I'm guessing the application is using threads to > > > >handle async processes, which is good, but spawning new threads with > > > >every request isn't going to work. > > > > > > > >hope that helps > > > > > > > >peter > > > > > > > > > > > >On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > > > > > We tried a new test :) > > > > > > > > > > We added code that spins up 8 threads inside just a plain servlet that > > > > > doesn't do anything else. When I fire up 10 different "instances" > > of that > > > > > servlet, I get hundreds of threads printing out as being active - > > and no > > > > > out of memory errors. > > > > > > > > > > So we added the same code to our (slightly more complex) application > > > > > servlet, and did the same thing. Now the reported number of threads > > > > > is > > > > > much higher, but I still get an OutOfMemory error at the same > > number of new > > > > > instances fired up (between 17-18 sites started up). > > > > > > > > > > Active Threads : WHEN STARTING INIT() 617 > > > > > [GC 57193K->46096K(129792K), 0.0353880 secs] > > > > > [GC 57936K->47554K(129792K), 0.0238210 secs] > > > > > [Full GC 49312K->47441K(129792K), 0.6546550 secs] > > > > > [Full GC 47732K->47129K(129792K), 0.7706500 secs] >
RE: OutOfMemoryError - 100 thread limit?
So let me get this straight, LeeAnn is specifying a maximum heap size of 512MB and a minimum size of 128MB. It looks like the heap doesn't get adjusted up when the servlets are initializing? Thus she needs to increase her minimum heapsize, so something like -Xms512m should fix it? -ryan -Original Message- From: Peter Lin [mailto:[EMAIL PROTECTED] Sent: Wednesday, April 20, 2005 8:35 PM To: Tomcat Users List Subject: Re: OutOfMemoryError - 100 thread limit? ok, I just tried starting tomcat with the default heap size and 18 instances of my webapp and it got OOME just as you see. If I set my heap to -Xms256m -Xmx512m I'm able to load the 18 webapps just fine. the total memory used after all the webapps are loaded is 152Megs. the total number of webapps is 25 webapps including the default tomcat webapps and a few more. so basically you'll need to set the heap larger enough and it should work. If i do some math, we can calculate a rough estimate for the min heap (154 - 64)/25 = 3.6megs so basically, you'll need to multiply the number of webapps by 4 megs to figure out the heap setting. peter lin On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > when you say 50 threads, do you mean 50 separate web applications? > > My concern right now is that we seem to be limited to 17-18 copies of our > web app on a particular installation of tomcat. And that > is just hitting the login page - not doing any work. Practically, for > applications with many users, we may be limited to 7-10 copies of our > application on a tomcat. > > Maybe I'm expecting too much? Perhaps that's a reasonable number of > concurrent webapps that can be run on a tomcat? But if that is the case, > it would be good to know what the limiting factor is, if it's not number of > threads or amount of memory. > > thanks for much for your help! > > > At 05:48 PM 4/20/2005, you wrote: > >ahh ok. .. my confusion. > > > >back to the problem you see. I've tested tomcat with 30-40 threads > >without any problems in the past. Even with heavy weight JSTL tags in > >JSP's, I'm able to go up to 50 threads with tomcat4.1. > > > >I'll try hitting tomcat's status servlet with 50 threads tonight and > >see what happens. > > > > > >peter > > > > > >On 4/20/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > > > I think we've gotten things a bit confused :) > > > > > > The application itself is not thread intensive - our original tests show > > > that each separate copy of the web application started up 3 active threads. > > > > > > Further testing, when we added in debugging code that would spin up dozens > > > of threads just to see if we could cause the tomcat to get the error if we > > > started one application up and had tons of threads, showed that we could > > > get 800+ threads with no problems (no outofmemory errors) and that the > > > problem does not in fact seem to be specifically related to the number of > > > threads the application creates. > > > > > > In fact, regardless of whether each instance of the application spins up 3 > > > threads (real situation) or 10+ (debug code only) threads, we still get the > > > out of memory error not when we hit a certain number of threads, but > > > actually when we hit a certain number of servlets having been initialized. > > > > > > Testing a very simple servlet, I can start up >25 instances of the servlet > > > with no OOM error. > > > Testing our application's servlet, I can only start up 17-18 instances of > > > the servlet before getting an OOM error. > > > > > > At this point I am going through our standard application servlet, trying > > > to isolate what work it does that may cause a problem when we do it 17 or > > > 18 times on one installation of tomcat. > > > > > > > > > At 05:16 PM 4/20/2005, you wrote: > > > >if your application is thread heavy, I would recommend changing it so > > > >that one thread can manage several processes. creating 800+ threads is > > > >going to hit a scalability and reliability wall very quickly. In > > > >general, you want to make sure the number of threads remain constant > > > >under constant load. If it doesn't the server will eventually crash as > > > >you currently see. I'm guessing the application is using threads to > > > >handle async processes, which is good, but spawning new threads with > > > >every request isn't going to work. > > > > > > > >hope th
RE: OutOfMemoryError - 100 thread limit?
> From: J. Ryan Earl [mailto:[EMAIL PROTECTED] > Subject: RE: OutOfMemoryError - 100 thread limit? > > So let me get this straight, LeeAnn is specifying a maximum > heap size of 512MB and a minimum size of 128MB. It looks > like the heap doesn't get adjusted up when the servlets are > initializing? Thus she needs to increase her minimum heapsize, > so something like -Xms512m should fix it? Probably not, since I suspect she's running out of perm gen space. IIRC, the perm gen size is calculated from -Xmx, which is already at 512m. Using the PrintGCDetails will tell us for sure. If that shows perm gen space to be exhausted, then adjusting -XX:MaxPermSize should fix it. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
my 2 bits. when I tried to replicate the issue LeeAnn saw, it was pretty clearn the JVM can't resize the heap fast enough to account for the large number of webapps being loaded. simply increasing the initial heap "should" solve the problem. peter On 4/21/05, Caldarale, Charles R <[EMAIL PROTECTED]> wrote: > > From: J. Ryan Earl [mailto:[EMAIL PROTECTED] > > Subject: RE: OutOfMemoryError - 100 thread limit? > > > > So let me get this straight, LeeAnn is specifying a maximum > > heap size of 512MB and a minimum size of 128MB. It looks > > like the heap doesn't get adjusted up when the servlets are > > initializing? Thus she needs to increase her minimum heapsize, > > so something like -Xms512m should fix it? > > Probably not, since I suspect she's running out of perm gen space. > IIRC, the perm gen size is calculated from -Xmx, which is already at > 512m. Using the PrintGCDetails will tell us for sure. If that shows > perm gen space to be exhausted, then adjusting -XX:MaxPermSize should > fix it. > > - Chuck > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you > received this in error, please contact the sender and delete the e-mail > and its attachments from all computers. > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
Peter Lin reproduced and fixed the problem LeeAnn is seeing, and said "If I set my heap to -Xms256m -Xmx512m I'm able to load the 18 webapps just fine. the total memory used after all the webapps are loaded is 152Megs." He didn't mention anything about adjusting the perm gen space. -ryan -Original Message- From: Caldarale, Charles R [mailto:[EMAIL PROTECTED] Sent: Thursday, April 21, 2005 9:19 AM To: Tomcat Users List Subject: RE: OutOfMemoryError - 100 thread limit? > From: J. Ryan Earl [mailto:[EMAIL PROTECTED] > Subject: RE: OutOfMemoryError - 100 thread limit? > > So let me get this straight, LeeAnn is specifying a maximum > heap size of 512MB and a minimum size of 128MB. It looks > like the heap doesn't get adjusted up when the servlets are > initializing? Thus she needs to increase her minimum heapsize, > so something like -Xms512m should fix it? Probably not, since I suspect she's running out of perm gen space. IIRC, the perm gen size is calculated from -Xmx, which is already at 512m. Using the PrintGCDetails will tell us for sure. If that shows perm gen space to be exhausted, then adjusting -XX:MaxPermSize should fix it. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
for what it's worth, adjusting the perm gen space rarely helps. I did a ton of benchmarks back in 2002 using all sorts of -X combinations and it rarely helped. You have to really know the memory allocation pattern to effectively tune perm gen space. adjusting the new eden may help, but again that requires quite a bit of profiling to really understand how an application is allocating memory. peter On 4/21/05, J. Ryan Earl <[EMAIL PROTECTED]> wrote: > Peter Lin reproduced and fixed the problem LeeAnn is seeing, and said "If I > set my heap to -Xms256m -Xmx512m I'm able to load the 18 webapps just fine. > the total memory used after all the webapps are loaded is 152Megs." He > didn't mention anything about adjusting the perm gen space. > > -ryan > > -Original Message- > From: Caldarale, Charles R [mailto:[EMAIL PROTECTED] > Sent: Thursday, April 21, 2005 9:19 AM > To: Tomcat Users List > Subject: RE: OutOfMemoryError - 100 thread limit? > > > From: J. Ryan Earl [mailto:[EMAIL PROTECTED] > > Subject: RE: OutOfMemoryError - 100 thread limit? > > > > So let me get this straight, LeeAnn is specifying a maximum > > heap size of 512MB and a minimum size of 128MB. It looks > > like the heap doesn't get adjusted up when the servlets are > > initializing? Thus she needs to increase her minimum heapsize, > > so something like -Xms512m should fix it? > > Probably not, since I suspect she's running out of perm gen space. > IIRC, the perm gen size is calculated from -Xmx, which is already at > 512m. Using the PrintGCDetails will tell us for sure. If that shows > perm gen space to be exhausted, then adjusting -XX:MaxPermSize should > fix it. > > - Chuck > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you > received this in error, please contact the sender and delete the e-mail > and its attachments from all computers. > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
> From: J. Ryan Earl [mailto:[EMAIL PROTECTED] > Subject: RE: OutOfMemoryError - 100 thread limit? > > Peter Lin reproduced and fixed the problem LeeAnn is seeing, > and said "If I set my heap to -Xms256m -Xmx512m I'm able to > load the 18 webapps just fine. the total memory used after > all the webapps are loaded is 152Megs." He didn't mention > anything about adjusting the perm gen space. Nor is he running the same webapps that LeeAnn is using (as far as I know), so that's a bit of an apples and oranges comparison. Also, his conjecture that the JVM "couldn't keep up" doesn't fit with my understanding of object allocation and GC in a HotSpot JVM. If needed, the JVM simply suspends all the application threads and does a full GC. Again, the -XX:+PrintGCDetails will tell us what's really going on. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
I may not be running the same app LeeAnn is running, but my webapp is fairly heavy and creates several threads per webapp. Even if it's not an apples-to-apples comparison, it's still a worth while trick to try. if it doesn't work, then back to square one :) if it does, then LeeAnn can go back to work. peter On 4/21/05, Caldarale, Charles R <[EMAIL PROTECTED]> wrote: > > From: J. Ryan Earl [mailto:[EMAIL PROTECTED] > > Subject: RE: OutOfMemoryError - 100 thread limit? > > > > Peter Lin reproduced and fixed the problem LeeAnn is seeing, > > and said "If I set my heap to -Xms256m -Xmx512m I'm able to > > load the 18 webapps just fine. the total memory used after > > all the webapps are loaded is 152Megs." He didn't mention > > anything about adjusting the perm gen space. > > Nor is he running the same webapps that LeeAnn is using (as far as I > know), so that's a bit of an apples and oranges comparison. Also, his > conjecture that the JVM "couldn't keep up" doesn't fit with my > understanding of object allocation and GC in a HotSpot JVM. If needed, > the JVM simply suspends all the application threads and does a full GC. > Again, the -XX:+PrintGCDetails will tell us what's really going on. > > - Chuck > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you > received this in error, please contact the sender and delete the e-mail > and its attachments from all computers. > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
> From: Peter Lin [mailto:[EMAIL PROTECTED] > Subject: Re: OutOfMemoryError - 100 thread limit? > > if it doesn't work, then back to square one :) It would be even better to get some real data with the -XX:+PrintGCDetails, so we can stop speculating... - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
I, for one, am awaiting LeeAnn's response with abated breath. =) -ryan -Original Message- From: Caldarale, Charles R [mailto:[EMAIL PROTECTED] Sent: Thursday, April 21, 2005 9:53 AM To: Tomcat Users List Subject: RE: OutOfMemoryError - 100 thread limit? > From: Peter Lin [mailto:[EMAIL PROTECTED] > Subject: Re: OutOfMemoryError - 100 thread limit? > > if it doesn't work, then back to square one :) It would be even better to get some real data with the -XX:+PrintGCDetails, so we can stop speculating... - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
Good Morning everyone :) Apparently I'm one of the few on the west coast - sorry for the delay this morning. I added the PrintGCDetails line to my CATALINA_OPTS and got a bit more information - although I'm afraid I'm not sure if this is the information Chuck was looking for? My catalina.out file showed the following lines just before tomcat tipped over at 17 sites: Attempting to Open Log File: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site17/WEB-INF/logs/EVJ.log Successfully finished init in SesameServlet. [Full GC [Tenured: 43103K->45066K(116544K), 0.6647620 secs] 55017K->45066K(129792K), 0.6648170 secs] [GC [DefNew: 11826K->1236K(13248K), 0.0184770 secs] 56893K->46303K(129792K), 0.0185260 secs] Start Init Method. The CONFIG_FILE from web.xml is : WEB-INF/configuration/Configuration.properties Using Configuration FILE: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site18/WEB-INF/configuration/Configuration.properties Successfully created a SesameConfig object from FILE: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site18/WEB-INF/configuration/Configuration.properties SesameProjectDir: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site18/WEB-INF Attempting to Open Log File: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site18/WEB-INF/logs/EVJ.log Successfully finished init in SesameServlet. [GC [DefNew: 13076K->686K(13248K), 0.0199470 secs] 58143K->46974K(129792K), 0.0199950 secs] [GC [DefNew: 12523K->1408K(13248K), 0.0300140 secs] 58810K->48267K(129792K), 0.0300740 secs] [Full GC [Tenured: 46859K->48385K(116544K), 0.6326870 secs] 50745K->48385K(129792K), 0.6327540 secs] [Full GC [Tenured: 48385K->47888K(116544K), 0.7671290 secs] 48385K->47888K(129792K), 0.7671810 secs] [Full GC [Tenured: 47888K->47924K(116544K), 0.6296020 secs] 48999K->47924K(129792K), 0.6296660 secs] [Full GC [Tenured: 47924K->47930K(116544K), 0.6508840 secs] 47939K->47930K(129792K), 0.6509620 secs] [Full GC [Tenured: 47930K->47930K(116544K), 0.6475860 secs] 47931K->47930K(129792K), 0.6476490 secs] [Full GC [Tenured: 47930K->47912K(116544K), 0.7901650 secs] 47932K->47912K(129792K), 0.7902320 secs] I will try increasing the Xms parameter now, although I believe we did try that earlier in the debugging process. More in a bit. At 08:25 AM 4/21/2005, you wrote: I, for one, am awaiting LeeAnn's response with abated breath. =) -ryan -Original Message- From: Caldarale, Charles R [mailto:[EMAIL PROTECTED] Sent: Thursday, April 21, 2005 9:53 AM To: Tomcat Users List Subject: RE: OutOfMemoryError - 100 thread limit? > From: Peter Lin [mailto:[EMAIL PROTECTED] > Subject: Re: OutOfMemoryError - 100 thread limit? > > if it doesn't work, then back to square one :) It would be even better to get some real data with the -XX:+PrintGCDetails, so we can stop speculating... - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
Ok, this time I used -Xms512m -Xmx512m with the same results - Successfully created a SesameConfig object from FILE: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site19/WEB-INF/configuration/Configuration.properties SesameProjectDir: /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site19/WEB-INF [Full GC [Tenured: 45429K->48110K(466048K), 0.6853640 secs] 80819K->48110K(518464K), 0.6854250 secs] [Full GC [Tenured: 48110K->47709K(466048K), 0.7746580 secs] 48119K->47709K(518464K), 0.7747100 secs] Right on the 18th site spin-up - I'm not getting OutOfMemory errors in the catalina.out trace, but the servlet didn't load and gave me an error in the browser: pe Exception report message description The server encountered an internal error () that prevented it from fulfilling this request. exception javax.servlet.ServletException: Servlet.init() for servlet ExtraView threw exception org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118) org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160) org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:300) org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:374) org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:743) org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:675) org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:866) org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683) java.lang.Thread.run(Thread.java:536) root cause java.lang.OutOfMemoryError note The full stack trace of the root cause is available in the Apache Tomcat/5.0.28 logs. At 08:25 AM 4/21/2005, you wrote: I, for one, am awaiting LeeAnn's response with abated breath. =) -ryan -Original Message- From: Caldarale, Charles R [mailto:[EMAIL PROTECTED] Sent: Thursday, April 21, 2005 9:53 AM To: Tomcat Users List Subject: RE: OutOfMemoryError - 100 thread limit? > From: Peter Lin [mailto:[EMAIL PROTECTED] > Subject: Re: OutOfMemoryError - 100 thread limit? > > if it doesn't work, then back to square one :) It would be even better to get some real data with the -XX:+PrintGCDetails, so we can stop speculating... - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115
Re: OutOfMemoryError - 100 thread limit?
wow, your webapp must be pretty heavy and have lots of jar and/or classes. The behavior that I saw was that 16 of the webapps would load with the default JVM heap settings. When I went to Tomcat's manager page, and click "start", I got a similar OOME and the webapp would not start. At that point, there was nothing I could do to get it to start. once I shut down tomcat, change the heap and restart, tomcat was able to load all the webapps. Here's a trick that might help. Start tomcat, but with only 10 copies of your webapp. look at the total memory used and substract 64meg. Divide that number by 10 and see what the memory requirements are for each webapp. (Total mem - 64mb ) / number of webapps = mem per webapp. once you have that, you can figure what you really need. if the total for 18 webapps is less than 512Mb, it means something else is happening, which means you can rule out Tomcat as the cause. Since all tomcat does is start the webapp and call the servletContextListeners, it really isn't doing much. hope that helps peter On 4/21/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > Ok, this time I used -Xms512m -Xmx512m with the same results - > > Successfully created a SesameConfig object from FILE: > /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site19/WEB-INF/configuration/Configuration.properties > SesameProjectDir: > /usr/local/extraview/stage2/jakarta-tomcat-5.0.28/webapps/site19/WEB-INF > [Full GC [Tenured: 45429K->48110K(466048K), 0.6853640 secs] > 80819K->48110K(518464K), 0.6854250 secs] > [Full GC [Tenured: 48110K->47709K(466048K), 0.7746580 secs] > 48119K->47709K(518464K), 0.7747100 secs] > > Right on the 18th site spin-up - I'm not getting OutOfMemory errors in the > catalina.out trace, but the servlet didn't load and gave me an error in the > browser: > > pe Exception report > > message > > description The server encountered an internal error () that prevented it > from fulfilling this request. > > exception > > javax.servlet.ServletException: Servlet.init() for servlet ExtraView threw > exception > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118) > > org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160) > org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:300) > org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:374) > org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:743) > > org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:675) > org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:866) > > org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683) > java.lang.Thread.run(Thread.java:536) > > root cause > > java.lang.OutOfMemoryError > > note The full stack trace of the root cause is available in the Apache > Tomcat/5.0.28 logs. > > > At 08:25 AM 4/21/2005, you wrote: > >I, for one, am awaiting LeeAnn's response with abated breath. =) > > > >-ryan > > > >-Original Message- > >From: Caldarale, Charles R [mailto:[EMAIL PROTECTED] > >Sent: Thursday, April 21, 2005 9:53 AM > >To: Tomcat Users List > >Subject: RE: OutOfMemoryError - 100 thread limit? > > > > > > > From: Peter Lin [mailto:[EMAIL PROTECTED] > > > Subject: Re: OutOfMemoryError - 100 thread limit? > > > > > > if it doesn't work, then back to square one :) > > > >It would be even better to get some real data with the > >-XX:+PrintGCDetails, so we can stop speculating... > > > > - Chuck > > > > > >THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > >MATERIAL and is thus for use only by the intended recipient. If you > >received this in error, please contact the sender and delete the e-mail > >and its attachments from all computers. > > > >- > >To unsubscribe, e-mail: [EMAIL PROTECTED] > >For additional commands, e-mail: [EMAIL PROTECTED] > > > > > >- > >To unsubscribe, e-mail: [EMAIL PROTECTED] > >For additional commands, e-mail: [EMAIL PROTECTED] > > LeeAnn Pultz > ExtraView Corporation > [EMAIL PROTECTED] > 831-461-7100 x115 > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
silly question, does this webapp have like thousands of JSP and servlets and preload the JSP's? peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
Not a silly question :) We have 1 servlet that is the main entry point, everything else is done with standard java classes. We have no jsp's. At 09:13 AM 4/21/2005, you wrote: silly question, does this webapp have like thousands of JSP and servlets and preload the JSP's? peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
Possible success! I went back to my weblogic installation which I was successfully able to open up >25 sites to see what it was doing when starting up. It had -XX:MaxPermSize=128m in the process description. I added that to my catalina_opts and I got past the 18 site limitation of just firing them up. Can anyone give me any information on what this does? Is this the "perm gen" that Chuck mentioned before? Are there any caveats to setting this memory option? thanks! LeeAnn At 09:13 AM 4/21/2005, you wrote: silly question, does this webapp have like thousands of JSP and servlets and preload the JSP's? peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
glad I was wrong and you found a solution. http://java.sun.com/docs/hotspot/VMOptions.html I'm just guessing here, but by forcing the maxPermSize to 128, it leaves more space for the eden and prevents classes from getting promoted to perm. Atleast that's logical explanation I can think of. Another way to get the heap to resize in larger increments and prevent it from getting smaller is to set the -XX:MinHeapFreeRatio. peter lin On 4/21/05, LeeAnn Pultz <[EMAIL PROTECTED]> wrote: > Possible success! > > I went back to my weblogic installation which I was successfully able to > open up >25 sites to see what it was doing when starting up. > > It had -XX:MaxPermSize=128m in the process description. > > I added that to my catalina_opts and I got past the 18 site limitation of > just firing them up. > > Can anyone give me any information on what this does? Is this the "perm > gen" that Chuck mentioned before? Are there any caveats to setting this > memory option? > > thanks! > > LeeAnn > > > At 09:13 AM 4/21/2005, you wrote: > >silly question, does this webapp have like thousands of JSP and > >servlets and preload the JSP's? > > > >peter > > > >- > >To unsubscribe, e-mail: [EMAIL PROTECTED] > >For additional commands, e-mail: [EMAIL PROTECTED] > > LeeAnn Pultz > ExtraView Corporation > [EMAIL PROTECTED] > 831-461-7100 x115 > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
This is a really interesting and confusing problem... Can you tell how much memory the OS is actually committing to Java? ie Check the resident stable set (RSS) = how much of the virtual memory space has been physically commited to memory. # ps axl|grep java 0 501 2623 1 21 0 1264252 79112 - Sl ? 0:09 /usr/java/jre1.5.0_02/bin/java -Xmx1000m The above says I have 1264252KB allocated but only 79112KB committed to physical memory. I'm curious what yours looks like. I know the the JVM reported it was using less memory, but I'd be curious to see what the OS thinks. Maybe even try raising your maximum heap size to say 1024MB? Which JVM are you using again? Which other JVMs have you tried? -ryan PS I just checked a production instance, and it looks like: $ ps axl|grep java 0 101 24243 1 16 0 2275516 906312 322497 Sl ?465:48 /opt/sun-jdk-1.4.2.07/bin/java ~900MB of over 2GB of virtual memory commited, it's apparently got quite a few threads running: $ ps ax -L |grep java|wc -l 2016 How does your application compare when it runs out of memory? -Original Message- From: LeeAnn Pultz [mailto:[EMAIL PROTECTED] Sent: Thursday, April 21, 2005 11:21 AM To: Tomcat Users List Subject: Re: OutOfMemoryError - 100 thread limit? Not a silly question :) We have 1 servlet that is the main entry point, everything else is done with standard java classes. We have no jsp's. At 09:13 AM 4/21/2005, you wrote: >silly question, does this webapp have like thousands of JSP and >servlets and preload the JSP's? > >peter > >- >To unsubscribe, e-mail: [EMAIL PROTECTED] >For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
> From: LeeAnn Pultz [mailto:[EMAIL PROTECTED] > Subject: Re: OutOfMemoryError - 100 thread limit? > > It had -XX:MaxPermSize=128m in the process description. Ahah, as they say. > Can anyone give me any information on what this does? Is > this the "perm gen" that Chuck mentioned before? Yes, this controls the size of the permanent generation. The permanent generation portion of the heap is where class objects (instances of java.lang.Class) are allocated. If you have a lot of classes - which app servers frequently do - the size of this space needs to be increased. A quick scan of the 1.4.2 HotSpot source shows the default value to be 64m. > Are there any caveats to setting this memory option? Other than it subtracts from the total heap space for normal object, no. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
Awesome, I'm glad that was solved as I'm about to do the same thing with our application: running multiple instances of the application on a single tomcat instance housing many virtual hosts. Thanks for the http://java.sun.com/docs/hotspot/VMOptions.html link Peter. -ryan -Original Message- From: Caldarale, Charles R [mailto:[EMAIL PROTECTED] Sent: Thursday, April 21, 2005 11:52 AM To: Tomcat Users List Subject: RE: OutOfMemoryError - 100 thread limit? > From: LeeAnn Pultz [mailto:[EMAIL PROTECTED] > Subject: Re: OutOfMemoryError - 100 thread limit? > > It had -XX:MaxPermSize=128m in the process description. Ahah, as they say. > Can anyone give me any information on what this does? Is > this the "perm gen" that Chuck mentioned before? Yes, this controls the size of the permanent generation. The permanent generation portion of the heap is where class objects (instances of java.lang.Class) are allocated. If you have a lot of classes - which app servers frequently do - the size of this space needs to be increased. A quick scan of the 1.4.2 HotSpot source shows the default value to be 64m. > Are there any caveats to setting this memory option? Other than it subtracts from the total heap space for normal object, no. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
> From: Peter Lin [mailto:[EMAIL PROTECTED] > Subject: Re: OutOfMemoryError - 100 thread limit? > > I'm just guessing here, but by forcing the maxPermSize to 128, it > leaves more space for the eden and prevents classes from getting > promoted to perm. I think you're confusing tenured with perm. Nothing is ever promoted to perm - only known long-lived objects (primarily instances of java.lang.Class) are allocated in there, and no objects ever migrate to or from perm space. Regular objects are normally allocated in eden space (young gen), and if they live long enough, are migrated to the tenured area (old gen) during a GC. Increasing the perm size actually reduces, not increases, the amount of eden space available since the total of all three generations (young, old, perm) must come from the -Xmx setting. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
glad I can help. it's the little odd edge case problems that I find most interesting. it was fun to figure why that behavior was occuring. peter On 4/21/05, J. Ryan Earl <[EMAIL PROTECTED]> wrote: > Awesome, I'm glad that was solved as I'm about to do the same thing with our > application: running multiple instances of the application on a single > tomcat instance housing many virtual hosts. > > Thanks for the http://java.sun.com/docs/hotspot/VMOptions.html link Peter. > > -ryan > > -Original Message- > From: Caldarale, Charles R [mailto:[EMAIL PROTECTED] > Sent: Thursday, April 21, 2005 11:52 AM > To: Tomcat Users List > Subject: RE: OutOfMemoryError - 100 thread limit? > > > From: LeeAnn Pultz [mailto:[EMAIL PROTECTED] > > Subject: Re: OutOfMemoryError - 100 thread limit? > > > > It had -XX:MaxPermSize=128m in the process description. > > Ahah, as they say. > > > Can anyone give me any information on what this does? Is > > this the "perm gen" that Chuck mentioned before? > > Yes, this controls the size of the permanent generation. The permanent > generation portion of the heap is where class objects (instances of > java.lang.Class) are allocated. If you have a lot of classes - which > app servers frequently do - the size of this space needs to be > increased. A quick scan of the 1.4.2 HotSpot source shows the default > value to be 64m. > > > Are there any caveats to setting this memory option? > > Other than it subtracts from the total heap space for normal object, no. > > - Chuck > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you > received this in error, please contact the sender and delete the e-mail > and its attachments from all computers. > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
yup you're right, silly me. ignore my brain dead remark. as long as the problem gets solved, I don't mind looking stupid. so it would appear by setting the PermSize, the jvm is pushing all java.lang.Class instances to Perm rather than loading them in eden and then promoting them to tenured. If you run verbosegc again, the gc interval should be longer than previous settings. I can't think of any real trade-offs of setting Perm to 128 other than using more RAM. If I remember correctly Perm is also where static stuff goes. peter On 4/21/05, Caldarale, Charles R <[EMAIL PROTECTED]> wrote: > > From: Peter Lin [mailto:[EMAIL PROTECTED] > > Subject: Re: OutOfMemoryError - 100 thread limit? > > > > I'm just guessing here, but by forcing the maxPermSize to 128, it > > leaves more space for the eden and prevents classes from getting > > promoted to perm. > > I think you're confusing tenured with perm. Nothing is ever promoted to > perm - only known long-lived objects (primarily instances of > java.lang.Class) are allocated in there, and no objects ever migrate to > or from perm space. Regular objects are normally allocated in eden > space (young gen), and if they live long enough, are migrated to the > tenured area (old gen) during a GC. > > Increasing the perm size actually reduces, not increases, the amount of > eden space available since the total of all three generations (young, > old, perm) must come from the -Xmx setting. > > - Chuck > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you > received this in error, please contact the sender and delete the e-mail > and its attachments from all computers. > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
Thanks everyone! I really appreciate all your ideas and input and experience! I still am not completely clear on eden, gen, etc - obviously I need to go research how memory allocation is done in java. But, after setting the maxPermSize, I was able to log in and do work on >18 sites which is more than I could do before :) We will be scheduling some load testing next week to see if 128 is the right number for our real life problem, but at least I can see the light at the end of the tunnel now! thank you all again for all your help! At 10:35 AM 4/21/2005, you wrote: yup you're right, silly me. ignore my brain dead remark. as long as the problem gets solved, I don't mind looking stupid. so it would appear by setting the PermSize, the jvm is pushing all java.lang.Class instances to Perm rather than loading them in eden and then promoting them to tenured. If you run verbosegc again, the gc interval should be longer than previous settings. I can't think of any real trade-offs of setting Perm to 128 other than using more RAM. If I remember correctly Perm is also where static stuff goes. peter On 4/21/05, Caldarale, Charles R <[EMAIL PROTECTED]> wrote: > > From: Peter Lin [mailto:[EMAIL PROTECTED] > > Subject: Re: OutOfMemoryError - 100 thread limit? > > > > I'm just guessing here, but by forcing the maxPermSize to 128, it > > leaves more space for the eden and prevents classes from getting > > promoted to perm. > > I think you're confusing tenured with perm. Nothing is ever promoted to > perm - only known long-lived objects (primarily instances of > java.lang.Class) are allocated in there, and no objects ever migrate to > or from perm space. Regular objects are normally allocated in eden > space (young gen), and if they live long enough, are migrated to the > tenured area (old gen) during a GC. > > Increasing the perm size actually reduces, not increases, the amount of > eden space available since the total of all three generations (young, > old, perm) must come from the -Xmx setting. > > - Chuck > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you > received this in error, please contact the sender and delete the e-mail > and its attachments from all computers. > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] LeeAnn Pultz ExtraView Corporation [EMAIL PROTECTED] 831-461-7100 x115 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
> From: Peter Lin [mailto:[EMAIL PROTECTED] > Subject: Re: OutOfMemoryError - 100 thread limit? > > so it would appear by setting the PermSize, the jvm is pushing all > java.lang.Class instances to Perm rather than loading them in eden and > then promoting them to tenured. Instances of java.lang.Class are always allocated in the perm gen, never in an eden space, regardless of any heap or gen size settings. > I can't think of any real trade-offs of setting Perm to 128 > other than using more RAM. Unless you increase -Xmx correspondingly, enlarging perm space will reduce the amount available for regular objects. Generally, if you have the memory, increase -Xmx to whatever you can before swapping starts. > If I remember correctly Perm is also where static stuff goes. I believe that's true, since static data is associated with java.lang.Class instances. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
On 4/21/05, Caldarale, Charles R <[EMAIL PROTECTED]> wrote: > > From: Peter Lin [mailto:[EMAIL PROTECTED] > > Subject: Re: OutOfMemoryError - 100 thread limit? > > > > so it would appear by setting the PermSize, the jvm is pushing all > > java.lang.Class instances to Perm rather than loading them in eden and > > then promoting them to tenured. > > Instances of java.lang.Class are always allocated in the perm gen, never > in an eden space, regardless of any heap or gen size settings. logically, it makes sense that all java.lang.Class classes are loaded in Perm. But the result on the behavior LeeAnn saw is a bit perplexing. I'm just exploring here, so if it doesn't make sense, it's cuz I'm making wild guesses. ie, talking out of my rear. the default Perm is 64M. If LeeAnn's webapp have a large number of classes, maybe there's too many; therefore forcing the VM to resize the perm. Doing that, the VM probably is allocating a big block and copying all the data over. I believe that is much heavier task than an incremental GC and probably equally costly as a full GC. So once the new Perm is ready, the VM should make the old block available. So the end result of setting -XX:MaxPermSize is it avoid resizing Perm, which allows Tomcat to load the webapps quickly. Even though setting max Perm to 128 reduces the overall space for new/old generation, it results in better performance because the VM doesn't have to move perm? of course this is assuming the VM prefers to have a continuous block for Perm memory. > I believe that's true, since static data is associated with > java.lang.Class instances. > > - Chuck > peter - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: OutOfMemoryError - 100 thread limit?
> From: Peter Lin [mailto:[EMAIL PROTECTED] > Subject: Re: OutOfMemoryError - 100 thread limit? > > the default Perm is 64M. If LeeAnn's webapp have a large number of > classes, maybe there's too many; therefore forcing the VM to resize > the perm. IIRC (haven't looked at core HotSpot code in detail since 1.4.1 days), the virtual address boundary between perm and the other generations is fixed at JVM initialization and never moves. The perm gen does go through GC just like the other gens, but no memory ever moves between the perm and any other space, and the perm gen maximum virtual address range never changes. The HotSpot JVM allocates, but does not commit, all of the -Xmx space at startup in one contiguous block (although there were rumors of that changing in later JVMs). Only the -Xms portion is committed, split up into three regions - young, old, and perm. When a region fills and GC doesn't reduce the usage significantly, additional pages are committed in whichever region needs expanding. I think what was happening is that LeeAnn simply had more classes than would fit in 64 MB. Nothing to do with threads, just the total number of classes in all the applications added together. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
We're battling this same problem at our client's site. I think I've identified the same cause (running out of Perm Gen space) but I have to prove it to them first. Hooking a profiler into the Tomcat instance should clearly show this but they don't have one handy. Moving some of the applications to a second instance of Tomcat should also solve this problem. -- Virtually, Andre Van Klaveren Architect III, SCP Enterprise Transformation Services Unisys Corporation On 4/21/05, Caldarale, Charles R <[EMAIL PROTECTED]> wrote: > > From: Peter Lin [mailto:[EMAIL PROTECTED] > > Subject: Re: OutOfMemoryError - 100 thread limit? > > > > the default Perm is 64M. If LeeAnn's webapp have a large number of > > classes, maybe there's too many; therefore forcing the VM to resize > > the perm. > > IIRC (haven't looked at core HotSpot code in detail since 1.4.1 days), > the virtual address boundary between perm and the other generations is > fixed at JVM initialization and never moves. The perm gen does go > through GC just like the other gens, but no memory ever moves between > the perm and any other space, and the perm gen maximum virtual address > range never changes. > > The HotSpot JVM allocates, but does not commit, all of the -Xmx space at > startup in one contiguous block (although there were rumors of that > changing in later JVMs). Only the -Xms portion is committed, split up > into three regions - young, old, and perm. When a region fills and GC > doesn't reduce the usage significantly, additional pages are committed > in whichever region needs expanding. > > I think what was happening is that LeeAnn simply had more classes than > would fit in 64 MB. Nothing to do with threads, just the total number > of classes in all the applications added together. > > - Chuck > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you > received this in error, please contact the sender and delete the e-mail > and its attachments from all computers. > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: OutOfMemoryError - 100 thread limit?
the advice given in previous message to dump the Perm gen info out should tell you. running tomcat in a profiler may not show it. I use optimizeIt frequently and it doesn't show perm generation. peter On 4/27/05, Andre Van Klaveren <[EMAIL PROTECTED]> wrote: > We're battling this same problem at our client's site. I think I've > identified the same cause (running out of Perm Gen space) but I have > to prove it to them first. Hooking a profiler into the Tomcat > instance should clearly show this but they don't have one handy. > Moving some of the applications to a second instance of Tomcat should > also solve this problem. > > -- > Virtually, > Andre Van Klaveren > Architect III, SCP > Enterprise Transformation Services > Unisys Corporation > > On 4/21/05, Caldarale, Charles R <[EMAIL PROTECTED]> wrote: > > > From: Peter Lin [mailto:[EMAIL PROTECTED] > > > Subject: Re: OutOfMemoryError - 100 thread limit? > > > > > > the default Perm is 64M. If LeeAnn's webapp have a large number of > > > classes, maybe there's too many; therefore forcing the VM to resize > > > the perm. > > > > IIRC (haven't looked at core HotSpot code in detail since 1.4.1 days), > > the virtual address boundary between perm and the other generations is > > fixed at JVM initialization and never moves. The perm gen does go > > through GC just like the other gens, but no memory ever moves between > > the perm and any other space, and the perm gen maximum virtual address > > range never changes. > > > > The HotSpot JVM allocates, but does not commit, all of the -Xmx space at > > startup in one contiguous block (although there were rumors of that > > changing in later JVMs). Only the -Xms portion is committed, split up > > into three regions - young, old, and perm. When a region fills and GC > > doesn't reduce the usage significantly, additional pages are committed > > in whichever region needs expanding. > > > > I think what was happening is that LeeAnn simply had more classes than > > would fit in 64 MB. Nothing to do with threads, just the total number > > of classes in all the applications added together. > > > > - Chuck > > > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > > MATERIAL and is thus for use only by the intended recipient. If you > > received this in error, please contact the sender and delete the e-mail > > and its attachments from all computers. > > > > - > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
