RE: Memory creep
Thanks for the info Howard. It's working out very well so far - very happy with 5.2.1. Regards, Jim. -Original Message- From: Howard Lewis Ship [mailto:hls...@gmail.com] Sent: 03 November 2010 16:40 To: Tapestry users Subject: Re: Memory creep FYI: Using either jconsole of VisualVM you can force a GC remotely. In general, Tapestry is very good about memory management (though it likes to use a lot of it). There are a few user coding errors that can "trap" objects that should be reclaimed, but you have to work a bit hard to hit those. The biggest memory issue is the heavy use of PermGen space, especially under development. Java is very reticent about GC'ing PermGen, even when you request a GC explicitly. On Wed, Nov 3, 2010 at 6:29 AM, Jim O'Callaghan wrote: > Thanks for the suggestions guys. Josh was right in that what I was seeing > was not a problem, just what I thought was a problem. I reduced my heap > from about 1.5GB down to 512MB and saw GC happening as expected and no OOMEs > with a moderate consistent load on the app. I think I got the idea that it > needed a lot of headroom from the dev environment, where production mode was > disabled, and live class reloading uses up a lot of space (I think), > otherwise the process was crashing with heap issues. Thiago, your > suggestion of using VisualVM was very helpful - a very good tool. It looks > like Tapestry (at least 5.2.1) is very good at releasing resources - my heap > size vs. used heap is consistently bouncing between predictable levels even > after running tests 24 hours over a few days. Very happy with the result. > Summary, the creep was me misinterpreting the heap growing because I gave > it so much headroom it never really needed to GC. > > Regards, > Jim. > > -Original Message- > From: Ivano Luberti [mailto:lube...@archicoop.it] > Sent: 03 November 2010 13:11 > To: users@tapestry.apache.org > Subject: Re: Memory creep > > and make sure you trap the exception that can be thrown > > Il 03/11/2010 14.02, p.stavrini...@albourne.com ha scritto: > >> can you give me some > >>> detail on how to go about that? > > Just invoke System.gc(); > > > > regards, > > Peter > > > > > > > > - Original Message - > > From: "Thiago H. de Paula Figueiredo" > > To: "Tapestry users" > > Sent: Friday, 29 October, 2010 20:55:03 GMT +02:00 Athens, Beirut, > Bucharest, Istanbul > > Subject: Re: Memory creep > > > > On Fri, 29 Oct 2010 15:30:23 -0200, Josh Canfield < > joshcanfi...@gmail.com> > > wrote: > > > >>> but your suggestion of forcing garbage collection - can you give me > some > >>> detail on how to go about that? > >> > http://download.oracle.com/javase/6/docs/technotes/guides/management/jconsol e.html > > VisualVM (https://visualvm.dev.java.net/) has a very nice GUI for it. > > > > -- > == > dott. Ivano Mario Luberti > Archimede Informatica societa' cooperativa a r. l. > Sede Operativa > Via Gereschi 36 - 56126- Pisa > tel.: +39-050- 580959 > tel/fax: +39-050-9711344 > web: www.archicoop.it > == > > > - > To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org > For additional commands, e-mail: users-h...@tapestry.apache.org > > > - > To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org > For additional commands, e-mail: users-h...@tapestry.apache.org > > -- Howard M. Lewis Ship Creator of Apache Tapestry The source for Tapestry training, mentoring and support. Contact me to learn how I can get you up and productive in Tapestry fast! (971) 678-5210 http://howardlewisship.com - To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org For additional commands, e-mail: users-h...@tapestry.apache.org
Re: Memory creep
FYI: Using either jconsole of VisualVM you can force a GC remotely. In general, Tapestry is very good about memory management (though it likes to use a lot of it). There are a few user coding errors that can "trap" objects that should be reclaimed, but you have to work a bit hard to hit those. The biggest memory issue is the heavy use of PermGen space, especially under development. Java is very reticent about GC'ing PermGen, even when you request a GC explicitly. On Wed, Nov 3, 2010 at 6:29 AM, Jim O'Callaghan wrote: > Thanks for the suggestions guys. Josh was right in that what I was seeing > was not a problem, just what I thought was a problem. I reduced my heap > from about 1.5GB down to 512MB and saw GC happening as expected and no OOMEs > with a moderate consistent load on the app. I think I got the idea that it > needed a lot of headroom from the dev environment, where production mode was > disabled, and live class reloading uses up a lot of space (I think), > otherwise the process was crashing with heap issues. Thiago, your > suggestion of using VisualVM was very helpful - a very good tool. It looks > like Tapestry (at least 5.2.1) is very good at releasing resources - my heap > size vs. used heap is consistently bouncing between predictable levels even > after running tests 24 hours over a few days. Very happy with the result. > Summary, the creep was me misinterpreting the heap growing because I gave > it so much headroom it never really needed to GC. > > Regards, > Jim. > > -Original Message- > From: Ivano Luberti [mailto:lube...@archicoop.it] > Sent: 03 November 2010 13:11 > To: users@tapestry.apache.org > Subject: Re: Memory creep > > and make sure you trap the exception that can be thrown > > Il 03/11/2010 14.02, p.stavrini...@albourne.com ha scritto: > >> can you give me some > >>> detail on how to go about that? > > Just invoke System.gc(); > > > > regards, > > Peter > > > > > > > > - Original Message - > > From: "Thiago H. de Paula Figueiredo" > > To: "Tapestry users" > > Sent: Friday, 29 October, 2010 20:55:03 GMT +02:00 Athens, Beirut, > Bucharest, Istanbul > > Subject: Re: Memory creep > > > > On Fri, 29 Oct 2010 15:30:23 -0200, Josh Canfield < > joshcanfi...@gmail.com> > > wrote: > > > >>> but your suggestion of forcing garbage collection - can you give me > some > >>> detail on how to go about that? > >> > http://download.oracle.com/javase/6/docs/technotes/guides/management/jconsole.html > > VisualVM (https://visualvm.dev.java.net/) has a very nice GUI for it. > > > > -- > == > dott. Ivano Mario Luberti > Archimede Informatica societa' cooperativa a r. l. > Sede Operativa > Via Gereschi 36 - 56126- Pisa > tel.: +39-050- 580959 > tel/fax: +39-050-9711344 > web: www.archicoop.it > == > > > - > To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org > For additional commands, e-mail: users-h...@tapestry.apache.org > > > - > To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org > For additional commands, e-mail: users-h...@tapestry.apache.org > > -- Howard M. Lewis Ship Creator of Apache Tapestry The source for Tapestry training, mentoring and support. Contact me to learn how I can get you up and productive in Tapestry fast! (971) 678-5210 http://howardlewisship.com
Re: Memory creep
On Wed, 03 Nov 2010 11:29:18 -0200, Jim O'Callaghan wrote: Thiago, your suggestion of using VisualVM was very helpful - a very good tool. :) It looks like Tapestry (at least 5.2.1) is very good at releasing resources - my heap size vs. used heap is consistently bouncing between predictable levels even after running tests 24 hours over a few days. Very happy with the result. This seems consistent with the experiments described here: http://blog.gidley.co.uk/2009/05/tapestry-load-testing-longer-runs.html. These experiments were made with T5.1, which still had a page pool, while 5.2 doesn't. It would be nice to have them repeated with 5.2. Gidley, are you listening? :) -- Thiago H. de Paula Figueiredo Independent Java, Apache Tapestry 5 and Hibernate consultant, developer, and instructor Owner, Ars Machina Tecnologia da Informação Ltda. http://www.arsmachina.com.br - To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org For additional commands, e-mail: users-h...@tapestry.apache.org
RE: Memory creep
Thanks for the suggestions guys. Josh was right in that what I was seeing was not a problem, just what I thought was a problem. I reduced my heap from about 1.5GB down to 512MB and saw GC happening as expected and no OOMEs with a moderate consistent load on the app. I think I got the idea that it needed a lot of headroom from the dev environment, where production mode was disabled, and live class reloading uses up a lot of space (I think), otherwise the process was crashing with heap issues. Thiago, your suggestion of using VisualVM was very helpful - a very good tool. It looks like Tapestry (at least 5.2.1) is very good at releasing resources - my heap size vs. used heap is consistently bouncing between predictable levels even after running tests 24 hours over a few days. Very happy with the result. Summary, the creep was me misinterpreting the heap growing because I gave it so much headroom it never really needed to GC. Regards, Jim. -Original Message- From: Ivano Luberti [mailto:lube...@archicoop.it] Sent: 03 November 2010 13:11 To: users@tapestry.apache.org Subject: Re: Memory creep and make sure you trap the exception that can be thrown Il 03/11/2010 14.02, p.stavrini...@albourne.com ha scritto: >> can you give me some >>> detail on how to go about that? > Just invoke System.gc(); > > regards, > Peter > > > > - Original Message - > From: "Thiago H. de Paula Figueiredo" > To: "Tapestry users" > Sent: Friday, 29 October, 2010 20:55:03 GMT +02:00 Athens, Beirut, Bucharest, > Istanbul > Subject: Re: Memory creep > > On Fri, 29 Oct 2010 15:30:23 -0200, Josh Canfield > wrote: > >>> but your suggestion of forcing garbage collection - can you give me some >>> detail on how to go about that? >> http://download.oracle.com/javase/6/docs/technotes/guides/management/jconsole.html > VisualVM (https://visualvm.dev.java.net/) has a very nice GUI for it. > -- == dott. Ivano Mario Luberti Archimede Informatica societa' cooperativa a r. l. Sede Operativa Via Gereschi 36 - 56126- Pisa tel.: +39-050- 580959 tel/fax: +39-050-9711344 web: www.archicoop.it == - To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org For additional commands, e-mail: users-h...@tapestry.apache.org - To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org For additional commands, e-mail: users-h...@tapestry.apache.org
Re: Memory creep
and make sure you trap the exception that can be thrown Il 03/11/2010 14.02, p.stavrini...@albourne.com ha scritto: >> can you give me some >>> detail on how to go about that? > Just invoke System.gc(); > > regards, > Peter > > > > - Original Message - > From: "Thiago H. de Paula Figueiredo" > To: "Tapestry users" > Sent: Friday, 29 October, 2010 20:55:03 GMT +02:00 Athens, Beirut, Bucharest, > Istanbul > Subject: Re: Memory creep > > On Fri, 29 Oct 2010 15:30:23 -0200, Josh Canfield > wrote: > >>> but your suggestion of forcing garbage collection - can you give me some >>> detail on how to go about that? >> http://download.oracle.com/javase/6/docs/technotes/guides/management/jconsole.html > VisualVM (https://visualvm.dev.java.net/) has a very nice GUI for it. > -- == dott. Ivano Mario Luberti Archimede Informatica societa' cooperativa a r. l. Sede Operativa Via Gereschi 36 - 56126- Pisa tel.: +39-050- 580959 tel/fax: +39-050-9711344 web: www.archicoop.it == - To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org For additional commands, e-mail: users-h...@tapestry.apache.org
Re: Memory creep
> can you give me some >> detail on how to go about that? Just invoke System.gc(); regards, Peter - Original Message - From: "Thiago H. de Paula Figueiredo" To: "Tapestry users" Sent: Friday, 29 October, 2010 20:55:03 GMT +02:00 Athens, Beirut, Bucharest, Istanbul Subject: Re: Memory creep On Fri, 29 Oct 2010 15:30:23 -0200, Josh Canfield wrote: >> but your suggestion of forcing garbage collection - can you give me some >> detail on how to go about that? > > http://download.oracle.com/javase/6/docs/technotes/guides/management/jconsole.html VisualVM (https://visualvm.dev.java.net/) has a very nice GUI for it. -- Thiago H. de Paula Figueiredo Independent Java, Apache Tapestry 5 and Hibernate consultant, developer, and instructor Owner, Ars Machina Tecnologia da Informação Ltda. http://www.arsmachina.com.br - To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org For additional commands, e-mail: users-h...@tapestry.apache.org - To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org For additional commands, e-mail: users-h...@tapestry.apache.org
Re: Memory creep
On Fri, 29 Oct 2010 15:30:23 -0200, Josh Canfield wrote: but your suggestion of forcing garbage collection - can you give me some detail on how to go about that? http://download.oracle.com/javase/6/docs/technotes/guides/management/jconsole.html VisualVM (https://visualvm.dev.java.net/) has a very nice GUI for it. -- Thiago H. de Paula Figueiredo Independent Java, Apache Tapestry 5 and Hibernate consultant, developer, and instructor Owner, Ars Machina Tecnologia da Informação Ltda. http://www.arsmachina.com.br - To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org For additional commands, e-mail: users-h...@tapestry.apache.org
Re: Memory creep
> I think I'm would be getting OOM if I left the tests running for long enough Until you actually hit a barrier that causes a garbage collection you'll never know how much actual memory your application is using and how much garbage is sitting around. I just randomly grabbed this article which appears to have good information, but I only scanned it so YMMV. http://www.petefreitag.com/articles/gctuning/ > but your suggestion of forcing garbage collection - can you give me some > detail on how to go about that? http://download.oracle.com/javase/6/docs/technotes/guides/management/jconsole.html You can use jconsole to get a decent amount of info about the running vm. > I thought the JVM just took the GC related > settings at startup and managed things on its own from then on in? Yep, it does. There are lots of choices depending on your needs/wants. You aren't actually seeing a problem, you suspect you are seeing a problem. Everything could be working fine :) You could drop your max memory down to 700m and re-run your tests and see if you start getting OOM, or if the GC cleans things up then you've got nothing to worry about. Josh On Fri, Oct 29, 2010 at 9:39 AM, Jim O'Callaghan wrote: > Thanks for the suggestions Josh - I'm just starting to work through the app > with the Eclipse Memory Profiler to see if I can spot what's going on. > > I think I'm would be getting OOM if I left the tests running for long enough > but your suggestion of forcing garbage collection - can you give me some > detail on how to go about that? I thought the JVM just took the GC related > settings at startup and managed things on its own from then on in? > > Regards, > Jim. > > -Original Message- > From: Josh Canfield [mailto:joshcanfi...@gmail.com] > Sent: 29 October 2010 17:00 > To: Tapestry users > Subject: Re: Memory creep > > Can you run it with a profiler? Something like Yourkit will tell you what is > being used. > > Also, have you forced garbage collection? It doesn't sound like you're > getting OOM so maybe you've got garbage laying around? > On 29 Oct 2010 08:36, "Jim O'Callaghan" > wrote: >> I'm testing a Tapestry app running under Jetty on Windows using multiple >> Selenium IDE sessions and have noticed a creep of about 130MB RAM per hour >> on my java exe when running through a test loop that logs in to the app, >> runs through some expected user behaviour (some search screens, detail >> screens, minor amounts of data entry / persistence) and then logs out. >> Each test loop iteration takes about 40 seconds. Upon log out, the http >> session is destroyed and I am running six concurrent sessions at the >> moment. The test script does not vary the entities retrieved from the >> database so beyond the initial entity caching I would not expect the > memory >> usage to keep increasing, i.e. it should bottom out at some stage. >> >> I've found this with T.5.2.0 and also T.5.2.1, and the current JVM I'm > using >> is from jdk1.6.0_21_x64 on Jetty 6.1.18. I'm working my way through >> different JVMs at the moment to see if there is much of a difference. > Given >> that the entities loaded are the same during the test loops, the HTTP >> session is destroyed during each loop cycle, and the memory footprint >> appears to stay at the same level after the tests are stopped, could > anyone >> give any pointers to Tapestry areas to look at to try to identify the > cause >> of the memory creep? Or perhaps this is a JVM / Jetty issue? Having >> trawled through the list before on related GC issues, I'm using the >> following JVM settings: >> >> -server >> >> -Xms1408m >> >> -Xmx1536m >> >> -XX:MaxPermSize=256m >> >> -XX:NewSize=384m >> >> -XX:SurvivorRatio=2 >> >> With such a noticeable creep (initially approx 650MB with 6 sessions > rising >> to 780MB after 1 hour) reproducible on a dev machine I'm concerned that > this >> app won't make it out of dev unless I can get to the bottom of the > problem. >> >> >> >> Regards, >> >> Jim. >> > > > - > To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org > For additional commands, e-mail: users-h...@tapestry.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org For additional commands, e-mail: users-h...@tapestry.apache.org
RE: Memory creep
Thanks for the suggestions Josh - I'm just starting to work through the app with the Eclipse Memory Profiler to see if I can spot what's going on. I think I'm would be getting OOM if I left the tests running for long enough but your suggestion of forcing garbage collection - can you give me some detail on how to go about that? I thought the JVM just took the GC related settings at startup and managed things on its own from then on in? Regards, Jim. -Original Message- From: Josh Canfield [mailto:joshcanfi...@gmail.com] Sent: 29 October 2010 17:00 To: Tapestry users Subject: Re: Memory creep Can you run it with a profiler? Something like Yourkit will tell you what is being used. Also, have you forced garbage collection? It doesn't sound like you're getting OOM so maybe you've got garbage laying around? On 29 Oct 2010 08:36, "Jim O'Callaghan" wrote: > I'm testing a Tapestry app running under Jetty on Windows using multiple > Selenium IDE sessions and have noticed a creep of about 130MB RAM per hour > on my java exe when running through a test loop that logs in to the app, > runs through some expected user behaviour (some search screens, detail > screens, minor amounts of data entry / persistence) and then logs out. > Each test loop iteration takes about 40 seconds. Upon log out, the http > session is destroyed and I am running six concurrent sessions at the > moment. The test script does not vary the entities retrieved from the > database so beyond the initial entity caching I would not expect the memory > usage to keep increasing, i.e. it should bottom out at some stage. > > I've found this with T.5.2.0 and also T.5.2.1, and the current JVM I'm using > is from jdk1.6.0_21_x64 on Jetty 6.1.18. I'm working my way through > different JVMs at the moment to see if there is much of a difference. Given > that the entities loaded are the same during the test loops, the HTTP > session is destroyed during each loop cycle, and the memory footprint > appears to stay at the same level after the tests are stopped, could anyone > give any pointers to Tapestry areas to look at to try to identify the cause > of the memory creep? Or perhaps this is a JVM / Jetty issue? Having > trawled through the list before on related GC issues, I'm using the > following JVM settings: > > -server > > -Xms1408m > > -Xmx1536m > > -XX:MaxPermSize=256m > > -XX:NewSize=384m > > -XX:SurvivorRatio=2 > > With such a noticeable creep (initially approx 650MB with 6 sessions rising > to 780MB after 1 hour) reproducible on a dev machine I'm concerned that this > app won't make it out of dev unless I can get to the bottom of the problem. > > > > Regards, > > Jim. > - To unsubscribe, e-mail: users-unsubscr...@tapestry.apache.org For additional commands, e-mail: users-h...@tapestry.apache.org
Re: Memory creep
Can you run it with a profiler? Something like Yourkit will tell you what is being used. Also, have you forced garbage collection? It doesn't sound like you're getting OOM so maybe you've got garbage laying around? On 29 Oct 2010 08:36, "Jim O'Callaghan" wrote: > I'm testing a Tapestry app running under Jetty on Windows using multiple > Selenium IDE sessions and have noticed a creep of about 130MB RAM per hour > on my java exe when running through a test loop that logs in to the app, > runs through some expected user behaviour (some search screens, detail > screens, minor amounts of data entry / persistence) and then logs out. > Each test loop iteration takes about 40 seconds. Upon log out, the http > session is destroyed and I am running six concurrent sessions at the > moment. The test script does not vary the entities retrieved from the > database so beyond the initial entity caching I would not expect the memory > usage to keep increasing, i.e. it should bottom out at some stage. > > I've found this with T.5.2.0 and also T.5.2.1, and the current JVM I'm using > is from jdk1.6.0_21_x64 on Jetty 6.1.18. I'm working my way through > different JVMs at the moment to see if there is much of a difference. Given > that the entities loaded are the same during the test loops, the HTTP > session is destroyed during each loop cycle, and the memory footprint > appears to stay at the same level after the tests are stopped, could anyone > give any pointers to Tapestry areas to look at to try to identify the cause > of the memory creep? Or perhaps this is a JVM / Jetty issue? Having > trawled through the list before on related GC issues, I'm using the > following JVM settings: > > -server > > -Xms1408m > > -Xmx1536m > > -XX:MaxPermSize=256m > > -XX:NewSize=384m > > -XX:SurvivorRatio=2 > > With such a noticeable creep (initially approx 650MB with 6 sessions rising > to 780MB after 1 hour) reproducible on a dev machine I'm concerned that this > app won't make it out of dev unless I can get to the bottom of the problem. > > > > Regards, > > Jim. >
