RE: Tomcat Status Metrics: RRDTool Graph: Analysis
Thanks, Chris. I'll do some load testing. I was thinking of using siege or apache bench. _Tomcat: The Definitive Guide_ is my guide. Do you know of a better way of getting heap numbers out of Java? Right now I've got three metrics: jvm_mem_free; jvm_mem_max; jvm_mem_total. Examining the graphs, I see that 'free' and 'total' are not exact opposites. 'max' doesn't change and I believe this is set with the option -Xmx2048m. These are the java options relating to the heap: -Xmx2048m -Xms1024m -XX:MaxPermSize=256m -XX:+UseParallelGC __/ __/ Jvm_mem_total creeps up to the max over time (about 2 days): __/ During that time, jvm_mem_free fluctuates regularly, like this: /\/\/\/\/\ -Original Message- From: Christopher Schultz [mailto:[EMAIL PROTECTED] Sent: Thursday, March 06, 2008 11:54 AM To: Tomcat Users List Subject: Re: Tomcat Status Metrics: RRDTool Graph: Analysis -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 David, David Smithson wrote: | Hi, Christopher. Thanks for your reply. This is a typical hour on | any given day. What other things should I be looking at in Tomcat | and Java? I've got plenty of metrics from other parts of the | operating system. Typically, I look at heap activity (in Java, not just free memory from the OS's perspective) and CPU utilization. The cleaner your heap, the faster everything will run. | I've got my on the JVM memory metrics that come from the Tomcat | status page as well. If I'm not mistaken, these numbers represent | JVM memory fluctuations and relate to garbage collection routines. Probably. Look for heap activity that looks like this: /\/\/\/\/\/\ ~ __/ ~ ___/ ~ ___/ When you start seeing this: __/ You have a problem! Just because your threads are not being used too much doesn't mean there aren't other problems lurking. Even with low thread utilization, a bad heap graph can mean that you're just waiting around for an OOME. Consider performing load tests and watching the numbers instead of just watching a typical hour of a typical day. You need to plan for peak load, not for average load. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkfQIYoACgkQ9CaO5/Lv0PAcQwCfcXMtpU3JklRx074q8y2d1cn5 6mAAoLiPX9LMAAOjk31uAiQRFu1Uot+q =Yfxr -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat Status Metrics: RRDTool Graph: Analysis
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 David, David Smithson wrote: | Do you know of a better way of getting heap numbers out of Java? | Right now I've got three metrics: jvm_mem_free; jvm_mem_max; | jvm_mem_total. Java has multiple heaps (okay, one heap segmented into several generations), so you might want to find a tool that will give you more detailed information. Search the archives for names of such tools. | Examining the graphs, I see that 'free' and 'total' | are not exact opposites. That's an odd way of expressing their relative inverse relationship. | 'max' doesn't change and I believe this is | set with the option -Xmx2048m. Yes. Max = -Xmx setting while total = the current max (which should be = Max). | These are the java options relating to the heap: | | -Xmx2048m | -Xms1024m Save yourself some confusion and set -Xmx = -Xms. Admin lore suggests that it's easier on the GC since it will never have to resize the entire heap (and I tend to agree). | -XX:MaxPermSize=256m I've found that unless you are having problems, you probably don't need to set the -X:MaxPermSize setting. | -XX:+UseParallelGC Same here. | Jvm_mem_total creeps up to the max over time (about 2 days): During | that time, jvm_mem_free fluctuates regularly, like this [nice graph]. Sounds just about right: the JVM is allowing the heap to grow from 1GB to the max 2GB you gave it. The free memory fluctuates nicely. I would watch it for a week to make sure you aren't observing a slow memory leak. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkfR/OQACgkQ9CaO5/Lv0PDV7gCgiFj1T9zNd1SItJ7nSqAy6ia/ cUMAn3OdM4s1SasVlYYh4KWOnYAnyitD =AAvT -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat Status Metrics: RRDTool Graph: Analysis
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 David, David Smithson wrote: | I was wondering if | someone could have a look at [the graph] and tell me if I can | conclude that the performance directives in server.xml do not need to be | modified. That depends on your target performance. If you wanted an average of 1 active thread, you have failed ;) | To me, this graph says that there are very few active Tomcat | threads That is true for this particular hour. Is this a representative sample? | and that there is no reason to go mucking around with the | settings in Tomcat server.xml. Is this a fair analysis? It does not appear that you need more threads for any reason. Remember, there are lots of other things you can tweak besides the thread pool parameters. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkfP+zQACgkQ9CaO5/Lv0PDlsQCeOmBnrFTrxlV5Ee3na4I1jb7w +BcAn3g+QHy0NNz2mOLid+dLwV/FBLOA =dwcc -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tomcat Status Metrics: RRDTool Graph: Analysis
Hi, Christopher. Thanks for your reply. This is a typical hour on any given day. What other things should I be looking at in Tomcat and Java? I've got plenty of metrics from other parts of the operating system. I've got my on the JVM memory metrics that come from the Tomcat status page as well. If I'm not mistaken, these numbers represent JVM memory fluctuations and relate to garbage collection routines. -Original Message- From: Christopher Schultz [mailto:[EMAIL PROTECTED] Sent: Thursday, March 06, 2008 9:10 AM To: Tomcat Users List Subject: Re: Tomcat Status Metrics: RRDTool Graph: Analysis -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 David, David Smithson wrote: | I was wondering if | someone could have a look at [the graph] and tell me if I can | conclude that the performance directives in server.xml do not need to be | modified. That depends on your target performance. If you wanted an average of 1 active thread, you have failed ;) | To me, this graph says that there are very few active Tomcat | threads That is true for this particular hour. Is this a representative sample? | and that there is no reason to go mucking around with the | settings in Tomcat server.xml. Is this a fair analysis? It does not appear that you need more threads for any reason. Remember, there are lots of other things you can tweak besides the thread pool parameters. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkfP+zQACgkQ9CaO5/Lv0PDlsQCeOmBnrFTrxlV5Ee3na4I1jb7w +BcAn3g+QHy0NNz2mOLid+dLwV/FBLOA =dwcc -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat Status Metrics: RRDTool Graph: Analysis
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 David, David Smithson wrote: | Hi, Christopher. Thanks for your reply. This is a typical hour on | any given day. What other things should I be looking at in Tomcat | and Java? I've got plenty of metrics from other parts of the | operating system. Typically, I look at heap activity (in Java, not just free memory from the OS's perspective) and CPU utilization. The cleaner your heap, the faster everything will run. | I've got my on the JVM memory metrics that come from the Tomcat | status page as well. If I'm not mistaken, these numbers represent | JVM memory fluctuations and relate to garbage collection routines. Probably. Look for heap activity that looks like this: /\/\/\/\/\/\ ~ __/ ~ ___/ ~ ___/ When you start seeing this: __/ You have a problem! Just because your threads are not being used too much doesn't mean there aren't other problems lurking. Even with low thread utilization, a bad heap graph can mean that you're just waiting around for an OOME. Consider performing load tests and watching the numbers instead of just watching a typical hour of a typical day. You need to plan for peak load, not for average load. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkfQIYoACgkQ9CaO5/Lv0PAcQwCfcXMtpU3JklRx074q8y2d1cn5 6mAAoLiPX9LMAAOjk31uAiQRFu1Uot+q =Yfxr -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tomcat Status Metrics: RRDTool Graph: Analysis
I guess the list doesn't support attachments, so I've staged this on a web server. Here's the link to the graph: https://littleblue.activsupport.com/tomcat_graph.png -Original Message- From: David Smithson [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 05, 2008 10:18 PM To: users@tomcat.apache.org Subject: Tomcat Status Metrics: RRDTool Graph: Analysis Hi all, I was recently asked to create a way to analyze Tomcat performance. I decided to grab stats from the /manager/status?XML=true page and push those metrics into rrdtool. There are all sorts of other details to the analysis which I will leave out. In any case, I was wondering if someone could have a look at the attached graph and tell me if I can conclude that the performance directives in server.xml do not need to be modified. To me, this graph says that there are very few active Tomcat threads and that there is no reason to go mucking around with the settings in Tomcat server.xml. Is this a fair analysis? Thanks for your time. -- David - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat Status Metrics: RRDTool Graph: Analysis
On Wed, Mar 5, 2008 at 7:26 PM, David Smithson [EMAIL PROTECTED] wrote: I guess the list doesn't support attachments, so I've staged this on a web server. Here's the link to the graph: https://littleblue.activsupport.com/tomcat_graph.png 404 Forbidden. -Dave - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tomcat Status Metrics: RRDTool Graph: Analysis
Oops. Try now please. -Original Message- From: David Rees [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 05, 2008 10:44 PM To: Tomcat Users List Subject: Re: Tomcat Status Metrics: RRDTool Graph: Analysis On Wed, Mar 5, 2008 at 7:26 PM, David Smithson [EMAIL PROTECTED] wrote: I guess the list doesn't support attachments, so I've staged this on a web server. Here's the link to the graph: https://littleblue.activsupport.com/tomcat_graph.png 404 Forbidden. -Dave - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]