Re: Performance Tuning Tomcat 7...
On 24/02/2011 06:42, Tony Anecito wrote: Hi Mark, I setup a simple servlet with no code in the doGet() method running in Tomcat 7.0.8 and using jvm 1.6.22 and using httpclient from apache commons lib and I am lucky to get 2.8msec per request as measured at the client. I am thinking the commons httpclient is slow itself. So what did you use for the client side? I was using JMeter. Key configuration was: - servlet that did nothing - http keepalive enabled - dual core test machine - single JMeter thread - all on localhost The above configuration meant that there was only a single thread used on Tomcat. The results on OSX were *much* better than those on Windows although I don't recall the details. Also, I looked at the test results someone ran using Apache httpclient and they took the total number of requests and the time it took to run and came out with an absurd number because the response times were much longer than the number they calaculated. I could do the same thing by say taking 100 requests that ran in parallel and took 100msec each and divide by 100 and say it took 1msec per request but that is not what the user would see. I've done some tests along those lines in the past with the same simple servlet and the result of (total time)/(total number of requests) was largely similar regardless of how many requests were processed in parallel. In short, the more threads, the longer each request took. I saw the best throughput where parallel threads was at or just above the number of cores available (unsurprisingly). A real app is likely to be different. Just interesting how people can twist statistics. Indeed. We often say on this list that you can construct a load-test to show that any combination of Tomcat connector with/without httpd is the 'best'. It all depends on the test. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Tuning Tomcat 7...
Hi Mark, I setup a simple servlet with no code in the doGet() method running in Tomcat 7.0.8 and using jvm 1.6.22 and using httpclient from apache commons lib and I am lucky to get 2.8msec per request as measured at the client. I am thinking the commons httpclient is slow itself. So what did you use for the client side? Also, I looked at the test results someone ran using Apache httpclient and they took the total number of requests and the time it took to run and came out with an absurd number because the response times were much longer than the number they calaculated. I could do the same thing by say taking 100 requests that ran in parallel and took 100msec each and divide by 100 and say it took 1msec per request but that is not what the user would see. Just interesting how people can twist statistics. -Tony - Original Message From: Mark Thomas ma...@apache.org To: Tomcat Users List users@tomcat.apache.org Sent: Sat, February 12, 2011 11:45:29 AM Subject: Re: Performance Tuning Tomcat 7... On 12/02/2011 18:27, Tony Anecito wrote: Right now for most of my transactions I get less than 5 microseconds and around 1.2msec is spent on getting to and out of Tomcat and out of to/out of the client call. So people were saying look at better parsers or replacing whatever does the parsing. I am measuring round trip time at the client (before and after the jersey call) and getting to 1.47msec. At Tomcat it is less than 5microseconds at the beginning of the method the GET goes to the end of the method so the code is very fast. So I am thinking whatever parses the cmd and puts together the html response after the end of the method is where I should focus. Testing Tomcat on localhost with a simple servlet using my 3 year old laptop I can process around 20,000 requests a second on a single connection which is around 50 microseconds per request. That includes server and client processing. Looking at all these numbers suggests that there is a lot of additional overhead somewhere in your system. Based on past experience, guessing where that overhead might be is a waste of time. You need to use a profiler to track it down. I usually use Yourkit since they give free copies to the Tomcat committers for use with Tomcat development. Other profilers are available. Pick the one that works best for you. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Tuning Tomcat 7...
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Pid, On 2/15/2011 2:43 PM, Pid wrote: On 2/15/11 2:34 PM, Christopher Schultz wrote: Pid, On 2/14/2011 5:39 AM, Pid wrote: On 2/13/11 8:15 PM, Tony Anecito wrote: Hi Mark, Here is the link to the screenshot I tried to send as an attachement. http://www.myuniportal.com/tomcatjerseyservletprofile.jpg What are we looking for here? I don't know if there's any optimisation possible in the SocketProcessor.run() method in Tomcat. If I were to consider what Jersey is doing, I'd look at why there's an expensive ContainerResponse.write() call (costing 0.6ms) while it still seems to be processing the request. ... and a recursive call to _handleRequest. Or am I reading that incorrectly Could be, or there could be more than one _handleRequest method with different args. /guessing Duh. Didn't think of that. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk1cCJEACgkQ9CaO5/Lv0PCTQQCePKndmJbVkBsW7O17DFrlbbXU A/AAn0+iP1nxUqL3evB9r8cY/vY9zw61 =tlpV -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Tuning Tomcat 7...
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Pid, On 2/14/2011 5:39 AM, Pid wrote: On 2/13/11 8:15 PM, Tony Anecito wrote: Hi Mark, Here is the link to the screenshot I tried to send as an attachement. http://www.myuniportal.com/tomcatjerseyservletprofile.jpg What are we looking for here? I don't know if there's any optimisation possible in the SocketProcessor.run() method in Tomcat. If I were to consider what Jersey is doing, I'd look at why there's an expensive ContainerResponse.write() call (costing 0.6ms) while it still seems to be processing the request. ... and a recursive call to _handleRequest. Or am I reading that incorrectly? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk1ajvEACgkQ9CaO5/Lv0PBaWQCgpYt1fEB10vvTa2iiQTB+yEW9 1BwAnjqa0VzQyLs5gwstzznb6ohuJ1HU =y8q8 -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Tuning Tomcat 7...
On 2/15/11 2:34 PM, Christopher Schultz wrote: Pid, On 2/14/2011 5:39 AM, Pid wrote: On 2/13/11 8:15 PM, Tony Anecito wrote: Hi Mark, Here is the link to the screenshot I tried to send as an attachement. http://www.myuniportal.com/tomcatjerseyservletprofile.jpg What are we looking for here? I don't know if there's any optimisation possible in the SocketProcessor.run() method in Tomcat. If I were to consider what Jersey is doing, I'd look at why there's an expensive ContainerResponse.write() call (costing 0.6ms) while it still seems to be processing the request. ... and a recursive call to _handleRequest. Or am I reading that incorrectly Could be, or there could be more than one _handleRequest method with different args. /guessing p -chris - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org signature.asc Description: OpenPGP digital signature
Re: Performance Tuning Tomcat 7...
On 2/13/11 8:15 PM, Tony Anecito wrote: Hi Mark, Here is the link to the screenshot I tried to send as an attachement. http://www.myuniportal.com/tomcatjerseyservletprofile.jpg What are we looking for here? I don't know if there's any optimisation possible in the SocketProcessor.run() method in Tomcat. If I were to consider what Jersey is doing, I'd look at why there's an expensive ContainerResponse.write() call (costing 0.6ms) while it still seems to be processing the request. p - Original Message From: Mark Thomas ma...@apache.org To: Tomcat Users List users@tomcat.apache.org Sent: Sun, February 13, 2011 2:54:53 AM Subject: Re: Performance Tuning Tomcat 7... On 13/02/2011 04:33, Tony Anecito wrote: Thanks Pid. Odd though I did a GET with no attachments. I do return a complex object but that is only the response not the request. Pid meant that the mailing list strips attachments. We can't see the Visual VM output you are referring to. Mark -Tony - Original Message From: Pid p...@pidster.com To: Tomcat Users List users@tomcat.apache.org Sent: Sat, February 12, 2011 5:04:41 PM Subject: Re: Performance Tuning Tomcat 7... On 2/12/11 10:23 PM, Tony Anecito wrote: I attached the screenshot of visualvm showing what I mean. The list strips attachments... p signature.asc Description: OpenPGP digital signature
Re: Performance Tuning Tomcat 7...
All I am showing is a very long chain of method calls for Jersey that seem to compose most of the overall time. I suspect that the tests Mark was talking about where he got 50k requests did not have anything like what I am seeing. Just shows Jersey is not capable of high throughput. -Tony - Original Message From: Pid p...@pidster.com To: Tomcat Users List users@tomcat.apache.org Sent: Mon, February 14, 2011 3:39:04 AM Subject: Re: Performance Tuning Tomcat 7... On 2/13/11 8:15 PM, Tony Anecito wrote: Hi Mark, Here is the link to the screenshot I tried to send as an attachement. http://www.myuniportal.com/tomcatjerseyservletprofile.jpg What are we looking for here? I don't know if there's any optimisation possible in the SocketProcessor.run() method in Tomcat. If I were to consider what Jersey is doing, I'd look at why there's an expensive ContainerResponse.write() call (costing 0.6ms) while it still seems to be processing the request. p - Original Message From: Mark Thomas ma...@apache.org To: Tomcat Users List users@tomcat.apache.org Sent: Sun, February 13, 2011 2:54:53 AM Subject: Re: Performance Tuning Tomcat 7... On 13/02/2011 04:33, Tony Anecito wrote: Thanks Pid. Odd though I did a GET with no attachments. I do return a complex object but that is only the response not the request. Pid meant that the mailing list strips attachments. We can't see the Visual VM output you are referring to. Mark -Tony - Original Message From: Pid p...@pidster.com To: Tomcat Users List users@tomcat.apache.org Sent: Sat, February 12, 2011 5:04:41 PM Subject: Re: Performance Tuning Tomcat 7... On 2/12/11 10:23 PM, Tony Anecito wrote: I attached the screenshot of visualvm showing what I mean. The list strips attachments... p - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Tuning Tomcat 7...
On 13/02/2011 04:33, Tony Anecito wrote: Thanks Pid. Odd though I did a GET with no attachments. I do return a complex object but that is only the response not the request. Pid meant that the mailing list strips attachments. We can't see the Visual VM output you are referring to. Mark -Tony - Original Message From: Pid p...@pidster.com To: Tomcat Users List users@tomcat.apache.org Sent: Sat, February 12, 2011 5:04:41 PM Subject: Re: Performance Tuning Tomcat 7... On 2/12/11 10:23 PM, Tony Anecito wrote: I attached the screenshot of visualvm showing what I mean. The list strips attachments... p We won't tell. Get more on shows you hate to love (and love to hate): Yahoo! TV's Guilty Pleasures list. http://tv.yahoo.com/collections/265 - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Tuning Tomcat 7...
Thanks that helps. I also sent the image output to the Jersey group hopefully it does not strip attachements. I will setup a link later today to the snapshot. Thanks, -Tony - Original Message From: Mark Thomas ma...@apache.org To: Tomcat Users List users@tomcat.apache.org Sent: Sun, February 13, 2011 2:54:53 AM Subject: Re: Performance Tuning Tomcat 7... On 13/02/2011 04:33, Tony Anecito wrote: Thanks Pid. Odd though I did a GET with no attachments. I do return a complex object but that is only the response not the request. Pid meant that the mailing list strips attachments. We can't see the Visual VM output you are referring to. Mark -Tony - Original Message From: Pid p...@pidster.com To: Tomcat Users List users@tomcat.apache.org Sent: Sat, February 12, 2011 5:04:41 PM Subject: Re: Performance Tuning Tomcat 7... On 2/12/11 10:23 PM, Tony Anecito wrote: I attached the screenshot of visualvm showing what I mean. The list strips attachments... p _ We won't tell. Get more on shows you hate to love (and love to hate): Yahoo! TV's Guilty Pleasures list. http://tv.yahoo.com/collections/265 - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Tuning Tomcat 7...
Hi Mark, Here is the link to the screenshot I tried to send as an attachement. http://www.myuniportal.com/tomcatjerseyservletprofile.jpg Best Regards, Tony Anecito Founder, MyUniPortal (2010 JavaOne Duke's Award) http://www.myuniportal.com - Original Message From: Mark Thomas ma...@apache.org To: Tomcat Users List users@tomcat.apache.org Sent: Sun, February 13, 2011 2:54:53 AM Subject: Re: Performance Tuning Tomcat 7... On 13/02/2011 04:33, Tony Anecito wrote: Thanks Pid. Odd though I did a GET with no attachments. I do return a complex object but that is only the response not the request. Pid meant that the mailing list strips attachments. We can't see the Visual VM output you are referring to. Mark -Tony - Original Message From: Pid p...@pidster.com To: Tomcat Users List users@tomcat.apache.org Sent: Sat, February 12, 2011 5:04:41 PM Subject: Re: Performance Tuning Tomcat 7... On 2/12/11 10:23 PM, Tony Anecito wrote: I attached the screenshot of visualvm showing what I mean. The list strips attachments... p _ We won't tell. Get more on shows you hate to love (and love to hate): Yahoo! TV's Guilty Pleasures list. http://tv.yahoo.com/collections/265 - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Tuning Tomcat 7...
On 12/02/2011 18:14, Tony Anecito wrote: Is there any performance tuning that can be done to Tomcat 7 besides just the standard jvm arguments? 95%+ of tuning will need to be done in the app rather than Tomcat. My recommendation is to get yourself a profiler and go from there. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Tuning Tomcat 7...
Right now for most of my transactions I get less than 5 microseconds and around 1.2msec is spent on getting to and out of Tomcat and out of to/out of the client call. So people were saying look at better parsers or replacing whatever does the parsing. I am measuring round trip time at the client (before and after the jersey call) and getting to 1.47msec. At Tomcat it is less than 5microseconds at the beginning of the method the GET goes to the end of the method so the code is very fast. So I am thinking whatever parses the cmd and puts together the html response after the end of the method is where I should focus. The network goes over a 1Gb ethernet lan. Regards, -Tony - Original Message From: Mark Thomas ma...@apache.org To: Tomcat Users List users@tomcat.apache.org Sent: Sat, February 12, 2011 11:18:18 AM Subject: Re: Performance Tuning Tomcat 7... On 12/02/2011 18:14, Tony Anecito wrote: Is there any performance tuning that can be done to Tomcat 7 besides just the standard jvm arguments? 95%+ of tuning will need to be done in the app rather than Tomcat. My recommendation is to get yourself a profiler and go from there. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Tuning Tomcat 7...
On 2/12/2011 1:27 PM, Tony Anecito wrote: Right now for most of my transactions I get less than 5 microseconds and around 1.2msec is spent on getting to and out of Tomcat and out of to/out of the client call. So people were saying look at better parsers or replacing whatever does the parsing. I am measuring round trip time at the client (before and after the jersey call) and getting to 1.47msec. At Tomcat it is less than 5microseconds at the beginning of the method the GET goes to the end of the method so the code is very fast. So I am thinking whatever parses the cmd and puts together the html response after the end of the method is where I should focus. The network goes over a 1Gb ethernet lan. When you're looking at low- or sub-millisecond timing, you're going to need to look at your network router (if any), switch and wiring latency, as well as the raw throughput.. D Regards, -Tony - Original Message From: Mark Thomasma...@apache.org To: Tomcat Users Listusers@tomcat.apache.org Sent: Sat, February 12, 2011 11:18:18 AM Subject: Re: Performance Tuning Tomcat 7... On 12/02/2011 18:14, Tony Anecito wrote: Is there any performance tuning that can be done to Tomcat 7 besides just the standard jvm arguments? 95%+ of tuning will need to be done in the app rather than Tomcat. My recommendation is to get yourself a profiler and go from there. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Tuning Tomcat 7...
On 12/02/2011 18:27, Tony Anecito wrote: Right now for most of my transactions I get less than 5 microseconds and around 1.2msec is spent on getting to and out of Tomcat and out of to/out of the client call. So people were saying look at better parsers or replacing whatever does the parsing. I am measuring round trip time at the client (before and after the jersey call) and getting to 1.47msec. At Tomcat it is less than 5microseconds at the beginning of the method the GET goes to the end of the method so the code is very fast. So I am thinking whatever parses the cmd and puts together the html response after the end of the method is where I should focus. Testing Tomcat on localhost with a simple servlet using my 3 year old laptop I can process around 20,000 requests a second on a single connection which is around 50 microseconds per request. That includes server and client processing. Looking at all these numbers suggests that there is a lot of additional overhead somewhere in your system. Based on past experience, guessing where that overhead might be is a waste of time. You need to use a profiler to track it down. I usually use Yourkit since they give free copies to the Tomcat committers for use with Tomcat development. Other profilers are available. Pick the one that works best for you. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Tuning Tomcat 7...
Thanks the numbers help to give me an idea what might be possible. I do have visualvm 1.3.1 and I will try that. -Tony - Original Message From: Mark Thomas ma...@apache.org To: Tomcat Users List users@tomcat.apache.org Sent: Sat, February 12, 2011 11:45:29 AM Subject: Re: Performance Tuning Tomcat 7... On 12/02/2011 18:27, Tony Anecito wrote: Right now for most of my transactions I get less than 5 microseconds and around 1.2msec is spent on getting to and out of Tomcat and out of to/out of the client call. So people were saying look at better parsers or replacing whatever does the parsing. I am measuring round trip time at the client (before and after the jersey call) and getting to 1.47msec. At Tomcat it is less than 5microseconds at the beginning of the method the GET goes to the end of the method so the code is very fast. So I am thinking whatever parses the cmd and puts together the html response after the end of the method is where I should focus. Testing Tomcat on localhost with a simple servlet using my 3 year old laptop I can process around 20,000 requests a second on a single connection which is around 50 microseconds per request. That includes server and client processing. Looking at all these numbers suggests that there is a lot of additional overhead somewhere in your system. Based on past experience, guessing where that overhead might be is a waste of time. You need to use a profiler to track it down. I usually use Yourkit since they give free copies to the Tomcat committers for use with Tomcat development. Other profilers are available. Pick the one that works best for you. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Tuning Tomcat 7...
Yep visualvm showed me where the issue is. It is in the jersey servlet it makes so many calls it all adds up. And that is before it calls my class which only takes 5-6 microseconds to execute. I am not sure what I can do about this unless there is some advice from the Jersey team about how to short-circuit some of these calls. I attached the screenshot of visualvm showing what I mean. Thanks, -Tony - Original Message From: Mark Thomas ma...@apache.org To: Tomcat Users List users@tomcat.apache.org Sent: Sat, February 12, 2011 11:45:29 AM Subject: Re: Performance Tuning Tomcat 7... On 12/02/2011 18:27, Tony Anecito wrote: Right now for most of my transactions I get less than 5 microseconds and around 1.2msec is spent on getting to and out of Tomcat and out of to/out of the client call. So people were saying look at better parsers or replacing whatever does the parsing. I am measuring round trip time at the client (before and after the jersey call) and getting to 1.47msec. At Tomcat it is less than 5microseconds at the beginning of the method the GET goes to the end of the method so the code is very fast. So I am thinking whatever parses the cmd and puts together the html response after the end of the method is where I should focus. Testing Tomcat on localhost with a simple servlet using my 3 year old laptop I can process around 20,000 requests a second on a single connection which is around 50 microseconds per request. That includes server and client processing. Looking at all these numbers suggests that there is a lot of additional overhead somewhere in your system. Based on past experience, guessing where that overhead might be is a waste of time. You need to use a profiler to track it down. I usually use Yourkit since they give free copies to the Tomcat committers for use with Tomcat development. Other profilers are available. Pick the one that works best for you. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org Expecting? Get great news right away with email Auto-Check. Try the Yahoo! Mail Beta. http://advision.webevents.yahoo.com/mailbeta/newmail_tools.html - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Performance Tuning Tomcat 7...
On 2/12/11 10:23 PM, Tony Anecito wrote: I attached the screenshot of visualvm showing what I mean. The list strips attachments... p signature.asc Description: OpenPGP digital signature
Re: Performance Tuning Tomcat 7...
Thanks Pid. Odd though I did a GET with no attachments. I do return a complex object but that is only the response not the request. -Tony - Original Message From: Pid p...@pidster.com To: Tomcat Users List users@tomcat.apache.org Sent: Sat, February 12, 2011 5:04:41 PM Subject: Re: Performance Tuning Tomcat 7... On 2/12/11 10:23 PM, Tony Anecito wrote: I attached the screenshot of visualvm showing what I mean. The list strips attachments... p We won't tell. Get more on shows you hate to love (and love to hate): Yahoo! TV's Guilty Pleasures list. http://tv.yahoo.com/collections/265 - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
