Hi Oleg, Summary The following test results show me that using the localhost as the server name and the ApacheHttp client running on the server as done with the ApacheHttp tests by Apache components group gets close to or bettter than expected. Using localhost I am guessing bypasses using the network and running 500,000 requests over 20 threads gets a low average.
I still need to have the servlet setup to send data but I am wondering if the Apache component tests used Tomcat at all and instand just read a file from Apache Web Server. I will look further into this also to make sure I get closer to what the tests did. Since I think I am getting closer to how Tomcat responds for a bare servlet with no code in it and maybe no network component I am more comfortable in understanding what Tomcat can truely do from a performance perspective. If I go back to my 6 core server and run the same test (client running on the server where Tomcat is as the I did previously on my old laptop I get: httpClient execute time in nanoseconds: 209391137 httpClient execute time in nanoseconds: 1595734 httpClient execute time in nanoseconds: 1298489 httpClient execute time in nanoseconds: 1238355 httpClient execute time in nanoseconds: 1262800 httpClient execute time in nanoseconds: 1196312 httpClient execute time in nanoseconds: 1253512 httpClient execute time in nanoseconds: 1457866 httpClient execute time in nanoseconds: 1215867 httpClient execute time in nanoseconds: 1254000 Concurrency level: 1 Time taken for tests: 0.234 seconds Complete requests: 10 Failed requests: 0 Content transferred: 0 bytes Requests per second: 42.735043 [#/sec] (mean) Time per request: 23.4 [ms] (mean) If I run next with the server name set to localhost instead of a internet url as done on the Apache tests I get (This should cause the network to be bypassed perhaps even the TCP/IP Stack? note the much lower response times): httpClient execute time in nanoseconds: 100221746 httpClient execute time in nanoseconds: 890756 httpClient execute time in nanoseconds: 688845 httpClient execute time in nanoseconds: 688844 httpClient execute time in nanoseconds: 689823 httpClient execute time in nanoseconds: 614044 httpClient execute time in nanoseconds: 585200 httpClient execute time in nanoseconds: 604266 httpClient execute time in nanoseconds: 657066 httpClient execute time in nanoseconds: 408223 Concurrency level: 1 Time taken for tests: 0.109 seconds Complete requests: 10 Failed requests: 0 Content transferred: 0 bytes Requests per second: 91.74312 [#/sec] (mean) Time per request: 10.9 [ms] (mean) If I now run a 500,000 request thread over 20 sockets (localhost and running the client app on the Tomcat serverI get: Concurrency level: 20 Time taken for tests: 14.549 seconds Complete requests: 500000 Failed requests: 0 Content transferred: 0 bytes Requests per second: 34366.625 [#/sec] (mean) Time per request: 0.029098 [ms] (mean) ----- Original Message ---- From: Tony Anecito <adanec...@yahoo.com> To: HttpClient User Discussion <httpclient-us...@hc.apache.org> Sent: Thu, March 3, 2011 10:49:53 PM Subject: Re: Slowness of 4.1 Hi Oleg, Here is some numbers when running TestHttpClient4. I am running from Eclipse using jdk 1. 6.0.22 and a Aspire 5670 which has Intel Core Duo T2300 (1.66 GHz, 667 MHz FSB, 2MB L2 cache) and 2GB of DDR2 memory. Note how for a single thread running 10 requests the response time starts out very slow at 150msec then gets to 1.43msec at the end. httpClient execute time in nanoseconds: 150181377 httpClient execute time in nanoseconds: 2180445 httpClient execute time in nanoseconds: 1540419 httpClient execute time in nanoseconds: 1757207 httpClient execute time in nanoseconds: 1759721 httpClient execute time in nanoseconds: 1492368 httpClient execute time in nanoseconds: 1493765 httpClient execute time in nanoseconds: 1474489 httpClient execute time in nanoseconds: 1456610 httpClient execute time in nanoseconds: 1437613 Document URI: xxx Document Length: 0 bytes Concurrency level: 1 Time taken for tests: 0.187 seconds Complete requests: 10 Failed requests: 0 Content transferred: 0 bytes Requests per second: 53.475933 [#/sec] (mean) Time per request: 18.7 [ms] (mean) When I increase the number of threads to 50 and requests to 500000 I get a max req of 2.2K per second and around 400microseconds. The CPU utilization is between 50-60% on the laptop and around 5-7% on my 6 core server with Tomcat 7. Tommow I will setup the test to run from a faster client and see the results. ----- Original Message ---- From: Tony Anecito <adanec...@yahoo.com> To: HttpClient User Discussion <mailto:httpclient-us...@hc.apache.org> Sent: Thu, March 3, 2011 1:06:41 PM Subject: Re: Slowness of 4.1 Thanks Oleg for your explanation about comparitive vs real life. Most of the testing I do is for "real life" since the end goal is user experience thus my comments. I can understand what Apache goals are better now and adjust my expectations accordingly. I did find what I needed as far as examples and hope to get back to response time numbers I have seen with other client side libraries and configurations. I will look at the links you provided to better understand what Apache hopes to do. The only recommendation for stats I would make is have a time measurement for how long each thread takes to execute a request. And show three new stats. One for average time based on that measurement, a 90% of measurements measurement (Gives indication of distribution) and average response time over load for each average. Just some thoughts you and the team can decide how if at all those suggestions might fit your testing goals. Regards, -Tony ----- Original Message ---- From: Oleg Kalnichevski <ol...@apache.org> To: HttpClient User Discussion <mailto:httpclient-us...@hc.apache.org> Sent: Thu, March 3, 2011 1:52:37 PM Subject: Re: Slowness of 4.1 On Thu, 2011-03-03 at 09:20 -0800, Tony Anecito wrote: > Hi, > > Where can I find the client side code for this test? I think I found it once > before and discovered the way the response times were calculated were >incorrect. > What I saw indicated your test results were divideing the number of requests > into the time it took for all of them to complete when the tests were run in > parallel so it made it look like the requests were faster than they really >were. > > It was like taking ten 100ms requests run in parallel and getting 10ms per > request when in reality it was 100ms. > I did see your remarks about the benchmark on the tomcat user list. (1) the benchmark tries to closely simulate Apache Bench (ab), which is a well established and widely used tool for performance measurement. http://en.wikipedia.org/wiki/ApacheBench (2) the benchmark is not intended to calculate 'real-life' or 'accurate' response time, whatever that means. The purpose of the benchmark is to provide a _baseline_ for a comparative performance analysis and is intended to give a _rough_ indication as to whether an HTTP client A faster than an client HTTP, or whether data throughput increased or decreased after particular set of changes. That is it. Having said all that you are very welcome to suggest improvements to the benchmark or a more accurate algorithm for calculating performance numbers. Oleg > In either case I will see what if I can find again the client side code > soemwhere and compare it to what I have unless you have a link to it. > > Thanks, > -Tony > > > > ----- Original Message ---- > From: Oleg Kalnichevski <ol...@apache.org> > To: HttpClient User Discussion <mailto:httpclient-us...@hc.apache.org> > Sent: Thu, March 3, 2011 2:46:03 AM > Subject: Re: Slowness of 4.1 > > On Thu, 2011-03-03 at 01:38 -0800, Tony Anecito wrote: > > Hi All, > > > > I downloaded httpclient 4.1 and noticed it is significantly slower than > > 3.1. >I > > > > even used the threadsafe connection manager hoping for better performance. > > I > > used to get below 3msec and now it is above 150msec. > > Is keepalive used by default? If not where can I find an example that sets > > it > > > > > using either a scheme, connection manager httpclient instance or the > > httpget > > object? > > > > What kind of response time should I expect hitting a servlet with no code > > in > >the > > > > method on a 2.8Ghz 6 core AMD server? > > > > Any ideas on what it might be? I used code sample from the site and it > > works > > just very slow. > > > > Thanks, > > -Tony > > > > HttpClient 4.1 is known to be comfortably faster than 3.1. There is > likely to be a problem with how you are measuring performance / response > time. > > http://wiki.apache.org/HttpComponents/HttpClient3vsHttpClient4vsHttpCore > > Oleg > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: >mailto:mailto:mailto:mailto:httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: >mailto:mailto:mailto:mailto:httpclient-users-h...@hc.apache.org > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
