Thanks Oleg for your explanation about comparitive vs real life. Most of the testing I do is for "real life" since the end goal is user experience thus my comments. I can understand what Apache goals are better now and adjust my expectations accordingly.
I did find what I needed as far as examples and hope to get back to response time numbers I have seen with other client side libraries and configurations. I will look at the links you provided to better understand what Apache hopes to do. The only recommendation for stats I would make is have a time measurement for how long each thread takes to execute a request. And show three new stats. One for average time based on that measurement, a 90% of measurements measurement (Gives indication of distribution) and average response time over load for each average. Just some thoughts you and the team can decide how if at all those suggestions might fit your testing goals. Regards, -Tony ----- Original Message ---- From: Oleg Kalnichevski <[email protected]> To: HttpClient User Discussion <[email protected]> Sent: Thu, March 3, 2011 1:52:37 PM Subject: Re: Slowness of 4.1 On Thu, 2011-03-03 at 09:20 -0800, Tony Anecito wrote: > Hi, > > Where can I find the client side code for this test? I think I found it once > before and discovered the way the response times were calculated were >incorrect. > What I saw indicated your test results were divideing the number of requests > into the time it took for all of them to complete when the tests were run in > parallel so it made it look like the requests were faster than they really >were. > > It was like taking ten 100ms requests run in parallel and getting 10ms per > request when in reality it was 100ms. > I did see your remarks about the benchmark on the tomcat user list. (1) the benchmark tries to closely simulate Apache Bench (ab), which is a well established and widely used tool for performance measurement. http://en.wikipedia.org/wiki/ApacheBench (2) the benchmark is not intended to calculate 'real-life' or 'accurate' response time, whatever that means. The purpose of the benchmark is to provide a _baseline_ for a comparative performance analysis and is intended to give a _rough_ indication as to whether an HTTP client A faster than an client HTTP, or whether data throughput increased or decreased after particular set of changes. That is it. Having said all that you are very welcome to suggest improvements to the benchmark or a more accurate algorithm for calculating performance numbers. Oleg > In either case I will see what if I can find again the client side code > soemwhere and compare it to what I have unless you have a link to it. > > Thanks, > -Tony > > > > ----- Original Message ---- > From: Oleg Kalnichevski <[email protected]> > To: HttpClient User Discussion <mailto:[email protected]> > Sent: Thu, March 3, 2011 2:46:03 AM > Subject: Re: Slowness of 4.1 > > On Thu, 2011-03-03 at 01:38 -0800, Tony Anecito wrote: > > Hi All, > > > > I downloaded httpclient 4.1 and noticed it is significantly slower than > > 3.1. >I > > > > even used the threadsafe connection manager hoping for better performance. > > I > > used to get below 3msec and now it is above 150msec. > > Is keepalive used by default? If not where can I find an example that sets > > it > > > using either a scheme, connection manager httpclient instance or the > > httpget > > object? > > > > What kind of response time should I expect hitting a servlet with no code > > in > >the > > > > method on a 2.8Ghz 6 core AMD server? > > > > Any ideas on what it might be? I used code sample from the site and it > > works > > just very slow. > > > > Thanks, > > -Tony > > > > HttpClient 4.1 is known to be comfortably faster than 3.1. There is > likely to be a problem with how you are measuring performance / response > time. > > http://wiki.apache.org/HttpComponents/HttpClient3vsHttpClient4vsHttpCore > > Oleg > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: >mailto:mailto:[email protected] > For additional commands, e-mail: >mailto:mailto:[email protected] > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
