On 4 December 2011 20:24, Rainer Jung <[email protected]> wrote: > On 04.12.2011 20:54, Philippe Mouawad wrote: >> >> From my tests, I don't have such a drop in performances (max 2%). >> I also don't notice degradation on POST particularly. >> I agree with Sebb, issue are in 2.5 and 2.5.1 so we won't degrade things >> in >> a future 2.5.2. > > > I did a simple test using a very small file (2 bytes) to mostly check per > request overhead. I let it run with 10 threads for a total of 200.000 > samples and only took the last 20.000 samples to calculate results. > > Configuration was default, JVM was 1.6.0_29, System was Solaris Sparc with 2 > CPUs for JMeter and Apache on a separate one CPU system. > > CPU was not saturated, bandwidth neither. > > Those tests showed: > > - results for HttpClient3.1 and HttpClient4 are about the same > - results for JMeter 2.4, 2.5.1 and 2.5.2-dev are about the same > - response times measured with HttpClient are between 52% and 59% of the old > Java Sampler > - wallclock time needed for the 20.000 samples was only 0.3% to 2.2% bigger > than the sum of the response times, so overhead is minimal > - overhead, though minimal was about 2% for HttpClient and about 0.5 for the > old Java sampler. Overall it is a big difference, but both numbers are > pretty small. > - since overhead is small, throughput in requests per second behaves roughly > like average response time, namely about 740 requests per second for > HttpClient and about 400-440 for the old Java sampler. So throughput is > about 70% better for the newer samplers. > - CPU was higher for HttpClient, but only about 50-60%, so relative to > throughput (per request) it was a bit lower. > > "about the same" means differences were smaller than variability of test > runs, always less than 10%. > > It could be, that the test results will be very different, for bigger > response sizes, KeepAlive turned off, real live tests with cookies etc. etc. > > At least the base line looks good and I don't see a relevant difference > between 2.4 and 2.5.x.
Thanks very much, very useful analysis. > Regards, > > Rainer > > >> On Sun, Dec 4, 2011 at 5:27 PM, sebb<[email protected]> wrote: >> >>> On 4 December 2011 16:09, Rainer Jung<[email protected]> wrote: >>>> >>>> On 01.12.2011 22:57, Philippe Mouawad wrote: >>>>> >>>>> >>>>> Hello Sebb, >>>>> Don't you think we could make a release ? >>>>> >>>>> Lots of important fixes have been made and 2 months have passed since >>> >>> last >>>>> >>>>> release. >>>> >>>> >>>> >>>> First of all congrats to the huge progress you are making. >>>> >>>> What about BZ52189: "JMeter 2.5.1 slower than 2.4 for HTTP POST >>>> requests" >>>> >>>> Is that problem reproducible and really in the range described in the >>> >>> first >>>> >>>> comment, or was that due to comparing different http samplers? >>> >>> >>> Not sure; I've not been able to reproduce it yet, and the data so far >>> does not give much clue as to what is happening. >>> >>>> A drop in throughput from 130 to 80 just because of a newer version >>> >>> would be >>>> >>>> pretty serious IMHO. Unfortunately I didn't yet have the cycles to try >>>> it >>>> myself, but wanted to provide a heads up. >>> >>> >>> Agreed; however if the problem is difficult to solve I see no harm in >>> releasing another version so long as it is no worse than 2.5.1, and so >>> long as the problem is eventually resolved. >>> >>>> Regards, >>>> >>>> Rainer
