Hello, I have just made the same test with a JSP (doing a Thread.sleep(2000) => 2s) behind a Tomcat 6 with JMeter 2.6
I put 100 threads that loop 100 times in GUI mode (one JMETER): All results shown in Aggregate report are around 2s . sampler_label,aggregate_report_count,average,aggregate_report_median,aggregate_report_90%_line,aggregate_report_min,aggregate_report_max,aggregate_report_error%,aggregate_report_rate,aggregate_report_bandwidth HTTP Request,10000,2006,2004,2012,2001,2195,0.0,49.5793194742609,1224.082013722936 TOTAL,10000,2006,2004,2012,2001,2195,0.0,49.5793194742609,1224.082013722936 Summary listener shows this: 2012/02/10 14:36:00 INFO - jmeter.reporters.Summariser: Generate Summary Results + 2655 in 54,8s = 48,4/s Avg: 2010 Min: 2001 Max: 2195 Err: 0 (0,00%) 2012/02/10 14:38:26 INFO - jmeter.reporters.Summariser: Generate Summary Results + 7345 in 148,9s = 49,3/s Avg: 2005 Min: 2001 Max: 2088 Err: 0 (0,00%) 2012/02/10 14:38:26 INFO - jmeter.reporters.Summariser: Generate Summary Results = 10000 in 201,7s = 49,6/s Avg: 2006 Min: 2001 Max: 2195 Err: 0 (0,00%) So as you can see, nothing abnormal . JSP Code: <%@ page contentType="text/html; charset=UTF-8" %> <html> <body> <% Thread.sleep(2000); %> <!-- GENERATE around 25Ko page --> <%=org.apache.commons.lang.RandomStringUtils.random(25000, "abcdefghijklmnopqrstuvwxyz01234567890")%> <p>Test</p> </body> </html> Regards Philippe http://www.ubik-ingenierie.com On Thu, Feb 9, 2012 at 4:52 PM, sebb <seb...@gmail.com> wrote: > On 9 February 2012 15:38, Robin D. Wilson <rwils...@gmail.com> wrote: > > Thanks sebb for the replies... > > > > Here's the deal, I am running the same test script on JM2.4 and JM2.6. I > am running in GUI mode. The test script has 3 thread groups > > - but the first and the last thread group is just a 'timer' I created to > log the total elapsed time of the test (the first and last > > group has 1 thread, and 1 request, and take less than 1 second each to > run). The 'real' test is the middle thread group. It has 100 > > threads (0 ramp), and runs 100 iterations (10,000 total samples). It > simply does a 'POST' to a URL, with 15 > > > > So the 'elapsed time' I referring to in my test is actually the > timestamp taken in the first thread group (in ms since epoch) > > subtracted from the timestamp taken in the 3rd (last) thread group. That > part of my test may only add 2 total seconds to the test, > > so while it may skew my results slightly - it doesn't explain the vast > difference in the 'average' sample duration. According to the > > Summary Report docs, the "Average" is supposed to be "the average > elapsed time of a set of samples". But clearly, if the minimum > > time it takes to actually get the page is 2 seconds (due to the built-in > delay in the cgi-script), there is no way I could have an > > 'average' elapsed time of less than 2 seconds, yet I'm showing an > average elapsed time of ~750 ms... (My "Max" elapsed time shows as > > only 1198!). When I request the page in Firefox, it takes ~2104ms (using > a status bar timer), so I think the cgi script is working > > correctly.) > > > > Sebb asked: > > > >>Again, the throughput calculations are based on total test time. Are you > sure the test run times are comparable? > > > > The test run times are automatically calculated by the 1st and 3rd > thread groups. The ~210 seconds total elapsed time is accurate > > based on my external measurement too (e.g., it is close to what I can > observe with my stopwatch). > > > > Both the JM2.4 test and the JM2.6 test are using the exact same ".jmx" > test file. > > > >>There's clearly something else going on here. > > > > I don't believe that the Summary Report is accurately calculating > anything except the total number of samples and the Avg. Bytes... > > What makes you say that? > Are the Min and Max really incorrect? > Error %? > > It's easy enough to check the Summary Results if you can provide the > CSV sample result files. > > > The cgi-script I'm using definitely takes 2+ seconds to respond after it > gets the request (I've measured this with Firefox directly, > > and it _never_ gets a response in less than 2 seconds). I even changed > the 'sleep' to 9 seconds, and JMeter pauses for that long in > > recording results (e.g., it shows 100 threads run, then waits 9 seconds, > shows another 100 threads, etc.), but the numbers just go > > up to '1758' Average, and '2415' Max (which is impossible since it is > taking 9+ seconds to respond to each request!). It takes over > > 15 minutes to complete 10,000 samples (and that seems about right - > 10000 samples/100 threads * 9 seconds each = 900 seconds). > > > > I even went so far as to inject a 2 second sleep in the middle of the > response (e.g., pause 2 seconds - send part of the response - > > pause 2 more seconds - send the rest), I'm still getting average times > of ~1000 ms. (That's with 4 seconds of built-in delays, and 2 > > of those seconds are in the middle of the response.) The browser shows > this delay properly, but JMeter isn't calculating it > > properly. > > > >>Please recheck the individual sample response times and see how they > compare to the average. > > > > I'm not sure how to do that in JMeter. I can manually hit the page, and > it takes about 100ms longer than the built-in delay I have. > > Add a Table View Listener, or just check the CSV sample result files. > > >>If there still appears to be a problem, create a Bugzilla issue and > attach: > >>- JMX test case > > > > I'm trying to simplify the test case to the bare minimum case - so the > results will be indisputable. I will also include the > > 'cgi-bin' script that I'm using, so someone else can easily setup the > same test. > > Thanks. > > > > >>- log files for JMeter 2.4 and 2.6 > > > > Which log files are these? Is it just the 'jmeter.log' that gets created > in the 'bin' folder when I run the GUI mode, or do you need > > another log file? > > jmeter.log > > >>- CSV result files for 2.4 and 2.6 > > > > I can do this. > > > > -- > > Robin D. Wilson > > Sr. Director of Web Development > > KingsIsle Entertainment, Inc. > > VOICE: 512-777-1861 > > www.KingsIsle.com > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org > > For additional commands, e-mail: user-h...@jmeter.apache.org > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@jmeter.apache.org > For additional commands, e-mail: user-h...@jmeter.apache.org > > -- Cordialement. Philippe Mouawad.
