I've been away from jmeter for a while. Working from memory:
The scripts I used had functional breakdown with the topic of interest
being latency. Each functional area had latency samples associated with
it. For example "login" for run1 would have X samples and the same was for
run2. It is important that measurement conditions be identical for each
run; otherwise, the performance of interest may be affected by
uncontrolled conditions (e.g., time-dependent internet delays). It is also
VERY important that the system of interest be in exactly the same condition
when the runs are done; if not, there may be memory impacts. I then did
paired t-testing to see if the runs were statistically similar (I think I
used a confidence interval of 0.95). X should be greater than around 10;
otherwise, there may be too much variability in the measurements and you
may have false conclusions.
Bohdan L. Bodnar
Lead Performance Engineer
1-312-871-5163
E-mail: [email protected] 222 South Riverside
Plaza
Chicago, IL
60606
United
States
From: Marek Czernek <[email protected]>
To: [email protected]
Date: 08/30/2018 09:11 AM
Subject: Best way to compare two results of jmeter
Hi there,
is there any 'supported' way to compare the results of 2 jmeter runs? I
googled around and found an old email from 2004 [1] basically saying
that there is no recommended solution other than a custom-made analysis.
Have there been any solutions to this problem?
I can also see a Jenkins plugin [2] though I have no idea in what state
the plugin is, and as such, how viable it is to run it. Last but not
least, there's some Grafana integration blog [3]. Does anyone have any
other suggestions? How would you compare two results programmatically to
see if there is degraded performance?
[1]
http://mail-archives.apache.org/mod_mbox/jmeter-user/200401.mbox/%3c99805b014a26d211bc3100a0c9b72cb8059e2...@exchange-va.noblestar.com%3E
[2]
https://wiki.jenkins.io/display/JENKINS/Performance+Plugin
[3]
http://www.testautomationguru.com/jmeter-real-time-results-influxdb-grafana/
Cheers,
--
Marek Czernek
JWS/JBCS Associate Quality Engineer, RHCA
Find me at www.halfastack.com
