You might find https://bz.apache.org/bugzilla/show_bug.cgi?id=65353 interesting.
With the next nightly build or build from trunk, you should be able to use the new property "backend_metrics_percentile_estimator" with a value of "R_3" to lessen the difference between both reports. But keep in mind, that the HTML Report use a sliding window, while the Aggregate Report does not (and will eventually OOM). Felix Am 03.06.21 um 17:46 schrieb Deepak Chaudhari: > I need to send both the reports to the client and surely the question > will come "Why is there so much difference?" > If there is any way with which we can generate almost similar > aggregate and HTML reports? > > On Thu, Jun 3, 2021 at 9:00 PM Felix Schumacher > <[email protected] > <mailto:[email protected]>> wrote: > > Without the actual data it is impossible to say, if the reports > are wrong, or if they follow different ways to calculate a percentile. > > The problem here is, that the two reports are using different > algorithms to calculate the percentiles. We are working with > discrete numbers and here a percentile is more like a range than a > single (correct or wrong) number. Say, you have the values [1, 2, > 4, 8, 16, 32, 64, 128, 256, 512] want to calculate a 90% > percentile. For this you take any number that splits the (sorted) > list into two lists, where one list has all elements, that are > smaller (or equal) to the number and one list contains all > elements that are bigger than the number. For this example, any > number between 256 and 511.99 would be a valid 90% percentile. > (that is of course a broad description only) > > The Aggregate report currently uses a memory intensive way and > stores a list with all values and picks the smallest number, that > fits the above description. (This memory hogging behaviour should > be fixed and would lead to less accurate data and less OOMs or > slow JMeter instances) > > The HTML report gives you a value that would probably be a good > 90% percentile, if you would have more data like the one, you > already have by picking some value between the lower bound and the > upper bound of the described range (in our example something > between 256 and 511.999). > > As your report has very few samples (21 for the biggest difference > in the numbers), the skew between the reported percentiles can > look big, but are not necessarily wrong. > > Note, that if the numbers that both reports tell you are far > apart, that is probably a sign of having either too few samples, > or the samples (durations of the samples) are distributed sparsely > near the percentile ranges. > > We have discussed in the past to use the same algorithm for both > components, so that you get consistent values, but there were > always other issues, that seemed to be more important. > > Felix > > Am 02.06.21 um 15:28 schrieb Deepak Chaudhari: >> Hi, >> >> I collected JTL file during a test run and using that I'm >> generating HTML report and aggregate report. When I compare >> aggregate and HTML reports I found that 90th and 95th percentile >> values are wrong in HTML report. >> I tried to change "jmeter.reportgenerator.statistic_window" value >> in user.properties but still getting wrong values. >> >> *Aggregate report:* >> image.png >> >> *HTML report:* >> image.png >
OpenPGP_signature
Description: OpenPGP digital signature
