You rock Felix ! On Fri, Jun 4, 2021 at 5:39 PM Deepak Chaudhari <[email protected]> wrote:
> I downloaded JMeter 5.5 and compared aggregate and HTML reports. > Now the results are exactly the same > > Thank you very much Felix for fixing it on such short notice. :) > > [image: image.png] > > [image: image.png] > . > > On Thu, Jun 3, 2021 at 11:14 PM Felix Schumacher < > [email protected]> wrote: > >> You might find https://bz.apache.org/bugzilla/show_bug.cgi?id=65353 >> interesting. >> >> With the next nightly build or build from trunk, you should be able to use >> the new property "backend_metrics_percentile_estimator" with a value of >> "R_3" to lessen the difference between both reports. But keep in mind, that >> the HTML Report use a sliding window, while the Aggregate Report does not >> (and will eventually OOM). >> >> Felix >> >> >> Am 03.06.21 um 17:46 schrieb Deepak Chaudhari: >> >> I need to send both the reports to the client and surely the question >> will come "Why is there so much difference?" >> If there is any way with which we can generate almost similar aggregate >> and HTML reports? >> >> On Thu, Jun 3, 2021 at 9:00 PM Felix Schumacher < >> [email protected]> wrote: >> >>> Without the actual data it is impossible to say, if the reports are >>> wrong, or if they follow different ways to calculate a percentile. >>> >>> The problem here is, that the two reports are using different algorithms >>> to calculate the percentiles. We are working with discrete numbers and here >>> a percentile is more like a range than a single (correct or wrong) number. >>> Say, you have the values [1, 2, 4, 8, 16, 32, 64, 128, 256, 512] want to >>> calculate a 90% percentile. For this you take any number that splits the >>> (sorted) list into two lists, where one list has all elements, that are >>> smaller (or equal) to the number and one list contains all elements that >>> are bigger than the number. For this example, any number between 256 and >>> 511.99 would be a valid 90% percentile. (that is of course a broad >>> description only) >>> >>> The Aggregate report currently uses a memory intensive way and stores a >>> list with all values and picks the smallest number, that fits the above >>> description. (This memory hogging behaviour should be fixed and would lead >>> to less accurate data and less OOMs or slow JMeter instances) >>> >>> The HTML report gives you a value that would probably be a good 90% >>> percentile, if you would have more data like the one, you already have by >>> picking some value between the lower bound and the upper bound of the >>> described range (in our example something between 256 and 511.999). >>> >>> As your report has very few samples (21 for the biggest difference in >>> the numbers), the skew between the reported percentiles can look big, but >>> are not necessarily wrong. >>> >>> Note, that if the numbers that both reports tell you are far apart, that >>> is probably a sign of having either too few samples, or the samples >>> (durations of the samples) are distributed sparsely near the percentile >>> ranges. >>> >>> We have discussed in the past to use the same algorithm for both >>> components, so that you get consistent values, but there were always other >>> issues, that seemed to be more important. >>> >>> Felix >>> Am 02.06.21 um 15:28 schrieb Deepak Chaudhari: >>> >>> Hi, >>> >>> I collected JTL file during a test run and using that I'm generating >>> HTML report and aggregate report. When I compare aggregate and HTML reports >>> I found that 90th and 95th percentile values are wrong in HTML report. >>> I tried to change "jmeter.reportgenerator.statistic_window" value in >>> user.properties but still getting wrong values. >>> >>> *Aggregate report:* >>> [image: image.png] >>> >>> *HTML report:* >>> [image: image.png] >>> >>> -- Cordialement Philippe M. Ubik-Ingenierie
