On 2016-02-14 8:30 PM, Valentin Gosu wrote:
>Great! It'll be interesting to track how these change over time as well
>(or as versions get added to the list). Again, medians/means/etc may
>help with evaluating and tracking this (or automating notices, ala Talos)
>
I thought about doing this, but Talos is always using static content on a
local connection, whereas this goes over a real network which may vary
performance, and load real websites which may change content or optimize
for different situations. I expect it's useful for confirming certain
properties, such as if page loads are faster on Fx, Chrome or Nightly, and
by how much, but probably can't get results that make sense over a longer
period of time.
These things are true of Talos too, at least to an extent -- machine
configurations change, we modify tests, etc. As long as the numbers
remain relatively stable on a day-to-day basis, Perfherder might well be
able to generate useful alerts
(https://treeherder.allizom.org/perf.html#/alerts?status=-1&framework=1)
when someone checks something in that improves or regresses performance.
If you're interested in getting your performance framework submitting to
treeherder/perfherder, let me know. It's pretty trivial.
Will
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
