On 4/17/2013 3:07 PM, Matt Brubeck wrote:
analyze_talos.py does identify several real regressions in that dataset -- but it suppresses emails for them because each one is an increase of less than 2% (see bug 822249). Here are regressions identified by the latest version of analyze.py:
In case anyone wants to investigate these, I've added regression ranges below. These are automatically generated; I haven't double-checked them manually.
Since we run PGO builds only a few times a day, the ranges can be large. For those that include m-c merges, you could narrow them down using the m-c data. WebIDL seems to be a common theme.
changeset mem (KB) t-test % change ---------------------------------------- c1ee454506f6 3296860000 139.519 1.05%
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=88543d623c3f&tochange=c1ee454506f6 (Includes several Paris bindings changes, among others)
0acac77dd920 3330290000 11.809 0.99%
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=16ddbb6852ec&tochange=0acac77dd920 (More Paris binding changes, and others)
e6ca584f4fe7 3358010000 9.753 0.86%
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=a69f329fc7ee&tochange=e6ca584f4fe7 (includes the Win8 Metro merge)
80d52655c8b8 3398300000 74.742 0.45%
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=4637a1449900&tochange=80d52655c8b8 (includes some WebIDL-related changes)
d27445d1eac5 3450400000 26.214 0.99%
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=92824d900e25&tochange=d27445d1eac5
eaff15332579 3468970000 70.525 0.52%
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=e38c5c346840&tochange=eaff15332579 (Olli Pettay — Bug 822399 - Make Event to use Paris bindings)
Perhaps we should modify bug 822249 to ignore the 2% threshold for specific tests where we *do* care about small changes. Or for any change with a very high t-test score. I filed this bug: https://bugzilla.mozilla.org/show_bug.cgi?id=863061
A fix has been checked in, so we should start getting alerts for future linker memory regressions of any size.
_______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
