On Friday, May 30, 2014 5:14:20 AM UTC+3, avi...@gmail.com wrote: > So, wrt TART, I now took the time to carefully examine tab animation visually > on one system. > > > > TL;DR: > > - I think OMTC introduces a clearly visible regression with tab animation > compared to without OMTC. > > - I _think_ it regresses more with tab close than with tab open animation. > > - The actual throughput regression is probably bigger than indicated by TART > numbers. > > > > > > The reason for the negative bias is that the TART results are an average of > 10 different animations, but only one of those is close to pure graphics perf > numbers, and when you look only on this test, the regression is bigger than > 50-100% (more like 100-400%). > > > > The details: > > > > System: Windows 8.1 x64, i7-4500u, using Intel's iGPU (HD4400), and with > official Firefox nightly 32bit (2014-05-29). > > > > First, visually: both with and without ASAP mode, to my eyes, tab animation > with OMTC is less smooth, and seems to have lower frame rate than without > OMTC. > > > > As for what TART measures, of all the TART subtests, there are 3 which are > most suitable for testing pure graphics performance - they test the css > fade-in and fade-out (that's the close/open animation) of a tab without > actually opening or closing a browser tab, so whatever performance it has, > the limit comes only from the animation itself and it doesn't include other > overheads. > > > > These tests are the ones which have "fade" in their name, and only one of > them is enabled by default in talos - the other two are available only when > running TART locally and then manually selecting animations to run. > > > > I'll focus on a single number which is the average frame interval of the > entire animation (these are the ".all" numbers), for the fade animation at > default DPI (which is 1 on my system - so the most common). > > > > What TART measures locally on my system: > > > > OMTC without ASAP mode (as out of the box config as it gets): > > iconFade-close-DPIcurrent.all Average (5): 18.91 stddev: 0.86 > > iconFade-open-DPIcurrent.all Average (5): 17.61 stddev: 0.78 > > > > OMTC with ASAP: > > iconFade-close-DPIcurrent.all Average (5): 18.47 stddev: 0.46 > > iconFade-open-DPIcurrent.all Average (5): 10.08 stddev: 0.46 > > > > While this is an average of only 5 runs, stddev shows it's reasonably > consistent, and the results are also consistent when I tried more. > > > > We can already tell that close animation just doesn't get below ~18.5ms/frame > on this system, ASAP doesn't affect it at all. We can also see that the open > animation is around 60fps without ASAP (17.6 can happen with our inaccurate > interval timers) and with ASAP it goes down to about 10ms/frame. > > > > Without OMTC and without ASAP: > > iconFade-close-DPIcurrent.all Average (5): 16.54 stddev: 0.16 > > iconFade-open-DPIcurrent.all Average (5): 16.52 stddev: 0.12 > > > > Without OMTC and with ASAP: > > iconFade-close-DPIcurrent.all Average (5): 5.53 stddev: 0.07 > > iconFade-open-DPIcurrent.all Average (5): 6.37 stddev: 0.08 > > > > The results are _much_ more stable (stddev), and quite lower (in ASAP) and > closer to 16.7 in "normal" mode. > > > > While I obviously can't visually notice differences when the frame rate is > higher than my screen's 60hz, from what I've seen so far, both visually and > at the numbers, I think TART is not less reliable than before, it doesn't > look to me as if ASAP introduces very bad bias (I couldn't deduct any), and > OMTC does seem regress tab animations meaningfully. > > > > - avih
Earlier this week, after windows update which included an Intel driver update (to 10.18.10.3621 2014-05-16), I noticed a considerable improvement in Firefox performance and smoothness, with both release and nightly versions. So I got back to the lab and grabbed some numbers. Same build as before (nightly 32 bit 2014-05-29) and same system (i7-4500u, HD4400, win 8.1 64). Here are the measurements with the new Intel driver: OMTC without ASAP: iconFade-close-DPIcurrent.all Average (5): 16.35 stddev: 0.20 iconFade-open-DPIcurrent.all Average (5): 16.68 stddev: 0.15 This already looks considerably more stable than before. OMTC with ASAP: iconFade-close-DPIcurrent.all Average (5): 4.80 stddev: 0.23 iconFade-open-DPIcurrent.all Average (5): 3.32 stddev: 0.05 This is actually pretty fast! about 3x faster than before, or even a bit faster (vs 18ms, 10ms respectively). Yay! Without OMTC and without ASAP: same as before Without OMTC and with ASAP: iconFade-close-DPIcurrent.all Average (5): 2.16 stddev: 0.11 iconFade-open-DPIcurrent.all Average (5): 2.96 stddev: 0.06 Without OMTC it's flying! again, about 2.5x faster than before. The regression between yes and no OMTC is smaller with this driver, but still not negligible (150% on close tab animation, 10% on open tab animation, compared to before with ~200%, ~50%, respectively). Obviously with such numbers, but worth mentioning explicitly, I can't visually tell a difference between yes/no OMTC. And this is only from a newer Intel driver! I took the liberty to also run the scroll test (bookmarklet from bug 894128) on the same system and with the Firefox Wikipedia page. It scrolls for 5 seconds in ASAP mode and reports average interval and stddev, and I used full screen (1920x1080) with normal DPI of 1, with the same nightly build as above: OMTC: Average interval: 5.08 ms STDDEV intervals: 1.02 ms OMTC:off Average interval: 8.19 ms STDDEV intervals: 1.22 ms We've known it already, but here's another confirmation: OMTC improves scrolling very nicely. Yay #2 So, encouraged with these very nice numbers, I decided to also test the latest nightly (2014-06-16). It turns out that all the numbers (TART, scroll-test with[/out] OMTC) are more or less in the same ballpark as with the 2014-05-29 build. However, on this specific system, and I'm guessing many others with a recent Intel iGPU, these latest drivers make a huge difference, ~2-3x faster than before on All Firefox versions, with or without OMTC, and it's very noticeable. -avih _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
