Hey gandalf, Using WICG for browser start-up makes sense to me.
We should also consider adding a milestone for the "hero element" for the browser. There's some discussion fragments in bug 1369417 about this (which I see you're already involved in! Great!), captured from a very caffeinated discussion with jmaher, florian and a few other folks in SF this past all-hands. At any rate, this all sounds strictly better than what ts_paint currently captures. We just need to ensure that we measure paint times when they're presented to the user (so after composite), and using the timestamps on the MozAfterPaint events themselves[1] (and not taking a timestamp at the event-servicing time, as this adds noise and padding to the timestamp). So, uh, thumbs up from me. :) -Mike [1]: http://searchfox.org/mozilla-central/rev/f2a1911ad310bf8651f342d719e4f4ca0a7b9bfb/dom/webidl/NotifyPaintEvent.webidl#36 On 2017-08-31 6:51 PM, zbranie...@mozilla.com wrote: > Gecko has a pretty substantial number of metrics used for measuring startup > performance. > > We have probes in Telemetry [0], StartupTimeline [1], various uses of > MozAfterPaint, both in talos tests [2] and in production code [3][4]. > We also have first paint in compositor [5], before-first-paint [6], > timeToNonBlankPaint [7] which should not be confused with firstNonBlankPaint > [8] and probably a number of other probes that register different timestamps > and are used all around to mean different things. Some measure layout, others > composition. Many of them are misused by capturing the timestamp in the > callback to event listener fired asynchronously post-event. > We end up seeing huge swings in some "first-paint" [9] that are not > reproducible in another "first-paint" [10], and we know that things like > WebRender may not affect some of our "first-paint" because it measures only > part of paint that WebRender doesn't affect[11]. > > It doesn't help that some of them are chrome-only while others are available > in content. > > I believe that, while we can recognize the complexity of the systems in play > and how this complexity explains why different probes ended up being > constituted, this situation is counter productive to our ability to > understand the performance implication of our changes in product. > > I'd like to suggest establishing a single set of timestamps with unique names > to represent various stages of the product launch. > Those timestamps would be described based on the user-perceived results and > as such serve us as best-effort approximations of the impact of any change on > the user-perceived experience. > > In particular, my proposal is based on WICG Paint Timing proposal [12] and > establishes the 5 major timestamps to be executed at the latest event that > contributes to the user-perceived outcome. > For example, when selecting when to mark "xPaint" event, we will use the > consumer notion of the term "paint" and mark it after *all* operations > required for the paint to happen are done - layout, composition, rendering > and paint. > > My proposal is also based on the work on negotiated performance milestones > established for Firefox OS project [13]. > > The proposed milestones are: > > 1) firstPaint > > This milestone happens when the first paint that is influenced by the data > for the measured object is completed by the engine (and likely submitted to > the graphic driver). > > In the context of an HTML document, the first paint that is affected by the > document's background or title is completed. > > 2) firstContentfulPaint > > The first contentful paint of browser.xul happens when the first paint that > includes layout of DOM data from browser.xul is completed. > > 3) visuallyCompletedPaint > > This milestones is achieved after the first paint with the above-the-fold > part of the DOM ready is submitted. This event may require the document to > inform the engine that all the items in the visual field are ready, and the > next paint captures the timestamp. > > 4) chromeInteractive > > This milestone is achieved when the app reports the UI to be ready to be > interacted with. The definition of what constitutes of the UI being ready is > up to the app, and may just include the URL bar being ready to receive URL > input, or may wait for all core parts of the product to have event handlers > attached (urlbar, tabbar, main menu etc.) > This milestone may be reached before (3), but not after (5). > > 5) fullyLoaded > > This milestone also may require data from the document and it should mark the > timestamp when all startup operations are completed. > This should include delayed startup operations that may have waited for > previous stages to be achieved, but should not wait for non-startup delayed > operations like periodic updates of data from AMO etc. > > The last milestone is a good moment to reliably measure memory consumption > (possibly after performing GC), take a screenshot for any tests that compare > UI between starts and so on. > > Generally speaking, (5) is when we naively can say "the app is fully > launched". > > > ================== > > This system would require us to provide a way for each document to inform the > engine when some of the later stages are reached. Then the engine would take > the next full paint and capture the timestamp. > Such timestamp list would be available for chrome and behind a flag for > content, to be used by tests. > > The value of such milestone approach lies not only in unification of reading, > but also easier hooking of code into the bootstrap process. Having such a > consistent multi-stage bootstrap allows developers to decide at which stage > their code has to be executed to delay only what has to be affected by it and > in result developing a culture of adding code that doesn't affect early > stages of the bootstrap unnecessarily. > > Lastly of course, the value comes in our ability to say that all telemetry > probes, talos tests, tp6 tests etc. and automation can now rely on a single > set of timestamps which would increase developers ability to understand and > verify how their patches affect the user perceived experience. > > I do not expect us to place all of them at once, but I believe that there's a > value in having a conversation and establishing the list of marks we want to > unify around and then start moving toward them. > In the end, the bootstrap timestamps of browser.xul should be read in a very > similar fashion to how we read bootstrap timestamps of amazon.com with the > difference being that later stages like "visuallyLoaded" or "interactive" may > be triggered by browser.xul's JS code, while for amazon, we'll either not > capture them, or use some custom code in testing machinery to decide when the > hero elements are interactive. > > zb. > > > [0] > http://searchfox.org/mozilla-central/source/toolkit/components/startup/StartupTimeline.cpp#51 > [1] > http://searchfox.org/mozilla-central/source/toolkit/components/startup/StartupTimeline.h#15 > [2] > http://searchfox.org/mozilla-central/source/testing/talos/talos/tests/tpaint/file_tpaint_win.html#4 > [3] > http://searchfox.org/mozilla-central/source/browser/base/content/browser.js#1388 > [4] > http://searchfox.org/mozilla-central/source/browser/base/content/tabbrowser.xml#1124 > [5] > http://searchfox.org/mozilla-central/source/gfx/layers/ipc/UiCompositorControllerMessageTypes.h#24 > [6] http://searchfox.org/mozilla-central/source/dom/ipc/TabChild.cpp#2847 > [7] > http://searchfox.org/mozilla-central/source/dom/performance/PerformanceTiming.h#249 > [8] > http://searchfox.org/mozilla-central/source/layout/base/nsPresContext.h#1430 > [9] > https://telemetry.mozilla.org/new-pipeline/evo.html#!aggregates=75th-percentile!95th-percentile&cumulative=0&end_date=null&keys=&max_channel_version=nightly%252F57&measure=SIMPLE_MEASURES_FIRSTPAINT&min_channel_version=nightly%252F54&processType=*&product=Firefox&sanitize=1&sort_keys=submissions&start_date=null&trim=1&use_submission_date=0 > [10] > https://treeherder.mozilla.org/perf.html#/graphs?series=mozilla-central,1517547,1,1&series=mozilla-central,1535589,1,1&series=mozilla-central,1535590,1,1&series=mozilla-central,1535587,1,1&series=mozilla-central,1535588,1,1 > [11] https://bugzilla.mozilla.org/show_bug.cgi?id=1388157#c15 > [12] https://w3c.github.io/paint-timing/ > [13] > https://developer.mozilla.org/en-US/docs/Archive/B2G_OS/Developing_Gaia/Raptor/Responsiveness_guidelines > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
