Who's responsible for looking into the test/regression? Bas? Does the person looking into it need help from the performance or desktop teams?
What bug is tracking that work? Gavin On Wed, May 28, 2014 at 12:12 PM, Vladimir Vukicevic <vladi...@mozilla.com>wrote: > (Note: I have not looked into the details of CART/TART and their > interaction with OMTC) > > It's entirely possible that (b) is true *now* -- the test may have been > good and proper for the previous environment, but now the environment > characteristics were changed such that the test needs tweaks. Empirically, > I have not seen any regressions on any of my Windows machines (which is > basically all of them); things like tab animations and the like have > started feeling smoother even after a long-running browser session with > many tabs. I realize this is not the same as cold hard numbers, but it > does suggest to me that we need to take another look at the tests now. > > - Vlad > > ----- Original Message ----- > > From: "Gijs Kruitbosch" <gijskruitbo...@gmail.com> > > To: "Bas Schouten" <bschou...@mozilla.com>, "Gavin Sharp" < > ga...@gavinsharp.com> > > Cc: dev-tech-...@lists.mozilla.org, "mozilla.dev.platform group" < > dev-platform@lists.mozilla.org>, "release-drivers" > > <release-driv...@mozilla.org> > > Sent: Thursday, May 22, 2014 4:46:29 AM > > Subject: Re: OMTC on Windows > > > > Looking on from m.d.tree-management, on Fx-Team, the merge from this > > change caused a >40% CART regression, too, which wasn't listed in the > > original email. Was this unforeseeen, and if not, why was this > > considered acceptable? > > > > As gavin noted, considering how hard we fought for 2% improvements (one > > of the Australis folks said yesterday "1% was like Christmas!") despite > > our reasons of why things were really OK because of some of the same > > reasons you gave (e.g. running in ASAP mode isn't realistic, "TART is > > complicated", ...), this hurts - it makes it seem like (a) our > > (sometimes extremely hacky) work was done for no good reason, or (b) the > > test is fundamentally flawed and we're better off without it, or (c) > > when the gfx team decides it's OK to regress it, it's fine, but not when > > it happens to other people, quite irrespective of reasons given. > > > > All/any of those being true would give me the sad feelings. Certainly it > > feels to me like (b) is true if this is really meant to be a net > > perceived improvement despite causing a 40% performance regression in > > our automated tests. > > > > ~ Gijs > > > > On 18/05/2014 19:47, Bas Schouten wrote: > > > Hi Gavin, > > > > > > There have been several e-mails on different lists, and some > communication > > > on some bugs. Sadly the story is at this point not anywhere in a > condensed > > > form, but I will try to highlight a couple of core points, some of > these > > > will be updated further as the investigation continues. The official > bug > > > is bug 946567 but the numbers and the discussion there are far outdated > > > (there's no 400% regression ;)): > > > > > > - What OMTC does to tart scores differs wildly per machine, on some > > > machines we saw up to 10% improvements, on others up to 20% > regressions. > > > There also seems to be somewhat more of a regression on Win7 than > there is > > > on Win8. What the average is for our users is very hard to say, > frankly I > > > have no idea. > > > - One core cause of the regression is that we're now dealing with two > D3D > > > devices when using Direct2D since we're doing D2D drawing on one > thread, > > > and D3D11 composition on the other. This means we have DXGI locking > > > overhead to synchronize the two. This is unavoidable. > > > - Another cause is that we're now having two surfaces in order to do > double > > > buffering, this means we need to initialize more resources when new > layers > > > come into play. This again, is unavoidable. > > > - Yet another cause is that for some tests we composite 'ASAP' to get > > > interesting numbers, but this causes some contention scenario's which > are > > > less likely to occur in real-life usage. Since the double buffer might > > > copy the area validated in the last frame from the front buffer to the > > > backbuffer in order to prevent having to redraw much more. If the > > > compositor is compositing all the time this can block the main thread's > > > rasterization. I have some ideas on how to improve this, but I don't > know > > > how much they'll help TART, in any case, some cost here will be > > > unavoidable as a natural additional consequence of double buffering. > > > - The TART number story is complicated, sometimes it's hard to know > what > > > exactly they do, and don't measure (which might be different with and > > > without OMTC) and how that affects practical performance. I've been > told > > > this by Avi and it matches my practical experience with the numbers. I > > > don't know the exact reasons and Avi is probably a better person to > talk > > > about this than I am :-). > > > > > > These are the core reasons that we were able to identify from > profiling. > > > Other than that the things I said in my previous e-mail still apply. We > > > believe we're offering significant UX improvements with async video and > > > are enabling more significant improvements in the future. Once we've > fixed > > > the obvious problems we will continue to see if there's something that > can > > > be done, either through tiling or through other improvements, > particularly > > > in the last point I mentioned there might be some, not 'too' complex > > > things we can do to offer some small improvement. > > > > > > If we want to have a more detailed discussion we should probably pick a > > > list to have this on and try not to spam people too much :-). > > > > > > Bas > > > > > > ----- Original Message ----- > > > From: "Gavin Sharp" <ga...@gavinsharp.com> > > > To: "Bas Schouten" <bschou...@mozilla.com> > > > Cc: "dev-tree-management" <dev-tree-managem...@lists.mozilla.org>, > > > dev-tech-...@lists.mozilla.org, "release-drivers" > > > <release-driv...@mozilla.org>, "mozilla.dev.platform group" > > > <dev-platform@lists.mozilla.org> > > > Sent: Sunday, May 18, 2014 6:23:58 PM > > > Subject: Re: OMTC on Windows > > > > > >> but tart will regress by ~20%, and several other suites will regress > as > > >> well. > > >> We've investigated this extensively and we believe the majority of > these > > >> regressions are due to the nature of OMTC and the fact that we have > to do > > >> more work. > > > > > > Where can I read more about the TART investigations? I'd like to > > > understand why it is seen as inevitable, and get some of the details > > > of the regression. OMTC is important, and I'm excited to see it land > > > on Windows, but the Firefox and Performance teams have just come off a > > > months-long effort to make significant wins in TART, and the thought > > > of taking a 20% regression (huge compared to some of the improvements > > > we fought for) is pretty disheartening. > > > > > > Gavin > > > > > > On Sun, May 18, 2014 at 12:16 AM, Bas Schouten <bschou...@mozilla.com> > > > wrote: > > >> Hey all, > > >> > > >> After quite a lot of waiting we've switched on OMTC on Windows by > default > > >> today (bug 899785). This is a great move towards moving all our > platforms > > >> onto OMTC (only linux is left now), and will allow us to remove a lot > of > > >> code that we've currently been duplicating. Furthermore it puts us on > > >> track for enabling other features on desktop like APZ, off main thread > > >> animations and other improvements. > > >> > > >> Having said that we realize that what we've currently landed and > turned on > > >> is not completely bug free. There's several bugs still open (some more > > >> serious than others) which we will be addressing in the coming weeks, > > >> hopefully before the merge to Aurora. The main reason we've switched > it > > >> on now is that we want to get as much data as possible from the > nightly > > >> channel and our nightly user base before the aurora merge, as well as > > >> wanting to prevent any new regressions from creeping in while we fix > the > > >> remaining problems. This was extensively discussed both internally in > the > > >> graphics team and externally with other people and we believe we're > at a > > >> point now where things are sufficiently stabilized for our nightly > > >> audience. OMTC is enabled and disabled with a single pref so if > > >> unforeseen, serious consequences occur we can disable it quickly at > any > > >> stage. We will inevitably find new bugs in the coming weeks, please > link > > >> any bugs you happen to come across to bug 899785, if anything > > se > > >> ems very serious, please let us know, we'll attempt to come up with > a > > >> solution on the short-term rather than disabling OMTC and reducing > the > > >> amount of feedback we get. > > >> > > >> There's also some important notes to make on performance, which we > expect > > >> to be reported by our automated systems: > > >> > > >> - Bug 1000640 is about WebGL. Currently OMTC regresses WebGL > performance > > >> considerably, patches to fix this are underway and this should be > fixed > > >> on the very short term. > > >> > > >> - Several of the Talos test suite numbers will change considerably > > >> (especially with Direct2D enabled), this means Tscroll for example > will > > >> improve by ~25%, but tart will regress by ~20%, and several other > suites > > >> will regress as well. We've investigated this extensively and we > believe > > >> the majority of these regressions are due to the nature of OMTC and > the > > >> fact that we have to do more work. We see no value in holding off OMTC > > >> because of these regressions as we'll have to go there anyway. Once > the > > >> last correctness and stability problems are all solved we will go > back to > > >> trying to find ways to get back some of the performance regressions. > > >> We're also planning to move to a system more like tiling in desktop, > > >> which will change the performance characteristics significantly > again, so > > >> we don't want to sink too much time into optimizing the current > > >> situation. > > >> > > >> - Memory numbers will increase somewhat, this is unavoidable, there's > > >> several steps which have to be taken when doing off main thread > > >> compositing (like double-buffering), which inherently use more memory. > > >> > > >> - On a brighter note: Async video is also enabled by these patches. > This > > >> means that when the main thread is busy churning JavaScript, instead > of > > >> stuttering your video should now happily continue playing! > > >> > > >> - Also there's some indications that there's a subjective increase in > > >> scrolling performance as well. > > >> > > >> > > >> If you have any questions please feel free to reach out to myself or > other > > >> members of the graphics team! > > >> > > >> > > >> Bas > > >> _______________________________________________ > > >> dev-platform mailing list > > >> dev-platform@lists.mozilla.org > > >> https://lists.mozilla.org/listinfo/dev-platform > > > > _______________________________________________ > > dev-tech-gfx mailing list > > dev-tech-...@lists.mozilla.org > > https://lists.mozilla.org/listinfo/dev-tech-gfx > > > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
