I'd also like to add that we landed a series of performance patches to Music NGA on Friday which bring our cold launch startup times to within 100ms of the OGA app. We still have several more performance patches forthcoming that should bring that number down even more and also help cut our memory utilization (I actually wouldn't be surprised if we end up beating the old cold launch numbers). So, for dogfooders, they should not be seeing any noticeable startup time regressions anymore with today's build. All the while, we are getting more eyes on the new app and are addressing any new bugs reported that we would not be seeing if we hadn't landed. We waited until we felt we had a stable/feature-complete version of the app before we landed on master. While we knew the app was not fully optimized yet, the version of the app we initially landed should never have felt incomplete, buggy or drastically slower than the old app to anyone dogfooding master. Additionally, if we thought we would not be able to get performance right, we would not have landed in the first place. Now that we've practically closed the gap in the startup time metric, all that remains is tightening up the memory utilization.
-Justin > On Oct 5, 2015, at 6:47 PM, Eli Perelman <[email protected]> wrote: > > If by try runs you mean automated performance testing when opening a PR, then > no. Right now the best way to ensure performance is up-to-snuff with your > patch is to run Raptor during development. With Raptor installed, use a > performance profile on the device by using the same flags as `make raptor`, > then testing an app is as easy as: > > raptor test coldlaunch --app clock --runs 10 > raptor test coldlaunch --app communications --entry-point dialer --runs 20 > > During development use a small number of runs for a tighter feedback loop. > Before committing/landing, use many runs to ensure a better statistical > guarantee. See the Raptor docs for details on getting started [1]. If you > need any help getting up and running with Raptor, Rob Wood and myself would > be happy to lend a hand, just ping us [2]. > > [1] > https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Automated_testing/Raptor#Getting_Started > [2] We're in #fxos or #raptor > > Eli Perelman > >> On Mon, Oct 5, 2015 at 5:37 PM, Naoki Hirata <[email protected]> wrote: >> I think to add what dietrich is saying, I personally am not condeming >> performance on your own local branch or doing test runs there... >> >> I think we're condeming anything that lands on master as an experiment, >> without testing or without running through our jenkins setup etc for testing >> performance. >> >> I don't want to speak too much for the performance team, I'm not sure what >> their load is in handling requests though. Eli is there an easy way for >> dev to setup try runs w/ performance testing? >> >>> On Mon, Oct 5, 2015 at 1:53 PM, Dietrich Ayala <[email protected]> wrote: >>> +1 to the frog metaphor. >>> >>> History has shown it's *incredibly* hard to claw back from performance >>> regressions. And every moment spent doing so is done *at the cost* of >>> exactly the type of work Chris described - work that actually moves the >>> project *forward*. >>> >>> I strongly disagree with any acceptance of any performance regression for >>> any reason except emergency security patches. Only a zero tolerance policy >>> for perf regressions will result in performant software in such a large and >>> complex project. >>> >>> If you have a tension between perf and features, then it's time to cut the >>> slow features, or get some more time. >>> >>> The polish/bugs problems mentioned is fixed by landing fewer bugs (a >>> culture of detailed automated tests and a project-wide love and acceptance >>> of backouts), not by accepting perf regressions. >>> >>> Also, I recommend not using any subjective measure to compare app startup >>> times across different platforms. We used tools to do this in the past. >>> >>> (My first patch ever, in 2006, regressed Firefox startup time and I spent a >>> few days on the hook... until my feature could land with no startup hit. >>> Can you tell it had an impact on me :D) >>> >>>> On Mon, Oct 5, 2015 at 5:19 PM Fabrice Desré <[email protected]> wrote: >>>> On 10/05/2015 01:46 AM, Christopher Lord wrote: >>>> > I'll preface what I say with the hopefully obvious statement that we >>>> > should always aim for everything to be better. That said, however, I'd >>>> > take a 2mb memory regression and a half-second startup time regression >>>> > if it meant the app was polished and performed well. >>>> >>>> Some apps regressed by way more than 2MB. And also, beware of the >>>> boiling frog. >>>> >>>> > Have you guys used an Android phone recently? Their startup time for >>>> > apps is generally atrocious compared to ours (even on high-end devices) >>>> > - we shouldn't drop the ball, but it's not where we compare badly. Given >>>> > we aren't targeting 256mb devices anymore, I'd gladly have all our apps >>>> > use double the memory they did in 2.2 if it meant we had a consistent >>>> > 60Hz update, consistent transitions and snappy response. >>>> >>>> That's not what I see on a Nexus 4 running CM 12 and on a z3c running L. >>>> They are both super fast and snappy when launching the default apps. >>>> Still better than us on the same hardware. >>>> >>>> Fabrice >>>> -- >>>> Fabrice Desré >>>> b2g team >>>> Mozilla Corporation >>>> _______________________________________________ >>>> dev-fxos mailing list >>>> [email protected] >>>> https://lists.mozilla.org/listinfo/dev-fxos >>> >>> _______________________________________________ >>> dev-fxos mailing list >>> [email protected] >>> https://lists.mozilla.org/listinfo/dev-fxos > > _______________________________________________ > dev-fxos mailing list > [email protected] > https://lists.mozilla.org/listinfo/dev-fxos
_______________________________________________ dev-fxos mailing list [email protected] https://lists.mozilla.org/listinfo/dev-fxos
