> On Oct 6, 2015, at 12:39 PM, Fabrice Desré <[email protected]> wrote: > > On 10/06/2015 01:57 AM, Julien Wajsberg wrote: >> Le 02/10/2015 19:16, Fabrice Desré a écrit : >>> On 10/02/2015 09:49 AM, Justin D'Arcangelo wrote: >>>> I would also like to add that this policy of immediately pouncing on devs >>>> who attempt to try something new that may cause the perf numbers to >>>> momentarily dip is part of why we seem to have a culture problem in FxOS >>>> dev where everyone is afraid to take any kind of risks. If we are not >>>> allowed to have a 2-3 week window to optimize after a huge landing such as >>>> this, then how are we supposed to experiment or take risks? >>> You have all the time you want if you don't put dogfooders at risk. No >>> one is saying that you should not take the risk to try something new >>> (side note, you spent enough time on spark & flyweb to know that). But >>> when it comes to shipping there is a minimum bar to meet, and with >>> basically a x2 memory usage we are not meeting it in this app yet, >>> sorry. Feel free to ship a new app alongside the existing one instead >>> and ask people to try it, since we can't do A/B testing. >> >> Sorry, I disagree here. I don't completely disagree though, so bear with >> me :) >> >> I think the best way to find bugs and regressions is exposing the >> changes to users. _of course_ we need to make sure we don't badly break >> the phone first. But users of the master branch will have regressions. >> That's normal and expected. Any big feature will get at least a handful >> of regressions. Our goal is to track them and fix them, before we ship >> to less technical/less engaged users. IMO that's why we wanted the >> dogfood process in the first place. > > I guess we only disagree on the magnitude of the regressions we are > happy to ship to dogfooders. Your bar seems higher than mine.
TBH, if you look at the Raptor dashboards, the Music app startup times were terrible both before and after landing due to the massive NUWA regression. If someone was dogfooding daily builds from master during those two days, they likely would not have even noticed any difference because it was so bad to begin with. But, as others have already mentioned, dogfooders have not seen a new build in a while, so this is kind of a moot point. Also, if you look at the numbers from this week, we are ~100ms away from v2.2 startup times and RSS memory is within 6MB of the old app now due to optimizations that were landed in the 5 days following landing. I understand that you want to set the bar high, I feel the same way about code quality. But, if you put things into perspective, and remove the noise from the massive 2-week NUWA regression, the Music app is actually in really good shape and is continuing to improve even more throughout this week. We also now have the added benefit of code quality which helps us move much faster than we would be able to with the old codebase. So, even if we had been shipping regular builds to dogfooders during this time, and assuming that the NUWA regression didn’t happen, they would *maybe* notice a ~100ms slower startup time from the old app to the new app. > >> If you want that dogfooders don't get the master regressions, then don't >> use the master branch. BTW I personally think we should let the >> dogfooders choose between "master-dogfood" and "aurora-dogfood" branches. > > There's already "dogfood" (with QA sign off) and "dogfood-latest" > (nightlies, use at your own risk). > >> Now, I guess you're afraid that we're losing dogfooders, even those on >> master that are aware they can get issues. Big news, they don't leave >> the program because an app takes 2x memory. Users don't even see it. You >> can look at the list of foxfood bugs [1], very few bugs have "slow" or >> "performance" in their summary. >> >> So please don't mix and confuse topics and concerns. The performance >> concern is important, but it's not what puts dogfooders at risk. > > Well... we have almost no dogfooders, because we have been unable to > ship updates fixing a bunch of bugs that were submitted at the beginning > of the program. So right now I don't think we can draw any conclusions > from the foxfood feedback unfortunately. And it's not because they won't > notice memory regressions that they are not important. I was merely > pointing out that we have an overall quality issue, and memory/startup > time regressions are part of that. > > Fabrice > -- > Fabrice Desré > b2g team > Mozilla Corporation > _______________________________________________ > dev-fxos mailing list > [email protected] <mailto:[email protected]> > https://lists.mozilla.org/listinfo/dev-fxos > <https://lists.mozilla.org/listinfo/dev-fxos>
_______________________________________________ dev-fxos mailing list [email protected] https://lists.mozilla.org/listinfo/dev-fxos
