1. Ravi Randu and Bobby Chien head up the performance benchmarks definition, will input from relevant stakeholders.
2. For v2.5 I think so, if anything because there isn't one well-defined for Aries yet. 3. Bobby Chien should probably take point here and prioritize and triage these. Bobby? Thanks, Eli On Tue, Sep 15, 2015 at 4:23 PM, Naoki Hirata <nhir...@mozilla.com> wrote: > Ah ok. > > 1) Who's going to define these benchmarks? > > 2) Do we need a new baseline? > > 3) Is anyone investigating the memory regressions that we already have? > a) We still have not resolved the series of initial bugs going into > 2.5/3.0: > https://bugzilla.mozilla.org/show_bug.cgi?id=1155854 > https://bugzilla.mozilla.org/show_bug.cgi?id=1162535 > > b) I just did a quick search and found some open bugs in memory leaks; I > don't know if they all still occur: > > 1135278 - shared/js/media/video_player.js leaks memory > 1135256 - navigator.mozL10n.ready can cause memory leaks > 1032830 - Clock leaks memory while screen is off > 1203999 Pin the Web is leaking setting observers--- > 1202592 Activity chooser leaks settings observer > 1137995 research on power consumption shows privacy info leaks > 1118106 SecureElement : Resource management / resources leak to be > handled in erroneous conditions such as gecko crash / restart etc. > 1111526 [B2G][AudioChannel] Paused media element sound leaks when another > media element plays with loop = on . > 1046895 Wakelock leaks after several failed registration attempts > 1039068 notifications.js leaks the most recently used notification > <div> > 1020685 Fix leak via contextmenu handler > > > On Tue, Sep 15, 2015 at 2:10 PM, Eli Perelman <eperel...@mozilla.com> > wrote: > >> Naoki, yes, I believe that should probably be updated. It's OK to have >> multiple benchmarks, but for something that we are going to base blockers >> on, they should be well-established per release. For example: >> >> v2.2 >> flame-kk 319MB >> >> v2.5 >> flame-kk 319MB >> flame-kk 1024MB >> aries 2048MB >> >> v3.0 >> aries 2048MB >> aries-l 2048 >> >> Not that these are the actual baseline benchmarks, just that they should >> be well-defined, and there should be a common denominator between 2 >> adjacent releases. >> >> Thanks, >> >> Eli Perelman >> >> On Tue, Sep 15, 2015 at 3:28 PM, Naoki Hirata <nhir...@mozilla.com> >> wrote: >> >>> It sounds like : >>> https://wiki.mozilla.org/Firefox_OS/Performance/Benchmark needs to be >>> updated to include memory allocations? or are we talking about the same >>> thing? >>> >>> On Tue, Sep 15, 2015 at 12:01 PM, Eli Perelman <eperel...@mozilla.com> >>> wrote: >>> >>>> Raptor's performance tests also use 319MB as it is the only real >>>> baseline we tracked from v2.2. In order to compare the performance between >>>> the two branches of v2.2 and v2.5, we need a comparable baseline. We have >>>> been aggregating data for the Flame 1GB for future comparisons, but I would >>>> suggest proposing a new baseline sooner rather than later so Raptor can >>>> begin to aggregate data against that device and make it trackable between >>>> branches. >>>> >>>> Thanks, >>>> >>>> Eli Perelman >>>> >>>> On Tue, Sep 15, 2015 at 12:48 PM, Martijn <martijn.mart...@gmail.com> >>>> wrote: >>>> >>>>> On Tue, Sep 15, 2015 at 6:32 PM, Naoki Hirata <nhir...@mozilla.com> >>>>> wrote: >>>>> > >>>>> > I believe we still do care. >>>>> > >>>>> > Running Engineering build will make some difference as the 319 MB >>>>> takes into account the camera but not Marionette (adb + devtools) which >>>>> the engineering build runs. >>>>> > >>>>> > Having said that we run automation tests through jenkins, I'll check >>>>> with mwargers in regards to the issue to see if he already filed a bug or >>>>> not. >>>>> >>>>> >>>>> Yes, we're running the Gaia UI tests with the 319MB configuration and >>>>> we have various tests that are failing (intermittently) and disabled >>>>> because they just don't work with 319MB: >>>>> 1172167 [Flame][Cards View] Cards view loses the list of recently >>>>> opened apps >>>>> 1176502 Failure in test_homescreen_column_layout.py on the Flame >>>>> device, because Settings app gets killed >>>>> 1178859 test_browser_bookmark.py: "IOError: Connection to Marionette >>>>> server is lost." >>>>> 1181344 [Window Management] Settings will be a blank white screen when >>>>> returning from the "Built-in Keyboard" settings >>>>> 1188080 test_rocketbar_offline_behavior.py: "NoSuchWindowException: >>>>> None" >>>>> 1189262 Failure in test_browser_share_link.py using Flame 319MB >>>>> 1204280 [Flame] Intermittent failure in test_sms_contact.py in >>>>> tap_send_sms using Flame 319MB >>>>> 1188603 [Settings][Ringtones] Share activity closes when attempting to >>>>> add custom ringtone >>>>> >>>>> I think these bugs need to be fixed, because they stand for lost >>>>> functionality in those low memory conditions. >>>>> >>>>> All the bugs currently out there, regarding supporting 319MB memory >>>>> configuration: >>>>> >>>>> https://bugzilla.mozilla.org/buglist.cgi?list_id=12549261&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=[319MB-Flame-Support] >>>>> >>>>> Regards, >>>>> Martijn >>>>> >>>>> >>>>> > >>>>> > >>>>> > We still have not resolved the series of initial bugs going into >>>>> 2.5/3.0: >>>>> > https://bugzilla.mozilla.org/show_bug.cgi?id=1155854 >>>>> > https://bugzilla.mozilla.org/show_bug.cgi?id=1162535 >>>>> > >>>>> > On Tue, Sep 15, 2015 at 2:13 AM, Ting-Yu Chou <tc...@mozilla.com> >>>>> wrote: >>>>> >> >>>>> >> A bit off-topic. >>>>> >> >>>>> >> Raptor [1] just records a serious memory regression though, not >>>>> sure is it related to the issue you see. >>>>> >> >>>>> >> Ting >>>>> >> >>>>> >> [1] >>>>> http://raptor.mozilla.org/#/dashboard/script/apps-memory.js?device=flame-kk&branch=master&memory=319 >>>>> >> >>>>> >> On Tue, Sep 15, 2015 at 4:56 PM, Gabriele Svelto < >>>>> gsve...@mozilla.com> wrote: >>>>> >>> >>>>> >>> I was doing some testing in the past few days on a Flame using the >>>>> >>> 319MiB memory switch and found that our core apps are now almost >>>>> >>> unusable on it. Opening a single app seems to end up killing the >>>>> >>> homescreen and keeping two open at the same time is almost >>>>> impossible. >>>>> >>> >>>>> >>> So I've got two questions: the first one is, did we significantly >>>>> >>> regress memory consumption again? Or is it just gecko having grown >>>>> >>> significantly in the past few weeks making our minimum process size >>>>> >>> larger? The second question is obviously: do we still care about >>>>> 256MiB >>>>> >>> devices? Because if we still do then it looks like we're not in a >>>>> good >>>>> >>> spot for a 2.5 release running on those. If we don't care I >>>>> suppose it's >>>>> >>> fine though we should still keep our memory consumption under >>>>> control. >>>>> >>> If we move our minimum to 512MiB we'll have plenty of room and this >>>>> >>> might cause us to regress even further almost without noticing (*). >>>>> >>> >>>>> >>> I'm running an engineering build BTW, but I don't think this >>>>> should make >>>>> >>> much of a difference. >>>>> >>> >>>>> >>> Gabriele >>>>> >>> >>>>> >>> *) Unless the screen has a rather high resolution, in which case >>>>> 512MiB >>>>> >>> might still yield only a small amount of usable memory for apps as >>>>> most >>>>> >>> of it will go to the framebuffer and accompanying structures >>>>> (layers & co). >>>>> >>> >>>>> >>> >>>>> >>> _______________________________________________ >>>>> >>> dev-fxos mailing list >>>>> >>> dev-fxos@lists.mozilla.org >>>>> >>> https://lists.mozilla.org/listinfo/dev-fxos >>>>> >>> >>>>> >> >>>>> >> >>>>> >> _______________________________________________ >>>>> >> dev-fxos mailing list >>>>> >> dev-fxos@lists.mozilla.org >>>>> >> https://lists.mozilla.org/listinfo/dev-fxos >>>>> >> >>>>> > >>>>> > >>>>> > _______________________________________________ >>>>> > dev-fxos mailing list >>>>> > dev-fxos@lists.mozilla.org >>>>> > https://lists.mozilla.org/listinfo/dev-fxos >>>>> > >>>>> _______________________________________________ >>>>> dev-fxos mailing list >>>>> dev-fxos@lists.mozilla.org >>>>> https://lists.mozilla.org/listinfo/dev-fxos >>>>> >>>> >>>> >>> >> >
_______________________________________________ dev-fxos mailing list dev-fxos@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-fxos
