Re: Some data on mozilla-inbound

[Justin Lebar]

> The ratio of things landed on inbound which turn out to busted is really
> worrying

> * 116 of the 916 changesets (12.66%) were backed out

If 13% is "really worrying", what do you think our goal should be?

On Tue, Apr 23, 2013 at 12:39 AM, Ehsan Akhgari  wrote:
> This was a fantastic read, it almost made me shed happy tears!  Thanks a lot
> kats for doing this.
>
> The ratio of things landed on inbound which turn out to busted is really
> worrying, and it might be an indicator that (some?) developers have a poor
> judgement on how safe their patches are.  How hard would it be to gather a
> list of the total number of patches being backed out plus the amount of time
> that we spent building/testing those, hopefully in a style similar to
> ?  If we had
> such a list, perhaps we could reach out to the high offenders there and let
> them know about the problem, and see if that changes these stats a couple of
> weeks from now?
>
> Thanks!
> Ehsan
>
>
> On 2013-04-22 3:54 PM, Kartikaya Gupta wrote:
>>
>> TL;DR:
>> * Inbound is closed 25% of the time
>> * Turning off coalescing could increase resource usage by up to 60% (but
>> probably less than this).
>> * We spend 24% of our machine resources on changes that are later backed
>> out, or changes that are doing the backout
>> * The vast majority of changesets that are backed out from inbound are
>> detectable on a try push
>>
>> Because of the large effect from coalescing, any changes to the current
>> process must not require running the full set of tests on every push.
>> (In my proposal this is easily accomplished with trychooser syntax, but
>> other proposals include rotating through T-runs on pushes, etc.).
>>
>> --- Long verion below ---
>>
>> Following up from the infra load meeting we had last week, I spent some
>> time this weekend crunching various pieces of data on mozilla-inbound to
>> get a sense of how much coalescing actually helps us, how much backouts
>> hurt us, and generally to get some data on the impact of my previous
>> proposal for using a multi-headed tree. I didn't get all the data that I
>> wanted but as I probably won't get back to this for a bit, I thought I'd
>> share what I found so far and see if anybody has other specific pieces
>> of data they would like to see gathered.
>>
>> -- Inbound uptime --
>>
>> I looked at a ~9 day period from April 7th to April 16th. During this
>> time:
>> * inbound was closed for 24.9587% of the total time
>> * inbound was closed for 15.3068% of the total time due to "bustage".
>> * inbound was closed for 11.2059% of the total time due to "infra".
>>
>> Notes:
>> 1) "bustage" and "infra" were determined by grep -i on the data from
>> treestatus.mozilla.org.
>> 2) There is some overlap so bustage + infra != total.
>> 3) I also weighted the downtime using checkins-per-hour histogram from
>> joduinn's blog at [1], but this didn't have a significant impact: the
>> total, bustage, and infra downtime percentages moved to 25.5392%,
>> 15.7285%, and 11.3748% respectively.
>>
>> -- Backout changes --
>>
>> Next I did an analysis of the changes that landed on inbound during that
>> time period. The exact pushlog that I looked at (corresponding to the
>> same April 7 - April 16 time period) is at [2]. I removed all of the
>> merge changesets from this range, since I wanted to look at inbound in
>> as much isolation as possible.
>>
>> In this range:
>> * there were a total of 916 changesets
>> * there were a total of 553 "pushes"
>> * 74 of the 916 changesets (8.07%) were backout changesets
>> * 116 of the 916 changesets (12.66%) were backed out
>> * removing all backouts and changes backed out removed 114 pushes (20.6%)
>>
>> Of the 116 changesets that were backed out:
>> * 37 belonged to single-changeset pushes
>> * 65 belonged to multi-changeset pushes where the entire pushed was
>> backed out
>> * 14 belonged to multi-changeset pushes where the changesets were
>> selectively backed out
>>
>> Of the 74 backout changesets:
>> * 4 were for commit message problems
>> * 25 were for build failures
>> * 36 were for test failures
>> * 5 were for leaks/talos regressions
>> * 1 was for premature landing
>> * 3 were for unknown reasons
>>
>> Notes:
>> 1) There were actually 79 backouts, but I ignored 5 of them because they
>> backed out changes that happened prior to the start of my range).
>> 2) Additional changes at the end of my range may have been backed out,
>> but the backouts were not in my range so I didn't include them in my
>> analysis.
>> 3) The 14 csets that were selectively backed out is interesting to me
>> because it implies that somebody did some work to identify which changes
>> in the push were bad, and this naturally means that there is room to
>> save on doing that work.
>>
>> -- Merge conflicts --
>>
>> I also wanted to determine how many of these changes conflicted with
>> each other, and how far away the conflicting changes were. I 

Re: Some data on mozilla-inbound

[Ehsan Akhgari]

This was a fantastic read, it almost made me shed happy tears!  Thanks a 
lot kats for doing this.


The ratio of things landed on inbound which turn out to busted is really 
worrying, and it might be an indicator that (some?) developers have a 
poor judgement on how safe their patches are.  How hard would it be to 
gather a list of the total number of patches being backed out plus the 
amount of time that we spent building/testing those, hopefully in a 
style similar to 
?  If we 
had such a list, perhaps we could reach out to the high offenders there 
and let them know about the problem, and see if that changes these stats 
a couple of weeks from now?


Thanks!
Ehsan

On 2013-04-22 3:54 PM, Kartikaya Gupta wrote:

TL;DR:
* Inbound is closed 25% of the time
* Turning off coalescing could increase resource usage by up to 60% (but
probably less than this).
* We spend 24% of our machine resources on changes that are later backed
out, or changes that are doing the backout
* The vast majority of changesets that are backed out from inbound are
detectable on a try push

Because of the large effect from coalescing, any changes to the current
process must not require running the full set of tests on every push.
(In my proposal this is easily accomplished with trychooser syntax, but
other proposals include rotating through T-runs on pushes, etc.).

--- Long verion below ---

Following up from the infra load meeting we had last week, I spent some
time this weekend crunching various pieces of data on mozilla-inbound to
get a sense of how much coalescing actually helps us, how much backouts
hurt us, and generally to get some data on the impact of my previous
proposal for using a multi-headed tree. I didn't get all the data that I
wanted but as I probably won't get back to this for a bit, I thought I'd
share what I found so far and see if anybody has other specific pieces
of data they would like to see gathered.

-- Inbound uptime --

I looked at a ~9 day period from April 7th to April 16th. During this time:
* inbound was closed for 24.9587% of the total time
* inbound was closed for 15.3068% of the total time due to "bustage".
* inbound was closed for 11.2059% of the total time due to "infra".

Notes:
1) "bustage" and "infra" were determined by grep -i on the data from
treestatus.mozilla.org.
2) There is some overlap so bustage + infra != total.
3) I also weighted the downtime using checkins-per-hour histogram from
joduinn's blog at [1], but this didn't have a significant impact: the
total, bustage, and infra downtime percentages moved to 25.5392%,
15.7285%, and 11.3748% respectively.

-- Backout changes --

Next I did an analysis of the changes that landed on inbound during that
time period. The exact pushlog that I looked at (corresponding to the
same April 7 - April 16 time period) is at [2]. I removed all of the
merge changesets from this range, since I wanted to look at inbound in
as much isolation as possible.

In this range:
* there were a total of 916 changesets
* there were a total of 553 "pushes"
* 74 of the 916 changesets (8.07%) were backout changesets
* 116 of the 916 changesets (12.66%) were backed out
* removing all backouts and changes backed out removed 114 pushes (20.6%)

Of the 116 changesets that were backed out:
* 37 belonged to single-changeset pushes
* 65 belonged to multi-changeset pushes where the entire pushed was
backed out
* 14 belonged to multi-changeset pushes where the changesets were
selectively backed out

Of the 74 backout changesets:
* 4 were for commit message problems
* 25 were for build failures
* 36 were for test failures
* 5 were for leaks/talos regressions
* 1 was for premature landing
* 3 were for unknown reasons

Notes:
1) There were actually 79 backouts, but I ignored 5 of them because they
backed out changes that happened prior to the start of my range).
2) Additional changes at the end of my range may have been backed out,
but the backouts were not in my range so I didn't include them in my
analysis.
3) The 14 csets that were selectively backed out is interesting to me
because it implies that somebody did some work to identify which changes
in the push were bad, and this naturally means that there is room to
save on doing that work.

-- Merge conflicts --

I also wanted to determine how many of these changes conflicted with
each other, and how far away the conflicting changes were. I got a
partial result here but I need to do more analysis before I have numbers
worth posting.

-- Build farm resources --

Finally, I used a combination of gps' mozilla-build-analyzer tool [3]
and some custom tools to determine how much machine time was spent on
building all of these pushes and changes.

I looked at all the build.json files [4] from the 6th of April to the
17th of April and pulled out all the jobs that corresponding to the
"push" changesets in my range above. For this set of 553 changesets,
there were 500 (exactly!

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Nicholas Nethercote]

On Mon, Apr 22, 2013 at 6:35 PM, Justin Dolske  wrote:
>
> That said, I think it's critically important that we're working to make JS a
> acceptable -- nay, _excellent_ -- language/runtime for application
> development for the long run. We can't tell a credible story about why
> people should write HTML5 apps, if we're tearing out swaths of JS in our own
> products. Sometimes dogfooding is unpleasant or hard, but that's the whole
> point.

There's a big difference between apps on Firefox OS, which are likely
to have relatively short lifetimes and can be killed if they take up
too much memory, and the main process.  Bad memory behaviour in the
main process is a much bigger deal, and it's something that's
happening right now with some frequency.

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Justin Dolske]

On 4/21/13 4:51 PM, Justin Lebar wrote:


What I'd like to come out of this thread is a consensus one way or another as
to whether we continue along our current path of writing many features that are
enabled on B2G in JS, or whether we change course.


First -- B2G should clearly do whatever it needs to in order to get 
acceptable memory/perf/whatever in the short term. If B2G flops, it 
doesn't matter a damn what it's written in.


That said, I think it's critically important that we're working to make 
JS a acceptable -- nay, _excellent_ -- language/runtime for application 
development for the long run. We can't tell a credible story about why 
people should write HTML5 apps, if we're tearing out swaths of JS in our 
own products. Sometimes dogfooding is unpleasant or hard, but that's the 
whole point.


There will always be a place for C++ (well, until Rust, amirite? ;). 
Either as being the right tool for a specific job, or just because 
someone happens to know it better. But if we end up avoiding JS because 
it's inherently JS, then that's a big strategic problem.


Justin
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: JavaScript reference changes: looking for opinions

[Norbert Lindenberg]

On Apr 22, 2013, at 17:39 , Jeff Walden wrote:

> On 04/22/2013 04:34 PM, Norbert Lindenberg wrote:
>> 3) Related to that, some properties are documented as part of the wrong 
>> object. For example, the String.prototype documentation shows a length 
>> property. String.prototype doesn't have this property; String instances do.
> 
> String.prototype is a String object.  String objects have a length property.  
> So that's correct, as far as it goes.  (It's kind of useless to explicitly 
> call out "length" as a property of String.prototype, I think, but it's 
> correct to say so.)

You're right. I should have said that it doesn't matter that String.prototype 
has a length because every String instance/value has its own length, and that's 
the one you actually work with. Documenting it in the same way as the 
constructor property doesn't reflect the actual design.

Norbert
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Accelerating exact rooting work

[Robert O'Callahan]

On Tue, Apr 23, 2013 at 5:36 AM, Terrence Cole  wrote:

> Our exact rooting work is at a spot right now where we could easily use
> more hands to accelerate the process. The main problem is that the work
> is easy and tedious: a hard sell for pretty much any hacker at mozilla.
>

It sounds worthwhile to encourage developers who aren't currently working
on critical-path projects to pile onto the exact rooting project. Getting
GGC over the line reaps some pretty large benefits and it's an
all-or-nothing project, unlike say pursuing the long tail of WebIDL
conversions.

If that sounds right, put out a call for volunteers (by which I include
paid staff) to help push on exact rooting, with detailed instructions. I
know some people who could probably help.

Rob
-- 
q“qIqfq qyqoquq qlqoqvqeq qtqhqoqsqeq qwqhqoq qlqoqvqeq qyqoquq,q qwqhqaqtq
qcqrqeqdqiqtq qiqsq qtqhqaqtq qtqoq qyqoquq?q qEqvqeqnq qsqiqnqnqeqrqsq
qlqoqvqeq qtqhqoqsqeq qwqhqoq qlqoqvqeq qtqhqeqmq.q qAqnqdq qiqfq qyqoquq
qdqoq qgqoqoqdq qtqoq qtqhqoqsqeq qwqhqoq qaqrqeq qgqoqoqdq qtqoq qyqoquq,q
qwqhqaqtq qcqrqeqdqiqtq qiqsq qtqhqaqtq qtqoq qyqoquq?q qEqvqeqnq
qsqiqnqnqeqrqsq qdqoq qtqhqaqtq.q"
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: JavaScript reference changes: looking for opinions

[Jeff Walden]

On 04/22/2013 04:34 PM, Norbert Lindenberg wrote:
> 3) Related to that, some properties are documented as part of the wrong 
> object. For example, the String.prototype documentation shows a length 
> property. String.prototype doesn't have this property; String instances do.

String.prototype is a String object.  String objects have a length property.  
So that's correct, as far as it goes.  (It's kind of useless to explicitly call 
out "length" as a property of String.prototype, I think, but it's correct to 
say so.)

Jeff
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: JavaScript reference changes: looking for opinions

[Norbert Lindenberg]

It seems there are several distinct problems:

1) The page
https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/String
right now doesn't show the methods that String instances inherit from 
String.prototype. This is most likely a symptom of this bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=859169
The bug mentions how the problem can be temporarily fixed for any page, but it 
will recur each time the page is edited.

2) All the constructor pages have links "Properties of XXX instances" linking 
to the prototype page. This is wrong - properties of a prototype are not 
properties of instances.

3) Related to that, some properties are documented as part of the wrong object. 
For example, the String.prototype documentation shows a length property. 
String.prototype doesn't have this property; String instances do.

4) Maintaining a separate prototype page is a bit tedious to authors. See Tom's 
email.

I think any changes need to maintain (or actually create) a clear distinction 
between the properties of constructor, prototype, and instances. Which object a 
property belongs to is fundamental to software design in JavaScript, and 
without correct information developers will have a hard time getting their 
software to work. It's not necessary to express that distinction through 
separate pages though.

Norbert


On Apr 22, 2013, at 12:04 , Eric Shepherd wrote:

> Currently, the JavaScript reference content for the global classes (String, 
> Array, etc), are divided up such that the class methods and properties and 
> the prototype methods and properties are documented separately, with links 
> between them. For example, see:
> 
> https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/String
>  
> 
> and
> 
> https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/String/prototype
>  
> 
> While this might make sense to super-expert JavaScript folks (indeed, it was 
> their idea to do it this way), it's actually really confusing to everyone 
> else.
> 
> I'd like to propose we merge them back together, so that the stuff currently 
> documented on the "prototype" page is in the main body of the class's 
> documentation where most people would expect it to be. If useful, we can come 
> up with a badge to put next to items that are part of the prototype (or not) 
> to differentiate between them.
> 
> But the current organization is, well, kind of weird.
> 
> Any opinions on this pro or con before we actually try to find someone to do 
> the work?
> 
> -- 
> Eric Shepherd
> Developer Documentation Lead
> Mozilla
> Blog: http://www.bitstampede.com/
> Twitter: @sheppy
> 
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Gijs Kruitbosch]

On 23/04/13 24:18 , Nicolas B. Pierron wrote:

On 04/22/2013 07:53 AM, Justin Lebar wrote:

This is the wifi worker.  I think "script-sources" is code.  Note that
fragmentation (unused-arenas) is way too high, but even despite this
the worker uses too much memory.


2.38 MB (05.13%) -- worker(resource://gre/modules/wifi_worker.js,
0x45584800)
├──1.77 MB (03.81%) -- gc-heap
│  ├──1.74 MB (03.74%) ── unused-arenas


We have a parameter which set a low limit which is used to prevent GC
during start-up, I don't think it is used for workers, but this might be
something to check that this preference[1] is only used for the main
thread. Currently this preference is set to 3 MB before the first GC.

[1] javascript.options.mem.gc_allocation_threshold_mb



Knowing nothing about the code, this sure looks suspicious:

http://mxr.mozilla.org/mozilla-central/source/b2g/app/b2g.js#550

I cannot find code where this pref is changed using MXR, but I am 
jetlagged and don't know my way around this code, so I could totally 
have missed it...



~ Gijs
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Nicolas B. Pierron]

On 04/22/2013 07:53 AM, Justin Lebar wrote:

This is the wifi worker.  I think "script-sources" is code.  Note that
fragmentation (unused-arenas) is way too high, but even despite this
the worker uses too much memory.


2.38 MB (05.13%) -- worker(resource://gre/modules/wifi_worker.js, 0x45584800)
├──1.77 MB (03.81%) -- gc-heap
│  ├──1.74 MB (03.74%) ── unused-arenas


We have a parameter which set a low limit which is used to prevent GC during 
start-up, I don't think it is used for workers, but this might be something 
to check that this preference[1] is only used for the main thread. 
Currently this preference is set to 3 MB before the first GC.


[1] javascript.options.mem.gc_allocation_threshold_mb

--
Nicolas B. Pierron
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Boris Zbarsky]

On 4/22/13 5:22 PM, Jeff Muizelaar wrote:

I don't know if there's anything surprising here. Calling into JS from C++ goes 
through xpconnect which is a long standing slowness.


_That_ we can try to fix by converting to JS-implemented WebIDL.  Right 
now that performs about the same, but we know how to make it much faster...


-Boris

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Jeff Muizelaar]

On 2013-04-22, at 3:44 PM, Terrence Cole wrote:

> On 04/22/2013 12:12 PM, Jeff Muizelaar wrote:
>> On 2013-04-22, at 2:15 PM, Bill McCloskey wrote:
>> 
>>> I can't agree with you more, Justin. I think Boris is right that we should 
>>> make these decisions on a case-by-case basis. But in the case of these 
>>> workers, it seems clear that converting them to C++ is the way to go, 
>>> assuming we have the resources to do so.
>> So a specific case that I ran into during the Performance Workshop is 
>> RILContentHelper.js. During the startup of the settings app
>> we jank for 53ms while initializing the RILContentHelper.js: 
>> 
>> http://people.mozilla.com/~bgirard/cleopatra/#report=bf7077c6552fe2bc015d7074a338b673911f3ce8&search=Mobile
> 
> That link gives me this: "Error fetching profile :(. URL:
> 'http://profile-store.commondatastorage.googleapis.com/bf7077c6552fe2bc015d7074a338b673911f3ce8'.
> Did you set the CORS headers?"

That's weird. The link works for others and the CORS headers should be set.

> 
>> 
>> There doesn't seem to be anything specific taking that much time in the 
>> profile, just general JS overhead. In this case RILContentHelper.js is 
>> wrapped by by C++ code in dom/network/src/MobileConnection.cpp and so we end 
>> up spending a fair amount of time transitioning from JS to C++ to JS to C++.
> 
> That seems like the sort of thing that SpiderMonkey may be able to
> address in the short term, depending on what exactly it turns out to be.
> Is there a bug on file somewhere to coordinate the investigation?

I don't know if there's anything surprising here. Calling into JS from C++ goes 
through xpconnect which is a long standing slowness.

-Jeff
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Till Schneidereit]

On Mon, Apr 22, 2013 at 9:48 PM, Justin Lebar wrote:

> > This is all great stuff, but as mentioned elsewhere, B2G branched at
> > version 18 and so they need improvements that that can land quickly on
> > the relevant branches.
>

I understand that (but should certainly have made it more clear).

Maybe I'm underestimating the amount of work required to rebuild enough
(whatever "enough" means, in this case) modules in C++, but isn't that a
somewhat major undertaking, too?


>
> Well, to be clear, it would be great if we could land some
> improvements for v1.1 (which is based off version 18), but we're
> locking the tree down pretty hard already, so I suspect that e.g. a
> wifi worker rewrite is off the table for that version.  Hopefully v1.1
> is the last b2g version that will be based off b2g18.
>
> v1.1 is also, I found out last week, targeting a 512mb device, so
> memory usage probably isn't as critical there as it will be in future
> releases which target 256mb devices.
>

We might (should, actually) be able to land the lazy-cloning and
re-lazification parts without the lazy bytecode. I'm not saying it's a tiny
project, but at least we have almost all of the required infrastructure in
place, already.

Mmh, but backporting it to 18 might be a lot harder.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Nicholas Nethercote]

On Mon, Apr 22, 2013 at 1:32 PM, Nicholas Nethercote
 wrote:
>
> - Looking at the merged.json data:  the system principal compartment
> merging is happening on the main process, but doesn't appear to be
> happening on all the other processes:  Homescreen, Usage,
> (Preallocated app).

I filed https://bugzilla.mozilla.org/show_bug.cgi?id=864494 for this issue.

N
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Justin Lebar]

> This is all great stuff, but as mentioned elsewhere, B2G branched at
> version 18 and so they need improvements that that can land quickly on
> the relevant branches.

Well, to be clear, it would be great if we could land some
improvements for v1.1 (which is based off version 18), but we're
locking the tree down pretty hard already, so I suspect that e.g. a
wifi worker rewrite is off the table for that version.  Hopefully v1.1
is the last b2g version that will be based off b2g18.

v1.1 is also, I found out last week, targeting a 512mb device, so
memory usage probably isn't as critical there as it will be in future
releases which target 256mb devices.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Nicholas Nethercote]

On Mon, Apr 22, 2013 at 1:25 PM, Till Schneidereit
 wrote:
> There are a few things we're working on in SpiderMonkey that should improve
> this situation quite a bit:
>
> ...generational GC...
>
> making bytecode generation lazy...
> ...re-lazyfication of JSScripts...
> ...lazy cloning of JSScripts...

This is all great stuff, but as mentioned elsewhere, B2G branched at
version 18 and so they need improvements that that can land quickly on
the relevant branches.

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Justin Lebar]

> There are a few things we're working on in SpiderMonkey that should improve
> this situation quite a bit:

Thanks, but I again need to emphasize that these are large, long-term
plans.  Terrence tells me that GGC is planned for "sometime this
year".  Lazy bytecode generation has been on the roadmap for a long
time as well.

I understand that these are large projects, and I don't mean to
suggest that you guys aren't doing a good job with them, but I do not
think that the right solution for b2g system compartments / workers is
"JS should be able to meet your requirements one day," and "wait until
things get better."  In the meantime we continue to dig ourselves in a
hole by implementing new DOM APIs in JS.

We of course still want these engine improvements for reducing the
memory usage of web content.

Again, in the past, we've had the most success dealing with the JS
engine as it is today.  Then the engine optimizations, when they come,
are gravy.  That's all I'm suggesting we do here.

On Mon, Apr 22, 2013 at 4:25 PM, Till Schneidereit
 wrote:
> There are a few things we're working on in SpiderMonkey that should improve
> this situation quite a bit:
>
> Terrence already mentioned generational GC, which certainly is the largest
> piece by far. Getting rid of all or almost all memory lost to fragmentation
> makes the tradeoff a considerably different one, I'd say.
>
> Additionally, the work on making bytecode generation lazy[1] should vastly
> reduce the memory used for scripts. Based on that, I'm investigating
> several more options to reduce script memory:
> - re-lazyfication of JSScripts, reducing memory usage by removing the
> parsed represantation of scripts that are only run once or very rarely.
> - lazy cloning of JSScripts with the same source from other compartments
> containing the same script. Probably to go hand-in-hand with
> re-lazyfication.
>
> Combined, I'd very vaguely say that these measures should reduce the memory
> usage by an additional 20 or 30 percentage points. (Based on the current
> usage as 100%, and no guarantees, of course).
>
>
>
>
> On Mon, Apr 22, 2013 at 8:44 PM, Terrence Cole  wrote:
>
>> On 04/22/2013 12:12 PM, Jeff Muizelaar wrote:
>> > On 2013-04-22, at 2:15 PM, Bill McCloskey wrote:
>> >
>> >> I can't agree with you more, Justin. I think Boris is right that we
>> should make these decisions on a case-by-case basis. But in the case of
>> these workers, it seems clear that converting them to C++ is the way to go,
>> assuming we have the resources to do so.
>> > So a specific case that I ran into during the Performance Workshop is
>> RILContentHelper.js. During the startup of the settings app
>> > we jank for 53ms while initializing the RILContentHelper.js:
>> >
>> >
>> http://people.mozilla.com/~bgirard/cleopatra/#report=bf7077c6552fe2bc015d7074a338b673911f3ce8&search=Mobile
>>
>> That link gives me this: "Error fetching profile :(. URL:
>> '
>> http://profile-store.commondatastorage.googleapis.com/bf7077c6552fe2bc015d7074a338b673911f3ce8
>> '.
>> Did you set the CORS headers?"
>>
>> >
>> > There doesn't seem to be anything specific taking that much time in the
>> profile, just general JS overhead. In this case RILContentHelper.js is
>> wrapped by by C++ code in dom/network/src/MobileConnection.cpp and so we
>> end up spending a fair amount of time transitioning from JS to C++ to JS to
>> C++.
>>
>> That seems like the sort of thing that SpiderMonkey may be able to
>> address in the short term, depending on what exactly it turns out to be.
>> Is there a bug on file somewhere to coordinate the investigation?
>>
>> -Terrence
>>
>> > -Jeff
>> > ___
>> > dev-platform mailing list
>> > dev-platform@lists.mozilla.org
>> > https://lists.mozilla.org/listinfo/dev-platform
>>
>> ___
>> dev-platform mailing list
>> dev-platform@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-platform
>>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Nicholas Nethercote]

Some comments on this whole thread:

- I'm very sympathetic to Justin's concerns.  120 MiB is not much
memory.  While it's (somewhat) ok to kill apps that are using too much
memory, that doesn't work with the main B2G process, and I've been
CC'd on enough "the B2G main process is using too much memory" bugs to
understand this is an ongoing problem.

- Given the intense time pressure the B2G folks are under, saying
things like "there's no fundamental reason why JS can't be
memory-efficient" isn't much help when it currently isn't
memory-efficient.

- Justin knows more about B2G memory consumption than anyone;  he's
been looking at it closely for months.

- Look at the data -- code size is not the issue for JS code.

- Generational GC's timeline isn't even remotely feasible for B2G.
B2G branched on version 18!  Gen GC might be done in a few months.
(Which is fair enough;  Gen GC is a gigantic project.)

- I believe Justin identified off-list that the workers (and probably
the main JS runtime) are screaming out for more aggressive
decommitting of the JS heap:

>> 2.38 MB (05.13%) -- worker(resource://gre/modules/wifi_worker.js, 0x45584800)
>> ├──1.77 MB (03.81%) -- gc-heap
>> │  ├──1.74 MB (03.74%) ── unused-arenas

"unused-arenas" are empty 4 KiB JS heap arenas that could be
decommitted.  The merged.json data shows almost 5 MiB worth of
unused-arenas in 3 of the 4 workers.
https://bugzilla.mozilla.org/show_bug.cgi?id=829482 is currently open
on this issue.  That sounds easier to fix than rewriting modules in
C++.

- Looking at the merged.json data:  the system principal compartment
merging is happening on the main process, but doesn't appear to be
happening on all the other processes:  Homescreen, Usage,
(Preallocated app).  I've included the relevant data below.

Nick


Homescreen (pid 390)
Explicit Allocations

13,642,508 B (100.0%) -- explicit
├───4,403,684 B (32.28%) -- js-non-window
│   ├──3,865,124 B (28.33%) -- compartments
│   │  ├──3,358,912 B (24.62%) -- non-window-global
│   │  │  ├277,688 B (02.04%) ++ compartment([System Principal])
│   │  │  ├262,212 B (01.92%) ++ compartment([System Principal],
resource://gre/modules/DOMRequestHelper.jsm)
│   │  │  ├199,328 B (01.46%) ++ compartment([System Principal],
jar:file:///system/b2g/omni.ja!/components/Webapps.js)
│   │  │  ├187,400 B (01.37%) ++ compartment([System Principal],
resource://gre/modules/XPCOMUtils.jsm)
│   │  │  ├182,632 B (01.34%) ++ compartment([System Principal],
resource://gre/modules/CSPUtils.jsm)
│   │  │  ├134,456 B (00.99%) ++ compartment([System Principal],
resource://gre/modules/ObjectWrapper.jsm)
│   │  │  ├124,216 B (00.91%) ++ compartment([System Principal],
jar:file:///system/b2g/omni.ja!/components/SettingsManager.js)
│   │  │  ├114,360 B (00.84%) ++ compartment([System Principal],
resource://gre/modules/UserAgentOverrides.jsm)
│   │  │  ├110,008 B (00.81%) ++ compartment([System Principal],
jar:file:///system/b2g/omni.ja!/components/contentSecurityPolicy.js)
│   │  │  ├107,012 B (00.78%) ++ compartment([System Principal],
resource://gre/modules/AppsServiceChild.jsm)
│   │  │  ├105,264 B (00.77%) ++ compartment([System Principal],
jar:file:///system/b2g/omni.ja!/components/PushService.js)
│   │  │  ├104,688 B (00.77%) ++ compartment([System Principal],
jar:file:///system/b2g/omni.ja!/components/DirectoryProvider.js)
│   │  │  ├─98,664 B (00.72%) ++ compartment([System Principal],
jar:file:///system/b2g/omni.ja!/components/nsHandlerService.js)
│   │  │  ├─92,656 B (00.68%) ++ compartment([System Principal],
jar:file:///system/b2g/omni.ja!/components/nsPrompter.js)
│   │  │  ├─89,624 B (00.66%) ++ compartment([System Principal],
resource://gre/modules/BrowserElementPromptService.jsm)
│   │  │  ├─83,176 B (00.61%) ++ compartment([System Principal],
resource://gre/modules/AppsUtils.jsm)
│   │  │  ├─82,712 B (00.61%) ++ compartment([System Principal],
resource://gre/modules/services-common/preferences.js)
│   │  │  ├─81,536 B (00.60%) ++ compartment([System Principal],
resource://gre/modules/BrowserElementParent.jsm)
│   │  │  ├─77,520 B (00.57%) ++ compartment([System Principal],
jar:file:///system/b2g/omni.ja!/components/AppProtocolHandler.js)
│   │  │  ├─70,168 B (00.51%) ++ compartment([System Principal],
resource://gre/modules/Geometry.jsm)
│   │  │  ├─69,648 B (00.51%) ++ compartment([System Principal],
resource://gre/modules/SettingsDB.jsm)
│   │  │  ├─66,392 B (00.49%) ++ compartment([System Principal],
resource://gre/modules/NetUtil.jsm)
│   │  │  ├─65,768 B (00.48%) ++ compartment([System Principal],
resource://gre/modules/Services.jsm)
│   │  │  ├─65,112 B (00.48%) ++ compartment([System Principal],
resource://gre/modules/commonjs/promise/core.js)
│   │  │  ├─64,880 B (00.48%) ++ compartment([System Principal],
jar:file:///system/b2g/omni.ja!/components/BrowserElementParent.js)
│   │  │  ├─64,864 B (00.48%) ++ comp

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Till Schneidereit]

There are a few things we're working on in SpiderMonkey that should improve
this situation quite a bit:

Terrence already mentioned generational GC, which certainly is the largest
piece by far. Getting rid of all or almost all memory lost to fragmentation
makes the tradeoff a considerably different one, I'd say.

Additionally, the work on making bytecode generation lazy[1] should vastly
reduce the memory used for scripts. Based on that, I'm investigating
several more options to reduce script memory:
- re-lazyfication of JSScripts, reducing memory usage by removing the
parsed represantation of scripts that are only run once or very rarely.
- lazy cloning of JSScripts with the same source from other compartments
containing the same script. Probably to go hand-in-hand with
re-lazyfication.

Combined, I'd very vaguely say that these measures should reduce the memory
usage by an additional 20 or 30 percentage points. (Based on the current
usage as 100%, and no guarantees, of course).




On Mon, Apr 22, 2013 at 8:44 PM, Terrence Cole  wrote:

> On 04/22/2013 12:12 PM, Jeff Muizelaar wrote:
> > On 2013-04-22, at 2:15 PM, Bill McCloskey wrote:
> >
> >> I can't agree with you more, Justin. I think Boris is right that we
> should make these decisions on a case-by-case basis. But in the case of
> these workers, it seems clear that converting them to C++ is the way to go,
> assuming we have the resources to do so.
> > So a specific case that I ran into during the Performance Workshop is
> RILContentHelper.js. During the startup of the settings app
> > we jank for 53ms while initializing the RILContentHelper.js:
> >
> >
> http://people.mozilla.com/~bgirard/cleopatra/#report=bf7077c6552fe2bc015d7074a338b673911f3ce8&search=Mobile
>
> That link gives me this: "Error fetching profile :(. URL:
> '
> http://profile-store.commondatastorage.googleapis.com/bf7077c6552fe2bc015d7074a338b673911f3ce8
> '.
> Did you set the CORS headers?"
>
> >
> > There doesn't seem to be anything specific taking that much time in the
> profile, just general JS overhead. In this case RILContentHelper.js is
> wrapped by by C++ code in dom/network/src/MobileConnection.cpp and so we
> end up spending a fair amount of time transitioning from JS to C++ to JS to
> C++.
>
> That seems like the sort of thing that SpiderMonkey may be able to
> address in the short term, depending on what exactly it turns out to be.
> Is there a bug on file somewhere to coordinate the investigation?
>
> -Terrence
>
> > -Jeff
> > ___
> > dev-platform mailing list
> > dev-platform@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-platform
>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: JavaScript reference changes: looking for opinions

[Tom Schuster]

I agree, it is also super tedious to set up and update. You always
have to remember to go to /prototype and sometimes you need to clear
the caching of the pages that include it. I think some these pages
sometimes have extra information that is not included in the main
page, but I doubt many people even look there. When I created
https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/ParallelArray,
I also didn't bother to create the prototype page.


On Mon, Apr 22, 2013 at 9:04 PM, Eric Shepherd  wrote:
> Currently, the JavaScript reference content for the global classes (String,
> Array, etc), are divided up such that the class methods and properties and
> the prototype methods and properties are documented separately, with links
> between them. For example, see:
>
> https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/String
>
> and
>
> https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/String/prototype
>
> While this might make sense to super-expert JavaScript folks (indeed, it was
> their idea to do it this way), it's actually really confusing to everyone
> else.
>
> I'd like to propose we merge them back together, so that the stuff currently
> documented on the "prototype" page is in the main body of the class's
> documentation where most people would expect it to be. If useful, we can
> come up with a badge to put next to items that are part of the prototype (or
> not) to differentiate between them.
>
> But the current organization is, well, kind of weird.
>
> Any opinions on this pro or con before we actually try to find someone to do
> the work?
>
> --
> Eric Shepherd
> Developer Documentation Lead
> Mozilla
> Blog: http://www.bitstampede.com/
> Twitter: @sheppy
>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Some data on mozilla-inbound

[Kartikaya Gupta]

TL;DR:
* Inbound is closed 25% of the time
* Turning off coalescing could increase resource usage by up to 60% (but 
probably less than this).
* We spend 24% of our machine resources on changes that are later backed 
out, or changes that are doing the backout
* The vast majority of changesets that are backed out from inbound are 
detectable on a try push


Because of the large effect from coalescing, any changes to the current 
process must not require running the full set of tests on every push. 
(In my proposal this is easily accomplished with trychooser syntax, but 
other proposals include rotating through T-runs on pushes, etc.).


--- Long verion below ---

Following up from the infra load meeting we had last week, I spent some 
time this weekend crunching various pieces of data on mozilla-inbound to 
get a sense of how much coalescing actually helps us, how much backouts 
hurt us, and generally to get some data on the impact of my previous 
proposal for using a multi-headed tree. I didn't get all the data that I 
wanted but as I probably won't get back to this for a bit, I thought I'd 
share what I found so far and see if anybody has other specific pieces 
of data they would like to see gathered.


-- Inbound uptime --

I looked at a ~9 day period from April 7th to April 16th. During this time:
* inbound was closed for 24.9587% of the total time
* inbound was closed for 15.3068% of the total time due to "bustage".
* inbound was closed for 11.2059% of the total time due to "infra".

Notes:
1) "bustage" and "infra" were determined by grep -i on the data from 
treestatus.mozilla.org.

2) There is some overlap so bustage + infra != total.
3) I also weighted the downtime using checkins-per-hour histogram from 
joduinn's blog at [1], but this didn't have a significant impact: the 
total, bustage, and infra downtime percentages moved to 25.5392%, 
15.7285%, and 11.3748% respectively.


-- Backout changes --

Next I did an analysis of the changes that landed on inbound during that 
time period. The exact pushlog that I looked at (corresponding to the 
same April 7 - April 16 time period) is at [2]. I removed all of the 
merge changesets from this range, since I wanted to look at inbound in 
as much isolation as possible.


In this range:
* there were a total of 916 changesets
* there were a total of 553 "pushes"
* 74 of the 916 changesets (8.07%) were backout changesets
* 116 of the 916 changesets (12.66%) were backed out
* removing all backouts and changes backed out removed 114 pushes (20.6%)

Of the 116 changesets that were backed out:
* 37 belonged to single-changeset pushes
* 65 belonged to multi-changeset pushes where the entire pushed was 
backed out
* 14 belonged to multi-changeset pushes where the changesets were 
selectively backed out


Of the 74 backout changesets:
* 4 were for commit message problems
* 25 were for build failures
* 36 were for test failures
* 5 were for leaks/talos regressions
* 1 was for premature landing
* 3 were for unknown reasons

Notes:
1) There were actually 79 backouts, but I ignored 5 of them because they 
backed out changes that happened prior to the start of my range).

2) Additional changes at the end of my range may have been backed out,
but the backouts were not in my range so I didn't include them in my
analysis.
3) The 14 csets that were selectively backed out is interesting to me 
because it implies that somebody did some work to identify which changes 
in the push were bad, and this naturally means that there is room to 
save on doing that work.


-- Merge conflicts --

I also wanted to determine how many of these changes conflicted with 
each other, and how far away the conflicting changes were. I got a 
partial result here but I need to do more analysis before I have numbers 
worth posting.


-- Build farm resources --

Finally, I used a combination of gps' mozilla-build-analyzer tool [3] 
and some custom tools to determine how much machine time was spent on 
building all of these pushes and changes.


I looked at all the build.json files [4] from the 6th of April to the 
17th of April and pulled out all the jobs that corresponding to the 
"push" changesets in my range above. For this set of 553 changesets, 
there were 500 (exactly!) distinct "builders". 111 of these had "-pgo" 
or "_pgo" in the name, and I excluded them. I created a 553x389 matrix 
with the remaining builders and filled in how much time was spent on 
each changeset for each builder (in case of multiple jobs, I added the 
times).


Then I assumed that any empty field in the 553x389 matrix was a result 
of coalescing. This is a grossly simplifying assumption that I would 
like to revisit - I know for Android changes we can detect that in some 
cases and only run the relevant tests; my assumption means the rest of 
the platforms are considered "coalesced" for these changes. I filled in 
these fields in the matrix with the average time spent on all the other 
builds for that builder in the matri

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Terrence Cole]

On 04/22/2013 12:12 PM, Jeff Muizelaar wrote:
> On 2013-04-22, at 2:15 PM, Bill McCloskey wrote:
>
>> I can't agree with you more, Justin. I think Boris is right that we should 
>> make these decisions on a case-by-case basis. But in the case of these 
>> workers, it seems clear that converting them to C++ is the way to go, 
>> assuming we have the resources to do so.
> So a specific case that I ran into during the Performance Workshop is 
> RILContentHelper.js. During the startup of the settings app
> we jank for 53ms while initializing the RILContentHelper.js: 
>
> http://people.mozilla.com/~bgirard/cleopatra/#report=bf7077c6552fe2bc015d7074a338b673911f3ce8&search=Mobile

That link gives me this: "Error fetching profile :(. URL:
'http://profile-store.commondatastorage.googleapis.com/bf7077c6552fe2bc015d7074a338b673911f3ce8'.
Did you set the CORS headers?"

>
> There doesn't seem to be anything specific taking that much time in the 
> profile, just general JS overhead. In this case RILContentHelper.js is 
> wrapped by by C++ code in dom/network/src/MobileConnection.cpp and so we end 
> up spending a fair amount of time transitioning from JS to C++ to JS to C++.

That seems like the sort of thing that SpiderMonkey may be able to
address in the short term, depending on what exactly it turns out to be.
Is there a bug on file somewhere to coordinate the investigation?

-Terrence

> -Jeff
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Removing expando properties from window wrapper when freeing sandbox

[Matthew Gertner]

Boris - sorry, these are event listeners.

Bobby - that would rock. I'm removing the event listeners manually but this 
requires some hardcoded dependencies to jQuery that I'd love to get rid of.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Jeff Muizelaar]

On 2013-04-22, at 2:15 PM, Bill McCloskey wrote:

> I can't agree with you more, Justin. I think Boris is right that we should 
> make these decisions on a case-by-case basis. But in the case of these 
> workers, it seems clear that converting them to C++ is the way to go, 
> assuming we have the resources to do so.

So a specific case that I ran into during the Performance Workshop is 
RILContentHelper.js. During the startup of the settings app
we jank for 53ms while initializing the RILContentHelper.js: 

http://people.mozilla.com/~bgirard/cleopatra/#report=bf7077c6552fe2bc015d7074a338b673911f3ce8&search=Mobile

There doesn't seem to be anything specific taking that much time in the 
profile, just general JS overhead. In this case RILContentHelper.js is wrapped 
by by C++ code in dom/network/src/MobileConnection.cpp and so we end up 
spending a fair amount of time transitioning from JS to C++ to JS to C++.

-Jeff
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Terrence Cole]

On 04/22/2013 11:07 AM, Justin Lebar wrote:
>> I can't really agree or disagree without knowing why they use "too much"
>> memory.
> At the risk of sounding like a broken record, it's all in the memory
> reports.  You probably understand this data better than I do.  Extract
> and load in about:memory (button is at the bottom).
>
> http://people.mozilla.org/~jlebar/downloads/merged.json.xz
> http://people.mozilla.org/~jlebar/downloads/unmerged.json.xz
>
> As I said earlier, if the JS team wants to own B2G memory usage and
> commit to getting chrome JS memory usage down to C++ levels within a
> timeframe that's acceptable to the B2G team, that's fantastic.
>
> If on the other hand the JS team is not ready to commit to getting
> this work done on B2G's schedule, then by construction "wait for JS to
> get better" is not a solution that works for us.

I agree: I was not suggesting that as a general solution in any way.

>
> Given how long some of this prerequisite work (e.g. generational
> garbage collection) has been ongoing for, I'm highly dubious of the
> claim that our JS engine will improve at the requisite rate.  

Generational GC is an extremely ambitious undertaking. We have set
realistic milestones for completion and we are meeting our goal dates
more often than not: the project is on schedule. Whether this means it
will be done soon enough for B2G, I have no idea. What does B2G's
schedule look like?

> Where
> we've had success reducing our JS memory in the past (e.g. bug
> 798491), it's been by working within the current reality of the JS
> engine, instead of twiddling our thumbs waiting for the Right Fix
> (e.g. bug 759585, which did not come in time to be useful for B2G
> 1.x).

Agreed. Last week we finally got an actual physical Unagi posting
numbers to AWFY. Nicolas is now looking into our GC tuning parameters
with the goal of improving our numbers there.

>
> Please don't take this as a suggestion that I think you guys are doing
> a bad job -- I continue to characterize the JS team's work as heroic.
> I just think that there's a limit to how much we ought to expect from
> the JS folks, particularly given how many other high-priority projects
> you have.

I did not want to suggest that rewriting some of your modules to C++ is
the wrong solution, given your requirements. Sorry if my response was a
bit harsh; it is extremely frustrating from our side to be told now that
what we did 9 months ago was not good enough when you needed it 3 months
ago. Please keep in mind that we are also attacking the same problem
from the other direction and we'd very much like it if we could make our
work more helpful to you.

-Terrence

>
> On Mon, Apr 22, 2013 at 1:36 PM, Terrence Cole  wrote:
>> On 04/21/2013 04:51 PM, Justin Lebar wrote:
>>> I think we should consider using much less JS in the parts of Gecko that are
>>> used in B2G.  I'd like us to consider writing new modules in C++ where
>>> possible, and I'd like us to consider rewriting existing modules in C++.
>>>
>>> I'm only proposing a change for modules which are enabled for B2G.  For 
>>> modules
>>> which aren't enabled on B2G, I'm not proposing any change.
>>>
>>> What I'd like to come out of this thread is a consensus one way or another 
>>> as
>>> to whether we continue along our current path of writing many features that 
>>> are
>>> enabled on B2G in JS, or whether we change course.
>>>
>>> Since most of these features implemented in JS seem to be DOM features, I'm
>>> particularly interested in the opinions of the DOM folks.  I'm also 
>>> interested
>>> in the opinions of JS folks, particularly those who know about the memory 
>>> usage
>>> of our new JITs.
>>>
>>> In the remainder of this e-mail I'll first explain where our JS memory is
>>> going.  Then I'll address two arguments that might be made against my 
>>> proposal
>>> to use more C++.  Finally, I'll conclude by suggesting a plan of action.
>>>
>>> === Data ===
>>>
>>> Right now about 50% (16mb) of the memory used by the B2G main process
>>> immediately after rebooting is JS.   It is my hypothesis that we could 
>>> greatly
>>> reduce this by converting modules to C++.
>>>
>>> On our 256mb devices, we have about 120mb available to Gecko, so this 16mb
>>> represents 13% of all memory available to B2G.
>>>
>>> To break down the 16mb of JS memory, 8mb is from four workers: ril_worker,
>>> net_worker, wifi_worker (x2).  5mb of the 8mb is under "unused-arenas"; 
>>> this is
>>> fragmentation in the JS heap.  Based on my experience tackling 
>>> fragmentation in
>>> the jemalloc heap, I suspect reducing this would be difficult.  But even if 
>>> we
>>> eliminated all of the fragmentation, we'd still be spending 3mb on these 
>>> four
>>> workers, which I think is likely far more than we need.
>> Once exact rooting of the browser is complete we can implement heap
>> defragmentation easily. Generational GC should help as well here.
>>
>>> The other 8mb is everything else in the system compartme

JavaScript reference changes: looking for opinions

[Eric Shepherd]

Currently, the JavaScript reference content for the global classes 
(String, Array, etc), are divided up such that the class methods and 
properties and the prototype methods and properties are documented 
separately, with links between them. For example, see:


https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/String 



and

https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/String/prototype 



While this might make sense to super-expert JavaScript folks (indeed, 
it was their idea to do it this way), it's actually really confusing to 
everyone else.


I'd like to propose we merge them back together, so that the stuff 
currently documented on the "prototype" page is in the main body of the 
class's documentation where most people would expect it to be. If 
useful, we can come up with a badge to put next to items that are part 
of the prototype (or not) to differentiate between them.


But the current organization is, well, kind of weird.

Any opinions on this pro or con before we actually try to find someone 
to do the work?


--
Eric Shepherd
Developer Documentation Lead
Mozilla
Blog: http://www.bitstampede.com/
Twitter: @sheppy

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Taras Glek]

Mike Hommey wrote:

On Sun, Apr 21, 2013 at 07:51:18PM -0400, Justin Lebar wrote:

I think we should consider using much less JS in the parts of Gecko that are
used in B2G.  I'd like us to consider writing new modules in C++ where
possible, and I'd like us to consider rewriting existing modules in C++.

I'm only proposing a change for modules which are enabled for B2G.  For modules
which aren't enabled on B2G, I'm not proposing any change.

What I'd like to come out of this thread is a consensus one way or another as
to whether we continue along our current path of writing many features that are
enabled on B2G in JS, or whether we change course.

Since most of these features implemented in JS seem to be DOM features, I'm
particularly interested in the opinions of the DOM folks.  I'm also interested
in the opinions of JS folks, particularly those who know about the memory usage
of our new JITs.

In the remainder of this e-mail I'll first explain where our JS memory is
going.  Then I'll address two arguments that might be made against my proposal
to use more C++.  Finally, I'll conclude by suggesting a plan of action.


How about pre-compiling JS in JITed form? That would require the JIT
form to be relocatable if they isn't already, and wouldn't work well on
platforms where we use different instructions depending on the actual
target processor, but I guess that could work on our ARM targets. I
however don't know how much less memory that would take.


JS source is smaller than the compiled variety(atleast on x86), 
especially when compressed. It should be very hard to make JITed code 
smaller


Taras
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Bill McCloskey]

I can't agree with you more, Justin. I think Boris is right that we 
should make these decisions on a case-by-case basis. But in the case of 
these workers, it seems clear that converting them to C++ is the way to 
go, assuming we have the resources to do so.


-Bill

On 04/22/2013 11:07 AM, Justin Lebar wrote:

I can't really agree or disagree without knowing why they use "too much"
memory.

At the risk of sounding like a broken record, it's all in the memory
reports.  You probably understand this data better than I do.  Extract
and load in about:memory (button is at the bottom).

http://people.mozilla.org/~jlebar/downloads/merged.json.xz
http://people.mozilla.org/~jlebar/downloads/unmerged.json.xz

As I said earlier, if the JS team wants to own B2G memory usage and
commit to getting chrome JS memory usage down to C++ levels within a
timeframe that's acceptable to the B2G team, that's fantastic.

If on the other hand the JS team is not ready to commit to getting
this work done on B2G's schedule, then by construction "wait for JS to
get better" is not a solution that works for us.

Given how long some of this prerequisite work (e.g. generational
garbage collection) has been ongoing for, I'm highly dubious of the
claim that our JS engine will improve at the requisite rate.  Where
we've had success reducing our JS memory in the past (e.g. bug
798491), it's been by working within the current reality of the JS
engine, instead of twiddling our thumbs waiting for the Right Fix
(e.g. bug 759585, which did not come in time to be useful for B2G
1.x).

Please don't take this as a suggestion that I think you guys are doing
a bad job -- I continue to characterize the JS team's work as heroic.
I just think that there's a limit to how much we ought to expect from
the JS folks, particularly given how many other high-priority projects
you have.

On Mon, Apr 22, 2013 at 1:36 PM, Terrence Cole  wrote:

On 04/21/2013 04:51 PM, Justin Lebar wrote:

I think we should consider using much less JS in the parts of Gecko that are
used in B2G.  I'd like us to consider writing new modules in C++ where
possible, and I'd like us to consider rewriting existing modules in C++.

I'm only proposing a change for modules which are enabled for B2G.  For modules
which aren't enabled on B2G, I'm not proposing any change.

What I'd like to come out of this thread is a consensus one way or another as
to whether we continue along our current path of writing many features that are
enabled on B2G in JS, or whether we change course.

Since most of these features implemented in JS seem to be DOM features, I'm
particularly interested in the opinions of the DOM folks.  I'm also interested
in the opinions of JS folks, particularly those who know about the memory usage
of our new JITs.

In the remainder of this e-mail I'll first explain where our JS memory is
going.  Then I'll address two arguments that might be made against my proposal
to use more C++.  Finally, I'll conclude by suggesting a plan of action.

=== Data ===

Right now about 50% (16mb) of the memory used by the B2G main process
immediately after rebooting is JS.   It is my hypothesis that we could greatly
reduce this by converting modules to C++.

On our 256mb devices, we have about 120mb available to Gecko, so this 16mb
represents 13% of all memory available to B2G.

To break down the 16mb of JS memory, 8mb is from four workers: ril_worker,
net_worker, wifi_worker (x2).  5mb of the 8mb is under "unused-arenas"; this is
fragmentation in the JS heap.  Based on my experience tackling fragmentation in
the jemalloc heap, I suspect reducing this would be difficult.  But even if we
eliminated all of the fragmentation, we'd still be spending 3mb on these four
workers, which I think is likely far more than we need.

Once exact rooting of the browser is complete we can implement heap
defragmentation easily. Generational GC should help as well here.


The other 8mb is everything else in the system compartment (all our JSMs,
XPCOM components, etc).  In a default B2G build you don't get a lot of insight
into this, because most of the system compartments are squished together to save
memory (bug 798491).  If I set jsloader.reuseGlobal to false, the amount of
memory used increases from 8mb to 15mb, but now we can see where it's going.

The list of worst offenders follows, but because this data was collected with
reuseGlobal turned off, apply generous salt.

   0.74 MB modules/Webapps.jsm
   0.59 MB anonymous sandbox from devtools/dbg-server.jsm:41
   0.53 MB components/SettingsManager.js
   0.53 MB chrome://browser/content/shell.xul
   0.49 MB components/WifiWorker.js
   0.43 MB modules/DOMRequestHelper.jsm
   0.38 MB modules/XPCOMUtils.jsm
   0.34 MB RadioInterfaceLayer.js
   0.31 MB AppsUtils.jsm
   0.27 MB Webapps.js
   0.22 MB BrowserElementParent.jsm
   0.21 MB app://system.gaiamobile.org/index.html

Many (but certainly not all) of these modules could be rewritten in C++.

Beyond this list, it's deat

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Justin Lebar]

> I can't really agree or disagree without knowing why they use "too much"
> memory.

At the risk of sounding like a broken record, it's all in the memory
reports.  You probably understand this data better than I do.  Extract
and load in about:memory (button is at the bottom).

http://people.mozilla.org/~jlebar/downloads/merged.json.xz
http://people.mozilla.org/~jlebar/downloads/unmerged.json.xz

As I said earlier, if the JS team wants to own B2G memory usage and
commit to getting chrome JS memory usage down to C++ levels within a
timeframe that's acceptable to the B2G team, that's fantastic.

If on the other hand the JS team is not ready to commit to getting
this work done on B2G's schedule, then by construction "wait for JS to
get better" is not a solution that works for us.

Given how long some of this prerequisite work (e.g. generational
garbage collection) has been ongoing for, I'm highly dubious of the
claim that our JS engine will improve at the requisite rate.  Where
we've had success reducing our JS memory in the past (e.g. bug
798491), it's been by working within the current reality of the JS
engine, instead of twiddling our thumbs waiting for the Right Fix
(e.g. bug 759585, which did not come in time to be useful for B2G
1.x).

Please don't take this as a suggestion that I think you guys are doing
a bad job -- I continue to characterize the JS team's work as heroic.
I just think that there's a limit to how much we ought to expect from
the JS folks, particularly given how many other high-priority projects
you have.

On Mon, Apr 22, 2013 at 1:36 PM, Terrence Cole  wrote:
> On 04/21/2013 04:51 PM, Justin Lebar wrote:
>> I think we should consider using much less JS in the parts of Gecko that are
>> used in B2G.  I'd like us to consider writing new modules in C++ where
>> possible, and I'd like us to consider rewriting existing modules in C++.
>>
>> I'm only proposing a change for modules which are enabled for B2G.  For 
>> modules
>> which aren't enabled on B2G, I'm not proposing any change.
>>
>> What I'd like to come out of this thread is a consensus one way or another as
>> to whether we continue along our current path of writing many features that 
>> are
>> enabled on B2G in JS, or whether we change course.
>>
>> Since most of these features implemented in JS seem to be DOM features, I'm
>> particularly interested in the opinions of the DOM folks.  I'm also 
>> interested
>> in the opinions of JS folks, particularly those who know about the memory 
>> usage
>> of our new JITs.
>>
>> In the remainder of this e-mail I'll first explain where our JS memory is
>> going.  Then I'll address two arguments that might be made against my 
>> proposal
>> to use more C++.  Finally, I'll conclude by suggesting a plan of action.
>>
>> === Data ===
>>
>> Right now about 50% (16mb) of the memory used by the B2G main process
>> immediately after rebooting is JS.   It is my hypothesis that we could 
>> greatly
>> reduce this by converting modules to C++.
>>
>> On our 256mb devices, we have about 120mb available to Gecko, so this 16mb
>> represents 13% of all memory available to B2G.
>>
>> To break down the 16mb of JS memory, 8mb is from four workers: ril_worker,
>> net_worker, wifi_worker (x2).  5mb of the 8mb is under "unused-arenas"; this 
>> is
>> fragmentation in the JS heap.  Based on my experience tackling fragmentation 
>> in
>> the jemalloc heap, I suspect reducing this would be difficult.  But even if 
>> we
>> eliminated all of the fragmentation, we'd still be spending 3mb on these four
>> workers, which I think is likely far more than we need.
>
> Once exact rooting of the browser is complete we can implement heap
> defragmentation easily. Generational GC should help as well here.
>
>> The other 8mb is everything else in the system compartment (all our JSMs,
>> XPCOM components, etc).  In a default B2G build you don't get a lot of 
>> insight
>> into this, because most of the system compartments are squished together to 
>> save
>> memory (bug 798491).  If I set jsloader.reuseGlobal to false, the amount of
>> memory used increases from 8mb to 15mb, but now we can see where it's going.
>>
>> The list of worst offenders follows, but because this data was collected with
>> reuseGlobal turned off, apply generous salt.
>>
>>   0.74 MB modules/Webapps.jsm
>>   0.59 MB anonymous sandbox from devtools/dbg-server.jsm:41
>>   0.53 MB components/SettingsManager.js
>>   0.53 MB chrome://browser/content/shell.xul
>>   0.49 MB components/WifiWorker.js
>>   0.43 MB modules/DOMRequestHelper.jsm
>>   0.38 MB modules/XPCOMUtils.jsm
>>   0.34 MB RadioInterfaceLayer.js
>>   0.31 MB AppsUtils.jsm
>>   0.27 MB Webapps.js
>>   0.22 MB BrowserElementParent.jsm
>>   0.21 MB app://system.gaiamobile.org/index.html
>>
>> Many (but certainly not all) of these modules could be rewritten in C++.
>>
>> Beyond this list, it's death by a thousand cuts; there are 100 compartments 
>> in
>> there, and they each cost a small amount.
>>
>

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Terrence Cole]

On 04/21/2013 04:51 PM, Justin Lebar wrote:
> I think we should consider using much less JS in the parts of Gecko that are
> used in B2G.  I'd like us to consider writing new modules in C++ where
> possible, and I'd like us to consider rewriting existing modules in C++.
> 
> I'm only proposing a change for modules which are enabled for B2G.  For 
> modules
> which aren't enabled on B2G, I'm not proposing any change.
> 
> What I'd like to come out of this thread is a consensus one way or another as
> to whether we continue along our current path of writing many features that 
> are
> enabled on B2G in JS, or whether we change course.
> 
> Since most of these features implemented in JS seem to be DOM features, I'm
> particularly interested in the opinions of the DOM folks.  I'm also interested
> in the opinions of JS folks, particularly those who know about the memory 
> usage
> of our new JITs.
> 
> In the remainder of this e-mail I'll first explain where our JS memory is
> going.  Then I'll address two arguments that might be made against my proposal
> to use more C++.  Finally, I'll conclude by suggesting a plan of action.
> 
> === Data ===
> 
> Right now about 50% (16mb) of the memory used by the B2G main process
> immediately after rebooting is JS.   It is my hypothesis that we could greatly
> reduce this by converting modules to C++.
> 
> On our 256mb devices, we have about 120mb available to Gecko, so this 16mb
> represents 13% of all memory available to B2G.
> 
> To break down the 16mb of JS memory, 8mb is from four workers: ril_worker,
> net_worker, wifi_worker (x2).  5mb of the 8mb is under "unused-arenas"; this 
> is
> fragmentation in the JS heap.  Based on my experience tackling fragmentation 
> in
> the jemalloc heap, I suspect reducing this would be difficult.  But even if we
> eliminated all of the fragmentation, we'd still be spending 3mb on these four
> workers, which I think is likely far more than we need.

Once exact rooting of the browser is complete we can implement heap
defragmentation easily. Generational GC should help as well here.

> The other 8mb is everything else in the system compartment (all our JSMs,
> XPCOM components, etc).  In a default B2G build you don't get a lot of insight
> into this, because most of the system compartments are squished together to 
> save
> memory (bug 798491).  If I set jsloader.reuseGlobal to false, the amount of
> memory used increases from 8mb to 15mb, but now we can see where it's going.
> 
> The list of worst offenders follows, but because this data was collected with
> reuseGlobal turned off, apply generous salt.
> 
>   0.74 MB modules/Webapps.jsm
>   0.59 MB anonymous sandbox from devtools/dbg-server.jsm:41
>   0.53 MB components/SettingsManager.js
>   0.53 MB chrome://browser/content/shell.xul
>   0.49 MB components/WifiWorker.js
>   0.43 MB modules/DOMRequestHelper.jsm
>   0.38 MB modules/XPCOMUtils.jsm
>   0.34 MB RadioInterfaceLayer.js
>   0.31 MB AppsUtils.jsm
>   0.27 MB Webapps.js
>   0.22 MB BrowserElementParent.jsm
>   0.21 MB app://system.gaiamobile.org/index.html
> 
> Many (but certainly not all) of these modules could be rewritten in C++.
> 
> Beyond this list, it's death by a thousand cuts; there are 100 compartments in
> there, and they each cost a small amount.
> 
> I've attached two about:memory dumps collected on my hamachi device soon after
> reboot, so you can examine the situation more closely, if you like.
> merged.json was collected with the default config, and unmerged.json was
> collected with jsloader.reuseGlobal set to false.
> 
> Download and extract these files and then open them with the button at
> the bottom
> of about:memory in Nightly.
> 
> (Before you ask: Most of the heap-unclassified in these dumps is
> graphics memory,
> allocated in drivers.)
> 
> === Should we use JS because it's nicer than C++? ===
> 
> I recognize that in many ways JS is a more convenient language than C++.  But
> that's besides the point here.  The point is that in the environment we're
> targeting, our implementation of JS is too heavyweight.  We can either fix our
> implementation or use less JS, but we can't continue using as much JS as we
> like without doing one of these two things.
> 
> === Why not just make JS slimmer? ===
> 
> It's been suggested to me that instead of converting existing and future JS
> code to C++, we should focus on making our JS engine slimmer.  Such changes
> would of course have the advantage of improving our handling of web content on
> B2G.
> 
> I'm absolutely in favor of reducing JS memory usage, but I see this effort as
> orthogonal to the question of rewriting our current code and writing our 
> future
> code in C++, for a few reasons.
> 
> 1. Content JS does not run in the B2G main process, where the impact of high
> memory usage is strongest.  We can probably tolerate higher memory usage for
> content JS than we can for main-process code.  I think it makes sense for our
> JS team to focus their ef

Re: Nightly *very* crashy on OSX

[Kannan Vijayan]

On 13-04-22 12:50 PM, Benjamin Smedberg wrote:

On 4/22/2013 12:41 PM, Kannan Vijayan wrote:


There are a whole bunch of arguments-object related crashes showing 
up that I think are related to my push over the weekend which enabled 
arguments-object support in Ion.  I'm working on resolving these 
quickly.  :decoder seems to have generated a nice fuzz bug in debug 
mode that's simple and reproduces reliably, and I'm expecting that to 
lead to a fix.


Working on it as fast as I can.

Can we back out or turn off that support until you've found the problem?

--BDS


Sure, I'll push something quick to disable that support in the meantime.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

WebAPI Meeting: Tuesday 23 April @ 10 AM Pacific [1]

[Andrew Overholt]

Meeting Details:

* Agenda: https://wiki.mozilla.org/WebAPI/2013-04-23
* WebAPI Vidyo room
* Amoeba conf. room, San Francisco office (7A)
* Spadina conf. room, Toronto office
* Allo Allo conf. room, London office

* Vidyo Phone # +1-650-903-0800 x92 Conference #98413 (US/INTL)
* US Vidyo Phone # 1-800-707-2533 (PIN 369) Conference #98413 (US)

* Join irc.mozilla.org #webapi for back channel

Notes will be taken on etherpad:

https://etherpad.mozilla.org/webapi-meetingnotes

All are welcome!

Andrew

[1]
http://www.timeanddate.com/worldclock/fixedtime.html?msg=WebAPI+meeting&iso=20130423T10&p1=224&am=30 


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Nightly *very* crashy on OSX

[Benjamin Smedberg]

On 4/22/2013 12:41 PM, Kannan Vijayan wrote:


There are a whole bunch of arguments-object related crashes showing up 
that I think are related to my push over the weekend which enabled 
arguments-object support in Ion.  I'm working on resolving these 
quickly.  :decoder seems to have generated a nice fuzz bug in debug 
mode that's simple and reproduces reliably, and I'm expecting that to 
lead to a fix.


Working on it as fast as I can.

Can we back out or turn off that support until you've found the problem?

--BDS

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Nightly *very* crashy on OSX

[Kannan Vijayan]

There are a whole bunch of arguments-object related crashes showing up 
that I think are related to my push over the weekend which enabled 
arguments-object support in Ion.  I'm working on resolving these 
quickly.  :decoder seems to have generated a nice fuzz bug in debug mode 
that's simple and reproduces reliably, and I'm expecting that to lead to 
a fix.


Working on it as fast as I can.

Kannan

On 13-04-22 12:37 PM, Alex Keybl wrote:

To hopefully close the loop on this, 860438 – Remove custom cx-stack munging 
scattered around Gecko was backed out to resolve the crash in 863646 – Start up 
crash in nsContentUtils::GetCurrentJSContext in profile manager (-p , 
-profilemanager, always ask at startup).

That leaves the crash regression in 864125 – crash in 
js::ion::DoTypeMonitorFallback, which is still occurring in the most recent 
Nightly build (from my look at crash-stats). Including Naveed here.

-Alex

On Apr 21, 2013, at 6:55 AM, Axel Hecht  wrote:


Hi,

I'm having a very crashy nightly, uptime below an hour, not really bound to a 
site.

Might be https://bugzilla.mozilla.org/show_bug.cgi?id=864125, but I've 
experienced a bunch of crashes, all with pretty non-existing stack traces of no 
or one frame.

bp-48ad9b29-145f-49ec-b282-5538f2130421 4/21/13 3:27 PM
bp-bea9322a-ab85-4586-8f26-bfbcb2130421 4/21/13 2:45 PM
bp-b3b43fa7-4c37-4f92-8d17-c82802130420 4/20/13 10:59 PM
bp-7e7f70e9-85c9-4fd2-a2d6-31c892130420 4/20/13 8:34 PM
bp-3faed1dd-98bb-4448-997b-db6f22130420 4/20/13 8:16 PM
bp-5440caa0-7ebc-48e2-bd15-7fcf12130416 4/17/13 12:44 AM
bp-3dbd9606-7d63-4a90-957a-98f772130416 4/17/13 12:32 AM
bp-2b7ac91d-1110-4780-9370-89a372130416 4/17/13 12:31 AM

Any ideas?

Axel
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Nightly *very* crashy on OSX

[Alex Keybl]

To hopefully close the loop on this, 860438 – Remove custom cx-stack munging 
scattered around Gecko was backed out to resolve the crash in 863646 – Start up 
crash in nsContentUtils::GetCurrentJSContext in profile manager (-p , 
-profilemanager, always ask at startup).

That leaves the crash regression in 864125 – crash in 
js::ion::DoTypeMonitorFallback, which is still occurring in the most recent 
Nightly build (from my look at crash-stats). Including Naveed here.

-Alex

On Apr 21, 2013, at 6:55 AM, Axel Hecht  wrote:

> Hi,
> 
> I'm having a very crashy nightly, uptime below an hour, not really bound to a 
> site.
> 
> Might be https://bugzilla.mozilla.org/show_bug.cgi?id=864125, but I've 
> experienced a bunch of crashes, all with pretty non-existing stack traces of 
> no or one frame.
> 
> bp-48ad9b29-145f-49ec-b282-5538f2130421   4/21/13 3:27 PM
> bp-bea9322a-ab85-4586-8f26-bfbcb2130421   4/21/13 2:45 PM
> bp-b3b43fa7-4c37-4f92-8d17-c82802130420   4/20/13 10:59 PM
> bp-7e7f70e9-85c9-4fd2-a2d6-31c892130420   4/20/13 8:34 PM
> bp-3faed1dd-98bb-4448-997b-db6f22130420   4/20/13 8:16 PM
> bp-5440caa0-7ebc-48e2-bd15-7fcf12130416   4/17/13 12:44 AM
> bp-3dbd9606-7d63-4a90-957a-98f772130416   4/17/13 12:32 AM
> bp-2b7ac91d-1110-4780-9370-89a372130416   4/17/13 12:31 AM
> 
> Any ideas?
> 
> Axel
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

DOM Bindings Meeting - Monday @ 12:30 PM PDT

[Kyle Huey]

Our (ostensibly) weekly DOM bindings meetings continue on Monday April 22th
at 12:30 PM PDT.

Meeting details:

* Monday, April 22, 2013, 12:30 PM PDT (3:30 PM EDT/9:30 PM CEST)
* Conference room 7-N, San Francisco office, 7th floor.
* Dial-in Info:
 - Vidyo room: SFO-7N
 - In office or soft phone: extension 92
 - US/INTL: 650-903-0800 or 650-215-1282 then extension 92
 - Toll-free: 800-707-2533 then password 369
 - Conference num 95704
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Justin Lebar]

> So to the extent that b2g is in a "not enough hands" situation, implementing 
> in JS
> makes sense.

I don't feel like b2g is in this position.  We don't have a lot of
Gecko hackers with 2+ years of experience, but we do have a lot of
hackers.  Whether or not we have a lot of C++ versus JS hackers, I
can't say.

> To the extent that we have code whose performance we want to maximize or whose
> memory footprint we need to minimize, a JS implementation right now is not a 
> good
> idea.

My contention is that /everything that runs on B2G/ is code whose
memory footprint we need to minimize.  Fitting inside 120mb leaves
very little room for waste.

But it doesn't sound like you disagree too much with the idea of
re-implementing the worst offenders in C++, as opposed to sitting back
and hoping for heroics from the JS team.  (To be clear, if I had to
choose a team to deliver heroics, I'd choose the JS team in a
heartbeat.)

It seems foolhardy to me to on the one hand do the work of
re-implementing these workers in C++, and on the other hand continue
implementing DOM features in JS that eat up tens or hundreds of KB of
memory each.  But I suppose that's better than doing nothing, and if
we all can agree on doing that much, I'd be happy.

On Mon, Apr 22, 2013 at 10:40 AM, Boris Zbarsky  wrote:
> On 4/21/13 7:51 PM, Justin Lebar wrote:
>>
>> Since most of these features implemented in JS seem to be DOM features,
>> I'm
>> particularly interested in the opinions of the DOM folks.
>
>
> Opinions differ.  ;)
>
> I think for DOM features that are not invoked on most pages, implementing
> them in JS seems like a reasonable choice.  We are actively working on
> making this easier to do than it is now.
>
> This is only a viable implementation strategy for objects you don't expect
> to have lots of around, since there is a lot more per-object overhead for a
> JS-implemented DOM object.  But lots of things that are currently
> implemented in JS are per-page singletons, so that's ok.
>
> I'm very sympathetic to Andreas' points about memory-safety (especially for
> hard-to-fuzz things) and ease of prototyping and implementation. But on the
> flip side, there are security bugs that JS implementations are subject to
> (involving content passing in unexpected objects and whatnot) that a C++
> implementation simply never has to deal with.  See
> https://bugzilla.mozilla.org/show_bug.cgi?id=856042 for those with the
> access...  I'm hoping that putting a WebIDL layer in front of the JS
> implementations will help, but will of course add to the per-object
> overhead.
>
> One other thing to keep in mind is that doing an implementation in JS does
> not involve having to figure out cycle collection, etc, boilerplate...
>
> So to the extent that b2g is in a "not enough hands" situation, implementing
> in JS makes sense.
>
> To the extent that we have code whose performance we want to maximize or
> whose memory footprint we need to minimize, a JS implementation right now is
> not a good idea.
>
> I'm afraid in practice the right decision should probably be made on a
> per-component basis, and the hard part with that is making sure informed
> decisions are made.  :(
>
> -Boris
>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Justin Lebar]

> How about making the JS engine use jemalloc instead of its own
> allocator? Does anything actually rely on the arenas being independent
> at the paging level?

My understanding, which may be wrong, is that the JS engine needs to
be able to quickly map an object's address to a compartment.  They do
this by keeping a map (in the runtime?) of chunk addresses (1mb
aligned) to compartments.  Given an address, you can easily find its
chunk (zero the low-order bits), and then it's a quick lookup to find
its compartment.

We could imagine shrinking chunks to 4kb, but that doesn't solve the
fragmentation problem here; the fragmentation in "unused-arenas" is
/within/ a page.  (We also have a bunch of decommitted memory in these
workers due to fragmentation between pages, but that's not a problem.)
 What you really need, I guess, is a moving GC.

But I emphasize again that reducing the unused-arenas does not solve
all of our problems, only the single largest.

On Mon, Apr 22, 2013 at 11:05 AM, Mike Hommey  wrote:
> On Mon, Apr 22, 2013 at 10:53:40AM -0400, Justin Lebar wrote:
>> > How about pre-compiling JS in JITed form?
>>
>> While significant, it seems that memory used for script source isn't
>> the biggest offender.
>>
>> Full details are in the about:memory reports,
>>
>> http://people.mozilla.org/~jlebar/downloads/merged.json.xz
>> http://people.mozilla.org/~jlebar/downloads/unmerged.json.xz
>>
>> but here's a teaser.
>>
>> This is the wifi worker.  I think "script-sources" is code.  Note that
>> fragmentation (unused-arenas) is way too high, but even despite this
>> the worker uses too much memory.
>
> How about making the JS engine use jemalloc instead of its own
> allocator? Does anything actually rely on the arenas being independent
> at the paging level?
>
> Mike
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Mike Hommey]

On Mon, Apr 22, 2013 at 10:53:40AM -0400, Justin Lebar wrote:
> > How about pre-compiling JS in JITed form?
> 
> While significant, it seems that memory used for script source isn't
> the biggest offender.
> 
> Full details are in the about:memory reports,
> 
> http://people.mozilla.org/~jlebar/downloads/merged.json.xz
> http://people.mozilla.org/~jlebar/downloads/unmerged.json.xz
> 
> but here's a teaser.
> 
> This is the wifi worker.  I think "script-sources" is code.  Note that
> fragmentation (unused-arenas) is way too high, but even despite this
> the worker uses too much memory.

How about making the JS engine use jemalloc instead of its own
allocator? Does anything actually rely on the arenas being independent
at the paging level?

Mike
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Justin Lebar]

> How about pre-compiling JS in JITed form?

While significant, it seems that memory used for script source isn't
the biggest offender.

Full details are in the about:memory reports,

http://people.mozilla.org/~jlebar/downloads/merged.json.xz
http://people.mozilla.org/~jlebar/downloads/unmerged.json.xz

but here's a teaser.

This is the wifi worker.  I think "script-sources" is code.  Note that
fragmentation (unused-arenas) is way too high, but even despite this
the worker uses too much memory.

> 2.38 MB (05.13%) -- worker(resource://gre/modules/wifi_worker.js, 0x45584800)
> ├──1.77 MB (03.81%) -- gc-heap
> │  ├──1.74 MB (03.74%) ── unused-arenas
> │  └──0.03 MB (00.07%) -- (2 tiny)
> │ ├──0.03 MB (00.07%) ── chunk-admin
> │ └──0.00 MB (00.00%) ── unused-chunks
> └──0.61 MB (01.32%) -- (3 tiny)
>├──0.26 MB (00.57%) -- runtime
>│  ├──0.13 MB (00.27%) ── gc-marker
>│  ├──0.06 MB (00.12%) ── jaeger-code
>│  ├──0.04 MB (00.08%) ── runtime-object
>│  ├──0.02 MB (00.03%) ── atoms-table
>│  ├──0.01 MB (00.03%) ── script-sources
>│  ├──0.01 MB (00.01%) ── unused-code
>│  ├──0.00 MB (00.01%) ── dtoa
>│  ├──0.00 MB (00.01%) ── stack
>│  ├──0.00 MB (00.01%) ── temporary
>│  ├──0.00 MB (00.00%) ── contexts
>│  ├──0.00 MB (00.00%) ── script-filenames
>│  ├──0.00 MB (00.00%) ── ion-code
>│  ├──0.00 MB (00.00%) ── math-cache
>│  └──0.00 MB (00.00%) ── regexp-code
>├──0.25 MB (00.54%) -- compartment(web-worker)
>│  ├──0.13 MB (00.29%) -- gc-heap
>│  │  ├──0.07 MB (00.15%) ── unused-gc-things [2]
>│  │  ├──0.03 MB (00.06%) -- objects
>│  │  │  ├──0.02 MB (00.04%) ── non-function
>│  │  │  └──0.01 MB (00.02%) ── function
>│  │  ├──0.02 MB (00.04%) ── shapes/tree
>│  │  └──0.02 MB (00.03%) ── sundries [2]
>│  ├──0.06 MB (00.13%) ── analysis-temporary
>│  ├──0.03 MB (00.06%) ── other-sundries [2]
>│  ├──0.02 MB (00.04%) ── objects-extra/slots
>│  └──0.01 MB (00.02%) ── script-data
>└──0.10 MB (00.22%) -- compartment(web-worker-atoms)
>   ├──0.09 MB (00.19%) -- gc-heap
>   │  ├──0.09 MB (00.18%) ── strings
>   │  └──0.00 MB (00.01%) ── sundries
>   └──0.01 MB (00.02%) ── other-sundries

Here's the worst-offending compartment (webapps.js) with
jsloader.reuseGlobal turned off.  Recall that flipping this switch
expands memory usage by a factor of 2, and not all pieces of the
compartment are affected equally, so take this with salt.

> 0.74 MB (01.60%) -- compartment([System Principal], 
> resource://gre/modules/Webapps.jsm)
> ├──0.28 MB (00.61%) -- gc-heap
> │  ├──0.10 MB (00.21%) ── unused-gc-things
> │  ├──0.08 MB (00.17%) -- objects
> │  │  ├──0.06 MB (00.12%) ── non-function
> │  │  └──0.02 MB (00.04%) ── function
> │  ├──0.05 MB (00.11%) ── strings
> │  ├──0.03 MB (00.05%) ── shapes/tree
> │  ├──0.02 MB (00.04%) ── scripts
> │  └──0.01 MB (00.02%) ── sundries
> ├──0.17 MB (00.36%) -- string-chars
> │  ├──0.09 MB (00.20%) ── non-huge
> │  └──0.08 MB (00.17%) -- huge
> │ ├──0.04 MB (00.08%) ── string(length=9114, 
> "data:image//png;base64,iVBORw0KG...") [2]
> │ └──0.04 MB (00.08%) ── string(length=9646, 
> "data:image//png;base64,iVBORw0KG...") [2]
> ├──0.11 MB (00.23%) -- objects-extra
> │  ├──0.09 MB (00.20%) ── slots
> │  └──0.02 MB (00.03%) ── property-iterator-data
> ├──0.07 MB (00.16%) ── script-data
> ├──0.06 MB (00.13%) ── analysis-temporary
> ├──0.02 MB (00.04%) ── shapes-extra/tree-tables
> ├──0.02 MB (00.04%) ── other-sundries
> └──0.02 MB (00.03%) ── cross-compartment-wrappers

To me, this seems like death by a thousand cuts; there are /lots/ of
little things that we'd need to improve.

On Mon, Apr 22, 2013 at 10:31 AM, Mike Hommey  wrote:
> On Sun, Apr 21, 2013 at 07:51:18PM -0400, Justin Lebar wrote:
>> I think we should consider using much less JS in the parts of Gecko that are
>> used in B2G.  I'd like us to consider writing new modules in C++ where
>> possible, and I'd like us to consider rewriting existing modules in C++.
>>
>> I'm only proposing a change for modules which are enabled for B2G.  For 
>> modules
>> which aren't enabled on B2G, I'm not proposing any change.
>>
>> What I'd like to come out of this thread is a consensus one way or another as
>> to whether we continue along our current path of writing many features that 
>> are
>> enabled on B2G in JS, or whether we change course.
>>
>> Since most of these features implemented in JS seem to be DOM features, I'm
>> particularly interested in the opinions of the DOM folks.  I'm also 
>> interested
>> in the opinions of JS folks, particularly those who know about the memory 
>> usage
>> of our new JITs.
>>
>> In the remainder of this e-mail I'll first explain where our JS memory is
>> going.  Then I'll address two arguments that might be made against my 
>> proposal
>> to use more C++.  Finally, I'll conclude by suggesting a plan of action.
>
> How about pre-compiling JS in JITed form? That would require the

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Boris Zbarsky]

On 4/21/13 7:51 PM, Justin Lebar wrote:

Since most of these features implemented in JS seem to be DOM features, I'm
particularly interested in the opinions of the DOM folks.


Opinions differ.  ;)

I think for DOM features that are not invoked on most pages, 
implementing them in JS seems like a reasonable choice.  We are actively 
working on making this easier to do than it is now.


This is only a viable implementation strategy for objects you don't 
expect to have lots of around, since there is a lot more per-object 
overhead for a JS-implemented DOM object.  But lots of things that are 
currently implemented in JS are per-page singletons, so that's ok.


I'm very sympathetic to Andreas' points about memory-safety (especially 
for hard-to-fuzz things) and ease of prototyping and implementation. 
But on the flip side, there are security bugs that JS implementations 
are subject to (involving content passing in unexpected objects and 
whatnot) that a C++ implementation simply never has to deal with.  See 
https://bugzilla.mozilla.org/show_bug.cgi?id=856042 for those with the 
access...  I'm hoping that putting a WebIDL layer in front of the JS 
implementations will help, but will of course add to the per-object 
overhead.


One other thing to keep in mind is that doing an implementation in JS 
does not involve having to figure out cycle collection, etc, boilerplate...


So to the extent that b2g is in a "not enough hands" situation, 
implementing in JS makes sense.


To the extent that we have code whose performance we want to maximize or 
whose memory footprint we need to minimize, a JS implementation right 
now is not a good idea.


I'm afraid in practice the right decision should probably be made on a 
per-component basis, and the hard part with that is making sure informed 
decisions are made.  :(


-Boris
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Mike Hommey]

On Sun, Apr 21, 2013 at 07:51:18PM -0400, Justin Lebar wrote:
> I think we should consider using much less JS in the parts of Gecko that are
> used in B2G.  I'd like us to consider writing new modules in C++ where
> possible, and I'd like us to consider rewriting existing modules in C++.
> 
> I'm only proposing a change for modules which are enabled for B2G.  For 
> modules
> which aren't enabled on B2G, I'm not proposing any change.
> 
> What I'd like to come out of this thread is a consensus one way or another as
> to whether we continue along our current path of writing many features that 
> are
> enabled on B2G in JS, or whether we change course.
> 
> Since most of these features implemented in JS seem to be DOM features, I'm
> particularly interested in the opinions of the DOM folks.  I'm also interested
> in the opinions of JS folks, particularly those who know about the memory 
> usage
> of our new JITs.
> 
> In the remainder of this e-mail I'll first explain where our JS memory is
> going.  Then I'll address two arguments that might be made against my proposal
> to use more C++.  Finally, I'll conclude by suggesting a plan of action.

How about pre-compiling JS in JITed form? That would require the JIT
form to be relocatable if they isn't already, and wouldn't work well on
platforms where we use different instructions depending on the actual
target processor, but I guess that could work on our ARM targets. I
however don't know how much less memory that would take.

Mike
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Justin Lebar]

> I think I would like to hear from the JS team on reducing memory use of JS and
> disabling any compilation for infrequently running code before we give
> up on it.

The issue isn't compilation of code; that doesn't stick out in the
memory reports.  The issue seems to be mostly the overhead of the JS
engine for each file (even if the file stores very little data, as
BrowserElementParent does) and also the unavoidable inefficiency
associated with assigning each worker its own runtime (in particular,
the fact that this greatly increases fragmentation).

> Using JS for infrequently executing code should be a memory win.

If the JS team wants to own B2G memory usage from here on out and can
commit the necessary resources to re-optimizing the JS engine
specifically for B2G, then that's great, I'd love to see us test this
hypothesis.

Otherwise, we're leaving a huge amount of memory for B2G on the table
when we have a solution in hand that requires zero heroics.  I think
that's folly.

On Mon, Apr 22, 2013 at 10:11 AM, Andreas Gal  wrote:
> JS is a big advantage for rapid implementation of features and it's
> easier to avoid exploitable mistakes. Also, in many cases JS code
> (bytecode, not data) should be slimmer than C++. Using JS for
> infrequently executing code should be a memory win. I think I would
> like to hear from the JS team on reducing memory use of JS and
> disabling any compilation for infrequently running code before we give
> up on it.
>
> Andreas
>
> Sent from Mobile.
>
> On Apr 22, 2013, at 7:05, Justin Lebar  wrote:
>
>> Of course attachments don't work great on newsgroups.  I've uploaded
>> the about:memory dumps I tried to attach to people.m.o:
>>
>> http://people.mozilla.org/~jlebar/downloads/merged.json.xz
>> http://people.mozilla.org/~jlebar/downloads/unmerged.json.xz
>>
>> On Sun, Apr 21, 2013 at 7:51 PM, Justin Lebar  wrote:
>>> I think we should consider using much less JS in the parts of Gecko that are
>>> used in B2G.  I'd like us to consider writing new modules in C++ where
>>> possible, and I'd like us to consider rewriting existing modules in C++.
>>>
>>> I'm only proposing a change for modules which are enabled for B2G.  For 
>>> modules
>>> which aren't enabled on B2G, I'm not proposing any change.
>>>
>>> What I'd like to come out of this thread is a consensus one way or another 
>>> as
>>> to whether we continue along our current path of writing many features that 
>>> are
>>> enabled on B2G in JS, or whether we change course.
>>>
>>> Since most of these features implemented in JS seem to be DOM features, I'm
>>> particularly interested in the opinions of the DOM folks.  I'm also 
>>> interested
>>> in the opinions of JS folks, particularly those who know about the memory 
>>> usage
>>> of our new JITs.
>>>
>>> In the remainder of this e-mail I'll first explain where our JS memory is
>>> going.  Then I'll address two arguments that might be made against my 
>>> proposal
>>> to use more C++.  Finally, I'll conclude by suggesting a plan of action.
>>>
>>> === Data ===
>>>
>>> Right now about 50% (16mb) of the memory used by the B2G main process
>>> immediately after rebooting is JS.   It is my hypothesis that we could 
>>> greatly
>>> reduce this by converting modules to C++.
>>>
>>> On our 256mb devices, we have about 120mb available to Gecko, so this 16mb
>>> represents 13% of all memory available to B2G.
>>>
>>> To break down the 16mb of JS memory, 8mb is from four workers: ril_worker,
>>> net_worker, wifi_worker (x2).  5mb of the 8mb is under "unused-arenas"; 
>>> this is
>>> fragmentation in the JS heap.  Based on my experience tackling 
>>> fragmentation in
>>> the jemalloc heap, I suspect reducing this would be difficult.  But even if 
>>> we
>>> eliminated all of the fragmentation, we'd still be spending 3mb on these 
>>> four
>>> workers, which I think is likely far more than we need.
>>>
>>> The other 8mb is everything else in the system compartment (all our JSMs,
>>> XPCOM components, etc).  In a default B2G build you don't get a lot of 
>>> insight
>>> into this, because most of the system compartments are squished together to 
>>> save
>>> memory (bug 798491).  If I set jsloader.reuseGlobal to false, the amount of
>>> memory used increases from 8mb to 15mb, but now we can see where it's going.
>>>
>>> The list of worst offenders follows, but because this data was collected 
>>> with
>>> reuseGlobal turned off, apply generous salt.
>>>
>>>  0.74 MB modules/Webapps.jsm
>>>  0.59 MB anonymous sandbox from devtools/dbg-server.jsm:41
>>>  0.53 MB components/SettingsManager.js
>>>  0.53 MB chrome://browser/content/shell.xul
>>>  0.49 MB components/WifiWorker.js
>>>  0.43 MB modules/DOMRequestHelper.jsm
>>>  0.38 MB modules/XPCOMUtils.jsm
>>>  0.34 MB RadioInterfaceLayer.js
>>>  0.31 MB AppsUtils.jsm
>>>  0.27 MB Webapps.js
>>>  0.22 MB BrowserElementParent.jsm
>>>  0.21 MB app://system.gaiamobile.org/index.html
>>>
>>> Many (but certainly not all) of these modules could

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Kyle Huey]

The things on Justin's list do not appear to be infrequently running code.
Wifi, RIL, Apps, mozbrowser, etc are going to be running all the time.

- Kyle


On Mon, Apr 22, 2013 at 7:11 AM, Andreas Gal  wrote:

> JS is a big advantage for rapid implementation of features and it's
> easier to avoid exploitable mistakes. Also, in many cases JS code
> (bytecode, not data) should be slimmer than C++. Using JS for
> infrequently executing code should be a memory win. I think I would
> like to hear from the JS team on reducing memory use of JS and
> disabling any compilation for infrequently running code before we give
> up on it.
>
> Andreas
>
> Sent from Mobile.
>
> On Apr 22, 2013, at 7:05, Justin Lebar  wrote:
>
> > Of course attachments don't work great on newsgroups.  I've uploaded
> > the about:memory dumps I tried to attach to people.m.o:
> >
> > http://people.mozilla.org/~jlebar/downloads/merged.json.xz
> > http://people.mozilla.org/~jlebar/downloads/unmerged.json.xz
> >
> > On Sun, Apr 21, 2013 at 7:51 PM, Justin Lebar 
> wrote:
> >> I think we should consider using much less JS in the parts of Gecko
> that are
> >> used in B2G.  I'd like us to consider writing new modules in C++ where
> >> possible, and I'd like us to consider rewriting existing modules in C++.
> >>
> >> I'm only proposing a change for modules which are enabled for B2G.  For
> modules
> >> which aren't enabled on B2G, I'm not proposing any change.
> >>
> >> What I'd like to come out of this thread is a consensus one way or
> another as
> >> to whether we continue along our current path of writing many features
> that are
> >> enabled on B2G in JS, or whether we change course.
> >>
> >> Since most of these features implemented in JS seem to be DOM features,
> I'm
> >> particularly interested in the opinions of the DOM folks.  I'm also
> interested
> >> in the opinions of JS folks, particularly those who know about the
> memory usage
> >> of our new JITs.
> >>
> >> In the remainder of this e-mail I'll first explain where our JS memory
> is
> >> going.  Then I'll address two arguments that might be made against my
> proposal
> >> to use more C++.  Finally, I'll conclude by suggesting a plan of action.
> >>
> >> === Data ===
> >>
> >> Right now about 50% (16mb) of the memory used by the B2G main process
> >> immediately after rebooting is JS.   It is my hypothesis that we could
> greatly
> >> reduce this by converting modules to C++.
> >>
> >> On our 256mb devices, we have about 120mb available to Gecko, so this
> 16mb
> >> represents 13% of all memory available to B2G.
> >>
> >> To break down the 16mb of JS memory, 8mb is from four workers:
> ril_worker,
> >> net_worker, wifi_worker (x2).  5mb of the 8mb is under "unused-arenas";
> this is
> >> fragmentation in the JS heap.  Based on my experience tackling
> fragmentation in
> >> the jemalloc heap, I suspect reducing this would be difficult.  But
> even if we
> >> eliminated all of the fragmentation, we'd still be spending 3mb on
> these four
> >> workers, which I think is likely far more than we need.
> >>
> >> The other 8mb is everything else in the system compartment (all our
> JSMs,
> >> XPCOM components, etc).  In a default B2G build you don't get a lot of
> insight
> >> into this, because most of the system compartments are squished
> together to save
> >> memory (bug 798491).  If I set jsloader.reuseGlobal to false, the
> amount of
> >> memory used increases from 8mb to 15mb, but now we can see where it's
> going.
> >>
> >> The list of worst offenders follows, but because this data was
> collected with
> >> reuseGlobal turned off, apply generous salt.
> >>
> >>  0.74 MB modules/Webapps.jsm
> >>  0.59 MB anonymous sandbox from devtools/dbg-server.jsm:41
> >>  0.53 MB components/SettingsManager.js
> >>  0.53 MB chrome://browser/content/shell.xul
> >>  0.49 MB components/WifiWorker.js
> >>  0.43 MB modules/DOMRequestHelper.jsm
> >>  0.38 MB modules/XPCOMUtils.jsm
> >>  0.34 MB RadioInterfaceLayer.js
> >>  0.31 MB AppsUtils.jsm
> >>  0.27 MB Webapps.js
> >>  0.22 MB BrowserElementParent.jsm
> >>  0.21 MB app://system.gaiamobile.org/index.html
> >>
> >> Many (but certainly not all) of these modules could be rewritten in C++.
> >>
> >> Beyond this list, it's death by a thousand cuts; there are 100
> compartments in
> >> there, and they each cost a small amount.
> >>
> >> I've attached two about:memory dumps collected on my hamachi device
> soon after
> >> reboot, so you can examine the situation more closely, if you like.
> >> merged.json was collected with the default config, and unmerged.json was
> >> collected with jsloader.reuseGlobal set to false.
> >>
> >> Download and extract these files and then open them with the button at
> >> the bottom
> >> of about:memory in Nightly.
> >>
> >> (Before you ask: Most of the heap-unclassified in these dumps is
> >> graphics memory,
> >> allocated in drivers.)
> >>
> >> === Should we use JS because it's nicer than C++? ===
> >>
> >> I recognize 

Re: Removing expando properties from window wrapper when freeing sandbox

[Bobby Holley]

I've filed bug 864313 for improving the situation here. Should be pretty
straightforward to do.


On Mon, Apr 22, 2013 at 9:20 AM, Boris Zbarsky  wrote:

> On 4/22/13 4:37 AM, Matthew Gertner wrote:
>
>> However, the sandbox lives on if it contains DOM event handlers that have
>> not been removed.
>>
>
> Event handlers, or event listeners?
>
> For event listeners, this happens because they're not holding
> cross-compartment wrappers, presumably... and that's not likely to change;
> see bug 862629.
>
> For event handlers this is surprising now but will stop being so once bug
> 862629 lands.
>
> -Boris
>
> __**_
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/**listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Andreas Gal]

JS is a big advantage for rapid implementation of features and it's
easier to avoid exploitable mistakes. Also, in many cases JS code
(bytecode, not data) should be slimmer than C++. Using JS for
infrequently executing code should be a memory win. I think I would
like to hear from the JS team on reducing memory use of JS and
disabling any compilation for infrequently running code before we give
up on it.

Andreas

Sent from Mobile.

On Apr 22, 2013, at 7:05, Justin Lebar  wrote:

> Of course attachments don't work great on newsgroups.  I've uploaded
> the about:memory dumps I tried to attach to people.m.o:
>
> http://people.mozilla.org/~jlebar/downloads/merged.json.xz
> http://people.mozilla.org/~jlebar/downloads/unmerged.json.xz
>
> On Sun, Apr 21, 2013 at 7:51 PM, Justin Lebar  wrote:
>> I think we should consider using much less JS in the parts of Gecko that are
>> used in B2G.  I'd like us to consider writing new modules in C++ where
>> possible, and I'd like us to consider rewriting existing modules in C++.
>>
>> I'm only proposing a change for modules which are enabled for B2G.  For 
>> modules
>> which aren't enabled on B2G, I'm not proposing any change.
>>
>> What I'd like to come out of this thread is a consensus one way or another as
>> to whether we continue along our current path of writing many features that 
>> are
>> enabled on B2G in JS, or whether we change course.
>>
>> Since most of these features implemented in JS seem to be DOM features, I'm
>> particularly interested in the opinions of the DOM folks.  I'm also 
>> interested
>> in the opinions of JS folks, particularly those who know about the memory 
>> usage
>> of our new JITs.
>>
>> In the remainder of this e-mail I'll first explain where our JS memory is
>> going.  Then I'll address two arguments that might be made against my 
>> proposal
>> to use more C++.  Finally, I'll conclude by suggesting a plan of action.
>>
>> === Data ===
>>
>> Right now about 50% (16mb) of the memory used by the B2G main process
>> immediately after rebooting is JS.   It is my hypothesis that we could 
>> greatly
>> reduce this by converting modules to C++.
>>
>> On our 256mb devices, we have about 120mb available to Gecko, so this 16mb
>> represents 13% of all memory available to B2G.
>>
>> To break down the 16mb of JS memory, 8mb is from four workers: ril_worker,
>> net_worker, wifi_worker (x2).  5mb of the 8mb is under "unused-arenas"; this 
>> is
>> fragmentation in the JS heap.  Based on my experience tackling fragmentation 
>> in
>> the jemalloc heap, I suspect reducing this would be difficult.  But even if 
>> we
>> eliminated all of the fragmentation, we'd still be spending 3mb on these four
>> workers, which I think is likely far more than we need.
>>
>> The other 8mb is everything else in the system compartment (all our JSMs,
>> XPCOM components, etc).  In a default B2G build you don't get a lot of 
>> insight
>> into this, because most of the system compartments are squished together to 
>> save
>> memory (bug 798491).  If I set jsloader.reuseGlobal to false, the amount of
>> memory used increases from 8mb to 15mb, but now we can see where it's going.
>>
>> The list of worst offenders follows, but because this data was collected with
>> reuseGlobal turned off, apply generous salt.
>>
>>  0.74 MB modules/Webapps.jsm
>>  0.59 MB anonymous sandbox from devtools/dbg-server.jsm:41
>>  0.53 MB components/SettingsManager.js
>>  0.53 MB chrome://browser/content/shell.xul
>>  0.49 MB components/WifiWorker.js
>>  0.43 MB modules/DOMRequestHelper.jsm
>>  0.38 MB modules/XPCOMUtils.jsm
>>  0.34 MB RadioInterfaceLayer.js
>>  0.31 MB AppsUtils.jsm
>>  0.27 MB Webapps.js
>>  0.22 MB BrowserElementParent.jsm
>>  0.21 MB app://system.gaiamobile.org/index.html
>>
>> Many (but certainly not all) of these modules could be rewritten in C++.
>>
>> Beyond this list, it's death by a thousand cuts; there are 100 compartments 
>> in
>> there, and they each cost a small amount.
>>
>> I've attached two about:memory dumps collected on my hamachi device soon 
>> after
>> reboot, so you can examine the situation more closely, if you like.
>> merged.json was collected with the default config, and unmerged.json was
>> collected with jsloader.reuseGlobal set to false.
>>
>> Download and extract these files and then open them with the button at
>> the bottom
>> of about:memory in Nightly.
>>
>> (Before you ask: Most of the heap-unclassified in these dumps is
>> graphics memory,
>> allocated in drivers.)
>>
>> === Should we use JS because it's nicer than C++? ===
>>
>> I recognize that in many ways JS is a more convenient language than C++.  But
>> that's besides the point here.  The point is that in the environment we're
>> targeting, our implementation of JS is too heavyweight.  We can either fix 
>> our
>> implementation or use less JS, but we can't continue using as much JS as we
>> like without doing one of these two things.
>>
>> === Why not just make JS slimmer? ===
>>
>

Re: Rethinking the amount of system JS we use in Gecko on B2G

[Justin Lebar]

Of course attachments don't work great on newsgroups.  I've uploaded
the about:memory dumps I tried to attach to people.m.o:

http://people.mozilla.org/~jlebar/downloads/merged.json.xz
http://people.mozilla.org/~jlebar/downloads/unmerged.json.xz

On Sun, Apr 21, 2013 at 7:51 PM, Justin Lebar  wrote:
> I think we should consider using much less JS in the parts of Gecko that are
> used in B2G.  I'd like us to consider writing new modules in C++ where
> possible, and I'd like us to consider rewriting existing modules in C++.
>
> I'm only proposing a change for modules which are enabled for B2G.  For 
> modules
> which aren't enabled on B2G, I'm not proposing any change.
>
> What I'd like to come out of this thread is a consensus one way or another as
> to whether we continue along our current path of writing many features that 
> are
> enabled on B2G in JS, or whether we change course.
>
> Since most of these features implemented in JS seem to be DOM features, I'm
> particularly interested in the opinions of the DOM folks.  I'm also interested
> in the opinions of JS folks, particularly those who know about the memory 
> usage
> of our new JITs.
>
> In the remainder of this e-mail I'll first explain where our JS memory is
> going.  Then I'll address two arguments that might be made against my proposal
> to use more C++.  Finally, I'll conclude by suggesting a plan of action.
>
> === Data ===
>
> Right now about 50% (16mb) of the memory used by the B2G main process
> immediately after rebooting is JS.   It is my hypothesis that we could greatly
> reduce this by converting modules to C++.
>
> On our 256mb devices, we have about 120mb available to Gecko, so this 16mb
> represents 13% of all memory available to B2G.
>
> To break down the 16mb of JS memory, 8mb is from four workers: ril_worker,
> net_worker, wifi_worker (x2).  5mb of the 8mb is under "unused-arenas"; this 
> is
> fragmentation in the JS heap.  Based on my experience tackling fragmentation 
> in
> the jemalloc heap, I suspect reducing this would be difficult.  But even if we
> eliminated all of the fragmentation, we'd still be spending 3mb on these four
> workers, which I think is likely far more than we need.
>
> The other 8mb is everything else in the system compartment (all our JSMs,
> XPCOM components, etc).  In a default B2G build you don't get a lot of insight
> into this, because most of the system compartments are squished together to 
> save
> memory (bug 798491).  If I set jsloader.reuseGlobal to false, the amount of
> memory used increases from 8mb to 15mb, but now we can see where it's going.
>
> The list of worst offenders follows, but because this data was collected with
> reuseGlobal turned off, apply generous salt.
>
>   0.74 MB modules/Webapps.jsm
>   0.59 MB anonymous sandbox from devtools/dbg-server.jsm:41
>   0.53 MB components/SettingsManager.js
>   0.53 MB chrome://browser/content/shell.xul
>   0.49 MB components/WifiWorker.js
>   0.43 MB modules/DOMRequestHelper.jsm
>   0.38 MB modules/XPCOMUtils.jsm
>   0.34 MB RadioInterfaceLayer.js
>   0.31 MB AppsUtils.jsm
>   0.27 MB Webapps.js
>   0.22 MB BrowserElementParent.jsm
>   0.21 MB app://system.gaiamobile.org/index.html
>
> Many (but certainly not all) of these modules could be rewritten in C++.
>
> Beyond this list, it's death by a thousand cuts; there are 100 compartments in
> there, and they each cost a small amount.
>
> I've attached two about:memory dumps collected on my hamachi device soon after
> reboot, so you can examine the situation more closely, if you like.
> merged.json was collected with the default config, and unmerged.json was
> collected with jsloader.reuseGlobal set to false.
>
> Download and extract these files and then open them with the button at
> the bottom
> of about:memory in Nightly.
>
> (Before you ask: Most of the heap-unclassified in these dumps is
> graphics memory,
> allocated in drivers.)
>
> === Should we use JS because it's nicer than C++? ===
>
> I recognize that in many ways JS is a more convenient language than C++.  But
> that's besides the point here.  The point is that in the environment we're
> targeting, our implementation of JS is too heavyweight.  We can either fix our
> implementation or use less JS, but we can't continue using as much JS as we
> like without doing one of these two things.
>
> === Why not just make JS slimmer? ===
>
> It's been suggested to me that instead of converting existing and future JS
> code to C++, we should focus on making our JS engine slimmer.  Such changes
> would of course have the advantage of improving our handling of web content on
> B2G.
>
> I'm absolutely in favor of reducing JS memory usage, but I see this effort as
> orthogonal to the question of rewriting our current code and writing our 
> future
> code in C++, for a few reasons.
>
> 1. Content JS does not run in the B2G main process, where the impact of high
> memory usage is strongest.  We can probably tolerate higher memory usage for
> con

Rethinking the amount of system JS we use in Gecko on B2G

[Justin Lebar]

I think we should consider using much less JS in the parts of Gecko that are
used in B2G.  I'd like us to consider writing new modules in C++ where
possible, and I'd like us to consider rewriting existing modules in C++.

I'm only proposing a change for modules which are enabled for B2G.  For modules
which aren't enabled on B2G, I'm not proposing any change.

What I'd like to come out of this thread is a consensus one way or another as
to whether we continue along our current path of writing many features that are
enabled on B2G in JS, or whether we change course.

Since most of these features implemented in JS seem to be DOM features, I'm
particularly interested in the opinions of the DOM folks.  I'm also interested
in the opinions of JS folks, particularly those who know about the memory usage
of our new JITs.

In the remainder of this e-mail I'll first explain where our JS memory is
going.  Then I'll address two arguments that might be made against my proposal
to use more C++.  Finally, I'll conclude by suggesting a plan of action.

=== Data ===

Right now about 50% (16mb) of the memory used by the B2G main process
immediately after rebooting is JS.   It is my hypothesis that we could greatly
reduce this by converting modules to C++.

On our 256mb devices, we have about 120mb available to Gecko, so this 16mb
represents 13% of all memory available to B2G.

To break down the 16mb of JS memory, 8mb is from four workers: ril_worker,
net_worker, wifi_worker (x2).  5mb of the 8mb is under "unused-arenas"; this is
fragmentation in the JS heap.  Based on my experience tackling fragmentation in
the jemalloc heap, I suspect reducing this would be difficult.  But even if we
eliminated all of the fragmentation, we'd still be spending 3mb on these four
workers, which I think is likely far more than we need.

The other 8mb is everything else in the system compartment (all our JSMs,
XPCOM components, etc).  In a default B2G build you don't get a lot of insight
into this, because most of the system compartments are squished together to save
memory (bug 798491).  If I set jsloader.reuseGlobal to false, the amount of
memory used increases from 8mb to 15mb, but now we can see where it's going.

The list of worst offenders follows, but because this data was collected with
reuseGlobal turned off, apply generous salt.

  0.74 MB modules/Webapps.jsm
  0.59 MB anonymous sandbox from devtools/dbg-server.jsm:41
  0.53 MB components/SettingsManager.js
  0.53 MB chrome://browser/content/shell.xul
  0.49 MB components/WifiWorker.js
  0.43 MB modules/DOMRequestHelper.jsm
  0.38 MB modules/XPCOMUtils.jsm
  0.34 MB RadioInterfaceLayer.js
  0.31 MB AppsUtils.jsm
  0.27 MB Webapps.js
  0.22 MB BrowserElementParent.jsm
  0.21 MB app://system.gaiamobile.org/index.html

Many (but certainly not all) of these modules could be rewritten in C++.

Beyond this list, it's death by a thousand cuts; there are 100 compartments in
there, and they each cost a small amount.

I've attached two about:memory dumps collected on my hamachi device soon after
reboot, so you can examine the situation more closely, if you like.
merged.json was collected with the default config, and unmerged.json was
collected with jsloader.reuseGlobal set to false.

Download and extract these files and then open them with the button at
the bottom
of about:memory in Nightly.

(Before you ask: Most of the heap-unclassified in these dumps is
graphics memory,
allocated in drivers.)

=== Should we use JS because it's nicer than C++? ===

I recognize that in many ways JS is a more convenient language than C++.  But
that's besides the point here.  The point is that in the environment we're
targeting, our implementation of JS is too heavyweight.  We can either fix our
implementation or use less JS, but we can't continue using as much JS as we
like without doing one of these two things.

=== Why not just make JS slimmer? ===

It's been suggested to me that instead of converting existing and future JS
code to C++, we should focus on making our JS engine slimmer.  Such changes
would of course have the advantage of improving our handling of web content on
B2G.

I'm absolutely in favor of reducing JS memory usage, but I see this effort as
orthogonal to the question of rewriting our current code and writing our future
code in C++, for a few reasons.

1. Content JS does not run in the B2G main process, where the impact of high
memory usage is strongest.  We can probably tolerate higher memory usage for
content JS than we can for main-process code.  I think it makes sense for our
JS team to focus their effort on optimizing for content JS, since that's far
more widespread.

2. We have a large team of B2G engineers, some of whom could work exclusively
on converting components from JS to C++.  In contrast, we have a relatively
small team of JS engineers, few of whom can work exclusively on optimizing the
JS engine for B2G's use-cases.

3. I know people get in trouble at Mozilla for suggesting tha

Completed: Tree closure for infrastructure work, Sunday 21April2013 10:00-14:00PT

[Hal Wine]

This work has been completed, and trees reopened as of 12:06 PT (19:06 UTC).

Thanks for everyone's help & cooperation.
--Hal

On 2013-04-18 17:22 , John O'Duinn wrote:
> hi;
> 
> We will have a treeclosing maintenance window Sunday, April 21 from
> 10:00-14:00PT (17:00-21:00UTC). This is for changes and upgrades to
> firewalls in the scl3 colo. More details in tracking bug
> https://bugzil.la/862956.
> 
> All trees will be closed during this period. Jobs in progress at start
> of treeclosure will likely burn, and can be retrigged afterwards if needed.
> 
> Hopefully, this is low-disruption timing. We scheduled this downtime for
> Sunday (21st), after the last of the people leave the b2g workweek in
> Madrid Sunday morning, and before everyone shows up for work Monday. If
> there is any reason we should not proceed as planned on Sunday, please
> let us know by email or in the bug.
> 
> As always, we will reopen the trees as soon as possible after completion
> of the work. Our conservative estimate is a total of 4 hours of tree
> closure. All systems should be operational by the end of the maintenance
> window.
> 
> tc
> John.
> 

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Removing expando properties from window wrapper when freeing sandbox

[Boris Zbarsky]

On 4/22/13 4:37 AM, Matthew Gertner wrote:

However, the sandbox lives on if it contains DOM event handlers that have not 
been removed.


Event handlers, or event listeners?

For event listeners, this happens because they're not holding 
cross-compartment wrappers, presumably... and that's not likely to 
change; see bug 862629.


For event handlers this is surprising now but will stop being so once 
bug 862629 lands.


-Boris
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

W3C Proposed Recommendation: Web Storage

[L. David Baron]

The Web Apps Working Group at W3C has published a Proposed
Recommendation, Web Storage (the stage before W3C's final stage,
Recommendation):
  http://www.w3.org/TR/2013/PR-webstorage-20130409/

There's a call for review to W3C member companies (of which Mozilla
is one) open until Tuesday, May 7.

If there are comments you think Mozilla should send as part of the
review, or if you think Mozilla should voice support or opposition
to the specification, please say so in this thread.  (I'd note,
however, that there have been many previous opportunities to make
comments, so it's somewhat bad form to bring up fundamental issues
for the first time at this stage.)

-David

-- 
𝄞   L. David Baron http://dbaron.org/   𝄂
𝄢   Mozilla   http://www.mozilla.org/   𝄂
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Revamping touch input on Windows

[Jim Mathies]

Someone who knows that code better should comment, but
I assume it should also replace the Metro javascript pan/zoom
stuff at some point, perhaps backed by native gesture detection
when the page doesn't have touch handlers, and using b2g's
gesture detection at other times? Maybe that's orthogonal to
this problem...


We plan to use the async pan zoom controller down is layers and yes it is 
supposed to replace the front end code.


We don't want to use native gesture detection unless we have to. We might 
have to make Win8 specific modifications to the APZC so that reactions to 
user input comply with Win8 input guidelines.


This shouldn't play into what/how events get delivered to content though 
since the apzc (from what I understand) acts as an input trap.


Jim 


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Fw: Revamping touch input on Windows

[Jim Mathies]

I don't think we fire touch events on OSX, so probably not an issue. I'm 
confident Android's behavior is more compliant that Win8 desktop. Metro is 
somewhere in between. Wes makes a good point in the next follow up about 
moving simulated click events into shared code to help standardize. For now 
behavior would be dependent on the widget backend. My goal here is to start 
by getting the two win widget layers acting the same.


As far as moz simple gestures go, they are moz/chrome specific, so they do 
not play into compliance. The metro front end doesn't consume these and we 
don't have add-ons yet, and the desktop front end uses them sparingly (for 
fake pinch zoom). I don't think there are major issues with removing support 
for these on Windows. There might be some addons that consume them. Authors 
could migrate uses to W3C touch events pretty easily I would think.


Jim

-Original Message- 
From: Justin Dolske

Sent: Sunday, April 21, 2013 9:01 PM Newsgroups: mozilla.dev.platform
To: dev-platform@lists.mozilla.org
Subject: Re: Revamping touch input on Windows

On 4/18/13 5:50 AM, Jim Mathies wrote:


One of the concerns here is that since we do not differentiate the metro
and desktop browsers via UA, the two should emulate each other closely.
The browser would appear completely broken to content if the same UA
sent two different event streams. So we need to take into account how
metrofx works as well.


How does the current/proposed state of affairs compare to other
platforms (notably OS X and Android)?

While the Metro-vs-Win8 case seems especially important to get right
(since the same user on the same device may commonly flip between both),
I'd hope web authors can write/test for on one platform and be
reasonable able to expect their code to work the same everywhere. It's
not clear to me how that ties in with what you're talking about.

Justin
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform 


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Removing expando properties from window wrapper when freeing sandbox

[Matthew Gertner]

On Wednesday, April 17, 2013 3:57:58 PM UTC+2, Bobby Holley wrote:
>  When you want the sandbox to die, you can do
> |Components.utils.nukeSandbox(sandbox)|, and it'll go away. Simple as that.
> :-)

I've been using this, and it indeed nukes the sandbox even if there are 
backreferences from expando propertie external to the sandbox. However, the 
sandbox lives on if it contains DOM event handlers that have not been removed. 
Is this by design?
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform