Re: Proposed W3C Charter: Timed Text Working Group (WebVTT and TTML)
On Monday 2014-03-17 13:31 -0400, Richard Eyre wrote: > Thanks for adding me Anne, you did get my email correct :-). > > Personally, I'm not interested in developing TTML. I still agree to all the > points from our previous discussion from the page you linked, Anne. > > I'm not really sure whether having a joint working group would be of > benefit, particularly if there isn't interest in developing TTML in the > WebVTT community, which from my experience is the case. It also scares me a > bit because having a joint group might result in WebVTT being modified for > TTML specific reasons, such as better interoperability, or otherwise, > (possibly?) which wouldn't be good for the spec moving forward due to our > previous concerns. Sorry for not catching up on this thread until it's too late, but anyway: I'm inclined to think that it's not worth putting up a massive fight over the group's organization here, which I think is what it would take to change this plan. I think I'd rather focus the bandwidth of our communication with W3C management on other issues. I think if the group goes off into the weeds, it's worth putting up a fight over that, primarily in the group itself. (On the other hand, I think it is worth listening to the real needs of producers who have large libraries of captions that they'd like to convert to WebVTT.) I suppose I should at least send late feedback over the decision process, and perhaps also that there should be more mention of the working group operating as two subgroups than "Teleconferences: Weekly for TTML, and as needed for WebVTT". -David > On Mon, Mar 17, 2014 at 9:42 AM, Anne van Kesteren wrote: > > > On Tue, Feb 25, 2014 at 7:56 AM, L. David Baron wrote: > > > The W3C is proposing a revised charter for: > > > > > > Timed Text Working Group > > > http://lists.w3.org/Archives/Public/public-new-work/2014Feb/0004.html > > > http://www.w3.org/2013/10/timed-text-charter.html > > > deadline for comments: March 20 > > > > > > This new charter is quite substantive, in that it recharters a > > > working group that was previously only for TTML to now be to develop > > > both TTML and WebVTT. My understanding is that the two halves of > > > the group are expected to operate somewhat separately but also > > > interact, although the charter doesn't seem to say that explicitly. > > > > > > Mozilla has the opportunity to send comments or objections through > > > March 20. Please reply to this thread if you think there's > > > something we should say. > > > > So we commented pretty strongly against this in the past: > > > > http://lists.w3.org/Archives/Public/www-archive/2013May/0034.html > > > > Has something changed? > > > > > > (Not sure I got the correct email address for Rick, I found it on old > > archived email from a year ago.) > > > > > > -- > > http://annevankesteren.nl/ > > -- 𝄞 L. David Baron http://dbaron.org/ 𝄂 𝄢 Mozilla https://www.mozilla.org/ 𝄂 Before I built a wall I'd ask to know What I was walling in or walling out, And to whom I was like to give offense. - Robert Frost, Mending Wall (1914) signature.asc Description: Digital signature ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Memory management in features implemented in JS
On 3/21/14, 10:23 PM, K. Gadd wrote: > A hypothetical scenario (please ignore any minor detail errors, I'm > using this to illustrate the scenario): Let's say I have a main > document and it spawns 3 popup windows as documents. The popup windows > have references to the parent document and use them to post messages > to the parent document for the purposes of collaboration. Now, let's > say I want the popups to *keep working* even if the parent document is > closed. By "keep working" do you mean that the popup windows should still be able to post messages to the main document, and its event handlers should still be firing? > In this scenario, the popups *need* to retain their references > to the parent document, but we don't want to leak the parent document > after all 3 of them are closed. Why are weak references useful here? Suppose we're just using strong references, i.e. the web as it is today. After the main window and all three popups are closed, why would the parent document leak? -j ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Memory management in features implemented in JS
In many cases the point at which an object becomes uninteresting is the point at which it is unreachable, and no deterministically identifiable point before that. It is true that in many cases you don't need anything resembling weak references, and can simply manually mark objects as dead. There are many application scenarios, though, where no single component is able to be responsible for managing the lifetime of a given object because the object is manipulated by multiple other objects. In such scenarios nothing short of reference counting (i.e. garbage collection and notification upon the object being unreachable/having no references) will suffice for preventing leaks. A hypothetical scenario (please ignore any minor detail errors, I'm using this to illustrate the scenario): Let's say I have a main document and it spawns 3 popup windows as documents. The popup windows have references to the parent document and use them to post messages to the parent document for the purposes of collaboration. Now, let's say I want the popups to *keep working* even if the parent document is closed. In this scenario, the popups *need* to retain their references to the parent document, but we don't want to leak the parent document after all 3 of them are closed. This is a scenario you cannot trivially implement without some form of GC notification. If you happen to write everything in C++, then you can be sure you don't need JS-accessible weak references here, but once enough logic lives in JavaScript (say for handling message ports, or releasing image/script resources used by the document, or terminating any playing background audio, or...) it is quite possible that JS will need a way to clean up when an object becomes unreachable. Sometimes these relationships are bidirectional, which makes it difficult to move all the cleanup logic into the target object's onDisposed-equivalent method without forming new cycles. If you want to be able to self-host the vast majority of the web platform and applications in JS, I think it's inevitable that you will need some sort of weak reference equivalent, and this is more or less the conclusion that was reached on es-discuss. Whether it should be what people commonly refer to as 'weak references' or some other abstraction with better properties, I don't know. (I do know that WeakMaps aren't it.) On Fri, Mar 21, 2014 at 7:14 PM, Jim Blandy wrote: > On 03/21/2014 05:03 PM, Boris Zbarsky wrote: >> >> On 3/21/14 6:34 PM, Jim Blandy wrote: >> I don't believe there are any DOM nodes involved in the situation that >> Kyle described at the start of this thread... > > > It's true that when I read, "We are discovering a lot of leaks in JS > implemented DOM objects", I wasn't sure what he was referring to... > > But the same question carries over: isn't there some way to tie the > registration / unregistration of observers / listeners to something more > directly connected to the notification recipient becoming uninteresting? > > That is, there's usually some point well before the notification recipient > becomes garbage that it becomes uninteresting. Hence my mention of DOM node > insertion / removal. > > > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Should nsIPrefBranch.set*Pref return NS_ERROR_UNEXPECTED on type mismatch?
On 2014-03-21 1:07 PM, Gavin Sharp wrote: On Fri, Mar 21, 2014 at 7:40 AM, Irving Reid wrote: extensions.blocklist.pingCountVersion (146 times out of ~1.5 million Nightly telemetry sessions) and extensions.shownSelectionUI (8 times in 1.5m) The prefs in question aren't likely targets for malware, though they could be collateral damage. They don't have default values in all.js. Doesn't corrupting them allow you to prevent the blocklist from working, and preventing the "choose which extensions to install" dialog from appearing on startup? Those sound like things malware would want to do. Gavin extensions.shownSelectionUI was introduced in bug 596343 to provide a run-once UI for users upgrading to Firefox 4, so that users could choose to disable third-party add-ons that had been silently installed in earlier versions of Firefox. The effect of corrupting this pref is that the user will see the selection UI every time their Fx version changes, instead of the compatibility update UI. You're right about extensions.blocklist.pingCountVersion, though. An invalid value will cause us to throw out of nsBlocklistService.notify() around http://dxr.mozilla.org/mozilla-central/source/toolkit/mozapps/extensions/nsBlocklistService.js#525 before we send the blocklist XHR request, so a profile with this pref broken won't update its blocklist. This method is called directly by nsUpdateTimerManager; the exception is caught there and logged to the error console for nobody to notice. - irving - ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Should nsIPrefBranch.set*Pref return NS_ERROR_UNEXPECTED on type mismatch?
On Thursday 2014-03-20 13:38 -0400, Irving Reid wrote: > In a way we're dancing around competing footguns here - do we > protect against bad code trying to break our preferences by setting > a value to the wrong type, or do we protect against a broken > preference messing up our code because we can't recover from a wrong > type? It seems possible to fix both problems (for prefs with values in a default preferences file) by making the pref code refuse to set a preference to a type other than the one in the default preferences file. If the set were in a user preferences file, it would just be ignored; if it were in a set*Pref call, it would throw an exception. Whether this is worth the work or the (probably not huge, but also nonzero) compatibility risk is another question. [ Written on an airplane; sorry if I missed responses in the last 11 hours. ] -David -- 𝄞 L. David Baron http://dbaron.org/ 𝄂 𝄢 Mozilla https://www.mozilla.org/ 𝄂 Before I built a wall I'd ask to know What I was walling in or walling out, And to whom I was like to give offense. - Robert Frost, Mending Wall (1914) signature.asc Description: Digital signature ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Memory management in features implemented in JS
On 3/21/14 10:14 PM, Jim Blandy wrote: It's true that when I read, "We are discovering a lot of leaks in JS implemented DOM objects", I wasn't sure what he was referring to... He means APIs exposed to the web whose implementation is a JS component. So typically these are objects whose useful lifetime depends on what the web page is doing with them. As long as they're reachable from the web page, they need to stay alive and functioning. And in cases when them being "functioning" means them being able to receive notifications from some other piece of chrome JS, we get the problem described here: how do we know when to tell that other piece of chrome JS to forget about us? But the same question carries over: isn't there some way to tie the registration / unregistration of observers / listeners to something more directly connected to the notification recipient becoming uninteresting? I'm not sure there is. :( That is, there's usually some point well before the notification recipient becomes garbage that it becomes uninteresting. Hence my mention of DOM node insertion / removal. Note that DOM nodes that are not in the document tree are "interesting" from the point of view of the web page: they can fire events, load images, be painted to canvases, etc, etc. Unfortunately, the way the web platform works is that anything reachable is potentially interesting. Attempts to cut that off lead to bugs like https://bugzilla.mozilla.org/show_bug.cgi?id=986542 :( -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Memory management in features implemented in JS
On 03/21/2014 05:03 PM, Boris Zbarsky wrote: On 3/21/14 6:34 PM, Jim Blandy wrote: I don't believe there are any DOM nodes involved in the situation that Kyle described at the start of this thread... It's true that when I read, "We are discovering a lot of leaks in JS implemented DOM objects", I wasn't sure what he was referring to... But the same question carries over: isn't there some way to tie the registration / unregistration of observers / listeners to something more directly connected to the notification recipient becoming uninteresting? That is, there's usually some point well before the notification recipient becomes garbage that it becomes uninteresting. Hence my mention of DOM node insertion / removal. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Memory management in features implemented in JS
On 3/21/14 6:34 PM, Jim Blandy wrote: What if these DOM nodes I don't believe there are any DOM nodes involved in the situation that Kyle described at the start of this thread... -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Memory management in features implemented in JS
On 3/21/14, 5:42 PM, Jim Blandy wrote: > On 03/21/2014 03:34 PM, Jim Blandy wrote: >> What if these DOM nodes could use a special class of observers / >> listeners that automatically set themselves aside when the node is >> deleted from the document, and re-instate themselves if the node is >> re-inserted in the document? Similarly for when the window goes away. > > Instead of addObserver or addMessageListener, you'd have > observeWhileInserted or listenWhileInserted. Implemented in some > clever and efficient way to avoid thrashing during heavy DOM manipulation. +1e9 More of this sort of thing please. Deterministic, declarative, does what it says on the label, easy to use correctly. The opposite of weak references on every axis. Stop me if you've heard this one. Some people, when confronted with a problem, think, "I know, I'll use weak references." —Oh you've heard it? Sorry. -j ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Memory management in features implemented in JS
On 03/21/2014 03:34 PM, Jim Blandy wrote: What if these DOM nodes could use a special class of observers / listeners that automatically set themselves aside when the node is deleted from the document, and re-instate themselves if the node is re-inserted in the document? Similarly for when the window goes away. Instead of addObserver or addMessageListener, you'd have observeWhileInserted or listenWhileInserted. Implemented in some clever and efficient way to avoid thrashing during heavy DOM manipulation. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Memory management in features implemented in JS
On 03/19/2014 04:39 PM, Kyle Huey wrote: Short of not implementing things in JS, what ideas do people have for fixing these issues? We have some ideas of how to add helpers to scope these things to the lifetime of the window (perhaps by adding an API that returns a promise that is resolved at inner-window-destroyed to provide a good cleanup hook that is not global) but that doesn't help with objects intended to have shorter lifetimes. Is it possible for us to implement some sort of useful weak reference in JS? The general principle of GC is that an object is removed when doing so could have no visible effect. What if these DOM nodes could use a special class of observers / listeners that automatically set themselves aside when the node is deleted from the document, and re-instate themselves if the node is re-inserted in the document? Similarly for when the window goes away. Then your behavior would be well-defined regardless of when GC happens or doesn't happen. People always want the GC to help them sort out these sorts of problems; but a good GC is tightly tuned for its workload, and trying to adapt it to serve similar purposes is generally not a path to happiness. It's better to identify the points at which an observer becomes useless --- hence my suggestion that element insertion and removal be the trigger for the corresponding observers / listeners. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On Fri, Mar 21, 2014 at 2:00 PM, Nicholas Nethercote wrote: >> >> What is the per-compartment overhead? > > It's hard to pin an exact number on it. It depends on exactly the > mixture of GC things allocated by the compartment. But in my current > session, the smallest two system compartments take up only 4,784 > bytes. Another three are 9,568 bytes. I should clarify: that's the memory consumption that's identified as belonging to the compartment, but that's not the whole picture. Some compartment data gets reported at the level of the runtime, including 'script-sources' and 'script-data'. Our increase for these over the past year is 1.5 MiB (see https://bugzilla.mozilla.org/show_bug.cgi?id=986323#c1). That's purely due to loading more code, and is unrelated to compartment overhead. Lazy loading will help there. Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
- Original Message - > From: "Nicholas Nethercote" > To: "Benjamin Smedberg" > Cc: "dev-platform" > Sent: Friday, March 21, 2014 2:00:01 PM > Subject: Re: Too many system compartments at start-up > > Marco identified a number of modules that could be lazily loaded, and > filed several bugs that block > ttps://bugzilla.mozilla.org/show_bug.cgi?id=986323. Hopefully others > can do likewise; this isn't something that a single person will be > able to do effectively, since it spans such a wide fraction of the > product. It would take some work, but it would be nice if modules could also be unloaded. Modules could call a function to suggest that it's okay to unload them. Later they could cancel that action. The module loader could periodically unload functions that signaled they were ready to be unloaded more than a minute ago without a later cancellation. -Bill ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On Fri, Mar 21, 2014 at 6:31 AM, Benjamin Smedberg wrote: > > What is the per-compartment overhead? It's hard to pin an exact number on it. It depends on exactly the mixture of GC things allocated by the compartment. But in my current session, the smallest two system compartments take up only 4,784 bytes. Another three are 9,568 bytes. They go up from there. Zones definitely helped. And most of the system compartments are in the same zone. But both CPG and zones have been around for a while, and start-up memory is still growing, because system compartments keep being added, and existing ones keep getting bigger. That's why my email focused mostly on lazy loading -- that seems like the easiest way to improve things. *And* it will help with start-up time. It's a no-brainer, but we're not doing enough of it. Marco identified a number of modules that could be lazily loaded, and filed several bugs that block ttps://bugzilla.mozilla.org/show_bug.cgi?id=986323. Hopefully others can do likewise; this isn't something that a single person will be able to do effectively, since it spans such a wide fraction of the product. Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On Fri, Mar 21, 2014 at 2:45 PM, Andrew McCreight wrote: > Looking at a recent AWSY run, in StartSettled, I see three non-window > zones. > > One is about 22mb and contains all the stuff you'd expect, browser.xul, > various .jsm files etc. > > The second is is 2.8mb, almost entirely strings. There's one string that > is about 10,000 long, but the rest are non-notable. That seems weird. > This is probably the atoms zone. But I'm curious why we have such a long atom in there. > The final one is 342kb, has the null principal, and seems to mostly be > random shapes and objects. I don't know what that is. > This is probably the self-hosting zone. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Running b2g reftests on mozilla-inbound on EC2 instances
We're now running b2g reftests on EC2 on every trunk branch side-by-side with the Fedora minis: https://tbpl.mozilla.org/?jobname=b2g_emulator.*reftest They're running around 30-40% slower. We are looking at enabling them on faster EC2 instances so we can disable the minis. regards, Armen On 14-03-14 11:14 AM, Armen Zambrano G. wrote: > Hello, > We're trying to move the b2g reftests from the old Mac minis to run on > the EC2 instances. > > We're going to run the jobs side-by-side on mozilla-inbound for few days > to see how it behaves with more load [1]. > > If you see the jobs misbehaving please hide them and let us know in bug > 818968. > > The jobs have been running green on Elm (R5 has been recently fixed). > > cheers, > Armen > > [1] > https://tbpl.mozilla.org/?tree=Mozilla-Inbound&jobname=b2g_emulator_vm.*elm%20opt%20test%20reftest&showall=1 > [2] https://bugzilla.mozilla.org/show_bug.cgi?id=818968 > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Running b2g reftests on mozilla-inbound on EC2 instances
We're now running b2g reftests on EC2 on every trunk branch side-by-side with the Fedora minis: https://tbpl.mozilla.org/?jobname=b2g_emulator.*reftest They're running around 30-40% slower. We are looking at enabling them on faster EC2 instances so we can disable the minis. regards, Armen On 14-03-14 11:14 AM, Armen Zambrano G. wrote: > Hello, > We're trying to move the b2g reftests from the old Mac minis to run on > the EC2 instances. > > We're going to run the jobs side-by-side on mozilla-inbound for few days > to see how it behaves with more load [1]. > > If you see the jobs misbehaving please hide them and let us know in bug > 818968. > > The jobs have been running green on Elm (R5 has been recently fixed). > > cheers, > Armen > > [1] > https://tbpl.mozilla.org/?tree=Mozilla-Inbound&jobname=b2g_emulator_vm.*elm%20opt%20test%20reftest&showall=1 > [2] https://bugzilla.mozilla.org/show_bug.cgi?id=818968 > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
Looking at a recent AWSY run, in StartSettled, I see three non-window zones. One is about 22mb and contains all the stuff you'd expect, browser.xul, various .jsm files etc. The second is is 2.8mb, almost entirely strings. There's one string that is about 10,000 long, but the rest are non-notable. That seems weird. The final one is 342kb, has the null principal, and seems to mostly be random shapes and objects. I don't know what that is. So it does look like all of the system-y stuff is in a single zone. Those other two zones are probably worthy of some investigation, though... Andrew - Original Message - > On 3/21/14 1:09 PM, Bobby Holley wrote: > > For JSMs and stuff, they should all go into the system zone. > > I realize that, emphasis on "should". Have we verified that this is > happening? > > -Boris > ___ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On Fri, Mar 21, 2014 at 2:23 PM, Boris Zbarsky wrote: > On 3/21/14 1:09 PM, Bobby Holley wrote: > >> For JSMs and stuff, they should all go into the system zone. >> > > I realize that, emphasis on "should". Have we verified that this is > happening? Maybe bill did, at some point? This is pretty hard to get wrong. See: http://mxr.mozilla.org/mozilla-central/source/js/xpconnect/loader/mozJSComponentLoader.cpp#656 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On 03/21/2014 12:06 PM, Bill McCloskey wrote:
The problem with doing measurements is that the per-compartment overhead is
very dependent on what's going on in the compartment. I tried to enable the B2G
compartment merging stuff in desktop Firefox to get a sense of how much of a
change there would be [1]. It's hard to tell if it actually saved anything. It
broke a lot of stuff, and I had a hard time getting consistent measurements of
memory use at startup since the numbers really fluctuate wildly. My best guess
is that explicit went from 90MB currently to 70MB with merging. That's a lot,
but keep in mind that a bunch of exceptions are thrown all over the place, and
we might just be failing to load some important stuff. Hopefully someone can
follow up on this.
There may indeed be a lot of breakage. See https://bugzil.la/980752
about the the reuseGlobal=true situation with the addon-sdk loader
Object.freeze()-ing Object.prototype. (Which is bad for idioms like
Foo.prototype.toString = function() {}.)
Andrew
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On 3/21/14 1:09 PM, Bobby Holley wrote: For JSMs and stuff, they should all go into the system zone. I realize that, emphasis on "should". Have we verified that this is happening? -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On Fri, Mar 21, 2014 at 1:37 PM, Boris Zbarsky wrote: > On 3/21/14 11:02 AM, Bobby Holley wrote: > >> I'm also curious to hear about the overhead of compartments now that we >> have zones. >> > > An important question: which of these system compartments actually share a > zone with each other? I know the goal is "all of them", but are we > actually getting there? > For JSMs and stuff, they should all go into the system zone. For window globals, they go into the zone of GetTop() if it exists, and get a new zone otherwise. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Should nsIPrefBranch.set*Pref return NS_ERROR_UNEXPECTED on type mismatch?
On Fri, Mar 21, 2014 at 7:40 AM, Irving Reid wrote: > extensions.blocklist.pingCountVersion (146 times out of ~1.5 million Nightly > telemetry sessions) and extensions.shownSelectionUI (8 times in 1.5m) > The prefs in question aren't likely targets for malware, though they could > be collateral damage. They don't have default values in all.js. Doesn't corrupting them allow you to prevent the blocklist from working, and preventing the "choose which extensions to install" dialog from appearing on startup? Those sound like things malware would want to do. Gavin ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On 3/21/14, 9:06 AM, Bill McCloskey wrote: - Original Message - From: "Bobby Holley" To: "Benjamin Smedberg" Cc: "dev-platform" , "Nicholas Nethercote" Sent: Friday, March 21, 2014 8:02:58 AM Subject: Re: Too many system compartments at start-up I'm also curious to hear about the overhead of compartments now that we have zones. I was under the impression that most of it was CCW overhead, which shouldn't really be an issue in the startup case, since we should have very few wrappers. As far as I know, we haven't done any measurements of compartment overhead since zones landed (bug 759585). The whole point of zones was that people shouldn't have to merge compartments just to save memory, so I hope people don't start doing that without any data. All of the measurements in bug 986323 simply say that we're loading a lot more code now than we were before. No amount of merging compartments will fix that. The problem with doing measurements is that the per-compartment overhead is very dependent on what's going on in the compartment. I tried to enable the B2G compartment merging stuff in desktop Firefox to get a sense of how much of a change there would be [1]. It's hard to tell if it actually saved anything. It broke a lot of stuff, and I had a hard time getting consistent measurements of memory use at startup since the numbers really fluctuate wildly. My best guess is that explicit went from 90MB currently to 70MB with merging. That's a lot, but keep in mind that a bunch of exceptions are thrown all over the place, and we might just be failing to load some important stuff. Hopefully someone can follow up on this. Great reply Bill! I'm interpreting your response as "the number of compartments isn't a problem: the total volume is." In other words, our incentive is to load as little code/JavaScript as possible at all times. (That should go without saying.) Or, ignore many of the responses earlier in this thread :) Let's talk about ways to make that happen. I'll start with a few ideas: 1) Require all new JS features to have a pref that can disable loading of extra modules/code. That way, we can disable prefs of features and measure the overhead of each feature. I would love a dashboard of per-feature overhead, especially one with a time axis so I can identify regressions over time. 2) Change how JavaScript "services" work so the js/jsm that is loaded as part of XPCOM startup is as minimal as possible and performs as few immediate imports as possible. This minimizes the import cost during XPCOM initialization. 3) Don't initialize a JS-based XPCOM service until it is actually accessed or needed. For optional browser services (like bookmarks), no JSMs get imported until someone makes an API call that accesses the service. Perhaps JS-based service objects/interfaces should be exposed as promises? 4) Explore a proper service management framework for initializing components, dependencies and all. This has been talked about before and is arguably long overdue. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On 3/21/14, 9:36 AM, Bill McCloskey wrote: - Original Message - From: "Gregory Szorc" To: "Nicholas Nethercote" Cc: "dev-platform" Sent: Friday, March 21, 2014 9:27:34 AM Subject: Re: Too many system compartments at start-up Also, the bug I was referring to is bug 807205. It was marked WONTFIX because gc now happens in zones rather than globals. But that bug was concerned mainly with the overhead of cross-compartment string copying, not compartment overhead. Strings are shared between compartments of the same zone. No copying happens. *Now*. My bug was marked WONTFIX after 759585 (which prevented compartment copying) landed. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On 3/21/14 11:02 AM, Bobby Holley wrote: I'm also curious to hear about the overhead of compartments now that we have zones. An important question: which of these system compartments actually share a zone with each other? I know the goal is "all of them", but are we actually getting there? -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
- Original Message - > From: "Gregory Szorc" > To: "Nicholas Nethercote" > Cc: "dev-platform" > Sent: Friday, March 21, 2014 9:27:34 AM > Subject: Re: Too many system compartments at start-up > > Also, the bug I was referring to is bug 807205. It was marked WONTFIX > because gc now happens in zones rather than globals. But that bug was > concerned mainly with the overhead of cross-compartment string copying, > not compartment overhead. Strings are shared between compartments of the same zone. No copying happens. -Bill ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On 3/21/14, 1:37 AM, Nicholas Nethercote wrote: On Fri, Mar 21, 2014 at 12:16 AM, Gregory Szorc wrote: Could someone please provide an update on reducing compartment overhead? Are we at a point where things like services/healthreport/HealthReport.jsm should be considered a "necessary evil" rather than a "gross hack?" What other solutions besides concatenating files and lazy loading are there? Neither sound particularly appealing to me. I'd really like to be able to "pin" specific JSMs to shared compartments. Kind of a hybrid between CPG and the monolithic compartment that came before. I filed a bug on that a year ago. Can't find it ATM... I am unaware of any compelling stories to be told about reducing compartment overhead at this time. Generational GC might help, though I'm not sure. If compartment overhead was zero, we wouldn't have to worry as much about this issue, but we live in an imperfect world and have to work within constraints presented to us. That's why I'm talking mostly about lazy loading -- it's a solution that potentially gives us decent wins without too much trouble. Firefox is a large program, and most users utilize only a fraction of its features. Lazy loading parts that many users don't require potentially helps memory consumption and start-up time. I also don't see HealthReport.jsm as a problem. For one, it's not even on the list. Furthermore, it seems like something that is needed right from start-up, unlike many other compartments on that list. If it's not in the list, then you aren't measuring properly. Firefox Health Report - like a number of JS features - delays its main module loading until a few seconds after profile-do-change. Components do this to prevent churn during startup. May I suggest changing your measurement technique to sample at least 60s after browser startup? Also, the bug I was referring to is bug 807205. It was marked WONTFIX because gc now happens in zones rather than globals. But that bug was concerned mainly with the overhead of cross-compartment string copying, not compartment overhead. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
- Original Message - > From: "Bobby Holley" > To: "Benjamin Smedberg" > Cc: "dev-platform" , "Nicholas Nethercote" > > Sent: Friday, March 21, 2014 8:02:58 AM > Subject: Re: Too many system compartments at start-up > > I'm also curious to hear about the overhead of compartments now that we > have zones. I was under the impression that most of it was CCW overhead, > which shouldn't really be an issue in the startup case, since we should > have very few wrappers. As far as I know, we haven't done any measurements of compartment overhead since zones landed (bug 759585). The whole point of zones was that people shouldn't have to merge compartments just to save memory, so I hope people don't start doing that without any data. All of the measurements in bug 986323 simply say that we're loading a lot more code now than we were before. No amount of merging compartments will fix that. The problem with doing measurements is that the per-compartment overhead is very dependent on what's going on in the compartment. I tried to enable the B2G compartment merging stuff in desktop Firefox to get a sense of how much of a change there would be [1]. It's hard to tell if it actually saved anything. It broke a lot of stuff, and I had a hard time getting consistent measurements of memory use at startup since the numbers really fluctuate wildly. My best guess is that explicit went from 90MB currently to 70MB with merging. That's a lot, but keep in mind that a bunch of exceptions are thrown all over the place, and we might just be failing to load some important stuff. Hopefully someone can follow up on this. -Bill [1] You have to make this code run unconditionally: http://mxr.mozilla.org/mozilla-central/source/js/xpconnect/loader/mozJSComponentLoader.cpp#354 The jsloader.reuseGlobal preference doesn't work because we haven't read in prefs yet. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to ship: CSS.escape
On Thu, 20 Mar 2014 17:11:01 +0100, Anne van Kesteren
wrote:
On Thu, Mar 20, 2014 at 3:17 PM, Boris Zbarsky wrote:
http://dev.w3.org/csswg/cssom/#the-css.escape%28%29-method which allows
web
pages to create a valid CSS identifier that will parse to a given
string. A
typical use case is:
document.querySelector('#' + CSS.escape(stringIDontControl))
Unless there are objections, I'm going to check this in without a
preference, for Firefox 31. The spec for this is very straightforward
and
not likely to change, apart from the feature disappearing altogether or
something. But I think this feature is pretty much necessary in some
form,
whether built into the platform or as a library, to safely use
querySelector-like APIs with non-literal strings.
Bug report: https://bugzilla.mozilla.org/show_bug.cgi?id=955860
Please let me know if there are objections.
Seems fine, specification should probably clarify surrogate handling.
I would expect a paired surrogate in JavaScript to end up as a single
escape.
A paired surrogate is left unchanged by CSS.escape().
--
Simon Pieters
Opera Software
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On Fri, Mar 21, 2014 at 10:31 AM, Benjamin Smedberg wrote: > On 3/21/2014 2:46 AM, Nicholas Nethercote wrote: > >> Hi, >> >> At start-up, with a new profile, Firefox creates more than 230 system >> compartments. This is about 90 more than a year ago, and it's part of >> the reason why Firefox uses almost twice as much physical memory at >> start-up than it did two years ago. (Have a look at >> https://areweslimyet.com/ for the measurements.) >> > > What is the per-compartment overhead? > I'm also curious to hear about the overhead of compartments now that we have zones. I was under the impression that most of it was CCW overhead, which shouldn't really be an issue in the startup case, since we should have very few wrappers. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On 21/03/2014 15:37, Benjamin Smedberg wrote: On 3/21/2014 10:34 AM, Marco Bonardo wrote: Or directly make Cu.import act like defineLazyModuleGetter... Would that be possible? I don't think that's a good idea. It's not an uncommon pattern to try/catch around a module import in case it's not present in a particular configuration or the expected symbols aren't exported, especially when writing addons that cross versions. I see, there's no way to touch Cu.import, though we could introduce Cu.lazyImport, and Cu.moduleExists and start deprecating Cu.import usage. It may stay for compatibility reasons, while we convert the codebase. Relegating it to add-ons (considered we may also reach add-on authors through amo editors/tools) would avoid increases like the one Nicholas is pointing out. (I'm just throwing out ideas here) -m ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Should nsIPrefBranch.set*Pref return NS_ERROR_UNEXPECTED on type mismatch?
On 2014-03-20 5:38 PM, Gavin Sharp wrote: On Thu, Mar 20, 2014 at 10:38 AM, Irving Reid wrote: For unknown reasons, internal bookkeeping prefs used by AddonManager and XPIProvider are set to values of the wrong type on some Firefox profiles, and are now stuck that way. I can write wrapper code on these calls to catch the error and clear the broken preference, but I wonder if it would be better to change set*Pref() to force the preference to the intended type. Which prefs, out of curiosity? extensions.blocklist.pingCountVersion (146 times out of ~1.5 million Nightly telemetry sessions) and extensions.shownSelectionUI (8 times in 1.5m) I could imagine some forms of malware installers clobbering prefs.js values (or even firefox.js/all.js values, if they have appdir write access) incorrectly. If that's what's happening here it should inform our ways of mitigating it. Any way to get more data from the affected installations? The prefs in question aren't likely targets for malware, though they could be collateral damage. They don't have default values in all.js. We could pull out the full telemetry details for sessions that see these failures and look for other signals, or we could add more telemetry probes. I'm not sure the problem is big enough to make those worthwhile. The exception reporting that I added in bug 952543, which revealed this issue, just rolled into Beta 29. I'll check the Beta telemetry data next week to see if it's any different from Nightly. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On 3/21/2014 10:34 AM, Marco Bonardo wrote: Or directly make Cu.import act like defineLazyModuleGetter... Would that be possible? I don't think that's a good idea. It's not an uncommon pattern to try/catch around a module import in case it's not present in a particular configuration or the expected symbols aren't exported, especially when writing addons that cross versions. --BDS ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
I started filing some low-hanging-fruit dependencies, though, looks like one of the main culprit is modules that are Cu.import-ed instead of defineLazyModuleGetter-ed. I wonder if we could somehow instrument Cu.import so that if a module is not used before N seconds (or M ticks?) from when it was imported it logs a warning to the console. Or directly make Cu.import act like defineLazyModuleGetter... Would that be possible? On 21/03/2014 07:46, Nicholas Nethercote wrote: > Hi, > > At start-up, with a new profile, Firefox creates more than 230 system > compartments. This is about 90 more than a year ago, and it's part of > the reason why Firefox uses almost twice as much physical memory at > start-up than it did two years ago. (Have a look at > https://areweslimyet.com/ for the measurements.) > > At the bottom of this email I have listed the system compartments. > (You can generate this list yourself from the > 'js-main-runtime-compartments' tree in about:memory.) Please look for > any there that you know could be created lazily. For example, there > are 16 that have "devtools" in the name, which sound unnecessary. > > If you find some, please file a bug that blocks > https://bugzilla.mozilla.org/show_bug.cgi?id=986323. > > Also, while modularization is admirable, compartments aren't free. So > if you see any compartments that could be merged easily, please > consider doing that as well. > > Thanks. > > Nick > ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Should nsIPrefBranch.set*Pref return NS_ERROR_UNEXPECTED on type mismatch?
On 2014-03-20 5:03 PM, Neil wrote: Irving Reid wrote: For unknown reasons, internal bookkeeping prefs used by AddonManager and XPIProvider are set to values of the wrong type on some Firefox profiles, and are now stuck that way. I can write wrapper code on these calls to catch the error and clear the broken preference, but I wonder if it would be better to change set*Pref() to force the preference to the intended type. a) if the pref has a default in all.js then you don't want to be able to change its type and in this case the prefs.js value should get ignored if it is the wrong type. Yes, this is what happens - if we've read a preference from any of the default files, like all.js, a value of the wrong type in the profile's prefs.js is discarded. b) by the time you're calling setPref it's too late because you know the type is wrong when your getPref failed so you might as well clear it as part of fixing the problem. Mostly yes - in one of my cases the code is setting the pref without reading it first. An intermediate approach would be to check PREF_HAS_DEFAULT - if we have a default value for the preference, the type ought to be correct. If the pref doesn't have a default value, it's likely a record keeping pref created on the fly by our (or extension) code so it's probably OK for the setter to change the type. - irving - ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
Nicholas Nethercote schrieb: Hi, At start-up, with a new profile, Firefox creates more than 230 system compartments. This is about 90 more than a year ago, and it's part of the reason why Firefox uses almost twice as much physical memory at start-up than it did two years ago. Hrm, reading your list sounds like the main culprits are modules and also bindings that all get their own compartment. I somewhat hazily seem to remember that we had an issue like that on FxOS as well and some kind of hack to merge modules into a single compartment was done, and the real solution being called out back then as doing "zones" or something where all those system things wouldn't get actual separate compartments? Did that not work out? It sounds bad that using modules and bindings would have a large overhead. :( KaiRo ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On 3/21/2014 2:46 AM, Nicholas Nethercote wrote: Hi, At start-up, with a new profile, Firefox creates more than 230 system compartments. This is about 90 more than a year ago, and it's part of the reason why Firefox uses almost twice as much physical memory at start-up than it did two years ago. (Have a look at https://areweslimyet.com/ for the measurements.) What is the per-compartment overhead? It seems like we have two possible solutions in general: * stop doing so much at startup/make sure feature code is only loaded when the feature is used * combine things that are currently in multiple compartments together into one compartment. Perhaps some manual combination of .jsms into a smaller number of compartments similar to what B2G does? I recently filed bug 982623 which might be able to help somewhat. Currently many frontend features are primarily written as a .jsm, but have a separate XPCOM component so that they can register for startup/profile/update-timer notifications. With that patch we should at least be able to get rid of the separate XPCOM component and only use the .jsm. I suspect that you'll have more luck if you direct questions or recommendations to specific module owners/teams based on this list. --BDS ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
If my memory serves, B2G has its own implementation of Cu.import that doesn't create components. Would it be possible to expose this implementation to Firefox so that we could use it for some modules that are designed to be always be imported together? I would definitely use this for OS.File. Similarly, I assume that a variant of Jetpack/devtools' require() that doesn't create components would be useful for the 16 devtools compartments. Cheers, David On 3/21/14 7:46 AM, Nicholas Nethercote wrote: > Hi, > > At start-up, with a new profile, Firefox creates more than 230 system > compartments. This is about 90 more than a year ago, and it's part of > the reason why Firefox uses almost twice as much physical memory at > start-up than it did two years ago. (Have a look at > https://areweslimyet.com/ for the measurements.) > > At the bottom of this email I have listed the system compartments. > (You can generate this list yourself from the > 'js-main-runtime-compartments' tree in about:memory.) Please look for > any there that you know could be created lazily. For example, there > are 16 that have "devtools" in the name, which sound unnecessary. > > If you find some, please file a bug that blocks > https://bugzilla.mozilla.org/show_bug.cgi?id=986323. > > Also, while modularization is admirable, compartments aren't free. So > if you see any compartments that could be merged easily, please > consider doing that as well. > > Thanks. > > Nick -- David Rajchenbach-Teller, PhD Performance Team, Mozilla ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
On Fri, Mar 21, 2014 at 12:16 AM, Gregory Szorc wrote: > Could someone please provide an update on reducing compartment overhead? > > Are we at a point where things like services/healthreport/HealthReport.jsm > should be considered a "necessary evil" rather than a "gross hack?" > > What other solutions besides concatenating files and lazy loading are there? > Neither sound particularly appealing to me. I'd really like to be able to > "pin" specific JSMs to shared compartments. Kind of a hybrid between CPG and > the monolithic compartment that came before. I filed a bug on that a year > ago. Can't find it ATM... I am unaware of any compelling stories to be told about reducing compartment overhead at this time. Generational GC might help, though I'm not sure. If compartment overhead was zero, we wouldn't have to worry as much about this issue, but we live in an imperfect world and have to work within constraints presented to us. That's why I'm talking mostly about lazy loading -- it's a solution that potentially gives us decent wins without too much trouble. Firefox is a large program, and most users utilize only a fraction of its features. Lazy loading parts that many users don't require potentially helps memory consumption and start-up time. I also don't see HealthReport.jsm as a problem. For one, it's not even on the list. Furthermore, it seems like something that is needed right from start-up, unlike many other compartments on that list. Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Intent to ship: CSS.escape
On 3/20/14 12:11 PM, Anne van Kesteren wrote: Seems fine, specification should probably clarify surrogate handling. I would expect a paired surrogate in JavaScript to end up as a single escape. Non-ASCII stuff is OK in CSS idents, so it doesn't need escaping at all: it's just left as-is. -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Too many system compartments at start-up
Could someone please provide an update on reducing compartment overhead? Are we at a point where things like services/healthreport/HealthReport.jsm should be considered a "necessary evil" rather than a "gross hack?" What other solutions besides concatenating files and lazy loading are there? Neither sound particularly appealing to me. I'd really like to be able to "pin" specific JSMs to shared compartments. Kind of a hybrid between CPG and the monolithic compartment that came before. I filed a bug on that a year ago. Can't find it ATM... > On Mar 20, 2014, at 23:47, Nicholas Nethercote wrote: > > Hi, > > At start-up, with a new profile, Firefox creates more than 230 system > compartments. This is about 90 more than a year ago, and it's part of > the reason why Firefox uses almost twice as much physical memory at > start-up than it did two years ago. (Have a look at > https://areweslimyet.com/ for the measurements.) > > At the bottom of this email I have listed the system compartments. > (You can generate this list yourself from the > 'js-main-runtime-compartments' tree in about:memory.) Please look for > any there that you know could be created lazily. For example, there > are 16 that have "devtools" in the name, which sound unnecessary. > > If you find some, please file a bug that blocks > https://bugzilla.mozilla.org/show_bug.cgi?id=986323. > > Also, while modularization is admirable, compartments aren't free. So > if you see any compartments that could be merged easily, please > consider doing that as well. > > Thanks. > > Nick > > > atoms > null-principal > [System Principal], about:blank [2] > [System Principal], about:newtab [2] > [System Principal], chrome://browser/content/baseMenuOverlay.xul > [System Principal], chrome://browser/content/browser.xul > [System Principal], chrome://browser/content/customizableui/panelUI.xml > [System Principal], chrome://browser/content/customizableui/toolbar.xml > [System Principal], chrome://browser/content/downloads/download.xml > [System Principal], chrome://browser/content/places/menu.xml > [System Principal], chrome://browser/content/places/placesOverlay.xul > [System Principal], chrome://browser/content/search/search.xml > [System Principal], chrome://browser/content/socialchat.xml > [System Principal], chrome://browser/content/tabbrowser.xml > [System Principal], chrome://browser/content/urlbarBindings.xml > [System Principal], chrome://global/content/bindings/autocomplete.xml > [System Principal], chrome://global/content/bindings/browser.xml > [System Principal], chrome://global/content/bindings/button.xml > [System Principal], chrome://global/content/bindings/general.xml > [System Principal], chrome://global/content/bindings/menu.xml > [System Principal], chrome://global/content/bindings/notification.xml > [System Principal], chrome://global/content/bindings/popup.xml > [System Principal], chrome://global/content/bindings/scrollbar.xml > [System Principal], chrome://global/content/bindings/scrollbox.xml > [System Principal], chrome://global/content/bindings/splitter.xml > [System Principal], chrome://global/content/bindings/stringbundle.xml > [System Principal], chrome://global/content/bindings/tabbox.xml > [System Principal], chrome://global/content/bindings/textbox.xml > [System Principal], chrome://global/content/bindings/text.xml > [System Principal], chrome://global/content/bindings/toolbarbutton.xml > [System Principal], chrome://global/content/bindings/toolbar.xml > [System Principal], chrome://global/content/editMenuOverlay.xul > [System Principal], chrome://global/content/platformHTMLBindings.xml > [System Principal], > data:application/vnd.mozilla.xul+xml;charset=utf-8, > [System Principal], > file:///home/njn/moz/miN/co64/dist/bin/browser/components/devtools-clhandler.js > [System Principal], > file:///home/njn/moz/miN/co64/dist/bin/browser/components/DownloadsStartup.js > [System Principal], > file:///home/njn/moz/miN/co64/dist/bin/browser/components/ExperimentsService.js > [System Principal], > file:///home/njn/moz/miN/co64/dist/bin/browser/components/nsBrowserContentHandler.js > [System Principal], > file:///home/njn/moz/miN/co64/dist/bin/browser/components/nsBrowserGlue.js > [System Principal], > file:///home/njn/moz/miN/co64/dist/bin/browser/components/nsSessionStartup.js > [System Principal], > file:///home/njn/moz/miN/co64/dist/bin/browser/components/nsSetDefaultBrowser.js > [System Principal], > file:///home/njn/moz/miN/co64/dist/bin/browser/components/WebContentConverter.js > [System Principal], > file:///home/njn/moz/miN/co64/dist/bin/components/addonManager.js > [System Principal], > file:///home/njn/moz/miN/co64/dist/bin/components/BrowserElementParent.js > [System Principal], > file:///home/njn/moz/miN/co64/dist/bin/components/CrashService.js > [System Principal], > file:///home/njn/moz/miN/co64/dist/bin/components/DataReportingService.js > [System Principal], > file:///home/njn/moz/miN/co64/dist/bin/components/For
