Everyone talks about how discussions on IRC are opaque unless you're there for them, but nobody ever does anything about it. :-) Given Till and I just had a spectacularly successful (in my super-humble opinion) discussion of self-hosting ideas (some short-term fixing/slight hacks, some longer-term), it seemed like a good idea to record it in less ephemeral form than just squirreled away in IRC channel logs. So here it is. Context is mostly:
Bug 851763 - Significant increase in minimum runtime size with large self-hosted objects https://bugzilla.mozilla.org/show_bug.cgi?id=851763 which is most particularly motivated by Intl code having mega-huge i18n read-only data tables that are semi-pointlessly cloned amongst compartments. But I feel like we went rather beyond it by the end, and into making the self-hosting setup not be so ad-hoc as it is now. Some good fodder for short-term fixing, some more mid-termish ideas. Jeff P.S. -- Thanks to everyone else in channel for saying about ten other lines throughout the entire discussion, so I didn't have to do too much work disentangling this discussion from others occurring at the time. ;-) ===== [2013-03-27 14:14:31] <till> Waldo: ping [2013-03-27 14:14:41] <Waldo> till: pong [2013-03-27 14:14:53] <till> Waldo: regarding bug 851763 [2013-03-27 14:15:07] <till> Waldo: I see only two options, really [2013-03-27 14:15:52] <till> 1. do shallow cloning and use a custom resolver to lazily clone referenced members [2013-03-27 14:16:14] <till> 2. somehow enable sharing objects across compartments [2013-03-27 14:16:39] <till> 2 would obviously be much more desirable, but I'm not sure we can implement it [2013-03-27 14:17:28] <Waldo> some of the self-hosted data we care about sharing [2013-03-27 14:17:32] <Waldo> some of it we don't [2013-03-27 14:17:35] <till> Do you think there's any chance we might be able to move deeply-frozen object trees into shared read-only memory or something like that [2013-03-27 14:17:42] <till> sure [2013-03-27 14:18:40] <Waldo> what can we think of that would allow data to only optionally be shared, somehow [2013-03-27 14:18:45] <Waldo> s/$/?/ [2013-03-27 14:19:03] <till> Object.freeze? [2013-03-27 14:19:28] <Waldo> I'm not sure the right thing to do is think about language-level primitives here [2013-03-27 14:19:46] <till> probably true, yes [2013-03-27 14:20:03] <Waldo> rather about intrinsics and such that could be written to use any internal mechanism we want, maybe [2013-03-27 14:20:19] <till> I'm trying to come up with something that'd be applicable beyond self-hosting, but yeah: much harder [2013-03-27 14:21:05] <Waldo> I think we restrict specifically to self-hosting for now, and if something comes up we can expand however [2013-03-27 14:21:46] <till> so maybe we could mark shareable objects as such and construct special CCWs that allow access to those from all compartments? [2013-03-27 14:22:28] <Waldo> that seems maybe possible -- something wrapperish [2013-03-27 14:22:45] <Waldo> and then the clone system would recognize that and produce a cross-compartment wrapper or such [2013-03-27 14:22:54] <till> yeah, that's what I mean [2013-03-27 14:23:04] <Waldo> there is a question of perf, to be sure -- as bz_away loves to remind me, proxies are bad for perf [2013-03-27 14:23:28] <Waldo> that said, for this big data stuff, it's unclear how much we'd actually be querying it [2013-03-27 14:23:39] <Waldo> and we could maybe minimize the time it takes to perform those queries somehow [2013-03-27 14:24:11] <till> for the Intl stuff, it probably doesn't matter at all. But yeah: for other things performance might be an issue [2013-03-27 14:24:17] <Waldo> and maybe we'd just mark *helper methods* as safe for sharing, ish [2013-03-27 14:24:44] <till> Waldo: ah, you mean we don't ever access the data itself from a client compartment? [2013-03-27 14:25:06] <Waldo> till: possibly [2013-03-27 14:25:16] <till> makes sense [2013-03-27 14:25:27] <till> I should talk to bholley, I guess? [2013-03-27 14:26:09] <Waldo> till: I'm not sure we need to rope him in, actually -- this seems all reusing existing JS engine concepts to me [2013-03-27 14:27:37] <till> Waldo: mmh, now that I think about it, I realize that Gecko or the JS engine has something like that. Using Components.utils.Sandbox, one can create a new global that can seamlessly access objects from other globals with the same principal. [2013-03-27 14:28:14] <till> Doing whatever that does with the correct principals configuration should do what we need, I guess [2013-03-27 14:29:20] <Waldo> var intlFunctions = (function() { var sharedIntlData = { big table }; function intlIntrinsicOne() { stuff } function intlIntrinsicTwo() { stuff } return { intlIntrinsicOne: markAsSingleton(intlIntrinsicOne), intlIntrinsicTwo: markAsSingleton(intlIntrinsicTwo) }; }; })(); Object.keys(intlFunctions).map(function(key) { global[key] = intlFunctions[key]; }); [2013-03-27 14:29:34] <Waldo> till: ^ is very vaguely what I had in mind [2013-03-27 14:29:51] <Waldo> till: with markAsSingleton doing something to indicate a wrapper should be created [2013-03-27 14:30:03] <till> Waldo: let me quickly pretty print that ;) [2013-03-27 14:30:12] <Waldo> word :-) [2013-03-27 14:30:29] <Waldo> also correct any thinkos I may have made ;-) [2013-03-27 14:33:22] <Waldo> till: now you have me thinking a little [2013-03-27 14:33:32] <till> Waldo: why not just mark the functions we don't want cloned with markAsSingleton, without the entire intlFunctions and Object.keys thing? [2013-03-27 14:33:47] <Waldo> till: what if we made self-hosted code more pythonic, almost, and had it declare its exports somehow? [2013-03-27 14:33:57] <till> ah [2013-03-27 14:34:15] <till> so only exports get cloned, the rest stays in the self-hosting compartment? [2013-03-27 14:34:15] <Waldo> till: and then only the contents of EXPORTS, say, were actually not wrapped, or something? [2013-03-27 14:34:24] <till> I like it! [2013-03-27 14:34:25] <Waldo> yes, something like that [2013-03-27 14:34:38] <Waldo> there's a question how much extra work this creates, timing-wise [2013-03-27 14:34:47] <till> yeah [2013-03-27 14:34:58] <Waldo> but that aside, this seems like an actual clean idea [2013-03-27 14:35:20] <Waldo> s/actually not wrapped/actually not cloned/ [2013-03-27 14:36:16] <till> Now you broke it [2013-03-27 14:36:41] <till> EXPORTS should be cloned, not not cloned, no? [2013-03-27 14:36:47] <till> Anyway [2013-03-27 14:37:13] <till> Here's another approach, which is a refinement of something I had in mind for a while [2013-03-27 14:37:42] <Waldo> er [2013-03-27 14:37:57] <Waldo> yeah, you're right, actually not wrapped was correct :-) [2013-03-27 14:38:05] <till> :) [2013-03-27 14:38:40] <till> How about we initialize all builtins in the self-hosting global, as we do now, then, during interpretation of the self-hosting script itself, install the self-hosted methods on those builtins, and then use those objects as, well, prototypes we clone into all other compartments upon creation? [2013-03-27 14:38:51] <Waldo> the right terminology for the splitting is cloning versus wrapping -- if I keep that in mind firmly I won't get confused [2013-03-27 14:40:05] <till> Crucially, we'd only clone those methods installed on builtins, no others [2013-03-27 14:40:13] <Waldo> that could be interesting as well [2013-03-27 14:40:36] * Waldo tries to think about how to break that, or something [2013-03-27 14:41:15] <till> It might be desirable to explicitly clone some helper functions that should deal with per-compartment data. Seems doable, though. [2013-03-27 14:43:05] <Waldo> you'd definitely have some ordering issues as regards cloning Function/Object as one coherent whole, before anything else [2013-03-27 14:43:25] <Waldo> perhaps those two, at least the initial steps, should remain primordial, and implemented in C++ as now [2013-03-27 14:44:48] <Waldo> there's also the issue of setting all the original [[Prototype]] objects in the new global/compartment, so that things like [] always work right [2013-03-27 14:45:10] <till> true, yeah [2013-03-27 14:45:11] <Waldo> but that seems like just a hurdle to leap, not a fundamental issue [2013-03-27 14:45:43] <till> V8 has a neat thing that's somewhat related [2013-03-27 14:46:01] <Waldo> you've been stealing concepts from them? [2013-03-27 14:46:03] <Waldo> good man [2013-03-27 14:46:15] <till> :D [2013-03-27 14:46:27] <Waldo> WARNING: Great Artists at work [2013-03-27 14:47:07] <till> They somehow serialize an entire global after first creating it and then, instead of initializing new globals, they just deserialize the stored one and fix up some addresses. [2013-03-27 14:47:29] <Waldo> ah, yes, that -- saw that in d8 commandline options sometime [2013-03-27 14:48:18] <till> Seems like it'd be nice to have. Both for the self-hosting that might not need its initialization to run at each startup, anymore, and for all globals, in general. [2013-03-27 14:49:06] <Waldo> chunk'o'work, but something that may be doable without too much grunge [2013-03-27 14:49:49] <till> That's my impression, too [2013-03-27 14:50:33] <till> My impression is not, however, that we can realistically get any of these ideas working - and proven to be safe - in time for FF22. [2013-03-27 14:51:40] <Waldo> till: I think we could get the markAsSingleton-implies-wrap-not-clone bit working pretty quickly [2013-03-27 14:52:00] <Waldo> till: the rest of the ideas seem a bit beyond that, of course -- but not all ridiculously so [2013-03-27 14:52:28] <till> Waldo: ok, that's certainly the easiest one. I'll dig into it and see how far I get. [2013-03-27 14:52:40] <Waldo> excellent [2013-03-27 14:53:17] <Waldo> for the first time I feel like our self-hosting stuff is progressing beyond expedient pile'o'hacks and into clean-design territory :-D _______________________________________________ dev-tech-js-engine-internals mailing list [email protected] https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals
