Re: Namespaces as Sugar (was: complexity tax)
On May 27, 2008, at 12:18 PM, Mike Shaver wrote: 2008/5/27 Maciej Stachowiak [EMAIL PROTECTED]: It could save a lot of complexity, by not requiring any first-class support for namespace lookup on arbitrary objects. Is the expectation then that having two lookup models, one for global objects and the other for non-global objects, going to provide less complexity? My proposal is that there is one for lexical scope lookup along the scope chain (which considers open namespaces when binding unqualified names) and another for property lookup qualified to an object reference (which does not). Yes, I believe this will provide less complexity, because the scope chain and prototype chain algorithms are quite complex in the current proposal and not the same as each other. My rough proposal would greatly simplify the prototype chain algorithm and not make the scope chain algorithm any more complex (it may become simpler). In a browser, window is the global object; would property lookup on window be namespaced when referenced as such? When you have a handle to another window? When you use |this.prop| in a global function? My proposed answer to all of these would be no. If we have namespace-aware lookup, it seems to me that it would be less complex for implementors and script authors alike for it to always apply, rather than just for one magical object. I think it would be simpler to limit the effect of namespaces to lexical bindings (where we know we can bind to the right name and namespace statically in any case of interest) and not have them affect object property lookup in general. Regards, Maciej ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: Namespaces as Sugar (was: complexity tax)
On Wed, May 28, 2008 at 12:48 AM, Maciej Stachowiak [EMAIL PROTECTED] wrote: I want to say thanks for making this proposal (open namespace search only for lexical references). It leaves most of the use-cases I cited intact. Well done, good compromise (not complete evisceration of property qualifiers, or dismissal of unqualified import). While I like the idea, it doesn't address at least two of the use cases for namespaced properties: - Having both a fully typed and an untyped version of the same method on the object, allowing code to switch through just opening a namespace if the programmer knows their code is type consistent. - Allowing user classes or objects to present interfaces overriding behaviour inherited from Object which were internal and unexposeed in ES3. The first point is mainly a convenience feature. The second point can be addressed using new syntactic forms for each such case. (If using the form operator [no newlines here] ident, that would be a safe way to do it since all code looking like that cause a syntax error in ES3.) I don't know if that's a good or bad idea though. -- David liorean Andersson ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: Namespaces as Sugar (was: complexity tax)
On May 28, 2008, at 12:48 AM, Maciej Stachowiak wrote: On May 27, 2008, at 1:07 PM, Brendan Eich wrote: I forgot one of the main use-case for 4, one you've expressed interest in already: 'use namespace intrinsic'. Without (4), this does not allow early binding for s.charAt(i) given var s:string to string.prototype.intrinsic::charAt. Yeah, but you can't early bind this anyway if s lacks a type annotation (or you otherwise have inferred that s is of string type). In fact if you can't infer the type of s then you get worse performance than if you hadn't opened any namespaces. That depends on the implementation, but let's say it's true. You still get the better-typed fixtures (the string class's intrinsic::charAt method) instead of possibly overridden prototype properties. The 'use namespace intrinsic' design for early binding grew out of the compact profile of ES3, which is a dead letter on the web because it's overtly incompatible (the spec is short enough to read in a few minutes: http://www.ecma-international.org/publications/ standards/Ecma-327.htm). A pragma for prioritizing fixed methods with stricter type signatures was prototyped by the strict mode in AS3. During collaborative ES4 development in TC39 TG1, we synthesized aspects of these precursors into 'use namespace intrinsic'. This design is implemented in the ES4 RI, and it will probably be in a couple of practical open source implementations this year. We want to get user as well as implementor feedback. But you do have my number - I like the potential for early binding both for program understandability and performance. I would love to see a way to make it possible without at the same time making property lookup in the face of unknown types slower. I will think about it myself but perhaps someone else will come up with something clever. Great. In the mean time, we'll be working on clever solutions to speed up all property lookups, even with open namespaces. Maybe we'll convince you that cleverness is better applied by implementors to enable programmers who benefit from new things like cross-cutting intrinsic, debugging, version2, etc. namespaces. As for my motherhood apple pie lecture, it is based on experience with a working on a high-performance production-quality JavaScript implementation. We managed to pretty much rewrite the core interpreter for a significant performance boost in about two months, and that was struggling against the existing complexity of de facto JavaScript (ES3 + some Mozilla extensions). Along the way we found some correctness bugs that exist in pretty much all existing implementations (sequencing considerations in the face of exceptions for instance). If the language were much more complex, then it would be much harder to make architectural changes of the implementation. Two months. Sorry, but if it took you three or four months to do that and also include namespace optimizations, but the benefit to the great many JS or would-be ES4 programmers were enough, then why wouldn't you take the extra month or three? Given the needs of the many and the scale of the web, I continue to believe that the main argument should be about maximizing usable utility for programmers writing in the new version of the language -- only secondarily about hardships for implementors who make the big bucks making JS go fast :-/. /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: Namespaces as Sugar (was: complexity tax)
On 5/27/08 8:45 AM, Maciej Stachowiak wrote: Is the cost too high? I think that depends on how the name lookup algorithm works on real-world code. AS3 developers have data to share. Let's get into that. You looking at me? ;-) I don't have that data, but I'll gladly see if I can find some. I'd love to hear the data. AS3 developers, can unqualified lookup of object properties on untyped references in the presence of property namespaces be as fast as when there aren't namespaces at all? Seems unlikely since more work is being done. If so, how? The most obvious way to do property lookup when there is no static type info is a hashtable lookup on each prototype chain entry, but I do not see an obvious way that a single hashtable lookup can look in multiple namespaces. I suspect the answer to this in AS3 is that if you want performance, you have to use type declarations. Some of the complexity in the name resolution algorithms is to ensure that references to fixed properties cannot be shadowed. This means, with or without static type information, references to fixtures need only be resolved once. So the incremental cost that is incurred during unqualified name lookup is amortized across multiple evaluations of the reference in which it occurs. Other bits of complexity in those algorithms is there to disambiguate names in the case of conflicts. It would be interesting to study how often this code gets invoked in practice. It might not come into play all that often. Jd ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: Namespaces as Sugar (was: complexity tax)
On Tue, May 27, 2008 at 07:32, Mark S. Miller [EMAIL PROTECTED] wrote: Yes, agreed. Namespaces-as-Sugar (hereafter NAS) is too big for ES3.1. Perhaps it's too small for ES4. Again, it is simply my attempt to explain what I meant by A language with some of ES4's syntactic conveniences but without ES4's semantics. I would be less happy with NAS added to a language than with nothing added -- the problem it addresses is a problem more in theory than in practice. The real problem in practice is module linkage. Dojo and YUI show that this is already adequately solved with patterns and libraries, with no new language mechanism needed. This is the second time I hear you say that the namespace pattern used by dojo and YUI is already adequate. Let me debunk that myth. Dojo flattened their namespaces from things like dojo.foo.bar.baz to dojo.baz since the ES3 namespace pattern is too vefrbose and painful to use. YUI people also complain that writing YAHOO.foo is painful (all caps is hard to type). Unqualified import of global objects is a must for programming at large. I agree with Maciej here, if we can simplify the name lookup significantly be removing unqualified import of property names then I think we should do that. That being said, having namespaced properties seems useful and I'd rather have that if it can be made to perform well as well as be made simpler to understand. -- erik ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: Namespaces as Sugar (was: complexity tax)
On May 27, 2008, at 8:45 AM, Maciej Stachowiak wrote: I don't see how any of this argues for namespacing of properties on non-global objects, or why that case in particular requires unqualified import. The topic was any single object, so if these are good for the global object, they may (not must) be good for other objects. That's the general (uniformity) argument, made further below (I'm hoping for a response). One can certainly question the need for any namespacing of object properties, let alone unqualified import of namespaces for such purposes. I hope so. I'd rather have someone question unqualified import than implicitly dismiss it, which is what seems to be happening. I don't see how your statement is responsive. Hey, I was agreeing with you (Mr. Assistant District Attorney Sir :- P). Questioning is fine. What was responsive about your non-sequitur One can certainly question the need ... anyway? No one objects to asking clear questions that are not of the when did you stop beating your wife kind. What's at issue is whether and why unqualified import matters in any object, even the global object only, since the NAS proposal did not allow unqualified import even at global level, and the use-case for unqualified import was dismissed as not compelling. Namespacing of non-object properties is poorly justified, in my opinion. Unqualified import of top-level names is well-justified. I don't see why you keep mixing the two together. Sure you do, below where I gave particulars that led to the generalized namespace scheme in ES4. but does not mean namespaces need to generalize beyond the global scope. For example, global unqualified namespace import could desugar (logically) into the injection of scope chain items instead of into a general property lookup mechanism. That's an interesting idea, although we use namespace qualification along the prototype chain all over the place in ES4, and for what seem like good reasons. Other languages with successful namespacing features don't have such a mechanism, so I am dubious of the goodness of these ideas. I am concerned that the namespace lookup algorithm for object property access is too complicated. Agreed, this is the big issue. I share your concern, but the conservative approach (esp. with reference to C++) of throwing out non-global open namespaces looks like an overreaction, and it may not save much complexity. It makes object property lookup depend on the set of open namespaces, which means obj.property may compile to entirely different code depending on the context, Lexical context, no dynamic open-namespaces scope. and it seems likely it will slow down property lookup when multiple namespaces are open but static type info is missing. It certainly could, although we think not in implementations under way. Opening multiple namespaces without is not free in a dynamic language. Is the name lookup algorithm much simpler if namespaces are top-level only? Since obj.prop could end up with obj referring to the (or I should write a) global object, I don't see it. Unless you're proposing outlawing such object references using the namespaces open at top-level when obj is the global object. If the only real justification is that it's a nice generalization, then I do not think it is worth the performance hit. The nice generalization followed from particular use-cases, it did not precede them. I cited those cases (briefly). How about being responsive to them? ES (any version) has objects as scopes, as well as prototypes. It's hard to keep the gander -- objects below the top level, or on the prototype chain -- from wanting the same sauce that the goose -- the global object -- is trying to hog all to itself. Is it really? Is there any other language where namespacing of the global namespace has led to namespacing at the sub-object level? C+ +, Java and Python all get by fine without namespacing of individual object properties. C++ and Java are not the right paradigms for JS/ES. Python is better, but Python *does* allow import in local scope. The reason namespacing at top level is essential to programming in the large is that the global namespace is a shared resource and must be partitioned in controlled ways to avoid collision in a large system. But I do not see how this argument applies to classes or objects. See Mark's big post, which discusses (in item (b)) extending objects, including standard ones. Saying the global object is a shared resource that must be partitioned, etc., but no others reachable from it, particularly class objects, are shared resources, is begging the question: what makes any object a shared resource? That the global is the only necessarily shared object does not reduce the benefit, or make the cost prohibitive, of sharing other objects reachable from it. Prototype (the Ajax
Re: Namespaces as Sugar (was: complexity tax)
On May 27, 2008, at 11:00 AM, Brendan Eich wrote: What's at issue is whether and why unqualified import matters in any object, even the global object only, since the NAS proposal did not allow unqualified import even at global level, and the use-case for unqualified import was dismissed as not compelling. There's really 4 separable issues: 1) Namespacing of names at global scope (via lexically scoped reference). 2) Unqualified import of names into global scope. 3) Namespacing of arbitrary object properties. 4) Unqualified import of namespaces for arbitrary object properties. I would claim 1 and 2 are essential, 3 can be done by convention in the absence of 4 (a la the NAS proposal) and 4 is unnecessary and harmful to performance. That's an interesting idea, although we use namespace qualification along the prototype chain all over the place in ES4, and for what seem like good reasons. Other languages with successful namespacing features don't have such a mechanism, so I am dubious of the goodness of these ideas. I am concerned that the namespace lookup algorithm for object property access is too complicated. Agreed, this is the big issue. I share your concern, but the conservative approach (esp. with reference to C++) of throwing out non-global open namespaces looks like an overreaction, and it may not save much complexity. It could save a lot of complexity, by not requiring any first-class support for namespace lookup on arbitrary objects. It makes object property lookup depend on the set of open namespaces, which means obj.property may compile to entirely different code depending on the context, Lexical context, no dynamic open-namespaces scope. Note I said compile to so I think this was clear. and it seems likely it will slow down property lookup when multiple namespaces are open but static type info is missing. It certainly could, although we think not in implementations under way. Opening multiple namespaces without is not free in a dynamic language. Is the name lookup algorithm much simpler if namespaces are top- level only? Since obj.prop could end up with obj referring to the (or I should write a) global object, I don't see it. Unless you're proposing outlawing such object references using the namespaces open at top-level when obj is the global object. I would propose that unqualified import only affects lexically scoped lookups, not object property access, even if the object in question is the global object. In particular, if you are willing to say global.property instead of property, it is not such a hardship to say global.ns::property. If the only real justification is that it's a nice generalization, then I do not think it is worth the performance hit. The nice generalization followed from particular use-cases, it did not precede them. I cited those cases (briefly). How about being responsive to them? I think many (perhaps all) of those cases either use namespaces gratuitously or work fine without unqualified import (and so could use namespaces by convention). For example it doesn't seem important to allow unqualified import of the meta namespace. ES (any version) has objects as scopes, as well as prototypes. It's hard to keep the gander -- objects below the top level, or on the prototype chain -- from wanting the same sauce that the goose -- the global object -- is trying to hog all to itself. Is it really? Is there any other language where namespacing of the global namespace has led to namespacing at the sub-object level? C+ +, Java and Python all get by fine without namespacing of individual object properties. C++ and Java are not the right paradigms for JS/ES. Python is better, but Python *does* allow import in local scope. Python allows import from inside a local namespace, but does it allow import from outside a local namespace to affect lookup into that namespace? I am not aware of such a feature but I'm not a Python expert. The reason namespacing at top level is essential to programming in the large is that the global namespace is a shared resource and must be partitioned in controlled ways to avoid collision in a large system. But I do not see how this argument applies to classes or objects. See Mark's big post, which discusses (in item (b)) extending objects, including standard ones. Saying the global object is a shared resource that must be partitioned, etc., but no others reachable from it, particularly class objects, are shared resources, is begging the question: what makes any object a shared resource? That the global is the only necessarily shared object does not reduce the benefit, or make the cost prohibitive, of sharing other objects reachable from it. The benefit is less, because you can use separate objects in different namespaces instead of a shared object with namespaces inside it. The cost is greater
Re: Namespaces as Sugar (was: complexity tax)
2008/5/27 Maciej Stachowiak [EMAIL PROTECTED]: It could save a lot of complexity, by not requiring any first-class support for namespace lookup on arbitrary objects. Is the expectation then that having two lookup models, one for global objects and the other for non-global objects, going to provide less complexity? In a browser, window is the global object; would property lookup on window be namespaced when referenced as such? When you have a handle to another window? When you use |this.prop| in a global function? If we have namespace-aware lookup, it seems to me that it would be less complex for implementors and script authors alike for it to always apply, rather than just for one magical object. Mike ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: Namespaces as Sugar (was: complexity tax)
Delurk to ask a question -- On Tue, May 27, 2008 at 12:18 PM, Mike Shaver [EMAIL PROTECTED] wrote: In a browser, window is the global object; would property lookup on window be namespaced when referenced as such? When you have a handle to another window? When you use |this.prop| in a global function? What is wrong with the following strawman syntax: Define a form -- import into target values namespace; which puts all the values namespace.* into target. This can be explained in terms of two first class objects, target and namespace. Now the global case is just a matter of defining an alias for the global target, in case it is not mentionable by a given JS environment. Like maybe -- import values namespace; The remainder of the asymmetry, where the global lexical scope is addressable as an object yet nested scopes are not mentionable as first-class objects, is endemic to JavaScript and perhaps not a valid topic of debate or change at this point. This seems to solve the problem for module linkage, yet does not introduce complexities in general purpose object key lookup. It can be modeled simply as an addition of (possibly virtual) keys to an object. And it (correctly) requires the client to have write access to the target. The problem occurs if you wish to fail late in case two namespaces define the same name. This is where the keys can be virtualized so that, at lookup time, an error occurs if a name is ambiguous. Yet this is still localized to one first-class object resolving string keys to first-class object values, which is well within the simple programmer model of existing JS. Ihab -- Ihab A.B. Awad, Palo Alto, CA ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: Namespaces as Sugar (was: complexity tax)
On May 27, 2008, at 11:42 AM, Maciej Stachowiak wrote: On May 27, 2008, at 11:00 AM, Brendan Eich wrote: What's at issue is whether and why unqualified import matters in any object, even the global object only, since the NAS proposal did not allow unqualified import even at global level, and the use- case for unqualified import was dismissed as not compelling. There's really 4 separable issues: 1) Namespacing of names at global scope (via lexically scoped reference). 2) Unqualified import of names into global scope. 3) Namespacing of arbitrary object properties. 4) Unqualified import of namespaces for arbitrary object properties. I would claim 1 and 2 are essential, 3 can be done by convention in the absence of 4 (a la the NAS proposal) and 4 is unnecessary and harmful to performance. Thanks, this is helpful, since the argument you joined was about (2) and/or (4) -- there is no unqualified import in the NAS sketch. I forgot one of the main use-case for 4, one you've expressed interest in already: 'use namespace intrinsic'. Without (4), this does not allow early binding for s.charAt(i) given var s:string to string.prototype.intrinsic::charAt. So you can't add one pragma and realize speedups or better type signatures for built-in methods. All you can do is access global intrinsic::foo bindings, which (in ES4, where opt-in versioning gets you immutable Object, Array, String, etc. without needing to open intrinsic) are few and not likely to realize a speed-up or more precise typing. So early binding via namespacing would be gone. It could be reintroduced via an ad-hoc pragma. But that takes up complexity budget in the spec and real implementations too. It could save a lot of complexity, by not requiring any first-class support for namespace lookup on arbitrary objects. If I understand your proposal, meta::get or iterator::get could still be defined in an arbitrary object, and called with full qualification. Lexical context, no dynamic open-namespaces scope. Note I said compile to so I think this was clear. Just dotting an i, especially for everyone following along at home :-). * internal namespace per compilation unit for information hiding -- hardcode as a special case? I'm not sure how this applies to unqualified import of namespaces on arbitrary object properties. Per the spec, internal is open in a separate set on the list of open namespaces. You don't have to qualify references to internal::foo. * iterator and meta hooks in objects. Ugly __get__, etc., names instead? Unqualified import is not necessary for iterator or meta hooks. Namespaces by convention (or __decoratedNames__) would be enough. Agreed :-). * helper_foo() and informative_bar() in the RI? I don't think any language feature should exist solely for the convenience of the RI. It's not just the RI. Namespacing for informative purposes in the RI is good programming style, akin to using helper namespaces in C++ where top-level (but often class-level as you can see from reading RI builtins/*.es). `grep 'use namespace ' builtins/*.es` runs to 41 lines. If more effort must be spent the necessary complexity of the language just to preserve current levels of performance, No, to increase the levels of performance for the current version of the language. That's the horse going over the first hill from the barn. Adding namespaces to the next major version could slip into the optimization frameworks under way in a couple of engines I know of. But you may doubt. That's ok. I maintain that the high order bit should not be the cost to implementors, rather utility vs. complexity facing users. then that takes away resources from implementing unnecessary complexity to improve performance beyond current levels. Or harmonizes with the unnecessary complexity that implementors trying to compete already face. Let's not prejudge the ability of competing implementations to (a) speed up; (b) optimize namespace search for common code. The first target has to be the programmer using the language, not the implementor. This does not mean implementor concerns do not matter, as I keep trying to reassure you -- only that they come second. In general, keeping the language simpler is good for performance. Yet focusing on performance can complicate the language for programmers. It certainly has for other languages. Other things being equal, simpler implementations are easier to change, and have more room to add complexity for optimization without becoming too complex for human understanding. I will agree that some added language features are essential but I think minor improvements in expressiveness that have large complexity cost are a poor tradeoff. Hard to disagree without more details. This is motherhood and apple pie stuff. This is a good trade-off if it can be done in reasonable footprint, since
Re: Namespaces as Sugar (was: complexity tax)
Brendan Eich wrote: I want to say thanks for making this proposal (open namespace search only for lexical references). It leaves most of the use-cases I cited intact. Well done, good compromise (not complete evisceration of property qualifiers, or dismissal of unqualified import). Likewise. It's an important distinction to have made and I'm glad you made it. Gives the discussion clearer texture. I'm mostly sitting out the argument here, but there are good points being raised and it's good to explore around them. The helper/informative usage in the RI does open those namespaces, and intrinsic works only as a cross-cutting namespace that can be opened via pragma. So we should focus on these, if only to agree to disagree on the importance of the former, argue for hardcoding the latter via a separate early-binding pragma, and so forth. Deploying a 'use namespace intrinsic' pragma does more than give early-binding opportunity to ES3 code (which, realistically, you'll need to do some extra type-level work to fully resolve). More importantly, it grants a one-switch migration point from mutable prototype slots (compatible) to fixed class slots (predictable). IOW it's a semantic, program-integrity feature. This is IMO the more serious issue we're missing in the discussion. I want to elaborate on it here for the sake of clarity. What we have now is the ability to take ES3 code such as: var x = hello; x.split(e); and, by putting use namespace intrinsic at the front of it, make a rather drastic upgrade in its integrity (by at least some measure, see below). The call x.split() changes from a prototype-resolved lookup on a dynamic property (likely to resolve at String.prototype.public::split()) to a lookup that stops with the fixture intrinsic::split() defined on the class __ES4__::string, that x is an instance of. This fixture has the following improved nature: - It has fixed semantics! the user knows what they're getting and no 3rd party code twiddling String.prototype will change it. - It so happens that we improved the semantics by sticking dynamic typechecks on the boundaries of intrinsic::split(), so you'll get more type-error checking. - It can be cached and reused w/o worrying about cache invalidation due to mutation of String.prototype. The implementation can get adequate performance without having to play as-subtle tricks. Moreover, a single use namespace intrinsic will upgrade an *entire* compilation unit at a time (or a limited scope within one) using this technique, without having to go through and modify every x.split() to x.improved_split(). I'd be curious to hear if there's another well-developed way to provide this sort of thing. I guess it could be done by some sort of pragma that is specific to the standard library, but the current technique will work for helping other library authors (dojo / yui / etc.) provide an integrity-upgrading facility for *their* users too. It'd be nice to make this technique usable to all. (The same technique can be used to e.g. provide debugging-heavy variants of methods, or side-by-side usable new versions in the presence of old versions, that you toggle by opening namespaces. I have less experience with this, but IIRC it was part of Waldemar's the initial motivation.) -Graydon ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
