Re: ES4 implementation process, teams, and meetings
On Feb 20, 2008, at 3:38 PM, Brendan Eich wrote: > To provide focused face-to-face time working together and building > rapport among the principals, we are thinking of meeting roughly > every month, with this strawman schedule: > > March 17-21 - Mountain View, CA > April 14-18 - Newton, MA > May 12-16 - Vancouver, BC New schedule, getting firmer: March 24-26 - Mountain View, CA (before the next TC39 meeting on March 27-28) April 14-18 - Newton, MA May 12-16 - Vancouver, BC (longer week option, first three days for sure) Feedback welcome, as ever. /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
RE: ES4 implementation process, teams, and meetings
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Brendan Eich > Sent: 26. februar 2008 09:49 > To: Waldemar Horwat; Maciej Stachowiak > Cc: Graydon Hoare; es4-discuss@mozilla.org es4-discuss > Subject: Re: ES4 implementation process, teams, and meetings > > On Feb 25, 2008, at 5:58 PM, Waldemar Horwat wrote: > > > Brendan Eich wrote: > >> Thanks for understanding. Now with that in mind, please re-read > >> Jeff's post and mine. We are talking about working > intensively in the > >> next three months on both specs and implementation. Now is > the time > >> to step up. Apple was /hors de combat/ for a long time in > Ecma TC39. > >> Kicking the > >> legs out from under the table and pointing at the floor is > not good > >> citizenship in my book, whatever our (real) failings in keeping > >> proto-specs up to date. > > > > This is getting a little too belligerent. > > > Rather than update wiki'ed proposals, we've been planning on > writing more integrated specs. Lars has a library spec that > should be reviewable. Jeff is working on an ES3 spec based on > the RI, I believe. If we pursue this course, then we need to > do something with the out of date parts of the wiki to avoid > confusion. Things should get better over the next month, but > not soon enough. Just to follow up quickly. Jeff and I will have reviewable material (at a minimum a language base document and the ES4 library spec) ready at least two weeks before the next TC39 f2f. We haven't talked about how to distribute these materials, but an announcement will be made here and on the TC39 reflector in due course. (The next TC39 f2f is March 27 and 28, so two weeks before that is March 13.) This morning I will also be posting a few small -- frankly trivial -- draft specs for some ES4 surface features, and any feedback you have on their substance or format is welcome. (If there's agreement on the substance then they may make it into the March TC39 submission, but that depends on what we have time for, too. In either case they end up in the spec: namespace on the wiki once we've converged.) Each of these proposals directly addresses a point in John Resig's spreadsheet: http://spreadsheets.google.com/pub?key=pFIHldY_CkszsFxMkQOReAQ&gid=2 For a number of surface and mid-level features, specs like these are probably just right in that they define the feature without needing to make use of heavyweight formalism or discuss every detail that the full spec will have to pin down. (For example, the ES3 spec for Array.prototype.reverse pins down the order of insertions and deletions in the array. That order may be observable by the program (at least in ES4 it is) so it's important for interoperability that it be specified. But it's a level of detail that's probably not very useful in these draft specs.) The point I'm making is simply that if any implementer among you feels that a feature listed in the spreadsheet above is interesting to you and well enough specified in the supporting documents (and in the RI if you're so inclined) then go forth and write the spec, and let it be informed by doing a trial implementation. And then post the spec here. --lars ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
On Feb 25, 2008, at 5:58 PM, Waldemar Horwat wrote: > Brendan Eich wrote: >> Thanks for understanding. Now with that in mind, please re-read >> Jeff's >> post and mine. We are talking about working intensively in the next >> three months on both specs and implementation. Now is the time to >> step >> up. Apple was /hors de combat/ for a long time in Ecma TC39. >> Kicking the >> legs out from under the table and pointing at the floor is not good >> citizenship in my book, whatever our (real) failings in keeping >> proto-specs up to date. > > This is getting a little too belligerent. My apologies, I did not intend that. > Maciej has a point which several of us share. For things that > haven't been discussed actively, I too have little idea about what > their status is or where is the correct place to look for the > current understanding of their design. This became a problem when > we went from using proposals to relying on individual trac > tickets. I think we should go back to maintaining proposals. I agree, and I've done that with a proposal (iterators and generators) I've tended, but it depends on structural types and type paramters. There, the RI and Cormac's paper would need to be turned into wiki'ed content. Is that a good use of time? This gets back to the question of the RI as an informative construct, which was what Graydon referred to with the "kicking the legs out from under the table" metaphor. If the RI is not acceptable to some, but nevertheless important for testability and significant parts of the final spec, then we face the task of translating parts of it into prose. Graydon's SML-lowered-to- English script may be helpful, but new prose is needed too. Rather than update wiki'ed proposals, we've been planning on writing more integrated specs. Lars has a library spec that should be reviewable. Jeff is working on an ES3 spec based on the RI, I believe. If we pursue this course, then we need to do something with the out of date parts of the wiki to avoid confusion. Things should get better over the next month, but not soon enough. /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
RE: ES4 implementation process, teams, and meetings
I think the important point to keep in mind here is that we need to move forward to sound specs based on commercial implementation. The burden is on those who "understand" the proposals + trac + RI to translate them into implementation and spec for others to evaluate. This is a short step forward for all but the deepest proposals. For the more fundamental changes, the proposal format has and will continue to be unsatisfactory. To address some of the concerns expressed last week, Lars has updated John Resig's ES4 status spreadsheet at http://spreadsheets.google.com/pub?key=pFIHldY_CkszsFxMkQOReAQ&gid=2 He has added references to some of the materials supporting each feature. We also intend to add a column to indicate the bakedness of each feature. For people who are more interested in self-standing specs, some will be coming in the next few weeks. Jd > -Original Message- > From: [EMAIL PROTECTED] [mailto:es4-discuss- > [EMAIL PROTECTED] On Behalf Of Waldemar Horwat > Sent: Monday, February 25, 2008 5:59 PM > To: Brendan Eich > Cc: Graydon Hoare; es4-discuss@mozilla.org es4-discuss > Subject: Re: ES4 implementation process, teams, and meetings > > Brendan Eich wrote: > > Thanks for understanding. Now with that in mind, please re-read Jeff's > > post and mine. We are talking about working intensively in the next > > three months on both specs and implementation. Now is the time to step > > up. Apple was /hors de combat/ for a long time in Ecma TC39. Kicking the > > legs out from under the table and pointing at the floor is not good > > citizenship in my book, whatever our (real) failings in keeping > > proto-specs up to date. > > This is getting a little too belligerent. > > Maciej has a point which several of us share. For things that haven't > been discussed actively, I too have little idea about what their status is > or where is the correct place to look for the current understanding of > their design. This became a problem when we went from using proposals to > relying on individual trac tickets. I think we should go back to > maintaining proposals. > > Waldemar > ___ > Es4-discuss mailing list > Es4-discuss@mozilla.org > https://mail.mozilla.org/listinfo/es4-discuss ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
Brendan Eich wrote: > Thanks for understanding. Now with that in mind, please re-read Jeff's > post and mine. We are talking about working intensively in the next > three months on both specs and implementation. Now is the time to step > up. Apple was /hors de combat/ for a long time in Ecma TC39. Kicking the > legs out from under the table and pointing at the floor is not good > citizenship in my book, whatever our (real) failings in keeping > proto-specs up to date. This is getting a little too belligerent. Maciej has a point which several of us share. For things that haven't been discussed actively, I too have little idea about what their status is or where is the correct place to look for the current understanding of their design. This became a problem when we went from using proposals to relying on individual trac tickets. I think we should go back to maintaining proposals. Waldemar ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 2:24 PM, Maciej Stachowiak wrote: On Feb 21, 2008, at 10:41 AM, Brendan Eich wrote: We'd like to be active participant. However, it seems like as newcomers/outsiders, we do not have enough information available to participate in early implementation. Neither does anyone else. That's the point of the co-evolved implementations and specs. Sorry if this is unclear. Michael O'Brien's messages have been painfully clear on where specs are lacking, and before there's an ES4 standard we need to fix those lacks. I am not asking for a finished, formal, final detailed spec. What I am asking is this: for each proposal where you'd like early implementations, before implementation commences please write down enough information about that proposal in some reasonably understandable form to represent the current shared understanding of the insiders/old-timers. That would be enough info for us relative outsiders/newcomers to participate. I don't think it's too much to ask for a rough but up-to-date and accurate first draft. I'm not sure how we are supposed participate otherwise. Maybe it is not expected that we would or should. No, we're not turning anyone away for not diving into the deep end of the pool Michael already dove into. We need specs, but they need to be informed by ongoing, well-ordered implementation work. Where we have good-enough specs already (Graydon listed many places near surface syntax), we're ready -- indeed JS1.7 included a bunch of these in order to shake out specs and provide implementation and field-testing feedback. The deeper unresolved issues to do with numbers, e.g., need more spec work, I agree. The proposals on the wiki are way out of date, it's not easy to find what trac tickets modified them, and there seem to be commonly understood planned changes that aren't even reflected in trac. That's a failure to file trac tickets -- could you please list these changes that aren't in the trac? There's no other bug system to track these planned changes, so they had better show up at http://bugs.ecmascript.org/ soon or they won't happen. I have no idea what changes aren't in trac. You wrote "there seem to be commonly understood planned changes that aren't even reflected in the trac." I'm asking you what exactly you meant. In the past I've asked questions on #jslang or elsewhere about particular proposals (such as the parametric types proposal) and been told that many things about it had been changed, and the person telling me wasn't sure if all these changes had trac tickets, or if so, what they were. A Mozilla IRC channel and an anonymous respondent -- I don't know what to say. Let's just start fresh here, ok? It really seems to me like in many cases there is a shared understanding among many of the insiders that is only recorded inside people's heads. Maybe that's not right, but that is certainly the impression I've gotten every time I have asked questions about where something is documented. The only way to proceed is case by case, but I think we've already identified the RI as an informative artifact, not just in people's heads, but something you are not able to use. That is a pity, but perhaps we can work around it. Great, if some proposals are accurate and up to date enough to drive an initial implementation, then my concern is addressed for those features. But I don't know how to tell which ones those are. Is there a list of which proposals are up to date? No, probably that's the first order of business, and it should not take that much time. Furthermore, this won't help when it comes time to implement proposals that *aren't* up to date. All I'm asking is that they be brought up to date first. The preference has been to work via the trac to produce sub-specs instead of editing the old (sometimes mis-named) proposals: pages on the wiki. Whatever we do, I agree we need to get rid of the out of date docs. This has been a recurrent problem; sorry for it. Then what is proposed? If I ask an engineer on my team to implement a feature such as type annotation, how should I ask them to proceed? A type annotation is not a "leaf feature" like a new tag in HTML5. It involves foundational issues. The answer at this point involves reading Cormac's paper, and the RI. We can work from here toward a final spec which will entail prose, but it may also use conventional formalisms for type rules, and it will be based on checkable RI code, even if that is not directly excerpted into the spec. In contrast, with CSS, Web API or HTML WG specifications, I can point engineers to a spec that is more or less accurate for a given feature and they only have to ask questions about the few missing details. And then Hixie goes and rewrites it. I am calling b.s. here, Maciej. We implemented offline web app support early in Firefox 3, based on such WHAT-
Re: ES4 implementation process, teams, and meetings
Maciej Stachowiak wrote: > We're unlikely to have much interest in working on implementing the RI. Ok. I'm sorry to hear that, but I understand. > As for reading the RI, it seems a lot harder to understand than specs > written in prose. As far as I can tell, only people who have coded > significant portions understand it. Fair enough. Comprehensibility is a good part of the measurable value of a spec, so if it the code prohibits that we are in an undesirable state. I wonder -- I do not mean to offend here -- if this is partly "sticker shock" at the initial barrier, which is simply that you have to digest SML, and you haven't read it before. It is not a terribly hard language to learn: it consists of value bindings, function expressions, function-application expressions, case expressions with destructuring pattern matching, if/then/else expressions, and a very small algebraic type system (function types, named types, records, disjoint sums, and sugar for lists). I also intended to do -- and have gradually been doing -- a conversion to the simplest possible syntactic forms of SML I could write, unpacking any syntactic short-hands or dense, idiomatic phrases that might have turned up during the more intense implementation stages. This is similar to rewriting english paragraphs for clarity, and is easy work for someone who speaks the language. Parallelizes easily. I believe this will help with the legibility significantly: for example, read evalCondExpr and tell me if it's illegible: evalCondExpr (regs:Mach.REGS) (cond:Ast.EXPR) (thn:Ast.EXPR) (els:Ast.EXPR) : Mach.VAL = let val v = evalExpr regs cond val b = toBoolean v in if b then evalExpr regs thn else evalExpr regs els end One of my goals -- which I have surely not achieved yet -- is for most of the RI to be distilled to this very pedestrian dialect. > On the one hand, it's useful to have a reference implementation to > validate what is being done, explore ideas, have something to test and > compare against, etc. > > But yes I think it is an incredibly bad idea for the only specification > to be a computer program. It's not approachable. I don't think I could > quickly grok a program of this complexity even in a programming language > I am familiar with. And by its nature it does not very cleanly partition > separate concepts. For example, below you pointed me to 6 places in the > code for "let" statements, and I doubt that reading those functions > alone will be enough to understand it. So in practice, I don't think > there is any way to understand "let" in detail without asking you or > another expert on the RI. These are ... points I nearly agree with, but not quite, and at the risk of being terribly long-winded I'd like to air the discussion a bit in public here, if we can back off from worrying that I'm saying anything about the schedule of auxiliary-doc-generation (which I've hopefully addressed in the other email): First I want to to point out that there is no established "right way" to publish language specifications. Language specifications range in style and formalisms employed. People frequently need to study spec, and implementations, and formal treatments in eg. proof assistants or reduced semantic models, *and* do impl-to-impl compatibility bakeoffs. And it still sometimes takes many years, many revisions, to nail down what people actually agree on or disagree on, what's "in" the language or "out" of it. Sometimes it takes 5 or 10 years to discover a horrible unsoundness in the language (or, gasp, that you accidentally made the type system turing complete!) No one approach is proven to "work". Not yet. The AS3 draft spec we were looking at two winters ago had sections containing pseudo-C++ code, as a way of describing relevant data structures. ES3 has pseudo-assembly, that has typos and nonsensical parts in addition to requiring readers to execute goto statements in their head to understand the flow of a rule. R6RS, for a different example, shipped most recently with a PLT Redex operational semantic model to accompany and illuminate it. We considered using PLT Redex too, and in fact rejected it in part out of the belief (perhaps mistaken!) that "normal" programmers would find a "normal" language like SML easier to read than one from the more academic setting of operational semantics descriptions. Possibly in the future (as the POPLMark challenge is hoping to establish) a standard metatheory will solidify for semantics such that machine-checked evaluation rules are no less common than machine-checked grammars in EBNF. But we're not there yet, so we picked something that seemed like it might help, and in at least some senses (see next point) it did. Second I want to point out that while much of the value of a spec is in informing/transmitting informa
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 5:49 PM, Maciej Stachowiak wrote: On Feb 21, 2008, at 5:03 PM, Graydon Hoare wrote: So presenting yourself as a participant with neither of those supports in place, you're sort of walking into a room, kicking the legs out from under a table and asking why it's suddenly on the floor. We need to revise the strategy a bit to help for your case, if we can't rely on those. Please try to be patient if this takes some time; none of us had put "write pre-impl docs" in our current work-assignment schedules. We'll need to make room to do it. I can probably dedicate a month of nearly-full-time energy to this starting in the first week of march. Soon enough? I'm totally willing to wait for documentation that is more clear and up to date. What concerned me was that it wasn't part of the plan at all, and that Brendan at least seemed unreceptive to the request. You keep misrepresenting things. I never said anything about not writing up-to-date specs. I said specs should be co-evolved with implementations to be up to date, and to the extent that they can be based on extracted RI code, they should be developed that way, so as to be mechanically checkable. This is a serious point. It deserves more than what I regard as flippant attitude about "ask Graydon" combined with overstatement of the value of all-prose specs. If you have to look at four files to understand let as implemented in the RI, so what? ES1-3 require multiple readings of sections 8-15 just to settle simple questions. The overview and evolutionary programming tutorial take more top-down approaches to presenting (not specifying) features. Programming languages with the kind of type system, binding rules, and compatibility constraints that ES4 has absolutely need executable semantic specifications, not merely all-prose specs. That it takes time is natural. Thanks for understanding. Now with that in mind, please re-read Jeff's post and mine. We are talking about working intensively in the next three months on both specs and implementation. Now is the time to step up. Apple was hors de combat for a long time in Ecma TC39. Kicking the legs out from under the table and pointing at the floor is not good citizenship in my book, whatever our (real) failings in keeping proto-specs up to date. Ultimately for ES4 to be a meaningful standard, it has to have a spec that is comprehensible without special insider information. That's true if you mean by "comprehensible" "things only in people's head". I claim it's false if you mean "things specified only in prose". My expectation was that at least parts of it would start to approach that point before they reached the implementation phase. I'm surprised that this wasn't the plan already, but I'm glad you are willing to be flexible. I'm not thrilled about people detouring into recapitulating in uncheckable, duplicative, buggy prose what is in the RI. I'd rather we focus on where the final spec will want to use prose anyway, not extracted, lowered RI code. Probably we will have to compromise and do both prose and RI work. How willing to be flexible are you? /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 5:03 PM, Graydon Hoare wrote: > Maciej Stachowiak wrote: > >> What I am asking is this: for each proposal where you'd like early >> implementations, before implementation commences please write down >> enough information about that proposal in some reasonably >> understandable form to represent the current shared understanding >> of the insiders/old-timers. > > Ok. Understand though that you're asking for a bit of a change to > the game plan. That's a reasonable demand but the "early-early" > imlpementations (RI, futhark, spidermonkey, esc) were all done by > people who *did* have that shared understanding (through earlier > participation and discussion) and *did* find the SML we were all > working on "reasonably understandable". > > So presenting yourself as a participant with neither of those > supports in place, you're sort of walking into a room, kicking the > legs out from under a table and asking why it's suddenly on the > floor. We need to revise the strategy a bit to help for your case, > if we can't rely on those. Please try to be patient if this takes > some time; none of us had put "write pre-impl docs" in our current > work-assignment schedules. We'll need to make room to do it. I can > probably dedicate a month of nearly-full-time energy to this > starting in the first week of march. Soon enough? I'm totally willing to wait for documentation that is more clear and up to date. What concerned me was that it wasn't part of the plan at all, and that Brendan at least seemed unreceptive to the request. That it takes time is natural. Ultimately for ES4 to be a meaningful standard, it has to have a spec that is comprehensible without special insider information. My expectation was that at least parts of it would start to approach that point before they reached the implementation phase. I'm surprised that this wasn't the plan already, but I'm glad you are willing to be flexible. Regards, Maciej ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 4:34 PM, Graydon Hoare wrote:
> Maciej Stachowiak wrote:
>
>> I don't think the sets are disjoint, but they are not identical
>> either.
>
> Agreed. I am trying to arrive at an understanding of which camp
> Apple aspires to ("designer", "implementor" or both) and in
> particular how you wish to enact that role.
Apple is not monolithic. Some of us hope to participate more in the
design process, but it's pretty likely that people who don't deeply
involve themselves in the design process will do a lot of the
implementation work. Those people who are implementing need to have
something to follow.
> Any Rhino hackers (or other implementors) may also wish to chime in.
> It sounds to me like "not having anything to do with the RI" is
> characteristic of how you wish to participate. Is this correct?
We're unlikely to have much interest in working on implementing the
RI. We have a small team and our hands are full implementing our own
JS engine (and the rest of the browser engine). And we think that
ultimately implementing a production-quality implementation in our
engine is more valuable.
While we try to do our part to help with development and validation of
important web standards, learning a new programming language and
coding in it is a pretty high barrier to entry. Is that really the
only way to meaningfully participate in the process?
As for reading the RI, it seems a lot harder to understand than specs
written in prose. As far as I can tell, only people who have coded
significant portions understand it.
>> I don't think foundational documents are what we need to implement
>> specific features. What we need are rough cut but accurate specs
>> for the features to implement. I don't think SML + trac is a form
>> that anyone here can easily understand.
>
> Ok. Some of the features are deep and some are shallow. Many of the
> shallow ones are not in much flux, do not require much insight, and
> are well-described by the "proposals" on the wiki. You could readily
> show one to an engineer and have them implement, say, destructuring
> assignment, triple quotes, date and time extensions, regexp
> extensions, a hashcode function, getters and setters, block
> expressions (let expressions), enumerability control,
> String.prototype.trim, expression closures.
Great, can we start recording this list somewhere? Perhaps part of
breaking down features and figuring out the dependencies should be
recording which features have an up to date and accurate wiki proposal.
> Others, as you've noted, require some clean-up / consolidation to be
> brought into line with tickets. Decimal and numerics turned out to
> be an ongoing source of trouble, as did tail calls. We're still
> discussing these. We can probably do this in parallel with your team
> implementing the ones that are more stable.
A clear list of proposals that are clearly not ready would be valuable
as well. So let's also start recording this list.
> The thing we don't have "feature sketches" for are deep, systemic
> issues that affect the whole language: types, fixtures and
> namespaces. Also the rules governing packages, classes and
> interfaces. These came in "implicitly" from AS3, and the type system
> in particular has been modified extensively: Cormac's paper and the
> code in type.sml is probably the best we have to describe the
> intended type system. This is what I meant by "foundational"
> documents. Do you still feel that they are not needed?
Documentation will certainly be needed when it comes time to implement
said features.
>> Well, neither I nor anyone on my team know SML. Nor do we know the
>> internals of the reference implementation, what aspects of it are
>> normative, which are implementation details, and which are
>> considered bugs and are intended to change. Nor would I know where
>> in the RI to look to understand how to implement particular
>> features. For example, "let" binding was raised as an example of a
>> possible early implementation feature. I don't know where in the
>> ~40 klocs of SML in the repository I should look.
>
> I am trying to give you a guide, but am unsure if you want this form
> of guidance. SML is a small functional language, and we use mostly
> applicative style. Nothing crazy.
>
> Here is a general rule for researching an aspect of the language:
> start by identifying the AST node for a feature; if you can't find
> it by guesswork and reading ast.sml, ask someone on IRC or here.
> Then depending on what you want to do for that feature, read the
> parser.sml function that produces that node, the defn.sml function
> that elaborates it, the type.sml rule that checks it, or the
> eval.sml rule that evaluates it.
>
> It would also be good to read some of mach.sml to familiarize
> yourself with the runtime machine model eval.sml manipulates, if
> you're investigating a runtime
Re: ES4 implementation process, teams, and meetings
Maciej Stachowiak wrote: > What I am asking is this: for each proposal where you'd like early > implementations, before implementation commences please write down > enough information about that proposal in some reasonably understandable > form to represent the current shared understanding of the > insiders/old-timers. Ok. Understand though that you're asking for a bit of a change to the game plan. That's a reasonable demand but the "early-early" imlpementations (RI, futhark, spidermonkey, esc) were all done by people who *did* have that shared understanding (through earlier participation and discussion) and *did* find the SML we were all working on "reasonably understandable". So presenting yourself as a participant with neither of those supports in place, you're sort of walking into a room, kicking the legs out from under a table and asking why it's suddenly on the floor. We need to revise the strategy a bit to help for your case, if we can't rely on those. Please try to be patient if this takes some time; none of us had put "write pre-impl docs" in our current work-assignment schedules. We'll need to make room to do it. I can probably dedicate a month of nearly-full-time energy to this starting in the first week of march. Soon enough? (Also note that about half of the RI is written in ES4, not SML, and that is surely understandable to your team with minimal effort) > I have no idea what changes aren't in trac. In the past I've asked > questions on #jslang or elsewhere about particular proposals (such as > the parametric types proposal) and been told that many things about it > had been changed, and the person telling me wasn't sure if all these > changes had trac tickets, or if so, what they were. Unfortunately parametric types are one of the cases where you are stabbing at both a terribly in-flux and terribly deep part of the language; they only make sense in the context of types in general, and the *entire* type system has been in flux for quite some time. I believe it is stable now, and I believe the way to understand it is to read Cormac's paper first and type.sml second, ignoring much of the residue in the wiki. I'm sorry about this. A consolidated write-up would be good, I agree. Part of what I was trying to get at by describing dependencies was that you don't really need to understand the entire type system to get a little miniature version of it hobbling along. The initial task to ask one of your engineers to do is "give each property and each value a pointer to a runtime-defined 'type' (TBD) and run a function that checks those 'match' (TBD) when someone performs an assignment". Then start enhancing the meaning of "type" and "match" as you learn more about them: mbedthis (and the RI) started with simple nominal types (classes in a hierarchy) and worked out from there. > It really seems to me like in many cases there is a shared understanding > among many of the insiders that is only recorded inside people's heads. > Maybe that's not right, but that is certainly the impression I've gotten > every time I have asked questions about where something is documented. This depends significantly on the feature in question. For types, and discounting both Cormac's paper and the RI as "something you can't extract the desired degree of meaning from", this is probably true. There's not a laid-out plan of the full type system in english alone. I don't know if you like reading LaTeX-y academic type judgment rules more than SML :) For simpler features, I think this is an over-generalization. With some simple markup about currentness, some of them are implementable as-is. But hopefully we can patch up the worst cases and/or mark the stable and easy pieces quickly, so you can make progress. > Great, if some proposals are accurate and up to date enough to drive an > initial implementation, then my concern is addressed for those features. > But I don't know how to tell which ones those are. Is there a list of > which proposals are up to date? No. You're right that we should sort them or mark their outdated-ness and revise them. This may cover features that stand as proposals. For deeper cross-language issues (say "how classes work"), it is not a matter of bringing a page up to date as writing one in the first place. Nobody "proposed" them since they were inherited from AS3. We may get to writing pre-implementation guides to these too, but it too will take time. > Furthermore, this won't help when it comes time to implement proposals > that *aren't* up to date. All I'm asking is that they be brought up to > date first. A reasonable request; one we haven't focused energy on lately. -Graydon ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
Maciej Stachowiak wrote:
> I don't think the sets are disjoint, but they are not identical either.
Agreed. I am trying to arrive at an understanding of which camp Apple
aspires to ("designer", "implementor" or both) and in particular how you
wish to enact that role. Any Rhino hackers (or other implementors) may
also wish to chime in. It sounds to me like "not having anything to do
with the RI" is characteristic of how you wish to participate. Is this
correct?
> I don't think foundational documents are what we need to implement
> specific features. What we need are rough cut but accurate specs for the
> features to implement. I don't think SML + trac is a form that anyone
> here can easily understand.
Ok. Some of the features are deep and some are shallow. Many of the
shallow ones are not in much flux, do not require much insight, and are
well-described by the "proposals" on the wiki. You could readily show
one to an engineer and have them implement, say, destructuring
assignment, triple quotes, date and time extensions, regexp extensions,
a hashcode function, getters and setters, block expressions (let
expressions), enumerability control, String.prototype.trim, expression
closures.
Others, as you've noted, require some clean-up / consolidation to be
brought into line with tickets. Decimal and numerics turned out to be an
ongoing source of trouble, as did tail calls. We're still discussing
these. We can probably do this in parallel with your team implementing
the ones that are more stable.
The thing we don't have "feature sketches" for are deep, systemic issues
that affect the whole language: types, fixtures and namespaces. Also the
rules governing packages, classes and interfaces. These came in
"implicitly" from AS3, and the type system in particular has been
modified extensively: Cormac's paper and the code in type.sml is
probably the best we have to describe the intended type system. This is
what I meant by "foundational" documents. Do you still feel that they
are not needed?
> Well, neither I nor anyone on my team know SML. Nor do we know the
> internals of the reference implementation, what aspects of it are
> normative, which are implementation details, and which are considered
> bugs and are intended to change. Nor would I know where in the RI to
> look to understand how to implement particular features. For example,
> "let" binding was raised as an example of a possible early
> implementation feature. I don't know where in the ~40 klocs of SML in
> the repository I should look.
I am trying to give you a guide, but am unsure if you want this form of
guidance. SML is a small functional language, and we use mostly
applicative style. Nothing crazy.
Here is a general rule for researching an aspect of the language: start
by identifying the AST node for a feature; if you can't find it by
guesswork and reading ast.sml, ask someone on IRC or here. Then
depending on what you want to do for that feature, read the parser.sml
function that produces that node, the defn.sml function that elaborates
it, the type.sml rule that checks it, or the eval.sml rule that
evaluates it.
It would also be good to read some of mach.sml to familiarize yourself
with the runtime machine model eval.sml manipulates, if you're
investigating a runtime entity. The interesting types are VAL, OBJ,
VAL_TAG, MAGIC, SCOPE, REGS, PROP_STATE, TEMP_STATE, TEMPS, PROP,
PROP_BINDINGS.
> If learning SML is really a prerequisite to being one of the early
> implementors, then I am not sure anyone working on WebKit/JavaScriptCore
> is qualified, which is a pity. If learning SML is a prerequisite to
> understanding the finished spec, then that will be a pretty high barrier
> to entry for even non-early implementors.
It certainly will not be a requirement for understanding the finished
spec, though depending on whether any pieces of the code make it in as
informative annexes, it may help. It *is* currently a requirement for
understanding the RI, which the committee agreed to work together on
last year rather than drafting "formal" pseudo-assembly / pseudo-english
language we could neither execute nor even automatically check for typos.
So far some members of the committee have taken to working on it, while
some have not. Are you interested in working on it, as part of Apple's
active participation in the process? Or learning to read it? Or is it a
completely opaque non-sequitur from Apple's perspective?
The idea here -- and feel free to say it's a bad idea or you disagree!
-- is that a very high level programming language carries a *different*
mixture of risks than very low level natural language, and possibly a
more desirable mixture. In particular, as you say, it risks
overspecifying and using unfamiliar technical notation; but the risks of
natural language (being logically contradictory and underspecifying) are
significant too!
> I tried reading over some of the SML files y
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 10:31 AM, Graydon Hoare wrote: > Maciej Stachowiak wrote: > >> To expand a bit on Geoff's comment: >> I'd like Apple and the WebKit project to get involved with ES4 >> implementation. > > Great! Though please keep in mind a point in the remainder of your > comments: WebKit (and Rhino) are operating from a somewhat > "newcomer" perspective, relative to the committee. The other 5 > implementations in question (RI, spidermonkey, mbedthis, futhark, ESC > +tamarin) are all written by engineers who are and have been closely > following the tickets, proposals and discussion, and modifying the > RI to encode their thoughts / experiment with a feature. > > So the implication that the language "designers" are a disjoint set > from the "implementors", or that they haven't been writing their > thoughts down, is not historically accurate. If that's becoming more > true now, ok, maybe we need to make some adjustments. But understand > where we're coming from. I don't think the sets are disjoint, but they are not identical either. >> Before attempting interoperable implementations of particular >> features, I think we need at minimum a form of the proposal for >> that feature that is complete and up to date. It doesn't have to >> be formal specification quality, but there has to be something >> accurate. > > I agree this would be nice, but it'd also be nice to have 9 finished > implementations and a finished spec! We don't have these yet. So: is > your team *completely* stalled until we have such documents, > presumably in english rather than SML? If so, I (or anyone else who > understands the issues clearly enough -- they're not tremendously > complex) can make a priority of building up foundational > "implementors documents" covering such basic concepts as namespaces, > names, multinames, types and fixtures. I think we had hoped the RI > and tickets on it to serve this role. I don't think foundational documents are what we need to implement specific features. What we need are rough cut but accurate specs for the features to implement. I don't think SML + trac is a form that anyone here can easily understand. >> Now, it may be that by telling someone to reverse engineer another >> implementation, or ask the ES4 crowd about every detail of how a >> feature should be implemented, someone could succeed in >> implementing. But it seems to me that this undermines the unstated >> assumption of interoperable *independent* implementations. > > I do not think it undermines the assumption of independent > implementations, but I also don't think there's a perfectly clear > line between "dependent" and "independent". Impls inform themselves > from somewhere, be it spec or spec-that-is-informed-from-other-impls > or other technical reference material. Information flows somehow, > and often between impls (even if indirectly). > > You're not going to be doing WebKit by studying Futhark or > Spidermonkey; but I *would* recommend studying the RI (and > contributing to it!) I would not worry greatly about the risk of > being "dependent" on it, since it is in a very different language > (SML) than WebKit's interpreter and is surely structured quite > differently. Study it and understand it, though, as it's as precise > as we currently get. The RI was meant to be studied (a.k.a. "reverse > engineered") and the risk of overspecificity from that is something > we all explicitly agreed was better than the risk of deploying > underspecified and incompatible impls to the field. Well, neither I nor anyone on my team know SML. Nor do we know the internals of the reference implementation, what aspects of it are normative, which are implementation details, and which are considered bugs and are intended to change. Nor would I know where in the RI to look to understand how to implement particular features. For example, "let" binding was raised as an example of a possible early implementation feature. I don't know where in the ~40 klocs of SML in the repository I should look. >> In contrast, with CSS, Web API or HTML WG specifications, I can >> point engineers to a spec that is more or less accurate for a >> given feature and they only have to ask questions about the few >> missing details. I would raise HTML5 as a particularly laudable >> example because it achieves this even though much implementation >> work is happening in parallel with writing the spec. > > HTML5 is a laudable example, and I hope we wind up producing > something of similar quality. It has also had more energy put into > it, more eyes on it, and is a much wider and flatter spec (fewer > subtle interlocking issues). > > Web browsers are also stunningly more complex than programming > languages, so the concept of a "reference implementation" is > completely fanciful (though you could do a reference implementation > of th
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 10:41 AM, Brendan Eich wrote: On Feb 21, 2008, at 8:30 AM, Maciej Stachowiak wrote: I'd like Apple and the WebKit project to get involved with ES4 implementation. But right now, as far as I can tell, there isn't a written record for any of ES4's features that I could point an engineer to and say "implement this". There's certainly no such spec, or you would be a passive observer of a standardization process that was all but done. That's not reality, and it arguably is not what you should want -- Apple people could be valued peers in the remaining work on ES4. If you want to be passive implementors of a finished spec, then wait till next year. We'd like to be active participant. However, it seems like as newcomers/outsiders, we do not have enough information available to participate in early implementation. I am not asking for a finished, formal, final detailed spec. What I am asking is this: for each proposal where you'd like early implementations, before implementation commences please write down enough information about that proposal in some reasonably understandable form to represent the current shared understanding of the insiders/old-timers. That would be enough info for us relative outsiders/newcomers to participate. I don't think it's too much to ask for a rough but up-to-date and accurate first draft. I'm not sure how we are supposed participate otherwise. Maybe it is not expected that we would or should. The proposals on the wiki are way out of date, it's not easy to find what trac tickets modified them, and there seem to be commonly understood planned changes that aren't even reflected in trac. That's a failure to file trac tickets -- could you please list these changes that aren't in the trac? There's no other bug system to track these planned changes, so they had better show up at http://bugs.ecmascript.org/ soon or they won't happen. I have no idea what changes aren't in trac. In the past I've asked questions on #jslang or elsewhere about particular proposals (such as the parametric types proposal) and been told that many things about it had been changed, and the person telling me wasn't sure if all these changes had trac tickets, or if so, what they were. It really seems to me like in many cases there is a shared understanding among many of the insiders that is only recorded inside people's heads. Maybe that's not right, but that is certainly the impression I've gotten every time I have asked questions about where something is documented. Before attempting interoperable implementations of particular features, I think we need at minimum a form of the proposal for that feature that is complete and up to date. It doesn't have to be formal specification quality, but there has to be something accurate. I've worked pretty hard to keep proposals such as iterators and generators up to date; it depends on other proposals which are also not formal spec quality, but stable and meaningful (structural types, type parameters). Cormac has done work recently in formalizing the type system which was important to Graydon's RI work. Great, if some proposals are accurate and up to date enough to drive an initial implementation, then my concern is addressed for those features. But I don't know how to tell which ones those are. Is there a list of which proposals are up to date? Furthermore, this won't help when it comes time to implement proposals that *aren't* up to date. All I'm asking is that they be brought up to date first. So I think you are generalizing unfairly here. It's true that decimal is out of date in the wiki, and there are open trac issues. This is true of other proposals. Now, it may be that by telling someone to reverse engineer another implementation, or ask the ES4 crowd about every detail of how a feature should be implemented, someone could succeed in implementing. Nice strawmen, but no one proposed those things. Then what is proposed? If I ask an engineer on my team to implement a feature such as type annotation, how should I ask them to proceed? But it seems to me that this undermines the unstated assumption of interoperable *independent* implementations. In contrast, with CSS, Web API or HTML WG specifications, I can point engineers to a spec that is more or less accurate for a given feature and they only have to ask questions about the few missing details. And then Hixie goes and rewrites it. I am calling b.s. here, Maciej. We implemented offline web app support early in Firefox 3, based on such WHAT-WG (ok, not HTML WG at the time) specs. They changed a great deal later. I'm not asking for a spec that won't substantially change. The whole point of early implementations is to improve the spec, and sometimes that may take significant redesign. Safari has been hit by this as well, and we accept that as a risk we take on as early i
Re: ES4 implementation process, teams, and meetings
On Thu, Feb 21, 2008 at 3:48 PM, Geoffrey Garen <[EMAIL PROTECTED]> wrote: > > However, since there is no spec -- at least, not one I can lay my > hands on -- I'll have to wait until the "people in the know" finish > their implementations What changes would you like to see in the process? What would make it easier for you to jump in, and follow along once you've started? - Rob ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 12:07 PM, Michael O'Brien wrote: > My recommendation would be: > > - Add comments and overview inside the RI code itself. > - Create an addendum to Lar's document that drills down on key > topics to > provide more exact operational semantics and details. Lively exchange here, want to provide context again for newcomers. We do not want a by-hand, mechanically uncheckable operational semantic spec in parallel with the RI. Our plan to excerpt (and possibly lower/pretty-print) pieces of the RI still seems best. It certainly beats the BASIC+ad-hoc-operational-semantics of ES1-3. Before choosing to write a definitional interpreter in SML and ES4, we (Dave Herman, really) looked hard at other options, including term rewriting systems, mostly Stratego but initially Maud too. But we always sought a mechanically testable spec -- not more by-hand, error- prone, and frankly hard to read at anything but a low "pretend I'm a Commodore 64 and execute the next (we hope not misnumbered) step" level. What I think would help implementors are, as you say, design notes or some such addenda to the overview doc that drill down by referencing pieces of the RI and illuminating them. Such addenda could answer the question "is this the only place I have to look at to understand (scope|namespace|...) rules?" /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 12:48 PM, Geoffrey Garen wrote: >>> I'd like Apple and the WebKit project to get involved with ES4 >>> implementation. But right now, as far as I can tell, there isn't a >>> written record for any of ES4's features that I could point an >>> engineer to and say "implement this". >> >> There's certainly no such spec, or you would be a passive observer >> of a standardization process that was all but done. That's not >> reality, and it arguably is not what you should want -- Apple people >> could be valued peers in the remaining work on ES4. >> >> If you want to be passive implementors of a finished spec, then wait >> till next year. > > I know that I, at least, don't want to be a passive implementor of a > finished spec. I want to be an active contributor to a spec in > progress. Great, glad to hear it. > However, since there is no spec -- at least, not one I can lay my > hands on Have you looked at the RI? > -- I'll have to wait until the "people in the know" finish > their implementations and then write a spec. C'mon, we're explicitly proposing an annealing process where we write specs or design notes, referencing the RI by all means but also illuminating it, and using prose where it's appropriate anyway -- where we won't be putting code in the final spec. We never proposed to implement first, then write a big spec. What's with all these straw men? /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
>> I'd like Apple and the WebKit project to get involved with ES4 >> implementation. But right now, as far as I can tell, there isn't a >> written record for any of ES4's features that I could point an >> engineer to and say "implement this". > > There's certainly no such spec, or you would be a passive observer > of a standardization process that was all but done. That's not > reality, and it arguably is not what you should want -- Apple people > could be valued peers in the remaining work on ES4. > > If you want to be passive implementors of a finished spec, then wait > till next year. I know that I, at least, don't want to be a passive implementor of a finished spec. I want to be an active contributor to a spec in progress. However, since there is no spec -- at least, not one I can lay my hands on -- I'll have to wait until the "people in the know" finish their implementations and then write a spec. That's OK with me, I guess, but it seems to bar collaboration in the short term. Once you've finished your implementation and written a spec about it, I hope you won't discount feedback by saying, "Well, our implementation is already done, and it's too late for major changes -- why didn't you give us this feedback earlier?" Geoff ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
Graydon, >> As a solution: I think we need an intermediate step. Not a spec, but >> some detailed design notes. Lar's document was a good >> overview, but drill down on exactly how the mechanisms are meant to >> work would be very helpful to broaden the base of >> implementations. Some examples of the things I'd like to see are: >> >> * Exactly how the scope rules work for various constructs >> * Order of initialization >> * More detail on namespace lookup rules >> * More detail on unqualified variable lookup >> * Type matching rules and conversions >> * More detail on nullability > > I see. I would have hoped these would be visible more or less directly > by inspection: > > * the scope rules and initialization order of each construct are > elaborated production-by-production in eval.sml > > * the namespace lookup scheme is elaborated in multiname.sml, > fixture.sml and defn.sml > > * the unqualified lookup rule starts in eval.sml > (evalLexicalRef -> evalRefExpr -> resolveOnScopeChain -> > Multiname.resolve) > > * the type matching and conversion rules begin with runtime judgments > in eval.sml (evalBinaryTypeOp) and then transition to type.sml > for their type-term-specific parts (groundMatchesGeneric and > findSpecialConversion) > > * the nullability system is more spread out, covering some part > of the type normalizer in type.sml and some part of the property > allocation scheme in eval.sml Thanks. Navigating the RI does take time and I'm pretty poor on ML. The above was not necessarily meant to be definitive, but rather an example of the kinds of things that crop up. Thanks for the tips anyway. > > I'm sad to hear that the RI was not sufficiently clear on these > points, but I guess this was a risk in the choice of language; SML is > terse but also not always familiar. > > Some of the SML code can be "lowered", via a prototype converter I > have written, to english. Notational details are erased but the names > and structure is preserved. Would this be useful? Another possibility > is to expand the internal commentary of the RI, or write some sort of > study guide. Inline comments may be better or easier. Separate documents are hard to maintain. Comments that say: "Here is where XYZ happens" would be helpful. As well as some overview comments that explain where various parts are implemented. What I find with the RI is that I can understand once piece, but I don't know if that is the sole place that implements or manages an issue. Take scope rules: an implementation can implement these in a variety of ways and places. So you don't know by looking at one piece of code if you have it all. Where the RI is great is to run test cases and see what it actually does. We do this with ASC and the RI quite a bit. > > I'm hesitant to commit to "yet another bit of plain english docs", > both because everyone who could write them already has lots to do, and > because there is nothing guaranteeing that such docs remain current or > honest. As we've seen, it's easy for an english doc -- such as a wiki > page -- to drift into obsolescence if there's not constant energy > spent updating it. Understand, but I've got no easy solution. My recommendation would be: - Add comments and overview inside the RI code itself. - Create an addendum to Lar's document that drills down on key topics to provide more exact operational semantics and details. > > -Graydon > ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
Michael O'Brien wrote: > As a solution: I think we need an intermediate step. Not a spec, but > some detailed design notes. Lar's document was a good > overview, but drill down on exactly how the mechanisms are meant to work > would be very helpful to broaden the base of > implementations. Some examples of the things I'd like to see are: > > * Exactly how the scope rules work for various constructs > * Order of initialization > * More detail on namespace lookup rules > * More detail on unqualified variable lookup > * Type matching rules and conversions > * More detail on nullability > > But there is a lot more. We can do this via Q&A, which is what we have > been doing, but it is hard and requires quite a bit of sleuthing. I see. I would have hoped these would be visible more or less directly by inspection: * the scope rules and initialization order of each construct are elaborated production-by-production in eval.sml * the namespace lookup scheme is elaborated in multiname.sml, fixture.sml and defn.sml * the unqualified lookup rule starts in eval.sml (evalLexicalRef -> evalRefExpr -> resolveOnScopeChain -> Multiname.resolve) * the type matching and conversion rules begin with runtime judgments in eval.sml (evalBinaryTypeOp) and then transition to type.sml for their type-term-specific parts (groundMatchesGeneric and findSpecialConversion) * the nullability system is more spread out, covering some part of the type normalizer in type.sml and some part of the property allocation scheme in eval.sml I'm sad to hear that the RI was not sufficiently clear on these points, but I guess this was a risk in the choice of language; SML is terse but also not always familiar. Some of the SML code can be "lowered", via a prototype converter I have written, to english. Notational details are erased but the names and structure is preserved. Would this be useful? Another possibility is to expand the internal commentary of the RI, or write some sort of study guide. I'm hesitant to commit to "yet another bit of plain english docs", both because everyone who could write them already has lots to do, and because there is nothing guaranteeing that such docs remain current or honest. As we've seen, it's easy for an english doc -- such as a wiki page -- to drift into obsolescence if there's not constant energy spent updating it. -Graydon ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
On Thu, 21 Feb 2008, Brendan Eich wrote: > > > > In contrast, with CSS, Web API or HTML WG specifications, I can point > > engineers to a spec that is more or less accurate for a given feature > > and they only have to ask questions about the few missing details. > > And then Hixie goes and rewrites it. I am calling b.s. here, Maciej. We > implemented offline web app support early in Firefox 3, based on such > WHAT-WG (ok, not HTML WG at the time) specs. They changed a great deal > later. In all fairness, they changed in response to feedback from the Mozilla implementors. To paraphrase you, if the spec didn't change, you would be a passive observer of a standardization process that was all but done. That's not reality, and it arguably is not what you should want -- Mozilla people are and hopefully will remain valued peers in the remaining work on HTML5. If you want to be passive implementors of a finished spec, then wait a few years. (Reply-To set to [EMAIL PROTECTED]) -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.' ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
If I may be so bold, I think Maciej is saying something similar to what I was saying, we need some more design notes to efficiently create implementations. Currently, we have a catch-22. We need a spec to efficiently create an implementation, but we won't have a spec until we have implementations. We can say: "let's all be manly and reverse engineer the RI" -- which is possible and certainly works. This is what we have been doing, along with some Q&A. But it is hard and requires quite a bit of sleuthing, particularly for those not familiar with ML. As a solution: I think we need an intermediate step. Not a spec, but some detailed design notes. Lar's document was a good overview, but drill down on exactly how the mechanisms are meant to work would be very helpful to broaden the base of implementations. Some examples of the things I'd like to see are: Exactly how the scope rules work for various constructs Order of initialization More detail on namespace lookup rules More detail on unqualified variable lookup Type matching rules and conversions More detail on nullability But there is a lot more. We can do this via Q&A, which is what we have been doing, but it is hard and requires quite a bit of sleuthing. Michael Brendan Eich wrote: On Feb 21, 2008, at 8:30 AM, Maciej Stachowiak wrote: I'd like Apple and the WebKit project to get involved with ES4 implementation. But right now, as far as I can tell, there isn't a written record for any of ES4's features that I could point an engineer to and say "implement this". There's certainly no such spec, or you would be a passive observer of a standardization process that was all but done. That's not reality, and it arguably is not what you should want -- Apple people could be valued peers in the remaining work on ES4. If you want to be passive implementors of a finished spec, then wait till next year. The proposals on the wiki are way out of date, it's not easy to find what trac tickets modified them, and there seem to be commonly understood planned changes that aren't even reflected in trac. That's a failure to file trac tickets -- could you please list these changes that aren't in the trac? There's no other bug system to track these planned changes, so they had better show up at http://bugs.ecmascript.org/ soon or they won't happen. Before attempting interoperable implementations of particular features, I think we need at minimum a form of the proposal for that feature that is complete and up to date. It doesn't have to be formal specification quality, but there has to be something accurate. I've worked pretty hard to keep proposals such as iterators and generators up to date; it depends on other proposals which are also not formal spec quality, but stable and meaningful (structural types, type parameters). Cormac has done work recently in formalizing the type system which was important to Graydon's RI work. So I think you are generalizing unfairly here. It's true that decimal is out of date in the wiki, and there are open trac issues. This is true of other proposals. Now, it may be that by telling someone to reverse engineer another implementation, or ask the ES4 crowd about every detail of how a feature should be implemented, someone could succeed in implementing. Nice strawmen, but no one proposed those things. But it seems to me that this undermines the unstated assumption of interoperable *independent* implementations. In contrast, with CSS, Web API or HTML WG specifications, I can point engineers to a spec that is more or less accurate for a given feature and they only have to ask questions about the few missing details. And then Hixie goes and rewrites it. I am calling b.s. here, Maciej. We implemented offline web app support early in Firefox 3, based on such WHAT-WG (ok, not HTML WG at the time) specs. They changed a great deal later. I would raise HTML5 as a particularly laudable example because it achieves this even though much implementation work is happening in parallel with writing the spec. You are misrepresenting what has actually happened there. I think we should strive to achieve the same standard for ES4. At feature granularity, someone should first write an up to date accurate document and implementations should be done against that, not against some separate shared understanding of the feature. That's the plan -- see Jeff's paragraph about "feature specs" which I cited in reply to Geoff. /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss ___ Es4-discuss mail
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 8:30 AM, Maciej Stachowiak wrote: I'd like Apple and the WebKit project to get involved with ES4 implementation. But right now, as far as I can tell, there isn't a written record for any of ES4's features that I could point an engineer to and say "implement this". There's certainly no such spec, or you would be a passive observer of a standardization process that was all but done. That's not reality, and it arguably is not what you should want -- Apple people could be valued peers in the remaining work on ES4. If you want to be passive implementors of a finished spec, then wait till next year. The proposals on the wiki are way out of date, it's not easy to find what trac tickets modified them, and there seem to be commonly understood planned changes that aren't even reflected in trac. That's a failure to file trac tickets -- could you please list these changes that aren't in the trac? There's no other bug system to track these planned changes, so they had better show up at http:// bugs.ecmascript.org/ soon or they won't happen. Before attempting interoperable implementations of particular features, I think we need at minimum a form of the proposal for that feature that is complete and up to date. It doesn't have to be formal specification quality, but there has to be something accurate. I've worked pretty hard to keep proposals such as iterators and generators up to date; it depends on other proposals which are also not formal spec quality, but stable and meaningful (structural types, type parameters). Cormac has done work recently in formalizing the type system which was important to Graydon's RI work. So I think you are generalizing unfairly here. It's true that decimal is out of date in the wiki, and there are open trac issues. This is true of other proposals. Now, it may be that by telling someone to reverse engineer another implementation, or ask the ES4 crowd about every detail of how a feature should be implemented, someone could succeed in implementing. Nice strawmen, but no one proposed those things. But it seems to me that this undermines the unstated assumption of interoperable *independent* implementations. In contrast, with CSS, Web API or HTML WG specifications, I can point engineers to a spec that is more or less accurate for a given feature and they only have to ask questions about the few missing details. And then Hixie goes and rewrites it. I am calling b.s. here, Maciej. We implemented offline web app support early in Firefox 3, based on such WHAT-WG (ok, not HTML WG at the time) specs. They changed a great deal later. I would raise HTML5 as a particularly laudable example because it achieves this even though much implementation work is happening in parallel with writing the spec. You are misrepresenting what has actually happened there. I think we should strive to achieve the same standard for ES4. At feature granularity, someone should first write an up to date accurate document and implementations should be done against that, not against some separate shared understanding of the feature. That's the plan -- see Jeff's paragraph about "feature specs" which I cited in reply to Geoff. /be___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
Maciej Stachowiak wrote:
> To expand a bit on Geoff's comment:
>
> I'd like Apple and the WebKit project to get involved with ES4
> implementation.
Great! Though please keep in mind a point in the remainder of your
comments: WebKit (and Rhino) are operating from a somewhat "newcomer"
perspective, relative to the committee. The other 5 implementations in
question (RI, spidermonkey, mbedthis, futhark, ESC+tamarin) are all
written by engineers who are and have been closely following the
tickets, proposals and discussion, and modifying the RI to encode their
thoughts / experiment with a feature.
So the implication that the language "designers" are a disjoint set from
the "implementors", or that they haven't been writing their thoughts
down, is not historically accurate. If that's becoming more true now,
ok, maybe we need to make some adjustments. But understand where we're
coming from.
> Before attempting interoperable implementations of particular
> features, I think we need at minimum a form of the proposal for that
> feature that is complete and up to date. It doesn't have to be formal
> specification quality, but there has to be something accurate.
I agree this would be nice, but it'd also be nice to have 9 finished
implementations and a finished spec! We don't have these yet. So: is
your team *completely* stalled until we have such documents, presumably
in english rather than SML? If so, I (or anyone else who understands the
issues clearly enough -- they're not tremendously complex) can make a
priority of building up foundational "implementors documents" covering
such basic concepts as namespaces, names, multinames, types and
fixtures. I think we had hoped the RI and tickets on it to serve this role.
> Now, it may be that by telling someone to reverse engineer another
> implementation, or ask the ES4 crowd about every detail of how a
> feature should be implemented, someone could succeed in implementing.
> But it seems to me that this undermines the unstated assumption of
> interoperable *independent* implementations.
I do not think it undermines the assumption of independent
implementations, but I also don't think there's a perfectly clear line
between "dependent" and "independent". Impls inform themselves from
somewhere, be it spec or spec-that-is-informed-from-other-impls or other
technical reference material. Information flows somehow, and often
between impls (even if indirectly).
You're not going to be doing WebKit by studying Futhark or Spidermonkey;
but I *would* recommend studying the RI (and contributing to it!) I
would not worry greatly about the risk of being "dependent" on it, since
it is in a very different language (SML) than WebKit's interpreter and
is surely structured quite differently. Study it and understand it,
though, as it's as precise as we currently get. The RI was meant to be
studied (a.k.a. "reverse engineered") and the risk of overspecificity
from that is something we all explicitly agreed was better than the risk
of deploying underspecified and incompatible impls to the field.
> In contrast, with CSS, Web API or HTML WG specifications, I can point
> engineers to a spec that is more or less accurate for a given feature
> and they only have to ask questions about the few missing details. I
> would raise HTML5 as a particularly laudable example because it
> achieves this even though much implementation work is happening in
> parallel with writing the spec.
HTML5 is a laudable example, and I hope we wind up producing something
of similar quality. It has also had more energy put into it, more eyes
on it, and is a much wider and flatter spec (fewer subtle interlocking
issues).
Web browsers are also stunningly more complex than programming
languages, so the concept of a "reference implementation" is completely
fanciful (though you could do a reference implementation of the parsing
rules, say).
The ES4 RI is small and digestible. Discounting the builtins and some
obvious support code, here is the guts of it according to wc -l:
595 ast.sml
134 multiname.sml
923 lexer.sml
7468 parser.sml
2687 defn.sml
1008 verify.sml
5558 eval.sml
913 type.sml
1254 mach.sml
20540 total
This is not a big program, it's really small enough to study and
participate in. I'd argue that you'd even be in a good place simply
reading ast, multiname, eval, type and mach (8.5 kloc). It's intended to
be a place for implementors to encode their thoughts and experiments,
refer back to, write canonical algorithms in, etc. Don't be shy about
using it that way. It's public source. It was intended to be
illustrative ("how is this supposed to even work") rather than simply
consultative ("what is the magic ES4 oracle answer to this").
(We're probably going to shift it out of monotone and into mercurial
soon, which will make it both easier to fetch and more web-accessible too).
Is this an unsatisfactory ans
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 8:14 AM, Geoffrey Garen wrote: > Is there a published specification that all these implementors will > be using? We're implementing from proposals augmented by trac tickets and other docs including Cormac's formalization of the type system, plus existing extended implementations (which may need to evolve in parallel), at the same time as we will be writing specs. Jeff's mail talked about this: > Feature spec - one of the participants from the implementation team > writes > up a description of the feature based on that implementation. > Feature specs > are complete and accurate descriptions of individual features to be > added to > the language. They are supported by production implementation, test > cases > and implementer commitment to release the feature. But then Graydon pointed out the dependencies among "features" requiring partial order of work, and care about getting foundational work done first. The RI has been through this, so it serves as a proto-spec in some says (namespaces and the multiname algorithm, structural types and type parameters in part). As noted, we do not plan on mythic waterfall development with specs entirely done before code -- more like straw/iron/diamond-man specs interleaved with implementation, for the hard cases and the full spec. We have many strawman specs, but not a rigorous and formal integrated spec. There are indeed easy cases of mini-specs, not foundational items, that can be written pretty completely ahead of implementation, don't get me wrong. But the totality of the language spec won't be done until we have some implementations interoperating, and the RI matching. /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
Re: ES4 implementation process, teams, and meetings
On Feb 21, 2008, at 8:14 AM, Geoffrey Garen wrote: > Is there a published specification that all these implementors will be > using? To expand a bit on Geoff's comment: I'd like Apple and the WebKit project to get involved with ES4 implementation. But right now, as far as I can tell, there isn't a written record for any of ES4's features that I could point an engineer to and say "implement this". The proposals on the wiki are way out of date, it's not easy to find what trac tickets modified them, and there seem to be commonly understood planned changes that aren't even reflected in trac. Before attempting interoperable implementations of particular features, I think we need at minimum a form of the proposal for that feature that is complete and up to date. It doesn't have to be formal specification quality, but there has to be something accurate. Now, it may be that by telling someone to reverse engineer another implementation, or ask the ES4 crowd about every detail of how a feature should be implemented, someone could succeed in implementing. But it seems to me that this undermines the unstated assumption of interoperable *independent* implementations. In contrast, with CSS, Web API or HTML WG specifications, I can point engineers to a spec that is more or less accurate for a given feature and they only have to ask questions about the few missing details. I would raise HTML5 as a particularly laudable example because it achieves this even though much implementation work is happening in parallel with writing the spec. I think we should strive to achieve the same standard for ES4. At feature granularity, someone should first write an up to date accurate document and implementations should be done against that, not against some separate shared understanding of the feature. Regards, Maciej > > > Thanks, > Geoff > > On Feb 20, 2008, at 3:38 PM, Brendan Eich wrote: > >> As Jeff has laid out, with helpful comments from Michael O'Brien, >> Lars, and Graydon, we are entering a phase of ES4 work where >> practical implementations will adopt and implement proposed parts of >> the new language. We need to do this to shake out proposals and gain >> critical feedback from implementors. We hope to get usability results >> from programmers too. >> >> I agree with Michael's point about the RI being both alpha and omega >> among implementations, so RI work will continue. But practical >> implementations, besides enabling usability testing with real >> programmers, will help weed out potential problems to do with >> performance and security that the RI blissfully ignores. >> >> As Graydon and Michael point out, the waterfall diagram (even if you >> put the RI earlier in the waterfall) does not reflect the wheels >> within wheels (waterwheels?) that must cycle at several levels of >> design, implementation, and even usability testing, in order to reach >> the ES4 spec we aspire to write. So take that waterfall diagram more >> as a management crutch ;-). >> >> Finally, we absolutely aspire to build our existing testsuites up to >> cover as much of the new language as we can. Test-driven development >> is the only way to go (I speak from painful experience back in the >> Netscape days :-/). >> >> The good news is that I believe we will have many ES4 implementations >> coming up in the next few months, working in parallel to improve the >> spec and RI. I know of at least these already under way: >> >> * ES4 RI (SML + ES4 self-hosted) >> * MbedThis (C + Java) >> * Rhino (Java) >> * SpiderMonkey (C) >> * Tamarin+ESC (C++ + ES4 self-hosted) >> >> If you are implementing any part of ES4 and want to join forces, >> please reply. >> >> We aim to track progress using the infrastructure created by John >> Resig: >> >> http://ejohn.org/blog/state-of-ecmascript-4-dec-07/ >> http://spreadsheets.google.com/pub?key=pFIHldY_CkszsFxMkQOReAQ&gid=2 >> >> I believe that the shared spreadsheet URL given above is correct, but >> John can provide the latest information as well as grant write >> access. My hope is that implementors can edit the spreadsheet to >> record progress, providing verifiable builds and even open source for >> their implementations, as they go. Again I'll defer to John on this. >> >> We propose to communicate among implementation teams using es4- >> [EMAIL PROTECTED], since (a) the list is not terribly high-traffic, >> (b) we aim to operate transparently, and (c) we believe most of you >> are interested at least as onlookers, if not as implementors. We can >> split lists if we run into a problem, but I don't foresee one. >> >> To provide focused face-to-face time working together and building >> rapport among the principals, we are thinking of meeting roughly >> every month, with this strawman schedule: >> >> March 17-21 - Mountain View, CA >> April 14-18 - Newton, MA >> May 12-16 - Vancouver, BC >> >> This is very straw, so please don't flame it or it will ignite! But >>
Re: ES4 implementation process, teams, and meetings
Is there a published specification that all these implementors will be
using?
Thanks,
Geoff
On Feb 20, 2008, at 3:38 PM, Brendan Eich wrote:
> As Jeff has laid out, with helpful comments from Michael O'Brien,
> Lars, and Graydon, we are entering a phase of ES4 work where
> practical implementations will adopt and implement proposed parts of
> the new language. We need to do this to shake out proposals and gain
> critical feedback from implementors. We hope to get usability results
> from programmers too.
>
> I agree with Michael's point about the RI being both alpha and omega
> among implementations, so RI work will continue. But practical
> implementations, besides enabling usability testing with real
> programmers, will help weed out potential problems to do with
> performance and security that the RI blissfully ignores.
>
> As Graydon and Michael point out, the waterfall diagram (even if you
> put the RI earlier in the waterfall) does not reflect the wheels
> within wheels (waterwheels?) that must cycle at several levels of
> design, implementation, and even usability testing, in order to reach
> the ES4 spec we aspire to write. So take that waterfall diagram more
> as a management crutch ;-).
>
> Finally, we absolutely aspire to build our existing testsuites up to
> cover as much of the new language as we can. Test-driven development
> is the only way to go (I speak from painful experience back in the
> Netscape days :-/).
>
> The good news is that I believe we will have many ES4 implementations
> coming up in the next few months, working in parallel to improve the
> spec and RI. I know of at least these already under way:
>
> * ES4 RI (SML + ES4 self-hosted)
> * MbedThis (C + Java)
> * Rhino (Java)
> * SpiderMonkey (C)
> * Tamarin+ESC (C++ + ES4 self-hosted)
>
> If you are implementing any part of ES4 and want to join forces,
> please reply.
>
> We aim to track progress using the infrastructure created by John
> Resig:
>
> http://ejohn.org/blog/state-of-ecmascript-4-dec-07/
> http://spreadsheets.google.com/pub?key=pFIHldY_CkszsFxMkQOReAQ&gid=2
>
> I believe that the shared spreadsheet URL given above is correct, but
> John can provide the latest information as well as grant write
> access. My hope is that implementors can edit the spreadsheet to
> record progress, providing verifiable builds and even open source for
> their implementations, as they go. Again I'll defer to John on this.
>
> We propose to communicate among implementation teams using es4-
> [EMAIL PROTECTED], since (a) the list is not terribly high-traffic,
> (b) we aim to operate transparently, and (c) we believe most of you
> are interested at least as onlookers, if not as implementors. We can
> split lists if we run into a problem, but I don't foresee one.
>
> To provide focused face-to-face time working together and building
> rapport among the principals, we are thinking of meeting roughly
> every month, with this strawman schedule:
>
> March 17-21 - Mountain View, CA
> April 14-18 - Newton, MA
> May 12-16 - Vancouver, BC
>
> This is very straw, so please don't flame it or it will ignite! But
> please do feel free to suggest alternative dates and locations. We
> hope to host anyone who has known reputation on this list and who
> wants to help, or who is already implementing. Jon Zeppieri has
> already helped with the RI, for example, and he's welcome to attend.
>
> More details on the meetings as we firm things up.
>
> Anxiety about commercial pre-release implementations gaining undue
> influence over the proposed standard naturally arises, and I wanted
> to address this briefly:
>
> Speaking for Mozilla, we do not intend to throw trump cards during
> this new phase of proposed-ES4 development based on implementations.
> We've implemented extensions in the past, some of which are popular,
> others not so popular. We carry what we create until we can drop it,
> ideally in favor of a standardized form that's an actual improvement.
> This has gone on since Netscape days.
>
> We try not to mess around with marginal extensions, so the few
> mistakes we're still carrying, e.g. the magic __proto__
> Object.prototype property, in both its read and write modes, reflect
> usability gaps in the language. For example, __proto__ is useful in
> the absence of ES4's Map to make an object be a "hash":
>
> var obj = {__proto__:null, "key1":val1, ... "keyN":valN};
>
> and that use-case should be better served by something like ES4's Map
> proposal. But as noted in the thread that started here, __proto__ is
> not proposed in any way for inclusion in ES4.
>
> I bet the other implementators cited above would make a similar "no
> trump cards" pledge. The point of this proposed-ES4 implementation
> exercise is not to develop and deploy production code to the Web,
> rather to gain crucial feedback from implementors and programmers --
> a kind of "open source" or "permanent beta" approach to
> standardization, where interoperation among ES4 impleme
Re: ES4 implementation process, teams, and meetings
On Feb 20, 2008, at 3:38 PM, Brendan Eich wrote: > But as noted in the thread that started here, __proto__ is > not proposed in any way for inclusion in ES4. Er, here: https://mail.mozilla.org/pipermail/es4-discuss/2007- September/001108.html /be ___ Es4-discuss mailing list Es4-discuss@mozilla.org https://mail.mozilla.org/listinfo/es4-discuss
ES4 implementation process, teams, and meetings
As Jeff has laid out, with helpful comments from Michael O'Brien,
Lars, and Graydon, we are entering a phase of ES4 work where
practical implementations will adopt and implement proposed parts of
the new language. We need to do this to shake out proposals and gain
critical feedback from implementors. We hope to get usability results
from programmers too.
I agree with Michael's point about the RI being both alpha and omega
among implementations, so RI work will continue. But practical
implementations, besides enabling usability testing with real
programmers, will help weed out potential problems to do with
performance and security that the RI blissfully ignores.
As Graydon and Michael point out, the waterfall diagram (even if you
put the RI earlier in the waterfall) does not reflect the wheels
within wheels (waterwheels?) that must cycle at several levels of
design, implementation, and even usability testing, in order to reach
the ES4 spec we aspire to write. So take that waterfall diagram more
as a management crutch ;-).
Finally, we absolutely aspire to build our existing testsuites up to
cover as much of the new language as we can. Test-driven development
is the only way to go (I speak from painful experience back in the
Netscape days :-/).
The good news is that I believe we will have many ES4 implementations
coming up in the next few months, working in parallel to improve the
spec and RI. I know of at least these already under way:
* ES4 RI (SML + ES4 self-hosted)
* MbedThis (C + Java)
* Rhino (Java)
* SpiderMonkey (C)
* Tamarin+ESC (C++ + ES4 self-hosted)
If you are implementing any part of ES4 and want to join forces,
please reply.
We aim to track progress using the infrastructure created by John Resig:
http://ejohn.org/blog/state-of-ecmascript-4-dec-07/
http://spreadsheets.google.com/pub?key=pFIHldY_CkszsFxMkQOReAQ&gid=2
I believe that the shared spreadsheet URL given above is correct, but
John can provide the latest information as well as grant write
access. My hope is that implementors can edit the spreadsheet to
record progress, providing verifiable builds and even open source for
their implementations, as they go. Again I'll defer to John on this.
We propose to communicate among implementation teams using es4-
[EMAIL PROTECTED], since (a) the list is not terribly high-traffic,
(b) we aim to operate transparently, and (c) we believe most of you
are interested at least as onlookers, if not as implementors. We can
split lists if we run into a problem, but I don't foresee one.
To provide focused face-to-face time working together and building
rapport among the principals, we are thinking of meeting roughly
every month, with this strawman schedule:
March 17-21 - Mountain View, CA
April 14-18 - Newton, MA
May 12-16 - Vancouver, BC
This is very straw, so please don't flame it or it will ignite! But
please do feel free to suggest alternative dates and locations. We
hope to host anyone who has known reputation on this list and who
wants to help, or who is already implementing. Jon Zeppieri has
already helped with the RI, for example, and he's welcome to attend.
More details on the meetings as we firm things up.
Anxiety about commercial pre-release implementations gaining undue
influence over the proposed standard naturally arises, and I wanted
to address this briefly:
Speaking for Mozilla, we do not intend to throw trump cards during
this new phase of proposed-ES4 development based on implementations.
We've implemented extensions in the past, some of which are popular,
others not so popular. We carry what we create until we can drop it,
ideally in favor of a standardized form that's an actual improvement.
This has gone on since Netscape days.
We try not to mess around with marginal extensions, so the few
mistakes we're still carrying, e.g. the magic __proto__
Object.prototype property, in both its read and write modes, reflect
usability gaps in the language. For example, __proto__ is useful in
the absence of ES4's Map to make an object be a "hash":
var obj = {__proto__:null, "key1":val1, ... "keyN":valN};
and that use-case should be better served by something like ES4's Map
proposal. But as noted in the thread that started here, __proto__ is
not proposed in any way for inclusion in ES4.
I bet the other implementators cited above would make a similar "no
trump cards" pledge. The point of this proposed-ES4 implementation
exercise is not to develop and deploy production code to the Web,
rather to gain crucial feedback from implementors and programmers --
a kind of "open source" or "permanent beta" approach to
standardization, where interoperation among ES4 implementations can
be demonstrated with tests and real code before the standard is
finalized.
Any questions? Feel free to reply-all.
/be
___
Es4-discuss mailing list
Es4-discuss@mozil
