Re: natively negotiating sync vs. async...without callbacks
I pretty much abandoned that line of investigation with the conclusion that
generators:
http://wiki.ecmascript.org/doku.php?id=strawman:generators
are a good (and well-tested, in Python and SpiderMonkey) design for
single-frame continuations. They hang together well; in particular, they don't
have the issues with `finally' that some of the alternatives I talked about do.
Moreover, the continuation-capture mechanisms based on call/cc or shift/reset
require additional power in the VM to essentially tail-call their argument
expression. When I tried prototyping this in SpiderMonkey, I found this to be
one of the biggest challenges -- and that was just in the straight-up
interpreter, not in the tracing JIT or method JIT.
Generators work well for lightweight concurrency. As a proof of concept, I put
together a little library of "tasks" based on generators:
http://github.com/dherman/jstask
Somebody reminded me that Neil Mix had written a very similar library several
years ago, called Thread.js:
http://www.neilmix.com/2007/02/07/threading-in-javascript-17/
and there's another library called Er.js that built off of that to create some
Erlang-like abstractions:
http://www.beatniksoftware.com/erjs/
Dave
On Dec 8, 2010, at 11:36 PM, Tom Van Cutsem wrote:
> The spirit of the proposal is that this special type of statement be a linear
> sequence of function executions (as opposed to nested function-reference
> callbacks delegating execution to other code).
>
> The special behavior is that in between each part/expression of the
> statement, evaluation of the statement itself (NOT the rest of the program)
> may be "suspended" until the previous part/expression is fulfilled. This
> would conceptually be like a yield/continuation localized to ONLY the
> statement in question, not affecting the linear execution of the rest of the
> program.
>
> This reminds me of a proposal by Kris Zyp a couple of months ago ("single
> frame continuations")
> https://mail.mozilla.org/pipermail/es-discuss/2010-March/010865.html
>
> I don't think that discussion lead to a clear outcome, but it's definitely
> related, both in terms of goals as well as in mechanism.
> I also recall it prompted Dave Herman to sketch the design space of
> (single-frame) continuations for JS:
> https://mail.mozilla.org/pipermail/es-discuss/2010-April/010894.html
>
> Cheers,
> Tom
> ___
> es-discuss mailing list
> es-discuss@mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: natively negotiating sync vs. async...without callbacks
PS To be concrete, here's an example code snippet using my jstask library that
chains several event-generated actions together in a natural way (i.e., in
direct style, i.e. not in CPS):
var task = new Task(function() {
var request = new HttpRequest();
try {
var foo = yield request.send(this, "foo.json");
var bar = yield request.send(this, "bar.json");
var baz = yield request.send(this, "baz.json");
} catch (errorResponse) {
console.log("failed HTTP request: " + errorResponse.statusText);
}
... foo.responseText ... bar.responseText ... baz.responseText ...
});
I should also point out that the core of jstask is 7 lines of code. :)
Dave
On Dec 9, 2010, at 7:55 AM, David Herman wrote:
> I pretty much abandoned that line of investigation with the conclusion that
> generators:
>
> http://wiki.ecmascript.org/doku.php?id=strawman:generators
>
> are a good (and well-tested, in Python and SpiderMonkey) design for
> single-frame continuations. They hang together well; in particular, they
> don't have the issues with `finally' that some of the alternatives I talked
> about do. Moreover, the continuation-capture mechanisms based on call/cc or
> shift/reset require additional power in the VM to essentially tail-call their
> argument expression. When I tried prototyping this in SpiderMonkey, I found
> this to be one of the biggest challenges -- and that was just in the
> straight-up interpreter, not in the tracing JIT or method JIT.
>
> Generators work well for lightweight concurrency. As a proof of concept, I
> put together a little library of "tasks" based on generators:
>
> http://github.com/dherman/jstask
>
> Somebody reminded me that Neil Mix had written a very similar library several
> years ago, called Thread.js:
>
> http://www.neilmix.com/2007/02/07/threading-in-javascript-17/
>
> and there's another library called Er.js that built off of that to create
> some Erlang-like abstractions:
>
> http://www.beatniksoftware.com/erjs/
>
> Dave
>
> On Dec 8, 2010, at 11:36 PM, Tom Van Cutsem wrote:
>
>> The spirit of the proposal is that this special type of statement be a
>> linear sequence of function executions (as opposed to nested
>> function-reference callbacks delegating execution to other code).
>>
>> The special behavior is that in between each part/expression of the
>> statement, evaluation of the statement itself (NOT the rest of the program)
>> may be "suspended" until the previous part/expression is fulfilled. This
>> would conceptually be like a yield/continuation localized to ONLY the
>> statement in question, not affecting the linear execution of the rest of the
>> program.
>>
>> This reminds me of a proposal by Kris Zyp a couple of months ago ("single
>> frame continuations")
>> https://mail.mozilla.org/pipermail/es-discuss/2010-March/010865.html
>>
>> I don't think that discussion lead to a clear outcome, but it's definitely
>> related, both in terms of goals as well as in mechanism.
>> I also recall it prompted Dave Herman to sketch the design space of
>> (single-frame) continuations for JS:
>> https://mail.mozilla.org/pipermail/es-discuss/2010-April/010894.html
>>
>> Cheers,
>> Tom
>> ___
>> es-discuss mailing list
>> es-discuss@mozilla.org
>> https://mail.mozilla.org/listinfo/es-discuss
>
> ___
> es-discuss mailing list
> es-discuss@mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: natively negotiating sync vs. async...without callbacks
It might surprise some who know me to hear this, but I agree with Dave on this. There's a huge gap between the single-frame mechanism and going the whole hog. Going all out does buy you some expressive power, but at the cost of complicating everything. If the simple mechanism gives you most of what you want (and I feel it goes pretty far), why commit to a lifetime of pain? Shriram ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: natively negotiating sync vs. async...without callbacks
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
The generators from JS 1.7 are too specific to provide much help with
promises, IMO. Adding a yield operator fundamentally alters the
semantics of the entire surrounding function, violating the principle
of locality. Consequently you need special mechanisms (like Neil's
library) to even call an async function, and you can't "return" values
either (with the return operator). Solving these problems was the
point behind the shallow continuation proposal that we worked on. Does
the generator strawman build on that work, or does it still fail to
preserve locality?
My understanding of Kyle's proposal was that he did not want to
introduce real continuations, but this was more of a syntax for a
specialized callback chain, reducing the verbosity of anon functions,
callback registration. I'd prefer more of a generalized solution for
reducing the size of anon functions (which has been discussed to great
lengths here in the lambda threads). However, my belief on async is
that the verbosity is only part of the pain, working with async within
various control flows (loops, branches, etc) is more of a burden,
hence the need for continuations (shallow/single frame, of course).
Kris
On 12/9/2010 8:55 AM, David Herman wrote:
> I pretty much abandoned that line of investigation with the
> conclusion that generators:
>
> http://wiki.ecmascript.org/doku.php?id=strawman:generators
>
> are a good (and well-tested, in Python and SpiderMonkey) design for
> single-frame continuations. They hang together well; in particular,
> they don't have the issues with `finally' that some of the
> alternatives I talked about do. Moreover, the continuation-capture
> mechanisms based on call/cc or shift/reset require additional power
> in the VM to essentially tail-call their argument expression. When I
> tried prototyping this in SpiderMonkey, I found this to be one of
> the biggest challenges -- and that was just in the straight-up
> interpreter, not in the tracing JIT or method JIT.
>
> Generators work well for lightweight concurrency. As a proof of
> concept, I put together a little library of "tasks" based on generators:
>
> http://github.com/dherman/jstask
>
> Somebody reminded me that Neil Mix had written a very similar
> library several years ago, called Thread.js:
>
> http://www.neilmix.com/2007/02/07/threading-in-javascript-17/
>
> and there's another library called Er.js that built off of that to
> create some Erlang-like abstractions:
>
> http://www.beatniksoftware.com/erjs/
>
> Dave
>
> On Dec 8, 2010, at 11:36 PM, Tom Van Cutsem wrote:
>
>> The spirit of the proposal is that this special type of
>> statement be a linear sequence of function executions (as
>> opposed to nested function-reference callbacks delegating
>> execution to other code).
>>
>> The special behavior is that in between each part/expression of
>> the statement, evaluation of the statement itself (NOT the rest
>> of the program) may be "suspended" until the previous
>> part/expression is fulfilled. This would conceptually be like a
>> yield/continuation localized to ONLY the statement in question,
>> not affecting the linear execution of the rest of the program.
>>
>>
>> This reminds me of a proposal by Kris Zyp a couple of months ago
>> ("single frame continuations")
>> https://mail.mozilla.org/pipermail/es-discuss/2010-March/010865.html
>>
>> I don't think that discussion lead to a clear outcome, but it's
>> definitely related, both in terms of goals as well as in mechanism.
>> I also recall it prompted Dave Herman to sketch the design space of
>> (single-frame) continuations for JS:
>> https://mail.mozilla.org/pipermail/es-discuss/2010-April/010894.html
>>
>> Cheers,
>> Tom
>> ___
>> es-discuss mailing list
>> es-discuss@mozilla.org
>> https://mail.mozilla.org/listinfo/es-discuss
>
>
>
> ___
> es-discuss mailing list
> es-discuss@mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss
- --
Kris Zyp
SitePen
(503) 806-1841
http://sitepen.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk0BA5kACgkQ9VpNnHc4zAy7nwCeJxL8Or+BUkYzfAi46EKEQG+O
nGEAn0nCErWiI5mbunUwD860Czeof1bt
=fHWD
-END PGP SIGNATURE-
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: natively negotiating sync vs. async...without callbacks
>
> var task = new Task(function() {
> var request = new HttpRequest();
> try {
> var foo = yield request.send(this, "foo.json");
> var bar = yield request.send(this, "bar.json");
> var baz = yield request.send(this, "baz.json");
> } catch (errorResponse) {
> console.log("failed HTTP request: " +
> errorResponse.statusText);
> }
> ... foo.responseText ... bar.responseText ... baz.responseText ...
> });
>
This style is similar to what I'm proposing, except for a few important
things:
1. I'm not trying to open the can-o'-worms around block level changes. The
above code suggests that a 'yield' suspension of execution is local to the
nearest container { } block, in this case the try { } block. While I
wouldn't be strongly opposed to it, it seems a little unnatural to have to
wrap a collection of like-behaving stuff in a { }... if that were the case,
just use an anonymous function (without or without the shorter syntax) and
not have to change the meaning of { } blocks. But I can see some merit in
this concept. Just not hugely in favor of it.
2. It breaks a fundamental thing that *I* happen to think is really
important... it puts the control of whether a function is yielded or not
into the hands of the calling code. I happen to think the function call
itself should get to decide if he wants to yield asynchronously, or finish
synchronously.
For instance, imagine a function that retrieves a value from some
"database", but employs some memoization/local caching. In the first call
case, that function will need to return asynchronously. But in the cached
case, that function can and should return immediately. It'd be nice for the
function to get to decide conditionally if it wants to "yield" or "defer"
itself based on that state.
And the calling code, by virtue of something like the native operator I
suggested, can be written just one way and not care about that hidden
detail.
lookupDBValue("something") @ printValue();
For async, `printValue` will be held up from executing until `lookupDBValue`
signals that it's complete by fulfilling its promise. For sync,
`lookupDBValue` will already have its promise fulfilled at the time it
returns, so the @ can move to immediately/synchronously executing
`printValue`.
--Kyle
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: RegExp syntax suggestion: allow CharacterClassEscape in CharacterRange
On Wed, 08 Dec 2010 21:43:06 +0100, Gavin Barraclough
wrote:
According to the ES5 spec a regular expression such as /[\w-_]/ should
generate a syntax error. Unfortunately there appears to be a
significant quantity of existing code that will break if this behavior
is implemented (I have been experimenting with bringing WebKit's RegExp
implementation into closer conformance to the spec), and looking at
other implementations it appears common for this error to be ignored.
It's far from the only extension to RegExp syntax that is common to most
implementations. In fact, the extensions are both extensive and consistent
across browsers. A quick check through the possible syntax errors show
the following:
// Invalid ControlEscape/IdentityEscape character treated as literal.
/\z/; // Invalid escape, same as /z/
// Incomplete/Invalid ControlEscape treated as either "\\c" or "c"
/\c/; // same as /c/ or /\\c/
/\c2/; // same as /c2/ or /\\c2/
// Incomplete HexEscapeSequence escape treated as either "\\x" or "x".
/\x/; // incomplete x-escape
/\x1/; // incomplete x-escape
/\x1z/; // incomplete x-escape
// Incomplete UnicodeEscapeSequence escape treated as either "\\u" or "u".
/\u/; // incomplete u-escape
/\uz/; // incomplete u-escape
/\u1/; // incomplete u-escape
/\u1z/; // incomplete u-escape
/\u12/; // incomplete u-escape
/\u12z/; // incomplete u-escape
/\u123/; // incomplete u-escape
/\u123z/; // incomplete u-escape
// Bad quantifier range:
/x{z/; // same as /x\{z/
/x{1z/; // same as /x\{1z/
/x{1,z/; // same as /x\{1,z/
/x{1,2z/; // same as /x\{1,2z/
/x{1,2z/; // same as /x\{1,2z/
// Notice: It needs arbitrary lookahead to determine the invalidity,
// except Mozilla that limits the numbers.
// Zero-initialized Octal escapes.
/\012/;// same as /\x0a/
// Nonexisting back-references treated as octal escapes:
/\5/; // same as /\x05/
// Invalid PatternCharacter accepted unescaped
/]/;
/{/;
/}/;
// Bad escapes also inside CharacterClass.
/[\z]/;
/[\c]/;
/[\c2]/;
/[\x]/;
/[\x1]/;
/[\x1z]/;
/[\u]/;
/[\uz]/;
/[\u1]/;
/[\u1z]/;
/[\u12]/;
/[\u12z]/;
/[\u123]/;
/[\u123z]/;
/[\012]/;
/[\5]/;
// And in addition:
/[\B]/;
/()()[\2]/; // Valid backreference should be invalid.
None of these RegExps cause a syntax error in any of the current "top-5"
browsers,
even though they are (AFAICS) invalid syntax.
Most of the RegExps treat a malformed (start of a multi-character) escape
sequence
as a simple identity escape or octal escape, and extends identity escapes
to all characters
that doesn't already have another meaning (ControlEscape,
CharacterClassEscape or
one of c, x, u, or b, and B outside a CharacterClass).
To match the current behavior, IdentityEscape shouldn't exclude all of
IdentifierPart,
but only the characters that already mean something else.
Allowing /\c2/ to match "c2", but requiring /\CB/ to match "\x02" seems
like it would
be better explained in prose than in the BNF.
...
I'd like to propose a minimal change to hopefully allow implementations
to come into line with the spec, without breaking the web. I'd suggest
changing the first step of CharacterRange to instead read:
1. If A does not contain exactly one character or B does not contain
exactly one character then create a CharSet AB containing the union of
the CharSets A and B, and return the union of CharSet AB and the CharSet
containing the one character -.
I think this matches the current actual behavior of all the browsers, and
is
short and understandable.
/Lasse R.H. Nielsen
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: natively negotiating sync vs. async...without callbacks
>
> This reminds me of a proposal by Kris Zyp a couple of months ago
> ("single frame continuations")
> https://mail.mozilla.org/pipermail/es-discuss/2010-March/010865.html
>
> I don't think that discussion lead to a clear outcome, but it's definitely
> related, both in terms of goals as well as in mechanism.
> I also recall it prompted Dave Herman to sketch the design space of
> (single-frame) continuations for JS:
> https://mail.mozilla.org/pipermail/es-discuss/2010-April/010894.html
>
> I have to throw up a red flag and claim some naivety/ignorance here. Even
after reading those links, I'm confused on what "single frame continuations"
means. Perhaps what I have in mind is like that, but perhaps not, and I need
a little help in understanding better the implications of that terminology.
what is a "single-frame"? I *think* I roughly understand a 20,000ft level
idea of what continuation is.
> It might surprise some who know me to hear this, but I agree with Dave
> on this. There's a huge gap between the single-frame mechanism and
> going the whole hog. Going all out does buy you some expressive
> power, but at the cost of complicating everything. If the simple
> mechanism gives you most of what you want (and I feel it goes pretty
> far), why commit to a lifetime of pain?
>
> As I said, I'm confused, so I'm not sure where my idea fits in the spectrum
of what you're suggesting. Is my idea closer to the simple mechanism
"single-frame continuation" side or is it closer to the "whole hog" side?
I feel like my idea is pretty simple and limited compared to what I've seen
from the broader scope ideas of full program continuations, true
concurrency, etc. But maybe I'm in completely the wrong "frame" of reference
and my idea is way far out there in complexity?
> Solving these problems was the
> point behind the shallow continuation proposal that we worked on. Does
> the generator strawman build on that work, or does it still fail to
> preserve locality?
>
>
OK, so is "shallow continuation" the same as "single-frame continuation" or
are we now talking about different concepts? Again, on the surface, "shallow
continuation" sounds like it might be akin to what my idea is, but I may be
completely wrong.
> My understanding of Kyle's proposal was that he did not want to
> introduce real continuations, but this was more of a syntax for a
> specialized callback chain, reducing the verbosity of anon functions,
> callback registration.
>
> In my naive understanding of the words, I think what I want to introduce is
statement-localized continuations.
I want for a statement, which can consist of two or more expressions, any or
all of them being function calls which can chose to yield/defer/suspend
their completion asynchronously.
Only a statement which would use the @ operator on two or more expression
operands would be able to take advantage of this statement localized
continuation, though. A function that called `p.defer()` inside itself, but
which was called in a normal expression/statement without @ operator, would
simply complete and program execution would continue as normal. Of course,
the function would still be able to complete asynchronously at a later time,
just like a callback to setTimeout can.
function foo() {
console.log("foo");
}
function bar() {
console.log("bar");
}
foo() @ bar(); // normal sync statement that prints "foobar".
function foo() {
var p = promise;
setTimeout(function(){
console.log("foo");
p.fulfill();
},1000);
p.defer();
}
function bar() {
console.log("bar");
}
foo() @ bar(); // async statement, which will print "foobar", but not until
1000ms from now.
And what I mean by "statement-localized continuation" is that
yield/defer/suspend/continuation would only affect that single statement
with the @ in that case, and any statements after the @ statement would
continue synchronously if the @ statement indeed gets suspended at any point
during its evaluation.
So:
function baz() {
console.log("baz");
}
baz() @ foo() @ bar();
console.log("yes");
would result in "bazyes" immediately, and 1000ms from now, "foobar" right
after it.
Does that make any more sense in terms of clarifying my idea, or does it
just complicate things worse?
--Kyle
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: natively negotiating sync vs. async...without callbacks
On Thu, Dec 9, 2010 at 12:22 PM, Getify Solutions wrote:
> 1. I'm not trying to open the can-o'-worms around block level changes. The
> above code suggests that a 'yield' suspension of execution is local to the
> nearest container { } block, in this case the try { } block
No, as implemented in JS1.7, the suspension is of the innermost frame,
not the innermost lexical block. (You think that it would pause that
block, and then jump to the statement following it, to execute from
there until the yield...yielded?)
https://developer.mozilla.org/en/Core_JavaScript_1.5_Guide/Iterators_and_Generators
has more.
Mike
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: natively negotiating sync vs. async...without callbacks
> I'm not trying to open the can-o'-worms around block level changes. The above
> code suggests that a 'yield' suspension of execution is local to the nearest
> container { } block, in this case the try { } block.
No, that's not the case. That code was not hypothetical: it works in
SpiderMonkey using extensions that have been around for several years. You
might want to take some time to explore generators by following the docs that
Mike linked to, and by playing with SpiderMonkey.
> I have to throw up a red flag and claim some naivety/ignorance here. Even
> after reading those links, I'm confused on what "single frame continuations"
> means.
Both "single-frame continuation" and "shallow continuation" are terms we've
used informally for the same idea: being able to suspend a single function
activation (aka stack frame) as a JavaScript value that you can use later to
resume the suspended activation.
> I feel like my idea is pretty simple and limited compared to what I've seen
> from the broader scope ideas of full program continuations, true concurrency,
> etc.
Nobody is recommending full continuations or heavyweight concurrency
facilities. *Especially* if it involves shared-memory threads (which most
straightforward extensions would) -- that will *definitely* not happen in
ECMAScript).
> But maybe I'm in completely the wrong "frame" of reference and my idea is way
> far out there in complexity?
I haven't fully understood your proposal yet, so FWIW it's hard for me to say
just yet.
> In my naive understanding of the words, I think what I want to introduce is
> statement-localized continuations.
>
> I want for a statement, which can consist of two or more expressions, any or
> all of them being function calls which can chose to yield/defer/suspend their
> completion asynchronously.
I think you might find you need to work through this part in more detail. Part
of the essential trickiness in nailing down the semantics of a continuation
operator is specifying *exactly* how much of the continuation is suspended.
I don't understand the |promise| "auto-variable" you mention. What does it do?
Does it affect control flow as well, or is it no difference from (new
Promise()) for some built-in Promise constructor? (BTW, automatically-bound
variables like |arguments| are a mis-feature IMO, and I'd urge you to
reconsider that part.)
What does it mean when you say in your post that "the @ operator will wait to
continue"? What does it mean in JS to "wait?" Does it mean that evaluation of
the entire statement suspends and the result is a new promise? Where does that
promise go? Statements discard their result, more or less (notwithstanding the
completion value, which in most cases in the language really is discarded), so
how does that promise not just disappear?
You mention a ternary form but I can't find it anywhere in the blog post. Do
you describe it in any more detail somewhere else?
What is the specification of p.defer()? Does it cause its containing function
to suspend? Or just its containing statement?
Dave
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
Re: natively negotiating sync vs. async...without callbacks
On Dec 9, 2010, at 3:27 PM, David Herman wrote:
>> I'm not trying to open the can-o'-worms around block level changes. The
>> above code suggests that a 'yield' suspension of execution is local to the
>> nearest container { } block, in this case the try { } block.
>
> No, that's not the case. That code was not hypothetical: it works in
> SpiderMonkey using extensions that have been around for several years.
Works in Rhino too, thanks to Norris Boyd, Steve Yegge, et al.
/be
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
