Re: Blocking message passing for Workers

[David Bruant]

Le 12/08/2014 20:51, Brendan Eich a écrit :

David Bruant wrote:
That's not what I had understood. So both types of APIs (sync and 
async) will be available to workers for say, IndexedDB?
If that's the case, I have no problem with it and we can stop the 
discussion.
What I remembered of the state of the consensus was that given sync 
APIs were considered needed in workers, they would be added to 
workers without the async counterpart (since that would duplicate the 
API surface).


Sorry I missed this -- do you have a link to the thread?
I proposed exposing both here 
http://lists.w3.org/Archives/Public/public-webapps/2013OctDec/0164.html
Jonas Sicking wasn't sold 
http://lists.w3.org/Archives/Public/public-webapps/2013OctDec/0165.html

And I haven't found later replies on this topic.

David

Re: Blocking message passing for Workers

[David Bruant]

Le 12/08/2014 19:36, Brendan Eich a écrit :

David Bruant wrote:

Le 12/08/2014 02:11, Brendan Eich a écrit :

David Bruant wrote:

Le 09/08/2014 16:22, Brian Kardell a écrit :


On Aug 9, 2014 10:16 AM, "David Bruant" <mailto:bruan...@gmail.com>> wrote:
> There is still a case for blocking primitives for projects that 
compile from other languages (C, C++, Python, Java, C#, etc.) to 
JS [3].

>

I'm glad to be switching last night's twitter discussion to a 
bigger medium.  My question here is: what is the proposal (if 
there is any) to balance these and simultaneously ensure that we 
don't wind up limiting ourselves or providing really bad foot guns 
or two APIs depending on whether you're in the main thread or a 
worker?


There isn't such proposal and I don't think that can exist which is 
one reason I'm opposed to the introduction of blocking primitives 
in workers.


I really hope the compile-to-JS use cases will find another way to 
be solved.


There is no other way.

Why are you arguing from dogma instead of reason?
It won't be possible to run the same code (libraries) in workers and 
main thread. That's a reason, not a dogma.


It's not much of a reason :-P.

Workers don't have all the APIs that main-thread JS has today. What's 
more, if one chooses to write async-only code for all contexts, then 
there's no problem.
That's not what I had understood. So both types of APIs (sync and async) 
will be available to workers for say, IndexedDB?
If that's the case, I have no problem with it and we can stop the 
discussion.
What I remembered of the state of the consensus was that given sync APIs 
were considered needed in workers, they would be added to workers 
without the async counterpart (since that would duplicate the API surface).


If I understand you, you're arguing for everyone manually inverting 
control flow, because that maximizes code re-use.
That's what everyone is already used to doing already (not because of 
code reuse, but because that's how JS APIs are designed).


Why not let authors maximize or not? Trade-offs exist along multiple 
dimensions here, and code reuse is only one good to consider.


Anyway, the Emscripten (and many other compilers) use-case remains. 
It's not something to hand-wave away.

I wasn't. That branch of the discussion was discussing hand-written code.
My position on Emscripten&co is "I really hope the compile-to-JS use 
cases will find another way to be solved.", to which you answered.


People already don't use workers that much (because of copy cost 
outweighing computing in most use cases and too many people are still 
unaware of transferables).


This has nothing to do with the subject.

I mean, sure: there are probably lots of reasons workers are underused 
(see above about missing main-thread APIs), but I doubt that among 
those reasons is the fear of blocking i/o primitives being exposed to 
workers and thereby limiting migration of blocking-worker code -- 
which would have to be written only for that content -- onto the main 
thread!


Once we have blocking i/o in workers, you may find workers used more 
in practice, certainly by Emscripten'ed code.
Assuming blocking i/o in workers gets widely implemented in browsers of 
course.


With C, Java and all, we already know where adding blocking I/O 
primitives leads to. Admittedly maybe dogma trying to learn from 
history.


You're appealing to something here, but I can't tell what. "C, Java 
and all" have not all failed as languages or successful server-side 
systems because of blocking i/o primitives. More the reverse.


Is this Node.js triumphalism? Be careful: notable Node hackers have 
left for Go. Node is not a proof that non-blocking is always and only 
ever the one true way or the best way to do i/o and utilize CPUs, by 
any means.
No Node.js triumphalism, but Node found an interesting balance of being 
resource efficient while remaining pleasant to read and write is all I'm 
saying.


David

Re: Blocking message passing for Workers

[David Bruant]

Le 12/08/2014 19:44, Domenic Denicola a écrit :

Realizing the difference between these is important background to realizing why 
async + sugar cannot replace synchronous code. (Apologies if this was stating 
the obvious...)

Is replacing sync APIs a goal? It sure isn't mine.
My point is that from a purely technical perspective async is better as 
it allows better reuse of process/threads. Historically async 
programming has been hard to read/write. Things are getting better as 
programming languages evolve (ES7) is all I'm saying.


David

Re: Blocking message passing for Workers

[David Bruant]

Le 12/08/2014 02:11, Brendan Eich a écrit :

David Bruant wrote:

Le 09/08/2014 16:22, Brian Kardell a écrit :


On Aug 9, 2014 10:16 AM, "David Bruant" <mailto:bruan...@gmail.com>> wrote:
> There is still a case for blocking primitives for projects that 
compile from other languages (C, C++, Python, Java, C#, etc.) to JS 
[3].

>

I'm glad to be switching last night's twitter discussion to a bigger 
medium.  My question here is: what is the proposal (if there is any) 
to balance these and simultaneously ensure that we don't wind up 
limiting ourselves or providing really bad foot guns or two APIs 
depending on whether you're in the main thread or a worker?


There isn't such proposal and I don't think that can exist which is 
one reason I'm opposed to the introduction of blocking primitives in 
workers.


I really hope the compile-to-JS use cases will find another way to be 
solved.


There is no other way.

Why are you arguing from dogma instead of reason?
It won't be possible to run the same code (libraries) in workers and 
main thread. That's a reason, not a dogma.
People already don't use workers that much (because of copy cost 
outweighing computing in most use cases and too many people are still 
unaware of transferables).


There's no *reason* to say worker overhead is so expensive we should 
not allow authors to create more workers as needed when some block 
(temporarily, let's hope

hope? Who was taking about reason? :-p
I don't think hoping for reasonable behaviors from authors works at 
scale, do you? People copy/paste, put copy/pasted code in loop bodies 
without looking into it too much (it's not even funny how often people 
ask me for JS perf advice which ends up being some equivalent of "cache 
this jQuery call in a variable").


With C, Java and all, we already know where adding blocking I/O 
primitives leads to. Admittedly maybe dogma trying to learn from history.



on non-main-thread sync input operations?

Why haven't workers begun with sync primitives?

David

Re: Blocking message passing for Workers

[David Bruant]

Le 12/08/2014 15:28, Glenn Maynard a écrit :
On Mon, Aug 11, 2014 at 6:52 PM, David Bruant <mailto:bruan...@gmail.com>> wrote:


Le 12/08/2014 00:40, Glenn Maynard a écrit :

On Sat, Aug 9, 2014 at 9:12 AM, David Bruant mailto:bruan...@gmail.com>> wrote:

This topic is on people minds [1]. My understanding of where
we're at is that "ECMAScript 7" will bring syntax
(async/await keywords [2]) that looks like sync syntax, but
acts asynchronously. This should eliminate the need for web
devs for blocking message passing primitives for workers.


Syntax sugar around async is not a replacement for synchronous APIs.

I have yet to find a use case for hand-written code that requires
sync APIs and cannot be achieved with async programming.


I have yet to find a use case for hand-written code that requires 
structured programming and cannot be achieved with raw assembly.

I guess I misundersood what you meant by "replacement".
What about sync APIs cannot be replaced by async APIs with sugar ?





I personally hope it won't happen as it would be a step
backwards. Blocking communication
(cross-thread/process/computer) was a mistake. We need a
culture shift. The browser and Node.js are a step in the
right direction (they did not initiate it, but helped
popularize it).


The problem wasn't that synchronous programming is bad, the
problem was that synchronous code in the UI thread blocks UI, and
the solution to that is asynchronous programming.  Saying
"therefore all synchronous programming is bad" is a very deep
misunderstanding of the issue.

If you block on workers, you'll mechanically need more workers.
That's what happened with Apache that was spawning more threads as
more HTTP requests were coming because the existing threads were
busy waiting for blocking I/O.


That's incorrect.  If I want to perform one CPU-intensive task per CPU 
on a 4-CPU machine, I'm going to have 4 workers whether it's 
implemented sync or async. Not all software is a web server.
Web servers are just a caricature of I/O intensive software that forced 
the programming culture to move to non-blocking I/O patterns.
I don't understand the mention of CPU-intensive tasks, this use case is 
taken care of by the current form of workers, isn't it?


David

Re: Blocking message passing for Workers

[David Bruant]

Le 09/08/2014 16:22, Brian Kardell a écrit :


On Aug 9, 2014 10:16 AM, "David Bruant" <mailto:bruan...@gmail.com>> wrote:
> There is still a case for blocking primitives for projects that 
compile from other languages (C, C++, Python, Java, C#, etc.) to JS [3].

>

I'm glad to be switching last night's twitter discussion to a bigger 
medium.  My question here is: what is the proposal (if there is any) 
to balance these and simultaneously ensure that we don't wind up 
limiting ourselves or providing really bad foot guns or two APIs 
depending on whether you're in the main thread or a worker?


There isn't such proposal and I don't think that can exist which is one 
reason I'm opposed to the introduction of blocking primitives in workers.


I really hope the compile-to-JS use cases will find another way to be 
solved.


David

Re: Blocking message passing for Workers

[David Bruant]

Le 12/08/2014 00:40, Glenn Maynard a écrit :
On Sat, Aug 9, 2014 at 9:12 AM, David Bruant <mailto:bruan...@gmail.com>> wrote:


This topic is on people minds [1]. My understanding of where we're
at is that "ECMAScript 7" will bring syntax (async/await keywords
[2]) that looks like sync syntax, but acts asynchronously. This
should eliminate the need for web devs for blocking message
passing primitives for workers.


Syntax sugar around async is not a replacement for synchronous APIs.
I have yet to find a use case for hand-written code that requires sync 
APIs and cannot be achieved with async programming.




I personally hope it won't happen as it would be a step backwards.
Blocking communication (cross-thread/process/computer) was a
mistake. We need a culture shift. The browser and Node.js are a
step in the right direction (they did not initiate it, but helped
popularize it).


The problem wasn't that synchronous programming is bad, the problem 
was that synchronous code in the UI thread blocks UI, and the solution 
to that is asynchronous programming.  Saying "therefore all 
synchronous programming is bad" is a very deep misunderstanding of the 
issue.
If you block on workers, you'll mechanically need more workers. That's 
what happened with Apache that was spawning more threads as more HTTP 
requests were coming because the existing threads were busy waiting for 
blocking I/O.


David

Re: Blocking message passing for Workers

[David Bruant]

Le 09/08/2014 15:51, Alan deLespinasse a écrit :
Thanks. Apparently I did a lousy job of searching for previous 
discussions.


I just found this later, longer thread:

http://lists.w3.org/Archives/Public/public-webapps/2011OctDec/0965.html
http://lists.w3.org/Archives/Public/public-webapps/2012JanMar/0678.html
(same thread, different year, so they're not linked)

Has anything changed since that thread? It seems like the discussion 
stalled in early 2012. But I'm glad to find that other people want the 
same thing.
This topic is on people minds [1]. My understanding of where we're at is 
that "ECMAScript 7" will bring syntax (async/await keywords [2]) that 
looks like sync syntax, but acts asynchronously. This should eliminate 
the need for web devs for blocking message passing primitives for workers.


There is still a case for blocking primitives for projects that compile 
from other languages (C, C++, Python, Java, C#, etc.) to JS [3].


I personally hope it won't happen as it would be a step backwards. 
Blocking communication (cross-thread/process/computer) was a mistake. We 
need a culture shift. The browser and Node.js are a step in the right 
direction (they did not initiate it, but helped popularize it).


David

[1] https://twitter.com/briankardell/status/497843660680351744
[2] https://github.com/lukehoban/ecmascript-asyncawait#example
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=783190#c26

Re: =[xhr]

[David Bruant]

Le 04/08/2014 15:30, nmork_consult...@cusa.canon.com a écrit :
This is an intranet application.  The server is in the next room 
(locked, of course.)

I was seeing this one coming when I wrote my paragraph :-p
If you're in a tightly controlled environment, one could question the 
choice of a web application, but that's beyond the point of this email.


However, many (not all) POST operations require only 1 channel to be 
active at a time, especially in intranet applications.
As someone else noted, how do you know two computers aren't connected at 
the same time? Or even that the same user isn't logged in in two 
different machines within the network?
Note also that as others said, sync xhr doesn't freeze the browser, so 
if your application isn't robust to the user changing tab, or going 
back, you may have a bug you want to fix.


If your answer resembles even vaguely "because I control the environment 
and want to prevent the user from doing x, y, z and will as long as I 
can", then, I'd like to question again whether you really want to make 
your intranet application run a web browser.
There are lots of other choices to make non-web applications, but the 
web platform took the path of prioritizing the end-user concerns above 
all else. See the "Priority of Constituencies" : 
http://www.w3.org/TR/html-design-principles/#priority-of-constituencies
We've had a foggy period at the beginning of the web, but it's clear now 
that end-users are considered as a priority in web standards and 
implementors (though obviously various parties will see end-user 
interests from a different perspective).


If you care about how your software ensures some guarantees for your 
database above providing a good user experience, the web might not be 
where you want to make your application and I think that you will keep 
meeting resistance in web standards mailing-lists.


No user wants to be given the freedom to accidentally screw up. 

Why would you write code that provides that freedom, though?
In cases like you describe, I would send the POST request asynchronously 
and then disable any user interface that allows to send another request 
until the server comes back with a 200 (or equivalent) via either 
removing the event listeners. In essence, that would look like:


var postUrl = '...';
postButton.addEventListener('click', function clickHandler(){
var xhr = new XMLHttpRequest();
xhr.open('POST', postUrl);
xhr.addEventListener('load', function(){
// POST came back, re-enable the button
postButton.removeEventListener('click', clickHandler);
postButton.setAttribute('disabled', 'disabled');
});
xhr.send();

// disable UI for POSTing
postButton.removeEventListener('click', clickHandler);
postButton.setAttribute('disabled', 'disabled');
// maybe add a spinner somewhere and/or invite the user
// to have a hot beverage of their choice as you suggested
});

Of course, disable any other UI element you need, error handling is 
missing, etc, but you get the idea.
The above code demonstrates that you can prevent the user from further 
interacting with the database, yet use async xhr.


David

Re: =[xhr]

[David Bruant]

Le 02/08/2014 11:11, Austin William Wright a écrit :
Maybe there's another reason: Good idea or no, removing this feature 
DOES break reverse compatibility with the de-facto behavior of many 
Web browsers.
Everyone who wants sync xhr to disappear is well-aware. That's the 
reason it hasn't been removed yet.


Just as a reminder, we live on Planet Earth where the distance between 
two computers who want to communicate with one another can be 20.000km. 
A signal at speed light takes 60ms. But our network isn't either at 
speed light nor in straight line, so the delay is easily multiplied by 
20 and that's very optimistic. And that's just the latency. Bandwidth 
considerations and data processing times aren't free.
On the other hand, we're human beings and notice 100ms delays and all 
evidence converge in the direction of saying that human beings are 
frustrated when they feel a delay, so making people wait is terrible. 
(interesting talk which discusses perceived performance at one point 
http://vimeo.com/71278954 )
Sync xhr forces a delay and there is no way around that, ergo, it's 
terrible.


I'm probably not old enough to accurately make this comparison, but 
blocking I/O looks like it's 2010-era GOTO. I heard when higher-level 
programming came around, some people were strong defenser of GOTO. Now, 
most people have moved to higher-level constructs.

I think the same fate is waiting for blocking I/O.

David

Re: Custom Elements: 'data-' attributes

[David Bruant]

Le 07/05/2014 21:36, Ryosuke Niwa a écrit :

On May 7, 2014, at 12:03 PM, Ian Hickson  wrote:


On Wed, 7 May 2014, Anne van Kesteren wrote:

On Tue, May 6, 2014 at 7:19 PM, Wilson Page  wrote:

I'm unsure whether or not it is safe to use custom attributes without
the 'data-', I've heard mixed opinions. How do we know that chosen
attributes won't someday be global attributes?

Yeah, we should figure something out here. From one perspective you've
already namespaced your element by using a dash. However, given the
existence of an ever growing set of global attributes you probably do
not want to clash with those.

Maybe we should allow any attribute as long as it contains a dash (and
does not match a set of existing names)? Still seems somewhat
suboptimal.

Requiring a dash is pretty ugly. I would allow any attribute, and we'll
just have to be careful when introducing new global ones.

I don't think "being careful" is a sound strategy.

How are you going to quantify the risk of adding a new global attribute in the 
future?

I don't want us to depend on some random search engines to make a guess as to 
which names are safe to use.
I want to agree with you, but it's impossible to prevent authors from 
using any sort of attributes in their content (willing to bet hard money 
web browers won't stop page rendering because some custom element 
attribute doesn't contain a dash; looking at the XHTML precedent if 
nothing else).

Such search will have to happen anyway.

It'd be nice if the tooling to make such search was available to 
inspection, though.


David

Re: Sync IO APIs in Shared Workers

[David Bruant]

Le 24/11/2013 02:23, Jonas Sicking a écrit :


On Nov 23, 2013 2:04 PM, "David Bruant" <mailto:bruan...@gmail.com>> wrote:

>
> Le 22/11/2013 03:33, Jonas Sicking a écrit :
>>
>> We could enable sync IO APIs in workers while still allowing the
>> worker thread to remain responsive to messages from the main thread.
>> One solution is the API in [2]. Using that API the main thread can
>> send a error-response to a sync request for IO before the IO has
>> finished.
>>
>> [2] 
http://lists.w3.org/Archives/Public/public-webapps/2013OctDec/0142.html

>
> I'm not sure I understand this proposal. If you allow messages from 
main thread while waiting for I/O, aren't you re-introducing the same 
complexities that sync APIs are expected to stay away from?


No, because the main thread can always "return early" and not wait for 
the result from the IO before sending a message.


And returning early cancels the I/O and the related pending message (and 
stack)?


David

Re: Sync IO APIs in Shared Workers

[David Bruant]

Le 22/11/2013 03:33, Jonas Sicking a écrit :

We could enable sync IO APIs in workers while still allowing the
worker thread to remain responsive to messages from the main thread.
One solution is the API in [2]. Using that API the main thread can
send a error-response to a sync request for IO before the IO has
finished.

[2] http://lists.w3.org/Archives/Public/public-webapps/2013OctDec/0142.html
I'm not sure I understand this proposal. If you allow messages from main 
thread while waiting for I/O, aren't you re-introducing the same 
complexities that sync APIs are expected to stay away from?


What about providing both APIs in workers? sync and async.
This way, everyone can get what they want.
There is something ugly in providing both, but Node.js did it and is 
doing fine. And the sync version would be uncompromised. For authors, it 
would then be a matter of making a choice, understanding the pros and 
cons of this choice and accepting them.


David

Re: XMLHttpRequest.prototype.responseURL

[David Bruant]

Le 18/11/2013 08:29, Anne van Kesteren a écrit :

We discussed this briefly during TPAC. Currently you can only get to
the response URL if you are loading a document resource. The idea here
is to add a new property that exposes the response URL for any
resource. It will work the same way as status and statusText, as when
those are available we know we will no longer be redirected.

Anyone see a problem with adding this feature?

No problem.

Why not all also intermediary URLs if multiple redirects are followed?

David

Re: Sync API for workers

[David Bruant]

Le 13/10/2013 21:39, Jonas Sicking a écrit :

Ok, this thread is clearly heading off the deep end. Let me clear up a
few points of confusion:

* You can not wrap a truly synchronous library around an asynchronous
API. Spinning the event loop gets you close, but breaks
run-to-completion. Furthermore, spinning the event loop is irrelevant
as we don't have an API to do that, nor are we planning to introduce
one.
* yield only works within generators in JS.
To be honest, I feel generators will be interoperably deployed 
cross-browser long before sync APIs in workers. V8 and SpiderMonkey 
already have generators. I'm not entirely sure it's 100% compliant in 
SpiderMonkey yet, but for sure it's being actively worked on and should 
be soon if not yet.



* You could solve the use case of compile-to-JS for code that uses
sync APIs using yield. However it requires changing all functions into
generators, and all function calls into yield* statements.
all? as is "all function in the application"? that sounds like a too 
violent constraint, especially if a small proportion of the code uses 
sync functions. Maybe only the functions that may call a sync function 
need to be changed to generators... oh... hmm... I don't know.
Taking the liberty to cc Alon Zakai to ask for his expert opinion on 
this topics.



That comes
at a performance overhead that is significant enough as to make it an
unacceptable solution (several times slower in current
implementations).
I guess this point depends on the previous one. Given that compile-to-JS 
has wind behind these days, generators may benefits from optimizations.



* yield would not solve the use-case of allowing libraries that use
features from the main thread as it, again, would require a rewrite of
all code that directly or indirectly uses that library to change all
functions into generators and all function calls into yield*.
I think this point is about interoperability between main thread and 
worker. I don't remember this point being discussed too much yet.

What about exposing both async and sync APIs to workers?

David

Re: Regarding: Making the W3C Web SQL Database Specification Active

[David Bruant]

Le 27/09/2013 23:23, Jonas Sicking a écrit :

On Wed, Sep 25, 2013 at 3:39 PM, Michael Fitchett
 wrote:

Dear Members of the W3C Consortium::

Regarding:  Making the W3C Web SQL Database Specification Active

I would like to request  that you make the W3C Web SQL Database
specification active again. The Web SQL Database Specification enables
developers to build web-based applications that can store, retrieve,
manipulate and query against data on the client machine. This technology is
similar to SQLite, Microsoft SQL Server, MySQL, etc. Web SQL combined with
Manifest enable developers to build web-based applications that work while
offline.

The Web SQL Database specification was on the W3C Recommendation track, but
the specification was stopped because Mozilla and Microsoft did not want to
implement a specification that lacked proper SQL definition. I know there is
a need for both a NoSQL and SQL solution. The two specifications (Web SQL
Database and Indexed Database API) that exist to date are acceptable..
However, as stated above, the problem is the lack of definition for SQL.
Since lack of definition is the issue, I would like to recommend a remedy.
I know SQL experts and great documentation writers who I would gladly hire
to further define the Web SQL Database specification and fill in the missing
SQL definition. Is this something that would be possible to help revive the
specification and get the remaining vendors on board?

The minimum requirements for bringing back WebSQL, or any other
SQL-based web spec is IMHO:

1. A specification for the SQL dialect being proposed.
2. *Two* independent, production quality, database implementations
being willing to implement exactly that SQL dialect. Not a subset of
it, and not a superset of it.
3. The two independent implementations need to have roughly the same
performance characteristics. I.e. it's not ok for an implementation to
generate correct results, but do it so slowly that it's in practice
unusable.
I'd like to add another requirement which is having a significant 
advantage over IndexedDB. If web devs want SQL, they can have it on top 
of IndexedDB in the form of an open source library (I'm willing to be it 
already exists). They don't need to wait for a standard to emerge, nor 
for browsers to consistently implement it.


If they really want a spec, they can create a W3C community group (or a 
Github repo). We don't need browsers to do all the work for us!


David

[File-writer API] Details on NotReadableError and the snapshot state

[David Bruant]

Hi,

For context, I'm currently working on a Tizen (based on a somewhat modern
Webkit) web app and trying to store a file (on an actual Tizen test
device). As far as documentation, Tizen linked to specs, so here I am :-)
I managed to get a FileSystem instance, but fail to get a FileEntry (code:
4) after fs.getFile(validString, {create: true}) (file isn't supposed to
exist, but that should irrelevant according to the spec)

First, what is error 4 for fs.getFile? Remote devtools suggests that error
codes are the same for fs.getFile than any other File error, but this isn't
very clear from the file-writer spec.
Second, the .getFile function lists a bunch of error cases, but nowhere is
written which error and code should be provided. The wording is always
"getFile *MUST* fail" which doesn't seem really helpful either for other
implementors to implement nor for authors to understand whats' going on.
Did I miss something on this section?

code:4 aka NotReadableError in FileError can happen "due to permission
problems that occur after a snapshot state has been established". I guess
I'm looking at a Webkit bug? (I bet more on a device bug...) The spec
doesn't help to say.

Any help in understanding how code:4 can happen on fs.getFile would be
appreciated especially.

Thanks,

David

[FILE-SYSTEM] Hoping the globals of the FileSystem spec are TEMPORARY and not PERSISTENT

[David Bruant]

Hi,

The LocalFileSystem interface [1] defines the TEMPORARY and PERSISTENT 
constants and has the [NoInterfaceObject] extended attribute. It seems 
that the intention behind that choice was to expose these constants on 
the global object (see code snippets of [3] and the fact that these are 
indeed exposed as global properties in the console of a recent Chrome) [2].


This is a bad idea: imagine all specs start defining constants on the 
global object! This is an unhealthy practice that endangers the 
scalability of the web platforms when it comes to exposing APIs.


For that reason, I would like to ask here for the removal of these 
constants properties from the global. This leads to 2 concerns:

1) is this change web compatible?
Google has been promoting this API and the use of these globals through 
code snippets (including [3]) pretty heavily. Among others Mega made 
pretty clear that they were using the API [5].


2) Assuming the answer to 1) is "yes", how and what's the replacement?
One idea is to use strings:
window.requestFileSystem("temporary", size, onInitFs, errorHandler);
That could work fine.

David

[1] 
http://dev.w3.org/2009/dap/file-system/file-dir-sys.html#using-localfilesystem
[2] This appears to be a misinterpretation of the WebIDL spec [4], but 
that's not what brings me here, let's solve this issue separately.

[3] http://www.html5rocks.com/en/tutorials/file/filesystem/
[4] 
http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0203.html

[5] https://mega.co.nz/#blog_1

Re: DOM: Futures

[David Bruant]

<3

Le 28/03/2013 17:04, Anne van Kesteren a écrit :

The Future has arrived:

   http://dom.spec.whatwg.org/#futures

Thanks in particular to the https://github.com/slightlyoff/DOMFuture
project for nailing down all of the details. If someone is interested
in contributing examples the DOM Standard accepts pull requests
https://github.com/whatwg/dom but you can also contact me directly if
you think that would be easier.

(I did a bcc to public-webapps just in case.)

Re: window.event and Event.srcElement

[David Bruant]

Le 25/03/2013 20:00, Boris Zbarsky a écrit :

On 3/25/13 2:47 PM, David Bruant wrote:

For the latter case, a scoping trick might work. In essence, on* code
could run within a scope "located" between the code scope and the 
global scope.


It already does: the scope is the element, with some more 
complications as to what's on the scope chain above that.

I wasn't aware of that, but I see it now. Step 6 of:
http://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#event-handler-content-attributes
(Lexical Environment Scope stuff)

We could complicate the scoping setup for event handlers even more, of 
course.  Would any UAs actually be willing to implement that (given 
that e.g. Chrome doesn't even implement the current, simpler, scoping 
setup sanely)? 

Any info on why they don't implement the spec scoping rules?
Is it going to be a case of de-facto standard too?


For the former case, things cannot be as easy as naively inserting an
intermediate scope, because the function definition is somewhere else,
but maybe we can get away with an equivalent trick. Thoughts?


I can't see any equivalent tricks given the scoping rules in JavaScript.

Basically, either UAs that currently implement window.event remove it 
or it's clearly required for web compat and hence needs to be 
specified so other UAs can implement it.  I don't see any other sane 
options; do you?

Only an insane option. Sharing as food for thoughts:
when about to call a function from an eval'ed on* attribute string, 
change the function's [[Scope]] to contain an extra scope with an 
"event" variable. Restore the [[Scope]] to its original value when the 
frame is dropped.
But of course, as soon as I talked about changing [[Scope]], I 
definitely fell in the "insane" category as I warned :-/


Since that case apparently requires window.event, no need to add extra 
work on on* scoping, I guess.




If you want specific site examples www.ocn.ne.jp (on "mobile" only,
natch, thanks to only testing in WebKit) is the most recent one I came
across.

Is it the on* case? (how do you test for "mobile only"?)


It's using event handlers, yes.  You test for "mobile only" by loading 
the site in Firefox on some desktop platform and Firefox on Android 
and observing that the site is UA-sniffing and sending different 
content, and that the site sent to desktop browsers works in Firefox 
but the site sent to "mobile" browsers does not.

Now I realize my question was stupid...

David

The need to re-subscribe to requestAnimationFrame

[David Bruant]

Hi,

If someone wants to reuse the same function for 
requestionAnimationFrame, he/she has to go through:

requestAnimationFrame(function f(){
requestAnimationFrame(f);
// do stuff
})

I was wondering why it was the case. Other event-like mechanism do not 
require to re-subscribe after an event happened.


Thanks,

David

[xhr] Appending a charset after an author added content-type

[David Bruant]

Hi,

Nicolas Chambrier (he's not registered to the list, please keep him in 
the CC in your replies) and I came across that case recently:

1) create an XHR object (POST in case it matters)
2) set the "Content-Type" header explicitly (it changes, but we noticed 
it with something like "application/vnd.ooz.user.v1+json")

3) .send() (what is sent is a JSON valid string)

Firefox adds a "; charset=UTF-8" after the manually set content-type value.
Chrome does not.
I'd like to report a bug, but I'm not sure where I should report it to :-s

My understanding of the spec is that the charset shouldn't be added, 
because:

1) the Content-Type header is in author request header
2) the value is a valid MIME-type (tell me if I'm wrong on this)
3) it does not have a charset parameter (so, the condition of step 4 
doesn't apply in my opinion).


Who should I file a bug against?

David

Re: document.register and ES6

[David Bruant]

Le 06/02/2013 11:27, Boris Zbarsky a écrit :

On 2/5/13 10:28 PM, Boris Zbarsky wrote:

And the point is that document.register changes the [[Construct]] of
MyButton but does nothing else with it?


Note that I'm still checking how feasible this is in SpiderMonkey on 
any sort of finite timeframe, if we do decide to do this. Functions 
right now don't have a [[Construct]] member in spidermonkey that's 
stored on a function directly; it's stored on a shared data 
structure.  So it's impossible to change the [[Construct]] of a single 
function as things stand...
As a band-aid short-term type of solution, the exposed function could be 
a proxy to the actual function with a specific construct trap. I've just 
tried the following on Firefox Aurora and it does what you'd expect:


function f(){
console.log('f call')
}

var pf = new Proxy(f, {
construct: function(target){
console.log('pf construct');
target();
}
})

pf();
new pf();

David

Re: Colliding FileWriters

[David Bruant]

Hi,

Sorry for the archaeologically late response. I'm currently documenting 
FileHandle and am trying to understand how it works, so I have a couple 
of questions.


I don't think this particular message received a response and I have an 
unanswered interogation about it.



On Wed, Feb 29, 2012 at 1:56 AM, Glenn Maynard mailto:gl...@zewt.org?Subject=Re%3A%20Colliding%20FileWriters&In-Reply-To=%253CCA%2Bc2ei-f7fiBSZeBR0j6EpzQnUCTyyUF3bhXK86eC5Xo1_NVkw%40mail.gmail.com%253E&References=%253CCA%2Bc2ei-f7fiBSZeBR0j6EpzQnUCTyyUF3bhXK86eC5Xo1_NVkw%40mail.gmail.com%253E>>
 wrote:
> On Mon, Feb 27, 2012 at 6:40 PM, Jonas Sicking mailto:jo...@sicking.cc?Subject=Re%3A%20Colliding%20FileWriters&In-Reply-To=%253CCA%2Bc2ei-f7fiBSZeBR0j6EpzQnUCTyyUF3bhXK86eC5Xo1_NVkw%40mail.gmail.com%253E&References=%253CCA%2Bc2ei-f7fiBSZeBR0j6EpzQnUCTyyUF3bhXK86eC5Xo1_NVkw%40mail.gmail.com%253E>>
 wrote:
>>
>> To do the locking without requiring calls to .close() or relying on GC
>> we use a similar setup to IndexedDB transactions. I.e. you get an
>> object which represents a locked file. As long as you use that lock to
>> read from and write to the file the lock keeps being held. However as
>> soon as you return to the event loop from the last progress
>> notification from the last read/write operation, the lock is
>> automatically released.

I LOVE THIS SO MUCH! (that's not my question, just a reaction :-) )


>> One downside of this is that it means that if you're doing a bunch of
>> separate read/write operations in separate locks, each lock is held
>> until we've had a chance to fire the final success event for the
>> operation. So if you queue up a ton of small write operations you can
>> end up mostly sitting waiting for the main thread to finish posting
>> events.
>
> It'd only slow things down if you attach an expensive, long-running event
> handler to a load/loadend event, which is an inherently bad idea if you're
> doing lots of tiny operations.  Is that actually a problem?

No, that's not correct.

Most likely the implementation of this will use two threads. The main
thread which runs the JS code running in the window or worker and an
IO thread which does the file reading/writing. The main thread is also
where event handlers run. Every time a read/write is requested by the
main thread, data about this operation is sent to the IO thread
allowing the main thread to continue.

If the main thread creates two separate locks which perform two small
write operations (...)
I'd like to stop here for a minute, because I'm not entirely clear as to 
what this assumptions means.
Does this part mean that during the same event loop turn, some JS code 
would open 2 separate locks for the same file? If so, that sounds like 
such an edge case it should just be forbidden like throwing an 
ALREADY_LOCKED error when asking a second lock.
Since the first lock is released at the end of turn, the code asking the 
second lock can recover by try/catching the error and deciding to lock 
and play with the file in another later turn (using setTimeout( , 0) or 
whatev's)


Preventing 2 locks for the same file in the same turn saves a lot of 
complications it seems.

What ended up happening for Firefox implementation?

David

Re: CfC: publish WD of XHR; deadline November 29

[David Bruant]

Le 25/11/2012 20:07, Kyle Huey a écrit :

Have you read Adam Barth's contributions to this discussion?

Sure, and I personally mostly agree with these points.

He has summarized the point well, I think.  There is a difference 
between what the license legally obligates one to do
I talked very briefly in terms of "legally" and spent more time talking 
about "author intent" on purpose.
As I said, licences have a legal part to them, but also (and mostly?) 
convey an intent. Actually, Anne made extremely clear what he meant in a 
recent post [1]:
"Most of the WHATWG documents are published in the public domain (CC0 
for countries that do not recognize the public domain). I think this is 
important because these are documents that define part of the 
architecture of the web. Nobody and no organization should be entitled 
to them."


The intent is clear: the WHATWG publishes documents in the public domain 
for very good reason. Anyone (W3C included!) can reuse them under close 
to no condition, not even credit.



and what professionals working in good faith towards similar goals do.
Now, after discussing the author intention, we can discuss "what 
professionals working in good faith do", but we're starting to get in a 
blurry field where people from different cultures will have very 
different definitions of what "professional", "work" and "good faith" mean.
Adam Barth has his definition (which I agree with, by the way). It 
appears from the latest messages on this thread that the W3C agrees with 
his definition so everything is fine.
In case the W3C didn't agree (which it was initially accused of), maybe 
adding the WHATWG definition of "what professionals working in good 
faith do" to its licence (as a show of intention, the legal part would 
just be a side effect) could clear up cross-cultural ambiguities and 
remove a good share of conflicts.


Once again, publishing work under public domain and expecting others to 
respect mystical things like "what professionals working in good faith 
do" is hypocritical. If you expect someone else to behave is some way, 
the first step is expressing your expectations (the licence being one 
clear way to do so), not expecting others to guess! Especially not on 
the web where people from so many different cultures meet!
Clearly communicating intent and expectations is something I'd expect 
from professionals working in good faith. I'm glad it happened in this 
thread, I'm sad it took conflicts and tensions.


David

Ps : The importance of the intention behind a licence (besides the 
purely legal aspect) can be seen elsewhere as the example of the AGPL 
Licence and the Neo4J interpretation shows: 
http://stackoverflow.com/questions/6500925/agpl-license-question-re-neo4j


[1] http://annevankesteren.nl/2012/11/copyright

Re: CfC: publish WD of XHR; deadline November 29

[David Bruant]

Le 22/11/2012 18:16, Ms2ger a écrit :

On 11/22/2012 02:01 PM, Arthur Barstow wrote:

TheXHR Editors  would  like to publish a new WD of XHR and this is a
Call for  Consensus to do so using the following ED (not yet using the
WD template) as the basis
.

Agreement to this proposal: a) indicates support for publishing a new
WD; and b) does not necessarily indicate support of the contents of 
the WD.


If you have any comments or concerns about this proposal, please reply
to this e-mail by December 29 at the latest.

Positive response to this CfC is preferred and encouraged and silence
will be assumed to mean agreement with the proposal.


I object unless the draft contains a clear pointer to the canonical 
spec on whatwg.org.
I'm unfamiliar with the W3C process, so sorry if my question is stupid, 
but why would it be necessary? (I assume you're talking about 
http://xhr.spec.whatwg.org/)


Quoting http://xhr.spec.whatwg.org/
"Editor:
Anne van Kesteren 

CC0 To the extent possible under law, the editor has waived all 
copyright and related or neighboring rights to this work. In addition, 
as of 22 November 2012, the editor has made this specification available 
under the Open Web Foundation Agreement Version 1.0, which is available 
at http://www.openwebfoundation.org/legal/the-owf-1-0-agreements/owfa-1-0. "


Quoting 
http://www.openwebfoundation.org/legal/the-owf-1-0-agreements/owfa-1-0 
(emphasis is mine)
"2.1.   Copyright Grant.  I grant to you a perpetual (for the duration 
of the applicable copyright), worldwide, non-exclusive, no-charge, 
royalty-free, copyright license, *without any obligation for accounting 
to me*, to reproduce, prepare derivative works of, publicly display, 
publicly perform, sublicense, distribute, and implement the 
Specification to the full extent of my copyright interest in the 
Specification. "


This wording makes pretty clear that pointing to the whatwg spec isn't 
required or necessary or anything.



It would be pretty hypocritical to put some work under CC0/public 
domain/OWFAV1.0 and expect or even demand to be credited. Some licences 
(CC-BY as an example) require crediting the original author. I assume a 
purposeful choice has been made by Anne and the WHATWG to put the work 
under a licence that doesn't have such a requirement.
Choosing a licence applied to some work shows an intention of how one 
expects the work to be reused. The intention here is pretty clear and 
says "I don't care of being credited".

Choosing a licence is a serious choice with serious implications.

If the WHATWG expects credit, maybe it should consider re-licence its 
work (which would be easy given the current licence ;-) ) to a licence 
expressing more clearly this intent instead of expecting others to guess 
the intent and throwing accusations of plagiarism.


David

Re: random numbers API

[David Bruant]

Le 16/11/2012 18:20, Florian Bösch a écrit :
I'll see that I can come up with a test suite that verifies 
statistical and runtime behavior of an array of algorithms implemented 
in JS, it'll probably take a while.

I don't think you need to go through such lengths.
If you do have the galaxy use case, implement it with the same C 
algorithm you used to get the 0.7s measurement and see if JS perf is 
actually a problem.


David

Re: random numbers API

[David Bruant]

Le 16/11/2012 17:35, Florian Bösch a écrit :
On Fri, Nov 16, 2012 at 5:20 PM, David Bruant <mailto:bruan...@gmail.com>> wrote:


That'd be a nonsense to add seeding in my opinion. If you want
security, you don't want to take the risk of people seeding and
loose all security property. If it's for debugging purposes, the
seeding should be part of a devtool, not of the web-facing API.

I agree that in the crypographic context seeding might not make sense 
(or even guarantees about repeatability).


The purpose of the proposal of a fast, reliable, statistically sound, 
repeatable, seedable PRNG in JS however is not to do cryptography. It 
would be to be able to perform procedural computation repeatably 
regardless of machine, VM, optimization and vendor differences. An 
example: Say you wanted to do a procedural universe consisting of 1 
million stars. At 3 cartesian coordinates per star and at each 
component having 8 bytes, you'd get 22MB of data. If you want to share 
this galaxy with anybody you'll have to pass them this 22mb blob. If 
you want multiple people in the same galaxy, you have to pass them 
that blob.
If you want repeatable, you actually don't want random (as your title 
suggests) but PRNG very specifically ("pseudo" being themost important 
part). In that case, I feel writing your own PRNG will be almost as fast 
as a native one with nowadays crazy JIT. Just write an algorithm that 
you're satisfied and pass around the algo and any parametrization you 
want. I feel it would solve your use case.


It takes about 0.7 seconds in C to generate 3 million statistically 
sound random numbers for longs.

Do you have measurements of how much the same algo takes in JS?

David

Re: random numbers API

[David Bruant]

Le 16/11/2012 16:30, Florian Bösch a écrit :
On Fri, Nov 16, 2012 at 4:24 PM, > wrote:


The W3C Web Cryptography working group [1]  has a draft that seems
to include a method to generate cryptographically random values [2].

It does include a random number generator. However it does not include 
seeding and consequentially no guarantees about the algorithm and 
repeatability.
That'd be a nonsense to add seeding in my opinion. If you want security, 
you don't want to take the risk of people seeding and loose all security 
property. If it's for debugging purposes, the seeding should be part of 
a devtool, not of the web-facing API.


David

Re: CSP 1.1 DOM design

[David Bruant]

Le 05/11/2012 13:57, Alex Russell a écrit :
On Mon, Nov 5, 2012 at 12:14 PM, David Bruant <mailto:bruan...@gmail.com>> wrote:


Le 05/11/2012 12:50, Alex Russell a écrit :

On Mon, Nov 5, 2012 at 10:56 AM, David Bruant mailto:bruan...@gmail.com>> wrote:

Le 05/11/2012 11:32, Alex Russell a écrit :

On Mon, Nov 5, 2012 at 1:08 AM, Boris Zbarsky
mailto:bzbar...@mit.edu>> wrote:

On 11/4/12 3:58 PM, Alex Russell wrote:

 DOMString toString();


This should probably be:

  stringifier;

instead (which in ES will produce a toString on the
prototype, but is more clear about the point, and might
do different things in other binding languages).


Other binding languages don't matter, but OK.

I heard Google is working on this "Dart" thing. Unless Google
redefine APIs for every single web browser capability, it
will probably need to define WebIDL bindings for Dart. But
what do I know...


You know, we should go talk to the guys who designed the Dart
DOM...oh, wait, that's me (and arv and jacobr and jmesserly)!

I wrote "every single web browser capability", not "DOM".


WebIDL is about DOM. Does it even have other users?
I think Anne pointed a potential misunderstanding in his message. I 
personally use "DOM" for what it literally stands for. For the rest, I 
say "browser capability" or "browser APIs" or "web APIs".
I read "WebIDL is about DOM" as a non-sense with my definition, but 
understanding that we have a different definition, it makes more sense.


If you don't expose an API from a C++ impl to a more dynamic host 
language, it doesn't make sense. To the extent that "DOM" is "C++ 
objects vended to dynamic languages", yes, I maintain that every API 
should be re-cast in terms of language-native idioms. WebIDL gets you 
part of the way there for JS (and formerly for Java). But not the 
whole way. When it prevents improvements (as it so often does), it's a 
bug and not a feature.

I largely agree. Now, it just means that WebIDL needs to be improved.


Yes, it's a lot of work, but if you're not taking care to create
a great API for one of your most frequently-used libraries,
you're screwing your language and your users. I posit that every
language with enough users to matter will do this exercise (JS
has done so many times over in the form of the ubiquitous library
tax that slows so many sites).

FWIW, we can still use WebIDL as a stepping stone to fix the
b0rken JS bindings. But we need to collectively stop pretending that:

 1. We should be designing JS APIs though the lens of what WebIDL
can/can't do


Is anyone really doing this?



Part of my job is reviewing this sort of proposal and let me assure 
you that /everyone/ does this. IDL /is handy. /More to the point, it's 
the language of the specs we have now, and the default mode for 
writing new ones is "copy/paste some IDL from another spec that looks 
close to what I need and then hack away until it's close". This M.O. 
is exacerbated by the reality that most of the folks writing these 
specs are C++ hackers, not JS developers. For many, WebIDL becomes a 
safety blanket that keeps them from having to ever think about the 
operational JS semantics or be confronted with the mismatches.

Interesting. I wasn't aware of that.
Any idea as to how to improve the state of things? It seems to be much 
more a human problem than a WebIDL problem. Even if WebIDL improves, 
people will likely still do the copy/paste dance, no?



WebIDL didn't exist a couple of years ago and people were
designing APIs anyways.


...in IDL. WebIDL is descended from MIDL/XPIDL which is descended from 
CORBA IDL. WebIDL is merely a compatible subset + JS-leaning superset.
The WebIDL defines an ECMAScript binding and that's a huge step forward. 
DOM2 ECMAScript bindings is how I discovered that specs really are made 
by human beings and are subject to being incomplete or inaccurate.
I agree WebIDL has limitations, but it's going in the right direction, I 
feel.



Also, WebIDL is still in flux; if WebIDL is limitating, just ask
for a change in WebIDL. I've seen a bunch of posts from Boris
Zbarsky in that direction.


Heh. Who do you think advocated for DOM prototypes in the right 
locations? Or has been continuiously advocating that DOM designs not 
use create* but instead lean on "new"? ;-)
I'm aware of your work in this field. Your QCon London 2012 talk [1] 
shows where you want to go to and it's promising. I was not trying to 
say that you haven't done anything, but that the solution is to improve 
WebIDL (regard

Re: CSP 1.1 DOM design

[David Bruant]

Le 05/11/2012 12:50, Alex Russell a écrit :
On Mon, Nov 5, 2012 at 10:56 AM, David Bruant <mailto:bruan...@gmail.com>> wrote:


Le 05/11/2012 11:32, Alex Russell a écrit :

On Mon, Nov 5, 2012 at 1:08 AM, Boris Zbarsky mailto:bzbar...@mit.edu>> wrote:

On 11/4/12 3:58 PM, Alex Russell wrote:

 DOMString toString();


This should probably be:

  stringifier;

instead (which in ES will produce a toString on the
prototype, but is more clear about the point, and might do
different things in other binding languages).


Other binding languages don't matter, but OK.

I heard Google is working on this "Dart" thing. Unless Google
redefine APIs for every single web browser capability, it will
probably need to define WebIDL bindings for Dart. But what do I
know...


You know, we should go talk to the guys who designed the Dart 
DOM...oh, wait, that's me (and arv and jacobr and jmesserly)!

I wrote "every single web browser capability", not "DOM".
I agree with you about the DOM. The Dart effort (and DOM4 to some 
extent) is excellent for what it did to the DOM API (great job guys!).


Yes, it's a lot of work, but if you're not taking care to create a 
great API for one of your most frequently-used libraries, you're 
screwing your language and your users. I posit that every language 
with enough users to matter will do this exercise (JS has done so many 
times over in the form of the ubiquitous library tax that slows so 
many sites).


FWIW, we can still use WebIDL as a stepping stone to fix the b0rken JS 
bindings. But we need to collectively stop pretending that:


 1. We should be designing JS APIs though the lens of what WebIDL
can/can't do

Is anyone really doing this? WebIDL didn't exist a couple of years ago 
and people were designing APIs anyways. Also, WebIDL is still in flux; 
if WebIDL is limitating, just ask for a change in WebIDL. I've seen a 
bunch of posts from Boris Zbarsky in that direction.



 1. That there are other language consumers of WebIDL-defined DOM APIs
that both matter and will not go their own way when presented with
un-idiomatic/kludgy designs.

If you've read WebIDL recently, you've realize that only an ECMAScript 
binding is defined. I'm not sure anyone pretends there is another 
language consuming WebIDL.


I feel you have some misconceptions regarding WebIDL.

Perhaps there's some future world in which we decide that having an 
IDL to describe API invariants is a good idea (although it doesn't do 
that today to any reasonable degree), but nobody I know is clamoring 
for that.




Another thing to think about is whether reportURIs should
really be an IDL array (which does NOT produce a JS array on
the JS side, so it really depends on the expected use cases).


I'll advocate for a JS array wherever we surface an array-like
collection. It's long past time that we stopped shitting on users
with ad-hoc collection types.

Arguably, ES6 symbols may give a re-birth to ad-hoc collection
types by allowing safe (uncollidable) extension of built-ins. I
think an IDL array is fine (as far as I can tell, the difference
with a regular array is just a different prototype).


That's enough to make it toxic.
I don't understand this point. I'm not 100% up-to-date on ES6 classes, 
but it seems that WebIDL arrays are the equivalent of doing "class 
MadeUpName extends Array{}". If that's the case, do you think extending 
Array using ES6 classes is toxic as well?


David

Re: Sync API for workers

[David Bruant]

Alon Zakai wrote:
  Technically it might
  be possible to automatically rewrite such code to use asynchronous
  coding patterns, but so far I don't think anyone has managed to do
  that.
>>> Naively, I would say, that once we've paid the price to compile code
>>> from one language to another, you're not that far off from compiling
>>> to
>>> a given coding pattern. Especially compiling from LLVM bytecode.
>>> Regardless, has anyone tried at all? Has anyone tried to compile to
>>> JS+Promises or JS+task.js?
>> Consider even the simple and very common case of
>>
>> ..
>> while (!feof(f)) {
>>   fread(buf, 100, 1, f);
>>   ..process data in buf..
>> }
>> ..
>>
>> I don't have any good ideas for something asynchronous to compile this into 
>> that does not currently substantially harm performance. Compiling 
>> synchronous code into continuation passing style, generators, control flow 
>> emulation, or some other async style would greatly reduce performance.
I can imagine, it sounds hard indeed. Do you have numbers on how it
affects performance? Or an intuition on these numbers? I don't need to
be convinced that it affects performance significantly, but just to get
an idea.

I remember that at some point (your JSConf.eu talk last October), in
order to be able to compile through Emscripten, the source codebase (in
C/C++) had to be manually tweaked sometimes. Is it still the case? If
it's an acceptable thing to ask to authors, then would there be easy
ways for authors to make their IO blocking code more easily translated
to async JS code? I'm pessimistic, but it seems like an interesting
question to explore.

David

Re: Sync API for workers

[David Bruant]

[Forwarding a response from Alon Zakai, who is behind Emscripten and
CC'ing him]
>>>  There is also another use-case which has been brought up. As the
>>>  web
>>>  platform is becoming more powerful, people have started converting
>>>  code written for other platforms to javascript+html.
>> By "html", here, do you mean something else than?
>> Is there something that compiles any Windows/Mac/Linus UI framework
>> into
>> "HTML5"?
> There is pyjamas that compiles Python into JS and GWT that compiles Java into 
> JS. Both are UI frameworks, and being able to use sync calls in both would be 
> more natural since the original languages have lots of sync stuff I believe. 
> I don't know how important this use case is though. But we would like to 
> compile open source UI frameworks like Qt and GTK into JS using emscripten, 
> and sync would help a lot there.
>
>>>  For example the
>>>  emscipten[1] and mandreel[2] allow recompiling C++ code to
>>>  javascript
>>>  which is then run in a web browser.
>> I've been following loosely this topic and all examples I've seen
>> were
>> either about pure computation (like turning a C GZIP library in JS)
>> or
>> graphics stuffs (using canvas, hence my above question).
>>
>>>  Many times such code relies on APIs or libraries which contain
>>>  blockingcalls.
>> Do you have an example of that? I haven't seen one so far, but that's
>> an
>> interesting point.
>> Specifically, I can't recall having seen any C/C++->JS example that
>> were
>> doing IO (maybe files, but not network),
>> Last I heard, Emscripten compiles to JS from LLVM bytecode. I'm not
>> sure
>> they rely on any library containing blocking calls.
>> But as I said, I have been following that loosely.
> The normal C/C++ IO calls are all synchronous - fopen, fread, etc.
>
> As you said above, this is indeed less of a problem for pure computation. But 
> when compiling a complete game engine for example (like in BananaBread), you 
> need to handle everything a complete app needs, including synchronous IO. 
> Both file IO and and network IO (for multiplayer, downloading assets, etc. - 
> we haven't gotten around to that yet though in BananaBread) are relevant, as 
> well as synchronous GL operations using WebGL.
>
>>>  Technically it might
>>>  be possible to automatically rewrite such code to use asynchronous
>>>  coding patterns, but so far I don't think anyone has managed to do
>>>  that.
>> Naively, I would say, that once we've paid the price to compile code
>> from one language to another, you're not that far off from compiling
>> to
>> a given coding pattern. Especially compiling from LLVM bytecode.
>> Regardless, has anyone tried at all? Has anyone tried to compile to
>> JS+Promises or JS+task.js?
> Consider even the simple and very common case of
>
> ..
> while (!feof(f)) {
>   fread(buf, 100, 1, f);
>   ..process data in buf..
> }
> ..
>
> I don't have any good ideas for something asynchronous to compile this into 
> that does not currently substantially harm performance. Compiling synchronous 
> code into continuation passing style, generators, control flow emulation, or 
> some other async style would greatly reduce performance. In theory JS engines 
> could optimize those styles, but it would be hard and I don't think this is 
> at the top of anyone's list of priorities for any JS engine.
>
> Now, we could say that sync code, mainly IO, will run slowly but that is ok 
> because it's mainly just done during startup. That's true to some extent, but 
> startup is very important too, in BananaBread we already have 5-10 seconds or 
> so to load the entire game engine, a lot of which is file IO and processing, 
> and we have gotten requests to improve that as much as possible because it is 
> very significant for the user's initial impression.
>
> Best,
>   Alon Zakai

Re: Sync API for workers

[David Bruant]

Le 04/09/2012 20:47, Glenn Maynard a écrit :
> On Tue, Sep 4, 2012 at 12:49 PM, David Bruant  <mailto:bruan...@gmail.com>> wrote:
>
> I'm sorry, but I have to disagree. Have you ever used promises in
> a large-scale project?
> I've been amazed to discover that promise-based API are
> ridiculously much easier to refactor than callback-based API.
> Obviously, refactoring necessitates well-scoped state. I can't
> show the commit I have in mind, because it's in closed-source
> software, but really, a promise-based API isn't less
> understandable and less well-scoped. That statement is at the
> opposite direction of my experience these last 8 months.
>
>
> You have to choose between scoping state to a class (poor scoping) or
> in closures (hard to debug) instead of using locals in a call stack
> (tightly scoped and easy to debug); the overall current state of
> execution is much harder to see compared to a stack trace; the basic
> idea of stepping through code in a debugger scarcely translates at all.
Tooling isn't perfect for async debugging. It's being worked on. Yet it
hasn't prevented web devs from buiding (and debugging) event-based code.

As someone else said in another message, async isn't going away. There
won't be new blocking API for the main thread, so all the costs of
learning async programming will have to be paid. Debugging included.
I'm less and less convinced there is really something substancial to win
from the JS developer perspective.

For small scripts, it will be possible to use blocking APIs, but the
cost of async in small scripts is bearable. For big scripts, blocking
APIs induce a performance cost that soon makes people move to async.

>
> I understand and agree, but you're not addressing the problem of
> the resource waste I've mentionned above.
>
>
> I don't feel like I need to, because I expect this question was
> explored before workers were introduced in the first place.
It likely hasn't because workers do not have access to blocking APIs
except sync xhr. Are there examples in the wild of people creating new
workers when one is doing a sync xhr or do people just turn their code
into async when performance becomes an issue?
If the question has been explored before, can anyone point to the
answer? Otherwise, the debate won't move forward on that point.

> You apparently want to argue against *all* sync APIs, but you should
> do that separately, rather than singling out one sync API at random.
As I said in a previous message, I'm arguing against the waste of
resources due to blocking APIs. If a sync API makes an actual use of the
worker and CPU, that's excellent. If it's blocking on IO, it's wasting
resources that could be doing other computations.

David

Re: Sync API for workers

[David Bruant]

Le 04/09/2012 18:46, Glenn Maynard a écrit :
> On Tue, Sep 4, 2012 at 10:32 AM, David Bruant  <mailto:bruan...@gmail.com>> wrote:
>
> Cognitive load is the only one mentioned so far. It is a serious
> issue since for the foreseeable future, only human beings will be
> writing code.
>
> However, as said, there are solutions to reduce this load.
> I wish to share an experience.
> Back in April, I gave a JavaScript/jQuery training to people who
> knew programming, but didn't know JavaScript. I made the decision
> to teach promises right away (jQuery has them built-in, so that's
> easy). It seems that it helped a lot understanding async programming.
> The cognitive load has its solutions.
>
>
> (Understanding asynchronous programming isn't really the issue.  I'm
> sure everyone in this discussion has an intuitive grasp of that.)
>
> Those are attempts at making asynchronous code easier to write;
> they're not substitutes for synchronous code.  They still result in
> code with less understandable, well-scoped state.
I'm sorry, but I have to disagree. Have you ever used promises in a
large-scale project?
I've been amazed to discover that promise-based API are ridiculously
much easier to refactor than callback-based API. Obviously, refactoring
necessitates well-scoped state. I can't show the commit I have in mind,
because it's in closed-source software, but really, a promise-based API
isn't less understandable and less well-scoped. That statement is at the
opposite direction of my experience these last 8 months.


>
> This is a very interesting example and I realize that I have used
> "blocking" and "sync" interchangeably by mistake. I'm against
> blocking, but not sync.
> What I'm fundamentally (to answer what you said above) against is
> the idea of blocking a computation unit (like a worker) that does
> nothing but idly waits (for IO or a message for instance). It
> seems that proposals so far make the worker wait for a message and
> do nothing meanwhile and that's a pure waste of resources. A
> worker has been paid for (memory, init time...) and it's waiting
> while it could be doing other things.
> The current JS event loop run-to-completion model prevents that
> waste by design.
>
>
> Workers broke away from requiring the "do a bit of work then keep
> returning to the event loop" model of the UI thread from the start. 
> This is no different than the APIs we already have.  To take an
> earlier example:
>
> var worker = createDictionaryWorker();
> worker.postMessage("elephant");
> var definition = getMessage(worker); // wait for the answer
>
> This is no different than a sync XHR or IndexedDB call to do the same
> thing:
>
> var xhr = new XMLHttpRequest();
> xhr.open("GET", "/dictionary?elephant", false); // sync
> xhr.send();
> var definition = xhr.responseText;
>
> It simply allows workers, not just native code, to implement these
> APIs.  That's a natural step.
I understand and agree, but you're not addressing the problem of the
resource waste I've mentionned above.
Even if you're doing sync xhr in a worker, you're wasting the worker
time, because it could be computing other things while waiting for the
network to respond. That problem was obvious in the main thread because
it was resulting in poor user experience, but the problem still holds
with workers.
What do you do if your worker is busy idling while waiting for network,
but still need some other work to be done? Open another worker? And when
this one is idling and you need work done? Open another worker?

To oppose both things in the same sentence, is the readability worth the
waste of resources?
That's a genuine question. My experience with Node.js (which also
provides sync methods for IO) is that for small scripts, sync methods
are more convenient that callbacks or even promises. But arguably, for
small scripts, readability isn't that big of a concern by nature of a
small script.

David

Re: Sync API for workers

[David Bruant]

Le 04/09/2012 17:03, Glenn Maynard a écrit :
On Tue, Sep 4, 2012 at 4:23 AM, David Bruant <mailto:bruan...@gmail.com>> wrote:


The proposed solution here throws away all benefits of async code
to reduce the complexity of writing async code by... writing sync
code.

I wish we'd explore more solutions to make async more workable
rather than throwing away async.


It seems like you're thinking of asynchronous code as fundamentally 
better than synchronous code.  It's not; it has a set of 
advantages--ones that the Web needs badly for the UI thread, in order 
for scripts and the browser to coexist.  It also has a set of serious 
disadvantages.
Cognitive load is the only one mentioned so far. It is a serious issue 
since for the foreseeable future, only human beings will be writing 
code. However, as said, there are solutions to reduce this load.

I wish to share an experience.
Back in April, I gave a JavaScript/jQuery training to people who knew 
programming, but didn't know JavaScript. I made the decision to teach 
promises right away (jQuery has them built-in, so that's easy). It seems 
that it helped a lot understanding async programming.

The cognitive load has its solutions.

We're not throwing away async; we're bringing sync back into the game 
where it's appropriate.

True. I was exagerating a bit here :-)



The problem with blocking workers is that it may create a culture
of creating always more and more blocking workers (like Apache
creates more and more threads to handle more blocking connections).


You're not talking about this particular API here, you're talking 
about every sync API in workers.  Having sync APIs in workers and 
performing blocking tasks in workers isn't something new.


This is one of the big reasons that we have workers at all.

I had never heard this argument before the topic of sync messaging
API for workers. Where does it come from?
When I read the worker API, I see a way to create a new
computation unit and to send messages back and forth, nothing
about writing sync code.
Regardless of goal, do people actually write more sync code with
workers?


The very first example in the spec is doing work synchronously. 
http://www.whatwg.org/specs/web-apps/current-work/#a-background-number-crunching-worker
This is a very interesting example and I realize that I have used 
"blocking" and "sync" interchangeably by mistake. I'm against blocking, 
but not sync.
What I'm fundamentally (to answer what you said above) against is the 
idea of blocking a computation unit (like a worker) that does nothing 
but idly waits (for IO or a message for instance). It seems that 
proposals so far make the worker wait for a message and do nothing 
meanwhile and that's a pure waste of resources. A worker has been paid 
for (memory, init time...) and it's waiting while it could be doing 
other things.
The current JS event loop run-to-completion model prevents that waste by 
design.




Taras Glek seems to think that the local storage API (which is
sync) is not a good fit for workers [3]:
"We could expose this API to workers, but then we run into an
ethical question of bringing crappy APIs to new environments."
(the article mentions that the localStorage API is synchronous as
part of the "crappy" aspect of it)


(The synchronous part is bad for the UI thread, but not a problem in 
workers, so this isn't a very good argument, at least as summarized here.)


Many times such code relies on APIs or libraries which contain
blocking calls.


Do you have an example of that? I haven't seen one so far, but
that's an interesting point.
Specifically, I can't recall having seen any C/C++->JS example
that were doing IO (maybe files, but not network),


I believe that was his point--it's very hard to programmatically 
convert synchronous code to asynchronous code.
True, but I mean, we could have read intentions or blog posts of people 
saying "it's way too hard"




Technically it might be possible to automatically rewrite such
code to use asynchronous
coding patterns, but so far I don't think anyone has managed to do
that.

Naively, I would say, that once we've paid the price to compile
code from one language to another, you're not that far off from
compiling to a given coding pattern. Especially compiling from
LLVM bytecode.
Regardless, has anyone tried at all? Has anyone tried to compile
to JS+Promises or JS+task.js?
All of the compile-to-the-web movement started recently. Only now
do we start seeing what it can do. it's just the beginning.
Also, I wish the demand came from the people who do work on
Emscripten or Mandreel, that they came to standards mailing-lis

Re: Sync API for workers

[David Bruant]

Le 04/09/2012 14:34, Boris Zbarsky a écrit :

On 9/4/12 5:23 AM, David Bruant wrote:

Also, I wish the demand came from the people who do work on Emscripten
or Mandreel


As far as I know, what Jonas is saying about Emscripten did come from 
the Emscripten folks.  I've certainly seen them say it in bugs in the 
recent past.  No guessing involved on our part there.

Ok I wasn't aware of that. Do you have bug numbers in mind by any chance?
It however doesn't change that it could be tried to use promises or 
task.js to generate sync-like code more easily.


David

Re: Sync API for workers

[David Bruant]

Hi,

Before anything else, thanks for this detailed and quite complete 
explanation.


Le 03/09/2012 23:32, Jonas Sicking a écrit :

The other thing that I wanted to talk about is use cases. It has been
claimed in this thread that synchronous message passing isn't needed
and that people can just write code using async patterns. While this
is absolutely true, I would absolutely say that writing asynchronous
code is dramatically more complicated than writing synchronous code.
I acknowledge that writing async code may be hard for some with 
JavaScript-as-it-is.
The solution I have found personally to this issue is using promises 
which makes async code look sync (+some noise due to JS-the-language). 
Others think promises is a good solution [1]
Dave Herman has created task.js [2] to solve the same problem 
differently (promises+generators).
Some have created compile-to-JS languages (like Roy as I said in a 
previous message) to solve the problem in yet another way.

Others find other solutions.

All these solutions have in common that they reduce the complexity of 
writing/reading async code while keeping the benefits of it against a 
small layer of code.


The proposed solution here throws away all benefits of async code to 
reduce the complexity of writing async code by... writing sync code.


I wish we'd explore more solutions to make async more workable rather 
than throwing away async.
The problem with blocking workers is that it may create a culture of 
creating always more and more blocking workers (like Apache creates more 
and more threads to handle more blocking connections).
I understand the benefit that may come with a sync API, but we all know 
that APIs aren't used the way they're primarily intended and sometimes 
with very bad consequences. I'm afraid the bad consequences of a sync 
API misuse haven't been explored. Maybe soon you'll have people filing 
bugs telling "poor Firefox memory usage when using a lot of workers".



This is one of the big reasons that we have workers at all.
I had never heard this argument before the topic of sync messaging API 
for workers. Where does it come from?
When I read the worker API, I see a way to create a new computation unit 
and to send messages back and forth, nothing about writing sync code.

Regardless of goal, do people actually write more sync code with workers?
Taras Glek seems to think that the local storage API (which is sync) is 
not a good fit for workers [3]:
"We could expose this API to workers, but then we run into an ethical 
question of bringing crappy APIs to new environments." (the article 
mentions that the localStorage API is synchronous as part of the 
"crappy" aspect of it)



There is also another use-case which has been brought up. As the web
platform is becoming more powerful, people have started converting
code written for other platforms to javascript+html.

By "html", here, do you mean something else than ?
Is there something that compiles any Windows/Mac/Linus UI framework into 
"HTML5"?



For example the
emscipten[1] and mandreel[2] allow recompiling C++ code to javascript
which is then run in a web browser.
I've been following loosely this topic and all examples I've seen were 
either about pure computation (like turning a C GZIP library in JS) or 
graphics stuffs (using canvas, hence my above question).



Many times such code relies on APIs or libraries which contain blocking calls.
Do you have an example of that? I haven't seen one so far, but that's an 
interesting point.
Specifically, I can't recall having seen any C/C++->JS example that were 
doing IO (maybe files, but not network),
Last I heard, Emscripten compiles to JS from LLVM bytecode. I'm not sure 
they rely on any library containing blocking calls.

But as I said, I have been following that loosely.


Technically it might
be possible to automatically rewrite such code to use asynchronous
coding patterns, but so far I don't think anyone has managed to do
that.
Naively, I would say, that once we've paid the price to compile code 
from one language to another, you're not that far off from compiling to 
a given coding pattern. Especially compiling from LLVM bytecode.
Regardless, has anyone tried at all? Has anyone tried to compile to 
JS+Promises or JS+task.js?
All of the compile-to-the-web movement started recently. Only now do we 
start seeing what it can do. it's just the beginning.
Also, I wish the demand came from the people who do work on Emscripten 
or Mandreel, that they came to standards mailing-list saying "it took us 
a billion hours to compile I/O to JS correctly, we tried promises, 
task.js and it didn't help. It would have taken 10 seconds if we had a 
sync API". But I can't recall having read such a message yet. 
Compile-to-the-web is a complicated field and I wish we didn't try to 
guess for them what they need before they ask.


David

[1] http://jeditoolkit.com/2012/04/26/code-logic-not-mechanics.html#post
[2] http://taskjs.org/
[3] (see before-l

Re: Sync API for workers

[David Bruant]

Le 01/09/2012 22:30, Rick Waldron a écrit :
> On Saturday, September 1, 2012 at 4:02 PM, Glenn Maynard wrote:
>> On Sat, Sep 1, 2012 at 11:49 AM, David Bruant > <mailto:bruan...@gmail.com>> wrote:
>>> # Discussion 2
>>> ## Joshua Bell [5]
>>> "This can be done today using bidirectional postMessage, but of course
>>> this requires the Worker to then be coded in now common asynchronous
>>> JavaScript fashion, with either a tangled mess of callbacks or some sort
>>> of Promises/Futures library, which removes some of the benefits of
>>> introducing sync APIs to Workers in the first place."
>>> => What are these benefits?
>>
>> The benefit of being able to write linear code.  I don't think anyone
>> who's written complex algorithms in JavaScript can seriously dispute
>> this as anything but a huge win.
>
> I can seriously dispute this, as someone who involved in research and
> development of JavaScript programming for hardware. Processing high
> volume serialport IO is relatively simple with streams and data
> events. It's just a matter of thinking differently about the program.
I dispute it too. It's been 8 months I work with Node.js and have
written algorithms with IO. I've used promises and I'm very happy with
it when it comes to readability. The code is very close to being linear.
JavaScript, because of its syntax imposes some noise (especially the
"function" keyword and the Q library imposes a .then/.fail), but I'm
confident a language that compiles to JS could have sugar to eliminate
this issue.
I recommand taking a look at Roy for that inspiration on that topic
(start at ~8'00'') http://blip.tv/jsconf/jsconf2012-brian-mckenna-6145371
(it doesn't use promises, but adds sugar to get out of the callback hell)

David

Sync API for workers

[David Bruant]

Hi,

A Sync API for workers is being implemented in Firefox [1].
I'd like to come back to the discussions mentionned in comment 4 of the bug.

A summary of points I find important and my comments, questions and concerns

# Discussion 1
## Glenn Maynard [2] Use case exposed:
Ability to cancel long-running synchronous worker task
"Terminating the whole worker thread is the blunt way to do it; that's
no good since it requires starting a new thread for every keystroke, and
there may be significant startup costs (eg. loading search data)."
=> It's a legitimate use case that has no good solution today other than
cutting the task in smaller tasks between which a cancellation message
can be interleaved.


## Tab Atkins [3]
"If we were to fix this, it needs to be done at the language level,
because there are language-level issues to be solved that can't be
hacked around by a specialized solution."
=> I agree a lot with that point. This is a discussion that should be
had on es-discuss since JavaScript is the underlying language.
ECMAScript per se doesn't define a concurrency model and it's not even
on the table for ES.next, but might be in ES.next.next (7?). See [concurr]

## Jonas Sicking [4]
Ideas of providing control (read-only) over pending messages in workers.
(not part of the current Sync API, but interesting nonetheless)



# Discussion 2
## Joshua Bell [5]
"This can be done today using bidirectional postMessage, but of course
this requires the Worker to then be coded in now common asynchronous
JavaScript fashion, with either a tangled mess of callbacks or some sort
of Promises/Futures library, which removes some of the benefits of
introducing sync APIs to Workers in the first place."
=> What are these benefits? Is the cost of the Promises/Future library
so high it
Back to Tab's point of the previous discussion, this is a language
issue, not an API issue. It ought to be solved at the language level


## Rick Waldron [6]
"This is counter to the whole premise of Workers, which should be
independent of their renderer process and certainly not block themselves
while waiting for responses from the renderer (which inherently
describes an async behaviour)."
=> Indeed. Having a blocked worker makes that when you need other tasks
to happen in parallel, you need to spawn new workers which is a waste of
resources, very much like Apache which opens a new thread for each HTTP
connection while some thread are idling (I don't know if it's still the
case, but it used to)


## Glenn Maynard [7]
"I think this is a fundamental missing piece to worker communication.  A
basic reason for having Workers in the first place is so you can write
linear code, instead of having to structure code to be able to return
regularly (often awkward and inconvenient), but currently in order to
receive messages in workers you still have to do that."
=> A basic reason for having workers is to move computation away from
window to a concurrent and parallel computation unit so that the UI is
not blocked by computation. End of story. Nothing to do with writing
linear code. If JavaScript as it is doesn't allow people to write code
as they wish, once again, it's a language issue. Either ask a change in
the language or create a language that looks the way you want and
compiles down to JavaScript.



I wish to add that adding a sync API (even if the sync aspect is
asymetrical as proposed in [1]) breaks the event-loop run-to-completion
model of in-browser-JavaScript which is intended to be formalized at
[concurr]. This model is what prevents web pages from ever freezing from
a deadlock. The proposed API preserves this, but create the threat of
deadlocks for workers.

Besides programmer convenience, few arguments have been advanced to
justify the breakage of the current concurrency model (I don't even
think the breakage has been mentionned at all!). And as said several
times, programmer convenience are more of a language issue than an API
issue for the specific things we're talking about.
Also, I don't think I have seen mentionned use cases of things that are
not possible without a Sync API. Everything presented is already
possible (sometimes at arguably high costs like Glenn Maynard's use case
in discussion [1]).

David

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=783190
[2] http://lists.w3.org/Archives/Public/public-webapps/2010OctDec/1075.html
[3] http://lists.w3.org/Archives/Public/public-webapps/2010OctDec/1082.html
[4] http://lists.w3.org/Archives/Public/public-webapps/2010OctDec/1086.html
[5] http://lists.w3.org/Archives/Public/public-webapps/2011OctDec/0965.html
[6] http://lists.w3.org/Archives/Public/public-webapps/2011OctDec/0966.html
[7] http://lists.w3.org/Archives/Public/public-webapps/2011OctDec/0967.html
[concurr] http://wiki.ecmascript.org/doku.php?id=strawman:concurrency

Re: Synchronous postMessage for Workers?

[David Bruant]

Le 14/02/2012 14:31, Arthur Barstow a écrit :
> On 2/14/12 2:02 AM, ext David Bruant wrote:
>> Le 13/02/2012 20:44, Ian Hickson a écrit :
>>> Should we just tell authors to get used to the async style?
>> I think we should. More constructs are coming in ECMAScript. Things
>> related to language concurrency should probably be left to the core
>> language unless there is an extreme need (which there isn't as far as I
>> know).
>
> David - if you have some recommended reading(s) re concurrency
> constructs that are relevant to this discussion and are coming to
> ECMAScript, please let me know.
* http://wiki.ecmascript.org/doku.php?id=harmony:generators (with yield)
Yield has been for quite some time in SpiderMonkey. i'm not sure the
syntax in the spec (especially the "function*" functions) is
implemented, but there is a form of it anyway
Unless 2012 ends because of some Mayan calendar issues, generators will
be part of ECMAScript 6.
** Task.js by Dave Herman which uses generators. Built on top of
SpiderMonkey generators (so not the upcoming standard), but really worth
having a look at.
http://taskjs.org/
Related blog post:
http://blog.mozilla.com/dherman/2011/03/11/who-says-javascript-io-has-to-be-ugly/


* Concurrency: http://wiki.ecmascript.org/doku.php?id=strawman:concurrency
Probably one of the most ambitious and promising work. Did not cut it to
ECMAScript 6, but I'm hopeful will be part of ES7.
In a nutshell, this brings the event loop in ECMAScript itself. Even in
ES6, ECMAScript will have no concurrency mechanism whatsoever.
Concurrency is currently defined in HTML5 (event loop, setTimeout, etc.).
Another addition will be promises.
An already working example of promises can be found at
https://github.com/kriskowal/q
** This proposal is championned by Mark S. Miller (added in copy).
The strawman on concurrency seems largely inspired by what is done in
the E programming language. I highly recommend reading the Part III of
Mark's thesis on the topic: http://erights.org/talks/thesis/markm-thesis.pdf

> (I tend to agree if there isn't some real urgency here, we should be
> careful [with our specs]).
I've been participating on es-discuss for more than a year now and
following progress in implementations and things are moving. They are
moving in the right direction (browser vendors cooperate). Maybe slowly,
but they are moving.

I really think all topics that are low level (like concurrency) should
be left to ECMAScript now or addressed first on es-discuss before
thinking of a DOM/WebApp API. Low level things belong to ECMAScript and
any API we think of should be a library built on top of that. That's the
way I see it at least and I'd be happy to discuss if some disagree.

David

Re: Synchronous postMessage for Workers?

[David Bruant]

Le 13/02/2012 20:44, Ian Hickson a écrit :
> On Thu, 17 Nov 2011, Joshua Bell wrote:
>> Wouldn't it be lovely if the Worker script could simply make a 
>> synchronous call to fetch data from the Window?
> It wouldn't be so much a synchronous call, so much as a blocking get.
>
>
> On Thu, 17 Nov 2011, Jonas Sicking wrote:
>> We can only allow child workers to block on parent workers. Never the 
>> other way around.
> Indeed. And it would have to be limited to the built-in ports that workers 
> have, because regular ports can be shunted all over the place and you 
> could end up with a deadlock situation.
>
> It would be easy enough to add a blocking get on DedicatedWorkerGlobalScope.
> Is this something for which there's a lot of demand?
>
>// blocking get
>// no self.onmessage handler needed
>...
>var message = self.getMessage();
>...
>
> An alternative is to add continuations to the platform:
>
>// continuation
>// (this is not a formal proposal, just a illustration of the concept)
>var message;
>self.onmessage = function (event) {
>  message = event;
>  signal('got message'); // queues a task to resume from yieldUntil()
>};
>...
>yieldUntil('got message');
>...
>
> This would be a more general solution and would be applicable in many 
> other parts of the platform. As we get more and more async APIs, I think 
> it might be worth considering adding this.
>
> We could add it to HTML as an API rather than adding it to JS as a 
> language construct. It would be relatively easy to define, if much harder 
> to implement:
>
>yieldUntil(id) - spin the event loop until the signal() API unblocks 
> this script
>
>signal(id) - unblock the script with the oldest invokation of
> yieldUntil() called with the given ID, if any
>
> Given our definition of "spin the event loop", this doesn't even result, 
> per the spec, in nested event loops or anything like that. Note that per 
> this definition, "signal" just queues a task to resume the earlier script, 
> it is not your typical coroutine. That is:
>
>setTimeout(function () {
>  console.log('2');
>  signal('test');
>  console.log('3');
>}, 1000);
>console.log('1');
>yieldUntil('test');
>console.log('4');
>
> ...logs 1, 2, 3, 4, not 1, 2, 4, 3.
>
> Anyone object to me adding something like this?
If a web author does a typo (and we all know that especially with a
string-based API, it will happen or people will generate dynamically the
strings), what happens? Are you just blocked can can't resume after the
yieldUntil?

> Are there any better solutions?
"yield" as a keyword is considered being added to ECMAScript 6:
http://wiki.ecmascript.org/doku.php?id=harmony:generators#generator_functions
Web browsers will start implementing this one alongside with other new
ECMAScript features.
The yieldUntil/signal API would be probably a duplication of this.

I also recommand having a look at Dave Herman's Task.js http://taskjs.org/

> Should we just tell authors to get used to the async style?
I think we should. More constructs are coming in ECMAScript. Things
related to language concurrency should probably be left to the core
language unless there is an extreme need (which there isn't as far as I
know).

David

Re: [XHR][XHR2] Same-origin policy protection

[David Bruant]

Le 15/06/2011 19:39, Adam Barth a écrit :
> The server still needs to opt-in to allowing the web site to read the
> response or you get into trouble with firewalls.  This functionality
> is already available in every modern browser.
>
> Adam
>
>
> On Wed, Jun 15, 2011 at 10:15 AM, Charles Pritchard  wrote:
>> There have been a few requests for an XHR which does not expose session data 
>> to the target. I believe IE9 has an interface for this; I know it's been 
>> requested on chromium bug list.
>>
>>
>>
>> On Jun 15, 2011, at 9:18 AM, Boris Zbarsky  wrote:
>>
>>> On 6/15/11 6:43 AM, David Bruant wrote:
>>>> Could someone explain how running in a web browser justify such a
>>>> difference? For instance, could someone explain a threat particular to
>>>> cross-origin XHR in web browser?
>>> Off the top of my head:
>>>
>>> 1)  XHR in the web browser sends the user's cookies, HTTP auth credentials, 
>>> etc. with the request.  Which means that if you're logged in to some site 
>>> A, and cross-site XHR to that site is allowed from some other other site B, 
>>> then B can access all the information you can access due to being logged in 
>>> to site A.
As said by Charles, for this one, an option on the XHR object
(.withCredentials?) could allow cross-origin XHR without credentials and
refuse (throw a SECURITY_ERR) to perform cross-origin XHR if their are
credentials without needing a server-side opt-in.

>>> 2)  XHR in the web browser gives (at the moment, at least) sites that are 
>>> outside a firewall that your browser is behind the ability to make requests 
>>> to hosts that are behind the firewall.
>>>
>>> Item 2 is somewhat of an issue for installed apps as well, of course, but 
>>> installing an app is a trust decision.  I would imagine browsers could 
>>> relax some of these restrictions if a similar trust decision is explicitly 
>>> made for a non-locally-hosted app.
However, this point is far more tricky and may not have a solution at
the XHR interface level. That's the kind of answer I was looking for.
Thanks.
You wrote "at the moment, at least". Is there some planned change that
could question this?

Thanks all for your responses,

David

[XHR][XHR2] Same-origin policy protection

[David Bruant]

Hi,

I've been thinking a lot about same-origin policy recently. I understand 
the protection it provides when it comes to cross-frame communication, 
but I'm having a hard time understanding what it protects from when it 
comes to cross-origin XHR.
Over the years, web sites have moved to web apps and to just apps. These 
apps are client applications able to download content from different 
origin and mashup content. Interestingly, the notion of origin does not 
apply to these apps. Basically, being installed as independent pieces of 
software, rather than from loaded from a particular source in a web 
browser, they are origin-free. This already applied to other client 
applications such as crawlers.
To summurize, the same application (if written in JS for instance) could 
perform cross-domain XHR if installed as stand-alone, but cannot if 
running within a web browser (which granted it an origin and applied 
same-origin restrictions).


Could someone explain how running in a web browser justify such a 
difference? For instance, could someone explain a threat particular to 
cross-origin XHR in web browser?


Thanks,

David

Re: [XHR][XHR2] HTTP redirects / XHR Version detection

[David Bruant]

Le 30/01/2010 04:41, Anne van Kesteren a écrit :
On Sat, 23 Jan 2010 23:05:50 +0100, David Bruant  
wrote:
My comments and questions are based on the editor's drafts 
(http://dev.w3.org/2006/webapi/XMLHttpRequest/ and 
http://dev.w3.org/2006/webapi/XMLHttpRequest-2).


Between the two versions of the same-origin request event rules 
(#same-origin-request-event-rules), for the "If the response is an 
HTTP redirect" rule, the word "transparently" has disappeared. I 
understood the word "transparently" as "don't consider any other step 
of the algorithm" (like switching to the HEADERS_RECEIVED state, 
because if a redirect has been detected, it means that the headers 
has been received at least the location header).
The absence of this word in XHR2 suggests that the state should be 
switched to HEADERS_RECEIVED even if it is a redirect.


So, more generally, my question is : for HTTP redirects, should the 
implementations of XHR and XHR2 switch the state to HEADERS_RECIEVED 
? I don't know how it is currently implemented by the current 
browsers, but I would be in favor of switching the state (and thus 
dispatch an event) so that authors could write tools allowing them to 
have an idea of the number of redirects generated by some URL and 
then do an analysis of their websites (for instance all the links 
within a HTML page) "from the client side".


The idea is actually that a redirect does not involve a "state" 
transition. I have suggested on this mailing list a way to control 
whether redirects are followed but due to lack of response I have not 
done anything with it so far.

Ok.
The use case that could require to not follow silently or to follow 
silently but still leaving so kind of trace of redirects is a project I 
am currently working on : http://wiki.github.com/DavidBruant/linkChecker/
This is still very experimental (and a mess !) and not really ready to 
be deployed on actual websites. But the idea is to execute some JS on 
the client side and create a "report" of all resource links that can be 
found. Because it is run on the client side, if some content brought 
asynchronously contains links, these can be tested too (what is not true 
about this static checker : http://validator.w3.org/checklink).


I think that the "clients" of my library could be interested in knowing 
if some links involve none or one or a lot of redirects. Redirects imply 
back and forth communication between client and server, as a 
consequence, they can affect performance of applications.
For that reason, it could be interesting for me to provide the number of 
redirects and the different intermediate locations in the report. It 
would be a good way to provide a full analysis of the HTTP traffic.


Would it be clearer if after "Once all HTTP headers have been received 
and the asynchronous flag is true" I added "(and this is not an HTTP 
redirect)"?
I think that because receiving HTTP headers are chronologically the 
first thing that happens, it could be the first written :

"Once all HTTP headers have been received and the asynchronous flag is true"
"if it is an HTTP redirect then blabla"
"if it is not, blabla".

Now I know that the semantic of the HEADERS_RECEIVED state is not 
"headers received", but "headers of the expected/final resource 
received", I think it would deserve some sort of note at this
(http://dev.w3.org/2006/webapi/XMLHttpRequest-2/#headers-received-state) 
moment to explain that it is not only a matter of recieving "some/any" 
headers, but there is also the notion of final/expected resource.



My next question should find its answer thanks to the previous one, 
but juste to be sure : Is it possible that the state be to DONE and 
the status be a redirect (300 <= status < 400) ?


If there is no Location header this is possible. (Have not tested 
whether this is correctly implemented.)

Ok, thank you.

If the behavior has changed between XHR and XHR2, how authors can 
know what version is implemented by user agents ? In other words, is 
there a way defined by the spec to make the difference between a XHR 
"conforming user agent" and a XHR2 "conforming user agent" since the 
interface has the same name (XMLHttpRequest) ?


No, because in general user agents do not implement everything from 
one or the other. E.g. a couple of user agents implemented features 
from XHR2 but are not at all conforming to XHR1 requirements that did 
not change in XHR2. Implementations evolve organically, not in 
concrete steps.
So, why having two specifications in a working draft state ? Wouldn't it 
be easier for maintenability of the specs to have only the XHR2 spec and 
notes about legacy behaviors (and attributes/methods) ?

Will XHR1 reach the recommandation state one day ? Will it make sense ?

Thank you for the answer.

David

[XHR][XHR2] HTTP redirects / XHR Version detection

[David Bruant]

Hi,

My comments and questions are based on the editor's drafts 
(http://dev.w3.org/2006/webapi/XMLHttpRequest/ and 
http://dev.w3.org/2006/webapi/XMLHttpRequest-2).


Between the two versions of the same-origin request event rules 
(#same-origin-request-event-rules), for the "If the response is an HTTP 
redirect" rule, the word "transparently" has disappeared. I understood 
the word "transparently" as "don't consider any other step of the 
algorithm" (like switching to the HEADERS_RECEIVED state, because if a 
redirect has been detected, it means that the headers has been received 
at least the location header).
The absence of this word in XHR2 suggests that the state should be 
switched to HEADERS_RECEIVED even if it is a redirect.


So, more generally, my question is : for HTTP redirects, should the 
implementations of XHR and XHR2 switch the state to HEADERS_RECIEVED ? I 
don't know how it is currently implemented by the current browsers, but 
I would be in favor of switching the state (and thus dispatch an event) 
so that authors could write tools allowing them to have an idea of the 
number of redirects generated by some URL and then do an analysis of 
their websites (for instance all the links within a HTML page) "from the 
client side".


My next question should find its answer thanks to the previous one, but 
juste to be sure : Is it possible that the state be to DONE and the 
status be a redirect (300 <= status < 400) ?


If the behavior has changed between XHR and XHR2, how authors can know 
what version is implemented by user agents ? In other words, is there a 
way defined by the spec to make the difference between a XHR "conforming 
user agent" and a XHR2 "conforming user agent" since the interface has 
the same name (XMLHttpRequest) ?


Thanks,

David