Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Ian Hickson]

On Wed, 9 Dec 2009, Alexey Proskuryakov wrote:
> On 09.12.2009, at 10:07, Ian Hickson wrote:
> > > > 
> > > > I'm absolutely fine with changing WebSocket to change readyState 
> > > > as part of the same task that dispatches the event
> > > 
> > > Yes, that is what I think the spec should say.
> > 
> > Done.
> 
> As discussed on IRC, this leaves us with a race condition between 
> close() and queued messages. The spec says that once close() is invoked, 
> incoming data should be dropped, but queued tasks to dispatch message 
> event will still be coming.
> 
> This race condition can of course be fixed without much difficulty.

I believe it has been fixed, as per my last e-mail.


> It may be more difficult to similarly rearrange XHR specs.

I'm happy to work with Anne to address this as necessary.


> I find it unfortunate that where the spec says "queue a task", most (or 
> all) implementations will perform those actions immediately. This may be 
> indistinguishable from JavaScript client code, but it will be a source 
> of confusion for developers.

I disagree with your interpretation. Wherever the spec says "queue a 
task", this can be interpreted as meaning a message posted by a network 
thread to the main UI event loop, for UAs where this makes sense. However, 
there are many other UA implementation strategies where explicitly saying 
this would be even further from the actual implementation. I think it is 
inadvisable for specs to be defining things that are not black-box 
testable.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Alexey Proskuryakov]

On 09.12.2009, at 10:07, Ian Hickson wrote:


I'm absolutely fine with changing WebSocket to change readyState as
part of the same task that dispatches the event


Yes, that is what I think the spec should say.


Done.



As discussed on IRC, this leaves us with a race condition between  
close() and queued messages. The spec says that once close() is  
invoked, incoming data should be dropped, but queued tasks to dispatch  
message event will still be coming.


This race condition can of course be fixed without much difficulty. It  
may be more difficult to similarly rearrange XHR specs.


I find it unfortunate that where the spec says "queue a task", most  
(or all) implementations will perform those actions immediately. This  
may be indistinguishable from JavaScript client code, but it will be a  
source of confusion for developers.


- WBR, Alexey Proskuryakov

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Ian Hickson]

On Wed, 9 Dec 2009, Alexey Proskuryakov wrote:
> On 09.12.2009, at 10:07, Ian Hickson wrote:
> > > > 
> > > > I'm absolutely fine with changing WebSocket to change readyState 
> > > > as part of the same task that dispatches the event
> > > 
> > > Yes, that is what I think the spec should say.
> > 
> > Done.
> 
> With the updated text, there is a race condition between closing a 
> socket and removing all data that was received. The spec says "When a 
> user agent is to close the Web Socket connection, it must drop all 
> subsequent data from the server" - but with the current spec text, you 
> can get message events after you call close() from onopen handler.

Fixed. This also makes readyState change to CLOSED as soon as you call 
close() (the 'close' event is still queued, though, it doesn't fire 
synchronously during the close() call).

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Alexey Proskuryakov]

On 09.12.2009, at 10:07, Ian Hickson wrote:


I'm absolutely fine with changing WebSocket to change readyState as
part of the same task that dispatches the event


Yes, that is what I think the spec should say.


Done.


With the updated text, there is a race condition between closing a  
socket and removing all data that was received. The spec says "When a  
user agent is to close the Web Socket connection, it must drop all  
subsequent data from the server" - but with the current spec text, you  
can get message events after you call close() from onopen handler.


- WBR, Alexey Proskuryakov

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Ian Hickson]

On Wed, 9 Dec 2009, Jonas Sicking wrote:
> >
> > I'm absolutely fine with changing WebSocket to change readyState as 
> > part of the same task that dispatches the event
> 
> Yes, that is what I think the spec should say.

Done.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Jonas Sicking]

On Wed, Dec 9, 2009 at 9:53 AM, Ian Hickson  wrote:
> On Wed, 9 Dec 2009, Jonas Sicking wrote:
>> On Wed, Dec 9, 2009 at 9:34 AM, Ian Hickson  wrote:
>> > On Wed, 9 Dec 2009, Alexey Proskuryakov wrote:
>> >>
>> >> Data from network is only processed as an event loop task
>> >
>> > I do not believe this is accurate. It certainly isn't what is specced.
>>
>> In XMLHttpRequest this is certainly accurate. You will never see
>> .responseText changing in the middle of an execution thread, it's
>> always changed in response of a task run from the event loop.
>
> Sure. From the spec's point of view, a task is queued by the background
> network thread to update the responseText attribute. (I haven't checked if
> that's how Anne actually specced it, but something to that effect is
> needed, otherwise you can see the attribute changing from under you in a
> running script.)
>
> I'm absolutely fine with changing WebSocket to change readyState as part
> of the same task that dispatches the event

Yes, that is what I think the spec should say.

/ Jonas

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Ian Hickson]

On Wed, 9 Dec 2009, Jonas Sicking wrote:
> On Wed, Dec 9, 2009 at 9:34 AM, Ian Hickson  wrote:
> > On Wed, 9 Dec 2009, Alexey Proskuryakov wrote:
> >>
> >> Data from network is only processed as an event loop task
> >
> > I do not believe this is accurate. It certainly isn't what is specced.
> 
> In XMLHttpRequest this is certainly accurate. You will never see
> .responseText changing in the middle of an execution thread, it's
> always changed in response of a task run from the event loop.

Sure. From the spec's point of view, a task is queued by the background 
network thread to update the responseText attribute. (I haven't checked if 
that's how Anne actually specced it, but something to that effect is 
needed, otherwise you can see the attribute changing from under you in a 
running script.)

I'm absolutely fine with changing WebSocket to change readyState as part 
of the same task that dispatches the event, but we can't just say there's 
no task, that makes no sense.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Jonas Sicking]

On Wed, Dec 9, 2009 at 9:34 AM, Ian Hickson  wrote:
> On Wed, 9 Dec 2009, Alexey Proskuryakov wrote:
>>
>> Data from network is only processed as an event loop task
>
> I do not believe this is accurate. It certainly isn't what is specced.

In XMLHttpRequest this is certainly accurate. You will never see
.responseText changing in the middle of an execution thread, it's
always changed in response of a task run from the event loop. This
task can (and IMO should) synchronously fire the 'progress' event
(within the usual limitations of never firing that event more often
than every 50ms).

/ Jonas

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Ian Hickson]

On Wed, 9 Dec 2009, Alexey Proskuryakov wrote:
> 
> Data from network is only processed as an event loop task

I do not believe this is accurate. It certainly isn't what is specced.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Alexey Proskuryakov]

On 09.12.2009, at 4:39, Ian Hickson wrote:


I could change the spec to make the readyState attribute changes be
queued along with the event dispatch,


I do not understand why the events are queued, to begin with.


How else would it work? We can't interrupt a running script to run an
event dispatch, or fire an event in the middle of the HTML parser  
running,

or fire an event while a plugin is doing sychronous DOM changes -- we
always have to wait for the event loop's current running task to have
finished. That's what queuing a task means.


This is only an issue if WebSocket handling is performed in some other  
thread of execution (and even then, it's not really an issue). But I  
don't think that's the best way to think about it.


Data from network is only processed as an event loop task, so it's  
possible and desirable to dispatch events right away. Both single- 
process and multi-process implementations I know of do so - and even  
if some future implementation wants to run WebSocket code on a  
different thread, they can easily implement synchronous event dispatch  
by posting an event and waiting for it to be processed.


This is no different from how XMLHttpRequest works - browser engines  
do not need to queue tasks for onreadystate events, and do not - those  
are dispatched right away.



but then we'd have a situation whereby the "actual" state of the
websocket object might not match the state returned by the  
attribute.


This doesn't sounds like an issue to me. The "actual" state is always
out of sync, because client and server have different ideas of actual
state. So does an Ethernet controller, OS interrupt handler, low  
level
OS networking code, etc. The only state that matters client-side is  
what
JavaScript code sees, and all the intermediaries must (and do) act  
as if

that were the real thing.


I'm not talking about the state of the network, but about the state  
of the
object itself. You would end up in situations where, for instance,  
sending

would work fine, even though the state attribute said that you hadn't
connected yet.

I suppose we could get around that by saying that the attribute _is_  
the
state, so even if the object thinks it has a connection, the send()  
would
still fail until the first event has fired and the state has  
changed...

Would that be acceptable?



I don't understand what this means.

- WBR, Alexey Proskuryakov

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Ian Hickson]

On Fri, 4 Dec 2009, Alexey Proskuryakov wrote:
> On 03.12.2009, at 18:07, Ian Hickson wrote:
> 
> > I could change the spec to make the readyState attribute changes be 
> > queued along with the event dispatch,
> 
> I do not understand why the events are queued, to begin with. 

How else would it work? We can't interrupt a running script to run an 
event dispatch, or fire an event in the middle of the HTML parser running, 
or fire an event while a plugin is doing sychronous DOM changes -- we 
always have to wait for the event loop's current running task to have 
finished. That's what queuing a task means.


> > but then we'd have a situation whereby the "actual" state of the 
> > websocket object might not match the state returned by the attribute.
> 
> This doesn't sounds like an issue to me. The "actual" state is always 
> out of sync, because client and server have different ideas of actual 
> state. So does an Ethernet controller, OS interrupt handler, low level 
> OS networking code, etc. The only state that matters client-side is what 
> JavaScript code sees, and all the intermediaries must (and do) act as if 
> that were the real thing.

I'm not talking about the state of the network, but about the state of the 
object itself. You would end up in situations where, for instance, sending 
would work fine, even though the state attribute said that you hadn't 
connected yet.

I suppose we could get around that by saying that the attribute _is_ the 
state, so even if the object thinks it has a connection, the send() would 
still fail until the first event has fired and the state has changed... 
Would that be acceptable?

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Alexey Proskuryakov]

On 04.12.2009, at 14:30, Jonas Sicking wrote:


However for the events that are fired as a result of network activity,
I see no reason to fire these events asynchronously from that code.



Sounds like we're in complete agreement here. I've missed the change  
from sync to async dispatch when it was made in XHR specs (both v1 and  
v2), and I think that it should be reverted.


- WBR, Alexey Proskuryakov

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Jonas Sicking]

On Fri, Dec 4, 2009 at 9:52 AM, Alexey Proskuryakov  wrote:
>
> On 04.12.2009, at 4:12, Anne van Kesteren wrote:
>
>>> On the other hand, it can't possibly work like XMLHttpRequest - for
>>> whatever reason, the Web Sockets spec says that events are posted
>>> asynchronously. Maybe I'm not the last an only one to get confused by this
>>> difference from XMLHttpRequest events.
>>
>> XMLHttpRequest network events are "asynchronous" too.
>
>
> Looks like the spec says so now. Does any browser post XMLHttpRequest events
> asynchronously? This change in the spec is not harmless, as it seems to make
> readystatechange event pretty much useless.

Depends which events we are talking about. Events that are fired as a
result of network activity is inheritly asynchronous since network
events arrive asynchronously. For example "progress" events or "load"
events.

Other events must be synchronous, for example the "readystatechange"
event that is fired during the call to XMLHttpRequest.open, or the
"loadstart" event fired from XMLHttpRequest.send must be fired
synchronously.

However for the events that are fired as a result of network activity,
I see no reason to fire these events asynchronously from that code.
I.e. when a network request is finished, XMLHttpRequest takes several
actions:

* Sets readystate to 4
* Fires "readystatechange"
* Fires "load"
* Fires "loadend"
* Makes responseXML non-null

I see no reason to make these events be fired asynchronously *from the
code that takes all these actions*. In other words, I think the
XMLHttpRequest implementation should look something like:

XMLHttpRequest::OnRequestDone(status) {
  if (status != success) {
... error handling ...
return;
  }

  if (isXMLContentType(contentType)) {
responseXML = parseAsXML(responseText);
  }

  readystate = 4;
  synchronousDispatch("readystatechange");
  synchronousDispatch("load");
  synchronousDispatch("loadend");
}


Another way to look at it is that I think the following code:

xhr = new XMLHttpRequest;
timerId = setInterval(function() { if(xhr.responseXML)
alert(xhr.responseXML); }, 0);
xhr.open(...);
xhr.send();
xhr.onloadend = function() {
  cancelInterval(timerId);
}

should never ever fire the alert.

The whole purpose of firing events asynchronously is to aid
implementations so that they don't need to have a deep callstack when
the event fires, but instead can ensure to be in a stable state when
firing the event. However if the code in question is already started
asynchronously, such as as a result of a network event, then there is
no reason not to fire synchronously.

Firing events asynchronously always introduces hassles. Typically that
conditions change between when the event was scheduled to be fired,
and when the event actually fires. Many times this means that you have
to keep track of the task that will fire the event and under certain
conditions cancel it if what the event indicates is no longer true.
Thus I think we should as a general rule try to fire synchronously
when this doesn't risk destabilizing implementations.

/ Jonas

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Alexey Proskuryakov]

On 03.12.2009, at 18:07, Ian Hickson wrote:


I could change the spec to make the readyState attribute changes be
queued along with the event dispatch,


I do not understand why the events are queued, to begin with.  
Synchronous dispatch seems more useful to authors, as it gives more  
guarantees about connection state when handling the event.  
Implementations have always dealt with the necessary book-keeping to  
present asynchronous networking events in a synchronous manner, and I  
don't think think it's been a problem.


An implementation that keeps parts of WebSocket logic in a different  
thread or process will need to queue readyState event changes as you  
describe, but that will be an implementation detail.



but then we'd have a situation
whereby the "actual" state of the websocket object might not match the
state returned by the attribute.



This doesn't sounds like an issue to me. The "actual" state is always  
out of sync, because client and server have different ideas of actual  
state. So does an Ethernet controller, OS interrupt handler, low level  
OS networking code, etc. The only state that matters client-side is  
what JavaScript code sees, and all the intermediaries must (and do)  
act as if that were the real thing.


- WBR, Alexey Proskuryakov

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Alexey Proskuryakov]

On 04.12.2009, at 4:12, Anne van Kesteren wrote:

On the other hand, it can't possibly work like XMLHttpRequest - for  
whatever reason, the Web Sockets spec says that events are posted  
asynchronously. Maybe I'm not the last an only one to get confused  
by this difference from XMLHttpRequest events.


XMLHttpRequest network events are "asynchronous" too.



Looks like the spec says so now. Does any browser post XMLHttpRequest  
events asynchronously? This change in the spec is not harmless, as it  
seems to make readystatechange event pretty much useless.


- WBR, Alexey Proskuryakov

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Anne van Kesteren]

On Thu, 03 Dec 2009 22:43:10 +0100, Alexey Proskuryakov   
wrote:
On the other hand, it can't possibly work like XMLHttpRequest - for  
whatever reason, the Web Sockets spec says that events are posted  
asynchronously. Maybe I'm not the last an only one to get confused by  
this difference from XMLHttpRequest events.


XMLHttpRequest network events are "asynchronous" too.


--
Anne van Kesteren
http://annevankesteren.nl/

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Ian Hickson]

On Thu, 3 Dec 2009, Fumitoshi Ukai (��~\飼�~V~G�~U~O) wrote:
>
> I've question about thread to run Web Socket feedback from the protocol.
> 
> If server sends back handshake response and a data frame, and close 
> immediately, fast enough to run JavaScript on browser, how readyState 
> should be?
> 1) When client recognizes handshake response header, it changes readyState
> to OPEN  and queue a task to fire a simple event named "open".
> 2) Then client reads a data frame, and queue a task to fire a MessageEvent.
> 3) Then client recognizes the socket is closed, it changes readyState to
> CLOSED and queue a task to fire a simple event named "close".
> 
> I'm wondering this process should be done on the same thread of the
> JavaScript, or may be on the different thread.

It may be done on a different thread. (It doesn't _have_ to be done on a 
different thread; that would be indistinguishable from a slower network.)


> If it should run on the same thread,  1) would be processed after JavaScript
> code is processed. Thus, readyState is OPEN when it receives "open" event.
> However, if it may run on the different thread, 1)-3) might be processed
> before JavaScript code is processed.  So, readyState might be CLOSED when it
> receives "open" event.
> 
> Which is correct behavior?
> It seems WebKit/Mac does former, and Chromium does latter, so
> LayoutTests/websocket/tests/simple.html would fail with Chromium.

Both are correct. The test is incorrect in that it cannot predict what the 
readyState will be unless it has very strict timing relative to the 
server. (Such tests should exist, of course, but they are more complicated 
to write.)


On Thu, 3 Dec 2009, Alexey Proskuryakov wrote:
> 
> I'd expect it to work in the same way it works for XMLHttpRequest - 
> e.g., in an onreadystate handler of XMLHttpRequest, readyState does not 
> change due to background processing of incoming data.

On Thu, 3 Dec 2009, Alexey Proskuryakov wrote:
> 
> On the other hand, it can't possibly work like XMLHttpRequest - for 
> whatever reason, the Web Sockets spec says that events are posted 
> asynchronously. Maybe I'm not the last an only one to get confused by 
> this difference from XMLHttpRequest events.

I could change the spec to make the readyState attribute changes be 
queued along with the event dispatch, but then we'd have a situation 
whereby the "actual" state of the websocket object might not match the 
state returned by the attribute. Is that preferred?

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Alexey Proskuryakov]

On 03.12.2009, at 9:50, Alexey Proskuryakov wrote:

If server sends back handshake response and a data frame, and close  
immediately, fast enough to run JavaScript on browser, how  
readyState should be?


I'd expect it to work in the same way it works for XMLHttpRequest -  
e.g., in an onreadystate handler of XMLHttpRequest, readyState does  
not change due to background processing of incoming data.



On the other hand, it can't possibly work like XMLHttpRequest - for  
whatever reason, the Web Sockets spec says that events are posted  
asynchronously. Maybe I'm not the last an only one to get confused by  
this difference from XMLHttpRequest events.


In light of this, I think that the test in question is incorrect.

- WBR, Alexey Proskuryakov

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[Alexey Proskuryakov]

On 02.12.2009, at 23:46, Fumitoshi Ukai (鵜飼文敏) wrote:

If server sends back handshake response and a data frame, and close  
immediately, fast enough to run JavaScript on browser, how  
readyState should be?



I'd expect it to work in the same way it works for XMLHttpRequest -  
e.g., in an onreadystate handler of XMLHttpRequest, readyState does  
not change due to background processing of incoming data.


It seems WebKit/Mac does former, and Chromium does latter, so  
LayoutTests/websocket/tests/simple.html would fail with Chromium.


For those not involved with WebKit development, the test is available  
at , and its expected results at .


- WBR, Alexey Proskuryakov

Re: [whatwg] Thread to run Web Socket feedback from the protocol ?

[鵜飼文敏]

I've question about thread to run Web Socket feedback from the protocol.

If server sends back handshake response and a data frame, and close
immediately, fast enough to run JavaScript on browser, how readyState should
be?
1) When client recognizes handshake response header, it changes readyState
to OPEN  and queue a task to fire a simple event named "open".
2) Then client reads a data frame, and queue a task to fire a MessageEvent.
3) Then client recognizes the socket is closed, it changes readyState to
CLOSED and queue a task to fire a simple event named "close".

I'm wondering this process should be done on the same thread of the
JavaScript, or may be on the different thread.
If it should run on the same thread,  1) would be processed after JavaScript
code is processed. Thus, readyState is OPEN when it receives "open" event.
However, if it may run on the different thread, 1)-3) might be processed
before JavaScript code is processed.  So, readyState might be CLOSED when it
receives "open" event.

Which is correct behavior?
It seems WebKit/Mac does former, and Chromium does latter, so
LayoutTests/websocket/tests/simple.html would fail with Chromium.

-- 
ukai