Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, Jun 24, 2011 at 4:27 PM, Kenneth Russell wrote: > On Fri, Jun 24, 2011 at 3:43 PM, Ian Hickson wrote: >> On Fri, 24 Jun 2011, Kenneth Russell wrote: >>> >>> Slightly larger issue. In the typed array spec, views like Float32Array >>> refer to an ArrayBuffer instance. It's desired to be able to transfer >>> multiple views of the same ArrayBuffer in the same postMessage call. >>> Currently, because each Transferable is transferred independently, >>> transferring the first view will cause the view and underlying >>> ArrayBuffer to be neutered, so upon encountering the second view, an >>> exception will be thrown since its ArrayBuffer was already transferred. >> >> The views shouldn't be Transferable. Only the buffer should be. The views >> should continue to have the behaviour you had described before, where they >> recurse to clone their buffer then just clone the view. Since the buffers >> would already be transferred (the transfering happens before the cloning), >> it'll all just work. > > Thanks, I think I see how this is supposed to work now. I'll rewrite > the relevant sections of the typed array spec soon and ping you or > post again if it looks like there are any other issues. I apologize for how long this took, but the editor's draft of the typed array spec has finally been updated with these changes. I found it a little difficult to specify the desired behavior; please let me know if you see a better or simpler way. https://www.khronos.org/registry/typedarray/specs/latest/ -Ken
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, Jun 24, 2011 at 3:43 PM, Ian Hickson wrote: > On Fri, 24 Jun 2011, Kenneth Russell wrote: >> >> Slightly larger issue. In the typed array spec, views like Float32Array >> refer to an ArrayBuffer instance. It's desired to be able to transfer >> multiple views of the same ArrayBuffer in the same postMessage call. >> Currently, because each Transferable is transferred independently, >> transferring the first view will cause the view and underlying >> ArrayBuffer to be neutered, so upon encountering the second view, an >> exception will be thrown since its ArrayBuffer was already transferred. > > The views shouldn't be Transferable. Only the buffer should be. The views > should continue to have the behaviour you had described before, where they > recurse to clone their buffer then just clone the view. Since the buffers > would already be transferred (the transfering happens before the cloning), > it'll all just work. Thanks, I think I see how this is supposed to work now. I'll rewrite the relevant sections of the typed array spec soon and ping you or post again if it looks like there are any other issues. -Ken
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, 24 Jun 2011, Kenneth Russell wrote: > > Slightly larger issue. In the typed array spec, views like Float32Array > refer to an ArrayBuffer instance. It's desired to be able to transfer > multiple views of the same ArrayBuffer in the same postMessage call. > Currently, because each Transferable is transferred independently, > transferring the first view will cause the view and underlying > ArrayBuffer to be neutered, so upon encountering the second view, an > exception will be thrown since its ArrayBuffer was already transferred. The views shouldn't be Transferable. Only the buffer should be. The views should continue to have the behaviour you had described before, where they recurse to clone their buffer then just clone the view. Since the buffers would already be transferred (the transfering happens before the cloning), it'll all just work. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 23, 2011 at 4:52 PM, Ian Hickson wrote: > On Tue, 21 Jun 2011, Ian Hickson wrote: >> >> How about we just make postMessage() take the object to clone in the first >> argument, an array of objects to transfer in the second; on the other >> side, the author receives the object cloned, with anything listed in the >> array and in the structured data being transferred instead of cloned, and, >> in addition, in the event.ports array, a list of the ports that were given >> in the transfer array. >> >> This has the advantage of being completely backwards-compatible. >> >> So for example, on the sending side: >> >> postMessage(['copied', copiedArrayBuffer, transferredArrayBuffer, >> transferredPort1], // could be an object too >> [transferredArrayBuffer, transferredPort1, >> transferredPort2]); >> >> ...on the receiving side, the event has >> >> data == ['copied', newCopiedArrayBuffer, newTransferredArrayBuffer, >> newTransferredPort1]; >> ports == [newTransferredPort1, newTransferredPort2]; >> >> It's not going to win any design awards, but I think that boat has sailed >> for this particular API anyway, given the contraints we're operating under. > > Since no serious problems were raised with this and it seems to address > all the constraints, I've now specced this. > > One edge case that wasn't mentioned above is what happens when a non-port > Transferable object is in the second list but not the first. I defined it > such that it still gets cloned (and thus the original is neutered), but it > is then discarded. (That definition more or less fell out of the way one'd > have to implement this to make it simple to understand, but I'm happy to > do it a different way if there's a good reason to.) > > http://html5.org/tools/web-apps-tracker?from=6272&to=6273 Thanks, this looks great! Minor issue: there are a few places where "transfered" should be "transferred". Slightly larger issue. In the typed array spec, views like Float32Array refer to an ArrayBuffer instance. It's desired to be able to transfer multiple views of the same ArrayBuffer in the same postMessage call. Currently, because each Transferable is transferred independently, transferring the first view will cause the view and underlying ArrayBuffer to be neutered, so upon encountering the second view, an exception will be thrown since its ArrayBuffer was already transferred. This could happen with any Transferable object that has a reference to another Transferable, so I don't think it's a problem overly specific to typed arrays. The way this was addressed in the current typed array strawman proposals was to split the transfer operation into two stages: cloning of, and closing of, the original object. First, all transferable objects were cloned, and then they all were closed. See http://www.khronos.org/registry/typedarray/specs/latest/#9.2 . This can be addressed in the current spec by stating in http://www.whatwg.org/specs/web-apps/current-work/multipage/urls.html#transferable-objects that the transfer map is provided to the steps defined by the type of the object in question. That way, transferring of a particular object can update the map during the transfer operation, possibly adding more associations to it. The map will provide enough state to allow transfer of dependent transferable objects. Thoughts? Should I file a bug about this? > kbr: Feel free to ping me if you need advice on how to use this with > ArrayBuffer in the short term. On the long term I'm happy to just spec > this in the same spec as the rest of this stuff. Thanks. In the long run I'd be very happy to have the semantics for ArrayBuffer and views defined in this spec. In the short term, I'd like to prototype this first and get some experience with it. -Ken
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Tue, 21 Jun 2011, Ian Hickson wrote: > > How about we just make postMessage() take the object to clone in the first > argument, an array of objects to transfer in the second; on the other > side, the author receives the object cloned, with anything listed in the > array and in the structured data being transferred instead of cloned, and, > in addition, in the event.ports array, a list of the ports that were given > in the transfer array. > > This has the advantage of being completely backwards-compatible. > > So for example, on the sending side: > >postMessage(['copied', copiedArrayBuffer, transferredArrayBuffer, > transferredPort1], // could be an object too >[transferredArrayBuffer, transferredPort1, > transferredPort2]); > > ...on the receiving side, the event has > >data == ['copied', newCopiedArrayBuffer, newTransferredArrayBuffer, > newTransferredPort1]; >ports == [newTransferredPort1, newTransferredPort2]; > > It's not going to win any design awards, but I think that boat has sailed > for this particular API anyway, given the contraints we're operating under. Since no serious problems were raised with this and it seems to address all the constraints, I've now specced this. One edge case that wasn't mentioned above is what happens when a non-port Transferable object is in the second list but not the first. I defined it such that it still gets cloned (and thus the original is neutered), but it is then discarded. (That definition more or less fell out of the way one'd have to implement this to make it simple to understand, but I'm happy to do it a different way if there's a good reason to.) http://html5.org/tools/web-apps-tracker?from=6272&to=6273 kbr: Feel free to ping me if you need advice on how to use this with ArrayBuffer in the short term. On the long term I'm happy to just spec this in the same spec as the rest of this stuff. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Tue, Jun 21, 2011 at 11:43 PM, Glenn Maynard wrote:
> On Wed, Jun 22, 2011 at 1:57 AM, David Levin wrote:
>>
>> Let's say the call doesn't throw when given a type B that isn't
>> transferrable.
>> Let's also say some later changes the javascript code and uses B after the
>> postMessage call.
>> Everything work. No throw is done and B isn't "gutted" because it isn't
>> transferrable.
>
> Throwing for unsupported objects will break common, reasonable uses, making
> everyone jump hoops to use transfer. Not throwing will only break code that
> seriously misuses the API, by listing objects for transfer and then
> continuing to use the object.
>
> Anyway, if this exception is thrown, everyone's going to use a helper like
> this:
>
> function filterTransferrable(list) {
> var ret = [];
> for(var i = 0; i < list.length; ++i) {
> if(list[i] instanceof Transferrable)
> ret.push(list[i]);
> }
> return ret;
> }
>
> postMessage([A, B], filterTransferrable([A, B]));
>
> ... which will trigger the case you describe anyway.
The structured cloning algorithm itself will throw an exception if it
encounters an object type it doesn't know how to handle, in particular
a host object. See
http://www.whatwg.org/specs/web-apps/current-work/multipage/urls.html#safe-passing-of-structured-data
. As browsers evolve, it will be host object types that are promoted
to transferable status. Therefore, applications that are written for
browser version N+1 which supports transferring host object type Foo
will throw an exception on browser version N when sending a Foo via
postMessage simply because of its presence in the object graph,
regardless of its presence in the transfer array.
For this reason, and because I also prefer fail-fast APIs, I agree
that unsupported objects in the transfer array should raise an
exception.
I doubt that authors will write a filter for the transferable array as
you suggest, since filtering of the object graph would be necessary
too. Rather, I think that applications will be written for browsers
that support transferring host objects of a particular type.
The current proposal sounds good to me.
-Ken
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 22, 2011 at 2:31 AM, Glenn Maynard wrote: > On Wed, Jun 22, 2011 at 4:33 AM, David Levin wrote: > >> Making people use a helper function like that is just making them jump an >>> unnecessary hoop. >>> >> >> It makes them jump through another hoop to potentially misuse the api. >> > > No, it's another hoop that *everyone* has to jump through to use the API at > all, so code you write in browser N+1 would also work in browser N where > fewer classes support transfer. > > Jumping that hoop is not the misuse; it's a direct requirement of the API. > *Because* everyone would be doing that, the misuse will also be possible. > The throwing aspect is a useful debugging tool to warn developers about misuse that they wouldn't be aware of otherwise. (Not everyone will read the spec as closely as you do -- may will just write code to get the job done and declare it done when it works -- a not throwing api may still be fast but will be deceptive in allowing parameters and not acting on them leading to the discussed misuse by accident -- it happens all the time by people who are trying to be careful even.) Your given code isn't the only possible solution. However, it is one that imposes M choose N combinations for the code resulting in lots of testing combinations. I could easily envision several alternatives to your snippet which could be written which would be better. For example, one could write code that either send all parameters using fast path or none. (This is only two test cases and they could have a debug switch to make everything be passed in the fast method to ensue that it worked correctly -- note this isn't possible if the api doesn't throw.) I'm fairly certain that we won't agree. Due to my many experiences with people including myself misusing apis and not realizing it, I strongly prefer an api that warns of misuse. You don't. I see either of us saying anything new so I don't plan to continue this discussion further because I believe that we have both laid our points of view in vivid detail :). best wishes, dave
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 22, 2011 at 2:31 AM, Glenn Maynard wrote: > On Wed, Jun 22, 2011 at 4:33 AM, David Levin wrote: > >> Making people use a helper function like that is just making them jump an >>> unnecessary hoop. >>> >> >> It makes them jump through another hoop to potentially misuse the api. >> > > No, it's another hoop that *everyone* has to jump through to use the API at > all, so code you write in browser N+1 would also work in browser N where > fewer classes support transfer. > I do understand Glenn's point about convenience, but I generally strongly prefer APIs that throw exceptions when passed unsupported arguments over APIs that silently ignore those arguments.
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 22, 2011 at 4:33 AM, David Levin wrote: > Making people use a helper function like that is just making them jump an >> unnecessary hoop. >> > > It makes them jump through another hoop to potentially misuse the api. > No, it's another hoop that *everyone* has to jump through to use the API at all, so code you write in browser N+1 would also work in browser N where fewer classes support transfer. Jumping that hoop is not the misuse; it's a direct requirement of the API. *Because* everyone would be doing that, the misuse will also be possible. Also, I haven't seen mention of Transferrable else where in the final > proposed solution which you used in that code. > It's an interface to indicate that an object is transferrable, to allows feature testing. http://krijnhoetmer.nl/irc-logs/whatwg/20110617#l-1427 -- Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 22, 2011 at 12:26 AM, Glenn Maynard wrote: > On Wed, Jun 22, 2011 at 3:14 AM, David Levin wrote: > >> On Tue, Jun 21, 2011 at 11:43 PM, Glenn Maynard wrote: >> >>> On Wed, Jun 22, 2011 at 1:57 AM, David Levin wrote: >>> Let's say the call doesn't throw when given a type B that isn't transferrable. Let's also say some later changes the javascript code and uses B after the postMessage call. Everything work. No throw is done and B isn't "gutted" because it isn't transferrable. >>> >>> Throwing for unsupported objects will break common, reasonable uses, >>> making everyone jump hoops to use transfer. Not throwing will only break >>> code that seriously misuses the API, by listing objects for transfer and >>> then continuing to use the object. >>> >> >> Code always seriously misuses apis. See Raymond Chen's blog for numerous >> examples if you have any doubt (http://blogs.msdn.com/b/oldnewthing/). >> > > You didn't respond to the rest of my mail, where I pointed out that the > misuse case will end up broken anyway as everyone will likely use a wrapper > to get the no-exception behavior. You'd have to, to support older browsers > which don't support transfer for as many types. > If you insist Making people use a helper function like that is just making them jump an > unnecessary hoop. > It makes them jump through another hoop to potentially misuse the api. It shouldn't be simple to misuse apis. Also, I haven't seen mention of Transferrable else where in the final proposed solution which you used in that code. dave
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 22, 2011 at 3:14 AM, David Levin wrote: > On Tue, Jun 21, 2011 at 11:43 PM, Glenn Maynard wrote: > >> On Wed, Jun 22, 2011 at 1:57 AM, David Levin wrote: >> >>> Let's say the call doesn't throw when given a type B that isn't >>> transferrable. >>> Let's also say some later changes the javascript code and uses B after >>> the postMessage call. >>> Everything work. No throw is done and B isn't "gutted" because it isn't >>> transferrable. >>> >> >> Throwing for unsupported objects will break common, reasonable uses, >> making everyone jump hoops to use transfer. Not throwing will only break >> code that seriously misuses the API, by listing objects for transfer and >> then continuing to use the object. >> > > Code always seriously misuses apis. See Raymond Chen's blog for numerous > examples if you have any doubt (http://blogs.msdn.com/b/oldnewthing/). > You didn't respond to the rest of my mail, where I pointed out that the misuse case will end up broken anyway as everyone will likely use a wrapper to get the no-exception behavior. You'd have to, to support older browsers which don't support transfer for as many types. Making people use a helper function like that is just making them jump an unnecessary hoop. -- Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Tue, Jun 21, 2011 at 11:43 PM, Glenn Maynard wrote: > On Wed, Jun 22, 2011 at 1:57 AM, David Levin wrote: > >> Let's say the call doesn't throw when given a type B that isn't >> transferrable. >> Let's also say some later changes the javascript code and uses B after the >> postMessage call. >> Everything work. No throw is done and B isn't "gutted" because it isn't >> transferrable. >> > > Throwing for unsupported objects will break common, reasonable uses, making > everyone jump hoops to use transfer. Not throwing will only break code that > seriously misuses the API, by listing objects for transfer and then > continuing to use the object. > Code always seriously misuses apis. See Raymond Chen's blog for numerous examples if you have any doubt (http://blogs.msdn.com/b/oldnewthing/).
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 22, 2011 at 1:57 AM, David Levin wrote:
> Let's say the call doesn't throw when given a type B that isn't
> transferrable.
> Let's also say some later changes the javascript code and uses B after the
> postMessage call.
> Everything work. No throw is done and B isn't "gutted" because it isn't
> transferrable.
>
Throwing for unsupported objects will break common, reasonable uses, making
everyone jump hoops to use transfer. Not throwing will only break code that
seriously misuses the API, by listing objects for transfer and then
continuing to use the object.
Anyway, if this exception is thrown, everyone's going to use a helper like
this:
function filterTransferrable(list) {
var ret = [];
for(var i = 0; i < list.length; ++i) {
if(list[i] instanceof Transferrable)
ret.push(list[i]);
}
return ret;
}
postMessage([A, B], filterTransferrable([A, B]));
... which will trigger the case you describe anyway.
--
Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Tue, Jun 21, 2011 at 10:48 PM, Glenn Maynard wrote: > On Wed, Jun 22, 2011 at 1:25 AM, David Levin wrote: > >> On Tue, Jun 21, 2011 at 9:27 PM, Glenn Maynard wrote: >> >>> What happens if an object is included in the second list that doesn't >>> support transfer? Ian said that it would throw, but I'm not sure that's >>> best. >>> >> >> If it doesn't throw, doesn't that introduce the backwards compat issue >> when something new is supported that wasn't before? >> > > The backwards-compat issue that we've talked about before is when transfer > happens without opting into it explicitly for each object or type. For > example, transferEverythingPossible([A, B]) would cause this problem: if A > supports transfer when you write the code and B does not, then B gaining > support a year later might break your code. > > I can't think of backwards-compat issues with not throwing. Can you give > an example? > You just gave it! :) Let's say the call doesn't throw when given a type B that isn't transferrable. Let's also say some later changes the javascript code and uses B after the postMessage call. Everything work. No throw is done and B isn't "gutted" because it isn't transferrable. Now let's say Firefox supports transferring B, the site breaks because it was never tested like this, so Firefox has to choose between compatibility and a supporting transferring B. (Of course, this will be in some sample that everyone copies :).) dave
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 22, 2011 at 1:25 AM, David Levin wrote: > On Tue, Jun 21, 2011 at 9:27 PM, Glenn Maynard wrote: > >> What happens if an object is included in the second list that doesn't >> support transfer? Ian said that it would throw, but I'm not sure that's >> best. >> > > If it doesn't throw, doesn't that introduce the backwards compat issue when > something new is supported that wasn't before? > The backwards-compat issue that we've talked about before is when transfer happens without opting into it explicitly for each object or type. For example, transferEverythingPossible([A, B]) would cause this problem: if A supports transfer when you write the code and B does not, then B gaining support a year later might break your code. I can't think of backwards-compat issues with not throwing. Can you give an example? -- Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Tue, Jun 21, 2011 at 9:27 PM, Glenn Maynard wrote: > What happens if an object is included in the second list that doesn't > support transfer? Ian said that it would throw, but I'm not sure that's > best. > If it doesn't throw, doesn't that introduce the backwards compat issue when something new is supported that wasn't before? > > Suppose Firefox N supports transferring ArrayBuffer, and Firefox N+1 adds > support for transferring ImageData. Developers working with Firefox N+1 > write the following: > >postMessage(obj, [obj.anArrayBuffer, obj.anImageData]); > > On Firefox N+1, both objects will be transferred, mutating the sender's > copy. In Firefox N, only the ArrayBuffer will be transferred, and the > ImageData is cloned. Developers don't need to write wrappers to scan > through the list and remove objects which don't support transfer. They > don't have to test code with every version of browsers to make sure that > their transfer lists are created correctly for every possible combination of > supported object transfers. They just list the objects which they're > prepared to have mutated and will discard after sending the message. > > > > postMessage([thisArrayBufferIsCopied, thisPortIsTransferred], > > [thisPortIsTransferred]); > > > > This also has the benefit of being backwards-compatible, at least if I > > keep an attribute on the event on the other side called "ports" that > > includes the transferred objects (maybe another attribute should be > > included that also returns the same array, since 'ports' would now be a > > confusing misnomer). > > I don't think the ports array should be expanded to include all transferred > objects. This would turn the list into part of the user's messaging > protocol, which I think is inherently less clear. Protocols using this API > will be much cleaner when they're based on a single object graph. > > I'd recommend that the "ports" array contain only the transferred ports, > for backwards-compatibility; anything else transferred should be accessed > via the object graph. > > This also means that the transfer list has no visible effects to receivers. > Senders can choose to add or not add objects for transfer based on their > needs, without having to worry that the receiver of the message might depend > on the transfer list having a particular format (with the exception of > message ports). Having the transfer list both act as part of the messaging > protocol *and* change the side-effects for the sender will create conflicts, > where you'll want to clone (not transfer) an object but be forced to include > it in the transfer list to satisfy the receiver. > > -- > Glenn Maynard >
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
What happens if an object is included in the second list that doesn't support transfer? Ian said that it would throw, but I'm not sure that's best. Suppose Firefox N supports transferring ArrayBuffer, and Firefox N+1 adds support for transferring ImageData. Developers working with Firefox N+1 write the following: postMessage(obj, [obj.anArrayBuffer, obj.anImageData]); On Firefox N+1, both objects will be transferred, mutating the sender's copy. In Firefox N, only the ArrayBuffer will be transferred, and the ImageData is cloned. Developers don't need to write wrappers to scan through the list and remove objects which don't support transfer. They don't have to test code with every version of browsers to make sure that their transfer lists are created correctly for every possible combination of supported object transfers. They just list the objects which they're prepared to have mutated and will discard after sending the message. > postMessage([thisArrayBufferIsCopied, thisPortIsTransferred], > [thisPortIsTransferred]); > > This also has the benefit of being backwards-compatible, at least if I > keep an attribute on the event on the other side called "ports" that > includes the transferred objects (maybe another attribute should be > included that also returns the same array, since 'ports' would now be a > confusing misnomer). I don't think the ports array should be expanded to include all transferred objects. This would turn the list into part of the user's messaging protocol, which I think is inherently less clear. Protocols using this API will be much cleaner when they're based on a single object graph. I'd recommend that the "ports" array contain only the transferred ports, for backwards-compatibility; anything else transferred should be accessed via the object graph. This also means that the transfer list has no visible effects to receivers. Senders can choose to add or not add objects for transfer based on their needs, without having to worry that the receiver of the message might depend on the transfer list having a particular format (with the exception of message ports). Having the transfer list both act as part of the messaging protocol *and* change the side-effects for the sender will create conflicts, where you'll want to clone (not transfer) an object but be forced to include it in the transfer list to satisfy the receiver. -- Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Tue, Jun 21, 2011 at 11:07 AM, Ian Hickson wrote:
> On Tue, 21 Jun 2011, Jonas Sicking wrote:
>> On Tue, Jun 21, 2011 at 12:39 AM, Ian Hickson wrote:
>> > On Mon, 20 Jun 2011, Jonas Sicking wrote:
>> >>
>> >> If data appears both in the .ports array and the .data property, then
>> >> people will be tempted to create protocols which only work if the
>> >> array buffer is transferred, i.e. if the receiver only looks in
>> >> .ports. I.e. people will likely end up with equally ugly solutions
>> >> like:
>> >>
>> >> postMessage('vend-reply', [createClone(arraybuffer)]);
>> >
>> > Yeah, that's a very good point.
>> >
>> > It really seems like part of the problem here is that ports and
>> > arrayBuffers are quite different (in that one will always want to be
>> > transferred, but for the other you might want to sometimes transfer
>> > and sometimes not).
>>
>> I don't think we should worry too much about suboptimal syntax when
>> transferring ports. We can always add more syntax sugar later if people
>> think it's annoying. Better keep the initial version simple than to try
>> to guess which use cases will be the most common ones.
>
> Keeping it simple is one of the reasons I don't want to require that
> authors list every port twice in order to send a port. (Sending a port is
> intended to be a common pattern, at least if the API is used as intended.)
>
>
> So the current situation is as follows:
>
> - we want to be able to send structured data that is cloned.
> - we want the structured data to be able to contain arraybuffers and ports.
> - we want it to be possible for the author to opt-in to transferring
> specific arraybuffers rather than just cloning them.
> - we want the author to have to explicitly list the ports that are to be
> transferred because it would be easy to accidentally transfer one
> otherwise, and that would cause hard-to-find bugs (e.g. if the data
> being cloned happened to contain debugging information that happened to
> contain a reference to a port).
> - we don't want to require that authors list ports twice in the simple
> case, since that's quite ugly
> - we don't want authors to design APIs where arraybuffers can only be
> transferred and not copied; the receiving side should not be able to
> influence the sending side's decision as to whether to clone or
> transfer arraybuffers.
>
> How about we just make postMessage() take the object to clone in the first
> argument, an array of objects to transfer in the second; on the other
> side, the author receives the object cloned, with anything listed in the
> array and in the structured data being transferred instead of cloned, and,
> in addition, in the event.ports array, a list of the ports that were given
> in the transfer array.
>
> This has the advantage of being completely backwards-compatible.
>
> So for example, on the sending side:
>
> postMessage(['copied', copiedArrayBuffer, transferredArrayBuffer,
> transferredPort1], // could be an object too
> [transferredArrayBuffer, transferredPort1,
> transferredPort2]);
>
> ...on the receiving side, the event has
>
> data == ['copied', newCopiedArrayBuffer, newTransferredArrayBuffer,
> newTransferredPort1];
> ports == [newTransferredPort1, newTransferredPort2];
>
> It's not going to win any design awards, but I think that boat has sailed
> for this particular API anyway, given the contraints we're operating under.
I still think it's a bigger API than what I'm proposing without adding
much value. For example I suspect people will not realize that they
can transfer ports in the main data graph when they want to transfer
them along with metadata about the port.
I'm also not sure that backwards compatibility is a big problem given
that we're not asking any of the current implementations to modify any
of their existing API (only remove parts of it whenever they feel that
that can be done without breaking content).
But it is an improvement over previous proposals.
/ Jonas
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Tue, 21 Jun 2011, Jonas Sicking wrote:
> On Tue, Jun 21, 2011 at 12:39 AM, Ian Hickson wrote:
> > On Mon, 20 Jun 2011, Jonas Sicking wrote:
> >>
> >> If data appears both in the .ports array and the .data property, then
> >> people will be tempted to create protocols which only work if the
> >> array buffer is transferred, i.e. if the receiver only looks in
> >> .ports. I.e. people will likely end up with equally ugly solutions
> >> like:
> >>
> >> postMessage('vend-reply', [createClone(arraybuffer)]);
> >
> > Yeah, that's a very good point.
> >
> > It really seems like part of the problem here is that ports and
> > arrayBuffers are quite different (in that one will always want to be
> > transferred, but for the other you might want to sometimes transfer
> > and sometimes not).
>
> I don't think we should worry too much about suboptimal syntax when
> transferring ports. We can always add more syntax sugar later if people
> think it's annoying. Better keep the initial version simple than to try
> to guess which use cases will be the most common ones.
Keeping it simple is one of the reasons I don't want to require that
authors list every port twice in order to send a port. (Sending a port is
intended to be a common pattern, at least if the API is used as intended.)
So the current situation is as follows:
- we want to be able to send structured data that is cloned.
- we want the structured data to be able to contain arraybuffers and ports.
- we want it to be possible for the author to opt-in to transferring
specific arraybuffers rather than just cloning them.
- we want the author to have to explicitly list the ports that are to be
transferred because it would be easy to accidentally transfer one
otherwise, and that would cause hard-to-find bugs (e.g. if the data
being cloned happened to contain debugging information that happened to
contain a reference to a port).
- we don't want to require that authors list ports twice in the simple
case, since that's quite ugly
- we don't want authors to design APIs where arraybuffers can only be
transferred and not copied; the receiving side should not be able to
influence the sending side's decision as to whether to clone or
transfer arraybuffers.
How about we just make postMessage() take the object to clone in the first
argument, an array of objects to transfer in the second; on the other
side, the author receives the object cloned, with anything listed in the
array and in the structured data being transferred instead of cloned, and,
in addition, in the event.ports array, a list of the ports that were given
in the transfer array.
This has the advantage of being completely backwards-compatible.
So for example, on the sending side:
postMessage(['copied', copiedArrayBuffer, transferredArrayBuffer,
transferredPort1], // could be an object too
[transferredArrayBuffer, transferredPort1,
transferredPort2]);
...on the receiving side, the event has
data == ['copied', newCopiedArrayBuffer, newTransferredArrayBuffer,
newTransferredPort1];
ports == [newTransferredPort1, newTransferredPort2];
It's not going to win any design awards, but I think that boat has sailed
for this particular API anyway, given the contraints we're operating under.
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Tue, Jun 21, 2011 at 12:39 AM, Ian Hickson wrote:
> On Mon, 20 Jun 2011, Jonas Sicking wrote:
>>
>> If data appears both in the .ports array and the .data property, then
>> people will be tempted to create protocols which only work if the array
>> buffer is transferred, i.e. if the receiver only looks in .ports. I.e.
>> people will likely end up with equally ugly solutions like:
>>
>> postMessage('vend-reply', [createClone(arraybuffer)]);
>
> Yeah, that's a very good point.
>
> It really seems like part of the problem here is that ports and
> arrayBuffers are quite different (in that one will always want to be
> transferred, but for the other you might want to sometimes transfer and
> sometimes not).
I don't think we should worry too much about suboptimal syntax when
transferring ports. We can always add more syntax sugar later if
people think it's annoying. Better keep the initial version simple
than to try to guess which use cases will be the most common ones.
/ Jonas
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Mon, 20 Jun 2011, Jonas Sicking wrote:
>
> If data appears both in the .ports array and the .data property, then
> people will be tempted to create protocols which only work if the array
> buffer is transferred, i.e. if the receiver only looks in .ports. I.e.
> people will likely end up with equally ugly solutions like:
>
> postMessage('vend-reply', [createClone(arraybuffer)]);
Yeah, that's a very good point.
It really seems like part of the problem here is that ports and
arrayBuffers are quite different (in that one will always want to be
transferred, but for the other you might want to sometimes transfer and
sometimes not).
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Mon, Jun 20, 2011 at 3:54 PM, Ian Hickson wrote:
>
> So the proposal that seems to address the most concerns raised in this
> thread would be to have postMessage() work like this:
>
> postMessage({ object }, [ array ]);
>
> ...with it resulting in an event that contains both {object} and [array],
> where everything in the array is transferred, and everything in the object
> is by default cloned unless it's in both the {object} and the [array] in
> which case the transferred copy is used instead.
>
> That way you can keep doing the simple thing to send a port:
>
> postMessage('vend-reply', [port]);
>
> ...and you can pass array buffers in complex data structures as copies:
>
> postMessage({ data: arrayBuffer });
>
> ...and you can send them as clones without losing the structure, though
> you have to walk your structure to list all the objects you care about:
>
> postMessage({ data: arrayBuffer }, [arrayBuffer]);
>
> ...but you don't _have_ to do something like this to send a port:
>
> postMessage(['vend-reply', port], [port]);
>
> ...which seems a bit ugly.
>
> It also means you can both transfer and copy, if that's what you want:
>
> postMessage([thisArrayBufferIsCopied, thisPortIsTransferred],
> [thisPortIsTransferred]);
>
> This also has the benefit of being backwards-compatible, at least if I
> keep an attribute on the event on the other side called "ports" that
> includes the transferred objects (maybe another attribute should be
> included that also returns the same array, since 'ports' would now be a
> confusing misnomer). Alternatively, we can rename 'ports' to something
> else and just have WebKit support 'ports' for legacy reasons. If we do
> this I can also change the 'connect' event to make more sense.
>
> Opinions?
I still think that that transferring both the data field and the
"array formerly known as ports array" is a bad idea. It creates a
strange duality in where data can appear on the receiving side which
complicates the protocol definitions that people have to define.
I agree that
postMessage(['vend-reply', port], [port]);
is a big ugly, but it still seems better to me since it creates a
simpler and more consistent API on the receiver side.
If data appears both in the .ports array and the .data property, then
people will be tempted to create protocols which only work if the
array buffer is transferred, i.e. if the receiver only looks in
.ports. I.e. people will likely end up with equally ugly solutions
like:
postMessage('vend-reply', [createClone(arraybuffer)]);
(which also suffers from being worse performance than simply having
postMessage copy the buffer).
/ Jonas
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
So the proposal that seems to address the most concerns raised in this
thread would be to have postMessage() work like this:
postMessage({ object }, [ array ]);
...with it resulting in an event that contains both {object} and [array],
where everything in the array is transferred, and everything in the object
is by default cloned unless it's in both the {object} and the [array] in
which case the transferred copy is used instead.
That way you can keep doing the simple thing to send a port:
postMessage('vend-reply', [port]);
...and you can pass array buffers in complex data structures as copies:
postMessage({ data: arrayBuffer });
...and you can send them as clones without losing the structure, though
you have to walk your structure to list all the objects you care about:
postMessage({ data: arrayBuffer }, [arrayBuffer]);
...but you don't _have_ to do something like this to send a port:
postMessage(['vend-reply', port], [port]);
...which seems a bit ugly.
It also means you can both transfer and copy, if that's what you want:
postMessage([thisArrayBufferIsCopied, thisPortIsTransferred],
[thisPortIsTransferred]);
This also has the benefit of being backwards-compatible, at least if I
keep an attribute on the event on the other side called "ports" that
includes the transferred objects (maybe another attribute should be
included that also returns the same array, since 'ports' would now be a
confusing misnomer). Alternatively, we can rename 'ports' to something
else and just have WebKit support 'ports' for legacy reasons. If we do
this I can also change the 'connect' event to make more sense.
Opinions?
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
RE: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
Bug: http://www.w3.org/Bugs/Public/show_bug.cgi?id=12947 -Original Message- From: Travis Leithead Sent: Monday, June 13, 2011 10:49 AM To: Arthur Barstow Cc: Andrew Wilson; Glenn Maynard; Jonas Sicking; Dmitry Lomov; David Levin; ben turner; public-webapps@w3.org; Ian Hickson; ext Kenneth Russell Subject: FW: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers] >From: Arthur Barstow [mailto:art.bars...@nokia.com] On Jun/8/2011 5:24 >PM, ext Kenneth Russell wrote: >> My understanding is that we have reached a proposal which respecifies >> the "ports" argument to postMessage as an array of objects to >> transfer, in such a way that we: >> >> - Maintain 100% backward compatibility >> - Enhance the ability to pass MessagePorts, so that the object >> graph can refer to them as well >> - Allow more object types to participate in transfer of ownership >> in the future >> >> To the best of my knowledge there are no active points of >> disagreement. I think we are only waiting for general consensus from >> all interested parties that this is the desired step to take. >> >> If it is, I would be happy to draft proposed edits to the associated >> specs; there are several, and the edits may be somewhat involved. I'd >> also be happy to share the work with Ian or anyone else. > >Concrete proposals should be helpful and it may make sense to first use >Bugzilla to capture the related issues for the various specs. I'll get a bug filed with the summarized proposal as discussed in this thread. Thanks!
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Jun/8/2011 5:24 PM, ext Kenneth Russell wrote: My understanding is that we have reached a proposal which respecifies the "ports" argument to postMessage as an array of objects to transfer, in such a way that we: - Maintain 100% backward compatibility - Enhance the ability to pass MessagePorts, so that the object graph can refer to them as well - Allow more object types to participate in transfer of ownership in the future To the best of my knowledge there are no active points of disagreement. I think we are only waiting for general consensus from all interested parties that this is the desired step to take. If it is, I would be happy to draft proposed edits to the associated specs; there are several, and the edits may be somewhat involved. I'd also be happy to share the work with Ian or anyone else. Concrete proposals should be helpful and it may make sense to first use Bugzilla to capture the related issues for the various specs. (If there are any access issues with Bugzilla, Mike Smith or I can help with that.) -AB
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
(Can you please reset your font?)
On Fri, Jun 10, 2011 at 1:54 AM, Travis Leithead <
travis.leith...@microsoft.com> wrote:
> We don’t really need to support JavaScript objects, arrays, complex
> graphs, etc. at all with the new API
>
Strongly disagree. I should be able to transfer objects naturally with the
existing API, with the same flexibility:
postMessage([
{ frameData: computedArrayBuffer, frameId: 1, complexNestedObject: obj1
},
{ frameData: secondComputedArrayBuffer, frameId: 2, complexNestedObject:
obj2 }
], [computedArrayBuffer, secondComputedArrayBuffer]);
Using a whole separate API that can't handle everything that structured
clone can would be an unnecessarily limiting kludge, forcing the receiver to
piece together transferred objects and their associated cloned objects,
which would be received in separate messages.
This proposal is also completely transparent to the receiving side, so an
API exposed through messages can make use of object transfer without
affecting users of the API.
(and since the current proposal requires the web developer to make an
> explicit list anyway for the 2nd param to post message, it’s no _*more*_
> work to do the same for a new API).
>
It's a lot more work for the developer if he has to use separate APIs for
transferred objects and everything else.
(Also, a minor side proposal is to allow including object constructors, so
you can say things like postMessage(obj, [ArrayBuffer]). That would allow
developers to avoid having to write an explicit list.)
--
Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 9, 2011 at 10:54 PM, Travis Leithead wrote: > Honestly, there’s something about this whole discussion that just doesn’t > feel right. > > > > I looks like we’re trying to graft-in this new concept of transfer of > ownership into the existing postMessage semantics (i.e., object cloning). > Any way I try to make it work, it just looks like peaches grafted into an > apple tree. > > > > What happened to Jonas’ other proposal about a new API? I’d like to direct > some mental energy into that proposal. > > > >>> Complexity comes in many forms and shapes. I much more like the idea > >>> of explicit APIs that make it clear what happens and make it hard to >>> shoot > >>> yourself in the foot. Yes, it can involve more typing, but if it results >>> in more > >>> resilient code which contains fewer subtle bugs, then I think we have >>> designed > >>> the API well. > >>> > >>> / Jonas > > > > Ex: void postMessageAndTransfer([in] any transferOwnershipToDestination…); > > > > We’re only talking about a scenario that makes sense primarily for Web > Workers and applies only to certain types like ArrayBuffer, > CanvasPixelArray+ImageData, Blob, File, etc., that have large underlying > memory buffers. > > > > We don’t really need to support JavaScript objects, arrays, complex graphs, > etc. at all with the new API (and since the current proposal requires the > web developer to make an explicit list anyway for the 2nd param to post > message, it’s no _more_ work to do the same for a new API). > > > > We could even try to graft MessagePorts into this API, but why? MessagePorts > are unique in function compared to the other objects we are discussing for > transfer of ownership (e.g., they facilitate further messaging and can’t be > re-posted once they are cloned once), and they already have well-defined > behavior in MessageEvents and SharedWorkers. > > > > I propose keeping postMessage exactly as it is. Let’s eliminate the > potential compatibility issues. Let’s not re-write the existing specs (that > feels like going backwards, not forwards). For transfer of ownership, let’s > bring this capability on-line through a new API, for the specific scenario > where it makes sense (Web Workers) and not pollute the current postMessage > concepts (object graph cloning and port-passing). Travis, I disagree with your statement that "MessagePorts are unique in function compared to the other objects we are discussing for transfer of ownership". Cloning a MessagePort per http://dev.w3.org/html5/postmsg/#clone-a-port is *exactly* transferring its ownership to the other side. The reason that a MessagePort object can only be cloned once is that its ownership has been transferred. There is no restriction in the current specification preventing the cloned port from being transferred to a new owner via postMessage. The current proposal on the table is 100% backward compatible in signature and semantics, and is an elegant generalization of the slightly over-specialized MessagePort mechanism into the desired transfer of ownership mechanism. In any other API I would personally want exactly postMessage's capability of sending full JavaScript object graphs over the wire, while still being able to transfer ownership of some of the objects contained within, to be able to add some structure to the messages being sent. I would not want to artificially restrict the API to only be able to send certain types of objects. -Ken
RE: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
Honestly, there's something about this whole discussion that just doesn't feel right. I looks like we're trying to graft-in this new concept of transfer of ownership into the existing postMessage semantics (i.e., object cloning). Any way I try to make it work, it just looks like peaches grafted into an apple tree. What happened to Jonas' other proposal about a new API? I'd like to direct some mental energy into that proposal. >> Complexity comes in many forms and shapes. I much more like the idea >> of explicit APIs that make it clear what happens and make it hard to shoot >> yourself in the foot. Yes, it can involve more typing, but if it results in >> more >> resilient code which contains fewer subtle bugs, then I think we have >> designed >> the API well. >> >> / Jonas Ex: void postMessageAndTransfer([in] any transferOwnershipToDestination...); We're only talking about a scenario that makes sense primarily for Web Workers and applies only to certain types like ArrayBuffer, CanvasPixelArray+ImageData, Blob, File, etc., that have large underlying memory buffers. We don't really need to support JavaScript objects, arrays, complex graphs, etc. at all with the new API (and since the current proposal requires the web developer to make an explicit list anyway for the 2nd param to post message, it's no _more_ work to do the same for a new API). We could even try to graft MessagePorts into this API, but why? MessagePorts are unique in function compared to the other objects we are discussing for transfer of ownership (e.g., they facilitate further messaging and can't be re-posted once they are cloned once), and they already have well-defined behavior in MessageEvents and SharedWorkers. I propose keeping postMessage exactly as it is. Let's eliminate the potential compatibility issues. Let's not re-write the existing specs (that feels like going backwards, not forwards). For transfer of ownership, let's bring this capability on-line through a new API, for the specific scenario where it makes sense (Web Workers) and not pollute the current postMessage concepts (object graph cloning and port-passing).
RE: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
From: Andrew Wilson [mailto:atwil...@google.com]
Sent: Friday, June 03, 2011 2:15 PM
On Fri, Jun 3, 2011 at 1:02 PM, Jonas Sicking
mailto:jo...@sicking.cc>> wrote:
On Fri, Jun 3, 2011 at 11:37 AM, Kenneth Russell
mailto:k...@google.com>> wrote:
> On Fri, Jun 3, 2011 at 9:46 AM, Glenn Maynard
> mailto:gl...@zewt.org>> wrote:
>> On Fri, Jun 3, 2011 at 11:12 AM, Dmitry Lomov
>> mailto:dslo...@google.com>> wrote:
>>> a) Recursive transfer lists. Allow arbitrary objects, not only ArrayBuffers,
>>> to appear in transfer lists. ArrayBuffers that are under objects in
>>> transfer lists are transferred, others are cloned.
>>
>> This again causes the same forwards-compatibility problem. If you do this:
>>
>> frame = { video: canvasPixelArrayInstance, histogram: arrayBuffer } }
>> postMessage({ frame: frame }, { transfer: frame });
>>
>> and you expect only histogram to be transferred (since that's all that
>> supports it when you write this code), your code breaks when
>> CanvasPixelArray later supports it.
>>
>>> b) Transfer lists + separate transferMessage method. We still equip
>>> postMessage with transfer lists, these transfer lists list ArrayBuffers, and
>>> we provide a separate method transferMessage with recursive transfer
>>> semantics.
>>> What do people think?
>>
>> Same problem.
>>
>> If you want a quicker way to transfer all messages of given types, see
>> my previous mail: { transfer: ArrayBuffer }.
>
> Agreed on these points. Using an object graph for the transfer list
> (which is what the recursive transfer list idea boils down to) also
> sounds overly complicated.
>
> May I suggest to reconsider adding another optional array argument to
> postMessage for the transfer list, rather than using an object with
> special properties?
>
> Points in favor of adding another optional array argument:
>
> 1. Less typing, and less possibility that a typo will cause incorrect
> behavior:
> worker.postMessage(objectGraph, null, [ arrayBuffer1ToTransfer,
> arrayBuffer2ToTransfer ]);
>vs.
> worker.postMessage(objectGraph, { transfer: [
> arrayBuffer1ToTransfer, arrayBuffer2ToTransfer] });
>
> 2. Possibility of using Web IDL to specify the type of the optional
> array argument (i.e., "Transferable[]?"). Would be harder to do using
> an object -- requiring either specifying another interface type with
> the "ports" and "transfer" attributes, or using "any" plus
> ECMAScript-specific text.
>
> Points in favor of using an object:
>
> 1. Could potentially overload the meaning of the optional ports array
> argument, avoiding adding another argument to postMessage.
>
> 2. More extensible in the future.
>
> Thoughts?
My first thought is that so far no implementer has stepped up and said
that changing the meaning of the 'ports' argument would not be
acceptable. Would be great if someone who is reading this thread and
who works at Google/Apple/Opera could check with the relevant people
to see if such a change would be possible.
It's certainly possible - there's nothing intrinsically difficult with making
this change from an implementor's point of view (I've had my fingers in both
the WebKit and Chromium MessagePort implementations so I'm relatively confident
that this would not be prohibitively hard).
Is it desirable? My knee-jerk reaction is that we should stick with changes
that are compatible with the existing API (like Ian's original suggestion, or
adding a separate optional array of transferable objects) - I have no data on
the number of sites that are using the current API but I don't think we should
be changing existing APIs with multiple implementations without significant
motivation. The stated motivations for breaking this API don't seem compelling
to me given the existence of backwards-compatible alternatives.
+1 Having worked on compatibility for a while now, it's going to break someone
if you change the existing signature in a non-backwards compatible way.
I for one, would prefer the option that adds a 3rd parameter with the list of
transferrable object references in the main graph of parameter 1. This seems
like the least ugly way of adding this capability.
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 9, 2011 at 11:13 AM, Glenn Maynard wrote: > On Thu, Jun 9, 2011 at 1:28 PM, Andrew Wilson wrote: > > 1) I'm not completely sure I understand what the new postMessage() > semantics > > look like. Since cloning a port is a destructive operation, I like the > fact > > that the current postMessage() API requires the developer to explicitly > pass > > a list of ports to clone. If we shift to a paradigm where merely > referencing > > a port from the object graph is sufficient to do a destructive clone of > that > > port, I'm concerned that this will lead to very-difficult-to-debug issues > > for developers. So I would say that there is value to still requiring the > > developer to pass a separate "objectsToTransfer" array, even if we > > automagically fixup references to those transferred objects within the > > object graph. > > I see the list as a set of objects whose mutating structured clone behavior > should be enabled. > > For ArrayBuffer (and most any other object that might be put in here except > ports), it's object transfer. For MessagePort, it's MessagePort's cloning > behavior. MessagePort would have no non-mutating structured clone behavior, > so putting a MessagePort in the object graph without putting it in the > transfer list would be an error, like putting an unclonable object in the > graph; it would throw DATA_CLONE_ERR. > > Likewise, putting a MessagePort in the object graph with APIs like History > or Web Storage, which use structured clone but not transfer, would throw > DATA_CLONE_ERR, since the (implicit) transfer set is empty. (In other > words, other APIs would be unaffected and throw the same error they do now.) Great, this makes sense to me too.
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 9, 2011 at 1:28 PM, Andrew Wilson wrote: > 1) I'm not completely sure I understand what the new postMessage() semantics > look like. Since cloning a port is a destructive operation, I like the fact > that the current postMessage() API requires the developer to explicitly pass > a list of ports to clone. If we shift to a paradigm where merely referencing > a port from the object graph is sufficient to do a destructive clone of that > port, I'm concerned that this will lead to very-difficult-to-debug issues > for developers. So I would say that there is value to still requiring the > developer to pass a separate "objectsToTransfer" array, even if we > automagically fixup references to those transferred objects within the > object graph. I see the list as a set of objects whose mutating structured clone behavior should be enabled. For ArrayBuffer (and most any other object that might be put in here except ports), it's object transfer. For MessagePort, it's MessagePort's cloning behavior. MessagePort would have no non-mutating structured clone behavior, so putting a MessagePort in the object graph without putting it in the transfer list would be an error, like putting an unclonable object in the graph; it would throw DATA_CLONE_ERR. Likewise, putting a MessagePort in the object graph with APIs like History or Web Storage, which use structured clone but not transfer, would throw DATA_CLONE_ERR, since the (implicit) transfer set is empty. (In other words, other APIs would be unaffected and throw the same error they do now.) -- Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 8, 2011 at 6:26 PM, Kenneth Russell wrote: > > Thinking about this more, that port could be sent as the data > attribute of the event instead of the empty string. Then the ports > attribute on MessageEvent could be safely deprecated. > > -Ken > > So a number of different variations have floated around, and I have some concerns: 1) I'm not completely sure I understand what the new postMessage() semantics look like. Since cloning a port is a destructive operation, I like the fact that the current postMessage() API requires the developer to explicitly pass a list of ports to clone. If we shift to a paradigm where merely referencing a port from the object graph is sufficient to do a destructive clone of that port, I'm concerned that this will lead to very-difficult-to-debug issues for developers. So I would say that there is value to still requiring the developer to pass a separate "objectsToTransfer" array, even if we automagically fixup references to those transferred objects within the object graph. 2) Rather than having some mix of deprecated attributes on MessageEvent to support SharedWorkers, I'd rather just change onconnect() to take a distinct event type that has a non-deprecated ports attribute - I've never been particularly clear why we're reusing the MessageEvent interface for SharedWorker connect events anyway. Since Safari and Chrome have supported SharedWorkers for well over a year now, I would be careful about changing this interface in a non-backwards-compatible manner - however, if we are not concerned with backwards compatibility, I would change this ConnectEvent to just have a port attribute that references a single port rather than a ports[] attribute that references an array that always contains a single port. -atw
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 8, 2011 at 6:26 PM, Kenneth Russell wrote: > On Wed, Jun 8, 2011 at 6:14 PM, Jonas Sicking wrote: >> On Wed, Jun 8, 2011 at 4:27 PM, Kenneth Russell wrote: >>> On Wed, Jun 8, 2011 at 2:39 PM, David Levin wrote: On Wed, Jun 8, 2011 at 2:33 PM, Kenneth Russell wrote: > > I prefer continuing to use an array for several reasons: simpler > syntax, better type checking at the Web IDL level, and fewer > ECMAScript-specific semantics. An array makes it harder to do future modifications. >>> >>> Possibly, but it makes the design of this modification cleaner. >>> Also with the array, how does "Enhance the ability to pass MessagePorts, so that the object graph can refer to them as well" work? Specifically, consider an array that contains [arrayBuffer1, port1]. Is port1 something in the object graph or a port to be transfer as before? >>> >>> In order to maintain backward compatibility, the clone of port1 would >>> show up in the "ports" attribute of the MessageEvent on the other >>> side. Additionally, during the structured clone of the object graph, >>> any references to port1 would be updated to point to the clone of >>> port1. (The latter is new behavior, and brings MessagePorts in line >>> with the desired transfer-of-ownership semantics.) >>> >>> All other objects in the array (which, as Ian originally proposed, >>> would implement some interface like "Transferable" for better Web IDL >>> type checking) would simply indicate objects in the graph to be >>> transferred rather than copied. >> >> This all sounds great to me, but I think we should additionally make >> the 'ports' attribute on the MessageEvent interface deprecated. >> >> The only use case for it is to support existing code which doesn't >> pass ports in the object graph but rather only in the array in the >> second argument (i.e. the formerly "ports" argument). > > That's not quite correct; as was pointed out earlier in the thread, > SharedWorkerGlobalScope's onconnect message relies on receiving the > message port with which to communicate to the outside world in the > zeroth element of the ports array. See step 7.7.5 in > http://www.whatwg.org/specs/web-workers/current-work/#shared-workers-and-the-sharedworker-interface > . > > Thinking about this more, that port could be sent as the data > attribute of the event instead of the empty string. Then the ports > attribute on MessageEvent could be safely deprecated. Sounds great to me. / Jonas
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 8, 2011 at 9:14 PM, Jonas Sicking wrote: > This all sounds great to me, but I think we should additionally make > the 'ports' attribute on the MessageEvent interface deprecated. > > The only use case for it is to support existing code which doesn't > pass ports in the object graph but rather only in the array in the > second argument (i.e. the formerly "ports" argument). > > By deprecating it, I mean either: > > 1. Mark it, using prose, as deprecated in the specification. > 2. Remove it from the specification but allow existing implementations > of it to keep it as long as they feel needed to retain compatibility > with existing code. MessageEvent is also used by the onconnect event. -- Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 8, 2011 at 6:14 PM, Jonas Sicking wrote: > On Wed, Jun 8, 2011 at 4:27 PM, Kenneth Russell wrote: >> On Wed, Jun 8, 2011 at 2:39 PM, David Levin wrote: >>> >>> >>> On Wed, Jun 8, 2011 at 2:33 PM, Kenneth Russell wrote: I prefer continuing to use an array for several reasons: simpler syntax, better type checking at the Web IDL level, and fewer ECMAScript-specific semantics. >>> >>> An array makes it harder to do future modifications. >> >> Possibly, but it makes the design of this modification cleaner. >> >>> Also with the array, how does "Enhance the ability to pass MessagePorts, so >>> that the object graph can refer to them as well" work? Specifically, >>> consider an array that contains [arrayBuffer1, port1]. Is port1 something in >>> the object graph or a port to be transfer as before? >> >> In order to maintain backward compatibility, the clone of port1 would >> show up in the "ports" attribute of the MessageEvent on the other >> side. Additionally, during the structured clone of the object graph, >> any references to port1 would be updated to point to the clone of >> port1. (The latter is new behavior, and brings MessagePorts in line >> with the desired transfer-of-ownership semantics.) >> >> All other objects in the array (which, as Ian originally proposed, >> would implement some interface like "Transferable" for better Web IDL >> type checking) would simply indicate objects in the graph to be >> transferred rather than copied. > > This all sounds great to me, but I think we should additionally make > the 'ports' attribute on the MessageEvent interface deprecated. > > The only use case for it is to support existing code which doesn't > pass ports in the object graph but rather only in the array in the > second argument (i.e. the formerly "ports" argument). That's not quite correct; as was pointed out earlier in the thread, SharedWorkerGlobalScope's onconnect message relies on receiving the message port with which to communicate to the outside world in the zeroth element of the ports array. See step 7.7.5 in http://www.whatwg.org/specs/web-workers/current-work/#shared-workers-and-the-sharedworker-interface . Thinking about this more, that port could be sent as the data attribute of the event instead of the empty string. Then the ports attribute on MessageEvent could be safely deprecated. -Ken > By deprecating it, I mean either: > > 1. Mark it, using prose, as deprecated in the specification. > 2. Remove it from the specification but allow existing implementations > of it to keep it as long as they feel needed to retain compatibility > with existing code. > > / Jonas >
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 8, 2011 at 4:27 PM, Kenneth Russell wrote: > On Wed, Jun 8, 2011 at 2:39 PM, David Levin wrote: >> >> >> On Wed, Jun 8, 2011 at 2:33 PM, Kenneth Russell wrote: >>> >>> I prefer continuing to use an array for several reasons: simpler >>> syntax, better type checking at the Web IDL level, and fewer >>> ECMAScript-specific semantics. >> >> An array makes it harder to do future modifications. > > Possibly, but it makes the design of this modification cleaner. > >> Also with the array, how does "Enhance the ability to pass MessagePorts, so >> that the object graph can refer to them as well" work? Specifically, >> consider an array that contains [arrayBuffer1, port1]. Is port1 something in >> the object graph or a port to be transfer as before? > > In order to maintain backward compatibility, the clone of port1 would > show up in the "ports" attribute of the MessageEvent on the other > side. Additionally, during the structured clone of the object graph, > any references to port1 would be updated to point to the clone of > port1. (The latter is new behavior, and brings MessagePorts in line > with the desired transfer-of-ownership semantics.) > > All other objects in the array (which, as Ian originally proposed, > would implement some interface like "Transferable" for better Web IDL > type checking) would simply indicate objects in the graph to be > transferred rather than copied. This all sounds great to me, but I think we should additionally make the 'ports' attribute on the MessageEvent interface deprecated. The only use case for it is to support existing code which doesn't pass ports in the object graph but rather only in the array in the second argument (i.e. the formerly "ports" argument). By deprecating it, I mean either: 1. Mark it, using prose, as deprecated in the specification. 2. Remove it from the specification but allow existing implementations of it to keep it as long as they feel needed to retain compatibility with existing code. / Jonas
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 8, 2011 at 5:33 PM, Kenneth Russell wrote: > I prefer continuing to use an array for several reasons: simpler > syntax, better type checking at the Web IDL level, and fewer > ECMAScript-specific semantics. > > Possibly, but it makes the design of this modification cleaner. These don't sound too critical to me, and WebIDL supports using objects like this now. I suspect we'll start to see more of the optional-parameters-in-an-object pattern, as APIs grow more complex and start to need it more often. I don't think it's inherently cleaner or simpler either way--parameters in an object is very natural in JS, and much clearer once you reach four or five parameters. But if the simple array approach works, then there's just no need. -- Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
ok.
On Wed, Jun 8, 2011 at 4:27 PM, Kenneth Russell wrote:
> On Wed, Jun 8, 2011 at 2:39 PM, David Levin wrote:
> >
> >
> > On Wed, Jun 8, 2011 at 2:33 PM, Kenneth Russell wrote:
> >>
> >> I prefer continuing to use an array for several reasons: simpler
> >> syntax, better type checking at the Web IDL level, and fewer
> >> ECMAScript-specific semantics.
> >
> > An array makes it harder to do future modifications.
>
> Possibly, but it makes the design of this modification cleaner.
>
> > Also with the array, how does "Enhance the ability to pass MessagePorts,
> so
> > that the object graph can refer to them as well" work? Specifically,
> > consider an array that contains [arrayBuffer1, port1]. Is port1 something
> in
> > the object graph or a port to be transfer as before?
>
> In order to maintain backward compatibility, the clone of port1 would
> show up in the "ports" attribute of the MessageEvent on the other
> side. Additionally, during the structured clone of the object graph,
> any references to port1 would be updated to point to the clone of
> port1. (The latter is new behavior, and brings MessagePorts in line
> with the desired transfer-of-ownership semantics.)
>
> All other objects in the array (which, as Ian originally proposed,
> would implement some interface like "Transferable" for better Web IDL
> type checking) would simply indicate objects in the graph to be
> transferred rather than copied.
>
> Note: it would still be possible to evolve the API to transfer all
> objects of a certain type. We would just need to change the type of
> the "ports" or "transfer" array from Transferable[] to any[] and spec
> what happens when a constructor function is placed in the array.
>
> -Ken
>
> > dave
> >
> >>
> >> -Ken
> >>
> >> On Wed, Jun 8, 2011 at 2:29 PM, David Levin wrote:
> >> >
> >> >
> >> > On Wed, Jun 8, 2011 at 2:24 PM, Kenneth Russell
> wrote:
> >> >>
> >> >> My understanding is that we have reached a proposal which respecifies
> >> >> the "ports" argument to postMessage as an array of objects to
> >> >> transfer, in such a way that we:
> >> >
> >> > Array or object? (by object I mean: {transfer: [arrayBuffer1], ports:
> >> > [port]})
> >> >
> >> >>
> >> >> - Maintain 100% backward compatibility
> >> >> - Enhance the ability to pass MessagePorts, so that the object graph
> >> >> can refer to them as well
> >> >> - Allow more object types to participate in transfer of ownership in
> >> >> the
> >> >> future
> >> >>
> >> >> To the best of my knowledge there are no active points of
> >> >> disagreement. I think we are only waiting for general consensus from
> >> >> all interested parties that this is the desired step to take.
> >> >>
> >> >> If it is, I would be happy to draft proposed edits to the associated
> >> >> specs; there are several, and the edits may be somewhat involved. I'd
> >> >> also be happy to share the work with Ian or anyone else.
> >> >>
> >> >> I don't know the various processes for web specs, but the Web
> >> >> Messaging spec will definitely need to be updated if we decide to
> move
> >> >> in this direction.
> >> >>
> >> >> -Ken
> >> >>
> >> >> On Wed, Jun 8, 2011 at 4:30 AM, Arthur Barstow <
> art.bars...@nokia.com>
> >> >> wrote:
> >> >> > Now that the responses on this thread have slowed, I would
> appreciate
> >> >> > if
> >> >> > the
> >> >> > participants would please summarize where they think we are on this
> >> >> > issue,
> >> >> > e.g. the points of agreement and disagreement, how to move forward,
> >> >> > etc.
> >> >> >
> >> >> > Also, coming back to the question in the subject (and I apologize
> if
> >> >> > my
> >> >> > premature subject change caused any confusion or problems), since
> we
> >> >> > have an
> >> >> > open CfC (ends June 9 [1]) to publish a Candidate Recommendation of
> >> >> > Web
> >> >> > Messaging, is the Messaging spec going to need to change to address
> >> >> > the
> >> >> > issues raised in this thread?
> >> >> >
> >> >> > -Art Barstow
> >> >> >
> >> >> > [1]
> >> >> >
> >> >> >
> http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/0797.html
> >> >> >
> >> >
> >
> >
>
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 8, 2011 at 2:39 PM, David Levin wrote:
>
>
> On Wed, Jun 8, 2011 at 2:33 PM, Kenneth Russell wrote:
>>
>> I prefer continuing to use an array for several reasons: simpler
>> syntax, better type checking at the Web IDL level, and fewer
>> ECMAScript-specific semantics.
>
> An array makes it harder to do future modifications.
Possibly, but it makes the design of this modification cleaner.
> Also with the array, how does "Enhance the ability to pass MessagePorts, so
> that the object graph can refer to them as well" work? Specifically,
> consider an array that contains [arrayBuffer1, port1]. Is port1 something in
> the object graph or a port to be transfer as before?
In order to maintain backward compatibility, the clone of port1 would
show up in the "ports" attribute of the MessageEvent on the other
side. Additionally, during the structured clone of the object graph,
any references to port1 would be updated to point to the clone of
port1. (The latter is new behavior, and brings MessagePorts in line
with the desired transfer-of-ownership semantics.)
All other objects in the array (which, as Ian originally proposed,
would implement some interface like "Transferable" for better Web IDL
type checking) would simply indicate objects in the graph to be
transferred rather than copied.
Note: it would still be possible to evolve the API to transfer all
objects of a certain type. We would just need to change the type of
the "ports" or "transfer" array from Transferable[] to any[] and spec
what happens when a constructor function is placed in the array.
-Ken
> dave
>
>>
>> -Ken
>>
>> On Wed, Jun 8, 2011 at 2:29 PM, David Levin wrote:
>> >
>> >
>> > On Wed, Jun 8, 2011 at 2:24 PM, Kenneth Russell wrote:
>> >>
>> >> My understanding is that we have reached a proposal which respecifies
>> >> the "ports" argument to postMessage as an array of objects to
>> >> transfer, in such a way that we:
>> >
>> > Array or object? (by object I mean: {transfer: [arrayBuffer1], ports:
>> > [port]})
>> >
>> >>
>> >> - Maintain 100% backward compatibility
>> >> - Enhance the ability to pass MessagePorts, so that the object graph
>> >> can refer to them as well
>> >> - Allow more object types to participate in transfer of ownership in
>> >> the
>> >> future
>> >>
>> >> To the best of my knowledge there are no active points of
>> >> disagreement. I think we are only waiting for general consensus from
>> >> all interested parties that this is the desired step to take.
>> >>
>> >> If it is, I would be happy to draft proposed edits to the associated
>> >> specs; there are several, and the edits may be somewhat involved. I'd
>> >> also be happy to share the work with Ian or anyone else.
>> >>
>> >> I don't know the various processes for web specs, but the Web
>> >> Messaging spec will definitely need to be updated if we decide to move
>> >> in this direction.
>> >>
>> >> -Ken
>> >>
>> >> On Wed, Jun 8, 2011 at 4:30 AM, Arthur Barstow
>> >> wrote:
>> >> > Now that the responses on this thread have slowed, I would appreciate
>> >> > if
>> >> > the
>> >> > participants would please summarize where they think we are on this
>> >> > issue,
>> >> > e.g. the points of agreement and disagreement, how to move forward,
>> >> > etc.
>> >> >
>> >> > Also, coming back to the question in the subject (and I apologize if
>> >> > my
>> >> > premature subject change caused any confusion or problems), since we
>> >> > have an
>> >> > open CfC (ends June 9 [1]) to publish a Candidate Recommendation of
>> >> > Web
>> >> > Messaging, is the Messaging spec going to need to change to address
>> >> > the
>> >> > issues raised in this thread?
>> >> >
>> >> > -Art Barstow
>> >> >
>> >> > [1]
>> >> >
>> >> > http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/0797.html
>> >> >
>> >
>
>
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 8, 2011 at 2:33 PM, Kenneth Russell wrote:
> I prefer continuing to use an array for several reasons: simpler
> syntax, better type checking at the Web IDL level, and fewer
> ECMAScript-specific semantics.
>
An array makes it harder to do future modifications.
Also with the array, how does "Enhance the ability to pass MessagePorts, so
that the object graph can refer to them as well" work? Specifically,
consider an array that contains [arrayBuffer1, port1]. Is port1 something in
the object graph or a port to be transfer as before?
dave
>
> -Ken
>
> On Wed, Jun 8, 2011 at 2:29 PM, David Levin wrote:
> >
> >
> > On Wed, Jun 8, 2011 at 2:24 PM, Kenneth Russell wrote:
> >>
> >> My understanding is that we have reached a proposal which respecifies
> >> the "ports" argument to postMessage as an array of objects to
> >> transfer, in such a way that we:
> >
> > Array or object? (by object I mean: {transfer: [arrayBuffer1], ports:
> > [port]})
> >
> >>
> >> - Maintain 100% backward compatibility
> >> - Enhance the ability to pass MessagePorts, so that the object graph
> >> can refer to them as well
> >> - Allow more object types to participate in transfer of ownership in
> the
> >> future
> >>
> >> To the best of my knowledge there are no active points of
> >> disagreement. I think we are only waiting for general consensus from
> >> all interested parties that this is the desired step to take.
> >>
> >> If it is, I would be happy to draft proposed edits to the associated
> >> specs; there are several, and the edits may be somewhat involved. I'd
> >> also be happy to share the work with Ian or anyone else.
> >>
> >> I don't know the various processes for web specs, but the Web
> >> Messaging spec will definitely need to be updated if we decide to move
> >> in this direction.
> >>
> >> -Ken
> >>
> >> On Wed, Jun 8, 2011 at 4:30 AM, Arthur Barstow
> >> wrote:
> >> > Now that the responses on this thread have slowed, I would appreciate
> if
> >> > the
> >> > participants would please summarize where they think we are on this
> >> > issue,
> >> > e.g. the points of agreement and disagreement, how to move forward,
> etc.
> >> >
> >> > Also, coming back to the question in the subject (and I apologize if
> my
> >> > premature subject change caused any confusion or problems), since we
> >> > have an
> >> > open CfC (ends June 9 [1]) to publish a Candidate Recommendation of
> Web
> >> > Messaging, is the Messaging spec going to need to change to address
> the
> >> > issues raised in this thread?
> >> >
> >> > -Art Barstow
> >> >
> >> > [1]
> >> >
> http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/0797.html
> >> >
> >
>
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
I agree with Kenneth.
-Ben Turner
On Wed, Jun 8, 2011 at 2:33 PM, Kenneth Russell wrote:
> I prefer continuing to use an array for several reasons: simpler
> syntax, better type checking at the Web IDL level, and fewer
> ECMAScript-specific semantics.
>
> -Ken
>
> On Wed, Jun 8, 2011 at 2:29 PM, David Levin wrote:
>>
>>
>> On Wed, Jun 8, 2011 at 2:24 PM, Kenneth Russell wrote:
>>>
>>> My understanding is that we have reached a proposal which respecifies
>>> the "ports" argument to postMessage as an array of objects to
>>> transfer, in such a way that we:
>>
>> Array or object? (by object I mean: {transfer: [arrayBuffer1], ports:
>> [port]})
>>
>>>
>>> - Maintain 100% backward compatibility
>>> - Enhance the ability to pass MessagePorts, so that the object graph
>>> can refer to them as well
>>> - Allow more object types to participate in transfer of ownership in the
>>> future
>>>
>>> To the best of my knowledge there are no active points of
>>> disagreement. I think we are only waiting for general consensus from
>>> all interested parties that this is the desired step to take.
>>>
>>> If it is, I would be happy to draft proposed edits to the associated
>>> specs; there are several, and the edits may be somewhat involved. I'd
>>> also be happy to share the work with Ian or anyone else.
>>>
>>> I don't know the various processes for web specs, but the Web
>>> Messaging spec will definitely need to be updated if we decide to move
>>> in this direction.
>>>
>>> -Ken
>>>
>>> On Wed, Jun 8, 2011 at 4:30 AM, Arthur Barstow
>>> wrote:
>>> > Now that the responses on this thread have slowed, I would appreciate if
>>> > the
>>> > participants would please summarize where they think we are on this
>>> > issue,
>>> > e.g. the points of agreement and disagreement, how to move forward, etc.
>>> >
>>> > Also, coming back to the question in the subject (and I apologize if my
>>> > premature subject change caused any confusion or problems), since we
>>> > have an
>>> > open CfC (ends June 9 [1]) to publish a Candidate Recommendation of Web
>>> > Messaging, is the Messaging spec going to need to change to address the
>>> > issues raised in this thread?
>>> >
>>> > -Art Barstow
>>> >
>>> > [1]
>>> > http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/0797.html
>>> >
>>
>
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
I prefer continuing to use an array for several reasons: simpler
syntax, better type checking at the Web IDL level, and fewer
ECMAScript-specific semantics.
-Ken
On Wed, Jun 8, 2011 at 2:29 PM, David Levin wrote:
>
>
> On Wed, Jun 8, 2011 at 2:24 PM, Kenneth Russell wrote:
>>
>> My understanding is that we have reached a proposal which respecifies
>> the "ports" argument to postMessage as an array of objects to
>> transfer, in such a way that we:
>
> Array or object? (by object I mean: {transfer: [arrayBuffer1], ports:
> [port]})
>
>>
>> - Maintain 100% backward compatibility
>> - Enhance the ability to pass MessagePorts, so that the object graph
>> can refer to them as well
>> - Allow more object types to participate in transfer of ownership in the
>> future
>>
>> To the best of my knowledge there are no active points of
>> disagreement. I think we are only waiting for general consensus from
>> all interested parties that this is the desired step to take.
>>
>> If it is, I would be happy to draft proposed edits to the associated
>> specs; there are several, and the edits may be somewhat involved. I'd
>> also be happy to share the work with Ian or anyone else.
>>
>> I don't know the various processes for web specs, but the Web
>> Messaging spec will definitely need to be updated if we decide to move
>> in this direction.
>>
>> -Ken
>>
>> On Wed, Jun 8, 2011 at 4:30 AM, Arthur Barstow
>> wrote:
>> > Now that the responses on this thread have slowed, I would appreciate if
>> > the
>> > participants would please summarize where they think we are on this
>> > issue,
>> > e.g. the points of agreement and disagreement, how to move forward, etc.
>> >
>> > Also, coming back to the question in the subject (and I apologize if my
>> > premature subject change caused any confusion or problems), since we
>> > have an
>> > open CfC (ends June 9 [1]) to publish a Candidate Recommendation of Web
>> > Messaging, is the Messaging spec going to need to change to address the
>> > issues raised in this thread?
>> >
>> > -Art Barstow
>> >
>> > [1]
>> > http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/0797.html
>> >
>
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Wed, Jun 8, 2011 at 2:24 PM, Kenneth Russell wrote:
> My understanding is that we have reached a proposal which respecifies
> the "ports" argument to postMessage as an array of objects to
> transfer, in such a way that we:
>
Array or object? (by object I mean: {transfer: [arrayBuffer1], ports:
[port]})
>
> - Maintain 100% backward compatibility
> - Enhance the ability to pass MessagePorts, so that the object graph
> can refer to them as well
> - Allow more object types to participate in transfer of ownership in the
> future
>
> To the best of my knowledge there are no active points of
> disagreement. I think we are only waiting for general consensus from
> all interested parties that this is the desired step to take.
>
> If it is, I would be happy to draft proposed edits to the associated
> specs; there are several, and the edits may be somewhat involved. I'd
> also be happy to share the work with Ian or anyone else.
>
> I don't know the various processes for web specs, but the Web
> Messaging spec will definitely need to be updated if we decide to move
> in this direction.
>
> -Ken
>
> On Wed, Jun 8, 2011 at 4:30 AM, Arthur Barstow
> wrote:
> > Now that the responses on this thread have slowed, I would appreciate if
> the
> > participants would please summarize where they think we are on this
> issue,
> > e.g. the points of agreement and disagreement, how to move forward, etc.
> >
> > Also, coming back to the question in the subject (and I apologize if my
> > premature subject change caused any confusion or problems), since we have
> an
> > open CfC (ends June 9 [1]) to publish a Candidate Recommendation of Web
> > Messaging, is the Messaging spec going to need to change to address the
> > issues raised in this thread?
> >
> > -Art Barstow
> >
> > [1]
> http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/0797.html
> >
>
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
My understanding is that we have reached a proposal which respecifies
the "ports" argument to postMessage as an array of objects to
transfer, in such a way that we:
- Maintain 100% backward compatibility
- Enhance the ability to pass MessagePorts, so that the object graph
can refer to them as well
- Allow more object types to participate in transfer of ownership in the future
To the best of my knowledge there are no active points of
disagreement. I think we are only waiting for general consensus from
all interested parties that this is the desired step to take.
If it is, I would be happy to draft proposed edits to the associated
specs; there are several, and the edits may be somewhat involved. I'd
also be happy to share the work with Ian or anyone else.
I don't know the various processes for web specs, but the Web
Messaging spec will definitely need to be updated if we decide to move
in this direction.
-Ken
On Wed, Jun 8, 2011 at 4:30 AM, Arthur Barstow wrote:
> Now that the responses on this thread have slowed, I would appreciate if the
> participants would please summarize where they think we are on this issue,
> e.g. the points of agreement and disagreement, how to move forward, etc.
>
> Also, coming back to the question in the subject (and I apologize if my
> premature subject change caused any confusion or problems), since we have an
> open CfC (ends June 9 [1]) to publish a Candidate Recommendation of Web
> Messaging, is the Messaging spec going to need to change to address the
> issues raised in this thread?
>
> -Art Barstow
>
> [1] http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/0797.html
>
> On Jun/3/2011 8:47 PM, ext Kenneth Russell wrote:
>>
>> On Fri, Jun 3, 2011 at 4:15 PM, Andrew Wilson wrote:
>>>
>>> On Fri, Jun 3, 2011 at 3:23 PM, Glenn Maynard wrote:
On Fri, Jun 3, 2011 at 5:15 PM, Andrew Wilson
wrote:
>
> significant motivation. The stated motivations for breaking this API
> don't
> seem compelling to me given the existence of backwards-compatible
> alternatives.
This proposal is backwards-compatible. If the argument is an array,
nothing changes, so postMessage(..., [ports]) is equivalent to
postMessage(..., {ports: [ports]}). (The array-only approach can be
done compatibly, too; the object version is just an alternative to
that.) What's backwards-incompatible?
>>>
>>> Ah, I missed that piece (to be honest, I haven't been following this
>>> discussion in every detail - I only chimed in because of Jonas' request
>>> for
>>> implementation feedback).
>>>
For anyone not looking closely at the IDL while reading this, this
means deprecating (for whatever value "deprecate" has on the web) the
ports array in MessageEvent--not the ports parameter to postMessage
(that's a sequence).
>>>
>>> Does this affect the API for the SharedWorker onconnect message as well?
>>
>> Good point; that might inform not deprecating the ports array in
>> MessageEvent, but leaving it as is.
>>
>> -Ken
>>
>
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
Now that the responses on this thread have slowed, I would appreciate if
the participants would please summarize where they think we are on this
issue, e.g. the points of agreement and disagreement, how to move
forward, etc.
Also, coming back to the question in the subject (and I apologize if my
premature subject change caused any confusion or problems), since we
have an open CfC (ends June 9 [1]) to publish a Candidate Recommendation
of Web Messaging, is the Messaging spec going to need to change to
address the issues raised in this thread?
-Art Barstow
[1] http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/0797.html
On Jun/3/2011 8:47 PM, ext Kenneth Russell wrote:
On Fri, Jun 3, 2011 at 4:15 PM, Andrew Wilson wrote:
On Fri, Jun 3, 2011 at 3:23 PM, Glenn Maynard wrote:
On Fri, Jun 3, 2011 at 5:15 PM, Andrew Wilson wrote:
significant motivation. The stated motivations for breaking this API
don't
seem compelling to me given the existence of backwards-compatible
alternatives.
This proposal is backwards-compatible. If the argument is an array,
nothing changes, so postMessage(..., [ports]) is equivalent to
postMessage(..., {ports: [ports]}). (The array-only approach can be
done compatibly, too; the object version is just an alternative to
that.) What's backwards-incompatible?
Ah, I missed that piece (to be honest, I haven't been following this
discussion in every detail - I only chimed in because of Jonas' request for
implementation feedback).
For anyone not looking closely at the IDL while reading this, this
means deprecating (for whatever value "deprecate" has on the web) the
ports array in MessageEvent--not the ports parameter to postMessage
(that's a sequence).
Does this affect the API for the SharedWorker onconnect message as well?
Good point; that might inform not deprecating the ports array in
MessageEvent, but leaving it as is.
-Ken
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, Jun 3, 2011 at 4:15 PM, Andrew Wilson wrote:
>
>
> On Fri, Jun 3, 2011 at 3:23 PM, Glenn Maynard wrote:
>>
>> On Fri, Jun 3, 2011 at 5:15 PM, Andrew Wilson wrote:
>> > significant motivation. The stated motivations for breaking this API
>> > don't
>> > seem compelling to me given the existence of backwards-compatible
>> > alternatives.
>>
>> This proposal is backwards-compatible. If the argument is an array,
>> nothing changes, so postMessage(..., [ports]) is equivalent to
>> postMessage(..., {ports: [ports]}). (The array-only approach can be
>> done compatibly, too; the object version is just an alternative to
>> that.) What's backwards-incompatible?
>
> Ah, I missed that piece (to be honest, I haven't been following this
> discussion in every detail - I only chimed in because of Jonas' request for
> implementation feedback).
>
>>
>> For anyone not looking closely at the IDL while reading this, this
>> means deprecating (for whatever value "deprecate" has on the web) the
>> ports array in MessageEvent--not the ports parameter to postMessage
>> (that's a sequence).
>
> Does this affect the API for the SharedWorker onconnect message as well?
Good point; that might inform not deprecating the ports array in
MessageEvent, but leaving it as is.
-Ken
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, Jun 3, 2011 at 3:23 PM, Glenn Maynard wrote:
> On Fri, Jun 3, 2011 at 5:15 PM, Andrew Wilson wrote:
> > significant motivation. The stated motivations for breaking this API
> don't
> > seem compelling to me given the existence of backwards-compatible
> > alternatives.
>
> This proposal is backwards-compatible. If the argument is an array,
> nothing changes, so postMessage(..., [ports]) is equivalent to
> postMessage(..., {ports: [ports]}). (The array-only approach can be
> done compatibly, too; the object version is just an alternative to
> that.) What's backwards-incompatible?
>
Ah, I missed that piece (to be honest, I haven't been following this
discussion in every detail - I only chimed in because of Jonas' request for
implementation feedback).
> For anyone not looking closely at the IDL while reading this, this
> means deprecating (for whatever value "deprecate" has on the web) the
> ports array in MessageEvent--not the ports parameter to postMessage
> (that's a sequence).
>
Does this affect the API for the SharedWorker onconnect message as well?
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
> On Fri, Jun 3, 2011 at 6:02 PM, Jonas Sicking wrote: >> e) Keep "MessagePort[] ports" the way it is but deprecate it. > > For anyone not looking closely at the IDL while reading this, this > means deprecating (for whatever value "deprecate" has on the web) the > ports array in MessageEvent--not the ports parameter to postMessage > (that's a sequence). > > I agree that it'd be great to be able to pass MessagePorts around like > other objects. I havn't had to use them with any complexity yet, but > the current mechanism seems cumbersome for moving more than one port > around at a time. In this case, deprecating would mean that browsers that hasn't implemented the property yet wouldn't do so. And if they wanted to, browsers that do implement it could add a warning in the developer console any time the property is used and eventually remove the property once it seems likely that people have stopped using it. / Jonas
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, Jun 3, 2011 at 5:15 PM, Andrew Wilson wrote:
> significant motivation. The stated motivations for breaking this API don't
> seem compelling to me given the existence of backwards-compatible
> alternatives.
This proposal is backwards-compatible. If the argument is an array,
nothing changes, so postMessage(..., [ports]) is equivalent to
postMessage(..., {ports: [ports]}). (The array-only approach can be
done compatibly, too; the object version is just an alternative to
that.) What's backwards-incompatible?
On Fri, Jun 3, 2011 at 5:25 PM, Dmitry Lomov wrote:
>> >> If you want a quicker way to transfer all messages of given types, see
>> >> my previous mail: { transfer: ArrayBuffer }.
>
> (sorry I somehow missed your previous mail when replying)
> I do not think filtering by types solve much.
It solves the "very large transfer list" performance question, though
it hasn't been shown whether that's an actual problem. It also allows
writing functions like postMessageTransfer(port, obj) {
port.postMessage(obj, { transfer: [ArrayBuffer] }); }, without having
to do the recursion yourself to find the ArrayBuffers.
(Dmitry: I'm receiving each of your mails twice; once from
dslo...@chromium.org and once from dslo...@google.com.)
On Fri, Jun 3, 2011 at 5:54 PM, Kenneth Russell wrote:
> b) Leave the "MessagePort[] ports" property. Filter out all
> MessagePorts from the incoming Transferables[], and have only them
> show up in the ports array on the other side.
FYI, this is the same as what I described earlier.
http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/0812.html
(second paragraph)
On Fri, Jun 3, 2011 at 6:02 PM, Jonas Sicking wrote:
> e) Keep "MessagePort[] ports" the way it is but deprecate it.
For anyone not looking closely at the IDL while reading this, this
means deprecating (for whatever value "deprecate" has on the web) the
ports array in MessageEvent--not the ports parameter to postMessage
(that's a sequence).
I agree that it'd be great to be able to pass MessagePorts around like
other objects. I havn't had to use them with any complexity yet, but
the current mechanism seems cumbersome for moving more than one port
around at a time.
--
Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, Jun 3, 2011 at 2:54 PM, Kenneth Russell wrote:
> On Fri, Jun 3, 2011 at 2:15 PM, Andrew Wilson wrote:
>>
>>
>> On Fri, Jun 3, 2011 at 1:02 PM, Jonas Sicking wrote:
>>>
>>> On Fri, Jun 3, 2011 at 11:37 AM, Kenneth Russell wrote:
>>> > On Fri, Jun 3, 2011 at 9:46 AM, Glenn Maynard wrote:
>>> >> On Fri, Jun 3, 2011 at 11:12 AM, Dmitry Lomov
>>> >> wrote:
>>> >>> a) Recursive transfer lists. Allow arbitrary objects, not only
>>> >>> ArrayBuffers,
>>> >>> to appear in transfer lists. ArrayBuffers that are under objects in
>>> >>> transfer lists are transferred, others are cloned.
>>> >>
>>> >> This again causes the same forwards-compatibility problem. If you do
>>> >> this:
>>> >>
>>> >> frame = { video: canvasPixelArrayInstance, histogram: arrayBuffer } }
>>> >> postMessage({ frame: frame }, { transfer: frame });
>>> >>
>>> >> and you expect only histogram to be transferred (since that's all that
>>> >> supports it when you write this code), your code breaks when
>>> >> CanvasPixelArray later supports it.
>>> >>
>>> >>> b) Transfer lists + separate transferMessage method. We still equip
>>> >>> postMessage with transfer lists, these transfer lists list
>>> >>> ArrayBuffers, and
>>> >>> we provide a separate method transferMessage with recursive transfer
>>> >>> semantics.
>>> >>> What do people think?
>>> >>
>>> >> Same problem.
>>> >>
>>> >> If you want a quicker way to transfer all messages of given types, see
>>> >> my previous mail: { transfer: ArrayBuffer }.
>>> >
>>> > Agreed on these points. Using an object graph for the transfer list
>>> > (which is what the recursive transfer list idea boils down to) also
>>> > sounds overly complicated.
>>> >
>>> > May I suggest to reconsider adding another optional array argument to
>>> > postMessage for the transfer list, rather than using an object with
>>> > special properties?
>>> >
>>> > Points in favor of adding another optional array argument:
>>> >
>>> > 1. Less typing, and less possibility that a typo will cause incorrect
>>> > behavior:
>>> > worker.postMessage(objectGraph, null, [ arrayBuffer1ToTransfer,
>>> > arrayBuffer2ToTransfer ]);
>>> > vs.
>>> > worker.postMessage(objectGraph, { transfer: [
>>> > arrayBuffer1ToTransfer, arrayBuffer2ToTransfer] });
>>> >
>>> > 2. Possibility of using Web IDL to specify the type of the optional
>>> > array argument (i.e., "Transferable[]?"). Would be harder to do using
>>> > an object -- requiring either specifying another interface type with
>>> > the "ports" and "transfer" attributes, or using "any" plus
>>> > ECMAScript-specific text.
>>> >
>>> > Points in favor of using an object:
>>> >
>>> > 1. Could potentially overload the meaning of the optional ports array
>>> > argument, avoiding adding another argument to postMessage.
>>> >
>>> > 2. More extensible in the future.
>>> >
>>> > Thoughts?
>>>
>>> My first thought is that so far no implementer has stepped up and said
>>> that changing the meaning of the 'ports' argument would not be
>>> acceptable. Would be great if someone who is reading this thread and
>>> who works at Google/Apple/Opera could check with the relevant people
>>> to see if such a change would be possible.
>>
>> It's certainly possible - there's nothing intrinsically difficult with
>> making this change from an implementor's point of view (I've had my fingers
>> in both the WebKit and Chromium MessagePort implementations so I'm
>> relatively confident that this would not be prohibitively hard).
>> Is it desirable? My knee-jerk reaction is that we should stick with changes
>> that are compatible with the existing API (like Ian's original suggestion,
>> or adding a separate optional array of transferable objects) - I have no
>> data on the number of sites that are using the current API but I don't think
>> we should be changing existing APIs with multiple implementations without
>> significant motivation. The stated motivations for breaking this API don't
>> seem compelling to me given the existence of backwards-compatible
>> alternatives.
>
> Drew pointed me off-list to the original discussion motivating the
> addition of the optional argument for the array of ports:
>
> http://lists.w3.org/Archives/Public/public-html-comments/2009Mar/0001.html
>
> As I've been thinking more about Ian's original proposal, it seems to
> me that I might have misunderstood the intent.
>
> The current optional array of MessagePorts is very close to supporting
> the semantic of transfer of ownership that is desired. If a
> MessagePort is passed in this array, its ownership is transferred to
> the other side. The undesirable aspects seem to be:
>
> 1) It doesn't seem to be allowed to refer to those MessagePort objects
> in the object graph which is the first argument to postMessage. At
> least, it looks to me like the structured clone algorithm will throw a
> DATA_CLONE_ERR exception if it encounters a MessagePort object.
>
> 2) The array of ports shows up on the other side as the "ports"
> propert
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, Jun 3, 2011 at 2:15 PM, Andrew Wilson wrote:
>
>
> On Fri, Jun 3, 2011 at 1:02 PM, Jonas Sicking wrote:
>>
>> On Fri, Jun 3, 2011 at 11:37 AM, Kenneth Russell wrote:
>> > On Fri, Jun 3, 2011 at 9:46 AM, Glenn Maynard wrote:
>> >> On Fri, Jun 3, 2011 at 11:12 AM, Dmitry Lomov
>> >> wrote:
>> >>> a) Recursive transfer lists. Allow arbitrary objects, not only
>> >>> ArrayBuffers,
>> >>> to appear in transfer lists. ArrayBuffers that are under objects in
>> >>> transfer lists are transferred, others are cloned.
>> >>
>> >> This again causes the same forwards-compatibility problem. If you do
>> >> this:
>> >>
>> >> frame = { video: canvasPixelArrayInstance, histogram: arrayBuffer } }
>> >> postMessage({ frame: frame }, { transfer: frame });
>> >>
>> >> and you expect only histogram to be transferred (since that's all that
>> >> supports it when you write this code), your code breaks when
>> >> CanvasPixelArray later supports it.
>> >>
>> >>> b) Transfer lists + separate transferMessage method. We still equip
>> >>> postMessage with transfer lists, these transfer lists list
>> >>> ArrayBuffers, and
>> >>> we provide a separate method transferMessage with recursive transfer
>> >>> semantics.
>> >>> What do people think?
>> >>
>> >> Same problem.
>> >>
>> >> If you want a quicker way to transfer all messages of given types, see
>> >> my previous mail: { transfer: ArrayBuffer }.
>> >
>> > Agreed on these points. Using an object graph for the transfer list
>> > (which is what the recursive transfer list idea boils down to) also
>> > sounds overly complicated.
>> >
>> > May I suggest to reconsider adding another optional array argument to
>> > postMessage for the transfer list, rather than using an object with
>> > special properties?
>> >
>> > Points in favor of adding another optional array argument:
>> >
>> > 1. Less typing, and less possibility that a typo will cause incorrect
>> > behavior:
>> > worker.postMessage(objectGraph, null, [ arrayBuffer1ToTransfer,
>> > arrayBuffer2ToTransfer ]);
>> > vs.
>> > worker.postMessage(objectGraph, { transfer: [
>> > arrayBuffer1ToTransfer, arrayBuffer2ToTransfer] });
>> >
>> > 2. Possibility of using Web IDL to specify the type of the optional
>> > array argument (i.e., "Transferable[]?"). Would be harder to do using
>> > an object -- requiring either specifying another interface type with
>> > the "ports" and "transfer" attributes, or using "any" plus
>> > ECMAScript-specific text.
>> >
>> > Points in favor of using an object:
>> >
>> > 1. Could potentially overload the meaning of the optional ports array
>> > argument, avoiding adding another argument to postMessage.
>> >
>> > 2. More extensible in the future.
>> >
>> > Thoughts?
>>
>> My first thought is that so far no implementer has stepped up and said
>> that changing the meaning of the 'ports' argument would not be
>> acceptable. Would be great if someone who is reading this thread and
>> who works at Google/Apple/Opera could check with the relevant people
>> to see if such a change would be possible.
>
> It's certainly possible - there's nothing intrinsically difficult with
> making this change from an implementor's point of view (I've had my fingers
> in both the WebKit and Chromium MessagePort implementations so I'm
> relatively confident that this would not be prohibitively hard).
> Is it desirable? My knee-jerk reaction is that we should stick with changes
> that are compatible with the existing API (like Ian's original suggestion,
> or adding a separate optional array of transferable objects) - I have no
> data on the number of sites that are using the current API but I don't think
> we should be changing existing APIs with multiple implementations without
> significant motivation. The stated motivations for breaking this API don't
> seem compelling to me given the existence of backwards-compatible
> alternatives.
Drew pointed me off-list to the original discussion motivating the
addition of the optional argument for the array of ports:
http://lists.w3.org/Archives/Public/public-html-comments/2009Mar/0001.html
As I've been thinking more about Ian's original proposal, it seems to
me that I might have misunderstood the intent.
The current optional array of MessagePorts is very close to supporting
the semantic of transfer of ownership that is desired. If a
MessagePort is passed in this array, its ownership is transferred to
the other side. The undesirable aspects seem to be:
1) It doesn't seem to be allowed to refer to those MessagePort objects
in the object graph which is the first argument to postMessage. At
least, it looks to me like the structured clone algorithm will throw a
DATA_CLONE_ERR exception if it encounters a MessagePort object.
2) The array of ports shows up on the other side as the "ports"
property of the MessageEvent.
(1) could be solved, and in a backward-compatible way, by defining the
behavior of MessagePort under structured clone. "Cloning" a
MessagePort would cause a
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, Jun 3, 2011 at 2:25 PM, Dmitry Lomov wrote:
> (I am answering on multiple points - I do not want to fork the thread)
>
> On Fri, Jun 3, 2011 at 1:02 PM, Jonas Sicking wrote:
>>
>> On Fri, Jun 3, 2011 at 11:37 AM, Kenneth Russell wrote:
>> > On Fri, Jun 3, 2011 at 9:46 AM, Glenn Maynard wrote:
>> >> On Fri, Jun 3, 2011 at 11:12 AM, Dmitry Lomov
>> >> wrote:
>> >>> a) Recursive transfer lists. Allow arbitrary objects, not only
>> >>> ArrayBuffers,
>> >>> to appear in transfer lists. ArrayBuffers that are under objects in
>> >>> transfer lists are transferred, others are cloned.
>> >>
>> >> This again causes the same forwards-compatibility problem. If you do
>> >> this:
>> >>
>> >> frame = { video: canvasPixelArrayInstance, histogram: arrayBuffer } }
>> >> postMessage({ frame: frame }, { transfer: frame });
>> >>
>> >> and you expect only histogram to be transferred (since that's all that
>> >> supports it when you write this code), your code breaks when
>> >> CanvasPixelArray later supports it.
>
> Right, but you should not write { transfer : frame }, you should write:
> postMessage( { frame: frame }, { transfer: frame.histogram })
> in this case. The guidance for transferring objects should be: do not
> transfer objects you intend to use on this "thread" (on main thread, worker)
> - only transfer parts that you do not intend to use. This version of
> postMessage gives a high-fidelity tool for you to do so if desired, but
> handles the opaque case as well.
Right, but we all know that web authors (and us too) quite often have
bugs. Web authors in particular are infamous for using things as long
as they work even if it happens to rely on undefined behavior, browser
bugs, or goes against recommendations in the spec.
The point is that if we allow whole object graphs to be transferred,
we are making it much riskier for ourselves in the future to add
additional transferable types. It does not seem unlikely at all that
we'll end up wanting to make a type transferable but are unable to do
so because a couple of big sites rely on them not to be.
>> > Agreed on these points. Using an object graph for the transfer list
>> > (which is what the recursive transfer list idea boils down to) also
>> > sounds overly complicated.
>
> It feels that but not solving this as part of messaging spec we just offload
> the complexity to the users.
Complexity comes in many forms and shapes. I much more like the idea
of explicit APIs that make it clear what happens and make it hard to
shoot yourself in the foot. Yes, it can involve more typing, but if it
results in more resilient code which contains fewer subtle bugs, then
I think we have designed the API well.
/ Jonas
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, Jun 3, 2011 at 2:15 PM, Andrew Wilson wrote:
>
>
> On Fri, Jun 3, 2011 at 1:02 PM, Jonas Sicking wrote:
>
>> On Fri, Jun 3, 2011 at 11:37 AM, Kenneth Russell wrote:
>> > On Fri, Jun 3, 2011 at 9:46 AM, Glenn Maynard wrote:
>> >> On Fri, Jun 3, 2011 at 11:12 AM, Dmitry Lomov
>> wrote:
>> >>> a) Recursive transfer lists. Allow arbitrary objects, not only
>> ArrayBuffers,
>> >>> to appear in transfer lists. ArrayBuffers that are under objects in
>> >>> transfer lists are transferred, others are cloned.
>> >>
>> >> This again causes the same forwards-compatibility problem. If you do
>> this:
>> >>
>> >> frame = { video: canvasPixelArrayInstance, histogram: arrayBuffer } }
>> >> postMessage({ frame: frame }, { transfer: frame });
>> >>
>> >> and you expect only histogram to be transferred (since that's all that
>> >> supports it when you write this code), your code breaks when
>> >> CanvasPixelArray later supports it.
>>
>
Right, but you should not write { transfer : frame }, you should write:
postMessage( { frame: frame }, { transfer: frame.histogram })
in this case. The guidance for transferring objects should be: do not
transfer objects you intend to use on this "thread" (on main thread, worker)
- only transfer parts that you do not intend to use. This version of
postMessage gives a high-fidelity tool for you to do so if desired, but
handles the opaque case as well.
> >>
>> >>> b) Transfer lists + separate transferMessage method. We still equip
>> >>> postMessage with transfer lists, these transfer lists list
>> ArrayBuffers, and
>> >>> we provide a separate method transferMessage with recursive transfer
>> >>> semantics.
>> >>> What do people think?
>> >>
>> >> Same problem.
>> >>
>> >> If you want a quicker way to transfer all messages of given types, see
>> >> my previous mail: { transfer: ArrayBuffer }.
>>
>
(sorry I somehow missed your previous mail when replying)
I do not think filtering by types solve much.
Consider again a case of two library functions compute() returning some data
and present(data) rendering some data, where library user treats the data as
a black box. In first version, library author used ArrayBuffers as part of
data internally, because it is the only data type that is transferrable.
Now, CanvasPixelArray became transferrable as well, and library author
changes the library to compute those as part of the data, to offload more of
the computation to "compute" phase. With filtering by types, all library
user will need to modify their code as well, to add CanvasPixelArray to a
list of transferrable types in their postMessages.
Again, I think having a transfer list (as some argument to postMessage) is
great solution for high-fidelity control. Low-fidelity, opaque, "don't show
me the implementation" use cases is what is missing.
> >
>> > Agreed on these points. Using an object graph for the transfer list
>> > (which is what the recursive transfer list idea boils down to) also
>> > sounds overly complicated.
>>
>
It feels that but not solving this as part of messaging spec we just offload
the complexity to the users.
> >
>> > May I suggest to reconsider adding another optional array argument to
>> > postMessage for the transfer list, rather than using an object with
>> > special properties?
>> >
>> > Points in favor of adding another optional array argument:
>> >
>> > 1. Less typing, and less possibility that a typo will cause incorrect
>> behavior:
>> > worker.postMessage(objectGraph, null, [ arrayBuffer1ToTransfer,
>> > arrayBuffer2ToTransfer ]);
>> >vs.
>> > worker.postMessage(objectGraph, { transfer: [
>> > arrayBuffer1ToTransfer, arrayBuffer2ToTransfer] });
>> >
>> > 2. Possibility of using Web IDL to specify the type of the optional
>> > array argument (i.e., "Transferable[]?"). Would be harder to do using
>> > an object -- requiring either specifying another interface type with
>> > the "ports" and "transfer" attributes, or using "any" plus
>> > ECMAScript-specific text.
>> >
>> > Points in favor of using an object:
>> >
>> > 1. Could potentially overload the meaning of the optional ports array
>> > argument, avoiding adding another argument to postMessage.
>> >
>> > 2. More extensible in the future.
>> >
>> > Thoughts?
>>
>> My first thought is that so far no implementer has stepped up and said
>> that changing the meaning of the 'ports' argument would not be
>> acceptable. Would be great if someone who is reading this thread and
>> who works at Google/Apple/Opera could check with the relevant people
>> to see if such a change would be possible.
>>
>
> It's certainly possible - there's nothing intrinsically difficult with
> making this change from an implementor's point of view (I've had my fingers
> in both the WebKit and Chromium MessagePort implementations so I'm
> relatively confident that this would not be prohibitively hard).
>
> Is it desirable? My knee-jerk reaction is that we should stick with changes
> that are compatible with the existing
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, Jun 3, 2011 at 1:02 PM, Jonas Sicking wrote:
> On Fri, Jun 3, 2011 at 11:37 AM, Kenneth Russell wrote:
> > On Fri, Jun 3, 2011 at 9:46 AM, Glenn Maynard wrote:
> >> On Fri, Jun 3, 2011 at 11:12 AM, Dmitry Lomov
> wrote:
> >>> a) Recursive transfer lists. Allow arbitrary objects, not only
> ArrayBuffers,
> >>> to appear in transfer lists. ArrayBuffers that are under objects in
> >>> transfer lists are transferred, others are cloned.
> >>
> >> This again causes the same forwards-compatibility problem. If you do
> this:
> >>
> >> frame = { video: canvasPixelArrayInstance, histogram: arrayBuffer } }
> >> postMessage({ frame: frame }, { transfer: frame });
> >>
> >> and you expect only histogram to be transferred (since that's all that
> >> supports it when you write this code), your code breaks when
> >> CanvasPixelArray later supports it.
> >>
> >>> b) Transfer lists + separate transferMessage method. We still equip
> >>> postMessage with transfer lists, these transfer lists list
> ArrayBuffers, and
> >>> we provide a separate method transferMessage with recursive transfer
> >>> semantics.
> >>> What do people think?
> >>
> >> Same problem.
> >>
> >> If you want a quicker way to transfer all messages of given types, see
> >> my previous mail: { transfer: ArrayBuffer }.
> >
> > Agreed on these points. Using an object graph for the transfer list
> > (which is what the recursive transfer list idea boils down to) also
> > sounds overly complicated.
> >
> > May I suggest to reconsider adding another optional array argument to
> > postMessage for the transfer list, rather than using an object with
> > special properties?
> >
> > Points in favor of adding another optional array argument:
> >
> > 1. Less typing, and less possibility that a typo will cause incorrect
> behavior:
> > worker.postMessage(objectGraph, null, [ arrayBuffer1ToTransfer,
> > arrayBuffer2ToTransfer ]);
> >vs.
> > worker.postMessage(objectGraph, { transfer: [
> > arrayBuffer1ToTransfer, arrayBuffer2ToTransfer] });
> >
> > 2. Possibility of using Web IDL to specify the type of the optional
> > array argument (i.e., "Transferable[]?"). Would be harder to do using
> > an object -- requiring either specifying another interface type with
> > the "ports" and "transfer" attributes, or using "any" plus
> > ECMAScript-specific text.
> >
> > Points in favor of using an object:
> >
> > 1. Could potentially overload the meaning of the optional ports array
> > argument, avoiding adding another argument to postMessage.
> >
> > 2. More extensible in the future.
> >
> > Thoughts?
>
> My first thought is that so far no implementer has stepped up and said
> that changing the meaning of the 'ports' argument would not be
> acceptable. Would be great if someone who is reading this thread and
> who works at Google/Apple/Opera could check with the relevant people
> to see if such a change would be possible.
>
It's certainly possible - there's nothing intrinsically difficult with
making this change from an implementor's point of view (I've had my fingers
in both the WebKit and Chromium MessagePort implementations so I'm
relatively confident that this would not be prohibitively hard).
Is it desirable? My knee-jerk reaction is that we should stick with changes
that are compatible with the existing API (like Ian's original suggestion,
or adding a separate optional array of transferable objects) - I have no
data on the number of sites that are using the current API but I don't think
we should be changing existing APIs with multiple implementations without
significant motivation. The stated motivations for breaking this API don't
seem compelling to me given the existence of backwards-compatible
alternatives.
>
> / Jonas
>
>
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, Jun 3, 2011 at 11:37 AM, Kenneth Russell wrote:
> On Fri, Jun 3, 2011 at 9:46 AM, Glenn Maynard wrote:
>> On Fri, Jun 3, 2011 at 11:12 AM, Dmitry Lomov wrote:
>>> a) Recursive transfer lists. Allow arbitrary objects, not only ArrayBuffers,
>>> to appear in transfer lists. ArrayBuffers that are under objects in
>>> transfer lists are transferred, others are cloned.
>>
>> This again causes the same forwards-compatibility problem. If you do this:
>>
>> frame = { video: canvasPixelArrayInstance, histogram: arrayBuffer } }
>> postMessage({ frame: frame }, { transfer: frame });
>>
>> and you expect only histogram to be transferred (since that's all that
>> supports it when you write this code), your code breaks when
>> CanvasPixelArray later supports it.
>>
>>> b) Transfer lists + separate transferMessage method. We still equip
>>> postMessage with transfer lists, these transfer lists list ArrayBuffers, and
>>> we provide a separate method transferMessage with recursive transfer
>>> semantics.
>>> What do people think?
>>
>> Same problem.
>>
>> If you want a quicker way to transfer all messages of given types, see
>> my previous mail: { transfer: ArrayBuffer }.
>
> Agreed on these points. Using an object graph for the transfer list
> (which is what the recursive transfer list idea boils down to) also
> sounds overly complicated.
>
> May I suggest to reconsider adding another optional array argument to
> postMessage for the transfer list, rather than using an object with
> special properties?
>
> Points in favor of adding another optional array argument:
>
> 1. Less typing, and less possibility that a typo will cause incorrect
> behavior:
> worker.postMessage(objectGraph, null, [ arrayBuffer1ToTransfer,
> arrayBuffer2ToTransfer ]);
> vs.
> worker.postMessage(objectGraph, { transfer: [
> arrayBuffer1ToTransfer, arrayBuffer2ToTransfer] });
>
> 2. Possibility of using Web IDL to specify the type of the optional
> array argument (i.e., "Transferable[]?"). Would be harder to do using
> an object -- requiring either specifying another interface type with
> the "ports" and "transfer" attributes, or using "any" plus
> ECMAScript-specific text.
>
> Points in favor of using an object:
>
> 1. Could potentially overload the meaning of the optional ports array
> argument, avoiding adding another argument to postMessage.
>
> 2. More extensible in the future.
>
> Thoughts?
My first thought is that so far no implementer has stepped up and said
that changing the meaning of the 'ports' argument would not be
acceptable. Would be great if someone who is reading this thread and
who works at Google/Apple/Opera could check with the relevant people
to see if such a change would be possible.
/ Jonas
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, Jun 3, 2011 at 9:46 AM, Glenn Maynard wrote:
> On Fri, Jun 3, 2011 at 11:12 AM, Dmitry Lomov wrote:
>> a) Recursive transfer lists. Allow arbitrary objects, not only ArrayBuffers,
>> to appear in transfer lists. ArrayBuffers that are under objects in
>> transfer lists are transferred, others are cloned.
>
> This again causes the same forwards-compatibility problem. If you do this:
>
> frame = { video: canvasPixelArrayInstance, histogram: arrayBuffer } }
> postMessage({ frame: frame }, { transfer: frame });
>
> and you expect only histogram to be transferred (since that's all that
> supports it when you write this code), your code breaks when
> CanvasPixelArray later supports it.
>
>> b) Transfer lists + separate transferMessage method. We still equip
>> postMessage with transfer lists, these transfer lists list ArrayBuffers, and
>> we provide a separate method transferMessage with recursive transfer
>> semantics.
>> What do people think?
>
> Same problem.
>
> If you want a quicker way to transfer all messages of given types, see
> my previous mail: { transfer: ArrayBuffer }.
Agreed on these points. Using an object graph for the transfer list
(which is what the recursive transfer list idea boils down to) also
sounds overly complicated.
May I suggest to reconsider adding another optional array argument to
postMessage for the transfer list, rather than using an object with
special properties?
Points in favor of adding another optional array argument:
1. Less typing, and less possibility that a typo will cause incorrect behavior:
worker.postMessage(objectGraph, null, [ arrayBuffer1ToTransfer,
arrayBuffer2ToTransfer ]);
vs.
worker.postMessage(objectGraph, { transfer: [
arrayBuffer1ToTransfer, arrayBuffer2ToTransfer] });
2. Possibility of using Web IDL to specify the type of the optional
array argument (i.e., "Transferable[]?"). Would be harder to do using
an object -- requiring either specifying another interface type with
the "ports" and "transfer" attributes, or using "any" plus
ECMAScript-specific text.
Points in favor of using an object:
1. Could potentially overload the meaning of the optional ports array
argument, avoiding adding another argument to postMessage.
2. More extensible in the future.
Thoughts?
-Ken
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, Jun 3, 2011 at 11:12 AM, Dmitry Lomov wrote:
> a) Recursive transfer lists. Allow arbitrary objects, not only ArrayBuffers,
> to appear in transfer lists. ArrayBuffers that are under objects in
> transfer lists are transferred, others are cloned.
This again causes the same forwards-compatibility problem. If you do this:
frame = { video: canvasPixelArrayInstance, histogram: arrayBuffer } }
postMessage({ frame: frame }, { transfer: frame });
and you expect only histogram to be transferred (since that's all that
supports it when you write this code), your code breaks when
CanvasPixelArray later supports it.
> b) Transfer lists + separate transferMessage method. We still equip
> postMessage with transfer lists, these transfer lists list ArrayBuffers, and
> we provide a separate method transferMessage with recursive transfer
> semantics.
> What do people think?
Same problem.
If you want a quicker way to transfer all messages of given types, see
my previous mail: { transfer: ArrayBuffer }.
--
Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 11:44 PM, Dmitry Lomov wrote:
>
>
> On Thu, Jun 2, 2011 at 10:17 PM, Jonas Sicking wrote:
>
>> On Thu, Jun 2, 2011 at 4:41 PM, David Levin wrote:
>> >
>> >
>> > On Thu, Jun 2, 2011 at 4:24 PM, Jonas Sicking wrote:
>> >>
>> >> On Thu, Jun 2, 2011 at 2:01 PM, David Levin
>> wrote:
>> >> >
>> >> >
>> >> > On Thu, Jun 2, 2011 at 1:27 PM, Glenn Maynard
>> wrote:
>> >> >> port.postMessage({frameBuffer: frame}, {transfer: [frame], ports:
>> >> >> [port]});
>> >>
>> >> There are two properties of this approach that I like:
>> >>
>> >> 1. It means that objects which you'd like to transfer ownership are
>> >> not "second class citizens" and can live as part of the normal object
>> >> graph that is posted, together with metadata that goes with it (or
>> >> even as metadata for other things).
>> >>
>> >> 2. The receiving side doesn't need to worry about the difference, all
>> >> it gets is the graph of objects that was sent to it.
>> >
>> > Yep, I totally agree with this. All of the current solutions being
>> discussed
>> > satisfy both of these in fact.
>> >
>> >>
>> >> > It also raises questions when I see it. When I list an object there
>> does
>> >> > it
>> >> > imply that all children are also transfered or do I have to list each
>> of
>> >> > them explicitly as well?)
>> >>
>> >> None of the objects which allow transferring of ownership has children
>> >> so this doesn't appear to be a problem at this time. If it indeed does
>> >> turn into a problem, it would seem like a problem no matter what
>> >> solution is used, no?
>> >
>> > Not if all objects are transferred.
>>
>> Define "all objects". Consider something like:
>>
>> a = { x: myArrayBuffer1, y: myArrayBuffer2 };
>> worker.postMessage(a, { transfer: true });
>>
>> In this case the 'a' object is obviously not transferred. Or are you
>> proposing that it'd be transferred too somehow?
>>
>> >> > Then I wonder what is the use case for this complexity.
>> >> > Why not something simpler like this?
>> >> > port.postMessage({frameBuffer: frame}, {transfer: true, ports:
>> >> > [port]});
>> >> > where you can just say indicate that you want the message transfered.
>> >>
>> >> This means that you have to choose between transferring all arrays and
>> >> transferring none of them.
>> >
>> > Yep, why add the complication of picking individual items to transfer
>> > over? (I'm wary of second system syndrome, which I've seen happen many
>> > times in various designs.)
>> >
>> >>
>> >> It also makes it much less explicit which
>> >> objects ends up being mutated.
>> >
>> > It is all of them.
>>
>> Sure, but what "all" includes might not be obvious to the web
>> developer in all cases. For example if you're receiving an object from
>> some subsystem that you intend to do processing on. You later decide
>> to do the processing in a thread and throw that object into an array
>> alongside with some other data. Some of the "other data" you are
>> throwing in includes some big arrays that you want to avoid copying
>> and so you choose to use the transfer rather than copy mode.
>>
>> Now you all of a sudden start modifying the data that you got from the
>> other subsystem. This is data that you might normally not be even
>> looking at. The fact that it happened to contain some ArrayBuffers was
>> just a side effect of that that was the data structures that the other
>> subsystem uses and was easy for it to return to you.
>>
>> > Here's a simple use case, suppose I create an array of arrays (a 2d
>> array)
>> > which contains ArrayBuffers.Now I want to transfer this as fast as
>> possible
>> > using postMessage.
>> > What does my code look like for each of these apis?
>>
>> Your proposal:
>> w.postMessage(my2darray, {transfer: true});
>> vs.
>> w.postMessage(my2darray, Array.concat.apply(Array, my2darray));
>>
>
> Hmm, if my2darray is large, the latter is pretty wasteful.
>
>
>>
>> Now show me the code needed to send a message which contains one big
>> buffer from you that you want to transfer, along with some data that
>> you got from some other piece of code and which you do not want to
>> modify and which may or may not contain ArrayBuffers.
>>
>
> I think this can be reversed: what if you got some data from some other
> piece of code that you do not really control and you want to transfer (i.e.
> transfer all the ArrayBuffers that are inside that data)?
>
> As in, the library gives you two methods, compute() that returns data and
> present(data) that presents it, and you want to pass data from worker to
> main thread without really caring what is inside that data.
>
>
>> / Jonas
>>
>>
So, to clarify, we have two competing requirements here:
1) exercise fine-grained control over which parts of the message are
transfrerred
2) be able to transfer objects opaquely, for modularity or performance
reasons.
Here are two proposals that satisfy both:
a) *Recursive transfer lists*. Allow arbitrary objects, not only
ArrayBuffers, to appear in transfe
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 10:17 PM, Jonas Sicking wrote:
> On Thu, Jun 2, 2011 at 4:41 PM, David Levin wrote:
> >
> >
> > On Thu, Jun 2, 2011 at 4:24 PM, Jonas Sicking wrote:
> >>
> >> On Thu, Jun 2, 2011 at 2:01 PM, David Levin wrote:
> >> >
> >> >
> >> > On Thu, Jun 2, 2011 at 1:27 PM, Glenn Maynard wrote:
> >> >> port.postMessage({frameBuffer: frame}, {transfer: [frame], ports:
> >> >> [port]});
> >>
> >> There are two properties of this approach that I like:
> >>
> >> 1. It means that objects which you'd like to transfer ownership are
> >> not "second class citizens" and can live as part of the normal object
> >> graph that is posted, together with metadata that goes with it (or
> >> even as metadata for other things).
> >>
> >> 2. The receiving side doesn't need to worry about the difference, all
> >> it gets is the graph of objects that was sent to it.
> >
> > Yep, I totally agree with this. All of the current solutions being
> discussed
> > satisfy both of these in fact.
> >
> >>
> >> > It also raises questions when I see it. When I list an object there
> does
> >> > it
> >> > imply that all children are also transfered or do I have to list each
> of
> >> > them explicitly as well?)
> >>
> >> None of the objects which allow transferring of ownership has children
> >> so this doesn't appear to be a problem at this time. If it indeed does
> >> turn into a problem, it would seem like a problem no matter what
> >> solution is used, no?
> >
> > Not if all objects are transferred.
>
> Define "all objects". Consider something like:
>
> a = { x: myArrayBuffer1, y: myArrayBuffer2 };
> worker.postMessage(a, { transfer: true });
>
> In this case the 'a' object is obviously not transferred. Or are you
> proposing that it'd be transferred too somehow?
>
> >> > Then I wonder what is the use case for this complexity.
> >> > Why not something simpler like this?
> >> > port.postMessage({frameBuffer: frame}, {transfer: true, ports:
> >> > [port]});
> >> > where you can just say indicate that you want the message transfered.
> >>
> >> This means that you have to choose between transferring all arrays and
> >> transferring none of them.
> >
> > Yep, why add the complication of picking individual items to transfer
> > over? (I'm wary of second system syndrome, which I've seen happen many
> > times in various designs.)
> >
> >>
> >> It also makes it much less explicit which
> >> objects ends up being mutated.
> >
> > It is all of them.
>
> Sure, but what "all" includes might not be obvious to the web
> developer in all cases. For example if you're receiving an object from
> some subsystem that you intend to do processing on. You later decide
> to do the processing in a thread and throw that object into an array
> alongside with some other data. Some of the "other data" you are
> throwing in includes some big arrays that you want to avoid copying
> and so you choose to use the transfer rather than copy mode.
>
> Now you all of a sudden start modifying the data that you got from the
> other subsystem. This is data that you might normally not be even
> looking at. The fact that it happened to contain some ArrayBuffers was
> just a side effect of that that was the data structures that the other
> subsystem uses and was easy for it to return to you.
>
> > Here's a simple use case, suppose I create an array of arrays (a 2d
> array)
> > which contains ArrayBuffers.Now I want to transfer this as fast as
> possible
> > using postMessage.
> > What does my code look like for each of these apis?
>
> Your proposal:
> w.postMessage(my2darray, {transfer: true});
> vs.
> w.postMessage(my2darray, Array.concat.apply(Array, my2darray));
>
Hmm, if my2darray is large, the latter is pretty wasteful.
>
> Now show me the code needed to send a message which contains one big
> buffer from you that you want to transfer, along with some data that
> you got from some other piece of code and which you do not want to
> modify and which may or may not contain ArrayBuffers.
>
I think this can be reversed: what if you got some data from some other
piece of code that you do not really control and you want to transfer (i.e.
transfer all the ArrayBuffers that are inside that data)?
As in, the library gives you two methods, compute() that returns data and
present(data) that presents it, and you want to pass data from worker to
main thread without really caring what is inside that data.
> / Jonas
>
>
Thanks!
Dmitry
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Fri, Jun 3, 2011 at 2:44 AM, Dmitry Lomov wrote:
>> Now show me the code needed to send a message which contains one big
>> buffer from you that you want to transfer, along with some data that
>> you got from some other piece of code and which you do not want to
>> modify and which may or may not contain ArrayBuffers.
>
> I think this can be reversed: what if you got some data from some other
> piece of code that you do not really control and you want to transfer (i.e.
> transfer all the ArrayBuffers that are inside that data)?
> As in, the library gives you two methods, compute() that returns data and
> present(data) that presents it, and you want to pass data from worker to
> main thread without really caring what is inside that data.
One possibility would be, in addition to instances of object, to also
allow providing the object constructor, indicating that all objects of
that type should be transferred if supported. {transfer:
[ArrayBuffer, ImageData]}
--
Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 10:17 PM, Jonas Sicking wrote:
> On Thu, Jun 2, 2011 at 4:41 PM, David Levin wrote:
> >> None of the objects which allow transferring of ownership has children
> >> so this doesn't appear to be a problem at this time. If it indeed does
> >> turn into a problem, it would seem like a problem no matter what
> >> solution is used, no?
> >
> > Not if all objects are transferred.
>
> Define "all objects". Consider something like:
>
> a = { x: myArrayBuffer1, y: myArrayBuffer2 };
> worker.postMessage(a, { transfer: true });
>
> In this case the 'a' object is obviously not transferred. Or are you
> proposing that it'd be transferred too somehow?
>
Well the algorithm could empty 'a'. As far as what happens underneath the
covers that is up to the implementation. (I suspect most javascript engines
today wouldn't allow for actually transferring the memory cross thread.)
>
> > Here's a simple use case, suppose I create an array of arrays (a 2d
> array)
> > which contains ArrayBuffers.Now I want to transfer this as fast as
> possible
> > using postMessage.
> > What does my code look like for each of these apis?
>
> Your proposal:
> w.postMessage(my2darray, {transfer: true});
> vs.
> w.postMessage(my2darray, Array.concat.apply(Array, my2darray));
I thought this would be:
w.postMessage(my2darray, {transfer: Array.concat.apply(Array,
my2darray)});
> Now show me the code needed to send a message which contains one big
> buffer from you that you want to transfer, along with some data that
> you got from some other piece of code and which you do not want to
> modify and which may or may not contain ArrayBuffers.
>
Fair enough. Way more complicated for the "transfer: true" case. :)
The thing that seemed odd to me about this extra array is that it seemed to
make the code more complicated and harder to understand. However, I
understand that folks want to support involved scenarios.
*Let's go with the transfer list.* (I suspect that something like transfer:
true or transfer: "all" would still be possible in the future if it proved
desirable since bools/strings won't be valid there.)
dave
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 4:41 PM, David Levin wrote:
>
>
> On Thu, Jun 2, 2011 at 4:24 PM, Jonas Sicking wrote:
>>
>> On Thu, Jun 2, 2011 at 2:01 PM, David Levin wrote:
>> >
>> >
>> > On Thu, Jun 2, 2011 at 1:27 PM, Glenn Maynard wrote:
>> >> port.postMessage({frameBuffer: frame}, {transfer: [frame], ports:
>> >> [port]});
>>
>> There are two properties of this approach that I like:
>>
>> 1. It means that objects which you'd like to transfer ownership are
>> not "second class citizens" and can live as part of the normal object
>> graph that is posted, together with metadata that goes with it (or
>> even as metadata for other things).
>>
>> 2. The receiving side doesn't need to worry about the difference, all
>> it gets is the graph of objects that was sent to it.
>
> Yep, I totally agree with this. All of the current solutions being discussed
> satisfy both of these in fact.
>
>>
>> > It also raises questions when I see it. When I list an object there does
>> > it
>> > imply that all children are also transfered or do I have to list each of
>> > them explicitly as well?)
>>
>> None of the objects which allow transferring of ownership has children
>> so this doesn't appear to be a problem at this time. If it indeed does
>> turn into a problem, it would seem like a problem no matter what
>> solution is used, no?
>
> Not if all objects are transferred.
Define "all objects". Consider something like:
a = { x: myArrayBuffer1, y: myArrayBuffer2 };
worker.postMessage(a, { transfer: true });
In this case the 'a' object is obviously not transferred. Or are you
proposing that it'd be transferred too somehow?
>> > Then I wonder what is the use case for this complexity.
>> > Why not something simpler like this?
>> > port.postMessage({frameBuffer: frame}, {transfer: true, ports:
>> > [port]});
>> > where you can just say indicate that you want the message transfered.
>>
>> This means that you have to choose between transferring all arrays and
>> transferring none of them.
>
> Yep, why add the complication of picking individual items to transfer
> over? (I'm wary of second system syndrome, which I've seen happen many
> times in various designs.)
>
>>
>> It also makes it much less explicit which
>> objects ends up being mutated.
>
> It is all of them.
Sure, but what "all" includes might not be obvious to the web
developer in all cases. For example if you're receiving an object from
some subsystem that you intend to do processing on. You later decide
to do the processing in a thread and throw that object into an array
alongside with some other data. Some of the "other data" you are
throwing in includes some big arrays that you want to avoid copying
and so you choose to use the transfer rather than copy mode.
Now you all of a sudden start modifying the data that you got from the
other subsystem. This is data that you might normally not be even
looking at. The fact that it happened to contain some ArrayBuffers was
just a side effect of that that was the data structures that the other
subsystem uses and was easy for it to return to you.
> Here's a simple use case, suppose I create an array of arrays (a 2d array)
> which contains ArrayBuffers.Now I want to transfer this as fast as possible
> using postMessage.
> What does my code look like for each of these apis?
Your proposal:
w.postMessage(my2darray, {transfer: true});
vs.
w.postMessage(my2darray, Array.concat.apply(Array, my2darray));
Now show me the code needed to send a message which contains one big
buffer from you that you want to transfer, along with some data that
you got from some other piece of code and which you do not want to
modify and which may or may not contain ArrayBuffers.
/ Jonas
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 4:24 PM, Jonas Sicking wrote:
> On Thu, Jun 2, 2011 at 2:01 PM, David Levin wrote:
> >
> >
> > On Thu, Jun 2, 2011 at 1:27 PM, Glenn Maynard wrote:
> >> port.postMessage({frameBuffer: frame}, {transfer: [frame], ports:
> >> [port]});
>
> There are two properties of this approach that I like:
>
> 1. It means that objects which you'd like to transfer ownership are
> not "second class citizens" and can live as part of the normal object
> graph that is posted, together with metadata that goes with it (or
> even as metadata for other things).
>
> 2. The receiving side doesn't need to worry about the difference, all
> it gets is the graph of objects that was sent to it.
>
Yep, I totally agree with this. All of the current solutions being discussed
satisfy both of these in fact.
>
> > It also raises questions when I see it. When I list an object there does
> it
> > imply that all children are also transfered or do I have to list each of
> > them explicitly as well?)
>
> None of the objects which allow transferring of ownership has children
> so this doesn't appear to be a problem at this time. If it indeed does
> turn into a problem, it would seem like a problem no matter what
> solution is used, no?
>
Not if all objects are transferred.
> > Then I wonder what is the use case for this complexity.
> > Why not something simpler like this?
> > port.postMessage({frameBuffer: frame}, {transfer: true, ports:
> [port]});
> > where you can just say indicate that you want the message transfered.
>
> This means that you have to choose between transferring all arrays and
> transferring none of them.
Yep, why add the complication of picking individual items to transfer
over? (I'm wary of second system syndrome, which I've seen happen many
times in various designs.)
> It also makes it much less explicit which
> objects ends up being mutated.
>
It is all of them.
Here's a simple use case, suppose I create an array of arrays (a 2d array)
which contains ArrayBuffers.Now I want to transfer this as fast as possible
using postMessage.
What does my code look like for each of these apis?
dave
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 5:30 PM, Glenn Maynard wrote: > On Thu, Jun 2, 2011 at 5:01 PM, David Levin wrote: >> It feels like this array of objects given to transfer may complicate (and >> slow down) both the implementation of this as well as the developer's use of >> it. > > Even with thousands of objects, creating an array containing them is > quick (and only needs to be done once), and the implementation would > presumably convert it to a set internally for quick lookups. I doubt > most use cases will transfer so many separate objects, though. > > (And Ian keeps drilling into our head that implementation complexity > isn't a major concern, though I don't imagine converting a list of > objects to a hash table is complex.) It's spec complexity he was talking about--but anyway, making a hash table or tree from a list doesn't seem complicated; maybe there are other issues. I suppose this could also be done with a per-object flag, eg. arrayBuffer.setCloneTransfer(true), but as it's the postMessage call being modified I'm not sure flagging it on the object itself is clearer. -- Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 2:01 PM, David Levin wrote:
>
>
> On Thu, Jun 2, 2011 at 1:27 PM, Glenn Maynard wrote:
>>
>> On Thu, Jun 2, 2011 at 4:16 PM, Kenneth Russell wrote:
>> > On Thu, Jun 2, 2011 at 12:53 PM, David Levin wrote:
>> > The desire would be for this change to apply not just to the
>> > postMessage method on MessagePort and Worker but also to that on
>> > Window.
>>
>> I agree--the postMessage interfaces shouldn't drift apart more than
>> necessary. Adding another argument to window.postMessage would be
>> unfortunate, though: that brings it up to four, which is hitting the
>> limit of a sane, rememberable API.
>>
>> Alternatively--and this has come up repeatedly of late--allow the
>> final argument to be an object. If it's an object, it looks like
>> this:
>>
>> port.postMessage({frameBuffer: frame}, {transfer: [frame], ports:
>> [port]});
>>
>> where "transfer" and "port" are both optional. If it's an array, it
>> behaves as it does now.
>>
>> This also allows preserving MessagePort error checking: you can still
>> throw INVALID_STATE_ERR if something other than a MessagePort is
>> included in ports.
>
>
> It feels like this array of objects given to transfer may complicate (and
> slow down) both the implementation of this as well as the developer's use of
> it.
How so? You basically use the normal API, but if there are any objects
which you'd like to have transfer the ownership of, you additionally
list them in the second argument.
There are two properties of this approach that I like:
1. It means that objects which you'd like to transfer ownership are
not "second class citizens" and can live as part of the normal object
graph that is posted, together with metadata that goes with it (or
even as metadata for other things).
2. The receiving side doesn't need to worry about the difference, all
it gets is the graph of objects that was sent to it.
> It also raises questions when I see it. When I list an object there does it
> imply that all children are also transfered or do I have to list each of
> them explicitly as well?)
None of the objects which allow transferring of ownership has children
so this doesn't appear to be a problem at this time. If it indeed does
turn into a problem, it would seem like a problem no matter what
solution is used, no?
> Then I wonder what is the use case for this complexity.
> Why not something simpler like this?
> port.postMessage({frameBuffer: frame}, {transfer: true, ports: [port]});
> where you can just say indicate that you want the message transfered.
This means that you have to choose between transferring all arrays and
transferring none of them. It also makes it much less explicit which
objects ends up being mutated.
/ Jonas
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 5:01 PM, David Levin wrote:
> It feels like this array of objects given to transfer may complicate (and
> slow down) both the implementation of this as well as the developer's use of
> it.
Even with thousands of objects, creating an array containing them is
quick (and only needs to be done once), and the implementation would
presumably convert it to a set internally for quick lookups. I doubt
most use cases will transfer so many separate objects, though.
(And Ian keeps drilling into our head that implementation complexity
isn't a major concern, though I don't imagine converting a list of
objects to a hash table is complex.)
In the typical cases, this seems both simple for users and fast.
> It also raises questions when I see it. When I list an object there does it
> imply that all children are also transfered or do I have to list each of
> them explicitly as well?)
Objects with children wouldn't support transfer--you don't transfer a
Javascript array or a basic Object this way.
Note that ImageData's ImagePixelData isn't a "child" as far as
structured clone is concerned; only Array and Object are cloned
recursively. If ImageData gains support for transfer, then you
wouldn't include the ImagePixelData in the list; only the ImageData
itself. (DATA_CLONE_ERR should probably be thrown if an item is
marked for transfer that isn't supported by structured clone, to make
errors related to this more obvious.)
> Then I wonder what is the use case for this complexity.
> Why not something simpler like this?
> port.postMessage({frameBuffer: frame}, {transfer: true, ports: [port]});
> where you can just say indicate that you want the message transfered.
Adding transfer support to more types in the future would become
backwards-incompatible.
--
Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 1:27 PM, Glenn Maynard wrote:
> On Thu, Jun 2, 2011 at 4:16 PM, Kenneth Russell wrote:
> > On Thu, Jun 2, 2011 at 12:53 PM, David Levin wrote:
> > The desire would be for this change to apply not just to the
> > postMessage method on MessagePort and Worker but also to that on
> > Window.
>
> I agree--the postMessage interfaces shouldn't drift apart more than
> necessary. Adding another argument to window.postMessage would be
> unfortunate, though: that brings it up to four, which is hitting the
> limit of a sane, rememberable API.
>
> Alternatively--and this has come up repeatedly of late--allow the
> final argument to be an object. If it's an object, it looks like
> this:
>
> port.postMessage({frameBuffer: frame}, {transfer: [frame], ports: [port]});
>
> where "transfer" and "port" are both optional. If it's an array, it
> behaves as it does now.
>
> This also allows preserving MessagePort error checking: you can still
> throw INVALID_STATE_ERR if something other than a MessagePort is
> included in ports.
>
It feels like this array of objects given to transfer may complicate (and
slow down) both the implementation of this as well as the developer's use of
it.
It also raises questions when I see it. When I list an object there does it
imply that all children are also transfered or do I have to list each of
them explicitly as well?)
Then I wonder what is the use case for this complexity.
Why not something simpler like this?
port.postMessage({frameBuffer: frame}, {transfer: true, ports: [port]});
where you can just say indicate that you want the message transfered.
dave
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 4:16 PM, Kenneth Russell wrote:
> On Thu, Jun 2, 2011 at 12:53 PM, David Levin wrote:
> The desire would be for this change to apply not just to the
> postMessage method on MessagePort and Worker but also to that on
> Window.
I agree--the postMessage interfaces shouldn't drift apart more than
necessary. Adding another argument to window.postMessage would be
unfortunate, though: that brings it up to four, which is hitting the
limit of a sane, rememberable API.
Alternatively--and this has come up repeatedly of late--allow the
final argument to be an object. If it's an object, it looks like
this:
port.postMessage({frameBuffer: frame}, {transfer: [frame], ports: [port]});
where "transfer" and "port" are both optional. If it's an array, it
behaves as it does now.
This also allows preserving MessagePort error checking: you can still
throw INVALID_STATE_ERR if something other than a MessagePort is
included in ports.
--
Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 1:13 PM, Boris Zbarsky wrote: > On 6/2/11 3:53 PM, David Levin wrote: > >> The mechanism: >> >>* needs to have an intuitive feel for developers, >>* must preserve backwards compatibility, >>* should ideally allow the port to function the same regardless of >> whether the message was cloned or transferred. >> > > I'm not sure what you mean by that third item... the obvious meaning, > which is that clone vs transfer is not black-box observable to the code > calling postMessage makes no sense. The receiver of the message is what I meant to say. My edits lost some of the context. dave > There are three ideas for how to accomplish this: >> > > 4. Having separate arguments (in some order) for the ports and the list of > objects to transfer. > > -Boris > >
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 12:53 PM, David Levin wrote: > In summary, there is a desire for a mechanism to transfer objects (to allow > for potentially better perf) across a MessagePort. > The mechanism: > > needs to have an intuitive feel for developers, > must preserve backwards compatibility, > should ideally allow the port to function the same regardless of whether the > message was cloned or transferred. > should be easy to use. > > There are three ideas for how to accomplish this: > 1. Mixing in the list of objects to be cloned with the ports and use that > list to determine what objects in the message should be cloned. > > This allows a lot of flexibility. > It feels odd mixing in a list of objects with the ports when the two have > nothing related. > It also feels complicated having to add objects in two places (the message > and this extra array). Another option which was mentioned: 1a) Add another optional argument to postMessage after the array of ports, which would be an array of objects to transfer rather than clone. I think 1a) is the best way to expose the functionality. As Glenn pointed out, for backward and forward compatibility reasons, it's best if the developer explicitly selects the objects to transfer. The desire would be for this change to apply not just to the postMessage method on MessagePort and Worker but also to that on Window. -Ken > 2. Adding another parameter to postMessage "clone"/"transfer" or true/false, > etc. > > It is less flexible than 1. > It is very simple and easy to use. > It may not be as noticeable when reading the code that this postMessage does > a transfer of items. > > 3. Adding another method transferMessage with the same parameters as > postMessage. > > It is less flexible than 1. > It is very simple and easy to use. > It may be a pain to keep this in sync with postMessage. > It should be very noticeable when reading code. > > What do you think is the best way to expose this to web developers? > dave > >
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On 6/2/11 3:53 PM, David Levin wrote: The mechanism: * needs to have an intuitive feel for developers, * must preserve backwards compatibility, * should ideally allow the port to function the same regardless of whether the message was cloned or transferred. I'm not sure what you mean by that third item... the obvious meaning, which is that clone vs transfer is not black-box observable to the code calling postMessage makes no sense. There are three ideas for how to accomplish this: 4. Having separate arguments (in some order) for the ports and the list of objects to transfer. -Boris
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
In summary, there is a desire for a mechanism to transfer objects (to allow for potentially better perf) across a MessagePort. The mechanism: - needs to have an intuitive feel for developers, - must preserve backwards compatibility, - should ideally allow the port to function the same regardless of whether the message was cloned or transferred. - should be easy to use. There are three ideas for how to accomplish this: 1. Mixing in the list of objects to be cloned with the ports and use that list to determine what objects in the message should be cloned. This allows a lot of flexibility. It feels odd mixing in a list of objects with the ports when the two have nothing related. It also feels complicated having to add objects in two places (the message and this extra array). 2. Adding another parameter to postMessage "clone"/"transfer" or true/false, etc. It is less flexible than 1. It is very simple and easy to use. It may not be as noticeable when reading the code that this postMessage does a transfer of items. 3. Adding another method transferMessage with the same parameters as postMessage. It is less flexible than 1. It is very simple and easy to use. It may be a pain to keep this in sync with postMessage. It should be very noticeable when reading code. What do you think is the best way to expose this to web developers? dave
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 11:54 AM, Ian Hickson wrote:
> On Thu, 2 Jun 2011, ben turner wrote:
>>
>> I interpreted the proposal differently... This is what I envisioned:
>>
>> var bufferToTransfer = /* make ArrayBuffer */;
>> var bufferToCopy = /* make ArrayBuffer */;
>> var worker = /* make Worker */;
>> var message = { buf1: bufferToTransfer, buf2: bufferToCopy };
>> worker.postMessage(message, [bufferToTransfer]);
>>
>> We'd keep the structure exactly the same, it's just that when we clone
>> 'message' we compare each object value to one of the ones passed in the
>> transfer array. If the objects match (===) then we'd transfer it rather
>> than copy it.
>>
>> Does that sound like what you were hoping for?
>
> That's fine except for breaking compatibility with the current MessagePort
> model, which is a pretty big problem.
I'll note that Firefox does not support MessagePorts or the ports
argument, so it wouldn't be an backwards incompatible change for us.
I do seem to recall the Chrome does support it though so google would
have to chime in how happy they are about breaking compat here.
I don't know what the status is for Opera and Safari.
/ Jonas
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, 2 Jun 2011, ben turner wrote:
>
> I interpreted the proposal differently... This is what I envisioned:
>
> var bufferToTransfer = /* make ArrayBuffer */;
> var bufferToCopy = /* make ArrayBuffer */;
> var worker = /* make Worker */;
> var message = { buf1: bufferToTransfer, buf2: bufferToCopy };
> worker.postMessage(message, [bufferToTransfer]);
>
> We'd keep the structure exactly the same, it's just that when we clone
> 'message' we compare each object value to one of the ones passed in the
> transfer array. If the objects match (===) then we'd transfer it rather
> than copy it.
>
> Does that sound like what you were hoping for?
That's fine except for breaking compatibility with the current MessagePort
model, which is a pretty big problem.
On Thu, 2 Jun 2011, Jonas Sicking wrote:
>
> I also don't understand the reason for the requirement to have the type
> participate in some specific inheritance chain. All that seems needed to
> me is to enumerate the types which can be transferred rather than copied
> as obviously transferring normal JS arrays or JS objects is unlikely to
> be possible in todays JS implementations.
The idea here was just to have WebIDL do the type-checking instead of me.
The behaviour is black-box indistinguishable.
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 10:22 AM, ben turner wrote:
> On Thu, Jun 2, 2011 at 9:58 AM, Travis Leithead
> wrote:
>>
>> This plan requires all objects that want to opt-in to a new
>> transfer-of-ownership (or really
>> any special custom behavior for postMessage) to 1) participate in the
>> special inheritance
>> interface and 2) be isolated from the primary object graph being passed to
>> the first
>> parameter.
>
> I interpreted the proposal differently... This is what I envisioned:
>
> var bufferToTransfer = /* make ArrayBuffer */;
> var bufferToCopy = /* make ArrayBuffer */;
> var worker = /* make Worker */;
> var message = { buf1: bufferToTransfer, buf2: bufferToCopy };
> worker.postMessage(message, [bufferToTransfer]);
>
> We'd keep the structure exactly the same, it's just that when we clone
> 'message' we compare each object value to one of the ones passed in
> the transfer array. If the objects match (===) then we'd transfer it
> rather than copy it.
>
> Does that sound like what you were hoping for?
This would work for me. It's also a better design for MessagePorts IMO.
I also don't understand the reason for the requirement to have the
type participate in some specific inheritance chain. All that seems
needed to me is to enumerate the types which can be transferred rather
than copied as obviously transferring normal JS arrays or JS objects
is unlikely to be possible in todays JS implementations.
/ Jonas
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 12:58 PM, Travis Leithead
wrote:
> I'm a little concerned about the inherit approach that Ian outlines...
>
> This plan requires all objects that want to opt-in to a new
> transfer-of-ownership (or really any special custom behavior for postMessage)
> to 1) participate in the special inheritance interface and 2) be isolated
> from the primary object graph being passed to the first parameter. For
> example, instead of allowing the structured clone algorithm to preserve all
> my object relationships as-is, I will have to detach some objects from the
> graph (to include in the proposed 2nd parameter to postMessage) and then
> re-attach them on the destination side.
I interpreted (on only a quick reading) his proposal as saying that
the second list is a set of objects which, if included in the first
list, will be transferred rather than cloned. I'd agree that it
doesn't make sense to have a separate, isolated list to transfer;
that's cumbersome.
I assume the proposal was a bit different than I read it, since that's
not how MessagePorts work. I'd imagine it wouldn't be hard for the
list to do both: filter all objects that aren't MessagePorts, use that
set to flag transfers, and the remaining MessagePorts act as they do
now. Sort of ugly, though.
> Isn't there some way we can flag the objects that want transfer of ownership
> by a property or identifier on the objects themselves? That way, they won't
> have to be special-cased into a separate list. For example, if the
> ArrayBuffer added a property indicating transfer-of-ownership? Or a new
> configuration option for postMessage ( { transferOwnership: true } )?
It should be possible to request transfer ownership of some objects in
the tree and not others. Otherwise, adding this feature to other
objects later won't be backwards-compatible, since a tree may contain
transferrable and non-transferrable objects.
--
Glenn Maynard
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
On Thu, Jun 2, 2011 at 9:58 AM, Travis Leithead
wrote:
>
> This plan requires all objects that want to opt-in to a new
> transfer-of-ownership (or really
> any special custom behavior for postMessage) to 1) participate in the special
> inheritance
> interface and 2) be isolated from the primary object graph being passed to
> the first
> parameter.
I interpreted the proposal differently... This is what I envisioned:
var bufferToTransfer = /* make ArrayBuffer */;
var bufferToCopy = /* make ArrayBuffer */;
var worker = /* make Worker */;
var message = { buf1: bufferToTransfer, buf2: bufferToCopy };
worker.postMessage(message, [bufferToTransfer]);
We'd keep the structure exactly the same, it's just that when we clone
'message' we compare each object value to one of the ones passed in
the transfer array. If the objects match (===) then we'd transfer it
rather than copy it.
Does that sound like what you were hoping for?
-Ben Turner
Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
(It would have been better not to fork the thread with a different
subject line...)
On Thu, Jun 2, 2011 at 9:58 AM, Travis Leithead
wrote:
> I'm a little concerned about the inherit approach that Ian outlines...
>
> This plan requires all objects that want to opt-in to a new
> transfer-of-ownership (or really any special custom behavior for postMessage)
> to 1) participate in the special inheritance interface and 2) be isolated
> from the primary object graph being passed to the first parameter. For
> example, instead of allowing the structured clone algorithm to preserve all
> my object relationships as-is, I will have to detach some objects from the
> graph (to include in the proposed 2nd parameter to postMessage) and then
> re-attach them on the destination side.
I realize I misunderstood the proposal. I thought the additional array
of objects was supposed to indicate those in the object graph that
wanted ownership to be transferred. Having to send these objects
separately from the rest of those in the graph is undesirable and in
my opinion unworkable.
> Isn't there some way we can flag the objects that want transfer of ownership
> by a property or identifier on the objects themselves? That way, they won't
> have to be special-cased into a separate list. For example, if the
> ArrayBuffer added a property indicating transfer-of-ownership? Or a new
> configuration option for postMessage ( { transferOwnership: true } )?
A per-object property is undesirable because it forces all types that
want to participate in transfer of ownership to add a new property
specifically for the Web Messaging spec.
I still like the idea of identifying those objects in the graph passed
to postMessage which should be transferred rather than copied, since
it's flexible and should be easy to use. (Though maybe not, if the
object graph is deep and it's difficult to pull out all the objects
you want to transfer?) Adding a flag to postMessage, or adding a new
entry point like transferMessage, also seem fine.
-Ken
> -Original Message-
> From: Arthur Barstow [mailto:art.bars...@nokia.com]
> Sent: Thursday, June 02, 2011 9:02 AM
> To: ext Jonas Sicking; Kenneth Russell; Ian Hickson
> Cc: Travis Leithead; g...@google.com; cmar...@apple.com; gl...@zewt.org;
> public-webapps@w3.org
> Subject: What changes to Web Messaging spec are proposed? [Was: Re: Using
> ArrayBuffer as payload for binary data to/from Web Workers]
>
> What are the specific change(s) to the Web Messaging spec being proposed:
>
> http://dev.w3.org/html5/postmsg/
>
> -AB
>
> On Jun/2/2011 11:25 AM, ext Jonas Sicking wrote:
>> On Wed, Jun 1, 2011 at 4:55 PM, Kenneth Russell wrote:
>>> On Tue, May 31, 2011 at 3:35 PM, Ian Hickson wrote:
On Tue, 31 May 2011, Kenneth Russell wrote:
> Jonas's suggestion of adding another argument to postMessage, and
> Gregg's generalization to declare it as an array of objects to be
> transferred rather than copied, sounds good.
We could change make MessagePort and ArrayBuffer both inherit from a
[NoInterfaceObject] empty interface, and then make the MessagePort[]
argument of the various postMessage() methods instead take an array
of this new interface, and then just have ArrayBuffer and
MessagePort both define how to be cloned in this way.
If people like this approach I can work with Kenneth on getting the
wording right in the various specs.
>>> This sounds good to me; in the interest of moving things forward, are
>>> there any objections?
>> No, this sounded good to the people here at mozilla that I talked with
>> about this.
>>
>> / Jonas
>>
>
>
RE: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]
I'm a little concerned about the inherit approach that Ian outlines...
This plan requires all objects that want to opt-in to a new
transfer-of-ownership (or really any special custom behavior for postMessage)
to 1) participate in the special inheritance interface and 2) be isolated from
the primary object graph being passed to the first parameter. For example,
instead of allowing the structured clone algorithm to preserve all my object
relationships as-is, I will have to detach some objects from the graph (to
include in the proposed 2nd parameter to postMessage) and then re-attach them
on the destination side.
Isn't there some way we can flag the objects that want transfer of ownership by
a property or identifier on the objects themselves? That way, they won't have
to be special-cased into a separate list. For example, if the ArrayBuffer added
a property indicating transfer-of-ownership? Or a new configuration option for
postMessage ( { transferOwnership: true } )?
-Original Message-
From: Arthur Barstow [mailto:art.bars...@nokia.com]
Sent: Thursday, June 02, 2011 9:02 AM
To: ext Jonas Sicking; Kenneth Russell; Ian Hickson
Cc: Travis Leithead; g...@google.com; cmar...@apple.com; gl...@zewt.org;
public-webapps@w3.org
Subject: What changes to Web Messaging spec are proposed? [Was: Re: Using
ArrayBuffer as payload for binary data to/from Web Workers]
What are the specific change(s) to the Web Messaging spec being proposed:
http://dev.w3.org/html5/postmsg/
-AB
On Jun/2/2011 11:25 AM, ext Jonas Sicking wrote:
> On Wed, Jun 1, 2011 at 4:55 PM, Kenneth Russell wrote:
>> On Tue, May 31, 2011 at 3:35 PM, Ian Hickson wrote:
>>> On Tue, 31 May 2011, Kenneth Russell wrote:
Jonas's suggestion of adding another argument to postMessage, and
Gregg's generalization to declare it as an array of objects to be
transferred rather than copied, sounds good.
>>> We could change make MessagePort and ArrayBuffer both inherit from a
>>> [NoInterfaceObject] empty interface, and then make the MessagePort[]
>>> argument of the various postMessage() methods instead take an array
>>> of this new interface, and then just have ArrayBuffer and
>>> MessagePort both define how to be cloned in this way.
>>>
>>> If people like this approach I can work with Kenneth on getting the
>>> wording right in the various specs.
>> This sounds good to me; in the interest of moving things forward, are
>> there any objections?
> No, this sounded good to the people here at mozilla that I talked with
> about this.
>
> / Jonas
>
