Re: [whatwg] ConnectionPeer experiences
(Patrik has a couple of well deserved days off):
> - In your experimentation, did you find any reasonable underlying
> protocol to map sendFile, sendBitmap and their corresponding
> callbacks
> to, or did you just ignore them for now?
We just ignored them for the time being
> - In connecting, did you operate with connections going via a
> server, or
> did you go between browsers? If so, how did you identify the
> server vs
> identifying the remote participant? Was the
> "remoteConfiguration" method
> flexible enough?
The connection that is set up using ConnectionPeer is browser-to-browser in our
case (there is no need to use TURN in our setup). So what we do supply as
remoteConfiguration (note our proposal to change "addRemoteConfiguration" to
"setRemoteConfiguration") is basically the candidates obtained in contact with
the STUN server.
add/setRemoteConfiguration seems flexible enough (we have only experimented
with a simple case though). Of course the data supplied (acquired from
"getLocalConfig" at the remote sida) must be able to express all required
possibilities.
//Stefan
>
> Message: 4
> Date: Fri, 28 Jan 2011 16:14:52 -0800
> From: Harald Alvestrand
> To: whatwg@lists.whatwg.org
> Subject: Re: [whatwg] ConnectionPeer experiences
> Message-ID: <4d435bfc.7040...@alvestrand.no>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Thank you Patrik, I enjoyed reading that!
>
> Questions:
> - In your experimentation, did you find any reasonable underlying
> protocol to map sendFile, sendBitmap and their corresponding
> callbacks
> to, or did you just ignore them for now?
> - In connecting, did you operate with connections going via a
> server, or
> did you go between browsers? If so, how did you identify the
> server vs
> identifying the remote participant? Was the
> "remoteConfiguration" method
> flexible enough?
>
> I'm currently staring at the problem of defining a set of
> semantics for
> mapping this to RTC-Web protocols, and am having some problems
> interpreting what's currently in the spec, so would like your
> interpretation.
>
> Harald
>
> On 01/26/11 01:04, Patrik Persson J wrote:
> > We have done some experimentation with the ConnectionPeer
> API. We have
> > an initial implementation of a subset of the API, using ICE
> (RFC 5245)
> > for the peer-to-peer handshaking. Our implementation is
> > WebKit/GTK+/gstreamer-based, and we of course intend to submit it to
> > WebKit, but the implementation is not quite ready for that yet.
> >
> > More information about our work so far can be found here:
> >
> https://labs.ericsson.com/developer-community/blog/beyond-html
5-peer-peer-conversational-video
> >
> > However, we have bumped into some details that we'd like to discuss
> > here right away. The following is our mix of proposals and
> questions.
> >
> > 1. We propose adding a readyState attribute, to decouple the
> > onconnect() callback from any observers (such as the UI).
> >
> >const unsigned short CONNECTING = 0;
> >const unsigned short CONNECTED = 1;
> >const unsigned short CLOSED = 2;
> >readonly attribute unsigned short readyState;
> >
> > 2. We propose replacing the onstream event with custom
> events of type
> > RemoteStreamEvent, to distinguish between adding and removing
> > streams.
> >
> >attribute Function onstreamadded; // RemoteStreamEvent
> >attribute Function onstreamremoved; // RemoteStreamEvent
> >...
> >interface RemoteStreamEvent : Event {
> > readonly attribute Stream stream;
> >};
> >
> > The 'stream' attribute indicates which stream was added/removed.
> >
> > 3. We propose renaming addRemoteConfiguration to
> > setRemoteConfiguration. Our understanding of the
> ConnectionPeer is
> > that it provides a single-point-to-single-point
> connection; hence,
> > only one remote peer configuration is to be set, rather
> than many
> > to be added.
> >
> >void setRemoteConfiguration(in DOMString
> configuration, in optional DOMString remoteOrigin);
> >
> > 4. We propose swapping the ConnectionPeerConfigurationCallback
> > callback parameters. The current example seems to use
> only one (the
> > second one). Swapping them allows clients that care
> about 'server'
> > to do so, and clients that ignore it (such as the
> current examp
Re: [whatwg] ConnectionPeer experiences
Thank you Patrik, I enjoyed reading that!
Questions:
- In your experimentation, did you find any reasonable underlying
protocol to map sendFile, sendBitmap and their corresponding callbacks
to, or did you just ignore them for now?
- In connecting, did you operate with connections going via a server, or
did you go between browsers? If so, how did you identify the server vs
identifying the remote participant? Was the "remoteConfiguration" method
flexible enough?
I'm currently staring at the problem of defining a set of semantics for
mapping this to RTC-Web protocols, and am having some problems
interpreting what's currently in the spec, so would like your
interpretation.
Harald
On 01/26/11 01:04, Patrik Persson J wrote:
We have done some experimentation with the ConnectionPeer API. We have
an initial implementation of a subset of the API, using ICE (RFC 5245)
for the peer-to-peer handshaking. Our implementation is
WebKit/GTK+/gstreamer-based, and we of course intend to submit it to
WebKit, but the implementation is not quite ready for that yet.
More information about our work so far can be found here:
https://labs.ericsson.com/developer-community/blog/beyond-html5-peer-peer-conversational-video
However, we have bumped into some details that we'd like to discuss
here right away. The following is our mix of proposals and questions.
1. We propose adding a readyState attribute, to decouple the
onconnect() callback from any observers (such as the UI).
const unsigned short CONNECTING = 0;
const unsigned short CONNECTED = 1;
const unsigned short CLOSED = 2;
readonly attribute unsigned short readyState;
2. We propose replacing the onstream event with custom events of type
RemoteStreamEvent, to distinguish between adding and removing
streams.
attribute Function onstreamadded; // RemoteStreamEvent
attribute Function onstreamremoved; // RemoteStreamEvent
...
interface RemoteStreamEvent : Event {
readonly attribute Stream stream;
};
The 'stream' attribute indicates which stream was added/removed.
3. We propose renaming addRemoteConfiguration to
setRemoteConfiguration. Our understanding of the ConnectionPeer is
that it provides a single-point-to-single-point connection; hence,
only one remote peer configuration is to be set, rather than many
to be added.
void setRemoteConfiguration(in DOMString configuration, in optional
DOMString remoteOrigin);
4. We propose swapping the ConnectionPeerConfigurationCallback
callback parameters. The current example seems to use only one (the
second one). Swapping them allows clients that care about 'server'
to do so, and clients that ignore it (such as the current example)
to do so too.
[Callback=FunctionOnly, NoInterfaceObject]
interface ConnectionPeerConfigurationCallback {
void handleEvent(in DOMString configuration, in ConnectionPeer
server);
};
5. Should a size limit to text messages be specified? Text messages
with UDP-like behavior (unimportant=true) can't really be reliably
split into several UDP packets. For such long chunks of data, file
transfer seems like a better option anyway.
In summary, then, our proposal for a revised ConnectionPeer looks as follows:
[Constructor(in DOMString serverConfiguration)]
interface ConnectionPeer {
void sendText(in DOMString text, in optional boolean unimportant); // if
second arg is true, then use unreliable low-latency transport (UDP-like),
otherwise guarantee delivery (TCP-like)
attribute Function ontext; // receiving
void sendBitmap(in HTMLImageElement image);
attribute Function onbitmap; // receiving
void sendFile(in File file);
attribute Function onfile; // receiving
void addStream(in Stream stream, in optional DOMString metadata, in
optional String mediaFormat);
//Start stream, add meta data and encoding
parameters
void removeStream(in Stream stream);
readonly attribute Stream[] localStreams;
readonly attribute Stream[] remoteStreams;
attribute Function onstreamadded; // receiving new stream
attribute Function onstreamremoved; // stream not received any more
void getLocalConfiguration(in ConnectionPeerConfigurationCallback
callback); // maybe this should be in the constructor, or be an event
void setRemoteConfiguration(in DOMString configuration, in optional DOMString remoteOrigin);
// remote origin is assumed to be same-origin if not specified. If specified, has to match remote
origin (checked in handshake). Should support leading "*." to mean "any subdomain
of".
void close(); // disconnects and stops listening
attribute Function onconnect;
attribute Function onerror;
attribute Function ondisconnect;
const unsigned short CONNECT
Re: [whatwg] ConnectionPeer experiences
That is a very good point, and the name "add..." made me, too, suspect that ConnectionPeer could potentially be applied to multicast scenarios. However, it seems to me that this would quickly get rather complex. A single peer that connects to multiple others would still require the implementation to combine multiple addRemoteConfiguration() calls into a multicast configuration or something similar. I'm not quite sure how that would work, but if it can be done, it can also be done with multiple one-to-one ConnectionPeers. The alternative would be to use one ConnectionPeer for each peer you're connected to. I think this is more straight-forward from an API point of view. I also don't think this would restrict the potential for efficient multicast (for reasons outlined ebove). Feel free to set me straight. Perhaps question 3 in my original message could be read a follows: Is the ConnectionPeer intended to be used in a one-to-many setting (using a single ConnectionPeer in the 'one' end)? If so, how would that work? Cheers, > -Original Message- > From: whatwg-boun...@lists.whatwg.org > [mailto:whatwg-boun...@lists.whatwg.org] On Behalf Of Adam > Malcontenti-Wilson > Sent: Wednesday, January 26, 2011 2:41 PM > To: whatwg@lists.whatwg.org > Subject: Re: [whatwg] ConnectionPeer experiences > > Hi, > I was noticing how you were suggesting to change > addRemoteConfiguration to setRemoteConfiguration as it > appears as a single-point-to-single-point connection, is this > part of the current specification or could > single-point-to-multiple-points connections (or > "clouds") be implemented using the same API in the future? > This would be a big bandwidth saver for users in "group > chats" that would make some sense to use add rather than set > (and perhaps have another optional parameter to replace > rather than append or add). > > Thanks and sorry for butting in, > -- Adam M-W > > On Wed, Jan 26, 2011 at 9:09 PM, Patrik Persson J > wrote: > > Oh, thanks! > > > > Your question is a good one. We're looking into precisely > that, and expect to learn more in our continued implementation work. > > > > We're also participating in the discussion on these matters > in RTC-Web [http://rtc-web.alvestrand.com/]. > > > > -- Patrik > > > > -Original Message- > > From: Anne van Kesteren [mailto:ann...@opera.com] > > Sent: Wednesday, January 26, 2011 10:38 AM > > To: 'whatwg@lists.whatwg.org'; Patrik Persson J > > Subject: Re: [whatwg] ConnectionPeer experiences > > > > On Wed, 26 Jan 2011 10:34:11 +0100, Anne van Kesteren > > > > wrote: > >> Could you elaborate more on the details of the protocol? > >> Handshake/framing/etc. Is it something that can be independently > >> implemented? Agreement on that is what is holding back this API at > >> the moment. > > > > Oh, and also, extremely awesome! :-) > > > > > > -- > > Anne van Kesteren > > http://annevankesteren.nl/ > > > > > > -- > Adam Malcontenti-Wilson >
Re: [whatwg] ConnectionPeer experiences
Hi, I was noticing how you were suggesting to change addRemoteConfiguration to setRemoteConfiguration as it appears as a single-point-to-single-point connection, is this part of the current specification or could single-point-to-multiple-points connections (or "clouds") be implemented using the same API in the future? This would be a big bandwidth saver for users in "group chats" that would make some sense to use add rather than set (and perhaps have another optional parameter to replace rather than append or add). Thanks and sorry for butting in, -- Adam M-W On Wed, Jan 26, 2011 at 9:09 PM, Patrik Persson J wrote: > Oh, thanks! > > Your question is a good one. We're looking into precisely that, and expect > to learn more in our continued implementation work. > > We're also participating in the discussion on these matters in RTC-Web > [http://rtc-web.alvestrand.com/]. > > -- Patrik > > -Original Message- > From: Anne van Kesteren [mailto:ann...@opera.com] > Sent: Wednesday, January 26, 2011 10:38 AM > To: 'whatwg@lists.whatwg.org'; Patrik Persson J > Subject: Re: [whatwg] ConnectionPeer experiences > > On Wed, 26 Jan 2011 10:34:11 +0100, Anne van Kesteren > wrote: >> Could you elaborate more on the details of the protocol? >> Handshake/framing/etc. Is it something that can be independently >> implemented? Agreement on that is what is holding back this API at the >> moment. > > Oh, and also, extremely awesome! :-) > > > -- > Anne van Kesteren > http://annevankesteren.nl/ > -- Adam Malcontenti-Wilson
Re: [whatwg] ConnectionPeer experiences
Oh, thanks! Your question is a good one. We're looking into precisely that, and expect to learn more in our continued implementation work. We're also participating in the discussion on these matters in RTC-Web [http://rtc-web.alvestrand.com/]. -- Patrik -Original Message- From: Anne van Kesteren [mailto:ann...@opera.com] Sent: Wednesday, January 26, 2011 10:38 AM To: 'whatwg@lists.whatwg.org'; Patrik Persson J Subject: Re: [whatwg] ConnectionPeer experiences On Wed, 26 Jan 2011 10:34:11 +0100, Anne van Kesteren wrote: > Could you elaborate more on the details of the protocol? > Handshake/framing/etc. Is it something that can be independently > implemented? Agreement on that is what is holding back this API at the > moment. Oh, and also, extremely awesome! :-) -- Anne van Kesteren http://annevankesteren.nl/
Re: [whatwg] ConnectionPeer experiences
On Wed, 26 Jan 2011 10:34:11 +0100, Anne van Kesteren wrote: Could you elaborate more on the details of the protocol? Handshake/framing/etc. Is it something that can be independently implemented? Agreement on that is what is holding back this API at the moment. Oh, and also, extremely awesome! :-) -- Anne van Kesteren http://annevankesteren.nl/
Re: [whatwg] ConnectionPeer experiences
On Wed, 26 Jan 2011 10:04:22 +0100, Patrik Persson J wrote: We have done some experimentation with the ConnectionPeer API. We have an initial implementation of a subset of the API, using ICE (RFC 5245) for the peer-to-peer handshaking. Our implementation is WebKit/GTK+/gstreamer-based, and we of course intend to submit it to WebKit, but the implementation is not quite ready for that yet. More information about our work so far can be found here: https://labs.ericsson.com/developer-community/blog/beyond-html5-peer-peer-conversational-video Could you elaborate more on the details of the protocol? Handshake/framing/etc. Is it something that can be independently implemented? Agreement on that is what is holding back this API at the moment. -- Anne van Kesteren http://annevankesteren.nl/
[whatwg] ConnectionPeer experiences
We have done some experimentation with the ConnectionPeer API. We have
an initial implementation of a subset of the API, using ICE (RFC 5245)
for the peer-to-peer handshaking. Our implementation is
WebKit/GTK+/gstreamer-based, and we of course intend to submit it to
WebKit, but the implementation is not quite ready for that yet.
More information about our work so far can be found here:
https://labs.ericsson.com/developer-community/blog/beyond-html5-peer-peer-conversational-video
However, we have bumped into some details that we'd like to discuss
here right away. The following is our mix of proposals and questions.
1. We propose adding a readyState attribute, to decouple the
onconnect() callback from any observers (such as the UI).
const unsigned short CONNECTING = 0;
const unsigned short CONNECTED = 1;
const unsigned short CLOSED = 2;
readonly attribute unsigned short readyState;
2. We propose replacing the onstream event with custom events of type
RemoteStreamEvent, to distinguish between adding and removing
streams.
attribute Function onstreamadded; // RemoteStreamEvent
attribute Function onstreamremoved; // RemoteStreamEvent
...
interface RemoteStreamEvent : Event {
readonly attribute Stream stream;
};
The 'stream' attribute indicates which stream was added/removed.
3. We propose renaming addRemoteConfiguration to
setRemoteConfiguration. Our understanding of the ConnectionPeer is
that it provides a single-point-to-single-point connection; hence,
only one remote peer configuration is to be set, rather than many
to be added.
void setRemoteConfiguration(in DOMString configuration, in optional
DOMString remoteOrigin);
4. We propose swapping the ConnectionPeerConfigurationCallback
callback parameters. The current example seems to use only one (the
second one). Swapping them allows clients that care about 'server'
to do so, and clients that ignore it (such as the current example)
to do so too.
[Callback=FunctionOnly, NoInterfaceObject]
interface ConnectionPeerConfigurationCallback {
void handleEvent(in DOMString configuration, in ConnectionPeer server);
};
5. Should a size limit to text messages be specified? Text messages
with UDP-like behavior (unimportant=true) can't really be reliably
split into several UDP packets. For such long chunks of data, file
transfer seems like a better option anyway.
In summary, then, our proposal for a revised ConnectionPeer looks as follows:
[Constructor(in DOMString serverConfiguration)]
interface ConnectionPeer {
void sendText(in DOMString text, in optional boolean unimportant); // if
second arg is true, then use unreliable low-latency transport (UDP-like),
otherwise guarantee delivery (TCP-like)
attribute Function ontext; // receiving
void sendBitmap(in HTMLImageElement image);
attribute Function onbitmap; // receiving
void sendFile(in File file);
attribute Function onfile; // receiving
void addStream(in Stream stream, in optional DOMString metadata, in
optional String mediaFormat);
//Start stream, add meta data and encoding
parameters
void removeStream(in Stream stream);
readonly attribute Stream[] localStreams;
readonly attribute Stream[] remoteStreams;
attribute Function onstreamadded; // receiving new stream
attribute Function onstreamremoved; // stream not received any more
void getLocalConfiguration(in ConnectionPeerConfigurationCallback
callback); // maybe this should be in the constructor, or be an event
void setRemoteConfiguration(in DOMString configuration, in optional
DOMString remoteOrigin); // remote origin is assumed to be same-origin if not
specified. If specified, has to match remote origin (checked in handshake).
Should support leading "*." to mean "any subdomain of".
void close(); // disconnects and stops listening
attribute Function onconnect;
attribute Function onerror;
attribute Function ondisconnect;
const unsigned short CONNECTING = 0;
const unsigned short CONNECTED = 1;
const unsigned short CLOSED = 2;
readonly attribute unsigned short readyState;
};
interface RemoteStreamEvent : Event {
readonly attribute Stream stream;
};
[Callback=FunctionOnly, NoInterfaceObject]
interface ConnectionPeerConfigurationCallback {
void handleEvent(in DOMString configuration, in ConnectionPeer server);
};
What do you think?
In addition to the above there is a need to add support for
identifying streams (so that the receiving end can use the right
element for rendering) and for influencing the media format. Those
parts we're still working on.
--
Patrik Persson, Ericsson Research
mailto:patrik.j.pers...@ericsson.com
