Re: Extending createObjectUrl to MediaStream?
On Tue, Sep 3, 2013 at 12:29 AM, Robert O'Callahan rob...@ocallahan.org wrote: What are your use-cases where they're not the same? More importantly, what are the use-cases where they cannot be made the same by the developer? E.g. embedding a widget for video or audio manipulation. The widget could be written by a third-party of sorts. We're lifting the origin restriction on Blob too, or at least, I believe that's still the plan. If you want to transfer the object to this other global it seems fine to do so since you could also transfer the data in some other way. -- http://annevankesteren.nl/
Re: Extending createObjectUrl to MediaStream?
On 2013-09-03 01:29, Robert O'Callahan wrote: What are your use-cases where they're not the same? More importantly, what are the use-cases where they cannot be made the same by the developer? E.g. the main page delegating communication to someone else (perhaps via an iFrame). If the MediaStream is a transferable object the incoming MediaStream can be transferred to the main page, which in turn can control layout and rendering. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: Extending createObjectUrl to MediaStream?
On 09/03/2013 10:27 AM, Stefan Håkansson LK wrote:
On 2013-09-03 01:29, Robert O'Callahan wrote:
What are your use-cases where they're not the same? More importantly,
what are the use-cases where they cannot be made the same by the developer?
E.g. the main page delegating communication to someone else (perhaps via
an iFrame). If the MediaStream is a transferable object the incoming
MediaStream can be transferred to the main page, which in turn can
control layout and rendering.
One interesting thing about a MediaStream is that it's got multiple
links to the world around it. In particular, it can be connected to
multiple downstream objects.
This means that what happens to its peers when one transfers it has to
be defined.
For instance, if MediaStream were transferable, could I do this?
GetUserMedia(., success, failure)
function success(stream) {
myVideoTag.srcObject = stream
stream.transfer(myWorker)
}
and after this - would the video stop playing (because the stream is
gone from my context), or not?
Re: Extending createObjectUrl to MediaStream?
On 2013-09-03 12:01, Harald Alvestrand wrote:
On 09/03/2013 10:27 AM, Stefan Håkansson LK wrote:
On 2013-09-03 01:29, Robert O'Callahan wrote:
What are your use-cases where they're not the same? More importantly,
what are the use-cases where they cannot be made the same by the developer?
E.g. the main page delegating communication to someone else (perhaps via
an iFrame). If the MediaStream is a transferable object the incoming
MediaStream can be transferred to the main page, which in turn can
control layout and rendering.
One interesting thing about a MediaStream is that it's got multiple
links to the world around it. In particular, it can be connected to
multiple downstream objects.
This means that what happens to its peers when one transfers it has to
be defined.
For instance, if MediaStream were transferable, could I do this?
GetUserMedia(., success, failure)
function success(stream) {
myVideoTag.srcObject = stream
stream.transfer(myWorker)
}
and after this - would the video stop playing (because the stream is
gone from my context), or not?
I think it should stop playing since the object srcObject references is
gone. (It would work differently with createObjecURL + myVideoTag.src
since that would only be a handle to an underlying resource)
If you'd like it to continue playing you'd have to clone the MediaStream
and transfer the clone.
Re: Extending createObjectUrl to MediaStream?
On Tue, Sep 3, 2013 at 12:48 PM, Stefan Håkansson LK stefan.lk.hakans...@ericsson.com wrote: I think it should stop playing since the object srcObject references is gone. (It would work differently with createObjecURL + myVideoTag.src since that would only be a handle to an underlying resource) The object is not gone, it's neutered. That could mean e.g. that instead you'd get black and no sound. Kinda depends on how you define a neutered MediaStream to work. -- http://annevankesteren.nl/
Re: Extending createObjectUrl to MediaStream?
On 2013-09-03 14:02, Anne van Kesteren wrote: On Tue, Sep 3, 2013 at 12:48 PM, Stefan Håkansson LK stefan.lk.hakans...@ericsson.com wrote: I think it should stop playing since the object srcObject references is gone. (It would work differently with createObjecURL + myVideoTag.src since that would only be a handle to an underlying resource) The object is not gone, it's neutered. That could mean e.g. that instead you'd get black and no sound. Kinda depends on how you define a neutered MediaStream to work. Right. I should've known. Thanks.
Re: Extending createObjectUrl to MediaStream?
On Tue, Sep 3, 2013 at 6:31 PM, Anne van Kesteren ann...@annevk.nl wrote: On Tue, Sep 3, 2013 at 12:29 AM, Robert O'Callahan rob...@ocallahan.org wrote: What are your use-cases where they're not the same? More importantly, what are the use-cases where they cannot be made the same by the developer? E.g. embedding a widget for video or audio manipulation. The widget could be written by a third-party of sorts. The widget would not only have to be written by a third party, but actually hosted on their domain. And not just optionally, but for some reason the widget provider has decided not to allow the author to host it on their own domain. Sure, it could happen, but it seems somewhat far-fetched to me. On the other hand, allowing MediaStream graphs to span domains could have potentially far-reaching consequences. I don't see any need to rush into this. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: Extending createObjectUrl to MediaStream?
On Tue, Sep 3, 2013 at 1:25 PM, Robert O'Callahan rob...@ocallahan.org wrote: The widget would not only have to be written by a third party, but actually hosted on their domain. And not just optionally, but for some reason the widget provider has decided not to allow the author to host it on their own domain. That's fairly common though, consider eg YouTube. Sure, it could happen, but it seems somewhat far-fetched to me. On the other hand, allowing MediaStream graphs to span domains could have potentially far-reaching consequences. I don't see any need to rush into this. It's not entirely clear to me what you mean by this. Are these concerns specific to MediaStream and not applicable to Blob? -- http://annevankesteren.nl/
Re: Extending createObjectUrl to MediaStream?
On Wed, Sep 4, 2013 at 12:31 AM, Anne van Kesteren ann...@annevk.nl wrote: On Tue, Sep 3, 2013 at 1:25 PM, Robert O'Callahan rob...@ocallahan.org wrote: The widget would not only have to be written by a third party, but actually hosted on their domain. And not just optionally, but for some reason the widget provider has decided not to allow the author to host it on their own domain. That's fairly common though, consider eg YouTube. Youtube hosts embedded players because they're playing Youtube's content, not because Youtube's widget is particularly special. If you just want an HTML5 video player app to wrap around your own content, you don't use Youtube and you host it yourself. Sure, it could happen, but it seems somewhat far-fetched to me. On the other hand, allowing MediaStream graphs to span domains could have potentially far-reaching consequences. I don't see any need to rush into this. It's not entirely clear to me what you mean by this. Are these concerns specific to MediaStream and not applicable to Blob? Yes. For example there are plans to enable some kind of private mode for WebRTC MediaStreams that protects stream contents from inspection by the page. I don't know exactly how this is going to work, but if we allow MediaStreams to span domains it may get more complicated. More concretely, in Gecko we have experimental code to pipe HTML media element output into MediaStreams, so we already tag MediaStream data with origin information, but it's implemented in such a way that getUserMedia from one domain would be restricted in another domain (the other domain could render it in a media element, but it would be treated as cross-origin and thus would taint canvases it's drawn into, for example). I think it may make sense to provide cross-origin MediaStream transfer at some point in the future, but I think we have more important things to work on first. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: Extending createObjectUrl to MediaStream?
On Tue, Sep 3, 2013 at 2:01 PM, Robert O'Callahan rob...@ocallahan.org wrote: Yes. For example there are plans to enable some kind of private mode for WebRTC MediaStreams that protects stream contents from inspection by the page. I don't know exactly how this is going to work, but if we allow MediaStreams to span domains it may get more complicated. This scenario sounds very different from the one you outline next. More concretely, in Gecko we have experimental code to pipe HTML media element output into MediaStreams, so we already tag MediaStream data with origin information, but it's implemented in such a way that getUserMedia from one domain would be restricted in another domain (the other domain could render it in a media element, but it would be treated as cross-origin and thus would taint canvases it's drawn into, for example). It's not clear why if as a page I decide to share the MediaStream object I would not want all of that to be shared as I could share all of that regardless, it'd just require more hoops to jump through. I think it may make sense to provide cross-origin MediaStream transfer at some point in the future, but I think we have more important things to work on first. Again, what I'm trying to understand is why we have origin ties in the first place. So far no objects carry origin information in this regard. -- http://annevankesteren.nl/
Re: Extending createObjectUrl to MediaStream?
On 2013-09-02 01:44, Robert O'Callahan wrote: On Sat, Aug 31, 2013 at 1:14 AM, Stefan Håkansson LK stefan.lk.hakans...@ericsson.com mailto:stefan.lk.hakans...@ericsson.com wrote: One need I can see is when you want to display the video in another window. Let's say you want to have the video in a popout window - something I think we should definitely support - handing that window the URL (using postMessage) for use by its video element is a very convenient way to support this use-case. This works in Chrome today. Assuming the main window opened the popout window, and both pages have the same origin, this can be done without createObjectURL or any new features, because the main window has access to the DOM of the popout window (and vice versa). Yes, but I don't think we should constrain this to only meet cases when the origin of the other window is the same. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: Extending createObjectUrl to MediaStream?
On Mon, Sep 2, 2013 at 11:19 PM, Stefan Håkansson LK stefan.lk.hakans...@ericsson.com wrote: On 2013-09-02 01:44, Robert O'Callahan wrote: On Sat, Aug 31, 2013 at 1:14 AM, Stefan Håkansson LK stefan.lk.hakans...@ericsson.com mailto:stefan.lk.hakans...@ericsson.com wrote: One need I can see is when you want to display the video in another window. Let's say you want to have the video in a popout window - something I think we should definitely support - handing that window the URL (using postMessage) for use by its video element is a very convenient way to support this use-case. This works in Chrome today. Assuming the main window opened the popout window, and both pages have the same origin, this can be done without createObjectURL or any new features, because the main window has access to the DOM of the popout window (and vice versa). Yes, but I don't think we should constrain this to only meet cases when the origin of the other window is the same. What are your use-cases where they're not the same? More importantly, what are the use-cases where they cannot be made the same by the developer? Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: Extending createObjectUrl to MediaStream?
On Sat, Aug 31, 2013 at 1:14 AM, Stefan Håkansson LK stefan.lk.hakans...@ericsson.com wrote: One need I can see is when you want to display the video in another window. Let's say you want to have the video in a popout window - something I think we should definitely support - handing that window the URL (using postMessage) for use by its video element is a very convenient way to support this use-case. This works in Chrome today. Assuming the main window opened the popout window, and both pages have the same origin, this can be done without createObjectURL or any new features, because the main window has access to the DOM of the popout window (and vice versa). Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: Extending createObjectUrl to MediaStream?
On 2013-08-28 17:48, Dominique Hazael-Massieux wrote: Hi WebApps WG, The Media Capture Task Force (responsible for specs that deal with MediaStream objects [1]) has been considering whether one should be able to assign a MediaStream to video and audio elements via an URL obtained via createObjectURL or not, and is seeking feedback on the question. More precisely: A. we define a new srcObject attribute on HTMLMediaElement objects that can take directly a MediaStream object and make it playable video.srcObject = mediastream; B. the spec also supports assigning a MediaStream via the src attribute, via an URL obtained through createObjectURL: video.src = URL.createObjectURL(mediastream); While there are ongoing discussions on how to spec B properly [2] (which will require coordination with the WebApps Working Group), we are first and foremost wondering if that option is needed at all. We are thus looking for input on the use cases for createObjectURL as used for the File API, and whether these use cases would also apply to our MediaStream case. In general, is there a need for any object readable by media elements to support an URL-based approach for consistency with the rest of the platform? One need I can see is when you want to display the video in another window. Let's say you want to have the video in a popout window - something I think we should definitely support - handing that window the URL (using postMessage) for use by its video element is a very convenient way to support this use-case. This works in Chrome today. But, an alternative could perhaps be to make MediaStream a transferable (which means that the MediaStream object could be sent over using postMessage IIUC). Thanks! Dom for capture-ACTION-23 1. http://dev.w3.org/2011/webrtc/editor/getusermedia.html 2. https://www.w3.org/Bugs/Public/show_bug.cgi?id=19594
Re: Extending createObjectUrl to MediaStream?
On Thu, Aug 29, 2013 at 3:05 AM, Robert O'Callahan rob...@ocallahan.org wrote: I can't think of any good reason to support createObjectURL(MediaStream) --- except for compatibility with existing content, which may be an issue already. As I understand it, we have the createObjectURL() design because it's easier and allows integration with systems that have a poor API story, such as CSS. I'm not a fan and wish we had avoided it altogether in favor of objects everywhere, but I lost that battle. Given that MediaStream only works for a select set of end points, not supporting createObjectURL() would be somewhat better if that's indeed feasible. -- http://annevankesteren.nl/
Re: Extending createObjectUrl to MediaStream?
On Aug 28, 2013, at 9:48 AM, Dominique Hazael-Massieux wrote: Hi WebApps WG, The Media Capture Task Force (responsible for specs that deal with MediaStream objects [1]) has been considering whether one should be able to assign a MediaStream to video and audio elements via an URL obtained via createObjectURL or not, and is seeking feedback on the question. More precisely: A. we define a new srcObject attribute on HTMLMediaElement objects that can take directly a MediaStream object and make it playable video.srcObject = mediastream; B. the spec also supports assigning a MediaStream via the src attribute, via an URL obtained through createObjectURL: video.src = URL.createObjectURL(mediastream); While there are ongoing discussions on how to spec B properly [2] (which will require coordination with the WebApps Working Group), we are first and foremost wondering if that option is needed at all. We are thus looking for input on the use cases for createObjectURL as used for the File API, and whether these use cases would also apply to our MediaStream case. In general, is there a need for any object readable by media elements to support an URL-based approach for consistency with the rest of the platform? The fact that URL.createObjectURL exists at all is hitched to the ongoing use of strings as URLs in places like Image (img src=) and CSS. This also created the discussion about autoRevoke (namely the automatic revocation of these references), for which we added URL.createFor. As it stands, developers should bear the onus of calling URL.revokeObjectURL coupled to URL.createObjectURL, which may be a tall request. MediaStream may not be as ubiquitous as Blob, and you may be able to avoid URL strings altogether. If that's a fair assumption, option A seems like a good plan. If you go the option B route, then you'll have to also think about revocation and URL.createFor, since you'll have the same problem. On balance, I favor Option A, unless you make the case for strings :) If you choose Option B, I'm happy to work with your WG to make sure File API has the right hook for MediaStream objects, per https://www.w3.org/Bugs/Public/show_bug.cgi?id=19594 -- A* -- A*
Re: Extending createObjectUrl to MediaStream?
On Thu, Aug 29, 2013 at 3:48 AM, Dominique Hazael-Massieux d...@w3.orgwrote: We are thus looking for input on the use cases for createObjectURL as used for the File API, and whether these use cases would also apply to our MediaStream case. In general, is there a need for any object readable by media elements to support an URL-based approach for consistency with the rest of the platform? I can't think of any good reason to support createObjectURL(MediaStream) --- except for compatibility with existing content, which may be an issue already. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
