Re: [whatwg] Comments about the track element
Hi Silvia, [trimming a bit the text to keep the email short] Le 7/27/2012 12:35 AM, Silvia Pfeiffer a écrit : There is the SVG viewbox and there is the video viewbox. It is not immediately clear how they relate to each other. What I meant was: how to position the SVG viewbox within the boundaries of the video viewbox. It could fully cover it, but it may not need to. For example in your example with the clock, it could be positioned by coordinates of the video, e.g. left: 70%, top:30% or something like it. Then the SVG can be much smaller and it is possible to overlay other elements, too. I understand. What I'm saying is: when SVG is used as track, make it simple, both viewboxes are the same. You could, however, put SVG in WebVTT e.g. to provide overlay graphics that are non-moving or are in a loop for a certain duration of the video. E.g. an animated character (like your Rhino) could be rendered in a loop on top of a video for the first 3 minutes of the video. Agreed, why not. I don't want to take this discussion off track, but it is news to me that TTML can express frame-based animations. I indeed wouldn't mingle WebVTT and TTML layering since they satisfy the same use cases. I've seen examples like that, used to carry DVB subtitles, urgh! How does the browser support constructing SVG progressively right now? If there is a SVG-internal solution, that should be used. In this case, @mediagroup synchronization would again make the most sense. Or you just do everything in SVG. Browsers construct SVG progressively right now as they do for HTML. The parser parses the data as it receives it. @mediagroup is indeed a solution but the track has other advantages. Controlling the time when the data is received for instance with inband stored track is one. No, not really. What I meant was to draw the blue handle on top of the video not through cues, but directly in the browser. Then, the WebVTT file only delivers the according position changes for that particular time and all you need to do in JavaScript is to change the handle position in the SVG. That makes the WebVTT slimmer and not contain any SVG at all. Right, but that again requires JS, and hence incurs some processing penalty. And also, this requires a dedicated authoring while using an SVG track, you could just use any existing tool. The basic Track interface would be almost the same as the VideoTrack or AudioTrack. The GraphicsDocumentTrack interface would be used for tracks which have an underlying document (TTML, SVG, SMIL?, HTML?...) with a visual representation and not necessarily based on cues. For these documents, you are not interested in cues or cue changes (and it might not make sense to define cues). For these, you're only interested in: - the dispatch of the track content to the parser being done automatically by the browser (no need to use a JS DOMParser); - the rendering of the underlying document being synchronized (natively) by the browser, i.e. the timeline of the underlying document should be locked with the timeline of the video (or audio). No need to monitor cue changes to render the right SVG. You could discriminate between a TextTrack and a GraphicsDocumentTrack by a mime type or the inBandMetadataTrackDispatchType (not sure...). When the track carries SVG, the trackDocument object could be an SVGDocument. This would allow for controlling the SVG as if it was embedded in the HTML but for the synchronization done by the browser. What do you think? Why does it have to be a track at all? Video and audio can be synchronized to each other without one needing to be a track of the other. To use @mediagroup, you might need to consider what an SVG graphic has to provide for the MediaController [1]. There is no need to consider cues and tracks - we seem to agree on that. :-) We seem to agree on many things but not all ;) We agree that there are use cases for SVG graphics synchronize and overlayed on top of a video and that @mediagroup is a solution. The track mechanism (without cue) has several advantages: - synchronization equivalent to the @mediagroup solution - reuse of SVG native behavior (progressive loading) equivalent to the @mediagroup solution - selectability of the track using the default UI controls, just like with subtitles, without needing specific controls. - rendering on top of the video (just like subtitles) without interfering with the UI controls. - unified handling of out-of-band and in-band SVG tracks. Note, that this would be perfectly applicable to HTML/CSS animations on top of a video. It is just enlarging the scope of tracks, currently restricted (except for video and audio) to cue-based formats and text only. Regards, Cyril -- Cyril Concolato Maître de Conférences/Associate Professor Groupe Multimedia/Multimedia Group Telecom ParisTech 46 rue Barrault 75 013 Paris, France http://concolato.wp.mines-telecom.fr/
Re: [whatwg] Comments about the track element
Hi Silvia, Thank you for your reply. Your comments are quite helpful to understand how WebVTT can be used or not. See my comments below. Le 7/26/2012 12:44 AM, Silvia Pfeiffer a écrit : On Wed, Jul 25, 2012 at 11:45 PM, Cyril Concolato cyril.concol...@telecom-paristech.fr wrote: Right now it is fully defined how data in a TextTrack (of the defined kinds) is displayed on top of the video. As this is as yet unclear for SVG resources, I wouldn't say it's unclear, I'd say it needs to be specified ;) meaning that it probably doesn't require much specification. I was thinking that we could use the CSS box of the video element to position the SVG, as if the SVG was put in a div. Let's work on this basis and see where we get. There's also positioning issues What do you mean here by positioning issues? SVG handles the positioning within its viewbox and what I propose is to define the size and position of this viewbox in the parent coordinate system, i.e. with respect to the video. I don't see what else is needed? or do you mean when SVG is transported in cue, how do you use the cue settings? etc. so it's not as simple as just putting the SVG in a cue. I would suggest using the @metadata tack kind for now and providing the SVG as markup in a TextTrackCue (either from WebVTT cues I've tried this option but I'm facing several problems (Tested with Chrome Version 22.0.1216.0 canary). The first problem is how to embed SVG in a cue? Should the '', '' and other characters be escaped or not? According to Anne's validator, So, I assume you created WebVTT files. (You don't have to - you can directly use the TextTrack API.) Anne's validator validates the WebVTT rules for caption and subtitle kinds. For metadata kinds, there should be no parsing of the cues in browsers. Reading the spec again, I see that the parsing rules for WebVTT metadata text are different indeed. My mistake. A validator can only decide whether to parse the cues according to captions/subtitles, or chapters, or metadata rules if the WebVTT file has such an indicator. I've asked for such information to be included in WebVTT, but we don't currently have such markup/metadata. Do you mean that you would like to have some signaling in the WebVTT file (for instance in the header) to indicate the type of the cue payload? I think that'll be interesting. Otherwise, it'll be interesting to have a type selector in the validator. they should be. Actually, for @kind=metadata you don't need to escape '' or ''. Yes, I had missed that. But if I use them, then the parsing of the escaped string returns 'empty document' (http://perso.telecom-paristech.fr/~concolat/html5_tests/getcueasSVG-escaped.html). Which parsing? Anne's validator? Have you tried Chrome directly? Here I meant using a DOMParser object in JavaScript using Chrome. http://perso.telecom-paristech.fr/~concolat/html5_tests/svg-escaped.vtt does look very ugly. Indeed, but as you said, for the @kind=metadata it is not needed. However, if I don't escape them, the parsing doesn't fail and returns an SVG document (http://perso.telecom-paristech.fr/~concolat/html5_tests/getcueasSVG.html). cue.text is the SVG code? That's what we want, right? (http://perso.telecom-paristech.fr/~concolat/html5_tests/svg.vtt looks much nicer) Yes, cue.text seems to be the best option when using WebVTT. In any case, I think embedding the SVG in WEBVTT does not really make sense. Why not? I should have said forcing the embedding of SVG in WEBVTT does not make sense, since there is some overlap between the 2 (timing, positioning ...), some overhead, and limitations (see below). An other problem is in terms of design. SVG has a timing model (similar to TTML), WebVTT another. For instance, SVG can express things like repetitions of animations that WebVTT cannot. Are you saying that TTML should be carried in a WebVTT file? TTML in WebVTT probably doesn't make sense. But SVG's timing model can be a applied within the timeframe of a cue, so that does make sense. Maybe, yes. It might make sense if your cue has a long duration, otherwise the overhead of loading an SVG document for each cue might be too big. But in general, since you can structure an SVG document with a frame-based structure (see this cartoon for instance: http://perso.telecom-paristech.fr/~concolat/SVG/flash10.svg http://perso.telecom-paristech.fr/%7Econcolat/SVG/flash10.svg), I don't see the added value of WebVTT to carry SVG. How would you specify this with TTML? It would run into the same problems, wouldn't it? I think so, the problems would be similar. But again, TTML can also express frame-based animations, why should you add the WebVTT layer? Similarly, in terms of design, embedding SVG in cues requires repeating a lot of SVG content at each cue (see http://perso.telecom-paristech.fr/~concolat/html5_tests/svg.vtt), as this approach requires parsing an entire document at each cue. You could probably
Re: [whatwg] Comments about the track element
Hi Cyril,
On Thu, Jul 26, 2012 at 10:03 PM, Cyril Concolato
cyril.concol...@telecom-paristech.fr wrote:
What do you mean here by positioning issues? SVG handles the positioning
within its viewbox and what I propose is to define the size and position of
this viewbox in the parent coordinate system, i.e. with respect to the
video. I don't see what else is needed? or do you mean when SVG is
transported in cue, how do you use the cue settings?
There is the SVG viewbox and there is the video viewbox. It is not
immediately clear how they relate to each other. What I meant was: how
to position the SVG viewbox within the boundaries of the video
viewbox. It could fully cover it, but it may not need to. For example
in your example with the clock, it could be positioned by coordinates
of the video, e.g. left: 70%, top:30% or something like it. Then the
SVG can be much smaller and it is possible to overlay other elements,
too.
Do you mean that you would like to have some signaling in the WebVTT file
(for instance in the header) to indicate the type of the cue payload? I
think that'll be interesting.
Yes, we have a proposal for a metadata field in the WebVTT header to
signify the kind.
Otherwise, it'll be interesting to have a type
selector in the validator.
That can work, too, of course.
TTML in WebVTT probably doesn't make sense. But SVG's timing model can
be a applied within the timeframe of a cue, so that does make sense.
Maybe, yes. It might make sense if your cue has a long duration, otherwise
the overhead of loading an SVG document for each cue might be too big. But
in general, since you can structure an SVG document with a frame-based
structure (see this cartoon for instance:
http://perso.telecom-paristech.fr/~concolat/SVG/flash10.svg), I don't see
the added value of WebVTT to carry SVG.
Indeed, for this kind of use case, putting SVG in WebVTT makes no sense.
You could, however, put SVG in WebVTT e.g. to provide overlay graphics
that are non-moving or are in a loop for a certain duration of the
video. E.g. an animated character (like your Rhino) could be rendered
in a loop on top of a video for the first 3 minutes of the video.
How would you specify this with TTML? It would run into the same
problems, wouldn't it?
I think so, the problems would be similar. But again, TTML can also express
frame-based animations, why should you add the WebVTT layer?
I don't want to take this discussion off track, but it is news to me
that TTML can express frame-based animations.
I indeed wouldn't mingle WebVTT and TTML layering since they satisfy
the same use cases.
What would your preferred markup for
http://perso.telecom-paristech.fr/~concolat/html5_tests/svg.vtt be ?
How would you avoid the duplication?
For instance, you would want to be able to construct the SVG document
progressively, to have only one document that you modify by adding more
data. One way to do it would be to have the first cue contain the beginning
of the document and the following cues contain more data, but since
modifying the document after its load is tricky, this would require
concatenating all previous cue texts and then parsing that as a new document
(ugly!). I'd like to have the parsing step done under the hood by the
browser, as it usually do.
How does the browser support constructing SVG progressively right now?
If there is a SVG-internal solution, that should be used. In this
case, @mediagroup synchronization would again make the most sense. Or
you just do everything in SVG.
If you try my example here
(http://perso.telecom-paristech.fr/~concolat/html5_tests/getcueasSVG.html),
you'll see that changing the playback speed (even to 0.1) does not guarantee
synchronization either. By the time the JS has processed the content, it's
already too late. It might be an implementation issue but it's symptomatic
of the stacking, that's why I think we should leverage the native parsing,
synchronization and support for SVG rendering (not through JS). The clock
might be a (not so) extreme case, but I don't think I'm trying to do very
fancy things here, just trying to reproduce existing technologies
(proprietary or not) with existing web standards.
Sure.
I'm not sure. Having to repeatedly parse WebVTT cues and draw the SVG
image makes this particularly slow. Have you tried to paint the SVG
just once on the video and using TextTrackCues just to change the
transform value using JavaScript? Upon a cuechange event, you re-draw
the SVG.
I could give it a try if I have some time but I'm not really sure I
understand what you're suggesting. Do you mean using addCue? Could you give
an example? Are you suggesting something similar to the example in the spec
with
var sounds = sfx.addTextTrack('metadata');
No, not really. What I meant was to draw the blue handle on top of the
video not through cues, but directly in the browser. Then, the WebVTT
file only delivers the according position changes for
Re: [whatwg] Comments about the track element
On Wed, Jul 25, 2012 at 4:24 AM, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: You can even use getCueAsHTML() to simply hand the SVG to HTML for rendering. FWIW, that is not going to work. You need to parse the SVG using innerHTML or some such. getCueAsHTML() only gives a representation of a WebVTT tree which cannot contain SVG. -- http://annevankesteren.nl/
Re: [whatwg] Comments about the track element
Le 7/24/2012 5:04 PM, Ian Hickson a écrit : [only replied on the whatwg list; please if possible avoid cross-posting as it tends to fracture threads when people only on one list and not the other reply] [I'll forward the mails to the SVG mailing list separately to make sure people not subscribed to the whatwg list can still get the whole discussion] On Tue, 24 Jul 2012, Cyril Concolato wrote: During the ongoing SVG F2F meeting, the SVG WG discussed the use case of displaying SVG graphics on top of a video, in a synchronous manner. The SVG WG believes that for such use case, it is necessary to indicate to the browser that the SVG and video content should stay synchronized (no matter what happens to the video playback), and to let the browser handle the synchronization internally. The SVG WG resolved to include such indication as part of the Web Animation specification, for instance using the HTML mediagroup attribute or the MediaController API. However, the SVG WG thinks it would also be interesting to leverage the native UI controls of the video element to select (or deactivate) the display of the SVG content on top of the video, in a similar manner to a subtitle track. Obviously, the HTML 5 track element would be a suitable option for that. However, currently it only allows text tracks. So, the SVG WG would like HTML to allow the track element's URL to identify an SVG resource, and in that case the track kind would be 'graphics'. There would be a need to define how the graphics are displayed on top of the video, for instance reusing the viewport/viewbox negotiation phase. There would also be a need to make a more generic Track API or to replace the TextTrack API by the SVG API when the track is of kind 'graphics'. My understanding is that the expected way to solve this previously was using SMIL, putting the video in the SVG itself. That seems to make more sense that having the SVG layered on top of an HTML video. Why is this no longer a viable solution? First, the SVG WG resolved to align the SVG audio and video element with the HTML 5 ones. Then, in any case, I think both options are reasonable and viable. If your application is primarily SVG, you can indeed use a video element in SVG. But if your application is mainly HTML, and you just want to display SVG on top of the video, it makes perfect sense to stay in HTML I think. Regards, Cyril -- Cyril Concolato Maître de Conférences/Associate Professor Groupe Multimedia/Multimedia Group Telecom ParisTech 46 rue Barrault 75 013 Paris, France http://concolato.wp.mines-telecom.fr/
Re: [whatwg] Comments about the track element
Hi Silvia, Le 7/25/2012 4:24 AM, Silvia Pfeiffer a écrit : Expanding a bit on what Anne said... On Tue, Jul 24, 2012 at 11:18 PM, Cyril Concolato cyril.concol...@telecom-paristech.fr wrote: Dear WhatWG, During the ongoing SVG F2F meeting, the SVG WG discussed the use case of displaying SVG graphics on top of a video, in a synchronous manner. The SVG WG believes that for such use case, it is necessary to indicate to the browser that the SVG and video content should stay synchronized (no matter what happens to the video playback), and to let the browser handle the synchronization internally. The SVG WG resolved to include such indication as part of the Web Animation specification, for instance using the HTML mediagroup attribute or the MediaController API. However, the SVG WG thinks it would also be interesting to leverage the native UI controls of the video element to select (or deactivate) the display of the SVG content on top of the video, in a similar manner to a subtitle track. Obviously, the HTML 5 track element would be a suitable option for that. However, currently it only allows text tracks. So, the SVG WG would like HTML to allow the track element's URL to identify an SVG resource, and in that case the track kind would be 'graphics'. There would be a need to define how the graphics are displayed on top of the video, Right now it is fully defined how data in a TextTrack (of the defined kinds) is displayed on top of the video. As this is as yet unclear for SVG resources, I wouldn't say it's unclear, I'd say it needs to be specified ;) meaning that it probably doesn't require much specification. I was thinking that we could use the CSS box of the video element to position the SVG, as if the SVG was put in a div. I would suggest using the @metadata tack kind for now and providing the SVG as markup in a TextTrackCue (either from WebVTT cues I've tried this option but I'm facing several problems (Tested with Chrome Version 22.0.1216.0 canary). The first problem is how to embed SVG in a cue? Should the '', '' and other characters be escaped or not? According to Anne's validator, they should be. But if I use them, then the parsing of the escaped string returns 'empty document' (http://perso.telecom-paristech.fr/~concolat/html5_tests/getcueasSVG-escaped.html http://perso.telecom-paristech.fr/%7Econcolat/html5_tests/getcueasSVG-escaped.html). However, if I don't escape them, the parsing doesn't fail and returns an SVG document (http://perso.telecom-paristech.fr/~concolat/html5_tests/getcueasSVG.html http://perso.telecom-paristech.fr/%7Econcolat/html5_tests/getcueasSVG.html). In any case, I think embedding the SVG in WEBVTT does not really make sense. An other problem is in terms of design. SVG has a timing model (similar to TTML), WebVTT another. For instance, SVG can express things like repetitions of animations that WebVTT cannot. Are you saying that TTML should be carried in a WebVTT file? Similarly, in terms of design, embedding SVG in cues requires repeating a lot of SVG content at each cue (see http://perso.telecom-paristech.fr/~concolat/html5_tests/svg.vtt http://perso.telecom-paristech.fr/%7Econcolat/html5_tests/svg.vtt), as this approach requires parsing an entire document at each cue. You could probably envisage overlapping cues but that would require a lot of overhead. Leveraging the progressive loading of SVG cannot be done this way either. In general, I think it would make sense to leverage the browsers' support for SVG and not stack different technologies. Another problem is that I don't know if it's possible to display the SVG content in a layer between the video and the UI controls. Currently, I display the SVG on top of the video element, therefore the UI controls are not accessible for clicks. Having to embed my own UI controls for that is a bit of a pain. And, semantically, when reading the spec, 'metadata' tracks say Not displayed by the user agent. so I think this might be a bit confusing for users/authors. The third problem is performance-wise. In my example, the blue line (in SVG), when synchronized with the video, should be aligned with the moving (white-gray) edge of the pie. As you can see, this is not the case. Only 4-5 cuechange events seems to be processed properly. I noticed the same problem with 'timeupdate' events. Also, I've noticed that even though my WebVTT file is designed to have only one active cue at a time, for some cuechange events, there are 2. This might be an implementation issue but this might be a problem of reentrant code (the cuechange callback being called while it's not finished), but in general, I'm not sure it's a good idea to go through the Javascript engine to do that, for the processing overhead. or from JavaScript calls to addTextTrack()). Can you elaborate on this one? However, I suspect it'll have the same processing overhead. The timing/synchronization is provided by the
Re: [whatwg] Comments about the track element
Hi Silvia, Le 7/25/2012 3:42 PM, Silvia Pfeiffer a écrit : On Wed, Jul 25, 2012 at 10:55 PM, Henri Sivonenhsivo...@iki.fi wrote: On Wed, Jul 25, 2012 at 11:24 AM, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: But you can use cue.text and parse it as a SVG fragment. That would be RSS all over again. :-( To some extent. If we are very clear about what will be in the cues and that it will always be just SVG, we could just create a @kind=svg. The SVG WG resolved to write a document describing how to store SVG content into streaming packets (or cues) whether full documents or document fragments for progressive loading of content. I'll be the editor of this document. You can already have an idea of what would be in this document looking at http://www.w3.org/Graphics/SVG/WG/wiki/SVGStreaming. I plan to talk about cue content and associated signaling timing, random access point ... What I think we may need at this stage is a demo implementation using existing @kind=metadata functionality and then see what features are required in the cues and if that can be generalized as a new @kind. I think my previous email shows an attempt at providing such demo implementation (http://perso.telecom-paristech.fr/~concolat/html5_tests/getcueasSVG.html http://perso.telecom-paristech.fr/%7Econcolat/html5_tests/getcueasSVG.html) and why it is not sufficient. I'll be happy to take suggestions to improve it. Regards, Cyril Silvia. -- Cyril Concolato Maître de Conférences/Associate Professor Groupe Multimedia/Multimedia Group Telecom ParisTech 46 rue Barrault 75 013 Paris, France http://concolato.wp.mines-telecom.fr/
Re: [whatwg] Comments about the track element
On Wed, Jul 25, 2012 at 11:45 PM, Cyril Concolato cyril.concol...@telecom-paristech.fr wrote: Right now it is fully defined how data in a TextTrack (of the defined kinds) is displayed on top of the video. As this is as yet unclear for SVG resources, I wouldn't say it's unclear, I'd say it needs to be specified ;) meaning that it probably doesn't require much specification. I was thinking that we could use the CSS box of the video element to position the SVG, as if the SVG was put in a div. Let's work on this basis and see where we get. There's also positioning issues etc. so it's not as simple as just putting the SVG in a cue. I would suggest using the @metadata tack kind for now and providing the SVG as markup in a TextTrackCue (either from WebVTT cues I've tried this option but I'm facing several problems (Tested with Chrome Version 22.0.1216.0 canary). The first problem is how to embed SVG in a cue? Should the '', '' and other characters be escaped or not? According to Anne's validator, So, I assume you created WebVTT files. (You don't have to - you can directly use the TextTrack API.) Anne's validator validates the WebVTT rules for caption and subtitle kinds. For metadata kinds, there should be no parsing of the cues in browsers. A validator can only decide whether to parse the cues according to captions/subtitles, or chapters, or metadata rules if the WebVTT file has such an indicator. I've asked for such information to be included in WebVTT, but we don't currently have such markup/metadata. they should be. Actually, for @kind=metadata you don't need to escape '' or ''. But if I use them, then the parsing of the escaped string returns 'empty document' (http://perso.telecom-paristech.fr/~concolat/html5_tests/getcueasSVG-escaped.html). Which parsing? Anne's validator? Have you tried Chrome directly? http://perso.telecom-paristech.fr/~concolat/html5_tests/svg-escaped.vtt does look very ugly. However, if I don't escape them, the parsing doesn't fail and returns an SVG document (http://perso.telecom-paristech.fr/~concolat/html5_tests/getcueasSVG.html). cue.text is the SVG code? That's what we want, right? (http://perso.telecom-paristech.fr/~concolat/html5_tests/svg.vtt looks much nicer) In any case, I think embedding the SVG in WEBVTT does not really make sense. Why not? An other problem is in terms of design. SVG has a timing model (similar to TTML), WebVTT another. For instance, SVG can express things like repetitions of animations that WebVTT cannot. Are you saying that TTML should be carried in a WebVTT file? TTML in WebVTT probably doesn't make sense. But SVG's timing model can be a applied within the timeframe of a cue, so that does make sense. How would you specify this with TTML? It would run into the same problems, wouldn't it? Similarly, in terms of design, embedding SVG in cues requires repeating a lot of SVG content at each cue (see http://perso.telecom-paristech.fr/~concolat/html5_tests/svg.vtt), as this approach requires parsing an entire document at each cue. You could probably envisage overlapping cues but that would require a lot of overhead. Leveraging the progressive loading of SVG cannot be done this way either. In general, I think it would make sense to leverage the browsers' support for SVG and not stack different technologies. Sure, it should use existing SVG support. I'm not so sure I agree with not stacking - that depends. What would your preferred markup for http://perso.telecom-paristech.fr/~concolat/html5_tests/svg.vtt be ? How would you avoid the duplication? Another problem is that I don't know if it's possible to display the SVG content in a layer between the video and the UI controls. Currently, I display the SVG on top of the video element, therefore the UI controls are not accessible for clicks. Having to embed my own UI controls for that is a bit of a pain. And, semantically, when reading the spec, 'metadata' tracks say Not displayed by the user agent. so I think this might be a bit confusing for users/authors. All publishers that want the same controls in all browsers make their own controls anyway. If you make a library for SVG display on top of a video, you can also make one for the controls (or use one of the many existing ones). The third problem is performance-wise. In my example, the blue line (in SVG), when synchronized with the video, should be aligned with the moving (white-gray) edge of the pie. As you can see, this is not the case. Only 4-5 cuechange events seems to be processed properly. I noticed the same problem with 'timeupdate' events. Also, I've noticed that even though my WebVTT file is designed to have only one active cue at a time, for some cuechange events, there are 2. This might be an implementation issue but this might be a problem of reentrant code (the cuechange callback being called while it's not finished), but in general, I'm not sure it's a good idea to go through
Re: [whatwg] Comments about the track element
On Wed, Jul 25, 2012 at 11:51 PM, Cyril Concolato cyril.concol...@telecom-paristech.fr wrote: Hi Silvia, Le 7/25/2012 3:42 PM, Silvia Pfeiffer a écrit : On Wed, Jul 25, 2012 at 10:55 PM, Henri Sivonenhsivo...@iki.fi wrote: On Wed, Jul 25, 2012 at 11:24 AM, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: But you can use cue.text and parse it as a SVG fragment. That would be RSS all over again. :-( To some extent. If we are very clear about what will be in the cues and that it will always be just SVG, we could just create a @kind=svg. The SVG WG resolved to write a document describing how to store SVG content into streaming packets (or cues) whether full documents or document fragments for progressive loading of content. I'll be the editor of this document. You can already have an idea of what would be in this document looking at http://www.w3.org/Graphics/SVG/WG/wiki/SVGStreaming. I plan to talk about cue content and associated signaling timing, random access point ... I think the @mediagroup approach is indeed better, in particular for SVG content that should synchronize frame-by-frame. Silvia.
Re: [whatwg] Comments about the track element
On Tue, Jul 24, 2012 at 3:18 PM, Cyril Concolato cyril.concol...@telecom-paristech.fr wrote: The SVG WG would like to have your feedback on this matter. Once this feature is more widely deployed a library could use metadata tracks to implement this suggestion. It seems better to wait and study the patterns that emerge than standardize on something now. -- http://annevankesteren.nl/
Re: [whatwg] Comments about the track element
[only replied on the whatwg list; please if possible avoid cross-posting as it tends to fracture threads when people only on one list and not the other reply] On Tue, 24 Jul 2012, Cyril Concolato wrote: During the ongoing SVG F2F meeting, the SVG WG discussed the use case of displaying SVG graphics on top of a video, in a synchronous manner. The SVG WG believes that for such use case, it is necessary to indicate to the browser that the SVG and video content should stay synchronized (no matter what happens to the video playback), and to let the browser handle the synchronization internally. The SVG WG resolved to include such indication as part of the Web Animation specification, for instance using the HTML mediagroup attribute or the MediaController API. However, the SVG WG thinks it would also be interesting to leverage the native UI controls of the video element to select (or deactivate) the display of the SVG content on top of the video, in a similar manner to a subtitle track. Obviously, the HTML 5 track element would be a suitable option for that. However, currently it only allows text tracks. So, the SVG WG would like HTML to allow the track element's URL to identify an SVG resource, and in that case the track kind would be 'graphics'. There would be a need to define how the graphics are displayed on top of the video, for instance reusing the viewport/viewbox negotiation phase. There would also be a need to make a more generic Track API or to replace the TextTrack API by the SVG API when the track is of kind 'graphics'. My understanding is that the expected way to solve this previously was using SMIL, putting the video in the SVG itself. That seems to make more sense that having the SVG layered on top of an HTML video. Why is this no longer a viable solution? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Comments about the track element
Expanding a bit on what Anne said... On Tue, Jul 24, 2012 at 11:18 PM, Cyril Concolato cyril.concol...@telecom-paristech.fr wrote: Dear WhatWG, During the ongoing SVG F2F meeting, the SVG WG discussed the use case of displaying SVG graphics on top of a video, in a synchronous manner. The SVG WG believes that for such use case, it is necessary to indicate to the browser that the SVG and video content should stay synchronized (no matter what happens to the video playback), and to let the browser handle the synchronization internally. The SVG WG resolved to include such indication as part of the Web Animation specification, for instance using the HTML mediagroup attribute or the MediaController API. However, the SVG WG thinks it would also be interesting to leverage the native UI controls of the video element to select (or deactivate) the display of the SVG content on top of the video, in a similar manner to a subtitle track. Obviously, the HTML 5 track element would be a suitable option for that. However, currently it only allows text tracks. So, the SVG WG would like HTML to allow the track element's URL to identify an SVG resource, and in that case the track kind would be 'graphics'. There would be a need to define how the graphics are displayed on top of the video, Right now it is fully defined how data in a TextTrack (of the defined kinds) is displayed on top of the video. As this is as yet unclear for SVG resources, I would suggest using the @metadata tack kind for now and providing the SVG as markup in a TextTrackCue (either from WebVTT cues or from JavaScript calls to addTextTrack()). The timing/synchronization is provided by the TextTrackCue, which seems to be all you are asking for right now. The rendering is then done by JavaScript, which seems the most flexible approach. You can even use getCueAsHTML() to simply hand the SVG to HTML for rendering. @metadata tracks can be part of the native video controls and the @label on the text track would provide a description of what it is, e.g. Graphic overlays. for instance reusing the viewport/viewbox negotiation phase. There would also be a need to make a more generic Track API or to replace the TextTrack API by the SVG API when the track is of kind 'graphics'. I don't understand this requirement. What API needs are there aside from the synchronization? Trying to replicate SVG APIs through the TextTrack API seems like a repetition of the API and thus fragile. Regards, Silvia.
