Hi Silvia,
Le 7/25/2012 4:24 AM, Silvia Pfeiffer a écrit :
Expanding a bit on what Anne said...
On Tue, Jul 24, 2012 at 11:18 PM, Cyril Concolato
<cyril.concol...@telecom-paristech.fr> wrote:
Dear WhatWG,
During the ongoing SVG F2F meeting, the SVG WG discussed the use case of
displaying SVG graphics on top of a video, in a synchronous manner.
The SVG WG believes that for such use case, it is necessary to indicate to
the browser that the SVG and video content should stay synchronized (no
matter what happens to the video playback), and to let the browser handle
the synchronization internally. The SVG WG resolved to include such
indication as part of the Web Animation specification, for instance using
the HTML mediagroup attribute or the MediaController API.
However, the SVG WG thinks it would also be interesting to leverage the
native UI controls of the video element to select (or deactivate) the
display of the SVG content on top of the video, in a similar manner to a
subtitle track. Obviously, the HTML 5 track element would be a suitable
option for that. However, currently it only allows text tracks. So, the SVG
WG would like HTML to allow the track element's URL to identify an SVG
resource, and in that case the track kind would be 'graphics'. There would
be a need to define how the graphics are displayed on top of the video,
Right now it is fully defined how data in a TextTrack (of the defined
kinds) is displayed on top of the video. As this is as yet unclear for
SVG resources,
I wouldn't say it's unclear, I'd say it needs to be specified ;) meaning
that it probably doesn't require much specification. I was thinking that
we could use the CSS box of the video element to position the SVG, as if
the SVG was put in a div.
I would suggest using the @metadata tack kind for now
and providing the SVG as markup in a TextTrackCue (either from WebVTT
cues
I've tried this option but I'm facing several problems (Tested with
Chrome Version 22.0.1216.0 canary).
The first problem is how to embed SVG in a cue? Should the '<', '>' and
other characters be escaped or not? According to Anne's validator, they
should be. But if I use them, then the parsing of the escaped string
returns 'empty document'
(http://perso.telecom-paristech.fr/~concolat/html5_tests/getcueasSVG-escaped.html
<http://perso.telecom-paristech.fr/%7Econcolat/html5_tests/getcueasSVG-escaped.html>).
However, if I don't escape them, the parsing doesn't fail and returns an
SVG document
(http://perso.telecom-paristech.fr/~concolat/html5_tests/getcueasSVG.html <http://perso.telecom-paristech.fr/%7Econcolat/html5_tests/getcueasSVG.html>).
In any case, I think embedding the SVG in WEBVTT does not really make
sense.
An other problem is in terms of design. SVG has a timing model (similar
to TTML), WebVTT another. For instance, SVG can express things like
repetitions of animations that WebVTT cannot. Are you saying that TTML
should be carried in a WebVTT file?
Similarly, in terms of design, embedding SVG in cues requires repeating
a lot of SVG content at each cue (see
http://perso.telecom-paristech.fr/~concolat/html5_tests/svg.vtt
<http://perso.telecom-paristech.fr/%7Econcolat/html5_tests/svg.vtt>), as
this approach requires parsing an entire document at each cue. You could
probably envisage overlapping cues but that would require a lot of
overhead. Leveraging the progressive loading of SVG cannot be done this
way either.
In general, I think it would make sense to leverage the browsers'
support for SVG and not stack different technologies.
Another problem is that I don't know if it's possible to display the SVG
content in a layer between the video and the UI controls. Currently, I
display the SVG on top of the video element, therefore the UI controls
are not accessible for clicks. Having to embed my own UI controls for
that is a bit of a pain. And, semantically, when reading the spec,
'metadata' tracks say " Not displayed by the user agent. " so I think
this might be a bit confusing for users/authors.
The third problem is performance-wise. In my example, the blue line (in
SVG), when synchronized with the video, should be aligned with the
moving (white-gray) edge of the pie. As you can see, this is not the
case. Only 4-5 cuechange events seems to be processed properly. I
noticed the same problem with 'timeupdate' events. Also, I've noticed
that even though my WebVTT file is designed to have only one active cue
at a time, for some cuechange events, there are 2. This might be an
implementation issue but this might be a problem of reentrant code (the
cuechange callback being called while it's not finished), but in
general, I'm not sure it's a good idea to go through the Javascript
engine to do that, for the processing overhead.
or from JavaScript calls to addTextTrack()).
Can you elaborate on this one? However, I suspect it'll have the same
processing overhead.
The
timing/synchronization is provided by the TextTrackCue, which seems to
be all you are asking for right now. The rendering is then done by
JavaScript, which seems the most flexible approach. You can even use
getCueAsHTML() to simply hand the SVG to HTML for rendering.
@metadata tracks can be part of the native video controls and the
@label on the text track would provide a description of what it is,
e.g. "Graphic overlays".
for
instance reusing the viewport/viewbox negotiation phase. There would also be
a need to make a more generic Track API or to replace the TextTrack API by
the SVG API when the track is of kind 'graphics'.
I don't understand this requirement. What API needs are there aside
from the synchronization? Trying to replicate SVG APIs through the
TextTrack API seems like a repetition of the API and thus fragile.
Sorry for the confusion here. I didn't mean to replicate the SVG APIs
here but I just meant that the TextTrack API is very specific to 'pure'
text tracks (and even to WebVTT text tracks). You might want to expose
the SVG API when SVG content is used for the overlay to control it.
Regards,
Cyril
--
Cyril Concolato
Maître de Conférences/Associate Professor
Groupe Multimedia/Multimedia Group
Telecom ParisTech
46 rue Barrault
75 013 Paris, France
http://concolato.wp.mines-telecom.fr/
