[whatwg] Recording interface (Re: Peer-to-peer communication, video conferencing, and related topics (2))
-Original Message-
From: whatwg-boun...@lists.whatwg.org
[mailto:whatwg-boun...@lists.whatwg.org] On Behalf Of
whatwg-requ...@lists.whatwg.org
Sent: den 29 mars 2011 20:33
To: whatwg@lists.whatwg.org
Subject: whatwg Digest, Vol 84, Issue 69
I also believe that the recording interface should be
removed from
this part of the specification; there should be no requirement
that all streams be recordable.
Recording of streams is needed for some use cases unrelated
to video
conferencing, such as recording messages.
Having a recording function is needed in multiple use cases;
I think we all agree on that.
This is mostly a matter of style, which I'm happy to defer on.
The streams should be regarded as a control surface,
not as a data
channel; in many cases, the question of what is the
format of the stream at this point
is literally unanswerable; it may be represented as hardware
states, memory buffers, byte streams, or something
completely different.
Agreed.
Recording any of these requires much more specification
than just
record here.
Could you elaborate on what else needs specifying?
One thing I remember from an API design talk I viewed:
An ability to record to a file means that the file format is
part of your API.
For instance, for audio recording, it's likely that you want
control over whether the resulting file is in Ogg Vorbis
format or in MP3 format; for video, it's likely that you may
want to specify that it will be stored using the VP8 video
codec, the Vorbis audio codec and the Matroska container
format. These desires have to be communicated to the
underlying audio/video engine, so that the proper transforms
can be inserted into the processing stream, and I think they
have to be communicated across this interface; since the
output of these operations is a blob without any inherent
type information, the caller has to already know which format
the media is in.
This is absolutely correct, and it is not only about codecs or
container formats. Maybe you need to supply info like audio
sampling rate, video frame rate, video resolution, ...
There was an input on this already last November:
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-November/029069.html
Clearer?
--
Message: 2
Date: Tue, 29 Mar 2011 15:27:58 +0200
From: Wilhelm Joys Andersen wilhel...@opera.com
To: whatwg@lists.whatwg.org
Subject: [whatwg] details, summary and styling
Message-ID: op.vs3w0wvrm3w...@kunnskapet.oslo.osa
Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes
Hi,
I'm currently writing tests in preparation for Opera's implementation
of details and summary. In relation to this, I have a few
questions
about issues that, as far as I can tell, are currently
undefined in the
specification.
The spec says:
If there is no child summary element [of the details element], the
user agent should provide its own legend (e.g. Details). [1]
How exactly should this legend be provided? Should the user agent add
an implied summary element to the DOM, similar to tbody, a
pseudo-element, or a magic non-element behaving differently from both
of the above? In the current WebKit implementation[2], the UA-provided
legend behaves inconsistently from from an author-provided summary
in the following ways:
* Although it can be styled with rules applying to
summary, it does
not respond to :hover or :first-child.
* With regards to text selection, it behaves more like an input
type='submit' than a user-provided summary. Text within this
implied element may only be selected _together_ with the text
preceding and following it.
* A different mouse cursor is used.
This indicates that it is slightly more magic than I would prefer. I
believe a closer resemblance to an ordinary element would be more
convenient for authors - a ::summary pseudo element with Details as
its content() might be the cleanest approach, although that would
require a few more bytes in the author's stylesheet to cater to both
author- and UA-defined summaries:
summary, ::summary {
color: green;
}
Furthermore, the rendering spec says:
The first container is expected to contain at least one line box,
and that line box is expected to contain a disclosure
widget (typically
a triangle), horizontally positioned within the left padding of the
details element. [3]
For user agents aiming to support the suggested default rendering, how
should the disclosure widget be embedded? Ideally, graphical browsers
should all do this in a similar manner, and in a way that
allows authors
to style these elements to the same extent as any other element.
There are several options:
* A ::marker pseudo element[4].
* A default, non-repeating background image positioned within
the recommended 40 pixel left padding.
* A method similar to
Re: [whatwg] WebSockets and redirects
On 03/30/2011 12:12 AM, Jonas Sicking wrote: But I'm totally fine with punting on this for the future and just disallowing redirects on an API level for now. Yes, I think this is the right thing to do at the moment.
Re: [whatwg] Implementation difficulties for MediaController
On Mon, 28 Mar 2011 05:01:51 +0200, Ian Hickson i...@hixie.ch wrote: It would be helpful if you could describe exactly what is easy and what is hard (that is, glitchy or simply unsupported by common media frameworks) in terms of media synchronisation, in particular along the following axes: * multiple in-band tracks vs multiple independent files As long as all tracks are playing synchronously, in-band tracks is easier to deal with, as synchronization happens for free as a side effect of being able to play any plain audio+video file. Synchronizing independent files is not something that will be supported in all platform media frameworks, but probably possible to fake to sufficient precision. * playing tracks synchronised at different offsets * playing tracks at different rates For independent files, this is again not necessarily natively supported by all platform media frameworks, but as long as they can seek and play at arbitrary rates, it's probably possible to fake to sufficient precision. For in-band tracks, one would have to set up several instances of the same decoding pipeline. * changing any of the above while media is playing vs when it is stopped Having in-band tracks change between being in sync (same offset and rate) and being out of sync (different offset or rate) would be a major head-ache. There is no concept stopped in the current API, only paused, which is exactly like playing (pipeline set up and ready, buffers full) except time is standing still. In other words, it doesn't make a big difference if it's paused or playing. * adding or removing tracks while media is playing vs when it is stopped For independent files, this should not be much different to starting playback in the first place -- everything will stall until data for all tracks is available. However, when enabling another in-band track, enough information must be available for the browser to know if it can reuse an existing pipeline (sharing the demuxer) or if it has to set up another. In other words, the playback rate and offset that the author is intending to use must be known up-front and not be allowed to change later. * changing overall playback rate while a synced set of media is playing Should not be much more difficult than changing playback rates of individual media resources. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] input element's value should not be sanitized during parsing
On 03/12/2011 12:56 AM, Jonas Sicking wrote: inp = document.createElement(input); inp.setAttribute(value, foo\nbar); inp.setAttribute(type, hidden); and inp = document.createElement(input); inp.setAttribute(type, hidden); inp.setAttribute(value, foo\nbar); This does not seem desirable. I can't argue that it's desireable, but it's how the Web works, as I understand it. Gecko doesn't exhibit this behavior and I don't know of any sites that doesn't work in Gecko because of this. FWIW, it does. The first inp.value is 'foobar' while the second is 'foo bar'. See: http://software.hixie.ch/utilities/js/live-dom-viewer/saved/900 Though, I do not think this is related to the initial issue which is about setting attributes while creating the element from the parser. -- Mounir
[whatwg] Submitting datetime values should more clearly specify required timezone syntax
Hi, The spec states for submission of datetime controls that the value must be expressed in the UTC time-zone. It's not clear whether this requires the formatted string to state the timezone as an uppercase Z or +00:00, or whether either is acceptable. Most of the examples illustrate the use of the Z, but elsewhere in the spec, following the definition of the valid global date and time string, it recommends against using the Z. The *best representation of the global date and time string* datetime is the valid global date and time string representing datetime, with the valid time string component being given in its shortest possible form, with the last character of the string not being a U+005A LATIN CAPITAL LETTER Z character (Z), even if the time zone is UTC, and with a U+002B PLUS SIGN character (+) representing the sign of the time-zone offset when the time zone is UTC. http://whatwg.org/C#best-representation-of-the-global-date-and-time-string (Note that the spec defines this best representation... term, but never actually uses it anywhere. It's not clear what the purpose of that definition is.) Opera has always submitted the value with a Z and not +00:00, and we would prefer not to change this for backwards compatibility and ease of parsing on the server side. The spec should define in the value sanitization algorithm for the date and time time state, to express the UTC time-zone using a Z. http://whatwg.org/C#date-and-time-state -- Lachlan Hunt - Opera Software http://lachy.id.au/ http://www.opera.com/
Re: [whatwg] Implementation difficulties for MediaController
On Mar 29, 2011, at 9:05 PM, Ian Hickson wrote: On Tue, 29 Mar 2011, Jer Noble wrote: Contained is Eric and my feedback as to the difficulty of implementing this proposal in Apple's port of WebKit: Thank you very much for your feedback. I'll look into it more tomorrow when I update the spec, but in the meantime I had some additional questions: * playing tracks synchronised at different offsets However, if the in-band tracks will be played at a different time offsets, or at different rates, playback becomes just as inefficient as playing independent files. To implement this we will have to open two instances of a movie, enable different tracks on each, and then play the two instances in sync. Is that acceptable? That is, are you ok with implementing multiple file (or two instances of the same file at different offsets) synchronization? Yes, this would be acceptable. * playing tracks at different rates In addition to the limitation listed above, efficient playback of tracks at different rates will require all tracks to be played in the same direction. Ah, interesting. Is it acceptable to implement multiple playback at different rates if they're all in the same direction, or would you (at least for now) be significantly helped by forcing the playback rates to be the same for all slaved media tracks? It would be significantly easier to implement an across-the-board playback rate for all media elements in a media group. This seems like a reasonable restriction for the first version of the API. * changing any of the above while media is playing vs when it is stopped Modifying the media groups while the media is playing is probably impossible to do without stalling. The media engine may have thrown out unneeded data from disabled tracks and may have to rebuffer that data, even in the case of in-band tracks. That makes sense. There's several ways to handle this; the simplest is probably to say that when the list of synchronised tracks is changed, or when the individual offsets of each track or the individual playback rates of each track are changed, the playback of the entire group should be automatically stopped. Is that sufficient? I would say that, instead, it would be better to treat this as similar to seeking into an unbuffered region of a media file. Some implementers will handle this case better than others, so this seems to be a Quality of Service issue. (In the future, if media frameworks optimise these cases, or if hardware advances sufficiently that even inefficient implementations of this are adequate, we could add a separate flag that controls whether or not this automatic pausing happens.) It seems that this could be determined on the authors' side by pausing before operations that may cause significant buffering delays, without the need for a new flag. From a user's point of view, your proposal seems more complicated than the basic use cases merit. For example, attempting to fix the synchronization of improperly authored media with micro-adjustments of the playback rate isn't likely to be very successful or accurate. The metronome case, while an interesting experiment, would be better served through something like the proposed Audio API. Indeed. The use cases you mention here aren't the driving factor in this design, they're just enabled mostly as a side-effect. The driving factor is to avoid the symmetry problem described below: Slaving multiple media elements' playback rate and current time to a single master media element, Silvia and Eric's proposal, seems to achieve the needs of the broadest use cases. The problem with this design is that it is highly asymetric. The implementation of a media element needs to have basically two modes: slave and master, where the logic for both can be quite different. (Actually, three modes if you count the lone media element case as a separate mode.) This then also spills into the API, where the master is exposing both the network state of its own media, as well as the overall state of playback. We end up having to handle all kinds of special cases, such as what happens when the master track is shorter than a slaved track, or what happens when the master track is paused vs when a slaved track is paused. It's not impossible to do, but it is significantly more messy than simply having a distinct master object and having all the media elements only deal with one mode (two if a lone media element counts as separate), namely the slave mode. Any asymetry is reflected as differences between the controller and the media element. Each media element only has to deal with its own networking state, etc. For an example of why this matters, consider the use case of a movie site with the option of playing movies with a director's commentary track. Some director's commentaries are shorted than the movie (most,
Re: [whatwg] DOM Range: redefining behavior under DOM mutation
On 3/30/11 1:50 PM, Aryeh Gregor wrote: Hmm, wait a sec. I just reviewed the spec for insertBefore(), and I don't see any way this could happen. If anything would prevent it from completing successfully, an exception is supposed to be thrown before anything else is done. Under what conditions could it be removed from the old parent but not added to the new parent? The spec we're looking at is http://dvcs.w3.org/hg/domcore/raw-file/tip/Overview.html#dom-node-insertbefore as of March 26, I assume? This spec is all kinds of broken if you allow userdata handlers and mutation events. For example, consider the following situation: a.insertBefore(b, null); where a.ownerDocument == A, b.ownerDocument == B, A != B, and b has a userdata handler that calls B.adoptNode(b) if b is adopted out of document B. In this situation, following the spec algorithm will insert 'b' as a child of 'a' even though they have different owner documents. That's broken. Or consider a situation where a mutation listener on some ancestor of 'b', or a userdata handler on 'b', performs any operation which puts the DOM into a state where any of the checks in steps 3 or 4 of the algorithm fail. What's an implementation supposed to do then? The way Gecko handles this right now is that if in step 7 script ends up performing any DOM mutations we go back and redo the integrity checks of steps 3 and 4, throwing as needed. We also throw if the owner document is not what it should be after the adoptNode call. But at this point 'b' has already been removed from the old parent. The range is explicitly repositioned by the end of the surroundContents algorithm. That would result in pbFoo/bbbar/b/p but actually, the result in Gecko is pbFoobar/b/p reusing the existingb element. Ah, I misunderstood what the result was. Let me look up what Gecko actually does here and get back to you. -Boris
Re: [whatwg] DOM Range: redefining behavior under DOM mutation
On Wed, Mar 30, 2011 at 2:48 PM, Boris Zbarsky bzbar...@mit.edu wrote: The spec we're looking at is http://dvcs.w3.org/hg/domcore/raw-file/tip/Overview.html#dom-node-insertbefore as of March 26, I assume? This spec is all kinds of broken if you allow userdata handlers and mutation events. Oh, whoops. That's what I forgot about. Ryosuke Niwa of WebKit says that for editing commands, DOM mutation events for editing commands (not sure about things like insertBefore()) get delayed until after the command finishes executing: http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2011-March/030730.html But I'm not too happy trying to spec anything interesting here when you have this DOM mutation stuff hanging around, so for now I think I'll just spec what DOM 2 Range says. I'll leave a note in the spec to revisit this later. Ah, I misunderstood what the result was. Let me look up what Gecko actually does here and get back to you. Thanks.
Re: [whatwg] AppCache feature request: An https manifest should be able to list resources from other https origins.
Fyi: This change has been made in chrome. * respect no-store headers for cross-origin resources (only for HTTPS) * allow HTTPS cross-origin resources to be listed in manifest hosted on HTTPS On Mon, Feb 14, 2011 at 5:04 PM, Michael Nordman micha...@google.comwrote: Fyi... I'm planning on making a change along these lines to chrome soon... * respect no-store headers for cross-origin resources * allow HTTPS cross-origin resources On Tue, Feb 8, 2011 at 3:25 PM, Michael Nordman micha...@google.com wrote: Hi again, Just had an offline discussion about this and I think the answer can be much simpler than what's been proposed so far. All we have to do for cross-origin HTTPS resources is respect the cache-control no-store header. Let me explain the rationale... first let's back up to the motivation for the restrictions on HTTPS. They're there to defeat attacks that involve physical access the the client system, so the attacker cannot look at the cross-origin HTTS data stored in the appcache on disk. But the regular disk cache stores HTTPS data provided the cache-control header doesn't say no-store, so excluding this data from appcaching does nothing to defeat that attack. Maybe the spec changes to make are... 1) Examine the cache-control header for all cross-origin resources (not just HTTPS), and only allow them if they don't contain the no-store directive. 2) Remove the special-case restriction that is currently in place only for HTTPS cross-origin resources. WDYT? On Mon, Feb 7, 2011 at 5:27 PM, Michael Nordman micha...@google.com wrote: On Mon, Feb 7, 2011 at 4:35 PM, Jonas Sicking jo...@sicking.cc wrote: On Mon, Feb 7, 2011 at 3:31 PM, Ian Hickson i...@hixie.ch wrote: On Mon, 7 Feb 2011, Jonas Sicking wrote: On Mon, Jan 31, 2011 at 6:27 PM, Michael Nordman micha...@google.com wrote: But... the risk you outline is not possible... However, with the modification you are proposing, an attacker site could forever pin this page the users app-cache. This means that if there is a security bug in the page, the attacker site could exploit that security problem forever since any javascript in the page will continue to run in the security context of bunnies.com. So all of a sudden the CORS headers that the site added has now had a severe security impact. The bunnies.com page stored in the attacker's appcache will never be loaded into the context of bunnies.com. There are provisions in the the appcache system to prevent that. Those provisions guard against a this type of attack via HTTP. Your proposal means that we forever lock that constraint on the appcache. That is not currently the case. I.e. we'll never be able to say open an iframe using the resource which is available in my appcache or open this cross-site worker using the resource available in my appcache. Or at least we won't ever be able to do that for cross-site resources. That's intentional. We don't want it to be possible to get a cache of a third-party page vulnerable to some sort of XSS attack and then to be able to load that page with the credentials of its origin, since it would make it possible for hostile sites to lock in a vulnerability and keep using it even after the site had fixed the problem. It seems desirable that the third party site could opt in to allowing this. Especially if it can choose which sites should be able to cache it. Which I think is the feature request that Michael starts with in this thread. / Jonas My feature request is for an opt-in mechanism to facilitate cross-origin HTTPS resources. I'm not looking for an opt-in mechanism to allow execution of cached cross-origin resources at this time. Anne mentioned that CORS might be an option for my feature request... and here we are.
Re: [whatwg] WebSockets and redirects
On Tue, 2011-03-29 at 15:12 -0700, Jonas Sicking wrote: I'm totally fine with punting on this for the future and just disallowing redirects on an API level for now. +1. It has proved hard enough to get an acceptably-secure WebSocket interface on the bare minimum of features. Let's not introduce more issues by adding more features that aren't completely necessary now. -- And Clover mailto:a...@doxdesk.com http://www.doxdesk.com gtalk:chat?jid=bobi...@gmail.com
Re: [whatwg] Media elements statistics
Note that the Mozilla implementation I proposed has had a counter proposal by another mozilla developer and is being developed further. See: https://bugzilla.mozilla.org/show_bug.cgi?id=580531 Thanks. Taking a further look at that. Reviving thread... I have an initial patch in webkit (http://trac.webkit.org/changeset/77394) and the chromium work is underway - I wonder what might be a good approach to drive the apis closer together towards a real spec that everyone is happy with? There seems to be a lot of general agreement here (at least in principal :-) that this is needed. We'll be doing a bunch of experimentation once this has landed in chromium. I've landed support for some Mozilla-specific video decoding/painting statistics in Firefox trunk (tracking in Mozilla bug 580531 mentioned above), it should ship in Firefox 5 in a few months. For a few details, see: http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html Regards, Chris Pearce.
Re: [whatwg] Media elements statistics
Please note that I've started a wiki page at http://wiki.whatwg.org/wiki/Video_Metrics to try and collect all ideas around media element statistics. Please add freely! Silvia. On Thu, Mar 31, 2011 at 2:16 PM, Chris Pearce ch...@pearce.org.nz wrote: Note that the Mozilla implementation I proposed has had a counter proposal by another mozilla developer and is being developed further. See: https://bugzilla.mozilla.org/show_bug.cgi?id=580531 Thanks. Taking a further look at that. Reviving thread... I have an initial patch in webkit (http://trac.webkit.org/changeset/77394) and the chromium work is underway - I wonder what might be a good approach to drive the apis closer together towards a real spec that everyone is happy with? There seems to be a lot of general agreement here (at least in principal :-) that this is needed. We'll be doing a bunch of experimentation once this has landed in chromium. I've landed support for some Mozilla-specific video decoding/painting statistics in Firefox trunk (tracking in Mozilla bug 580531 mentioned above), it should ship in Firefox 5 in a few months. For a few details, see: http://blog.pearce.org.nz/2011/03/html5-video-painting-performance.html Regards, Chris Pearce.
