Re: [whatwg] exposing metadata on replaced elements (was: feedback)

[Rick Waldron]

On Monday, September 17, 2012 at 11:22 PM, Silvia Pfeiffer wrote:

> On Tue, Sep 18, 2012 at 1:19 PM, Rick Waldron  wrote:
> > > 
> > > While we have somewhat of a proposal for exposing metadata to media
> > > elements through the W3C media annotations WG [1] with the
> > > getMediaProperty()
> > > 
> > 
> > 
> > If the function is going to retrieve meta data, why is it named with words
> > that don't include "meta" or "data"?
> > 
> > Additionally, is it possible to avoid making the API async? This seems like
> > async for the sake of it. If this data is actually the meta data of the
> > media resource, then it make sense for this to be readily available after
> > any loadedmetadata event and therefore can be accesse via sync API.
> > 
> 
> 
> I agree - which is why I suggested to just use that work as an
> inspiration and not take it at face value. That WG has done extensive
> analysis of media metadata that is exposed by different media formats,
> which is useful to come up with a solution that works for UAs. Their
> solution was not directly meant to be used in UAs, I believe.
> 
> 


That's great news, thanks for the clarification. 

Rick 
> 
> Cheers,
> Silvia.
> 
> 

Re: [whatwg] exposing metadata on replaced elements (was: feedback)

[Silvia Pfeiffer]

On Tue, Sep 18, 2012 at 1:19 PM, Rick Waldron  wrote:
>>
>> While we have somewhat of a proposal for exposing metadata to media
>> elements through the W3C media annotations WG [1] with the
>> getMediaProperty()
>
> If the function is going to retrieve meta data, why is it named with words
> that don't include "meta" or "data"?
>
> Additionally, is it possible to avoid making the API async? This seems like
> async for the sake of it. If this data is actually the meta data of the
> media resource, then it make sense for this to be readily available after
> any loadedmetadata event and therefore can be accesse via sync API.

I agree - which is why I suggested to just use that work as an
inspiration and not take it at face value. That WG has done extensive
analysis of media metadata that is exposed by different media formats,
which is useful to come up with a solution that works for UAs. Their
solution was not directly meant to be used in UAs, I believe.

Cheers,
Silvia.

Re: [whatwg] exposing metadata on replaced elements (was: feedback)

[Rick Waldron]

On Monday, September 17, 2012 at 10:19 PM, Silvia Pfeiffer wrote:

> Hi Ralph, all,
> 
> > On Mon, 11 Jun 2012, Ralph Giles wrote:
> > > 
> > > Recently, we've been considering adding a 'tags' or 'metadata' attribute
> > > to HTML media elements in Firefox, to allow webcontent access to
> > > metadata from the playing media resource. In particular we're interested
> > > in tag data like creator, title, date, and so on.
> > > 
> > > My recollection is that this has been discussed a number of times in the
> > > past, but there was never suffient motivation to support the interface.
> > > Our particular motivation here is webapps that present a media file
> > > library. While it's certainly possible to parse the tag data out
> > > directly with javascript, it's more convenient if the HTML media element
> > > does so, and the underlying platform decoder libraries usually provide
> > > this data already.
> > > 
> > 
> > 
> > My recommendation would be to develop a specification for this (or use the
> > one(s) already available for this purpose), and in that specification
> > define how it is added to HTMLMediaElement, much as you suggest:
> > 
> > > partial interface HTMLMediaElement {
> > > [...]
> > > };
> > > 
> > 
> > 
> > (I don't have the bandwidth to define how to extract this kind of thing
> > from each video format. Even trying to define what little the spec already
> > says has required hours of reading obscure specifications and that's
> > without even testing to see if those specs match reality.)
> > 
> 
> 
> I think we have a more generic problem than just for media elements.
> 
> None of our elements that pull in external resources (including img,
> object/embed, video, audio, track) expose metadata to the Web page and
> the Web developer is required to implement a XHR to get this kind of
> information.
> 
> While we have somewhat of a proposal for exposing metadata to media
> elements through the W3C media annotations WG [1] with the
> getMediaProperty() 
> 
> 

If the function is going to retrieve meta data, why is it named with words that 
don't include "meta" or "data"?

Additionally, is it possible to avoid making the API async? This seems like 
async for the sake of it. If this data is actually the meta data of the media 
resource, then it make sense for this to be readily available after any 
loadedmetadata event and therefore can be accesse via sync API.

This may not it with "the plan"...

Rick

 
> function, we have no such thing for any of the
> other resource types. Also, it is not clear that this function is
> appropriate.
> 
> I suggest we need to develop a generic solution for this problem.
> 
> Cheers,
> Silvia.
> 
> [1] http://www.w3.org/TR/2011/WD-mediaont-api-1.0-2022/#async-api 

[whatwg] exposing metadata on replaced elements (was: feedback)

[Silvia Pfeiffer]

Hi Ralph, all,

> On Mon, 11 Jun 2012, Ralph Giles wrote:
>>
>> Recently, we've been considering adding a 'tags' or 'metadata' attribute
>> to HTML media elements in Firefox, to allow webcontent access to
>> metadata from the playing media resource. In particular we're interested
>> in tag data like creator, title, date, and so on.
>>
>> My recollection is that this has been discussed a number of times in the
>> past, but there was never suffient motivation to support the interface.
>> Our particular motivation here is webapps that present a media file
>> library. While it's certainly possible to parse the tag data out
>> directly with javascript, it's more convenient if the HTML media element
>> does so, and the underlying platform decoder libraries usually provide
>> this data already.
>
> My recommendation would be to develop a specification for this (or use the
> one(s) already available for this purpose), and in that specification
> define how it is added to HTMLMediaElement, much as you suggest:
>
>> partial interface HTMLMediaElement {
>>   [...]
>> };
>
> (I don't have the bandwidth to define how to extract this kind of thing
> from each video format. Even trying to define what little the spec already
> says has required hours of reading obscure specifications and that's
> without even testing to see if those specs match reality.)

I think we have a more generic problem than just for media elements.

None of our elements that pull in external resources (including img,
object/embed, video, audio, track) expose metadata to the Web page and
the Web developer is required to implement a XHR to get this kind of
information.

While we have somewhat of a proposal for exposing metadata to media
elements through the W3C media annotations WG [1] with the
getMediaProperty() function, we have no such thing for any of the
other resource types. Also, it is not clear that this function is
appropriate.

I suggest we need to develop a generic solution for this problem.

Cheers,
Silvia.

[1] http://www.w3.org/TR/2011/WD-mediaont-api-1.0-2022/#async-api

Re: [whatwg] [Web-storage] subdomains / cooperation and limits

[Ian Hickson]

On Mon, 17 Sep 2012, Brian Kardell wrote:
>
> Ian, you hit the nail on the head with the text section that raised the 
> issue but I still am not entirely sure that I understand... Doesn't this 
> imply that in a case like *.wordpress.com would have a (suggested) limit 
> of 5mb combined for all of its tons and tons of subdomains (at least 
> without additional/constant prompting)?

It wouldn't be "constant" prompting, but yes, the spec does suggest that 
if you visit a dozen WordPress-hosted blogs and they all try to load a 
bunch of content onto your machine, you should probably have to give 
consent or at least be aware of what's going on.


> There are a whole lot of what I would call "common" examples like where 
> it seems (to me anyway) unintuitive given the regularity with which this 
> kind of case would happen to think that that is what is actually 
> proposed.

What's the alternative? Allowing any site to overload your machine with 
infinite amounts of content isn't really a viable solution.


> I can understand blocking access to that data pretty easily, but with 
> postMessage, being in the same top-level domain doesn't even matter so 
> it seems that one could just as easily "subvert the limit" that way.

The difference is that getting a new domain costs money, whereas getting a 
subdomain does not. So the cost of attacking someone with subdomains is 
much lower than with domains.


> I think it isn't really implemented that way anywhere though, is it?
> That is, do browsers really share the limit across subdomains like 
> that...

If they do not, they are likely vulnerable to this kind of griefing.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] [Web-storage] subdomains / cooperation and limits

[Brian Kardell]

On Sep 17, 2012 5:22 PM, "Ian Hickson"  wrote:
>
> On Mon, 17 Sep 2012, Brian Kardell wrote:
> >
> > Essentially, x.wordpress.com and y.wordpress.com both allocate and use
> > space - no problem, right?  Access is subject to the browsers -general-
> > [same-origin policy], (leaving aside the ability to document.domain up
> > one), right?  If I have two affliate sites who communicate across an
> > explicit trust via postMessage - is this problematic?  I thought not,
> > and it doesn't seem to be - further - I cannot imagine how it could work
> > otherwise and still be useful for a host of common cases (like the
> > wordpress one I mentioned above).  I have been told that the draft
> > contradicts my understanding, but I don't think so.
>
> I don't really understand your question, but does this answer it?:
>
>
http://www.whatwg.org/specs/web-apps/current-work/multipage/webstorage.html#disk-space-0
>
> --
> Ian Hickson   U+1047E)\._.,--,'``.fL
> http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
> Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Ian, you hit the nail on the head with the text section that raised the
issue but I still am not entirely sure that I understand... Doesn't this
imply that in a case like *.wordpress.com would have a (suggested) limit of
5mb combined for all of its tons and tons of subdomains (at least without
additional/constant prompting)?  There are a whole lot of what I would call
"common" examples like where it seems (to me anyway) unintuitive given the
regularity with which this kind of case would happen to think that that is
what is actually proposed.  If so, I guess I am looking for some kind of
explanation which I haven't really been able to find to help me understand
how that came about.   I can understand blocking access to that data pretty
easily, but with postMessage, being in the same top-level domain doesn't
even matter so it seems that one could just as easily "subvert the limit"
that way.

I think it isn't really implemented that way anywhere though, is it?   That
is, do browsers really share the limit across subdomains like that... am I
just completely misunderstanding what that section is saying?

Re: [whatwg] StringEncoding open issues

[Glenn Maynard]

On Mon, Sep 17, 2012 at 4:17 PM, Anne van Kesteren  wrote:

> Why would we want the API to work different from how it works in
> markup (with  etc.)? Granted it's not super logical, but
> I don't really see why we should make it inconsistent and more
> complicated.
>

That algorithm always allows UTF-16, even when you explicitly say UTF-8.
 That would cause everyone using this API for UTF-8 to inadvertently
perpetuate UTF-16 support.

I hope we wouldn't want to apply the heuristic (#8) or user-locale steps
(#9), and the rest (user overrides, prescanning) is HTML-specific.

-- 
Glenn Maynard

Re: [whatwg] StringEncoding open issues

[Joshua Bell]

On Mon, Sep 17, 2012 at 2:17 PM, Anne van Kesteren  wrote:

> On Mon, Sep 17, 2012 at 11:13 PM, Joshua Bell  wrote:
> > I've attempted to distill the above into the spec in an algorithmic way:
> > http://wiki.whatwg.org/wiki/StringEncoding#TextDecoder
> >
> > English version: If you specify "utf-16" you get endian-agnostic UTF-16
> > encoding support. Failing that, if your encoding matches your BOM it is
> > consumed. Failing *that*, you get whatever behavior falls out of the
> decode
> > algorithm (garbage, error, etc).
>
> Why would we want the API to work different from how it works in
> markup (with  etc.)? Granted it's not super logical, but
> I don't really see why we should make it inconsistent and more
> complicated.
>

That's how the spec started out, so a recap of this thread would give you
the back-and-forth that led here. To summarize:

Having the BOM in the content be higher priority than the coding selected
by the developer was not seen as desirable (see earlier in the thread), and
potentially a source of errors. Selecting encoding via BOM (in general, or
to emulate , etc) was seen as something that could be done in
user code if desired, but unexpected otherwise.

Two desired behaviors remained: (1) developer need for BOM-specified
endian-agnostic UTF-16 encoding similar to ICU's handling that
distinguishes "utf-16" from "utf-16le", and (2) that matching BOMs should
be consumed and not appear in the decoded data.

Re: [whatwg] [Web-storage] subdomains / cooperation and limits

[Ian Hickson]

On Mon, 17 Sep 2012, Brian Kardell wrote:
> 
> Essentially, x.wordpress.com and y.wordpress.com both allocate and use 
> space - no problem, right?  Access is subject to the browsers -general- 
> [same-origin policy], (leaving aside the ability to document.domain up 
> one), right?  If I have two affliate sites who communicate across an 
> explicit trust via postMessage - is this problematic?  I thought not, 
> and it doesn't seem to be - further - I cannot imagine how it could work 
> otherwise and still be useful for a host of common cases (like the 
> wordpress one I mentioned above).  I have been told that the draft 
> contradicts my understanding, but I don't think so.

I don't really understand your question, but does this answer it?:

http://www.whatwg.org/specs/web-apps/current-work/multipage/webstorage.html#disk-space-0

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] StringEncoding open issues

[Anne van Kesteren]

On Mon, Sep 17, 2012 at 11:13 PM, Joshua Bell  wrote:
> I've attempted to distill the above into the spec in an algorithmic way:
> http://wiki.whatwg.org/wiki/StringEncoding#TextDecoder
>
> English version: If you specify "utf-16" you get endian-agnostic UTF-16
> encoding support. Failing that, if your encoding matches your BOM it is
> consumed. Failing *that*, you get whatever behavior falls out of the decode
> algorithm (garbage, error, etc).

Why would we want the API to work different from how it works in
markup (with  etc.)? Granted it's not super logical, but
I don't really see why we should make it inconsistent and more
complicated.


-- 
http://annevankesteren.nl/

Re: [whatwg] StringEncoding open issues

[Joshua Bell]

On Fri, Aug 17, 2012 at 5:19 PM, Jonas Sicking  wrote:

> On Fri, Aug 17, 2012 at 7:15 AM, Glenn Maynard  wrote:
> > On Fri, Aug 17, 2012 at 2:23 AM, Jonas Sicking  wrote:
> >>
> >> >   - If encoding is "utf-16" and the first bytes match 0xFF 0xFE or
> >> > 0xFE
> >> >   0xFF then set current encoding to "utf-16" or "utf-16be"
> >> > respectively and
> >> >   advance the stream past the BOM. The current encoding is used
> >> > until the
> >> >   stream is reset.
> >> >   - Otherwise, if the first bytes match 0xFF 0xFE, 0xFE 0xFF, or
> >> > 0xEF
> >> >   0xBB 0xBF then set current encoding to "utf-16", "utf-16be" or
> >> > "utf-8"
> >> >   respectively and advance the stream past the BOM. The current
> >> > encoding is
> >> >   used until the stream is reset.
> >>
> >> This doesn't sound right. The effect of the rules so far would be that
> >> if you create a decoder and specify "utf-16" as encoding, and the
> >> first bytes in the stream are 0xEF 0xBB 0xBF you'd silently switch to
> >> "utf-8" decoding.
> >
> > I think the scope of the "otherwise" is unclear, and this is meant to be
> > "otherwise (if encoding is not "utf-16")".
>
> Ah, that would make sense. It effectively means "if encoding is not set".
>
> / Jonas
>

I've attempted to distill the above into the spec in an algorithmic way:
http://wiki.whatwg.org/wiki/StringEncoding#TextDecoder

English version: If you specify "utf-16" you get endian-agnostic UTF-16
encoding support. Failing that, if your encoding matches your BOM it is
consumed. Failing *that*, you get whatever behavior falls out of the decode
algorithm (garbage, error, etc).

The JS shim has *not* been updated yet.

Only part of this edit has been live for the last few weeks - apologies to
the Moz folks who were trying to understand what the half-specified
internal useBOM flag was for. Any implementer feedback so far?

[whatwg] [Web-storage] subdomains / cooperation and limits

[Brian Kardell]

Cross posted from w3c list..

I have searched the archives and been unable to resolve this to a great
answer and I just want to make sure that my understanding is correct lest I
have to unwind things later as someone has recently made me second guess
what I thought was a logical understanding of things.  Essentially,
x.wordpress.com and y.wordpress.com both allocate and use space - no
problem, right?  Access is subject to the browsers -general- sop, (leaving
aside the ability to document.domain up one), right?  If I have two
affliate sites who communicate across an explicit trust via postMessage -
is this problematic?  I thought not, and it doesn't seem to be - further -
I cannot imagine how it could work otherwise and still be useful for a host
of common cases (like the wordpress one I mentioned above).  I have been
told that the draft contradicts my understanding, but I don't think so.
Thought that some manufactures/maybe Hixie could set me straight...?

Brian

[whatwg] feedback

[Ian Hickson]

On Thu, 7 Jun 2012, Kit Grose wrote:
> On 06/06/2012, at 7:44 AM, Ian Hickson wrote:
> > On Fri, 13 Jan 2012, Kit Grose wrote:
> >> 
> >> I'd argue that while we did receive in WebM "a common codec" it does 
> >> not enjoy the sort of universal adoption required to be able to 
> >> mandate its support in the spec, so on that logic, I think 
> >> establishing a declarative fallback mechanism is probably required to 
> >> prevent a situation where you cannot write a robust HTML5 page with 
> >> video and without resorting to JS.
> > 
> > I don't think it's time to give up yet, but maybe I'm overly 
> > optimistic...
> 
> Is there any reason why it wouldn't be prudent to render the content of 
> the  element as HTML if the video cannot be played, given that 
> fallback content in the video element is already supported for legacy 
> browsers in the spec:
>
> > Content may be provided inside the video element. User agents should 
> > not show this content to the user; it is intended for older Web 
> > browsers which do not support video, so that legacy video plugins can 
> > be tried, or to show text to the users of these older browsers 
> > informing them of how to access the video contents.
> 
> How are legacy UAs without  support appreciably different from a 
> UA with restrictive or limited  support? Surely the author's 
> intent in either case is delivering the content in a different way or 
> delivering appropriate alternate content.

There are several problems here. One is that designing  to fall 
back to rendering its contents in certain cases leads to a very flickery 
experience -- e.g. on a slow network:

   






   
   

It just doesn't work very well.


> Even if we eventually get a common codec (which I�perhaps overly 
> pessimistically�feel is unlikely), authors are already using the  
> element without supplying whatever that eventual codec will be, and 
> users are suffering without reasonable fallbacks.

Right now two codecs support everyone, and one of those is what you'd have 
to use for the fallback anyway, so when would you need to show fallback if 
the browser supports  at all?


> As it stands, it's almost better (and certainly easier) for authors to 
> default to Flash video and use the existing, declarative fallback 
> behaviour of the  element to use a  element instead. That 
> can't be in the best interest of the open web.

Why not? So long as they provide a non-proprietary non-encumbered 
alternative, there doesn't see to be much difference what the default is.


On Mon, 11 Jun 2012, Ralph Giles wrote:
>
> Recently, we've been considering adding a 'tags' or 'metadata' attribute 
> to HTML media elements in Firefox, to allow webcontent access to 
> metadata from the playing media resource. In particular we're interested 
> in tag data like creator, title, date, and so on.
> 
> My recollection is that this has been discussed a number of times in the 
> past, but there was never suffient motivation to support the interface. 
> Our particular motivation here is webapps that present a media file 
> library. While it's certainly possible to parse the tag data out 
> directly with javascript, it's more convenient if the HTML media element 
> does so, and the underlying platform decoder libraries usually provide 
> this data already.

My recommendation would be to develop a specification for this (or use the 
one(s) already available for this purpose), and in that specification 
define how it is added to HTMLMediaElement, much as you suggest:

> partial interface HTMLMediaElement {
>   [...]
> };

(I don't have the bandwidth to define how to extract this kind of thing 
from each video format. Even trying to define what little the spec already 
says has required hours of reading obscure specifications and that's 
without even testing to see if those specs match reality.)


On Fri, 15 Jun 2012, Charles Pritchard wrote:
> 
> It's a bit of extra work and won't be "compatible" with the img setsrc 
> technique. I think this is just a rehash of having Image.toBlob() and 
> video.frameToBlob() made available.

(I do not know what this refers to. The e-mail in question was top-posted 
with untrimmed context below. If you would like a detailed reply, I 
encourage you to follow the quoting style suggested in the FAQ:

   http://wiki.whatwg.org/wiki/FAQ#Should_I_top-post_or_reply_inline.3F

Thanks.)


On Mon, 18 Jun 2012, Simon Pieters wrote:
> On Fri, 15 Jun 2012 18:01:09 +0200, Ian Hickson  wrote:
> > > 
> > > When preload=none, step 2 of 
> > > http://www.whatwg.org/specs/web-apps/current-work/multipage/the-video-element.html#concept-media-load-resource
> > >  
> > > should not be optional.
> > > 
> > > The effective (internal) preload state should be defined.
> > > 
> > > It should also be defined that with preload=metadata, readyState 
> > > should never go beyond HAVE_CURRENT_DATA, even for a data: URL or 
> > > otherwise fully cached resource.
> >
> > Making it non-confo

Re: [whatwg] MediaController feedback

[Ian Hickson]

On Tue, 28 Aug 2012, Jer Noble wrote:
> On Aug 27, 2012, at 5:02 PM, Ian Hickson  wrote:
> >> 
> >> With JavaScript, it's certainly possible for a page author to play() 
> >> or pause() a slaved media element directly, but that author could 
> >> just as easily remove the media element from the media group / media 
> >> controller.
> >> 
> >>> [...]
> >>> 
> >>> That only works if there's JavaScript doing the removing. The idea 
> >>> here is that this should all work even without any JS, just with UA 
> >>> UI.
> >> 
> >> With just the UA UI, the behavior would be exactly the same [...]
> > 
> > If you remove the element from the media controller, the media 
> > controller's timeline changes.
> 
> So?  In the general case (alternative audio, sign-language) the 
> timelines will be exactly the same.  If there's an edge case where a 
> change in the timeline is a problem, a page author could hide the slaved 
> media element (e.g. "display:none" or "element.muted = true") instead.

Hiding it is a whole different thing than not playing. For example, text 
track events would still fire.


> > It'll be quite common for there to be videos that are not currently 
> > playing, e.g. sign-language tracks.
> 
> I think you're making an incorrect distinction.  The author may not want 
> the sign-language track to *display*.  Pausing the video is one 
> mechanism which achieves that (sort of).  Hiding it is another.  
> Removing the video from the MediaController and pausing it is a third.
> 
> The side effects of this particular mechanism are causing a lot of 
> confusion.  We gave a session at WWDC about the MediaController and 
> collected a lot of developer feedback at the labs, and the general theme 
> was that the API didn't make sense.  Here's a good example of the kinds 
> of bug reports we're seeing: "MediaController play() doesn't work" 
> .

I've renamed play() to unpause() and introduced a new play() that calls 
play() on each slaved media element then calls unpause().

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

[whatwg] Rename the 7-arg arcTo() to ellipseTo()?

[Tab Atkins Jr.]

Heya, we at the SVGWG just resolved today to add equivalents for the
CanvasPathMethods interface arc/ellipse/arcTo commands to the 
element's syntax.

Ideally, we'd be able to use the same names as Path, for minimal
confusion - "" producing a line
from (0,0) to (40,30), then a quarter-arc to (30,40).  However, this
is problematic for arcTo(), as it has both a 5-arg (for circular arcs)
and a 7-arg (for elliptical arcs) version.  For one, this is somewhat
inconsistent with the pattern established by the existing arc() and
ellipse() functions.  For two, and worse for SVG, SVG typically allows
you to omit the name of the path command if the immediately previous
command was the same name.  To do this, the command needs to have a
constant number of arguments, so you can just split on every Nth arg.

So, we're going to have to name the 7-arg version something different.
 The obvious answer is "ellipseTo".  Could we change the
CanvasPathMethods interface to do the same?  To the best of my
knowledge, no implementations exist yet for the 7-arg version, so
there's no backwards-compat issues.

~TJ