Re: [whatwg] Simplified picture element draft
On Tue, 7 Jan 2014, Yoav Weiss wrote: On Tue, Jan 7, 2014 at 6:20 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 1/7/14 12:01 PM, Yoav Weiss wrote: I'd love to get some more details regarding that. I'll start a mozilla.dev.platform thread on the subject, since it's Gecko specific. It's actually not entirely Gecko-specific. Consider a display:none iframe. How should viewport-size-related media queries be evaluated in such a thing? The specs don't define it, as far as I can tell. In fact, the specs don't actually define anything useful for the viewport of a framed document at all, as far as I can see. CSS just assumes a viewport exists, and HTML doesn't define anything about frames setting up a viewport for the document inside them... But say they defined it. How would the viewport of a display:none iframe be defined, exactly? Last I checked, UAs just end up doing wildly different things in this situation. I agree that iframes complicate things, since the parent document's external CSS can modify their viewport dimensions, so there's an inherent race condition there. It's sort of a race, but it settles, right? Since this case is not the majority case, we could bail out of it by delaying the iframe's subresource loading that rely on viewport dimensions until the parent's layout is considered done (e.g. all its head CSS was parsed and applied) The iframe can change dimensions long after that, still (e.g. if it's sized based on other content later in the page, as in a complex table). I'm sure browsers can optimise a bit here, but it's not clear to me that there's anything we need to explicitly require. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Simplified picture element draft
On Wed, Jan 8, 2014 at 10:05 AM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Wed, Jan 8, 2014 at 9:47 AM, Yoav Weiss y...@yoav.ws wrote: On Wed, Jan 8, 2014 at 6:18 PM, Maciej Stachowiak m...@apple.com wrote: - It seems this draft allows arbitrary media queries, with only a subset expected to give best performance (by correctly informing the preload scanner). In my opinion, this is worse than the alternative of only supporting the media queries that could plausibly be integrated with preload scanning. Creating a programming interface with a performance cliff - a point beyond which performance suddenly gets significantly worse for what seems like a small incremental change - is generally a bad idea. And the use cases for fully general media queries seem much more speculative than the ones for the simplified subset. I'm not opposed to having a white-list of performance-safe authorized media queries, and extending that list in the future if necessary, but we need to make sure that it's extensible without causing the content to break in browsers where the new MQs are not supported. As far as I can tell, the current syntax can support this. Right. We can't just ignore the MQ, as that's unfriendly. We could maybe ignore the source, but that feels like too much. Having it evaluated on the main thread is slower, but it's guaranteed to *work* in older browsers, which is nice. And, as previously stated, *every* MQ beyond resolution is preloader-unfriendly in at least some cases. I'd like to avoid making this complicated for authors as much as possible. Having to worry about if legacy browsers will display an image more slowly is much less troublesome than wondering if legacy browsers will display it at all; in most cases, you don't have to worry about it at all, because preloading is a nice benefit but not a necessity. Note, too, that though the full set of MQs is only for niche cases, I can certainly come up with reasonable cases nonetheless, particularly as we introduce new device-capability MQs. For example, you could switch from a GIF to a still picture based on the updates MQ that we're considering adding, or use a higher-contrast version based on whatever MQ we end up developing for detecting high-contrast needs. Disallowing these entirely just means that either devs will use JS to switch their images (then why did we spend all this effort on a markup-based solution?) or they just won't cater to those needs at all. The way we'd likely implement this feature in Blink is in couple stages. In the first stage, we'd implement the portions of the specification that don't involve media queries because we'll be able to provide developers a high-quality implementation of that part of the spec, including preload scanning. In the second stage, we'd implement the parts of the specification that relate to media queries. We'll likely wait to implement the second stage until we're able to provide a high-quality implementation, which means waiting until we're able to parse and process media queries in the preload scanner without joining the main thread. Adam
Re: [whatwg] Simplified picture element draft
On 1/22/14 3:54 PM, Adam Barth w...@adambarth.com wrote: The way we'd likely implement this feature in Blink is in couple stages. In the first stage, we'd implement the portions of the specification that don't involve media queries because we'll be able to provide developers a high-quality implementation of that part of the spec, including preload scanning. In the second stage, we'd implement the parts of the specification that relate to media queries. We'll likely wait to implement the second stage until we're able to provide a high-quality implementation, which means waiting until we're able to parse and process media queries in the preload scanner without joining the main thread. Adam Meanwhile, is there a way in which all vendors can prevent their pre-loaders from preloading (and loading at all for that matter) any img that has either: An html5 hidden attribute, or display:none; both inline or via a css class in the head, which is the current behavior of the object element in Webkit: https://twitter.com/pornelski/status/405704147678535680 I think it's safe to say that, for the browser to load assets marked as 'display:none;' makes little sense, and overrides a developer's ability to control when his/her assets are supposed to load. Browsers loading stuff that's voluntarily set as hidden is a bad performance predisposition, as well as a waste of bandwidth, especially when it comes to responsive mobile sites where the context of hidden *really* means hidden as in: Do *not* load unless actually displayed. Bruno
Re: [whatwg] Simplified picture element draft
On Mon, 13 Jan 2014 07:57:45 +0100, Shawn Jansepar shawn...@gmail.com wrote: Hi everyone, I spent a bit of time writing a polyfill for the picture element proposal that has been discussed in this thread. Feel free to check it out here: https://github.com/jansepar/picturefill. I was surprised to see one hadn't been created yet so I decided to give it a go. Any feedback is much appreciated! It looks like your implementation doesn't follow the specification. For instance, your implementation sorts the candidates while the specification uses markup order. Personally, I would like to see an implementation that implements the spec literally in order to better assess whether the specification is good or not. I'm not saying that you have any obligation to do that, though; maybe you have different goals. :-) -- Simon Pieters Opera Software
Re: [whatwg] Simplified picture element draft
Hi everyone, I spent a bit of time writing a polyfill for the picture element proposal that has been discussed in this thread. Feel free to check it out here: https://github.com/jansepar/picturefill. I was surprised to see one hadn't been created yet so I decided to give it a go. Any feedback is much appreciated! ~Shawn
Re: [whatwg] Simplified picture element draft
On Dec 31, 2013, at 7:17 AM, Yoav Weiss y...@yoav.ws wrote: On Mon, Nov 25, 2013 at 5:33 PM, Adam Barth w...@adambarth.com wrote: Is there an editor's draft or some other relatively self-contained write-up that I could review? Tab has rewritten the picture spec to match the latest proposal. You could review it at http://picture.responsiveimages.org/ This approach seems cleaner than src-n. I'll try to read it in more detail, but a few initial comments: - Is there any reason not to allow the sizes= attribute and the extended definition of srcset= on img as well as source? It seems like the picture wrapper is not helpful in cases where you only use one source element. - It seems this draft allows arbitrary media queries, with only a subset expected to give best performance (by correctly informing the preload scanner). In my opinion, this is worse than the alternative of only supporting the media queries that could plausibly be integrated with preload scanning. Creating a programming interface with a performance cliff - a point beyond which performance suddenly gets significantly worse for what seems like a small incremental change - is generally a bad idea. And the use cases for fully general media queries seem much more speculative than the ones for the simplified subset. Regards, Maciej
Re: [whatwg] Simplified picture element draft
On Wed, Jan 8, 2014 at 9:18 AM, Maciej Stachowiak m...@apple.com wrote: On Dec 31, 2013, at 7:17 AM, Yoav Weiss y...@yoav.ws wrote: On Mon, Nov 25, 2013 at 5:33 PM, Adam Barth w...@adambarth.com wrote: Is there an editor's draft or some other relatively self-contained write-up that I could review? Tab has rewritten the picture spec to match the latest proposal. You could review it at http://picture.responsiveimages.org/ This approach seems cleaner than src-n. I'll try to read it in more detail, but a few initial comments: - Is there any reason not to allow the sizes= attribute and the extended definition of srcset= on img as well as source? It seems like the picture wrapper is not helpful in cases where you only use one source element. Nope, no reason other than I was trying to minimize initial changes in the spec. I'm totally happy to add it, for the reason you list, and so that an early implementation of just img srcset will be compatible with a fuller picture implementation. - It seems this draft allows arbitrary media queries, with only a subset expected to give best performance (by correctly informing the preload scanner). In my opinion, this is worse than the alternative of only supporting the media queries that could plausibly be integrated with preload scanning. Creating a programming interface with a performance cliff - a point beyond which performance suddenly gets significantly worse for what seems like a small incremental change - is generally a bad idea. And the use cases for fully general media queries seem much more speculative than the ones for the simplified subset. The set of MQs that can be done on the preloader thread seems ill-defined and potentially growing. Even within the set that the spec currently requires, we've gotten feedback that those MQs are possible in some cases but not others (within an iframe you can't tell what the viewport dimensions are until the outer document has finished initial layout). ~TJ
Re: [whatwg] Simplified picture element draft
On Wed, Jan 8, 2014 at 6:18 PM, Maciej Stachowiak m...@apple.com wrote: On Dec 31, 2013, at 7:17 AM, Yoav Weiss y...@yoav.ws wrote: On Mon, Nov 25, 2013 at 5:33 PM, Adam Barth w...@adambarth.com wrote: Is there an editor's draft or some other relatively self-contained write-up that I could review? Tab has rewritten the picture spec to match the latest proposal. You could review it at http://picture.responsiveimages.org/ This approach seems cleaner than src-n. I'll try to read it in more detail, but a few initial comments: - Is there any reason not to allow the sizes= attribute and the extended definition of srcset= on img as well as source? It seems like the picture wrapper is not helpful in cases where you only use one source element. No reason. We have an open issue to add that [1] - It seems this draft allows arbitrary media queries, with only a subset expected to give best performance (by correctly informing the preload scanner). In my opinion, this is worse than the alternative of only supporting the media queries that could plausibly be integrated with preload scanning. Creating a programming interface with a performance cliff - a point beyond which performance suddenly gets significantly worse for what seems like a small incremental change - is generally a bad idea. And the use cases for fully general media queries seem much more speculative than the ones for the simplified subset. I'm not opposed to having a white-list of performance-safe authorized media queries, and extending that list in the future if necessary, but we need to make sure that it's extensible without causing the content to break in browsers where the new MQs are not supported. As far as I can tell, the current syntax can support this. Regards, Maciej [1] https://github.com/ResponsiveImagesCG/picture-element/issues/90
Re: [whatwg] Simplified picture element draft
On Wed, Jan 8, 2014 at 9:47 AM, Yoav Weiss y...@yoav.ws wrote: On Wed, Jan 8, 2014 at 6:18 PM, Maciej Stachowiak m...@apple.com wrote: - It seems this draft allows arbitrary media queries, with only a subset expected to give best performance (by correctly informing the preload scanner). In my opinion, this is worse than the alternative of only supporting the media queries that could plausibly be integrated with preload scanning. Creating a programming interface with a performance cliff - a point beyond which performance suddenly gets significantly worse for what seems like a small incremental change - is generally a bad idea. And the use cases for fully general media queries seem much more speculative than the ones for the simplified subset. I'm not opposed to having a white-list of performance-safe authorized media queries, and extending that list in the future if necessary, but we need to make sure that it's extensible without causing the content to break in browsers where the new MQs are not supported. As far as I can tell, the current syntax can support this. Right. We can't just ignore the MQ, as that's unfriendly. We could maybe ignore the source, but that feels like too much. Having it evaluated on the main thread is slower, but it's guaranteed to *work* in older browsers, which is nice. And, as previously stated, *every* MQ beyond resolution is preloader-unfriendly in at least some cases. I'd like to avoid making this complicated for authors as much as possible. Having to worry about if legacy browsers will display an image more slowly is much less troublesome than wondering if legacy browsers will display it at all; in most cases, you don't have to worry about it at all, because preloading is a nice benefit but not a necessity. Note, too, that though the full set of MQs is only for niche cases, I can certainly come up with reasonable cases nonetheless, particularly as we introduce new device-capability MQs. For example, you could switch from a GIF to a still picture based on the updates MQ that we're considering adding, or use a higher-contrast version based on whatever MQ we end up developing for detecting high-contrast needs. Disallowing these entirely just means that either devs will use JS to switch their images (then why did we spend all this effort on a markup-based solution?) or they just won't cater to those needs at all. ~TJ
Re: [whatwg] Simplified picture element draft
On Mon, Jan 6, 2014 at 6:47 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 1/6/14 12:42 PM, Yoav Weiss wrote: I don't know much about Gecko's implementation of the preloader, so I could be wrong, but from looking at nsHtml5TreeOpExecutor, it seems like in Gecko (similarly to current Blink) the preloader's products are sent to the main thread before the actual resource loading takes place. That's correct. But the main thread may _also_ not be in a position to evaluate, or even parse, a media query that early in the load process in Gecko Hmm. I'd love to get some more details regarding that. I'll start a mozilla.dev.platform thread on the subject, since it's Gecko specific.
Re: [whatwg] Simplified picture element draft
On 1/7/14 12:01 PM, Yoav Weiss wrote: I'd love to get some more details regarding that. I'll start a mozilla.dev.platform thread on the subject, since it's Gecko specific. It's actually not entirely Gecko-specific. Consider a display:none iframe. How should viewport-size-related media queries be evaluated in such a thing? The specs don't define it, as far as I can tell. In fact, the specs don't actually define anything useful for the viewport of a framed document at all, as far as I can see. CSS just assumes a viewport exists, and HTML doesn't define anything about frames setting up a viewport for the document inside them... But say they defined it. How would the viewport of a display:none iframe be defined, exactly? Last I checked, UAs just end up doing wildly different things in this situation. -Boris
Re: [whatwg] Simplified picture element draft
On Tue, 7 Jan 2014, Boris Zbarsky wrote: On 1/7/14 12:01 PM, Yoav Weiss wrote: I'd love to get some more details regarding that. I'll start a mozilla.dev.platform thread on the subject, since it's Gecko specific. It's actually not entirely Gecko-specific. Consider a display:none iframe. How should viewport-size-related media queries be evaluated in such a thing? The specs don't define it, as far as I can tell. In fact, the specs don't actually define anything useful for the viewport of a framed document at all, as far as I can see. CSS just assumes a viewport exists, and HTML doesn't define anything about frames setting up a viewport for the document inside them... But say they defined it. How would the viewport of a display:none iframe be defined, exactly? Last I checked, UAs just end up doing wildly different things in this situation. IMHO this should probably be in the CSS spec, but I'm happy to spec something if CSS doesn't want to pick this up. It's probably just a matter of saying, in the rendering section, that nested browsing contexts that aren't _being_rendered_ (or that are frames outside framesets) get a 0x0 viewport. (Viewports that are being rendered seem to have an unambiguous dimension, but let me know if I'm missing something here too.) Right now, Chrome and Firefox seem to report 0 as the body.offsetWidth for a hidden iframe. (Couldn't get a result for Safari; don't have IE here.) http://software.hixie.ch/utilities/js/live-dom-viewer/saved/2725 For Chrome (since it puts data: in the wrong origin) http://software.hixie.ch/utilities/js/live-dom-viewer/saved/2726 For Firefox (since for some reason the message in 2725 isn't sent?) Can you elaborate on the widly different things? What should I be testing? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Simplified picture element draft
On 1/7/14 2:24 PM, Ian Hickson wrote:
It's probably just a matter of saying, in the rendering section, that nested
browsing contexts that
aren't _being_rendered_ (or that are frames outside framesets) get a
0x0 viewport.
That doesn't seem unreasonable.
(Viewports that are being rendered seem to have an
unambiguous dimension
Actually, nothing defines this either. It should be the content box of
the iframe, assuming iframes can't get fragmented. If they can be
fragmented, I have no idea.
Right now, Chrome and Firefox seem to report 0 as the body.offsetWidth for
a hidden iframe. (Couldn't get a result for Safari; don't have IE here.)
http://software.hixie.ch/utilities/js/live-dom-viewer/saved/2725
For Chrome (since it puts data: in the wrong origin)
http://software.hixie.ch/utilities/js/live-dom-viewer/saved/2726
For Firefox (since for some reason the message in 2725 isn't sent?)
Can you elaborate on the widly different things? What should I be testing?
How about this testcase:
!DOCTYPE html
style
div { color: purple; }
/style
link rel=stylesheet href=data:text/css,%23one { color: yellow; }
media=(min-width: 300px)
link rel=stylesheet href=data:text/css,%23two { color: yellow; }
media=(max-width: 300px)
div id=one/div
div id=two/div
script
window.onload = function() {
alert(getComputedStyle(document.getElementById(one), ).color +
| +
getComputedStyle(document.getElementById(two), ).color);
}
/script
and variants on it. From my testing, in a display:none iframe:
1) Presto-based Opera treats the viewport width as 300px for purposes
of the media queries and reports a document.body.offsetWidth of 0.
2) Blink and WebKit both seems to treat the viewport width as 0 when
measured using either method.
3) Gecko doesn't allow creation of computed style objects in an iframe
that's not being rendered and reports a document.body.offsetWidth of 0.
4) IE11 seems to do what Blink/WebKit do.
-Boris
Re: [whatwg] Simplified picture element draft
On 1/7/14 3:06 PM, Boris Zbarsky wrote:
4) IE11 seems to do what Blink/WebKit do.
To test this, you'll need a testcase more like:
style
div { color: purple; }
@media (min-width: 300px) { #one { color: yellow; } }
@media (max-width: 300px) { #two { color: yellow; } }
/style
-Boris
Re: [whatwg] Simplified picture element draft
On Tue, Jan 7, 2014 at 6:20 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 1/7/14 12:01 PM, Yoav Weiss wrote: I'd love to get some more details regarding that. I'll start a mozilla.dev.platform thread on the subject, since it's Gecko specific. It's actually not entirely Gecko-specific. Consider a display:none iframe. How should viewport-size-related media queries be evaluated in such a thing? The specs don't define it, as far as I can tell. In fact, the specs don't actually define anything useful for the viewport of a framed document at all, as far as I can see. CSS just assumes a viewport exists, and HTML doesn't define anything about frames setting up a viewport for the document inside them... But say they defined it. How would the viewport of a display:none iframe be defined, exactly? Last I checked, UAs just end up doing wildly different things in this situation. I agree that iframes complicate things, since the parent document's external CSS can modify their viewport dimensions, so there's an inherent race condition there. But this is an issue that applies to any solution that relies on viewport dimensions for resource loading. Since this case is not the majority case, we could bail out of it by delaying the iframe's subresource loading that rely on viewport dimensions until the parent's layout is considered done (e.g. all its head CSS was parsed and applied)
Re: [whatwg] Simplified picture element draft
On 1/7/14 4:28 PM, Yoav Weiss wrote: Since this case is not the majority case, we could bail out of it by delaying the iframe's subresource loading that rely on viewport dimensions until the parent's layout is considered done That seems fairly undesirable, actually, though limiting it only to the things that have media queries might be better... (e.g. all its head CSS was parsed and applied) And all its HTML parsed (including whatever onload scripts it runs) and layout on that completed, right? This seems like it serializes things a lot... But also some of the same constraints apply to the toplevel viewport, not just subframes (e.g. its dimensions can change based on what the browser UI does, etc). At some point you just accept that the dimensions you have are what you have, even if they might change in the future. -Boris
Re: [whatwg] Simplified picture element draft
On Tue, Jan 7, 2014 at 1:53 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 1/7/14 4:28 PM, Yoav Weiss wrote: Since this case is not the majority case, we could bail out of it by delaying the iframe's subresource loading that rely on viewport dimensions until the parent's layout is considered done That seems fairly undesirable, actually, though limiting it only to the things that have media queries might be better... Yoav said [resources] that rely on viewport dimensions, so yes, it would only be MQs that query viewport size (or sizes='' attribute that use vw/vh units). (e.g. all its head CSS was parsed and applied) And all its HTML parsed (including whatever onload scripts it runs) and layout on that completed, right? This seems like it serializes things a lot... Only to the extent necessary. If you're relying on viewport information, you gotta wait for your viewport to get sized. There's no way around that, except not relying on viewport information. The most common use-cases of picture don't use viewport info, and so are friendly to this stuff. But also some of the same constraints apply to the toplevel viewport, not just subframes (e.g. its dimensions can change based on what the browser UI does, etc). At some point you just accept that the dimensions you have are what you have, even if they might change in the future. Indeed, and that's also fine. Small changes in viewport size from browser UI should only rarely have an impact on which resource is downloaded, anyway. (And adding a bit of hysteresis would solve those, too.) ~TJ
Re: [whatwg] Simplified picture element draft
On Tue, Jan 7, 2014 at 11:13 PM, Tab Atkins Jr. jackalm...@gmail.comwrote: On Tue, Jan 7, 2014 at 1:53 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 1/7/14 4:28 PM, Yoav Weiss wrote: Since this case is not the majority case, we could bail out of it by delaying the iframe's subresource loading that rely on viewport dimensions until the parent's layout is considered done That seems fairly undesirable, actually, though limiting it only to the things that have media queries might be better... Yoav said [resources] that rely on viewport dimensions, so yes, it would only be MQs that query viewport size (or sizes='' attribute that use vw/vh units). (e.g. all its head CSS was parsed and applied) And all its HTML parsed (including whatever onload scripts it runs) and layout on that completed, right? This seems like it serializes things a lot... Only to the extent necessary. If you're relying on viewport information, you gotta wait for your viewport to get sized. There's no way around that, except not relying on viewport information. The most common use-cases of picture don't use viewport info, and so are friendly to this stuff. One addition - Waiting for the parent document's onload seems a little too careful to me. At least some of the onus can also be on authors embedding the iframe (e.g. authors can declare the iframe's dimensions relatively early in the parent document's CSS, or at least avoid modifying its dimensions later on). But also some of the same constraints apply to the toplevel viewport, not just subframes (e.g. its dimensions can change based on what the browser UI does, etc). At some point you just accept that the dimensions you have are what you have, even if they might change in the future. Indeed, and that's also fine. Small changes in viewport size from browser UI should only rarely have an impact on which resource is downloaded, anyway. (And adding a bit of hysteresis would solve those, too.) ~TJ
Re: [whatwg] Simplified picture element draft
On Sat, 04 Jan 2014 06:36:27 -, Adam Barth w...@adambarth.com wrote: In order for the HTMLPreloadScanner to issue preload requests for picture elements, the HTMLPreloadScanner would need to be able to evaluate arbitrary media requests. That's difficult to do without joining the main thread because the media query engine works only on the main thread. Solution I've suggested originally was that when the selection algorithm encounters a media query it cannot evaluate yet, it aborts selection, waits until conditions change and retries selection form the beginning. This means that: * all images that can be selected by the preloader will be selected, and they'll be selected as soon as it is possible, * browser will never load any irrelevant image, * browsers can optimize when and which MQs match without affecting correctness. For example: picture source media=resolution:2dppx src=first source media=max-width:px src=second source media=super-complex-media-query-to-evaluate: on-main-thread-only src=third /picture in an iframe without layout. 1. If you know resolution and the first MQ matches, then load src=first immediately. Done! 2. If you don't know viewport size then wait until any conditions change (i.e. either viewport size becomes known OR resolution changes) and go to step 1. 3. If the second MQ matches then load src=second immediately (might still happen in the preloader). Done! 4. If you can't evaluate complex MQ in the preloader, then wait until control goes back to the main thread and go to step 1. By wait here I mean the selection algorithm is deferred for the given picture only, and nothing else is blocked. Equivalent of it in the current spec would be something like this: Before step 7 in http://picture.responsiveimages.org/#update-source-sets add: 6b. If child has a media attribute, and its value is a valid media query which UA temporarily cannot evaluate then exit this sub-algorithm and /select an image source/ again after a UA-specific delay. temporarily cannot evaluate is completely up to UA. It may mean unknown sizes in iframes, it may mean non-trivial queries in the preloader, etc. UA-specific delay could be waiting for any media query in the picture to change, or it could simply mean ignoring the picture in the preloader and doing evaluation properly on the main thread/when layout is calculated, etc. exit this sub-algorithm will either cause an earlier source that has unambiguously matched to be loaded or empty source set will cause selection algorithm to do nothing. -- regards, Kornel
Re: [whatwg] Simplified picture element draft
On 1/6/14 12:42 PM, Yoav Weiss wrote: I don't know much about Gecko's implementation of the preloader, so I could be wrong, but from looking at nsHtml5TreeOpExecutor, it seems like in Gecko (similarly to current Blink) the preloader's products are sent to the main thread before the actual resource loading takes place. That's correct. But the main thread may _also_ not be in a position to evaluate, or even parse, a media query that early in the load process in Gecko -Boris
Re: [whatwg] Simplified picture element draft
On Sat, Jan 4, 2014 at 7:36 AM, Adam Barth w...@adambarth.com wrote: On Tue, Dec 31, 2013 at 7:17 AM, Yoav Weiss y...@yoav.ws wrote: On Mon, Nov 25, 2013 at 5:33 PM, Adam Barth w...@adambarth.com wrote: Is there an editor's draft or some other relatively self-contained write-up that I could review? Tab has rewritten the picture spec to match the latest proposal. You could review it at http://picture.responsiveimages.org/ Thanks. I'm still concerned about the issue I raised with the original picture proposal in April 2012: ---8--- In order for the HTMLPreloadScanner to issue preload requests for picture elements, the HTMLPreloadScanner would need to be able to evaluate arbitrary media requests. That's difficult to do without joining the main thread because the media query engine works only on the main thread. ---8--- https://groups.google.com/a/chromium.org/d/msg/blink-dev/MlE9vYVUlzg/lohBvySEIegJ The proposal tries to address this issue in http://picture.responsiveimages.org/#preloader by providing a whitelist of media queries that must be recognized by the preload scanner and giving implementations license to evaluate other media queries incorrectly. In principle, we could implement these requirements in Blink, but my guess is that we won't be able to implement them anytime soon. For example, today we can't even parse media queries on the preload scanner thread. We're likely to improve our ability to process CSS on background threads in order to make existing content faster. It's possible in several years once we've developed fancier technology this design will look more feasible. Thanks for the review. I'd like to better understand what would be required to implement off-the-main-thread MQ evaluation in Blink. Since it's a Blink specific discussion, I'll start a Blink-dev thread on the subject. In the meantime, if you're interested in seeing implementations ship in the near term, I would focus on designs that don't require evaluating media queries in the preload scanner, even if that means we aren't able to address all the use cases that picture addresses. Adam
Re: [whatwg] Simplified picture element draft
On Sat Jan 04 2014 at 1:19:14 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 1/4/14 3:16 PM, Yoav Weiss wrote: Thanks for the review. I'd like to better understand what would be required to implement off-the-main-thread MQ evaluation in Blink. Since it's a Blink specific discussion, I'll start a Blink-dev thread on the subject. Note that Gecko has similar constraints here as far as I know, at the moment. If necessary for simplification, we could always relax the requirement that certain MQs be evaluatable on the preload scanner thread, and allow preloaders to bail on *any* MQs. That would still allow the basic use-cases to work nicely with the preloader, while allowing implementations to gradually optimize over time. ~TJ
Re: [whatwg] Simplified picture element draft
On Thu, Jan 2, 2014 at 12:28 AM, Ilya Grigorik i...@igvita.com wrote: On Tue, Dec 31, 2013 at 7:17 AM, Yoav Weiss y...@yoav.ws wrote: On Mon, Nov 25, 2013 at 5:33 PM, Adam Barth w...@adambarth.com wrote: Is there an editor's draft or some other relatively self-contained write-up that I could review? Tab has rewritten the picture spec to match the latest proposal. You could review it at http://picture.responsiveimages.org/ Kicking off 2014 on the right foot... awesome. :-) A couple of quick questions: a) Example 4 srcset uses w units, whereas Example 5 omits the w. Guessing, typos.. but want to be sure. Typos indeed. Good catch. b) What happens when source sizes= and img width= height= are both specified? Are the width/height on img ignored? AFAICT this is not currently specified, but it should be. I've opened an issue [1] regarding it. [1] https://github.com/ResponsiveImagesCG/picture-element/issues/82
Re: [whatwg] Simplified picture element draft
On Mon, Nov 25, 2013 at 5:33 PM, Adam Barth w...@adambarth.com wrote: Is there an editor's draft or some other relatively self-contained write-up that I could review? Tab has rewritten the picture spec to match the latest proposal. You could review it at http://picture.responsiveimages.org/ Thanks! Yoav
Re: [whatwg] Simplified picture element draft
On Wed, 27 Nov 2013 00:48:56 -, Simon Pieters sim...@opera.com wrote: You introduce a proxy that needs to be tested to see that it works in different scenarios (e.g. removing an attribute, that events are forwarded properly, that it does not affect parts it shouldn't like document.images, that the context menu works, etc.). You introduce a (or two) new fallback mechanism. You haven't specified that picture should be able to be drawn on a canvas in 2d (and WebGL?). Thanks, very good examples. Now I understand (although I wish specifying it exactly like img would make that easy enough). -- regards, Kornel
Re: [whatwg] Simplified picture element draft
On Mon, 25 Nov 2013 12:48:42 +0100, Kornel Lesiński kor...@geekhood.net wrote: The advantage of the scheme that zcorpan proposed is that there is no magic proxy; we just add a capability to img to select its source using more than just a src attribute. This has better fallback than your design and is easier to implement. I believe that from testing perspective both approaches are equivalent. I think they are not equivalent. You introduce a proxy that needs to be tested to see that it works in different scenarios (e.g. removing an attribute, that events are forwarded properly, that it does not affect parts it shouldn't like document.images, that the context menu works, etc.). You introduce a (or two) new fallback mechanism. You haven't specified that picture should be able to be drawn on a canvas in 2d (and WebGL?). You haven't specified the crossorigin attribute. You haven't specified that picture provides a paint source for CSS's element() feature. You haven't specified that picture participates in http://www.whatwg.org/html#using-the-a-element-to-define-a-command You haven't specified what the origin is for picture. You haven't specified that createImageBitmap() accepts picture. You haven't specified that picture should default its .draggable attribute to true. I'm sure I've missed a few things, but I think my proposal avoids the issues. The spec I propose *is* only another way to control src of an image. The only difference is that I don't expose the img to scripts. What is the motivation? That may make it even simpler, because you can't have odd cases like author moving/removing the controlling img or setting values directly on img that conflict with picture's definitions. I don't see what would conflict in my proposal. -- Simon Pieters Opera Software
Re: [whatwg] Simplified picture element draft
This is a part of the proposal that got people excited. The linked document defines picture's source selection algorithm. It contains some parts that I'm not sure have a consensus around them yet: * It defines picture as controlling img, where earlier on this list we discussed mostly the opposite (img querying its parent picture, if one exists) * It defines img as a part of picture's shadow DOM, which we need to see how it fits with having fallback img elements (which are necessary in the near future). This proposal does contain srcset as a subcomponent, but it's not the same srcset as defined in the HTML spec, but a modified version based on improvements from the src-N spec. (that cover the variable-width images use-case) The proposal will also require some changes to img and specifically, when not created by JS, img will have to avoid loading of resources until the element is added to the DOM, and can see if its direct parent is picture. If the parent is picture, img would then query the parent (or wait to be controlled by its parent), otherwise, it'll load its resources as usual. On Mon, Nov 25, 2013 at 8:33 AM, Adam Barth w...@adambarth.com wrote: I'm trying to sort out from the many messages on this topic which proposal I should read in detail. Is the one linked below the one that people are most excited about? I'm I correct in understand that this proposal contains srcset as a subcomponent? Thanks, Adam On Sat, Nov 23, 2013 at 3:04 PM, Kornel Lesiński kor...@geekhood.net wrote: I've written down proposal for the simplified source selection algorithm: http://geekhood.net/picture-element.html This also includes variant of the idea from the recent picture redux proposal to use an actual img element as the basis for the picture element definition. This draft doesn't include all features of src-N *yet*, but I expect this to be added either via extended srcset syntax to something like source sizes once there's consensus how to approach this. To simplify implementation even further I've allowed UAs to flatten fallback DOM to a plaintext string (in case they need to emulate img alt for existing screen readers or accessibility APIs). I've dropped usemap. It could be added, but I'm not sure if there is need for it. I've specified very few IDL attributes. This area may need to be extended. -- regards, Kornel
Re: [whatwg] Simplified picture element draft
On 25 November 2013 08:00:10 Yoav Weiss y...@yoav.ws wrote: It contains some parts that I'm not sure have a consensus around them yet: * It defines picture as controlling img, where earlier on this list we discussed mostly the opposite (img querying its parent picture, if one exists) Controlling image is a great idea. It greatly simplifies the spec and hopefully implementations as well. I chose not to expose that implementation detail, assuming that one day (when all UAs, crawlers implement it) we will not need explicit img fallback any more. If picture was explicitly controlled by img then websites could start depending on that behavior, and we'd be stuck with it. OTOH picture can have native DOM interface and still reuse img for implementation. * It defines img as a part of picture's shadow DOM, which we need to see how it fits with having fallback img elements (which are necessary in the near future). I've added section about preloader. The img in fallback content should be ignored by the preloader. It's purely for picture-less UAs. I do wonder however if fallback img should be used as equivalent of a source to save authors a bit of repetition. (in selection algorithm the first step would be for each source or img child...) or perhaps be used as last-resort fallback when no source matches (step 2 of the algorithm). This proposal does contain srcset as a subcomponent, but it's not the same srcset as defined in the HTML spec, but a modified version based on improvements from the src-N spec. (that cover the variable-width images use-case) Indeed. This part of the spec isn't ironed out yet. The proposal will also require some changes to img and specifically, when not created by JS, img will have to avoid loading of resources until the element is added to the DOM, and can see if its direct parent is picture. If the parent is picture, img would then query the parent (or wait to be controlled by its parent), otherwise, it'll load its resources as usual. I've specified something like that. I think it can be as simple as a flag that preload scanner uses internally. I think we don't need to add any runtime behavior changes for this, as scripts constructing picture will not insert explicit fallback img node - it makes more sense to rely on picture polyfill instead (that will use img with correct src from the start). -- regards, Kornel
Re: [whatwg] Simplified picture element draft
On Mon, Nov 25, 2013 at 11:32 AM, Kornel Lesiński kor...@geekhood.netwrote: On 25 November 2013 08:00:10 Yoav Weiss y...@yoav.ws wrote: It contains some parts that I'm not sure have a consensus around them yet: * It defines picture as controlling img, where earlier on this list we discussed mostly the opposite (img querying its parent picture, if one exists) Controlling image is a great idea. It greatly simplifies the spec and hopefully implementations as well. I'm not saying it isn't, but it's not what we previously discussed on the list. I chose not to expose that implementation detail, assuming that one day (when all UAs, crawlers implement it) we will not need explicit img fallback any more. If picture was explicitly controlled by img then websites could start depending on that behavior, and we'd be stuck with it. OTOH picture can have native DOM interface and still reuse img for implementation. I believe these interfaces would be something you'd need to test, so you would have testing duplication, even if you save code duplication. I'm not sure that the goal of getting rid of img 5-10 years from now, is worth that cost. * It defines img as a part of picture's shadow DOM, which we need to see how it fits with having fallback img elements (which are necessary in the near future). I've added section about preloader. The img in fallback content should be ignored by the preloader. It's purely for picture-less UAs. The problem here is not the preloader, but the HTMLImageElement itself, which currently downloads a resource upon creation. With that said, it's an issue we'd have to deal with in both cases, regardless of which element controls which, and if picture has the control, we can probably define that it controls only the first/last img it encounters. I do wonder however if fallback img should be used as equivalent of a source to save authors a bit of repetition. (in selection algorithm the first step would be for each source or img child...) or perhaps be used as last-resort fallback when no source matches (step 2 of the algorithm). I agree that it would make sense for authors. This proposal does contain srcset as a subcomponent, but it's not the same srcset as defined in the HTML spec, but a modified version based on improvements from the src-N spec. (that cover the variable-width images use-case) Indeed. This part of the spec isn't ironed out yet. The proposal will also require some changes to img and specifically, when not created by JS, img will have to avoid loading of resources until the element is added to the DOM, and can see if its direct parent is picture. If the parent is picture, img would then query the parent (or wait to be controlled by its parent), otherwise, it'll load its resources as usual. I've specified something like that. I think it can be as simple as a flag that preload scanner uses internally. Again, this is an issue with HTMLImageElement itself, not the preload scanner. It'd probably require modifications to the img section of the HTML spec. I think we don't need to add any runtime behavior changes for this, as scripts constructing picture will not insert explicit fallback img node - it makes more sense to rely on picture polyfill instead (that will use img with correct src from the start). -- regards, Kornel
Re: [whatwg] Simplified picture element draft
On 25/11/13 10:32, Kornel Lesiński wrote: On 25 November 2013 08:00:10 Yoav Weiss y...@yoav.ws wrote: It contains some parts that I'm not sure have a consensus around them yet: * It defines picture as controlling img, where earlier on this list we discussed mostly the opposite (img querying its parent picture, if one exists) Controlling image is a great idea. It greatly simplifies the spec and hopefully implementations as well. I chose not to expose that implementation detail, assuming that one day (when all UAs, crawlers implement it) we will not need explicit img fallback any more. This suffers from some of the same problems that were previously brought up with picture; because it defines a new element that should behave like img you have to test that the new element works in all the same places that img ought to work. The fact that the spec tries to define this in terms of the shadow DOM isn't really helpful; you still need to ensure that implementations actually proxy the underlying img correctly in all situations. The advantage of the scheme that zcorpan proposed is that there is no magic proxy; we just add a capability to img to select its source using more than just a src attribute. This has better fallback than your design and is easier to implement.
Re: [whatwg] Simplified picture element draft
On 25 November 2013 10:59:15 Yoav Weiss y...@yoav.ws wrote: On Mon, Nov 25, 2013 at 11:32 AM, Kornel Lesiński kor...@geekhood.netwrote: If picture was explicitly controlled by img then websites could start depending on that behavior, and we'd be stuck with it. OTOH picture can have native DOM interface and still reuse img for implementation. I believe these interfaces would be something you'd need to test, so you would have testing duplication, even if you save code duplication. Yes, you need to test the integration point, but you only need to test that assignment of one attribute affects the other. You don't need to repeat tests that test it deeper. I do wonder however if fallback img should be used as equivalent of a source to save authors a bit of repetition. (in selection algorithm the first step would be for each source or img child...) or perhaps be used as last-resort fallback when no source matches (step 2 of the algorithm). I agree that it would make sense for authors. Which variant you think is better? I've specified something like that. I think it can be as simple as a flag that preload scanner uses internally. Again, this is an issue with HTMLImageElement itself, not the preload scanner. It'd probably require modifications to the img section of the HTML spec. I believe it won't be an issue in the approach I've specified - when the fallback img is separate from controlling image. Scripts can avoid creating fallback img at all, because when scripting is enabled they will use polyfill and can treat all UAs as supporting picture. In that case fallback img would be like document.write(noscript) ;) Maybe the spec should have authoring guidelines for this? The controlling image starts with no src, so it won't download anything that wasn't deliberately chosen through picture. -- regards, Kornel
Re: [whatwg] Simplified picture element draft
The advantage of the scheme that zcorpan proposed is that there is no magic proxy; we just add a capability to img to select its source using more than just a src attribute. This has better fallback than your design and is easier to implement. I believe that from testing perspective both approaches are equivalent. The spec I propose *is* only another way to control src of an image. The only difference is that I don't expose the img to scripts. That may make it even simpler, because you can't have odd cases like author moving/removing the controlling img or setting values directly on img that conflict with picture's definitions. -- regards, Kornel
Re: [whatwg] Simplified picture element draft
Thanks Yoav. Tab asked me to provide some more feedback to this list, but it's hard to tell from the lengthy discussion what proposal folks are most interested in. Is there an editor's draft or some other relatively self-contained write-up that I could review? Thanks, Adam On Mon, Nov 25, 2013 at 12:00 AM, Yoav Weiss y...@yoav.ws wrote: This is a part of the proposal that got people excited. The linked document defines picture's source selection algorithm. It contains some parts that I'm not sure have a consensus around them yet: * It defines picture as controlling img, where earlier on this list we discussed mostly the opposite (img querying its parent picture, if one exists) * It defines img as a part of picture's shadow DOM, which we need to see how it fits with having fallback img elements (which are necessary in the near future). This proposal does contain srcset as a subcomponent, but it's not the same srcset as defined in the HTML spec, but a modified version based on improvements from the src-N spec. (that cover the variable-width images use-case) The proposal will also require some changes to img and specifically, when not created by JS, img will have to avoid loading of resources until the element is added to the DOM, and can see if its direct parent is picture. If the parent is picture, img would then query the parent (or wait to be controlled by its parent), otherwise, it'll load its resources as usual. On Mon, Nov 25, 2013 at 8:33 AM, Adam Barth w...@adambarth.com wrote: I'm trying to sort out from the many messages on this topic which proposal I should read in detail. Is the one linked below the one that people are most excited about? I'm I correct in understand that this proposal contains srcset as a subcomponent? Thanks, Adam On Sat, Nov 23, 2013 at 3:04 PM, Kornel Lesiński kor...@geekhood.net wrote: I've written down proposal for the simplified source selection algorithm: http://geekhood.net/picture-element.html This also includes variant of the idea from the recent picture redux proposal to use an actual img element as the basis for the picture element definition. This draft doesn't include all features of src-N *yet*, but I expect this to be added either via extended srcset syntax to something like source sizes once there's consensus how to approach this. To simplify implementation even further I've allowed UAs to flatten fallback DOM to a plaintext string (in case they need to emulate img alt for existing screen readers or accessibility APIs). I've dropped usemap. It could be added, but I'm not sure if there is need for it. I've specified very few IDL attributes. This area may need to be extended. -- regards, Kornel
Re: [whatwg] Simplified picture element draft
On Mon, Nov 25, 2013 at 8:33 AM, Adam Barth w...@adambarth.com wrote: Thanks Yoav. Tab asked me to provide some more feedback to this list, but it's hard to tell from the lengthy discussion what proposal folks are most interested in. Is there an editor's draft or some other relatively self-contained write-up that I could review? I'll be writing up a draft which more closely follows the discussion in WHATWG, and posting it soonish. I'm technically on vacation all this week*, so I might not get to it until next week, but I'll try to put something up sooner than that. * I'm just taking this week off to veg and play videogames. ~TJ
Re: [whatwg] Simplified picture element draft
I'm trying to sort out from the many messages on this topic which proposal I should read in detail. Is the one linked below the one that people are most excited about? I'm I correct in understand that this proposal contains srcset as a subcomponent? Thanks, Adam On Sat, Nov 23, 2013 at 3:04 PM, Kornel Lesiński kor...@geekhood.net wrote: I've written down proposal for the simplified source selection algorithm: http://geekhood.net/picture-element.html This also includes variant of the idea from the recent picture redux proposal to use an actual img element as the basis for the picture element definition. This draft doesn't include all features of src-N *yet*, but I expect this to be added either via extended srcset syntax to something like source sizes once there's consensus how to approach this. To simplify implementation even further I've allowed UAs to flatten fallback DOM to a plaintext string (in case they need to emulate img alt for existing screen readers or accessibility APIs). I've dropped usemap. It could be added, but I'm not sure if there is need for it. I've specified very few IDL attributes. This area may need to be extended. -- regards, Kornel
[whatwg] Simplified picture element draft
I've written down proposal for the simplified source selection algorithm: http://geekhood.net/picture-element.html This also includes variant of the idea from the recent picture redux proposal to use an actual img element as the basis for the picture element definition. This draft doesn't include all features of src-N *yet*, but I expect this to be added either via extended srcset syntax to something like source sizes once there's consensus how to approach this. To simplify implementation even further I've allowed UAs to flatten fallback DOM to a plaintext string (in case they need to emulate img alt for existing screen readers or accessibility APIs). I've dropped usemap. It could be added, but I'm not sure if there is need for it. I've specified very few IDL attributes. This area may need to be extended. -- regards, Kornel
