Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Mon, 14 May 2012 01:30:20 +0100, Odin Hørthe Omdal wrote: All optional replacements of the src will have to be fitted in the same box as the original src. That might actually require you to specify both width and height upfront. Of course, people won't really do that, so I guess we're bound to get differing behaviour... Hm. What do people think about that? What happens here? You have no info on the real size of the picture. I guess maybe the browser should never load any srcset alternatives then? If you have no information at all it's rather hard to make a judgement. A photo gallery wants to show you a fullscreen picture, and give you: In this example, us (humans :P) can easily see that one is 2048 px and the other 4096 px. If I'm viewing this on my highres Nokia N9, a naïve implementation could pick the 2x, because it knows that's nicely highres just like its own screen. But it would actually be wrong! It would never need anything else than the 2048 px for normal viewing because it is anyway exceeding its real pixels on the screen. If srcset/ provides authors with good way to serve images at most appropriate size, they won't need to resort to tricks with downsizing high-res images to smaller size. For a full-width image on a ~960px viewport (assuming author doesn't have better sizes available) this would be appropriate: style="width:100%"> -- regards, Kornel Lesiński
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Fri, 11 May 2012 20:52:43 +0200, Simon Pieters wrote: There seem to be two proposals for what syntax to use for the responsive images use case: several elements vs. an attribute. I think an attribute is simpler to implement and thus likely to result in fewer bugs in browsers, which in turn benefits Web developers. With , in the parser case, all the information is available when the tag has been parsed. Excellent! In the scripted case, we need to "await a stable state" when the src or srcset attributes are changed (because the same script might also change the other), and then process the attributes again. The spec already does this for src="" because of the crossorigin="" attribute, so it's basically a solved problem already for in the spec. Just for the record if people are interested in exploring possible syntaxes, *multiple* attributes on would have the same order of implementation complexity as a single attribute. Even an open-ended set of attributes. Here's a straw man example of how an open-ended set of attributes could look like (trying to solve both use cases presented), where two images are decided between using media queries, and each of those have several available resolutions, which are decided between by the browser: src-small="mobile-zoomed-out.jpg 32dpi, mobile.jpg 96dpi, mobile-hires.jpg 200dpi" media-small="max-width: 400px" src-big="desktop.jpg 96dpi, desktop-hires.jpg 200dpi, desktop-print-res.jpg 600dpi" media-big="min-width: 400px"> With ..., in the parser case, the browser can't know it has all the information is not available until the picture element has been popped off the stack (since there might be futher elements), which necessarily delays fetching the image. Moreover, scripts might have changed things before then, because scripts can run between the time that the "picture" element is added to the document and the time it is popped off the stack, which is a potential source for bugs. In the scripted case, we need to listen for changes to img elements' parents, changes to img element's attributes, changes to source elements' parents, and changes to source elements' attributes, and "await a stable state" and then process the picture element's img and source elements' attributes (for both the old and new picture element, if you moved an element between two picture elements), which is more complex than the attribute case. -- Simon Pieters Opera Software
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sun, 13 May 2012 00:43:48 +0200, Jason Grigsby wrote: On May 11, 2012, at 8:52 PM, Simon Pieters wrote: There seem to be two proposals for what syntax to use for the responsive images use case: several elements vs. an attribute. There are two proposals because they solve two different use cases. Both use cases are becoming increasingly important. Unfortunately, these use cases are commonly collapsed into one. I have done it myself in the past. I tried to clarify the use cases recently.[1] I agree, but that is orthogonal (the proposals for both use cases could use either attribute syntax or element syntax). Use case #1 --- Document author needs to display different versions of an image at different breakpoints based on what I’m calling, for a lack of a better phrase, art direction merits. * Example 1: News site shows photograph speaking at a auto factory. On wide screens, the news site includes a widescreen version of the photograph in which the cars being built can clearly be seen. On small screens, if the photograph is simply resized to fit the screen, Obama’s face is too small to be seen. Instead, the document author may choose to crop the photograph so that it focuses in on Obama before resizing to fit the smaller screen. [1] * Example 2: On the Nokia Browser site where it describes the Meego browser[2], the Nokia Lumia is show horizontally on wide screens[3]. As the screen narrows, the Nokia Lumia is then shown vertically and cropped[4]. Bryan and Stephanie Rieger, the designers of the site, have talked about how on a wide screen, showing the full phone horizontally showed the browser best, but on small screens, changing the img to vertical made more sense because it allowed the reader to still make out the features of the browser in the image. Current proposed solution: element[5] Use case #2 --- For a variety of reasons, images of various pixel density are needed. These reasons include current network connection speed, display pixel density, user data plan, and user preferences. * Example 1: The use of high-density images for the new iPad on Apple.com.[6] * Example 2: A user on a slow network or with limited data left may explicitly declare that he or she would like to download a high resolution because they need to see a sharper version of an image before buying product, etc. Current proposed solution for use case #2: [7] IMHO Neither proposed solution handles all of the use cases. I’m not convinced that one solution needs to solve both of them, but I do think if we’re getting close to implementing one of the proposed solutions, we need to consider how it would work in conjunction with a solution for the other use case. To be more specific, if were to be implemented in a browser--potentially solving use case #2, but leaving use case #1 open--what would happen when we realized that use case #1 still needed to be solved? Would we end up with some bastardized mixture of and syntax? When Ted proposed , he wrote[7]: Ultimately I don't think addressing the multiple-resolution case needs to wait for a solution to these other cases. We don't need to "SOLVE ALL THE PROBLEMS!" right now. In a similar vein, the responsive images community group, focused on use case #1 and explicitly chose to ignore the problems described in use case #2. While I agreed with that focus earlier, I now think this may be a mistake. As much as I don’t want to bog down solving either use case, it seems likely that if we don’t look at both at the same time, that we’ll end up with[8]: Which would make no one happy. -Jason [1] http://blog.cloudfour.com/a-framework-for-discussing-responsive-images-solutions/ [2] http://browser.nokia.com/smartphones.html [3] http://browser.nokia.com/resources/images/home-feature.png [4] http://browser.nokia.com/resources/images/smartphones/choose-me...@320.png [5] https://github.com/Wilto/respimg [6] http://blog.cloudfour.com/how-apple-com-will-serve-retina-images-to-new-ipads/ [7] http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-May/035746.html [8] Yes, yes, this is an exaggeration, but you get my point. -- Simon Pieters Opera Software
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On May 13, 2012, at 7:03 PM, Boris Zbarsky wrote: > On 5/13/12 3:20 PM, Mathew Marquis wrote: >> I doubt any UAs will be forced to misinterpret common media queries because >> they haven’t been accounted for. > > Opera has already been forced to do this. For example, in its projection > mode it matches both the "projection" and "screen" media queries (technically > a spec violation) because of all the sites that explicitly say "screen" when > they really mean "not print" (or in the absence of that query in downrev UAs > when they mean "screen, projection, tv"). Now of course part of the problem > here is that "screen", "projection", and "tv" are mutually exclusive, which > is in retrospect silly. > > This is an excellent example of the fundamental divide about optimism vs > pessimism here: the things that web authors doubt UA vendors will be forced > to do because of web authors making bogus assumptions are things that UA > vendors have already been forced to do for years because web authors ... make > bogus assumptions. > > -Boris You make an excellent case for standards bodies working more closely with developers during the implementation of things like media queries. With a better sense of the ways developers might misuse, misinterpret, or fail to understand standards, issues like you describe could potentially be better avoided. But I suppose we’ve gone off-topic, to an extent.
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On 5/13/12 3:20 PM, Mathew Marquis wrote: I doubt any UAs will be forced to misinterpret common media queries because they haven’t been accounted for. Opera has already been forced to do this. For example, in its projection mode it matches both the "projection" and "screen" media queries (technically a spec violation) because of all the sites that explicitly say "screen" when they really mean "not print" (or in the absence of that query in downrev UAs when they mean "screen, projection, tv"). Now of course part of the problem here is that "screen", "projection", and "tv" are mutually exclusive, which is in retrospect silly. This is an excellent example of the fundamental divide about optimism vs pessimism here: the things that web authors doubt UA vendors will be forced to do because of web authors making bogus assumptions are things that UA vendors have already been forced to do for years because web authors ... make bogus assumptions. -Boris
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On 5/13/12 12:21 PM, Mathew Marquis wrote: The amount of “developers can never be trusted with this” sentiment I’ve heard from the members of this group is incredibly depressing. For the record, developing a web browser and in the process realizing how much web content is fundamentally completely broken is incredibly depressing. Hence the general attitude of the UA vendors in this group. Now we may perhaps have a skewed perspective: sites that are well-designed and Just Work in all browsers all the time don't show up on browser vendors' radars, obviously. But such sites are actually rather rare, in my experience. Pretty much every single high-profile site I can think of has had completely broken code on it at various points; several do today. Not only that, but a good fraction of the time when you point the brokenness out to the site maintainers they don't bother fixing it. Non-high-profile sites tend to be even worse: they have less of a budget, and more broken things going on. :( And yes, it's all depressing as you said. -Boris
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On May 12, 2012, at 5:33 PM, Mathew Marquis wrote: > I worry that, when faced with this markup, developers will simply opt to > serve the largest possible image in a src. In fairness, that approach "works" > with far less headache. For the resolution-adaptation use case, that wouldn't do the right thing by itself because it wouldn't downscale. is the minimum you need to get proper fallback. When/if all browsers support srcset (assuming it is even selected), you could omit the src containing the 1x image and this would be a perfectly valid choice. The key issue here, and a big difference from the approach, is that to actually get higher resolution on a high-rez display, you need to rescale the image to below its normal intrinsic size. You can do this by explicitly styling it to a fixed size, by using an explicit image resolution, or through the 2x shorthand as above. But for compatibility reasons it can't just happen automatically or images meant to be viewed at 1x will be too small. Regards, Maciej
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
Kornel Lesiński said: > Odin said: >> Actually, for this to work, the user agent needs to know the size of the >> standard image. So: >> >> > srcset="d...@2.jpg 2x, dog-lo.jpg 500w"> >> >> So if you've got the smartphone held in portrait, it's 250 css pixels >> wide, and so 500 real pixels, it could opt to show dog-lo.jpg rather >> than dog.jpg. > >But still displayed at 960 CSS pixels or course? That'd be fine (and the >UA could even download dog@2x when user zooms in). Yes, that's a good thing to pinpoint. The picture will be in a 960 CSS pixels box, but depending on the stylesheet - maybe img { max-width: 100%; height: auto } It will ofc resize the backed picture down to fit that rule, when it comes to the layout part. But yes, the picture will behave as if it is 960 px wide, only with lower dpi (resolution). Just the opposite of 2x in fact. All optional replacements of the src will have to be fitted in the same box as the original src. That might actually require you to specify both width and height upfront. Of course, people won't really do that, so I guess we're bound to get differing behaviour... Hm. What do people think about that? What happens here? You have no info on the real size of the picture. I guess maybe the browser should never load any srcset alternatives then? If you have no information at all it's rather hard to make a judgement. A photo gallery wants to show you a fullscreen picture, and give you: In this example, us (humans :P) can easily see that one is 2048 px and the other 4096 px. If I'm viewing this on my highres Nokia N9, a naïve implementation could pick the 2x, because it knows that's nicely highres just like its own screen. But it would actually be wrong! It would never need anything else than the 2048 px for normal viewing because it is anyway exceeding its real pixels on the screen. -- Odin Hørthe Omdal (odinho, Velmont), Core, Opera
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sun, 13 May 2012 23:00:10 +0100, Bjartur Thorlacius wrote: I've got a hunch I'm over-thinking this, but might bandwidth-constrained users not prefer miniatures instead of huge pixelated images? Perhaps sometimes, but support for this would tie layout and bandwidth together, and that complicates things. It's easier for authors if images don't unexpectedly change displayed size. I think we can assume that authors won't provide image in resolution that is too low to be useful, so huge pixelation may not be a problem. Authors can decrease image filesize not only by decreasing pixel size, but also by using lossy image compression (lower JPEG quality, less colors in PNG/GIF files). For pure bandwidth optimisation on 100dpi displays (rather than avoiding sending too large 200dpi images to users with 100dpi displays) an explicit filesize information may be the solution: then UA can easily make decision how much bandwidth it can use (e.g. aim to download any page in 5 seconds, so try to get image sizes to add up to less than 5*network B/s). -- regards, Kornel Lesiński
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On 5/13/12, Kornel Lesiński wrote: > By resolution I mean pixel density (regular vs "Retina" display), so this > doesn't affect layout. > Ah, I must have misunderstood you. > > I can imagine layout complexity being tied to bandwidth (an image-rich > design vs minimalistic text-only design), but I'm not sure how that would > work in practice given that cache has "infinite" bandwidth, and network > speed can change any second on mobile connections. > The layout would not depend directly on bandwidth, but on an upper limit on "graphic-heaviness." > It would be weird if page design changed when you moved between cell > towers or left/entered a cafe that had public WiFi. And if bandwidth media > > query was defined to be fixed, then you'd sometimes end up stuck with > wrong design that was chosen based on a temporary network state. > There's no question that bandwidth media queries would be a bad idea. How would you measure bandwidth anyway? My thinnest downlink is a few Mb/s, but I'm charged an increasing amount for every 10Gb/mo of international downloads. Authors should of course not evaluate how important their graphics are to users against bandwidth scarcity (artificial or real). > There is no such problem if only same-CSS-pixel-size images are swapped > in-place. > True. I've got a hunch I'm over-thinking this, but might bandwidth-constrained users not prefer miniatures instead of huge pixelated images?
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sun, 13 May 2012 21:23:58 +0100, Odin Hørthe Omdal wrote: Instead of srcset it could be src2x or another attribute that specifies image for higher screen density and/or bandwidth. The point is that media="" would allow author to choose image version adapted to page layout, and another mechanism connected to would allow UA to choose image resolution. Seeing it here in code it's actually not such a monster that I'd said it'd be. So I like it even more, and it's the obvious way for these to interact. I think it'd be a mistake to call it src2x though, -- it feels very specific. You can scale up to double then, but you can't necessarily go beyond that: going down for e.g. mobile. OTOH, 2x will be the most common usage at least as far as I can tell. vs. is not really all that different, but the second should be more flexible. Also downscaling: Yes, good point. Actually, for this to work, the user agent needs to know the size of the standard image. So: So if you've got the smartphone held in portrait, it's 250 css pixels wide, and so 500 real pixels, it could opt to show dog-lo.jpg rather than dog.jpg. But still displayed at 960 CSS pixels or course? That'd be fine (and the UA could even download dog@2x when user zooms in). -- regards, Kornel Lesiński
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sun, 13 May 2012 20:55:08 +0100, Bjartur Thorlacius wrote: The problem with that, though, is that then bandwidth constraints can't affect layout. Users should be able to configure UAs to use downsized images even given a large viewport, if only to save bandwidth and reserve a larger fraction of the viewport for text columns. That wasn't my assumption. For optimisation case expected images to vary only in compression strength or DPI (low-res JPEG or 2x image for Retina display), but never in their dimension in CSS pixels. I think page should not be allowed to change layout based on bandwidth availability, because of browser caches. Imagine scenario: 1. User visits page on high-bandwidth connection and UA caches some high quality images. 2. Connection changes to slower one. 3. User visits the page again. HTML was non-cacheable, but images were cacheable. Now the page may have mix of high-bandwidth and low-bandwidth content. It's entirely possible to have mixed quality with images if they have same CSS pixel size (and use of higher quality images whenever possible is desirable), but a page cannot contain mix of multiple different layouts at the same time. If bandwidth was a media query, then entire page may have to be downgraded to low-bandwidth version even if UA had a lot of high-bandwidth content cached. Adaptation of images to the layout is page-specific. Adaptation of images to bandwidth/screen is UA/device-specific. Quite. But the latter just might affect the layout. I think optimisation case should be forbidden from affecting the layout. Author is in the best position to adapt image to page layout. User-agent is in the best position to determine speed/quality trade-offs. But low-res images usually don't look too good when upscaled. Thus few pixels should mean small image, UAs mustn't default to pixelation. I don't understand why UA would "default to pixelation"? The whole point of declaring available image versions is giving UA possibility to default to whatever it deems best. When user is on GPRS or roaming connection then pixelation is a great outcome compared to not being able to download images at all due to prohibitive size/price. When user is on broadband connection then UA can select high resolution images and avoid pixelation when network/hardware allows it. And when image properties are specified declaratively, a smart UA can even do a mix of both, e.g. download pixelated images on mobile connections and give user option to selectively download higher-res images, or download low-res first for fast initial display and then download high-res in the background. Media queries MUST be interpreted exactly as author specified them. Thus we mustn't force UAs to pretend to render to small viewports to find low-res images. That would have unwieldy side-effects. Indeed! User-agents need freedom to choose image resolution based on open set of factors, many of which are details authors should not have to think about (presence in cache, cost of bandwidth, available memory, external displays, etc.) But the chosen image resolution might be a factor for choosing layout. By resolution I mean pixel density (regular vs "Retina" display), so this doesn't affect layout. I can imagine layout complexity being tied to bandwidth (an image-rich design vs minimalistic text-only design), but I'm not sure how that would work in practice given that cache has "infinite" bandwidth, and network speed can change any second on mobile connections. It would be weird if page design changed when you moved between cell towers or left/entered a cafe that had public WiFi. And if bandwidth media query was defined to be fixed, then you'd sometimes end up stuck with wrong design that was chosen based on a temporary network state. There is no such problem if only same-CSS-pixel-size images are swapped in-place. -- regards, Kornel Lesiński
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
Kornel Lesiński wrote: > Selection of 1x/2x images is relevant only as long as we have 100dpi > screens and slow connections, and both will disappear over time. Well, the web is a huge place. I'm quite sure that'll take ages and ages if it ever happens at all (I don't think it'll ever disappear). Might get irrelevant for some small and specific markets/segments though. > How about that: > > > media="max-width:4in"> > > > > > > Instead of srcset it could be src2x or another attribute that specifies > image for higher screen density and/or bandwidth. The point is that > media="" would allow author to choose image version adapted to page > layout, and another mechanism connected to would allow UA to > choose image resolution. Seeing it here in code it's actually not such a monster that I'd said it'd be. So I like it even more, and it's the obvious way for these to interact. I think it'd be a mistake to call it src2x though, -- it feels very specific. You can scale up to double then, but you can't necessarily go beyond that: going down for e.g. mobile. OTOH, 2x will be the most common usage at least as far as I can tell. vs. is not really all that different, but the second should be more flexible. Also downscaling: Actually, for this to work, the user agent needs to know the size of the standard image. So: So if you've got the smartphone held in portrait, it's 250 css pixels wide, and so 500 real pixels, it could opt to show dog-lo.jpg rather than dog.jpg. -- Odin Hørthe Omdal, Core, Opera Software
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sun, May 13, 2012 at 8:55 PM, Bjartur Thorlacius wrote: > But the chosen image resolution might be a factor for choosing layout. Maybe we should think of a way to expose _that_ information to CSS, rather than going in the other direction. section { /* generic style rules */ } section! img:intrinsic-aspect-ratio(<2) { /* specific overrides for section when the UA picks the narrow image */ } -- Benjamin Hawkes-Lewis
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On 5/13/12, Kornel Lesiński wrote: > I think layout (media queries) and optimisation cases are orthogonal and > it would be a mistake to do both with the same mechanism. > My knee-jerk reaction to the above thought is that layout should be done using CSS and any optimizations left up to the UA. A bandwidth constrained UA could request a downsized thumbnail that fits the size of the /// element, or render an appropriately sized bitmap from a SVG. The problem with that, though, is that then bandwidth constraints can't affect layout. Users should be able to configure UAs to use downsized images even given a large viewport, if only to save bandwidth and reserve a larger fraction of the viewport for text columns. > Adaptation of images to the layout is page-specific. Adaptation of images > to bandwidth/screen is UA/device-specific. > Quite. But the latter just might affect the layout. > Author is in the best position to adapt image to page layout. User-agent > is in the best position to determine speed/quality trade-offs. > But low-res images usually don't look too good when upscaled. Thus few pixels should mean small image, UAs mustn't default to pixelation. > Media queries MUST be interpreted exactly as author specified them. Thus we mustn't force UAs to pretend to render to small viewports to find low-res images. That would have unwieldy side-effects. > User-agents need freedom to choose image resolution based on open set of > factors, many of which are details authors should not have to think about > (presence in cache, cost of bandwidth, available memory, external > displays, etc.) > But the chosen image resolution might be a factor for choosing layout.
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sun, May 13, 2012 at 6:01 PM, Benjamin Hawkes-Lewis wrote: > On Sun, May 13, 2012 at 5:21 PM, Mathew Marquis wrote: >> When we get ?something new to optimize for,? we start adding that thing >> going forward. The evolution of media queries?or, say, HTML5?hasn?t led to a >> need to constantly revisit every piece of client work a developer has ever >> produced. > > In the case of webkit prefixes, authors have needed to update their > work, have failed to do so, and now user agents are having to support > webkit prefixes. Agreed, but I don't think it's a fair comparison - we're not talking about deprecating with either new proposal. Existing work isn't becoming invalid. It'll be something we can use to do a better job from now on.
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sun, 13 May 2012 20:20:54 +0100, Mathew Marquis wrote: The key problem about designing a responsive images solution around user agent characteristics not image characteristics is that authors will inevitably make more false assumptions about what images match what user agent characteristics than user agents will. As a result, user agents may be forced to misinterpret media queries in order to provide their users with better user experiences. Correct me if I’m wrong, but wouldn’t this same reasoning apply to layouts that depend heavily on media queries? No, those are very different situations. Layout is under author's control and is known to the author when the page is authored. UA characteristics are not, and different UAs will have different requirements. Conversely from UA perspective: layouts vary wildly between pages, so UA can't really mess with them. Bandwidth/DPI requirements are known to UA and there can be same set of UA-specific rules that works for all pages. -- regards, Kornel Lesiński
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sun, 13 May 2012 01:33:25 +0100, Mathew Marquis wrote: I worry that, when faced with this markup, developers will simply opt to serve the largest possible image in a src. In fairness, that approach "works" with far less headache. In the long term that may be a very sensible approach. Selection of 1x/2x images is relevant only as long as we have 100dpi screens and slow connections, and both will disappear over time. Perhaps we should think about making syntax for 200dpi+ images with intrinsic dimensions easy? Selection between images for different screen sizes/orientations isn't a problem that will go away any time soon, so I think is a good idea regardless of solution for DPI/bandwidth problem. As long as HTML claims to be independent of CSS there is no solution for that (i.e authors shouldn't be adding multiple and showing one of them with CSS). How about that: media="max-width:4in"> Instead of srcset it could be src2x or another attribute that specifies image for higher screen density and/or bandwidth. The point is that media="" would allow author to choose image version adapted to page layout, and another mechanism connected to would allow UA to choose image resolution. -- regards, Kornel Lesiński
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On May 13, 2012, at 1:01 PM, Benjamin Hawkes-Lewis wrote: > On Sun, May 13, 2012 at 5:21 PM, Mathew Marquis wrote: >>> AND they have to update their sites and mediaqueries when we get >>> something new to optimize for. I don't think they will do that, based on >>> how extremely big the problem with -webkit-prefixes are. >>> >>> I've seen enough of the web to be sceptical. >> >> The amount of “developers can never be trusted with this” sentiment I’ve >> heard from the members of this group is incredibly depressing. > > That it depresses you does not mean that taking a more optimistic > viewpoint will produce specifications that will result in better > end-user experiences. > >> When we get “something new to optimize for,” we start adding that thing >> going forward. The evolution of media queries—or, say, HTML5—hasn’t led to a >> need to constantly revisit every piece of client work a developer has ever >> produced. > > In the case of webkit prefixes, authors have needed to update their > work, have failed to do so, and now user agents are having to support > webkit prefixes. > > The key problem about designing a responsive images solution around > user agent characteristics not image characteristics is that authors > will inevitably make more false assumptions about what images match > what user agent characteristics than user agents will. As a result, > user agents may be forced to misinterpret media queries in order to > provide their users with better user experiences. Correct me if I’m wrong, but wouldn’t this same reasoning apply to layouts that depend heavily on media queries? The primary characteristics in play are screen size and resolution. I doubt any UAs will be forced to misinterpret common media queries because they haven’t been accounted for. I doubt UAs could, in fact—or would care. It wouldn’t be a matter of neglecting to support their product. It would be a matter of neglecting to supporting a screen at 310px wide. This is to say nothing of the fact that the objective of a responsive layout _is_ to properly accommodate the widest possible range of displays. The things you’re trying to prevent are the things we’re trying to avoid anyway. > > What authors _can_ do and user agents _cannot_ do is describe their > images. Such metadata never needs to be misinterpreted and allows user > agents to iterate and improve the end-user experience even when the > author either does not care about them, or has moved on, or is long > dead. Is authors not caring about user agents a real problem? > Certainly is - witness Opera's failed efforts to get authors to > support more than Webkit, witness the widespread inaccessibility of > web services, etc. > > Depressing but true. > > -- > Benjamin Hawkes-Lewis
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sun, 13 May 2012 10:36:37 +0100, Jason Grigsby wrote: On May 13, 2012, at 9:51 AM, David Goss wrote: A common sentiment here seems to be that the two proposed responsive image solutions solve two different use cases: After skyping with Mat (@wilto) last night, I think I may be the only one who didn’t fully grok that the mediaqueries in could be used to address both use cases. It is still unclear to me if would address both. I think layout (media queries) and optimisation cases are orthogonal and it would be a mistake to do both with the same mechanism. Adaptation of images to the layout is page-specific. Adaptation of images to bandwidth/screen is UA/device-specific. Author is in the best position to adapt image to page layout. User-agent is in the best position to determine speed/quality trade-offs. Media queries MUST be interpreted exactly as author specified them. User-agents need freedom to choose image resolution based on open set of factors, many of which are details authors should not have to think about (presence in cache, cost of bandwidth, available memory, external displays, etc.) So even though both proposals are about selecting right images, their purposes and requirements are very different. -- regards, Kornel Lesiński
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sun, 13 May 2012 18:01:12 +0100, Benjamin Hawkes-Lewis wrote: What authors _can_ do and user agents _cannot_ do is describe their images. Such metadata never needs to be misinterpreted and allows user agents to iterate and improve the end-user experience even when the author either does not care about them, or has moved on, or is long dead. Indeed. An interesting use-case has come up on this list — use highest-resolution image when user chooses "Save As". While it's not an earth-shattering feature, it's a nice touch a browser could do having information about image. It wouldn't be possible/sensible to do with a media query. Even if there was support for something like media="user-action:save-as">, I can bet that less than 0.5% of pages would use it and do it correctly. -- regards, Kornel Lesiński
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sun, May 13, 2012 at 5:21 PM, Mathew Marquis wrote: >> AND they have to update their sites and mediaqueries when we get >> something new to optimize for. I don't think they will do that, based on >> how extremely big the problem with -webkit-prefixes are. >> >> I've seen enough of the web to be sceptical. > > The amount of “developers can never be trusted with this” sentiment I’ve > heard from the members of this group is incredibly depressing. That it depresses you does not mean that taking a more optimistic viewpoint will produce specifications that will result in better end-user experiences. > When we get “something new to optimize for,” we start adding that thing going > forward. The evolution of media queries—or, say, HTML5—hasn’t led to a need > to constantly revisit every piece of client work a developer has ever > produced. In the case of webkit prefixes, authors have needed to update their work, have failed to do so, and now user agents are having to support webkit prefixes. The key problem about designing a responsive images solution around user agent characteristics not image characteristics is that authors will inevitably make more false assumptions about what images match what user agent characteristics than user agents will. As a result, user agents may be forced to misinterpret media queries in order to provide their users with better user experiences. What authors _can_ do and user agents _cannot_ do is describe their images. Such metadata never needs to be misinterpreted and allows user agents to iterate and improve the end-user experience even when the author either does not care about them, or has moved on, or is long dead. Is authors not caring about user agents a real problem? Certainly is - witness Opera's failed efforts to get authors to support more than Webkit, witness the widespread inaccessibility of web services, etc. Depressing but true. -- Benjamin Hawkes-Lewis
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On May 13, 2012, at 9:42 AM, Odin Hørthe Omdal wrote: > >> Connection speed >> As an extension of the iPad example above, it would also be >> irresponsible to serve the high res image to users that do have a high >> pixel density display but are not on a fast internet connection for >> whatever reason. So you might write: >> >> >> >> >> > > As I said, this is one of my big gripes with this proposal. I don't > think this'll work satisfactorily, it puts the burden on figuring out > what is correct for that user into the page author's control. That model > skews the responsibility so that instead of only a few browser engines > having to get it right, millions of web page authors have to get it > right. We’re speaking in theoreticals here, which is what makes it hard for me to wrap my head around it being anyone’s major gripe with either proposal. Maybe a bandwidth media query isn’t the solution? Maybe, when the time comes, we do rely on the UA to make that decision. Maybe there’s a user preference that could be set. This line of reasoning doesn’t speak for or against either approach, however. > > AND they have to update their sites and mediaqueries when we get > something new to optimize for. I don't think they will do that, based on > how extremely big the problem with -webkit-prefixes are. > > I've seen enough of the web to be sceptical. The amount of “developers can never be trusted with this” sentiment I’ve heard from the members of this group is incredibly depressing. When we get “something new to optimize for,” we start adding that thing going forward. The evolution of media queries—or, say, HTML5—hasn’t led to a need to constantly revisit every piece of client work a developer has ever produced. We’re speaking in terms of optimization because that’s what it is: incremental improvements to the way assets are loaded. As more become available, requests become more tailored. > > > What if a the author doesn't write that min-connection-speed query > there? And who is the author of a page to decide such things anyway? > What about latency? Should there be a max-latency: 10ms, there as well? > What about cost? I have a fast line, but it costs money to download > stuff, so I'd like the smaller pictures. What about if I have slow > internet connection, but I'd want to print the page, thus getting the > best version of the image? Or if I'm sitting with my new fancy hires KDE > Tablet and loading your page in a background-tab, letting it render and > be finished while I continue to read the page I'm on (the browser might > very well want to load the highres picture then, even though the > connection is slow, but with MQ, not that lucky). I think this has has forked well into “should we have bandwidth available in a media query” territory. It’s a worthwhile topic for sure, but apart from being a minor and—again—purely speculative factor in considering an adaptive images approach, it is another conversation. Mat Marquis
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
Jason Grigsby wrote: > David Goss wrote: >> A common sentiment here seems to be that the two proposed responsive >> image solutions solve two different use cases: > > After skyping with Mat (@wilto) last night, I think I may be the only > one who didn’t fully grok that the mediaqueries in could be > used to address both use cases. > > It is still unclear to me if would address both. Oh but they *do* solve two different use cases. Mediaqueries in _may_ be able to address both, but is not, and never will. It's simply not designed for it. So, why then do I prefer srcset when it comes to solving the «save bandwidth»/use correct "weight" of resource use case? Because it's design is "browser choose" instead of "web author choose". It puts these decisions into the browsers control. The easiest path for the developers, is also what will make the browser be able to be a good agent to its user, and decide what it shall download. The srcset attribute is also much simpler, which makes me think it'll be used more and also in the correct way. There's strong correlation between simple and much used. for solving this use case is/will be over engineered, just because it can be done, doesn't mean it should. For the other use case, adapting the image for different content, it might be a good candidate though. inside should also get a srcset attribute then, so that it would be possible to choose different qualities if they exist. For relatively advanced sites, it will look like a monster though, so that's something to be looked more into. David Goss wrote: > Connection speed > As an extension of the iPad example above, it would also be > irresponsible to serve the high res image to users that do have a high > pixel density display but are not on a fast internet connection for > whatever reason. So you might write: > > > > > As I said, this is one of my big gripes with this proposal. I don't think this'll work satisfactorily, it puts the burden on figuring out what is correct for that user into the page author's control. That model skews the responsibility so that instead of only a few browser engines having to get it right, millions of web page authors have to get it right. AND they have to update their sites and mediaqueries when we get something new to optimize for. I don't think they will do that, based on how extremely big the problem with -webkit-prefixes are. I've seen enough of the web to be sceptical. What if a the author doesn't write that min-connection-speed query there? And who is the author of a page to decide such things anyway? What about latency? Should there be a max-latency: 10ms, there as well? What about cost? I have a fast line, but it costs money to download stuff, so I'd like the smaller pictures. What about if I have slow internet connection, but I'd want to print the page, thus getting the best version of the image? Or if I'm sitting with my new fancy hires KDE Tablet and loading your page in a background-tab, letting it render and be finished while I continue to read the page I'm on (the browser might very well want to load the highres picture then, even though the connection is slow, but with MQ, not that lucky). > (... containing element width/height) > As I understand it, the syntax would have to keep getting > extended every time we wanted to test a different property. No. It wouldn't, because it only describes the images, nothing more. Given: Say if you're in a browser optimizing for low bandwidth usage, and some quality at the cost of speed. The viewport is 800x600. In the normal case, the browser would choose hero.jpg because it fits well with its resource algorithm. However, since being in the special mode, it defers the prefetch of the image and waits for layout, where it can see that this picture lies inside a 150px wide box - so it fetches hero-lo.jpg because it doesn't need more. With the MediaQueries proposal, you'd need loads of advanced information to do the same. The browser could not just infer this on its own and change its algorithms to do it. Bandwidth might be expensive even though it's fast, so althought they have over 1mbit speed, they want to get low pictures. With media queries there's just so many variables, and so much to choose from. srcset only chooses between different *qualities* of the same image whereas who knows what mediaqueries does? It's not possible to not do anything the web page author hasn't told you about. That's why srcset is so much more powerful for its use case. The browser is in a better position to decide what quality of image it'd like to fetch. The content author, however, is in a better position to lay out the different content images based on their set of mediaqueries. I can see a few use cases there, but they are orthogonal to what we're talking about here. I think it's something worth solving, but doing it inside srcset is not where. -- Odin Hørthe Omdal (odinho/Velmont), Core, Opera Software
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On 5/13/12 7:26 AM, David Goss wrote: but it'd be irresponsible to just serve an with the high res source to all users, making them wait longer for the download even though they can't see the extra quality on their screen. Except when they can, e.g. by printing or moving the display to another screen. Basically, in this case not sending the high-res image is optimizing a bit for one common use case while degrading the user experience in other use cases. Maybe that's OK, but that requires a careful analysis of user behavior for the particular content involved. And in particular, I'm not sure what I think of labeling content that aims to be usable by the user in the widest variety of circumstances as "irresponsible". Past examples of us doing that sort of thing led to it being considered "irresponsible" to allow website layouts to scale above or below certain widths, amongst other current usability disasters. Note that these concerns argue, to a certian extent, *against* reusing a very general syntax that can express constraints that aren't relevant to the actual use cases, or that provide an attractive nuisance that encourages developers to do things that can't be implemented in a performant way. Dumping as it is won't prevent this, as it's already in the door in the form of. has quite different dynamic behavior from and quite different author expectations (e.g. authors don't expect videos to be preloaded in their entirety, by default; there are other significant expectation differences too). -Boris
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On 13 May 2012, at 13:19, Benjamin Hawkes-Lewis wrote: > On Sun, May 13, 2012 at 12:26 PM, David Goss wrote: >> As I understand it, the syntax would >> have to keep getting extended every time we wanted to test a different >> property. > > It doesn't "test" anything. It tells the UA metadata about the image > set it would otherwise have to obtain by downloading them all. > > UAs can pick the right image based on this metadata plus what they > know about dimensions and orientation of some container, pixel > density, and connectivity. > > It would need to get extended if we learnt of further image metadata > we needed to expose to UAs to help them pick the optimal download > (maybe MIME type). > > Media queries test client properties, but this syntax is attempting to > describe an image. If we were trying to avoid reinventing wheels at > all costs, we'd want to look at an image description vocabulary like > EXIF: > >http://www.w3.org/2003/12/exif/ > > -- > Benjamin Hawkes-Lewis Yes, you're right, I should have referred to "including more metadata" not "testing more properties" when referring to the set syntax.
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sun, May 13, 2012 at 12:26 PM, David Goss wrote: > As I understand it, the syntax would > have to keep getting extended every time we wanted to test a different > property. It doesn't "test" anything. It tells the UA metadata about the image set it would otherwise have to obtain by downloading them all. UAs can pick the right image based on this metadata plus what they know about dimensions and orientation of some container, pixel density, and connectivity. It would need to get extended if we learnt of further image metadata we needed to expose to UAs to help them pick the optimal download (maybe MIME type). Media queries test client properties, but this syntax is attempting to describe an image. If we were trying to avoid reinventing wheels at all costs, we'd want to look at an image description vocabulary like EXIF: http://www.w3.org/2003/12/exif/ -- Benjamin Hawkes-Lewis
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
> > > > Also note that there is a great difference in implementation complexity > > between various properties above. For example, viewport width/height is > > rather easy to work with because one can assume it won't change between > > prefetching and layout, so one can prefetch the right asset. On the other > > hand switching based on containing element width/height requires layout to > > happen before the right asset can be selected, so it has to be loaded late. > > This will significantly decrease the perceived responsiveness of the site. > > > > Other properties like connection speed are very difficult to work with > > because they can have high temporal variability e.g. due to sharing of one > > connection by many consumers, due to temporary environmental conditions > > (train goes into a tunnel) or due to switching transports (wifi to 3G, for > > example). My suspicion is that trying to write a solution for switching > > based on connection speed would lead to people getting the "wrong" assets > > much of the time. > > So, if someone's connection speed changes dramatically when the image > request has already started, they'll either get: > 1 - a high res image, but slowly > 2 - a low res image (relative to what their device can display) > > I'm not an expert on internet connections. I'd like to know, > statistically, how likely it is that a user's connection speed will > stay consistent (say +/- 20%) for a ten second period. Does anyone > know of any research like this? If wireless internet connections were > constantly all over the place it would be a concern, but as someone > that uses them all the time it doesn't feel that way to me. > > I do appreciate that standardised connection speed testing is going to > be hard. But if and when it comes, it should surely come in the form > of a media query (as well as a JS API) in which case would > support it automatically. > > > Note that these concerns argue, to a certian extent, *against* reusing a > > very general syntax that can express constraints that aren't relevant to the > > actual use cases, or that provide an attractive nuisance that encourages > > developers to do things that can't be implemented in a performant way. > > Dumping as it is won't prevent this, as it's already in the > door in the form of . This may not be as much of a problem as it seems though. A browser could quite easily keep track of the connection speed over time and determine a logical average. It could take into account things like the type of network connection (e.g. a laptop may switch between wireless, tethered phone or wired, and may move about between various networks of differing speeds). This may even be available at the OS level (I believe a Mac allows for various network configurations based on the network it's attached to) which would allow for a better experience for anyone using multiple browsers often. Thanks, Ash http://www.ashleysheridan.co.uk
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On 13 May 2012 10:14, James Graham wrote: > > > On Sun, 13 May 2012, David Goss wrote: > >> A common sentiment here seems to be that the two proposed responsive >> image solutions solve two different use cases: >> >> - for serving different resolutions of a content image >> (for bandwidth and dpi) >> - for serving different versions of a content image (for art >> direction) >> >> ...and that neither solution can deal with both issues. I disagree. I >> would describe it as a single, broad use case: >> Serving different sources of an image based on properties of the >> client. These properties could include: >> - Viewport width/height >> - Containing element width/height >> - Device orientation >> - Colour capability >> - Old-fashioned media type (screen/print) >> - Connection speed >> - Pixel density >> - Things we haven't thought about/aren't an issue yet > > > Which of hese things are actual requirements that people need to meet and > which are hypothetical? For example I think it is uncontroversial that > viewport width/height is a real requirement. On the other hand, I have never > heard of a site that switches assets based on display colour capability. Can > you point to sites actually switching assets based on each property you > listed? Before going on, I'll note that it's difficult to find many sites actually doing these things right now, the main reason being that the costs (wasted downloads, very hacky methods) outweigh the benefits much of the time and stop it being worthwhile to do. Viewport width/height As you said, viewport width/height is the obvious one. The Boston Globe (http://bostonglobe.com/) is an example of a site doing this now (and using a necessarily hacky method), as described in Mat's article (http://www.alistapart.com/articles/responsive-images-how-they-almost-worked-and-what-we-need/). Containing element width/height: This is kind of hypothetical, at least right now. But authors would like to be able to apply different CSS (and different image sources) based on the current context of the target (e.g. a module could be used multiple times on a site in different places, with various widths depending on where they are in the layout) - see http://blog.andyhume.net/responsive-containers/. I appreciate it would be difficult in terms of syntax and browser implementation. I'm sure it's no coincidence that no such media query exists yet. But this is the point: the syntax allows us to use future media queries if/when they arrive. As I understand it, the syntax would have to keep getting extended every time we wanted to test a different property. Device orientation This is essentially a part of viewport width/height, as it effectively denotes "is the viewport wider than it is tall, or vice versa". Perhaps I shouldn't have listed it separately. Pixel density Higher pixel density displays are getting more common. The new iPad is a good example. If you have a large, prominent image (especially a photo) on a web page that takes up the majority of the screen at once, you want to take advantage of the extra pixel density by serving a high resolution image, but it'd be irresponsible to just serve an with the high res source to all users, making them wait longer for the download even though they can't see the extra quality on their screen. So, you serve the image at an ordinary resolution by default, or serve the high res version if the pixel density is over a certain number. Apple are doing this with a JS method, as documented by Jason Grigsby (http://blog.cloudfour.com/how-apple-com-will-serve-retina-images-to-new-ipads/), and in cases where they do serve the high res image, the low res one is also downloaded. This is one of the problems solves: Connection speed As an extension of the iPad example above, it would also be irresponsible to serve the high res image to users that do have a high pixel density display but are not on a fast internet connection for whatever reason. So you might write: > > Also note that there is a great difference in implementation complexity > between various properties above. For example, viewport width/height is > rather easy to work with because one can assume it won't change between > prefetching and layout, so one can prefetch the right asset. On the other > hand switching based on containing element width/height requires layout to > happen before the right asset can be selected, so it has to be loaded late. > This will significantly decrease the perceived responsiveness of the site. > > Other properties like connection speed are very difficult to work with > because they can have high temporal variability e.g. due to sharing of one > connection by many consumers, due to temporary environmental conditions > (train goes into a tunnel) or due to switching transports (wifi to 3G, for > example). My suspicion is that trying to write a solution for switching > based on connection speed would lead to people getting the "wrong" assets > muc
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sun, May 13, 2012 at 10:36 AM, Jason Grigsby wrote: > It may be that the proposal is written in language that implementors > understand and that it needs to be rewritten to make it clearer for authors > how it would work. Or it could be an indication that the syntax is too terse > and confusing for authors (which is currently the feedback the community > group is receiving). Some people seem to think the filenames are part of the syntax. Using filenames that don't duplicate information in the syntax might actually be less confusing: Perhaps changing the syntax to avoid confusion with units might help too: -- Benjamin Hawkes-Lewis
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On May 13, 2012, at 9:51 AM, David Goss wrote: > A common sentiment here seems to be that the two proposed responsive > image solutions solve two different use cases: After skyping with Mat (@wilto) last night, I think I may be the only one who didn’t fully grok that the mediaqueries in could be used to address both use cases. It is still unclear to me if would address both. > I also agree with many other > developers that is definitely the more developer-friendly > syntax, in terms of reading, writing, maintaining and scripting. Edward’s original proposal was pretty straight forward, but as it has grown to try to address more use cases, the syntax has become more convoluted[1]. I read the latest proposal multiple times last night and still couldn’t figure out how it would work. It may be that the proposal is written in language that implementors understand and that it needs to be rewritten to make it clearer for authors how it would work. Or it could be an indication that the syntax is too terse and confusing for authors (which is currently the feedback the community group is receiving). -Jason [1] http://junkyard.damowmow.com/507
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sun, 13 May 2012, David Goss wrote: A common sentiment here seems to be that the two proposed responsive image solutions solve two different use cases: - for serving different resolutions of a content image (for bandwidth and dpi) - for serving different versions of a content image (for art direction) ...and that neither solution can deal with both issues. I disagree. I would describe it as a single, broad use case: Serving different sources of an image based on properties of the client. These properties could include: - Viewport width/height - Containing element width/height - Device orientation - Colour capability - Old-fashioned media type (screen/print) - Connection speed - Pixel density - Things we haven't thought about/aren't an issue yet Which of hese things are actual requirements that people need to meet and which are hypothetical? For example I think it is uncontroversial that viewport width/height is a real requirement. On the other hand, I have never heard of a site that switches assets based on display colour capability. Can you point to sites actually switching assets based on each property you listed? Also note that there is a great difference in implementation complexity between various properties above. For example, viewport width/height is rather easy to work with because one can assume it won't change between prefetching and layout, so one can prefetch the right asset. On the other hand switching based on containing element width/height requires layout to happen before the right asset can be selected, so it has to be loaded late. This will significantly decrease the perceived responsiveness of the site. Other properties like connection speed are very difficult to work with because they can have high temporal variability e.g. due to sharing of one connection by many consumers, due to temporary environmental conditions (train goes into a tunnel) or due to switching transports (wifi to 3G, for example). My suspicion is that trying to write a solution for switching based on connection speed would lead to people getting the "wrong" assets much of the time. Note that these concerns argue, to a certian extent, *against* reusing a very general syntax that can express constraints that aren't relevant to the actual use cases, or that provide an attractive nuisance that encourages developers to do things that can't be implemented in a performant way.
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
A common sentiment here seems to be that the two proposed responsive image solutions solve two different use cases: - for serving different resolutions of a content image (for bandwidth and dpi) - for serving different versions of a content image (for art direction) ...and that neither solution can deal with both issues. I disagree. I would describe it as a single, broad use case: Serving different sources of an image based on properties of the client. These properties could include: - Viewport width/height - Containing element width/height - Device orientation - Colour capability - Old-fashioned media type (screen/print) - Connection speed - Pixel density - Things we haven't thought about/aren't an issue yet There are media queries already standardised and implemented for many of the above. Others will surely follow. One of the main strengths of is that is incorporates media queries (rather than its own syntax for testing properties), which will be developed as time goes on to include other client properties as they become relevant. This is good DRY practise. The differences in the sources could include: - Resolution - Compression - Zoom - Cropping - Aspect ratio - Colour/monochrome This is all possible with . I also agree with many other developers that is definitely the more developer-friendly syntax, in terms of reading, writing, maintaining and scripting.
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On 5/12/12 9:28 AM, Mathew Marquis wrote: While that information may be available at the time the img tag is parsed, I don’t believe it will be available at the time of prefetching Which information? At least in Gecko, prefetching happens "when the tag is parsed". So in fact in Gecko the srcset approach would be much more amenable to prefetching than the approach. -Boris
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
I’ve put together a summary of potential use cases addressed by the picture markup and posted them to the WHATWG wiki, along with a few key implementation details: http://wiki.whatwg.org/wiki/Adaptive_images I don’t mind saying that the `img set` markup is inscrutable to the point where I may well be missing the mark on what it’s trying to achieve, but it certainly seems to overlap with many of the things for which media queries were intended—albeit in a completely siloed way. As media queries continue to advance over time, are there plans to continue adding functionality to `img set` in parallel? I would hate to think we could be painting ourselves into a corner for the sake of easier implementation on the UA side. I don’t claim to be any smarter than the average developer, but I am speaking as a core contributor to jQuery Mobile and having worked on the responsive BostonGlobe.com designs: tailoring things for client capabilities is kind of my thing. I fear the developer community has little hope of understanding this pattern, and there has been no small amount of consensus on that point: http://www.w3.org/community/respimg/2012/05/11/respimg-proposal/#comments I worry that, when faced with this markup, developers will simply opt to serve the largest possible image in a src. In fairness, that approach "works" with far less headache. -Mat Marquis On May 12, 2012, at 6:43 PM, Jason Grigsby wrote: > On May 11, 2012, at 8:52 PM, Simon Pieters wrote: > >> There seem to be two proposals for what syntax to use for the responsive >> images use case: several elements vs. an attribute. > > There are two proposals because they solve two different use cases. Both use > cases are becoming increasingly important. Unfortunately, these use cases are > commonly collapsed into one. I have done it myself in the past. I tried to > clarify the use cases recently.[1] > > Use case #1 > --- > Document author needs to display different versions of an image at different > breakpoints based on what I’m calling, for a lack of a better phrase, art > direction merits. > > * Example 1: News site shows photograph speaking at a auto factory. On wide > screens, the news site includes a widescreen version of the photograph in > which the cars being built can clearly be seen. On small screens, if the > photograph is simply resized to fit the screen, Obama’s face is too small to > be seen. Instead, the document author may choose to crop the photograph so > that it focuses in on Obama before resizing to fit the smaller screen. [1] > > * Example 2: On the Nokia Browser site where it describes the Meego > browser[2], the Nokia Lumia is show horizontally on wide screens[3]. As the > screen narrows, the Nokia Lumia is then shown vertically and cropped[4]. > Bryan and Stephanie Rieger, the designers of the site, have talked about how > on a wide screen, showing the full phone horizontally showed the browser > best, but on small screens, changing the img to vertical made more sense > because it allowed the reader to still make out the features of the browser > in the image. > > Current proposed solution: element[5] > > Use case #2 > --- > For a variety of reasons, images of various pixel density are needed. These > reasons include current network connection speed, display pixel density, user > data plan, and user preferences. > > * Example 1: The use of high-density images for the new iPad on Apple.com.[6] > > * Example 2: A user on a slow network or with limited data left may > explicitly declare that he or she would like to download a high resolution > because they need to see a sharper version of an image before buying product, > etc. > > Current proposed solution for use case #2: [7] > > IMHO > > Neither proposed solution handles all of the use cases. I’m not convinced > that one solution needs to solve both of them, but I do think if we’re > getting close to implementing one of the proposed solutions, we need to > consider how it would work in conjunction with a solution for the other use > case. > > To be more specific, if were to be implemented in a > browser--potentially solving use case #2, but leaving use case #1 open--what > would happen when we realized that use case #1 still needed to be solved? > Would we end up with some bastardized mixture of and > syntax? > > When Ted proposed , he wrote[7]: > >> Ultimately I don't think addressing the multiple-resolution case needs to >> wait for a solution to these other cases. We don't need to "SOLVE ALL THE >> PROBLEMS!" right now. > > > In a similar vein, the responsive images community group, focused on use case > #1 and explicitly chose to ignore the problems described in use case #2. > > While I agreed with that focus earlier, I now think this may be a mistake. As > much as I don’t want to bog down solving either use case, it seems likely > that if we don’t look at both at the same time, that
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
I didn’t want to cloud my previous email with my opinions on various solutions, but as you may expect, I have some thoughts on the solutions to these two use cases. On May 13, 2012, at 12:43 AM, Jason Grigsby wrote: > Use case #1 > --- > Document author needs to display different versions of an image at different > breakpoints based on what I’m calling, for a lack of a better phrase, art > direction merits. A solution to this use case should put 100% of the control in the hands of authors. It should provide the same control over images in html that is granted to authors over css images by media queries. > Use case #2 > --- > For a variety of reasons, images of various pixel density are needed. These > reasons include current network connection speed, display pixel density, user > data plan, and user preferences. A solution to this use case should allow an author to supply a list of images at different densities (or better yet an image in a format that supports multiple densities), BUT the user agent should decide which image density is right for the situation. Ideally, the user agent would make a decision about the appropriate image based on: * Connection speed * Display density * Data plan * User preference Obviously not all of these can be supported currently and the list isn’t necessarily comprehensive. The point I’m making is that there are multiple factors to picking the right density. The user agent is in the best position to make these decisions. -Jason
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On May 11, 2012, at 8:52 PM, Simon Pieters wrote: > There seem to be two proposals for what syntax to use for the responsive > images use case: several elements vs. an attribute. There are two proposals because they solve two different use cases. Both use cases are becoming increasingly important. Unfortunately, these use cases are commonly collapsed into one. I have done it myself in the past. I tried to clarify the use cases recently.[1] Use case #1 --- Document author needs to display different versions of an image at different breakpoints based on what I’m calling, for a lack of a better phrase, art direction merits. * Example 1: News site shows photograph speaking at a auto factory. On wide screens, the news site includes a widescreen version of the photograph in which the cars being built can clearly be seen. On small screens, if the photograph is simply resized to fit the screen, Obama’s face is too small to be seen. Instead, the document author may choose to crop the photograph so that it focuses in on Obama before resizing to fit the smaller screen. [1] * Example 2: On the Nokia Browser site where it describes the Meego browser[2], the Nokia Lumia is show horizontally on wide screens[3]. As the screen narrows, the Nokia Lumia is then shown vertically and cropped[4]. Bryan and Stephanie Rieger, the designers of the site, have talked about how on a wide screen, showing the full phone horizontally showed the browser best, but on small screens, changing the img to vertical made more sense because it allowed the reader to still make out the features of the browser in the image. Current proposed solution: element[5] Use case #2 --- For a variety of reasons, images of various pixel density are needed. These reasons include current network connection speed, display pixel density, user data plan, and user preferences. * Example 1: The use of high-density images for the new iPad on Apple.com.[6] * Example 2: A user on a slow network or with limited data left may explicitly declare that he or she would like to download a high resolution because they need to see a sharper version of an image before buying product, etc. Current proposed solution for use case #2: [7] IMHO Neither proposed solution handles all of the use cases. I’m not convinced that one solution needs to solve both of them, but I do think if we’re getting close to implementing one of the proposed solutions, we need to consider how it would work in conjunction with a solution for the other use case. To be more specific, if were to be implemented in a browser--potentially solving use case #2, but leaving use case #1 open--what would happen when we realized that use case #1 still needed to be solved? Would we end up with some bastardized mixture of and syntax? When Ted proposed , he wrote[7]: > Ultimately I don't think addressing the multiple-resolution case needs to > wait for a solution to these other cases. We don't need to "SOLVE ALL THE > PROBLEMS!" right now. In a similar vein, the responsive images community group, focused on use case #1 and explicitly chose to ignore the problems described in use case #2. While I agreed with that focus earlier, I now think this may be a mistake. As much as I don’t want to bog down solving either use case, it seems likely that if we don’t look at both at the same time, that we’ll end up with[8]: Which would make no one happy. -Jason [1] http://blog.cloudfour.com/a-framework-for-discussing-responsive-images-solutions/ [2] http://browser.nokia.com/smartphones.html [3] http://browser.nokia.com/resources/images/home-feature.png [4] http://browser.nokia.com/resources/images/smartphones/choose-me...@320.png [5] https://github.com/Wilto/respimg [6] http://blog.cloudfour.com/how-apple-com-will-serve-retina-images-to-new-ipads/ [7] http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-May/035746.html [8] Yes, yes, this is an exaggeration, but you get my point.
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On May 12, 2012, at 12:35 PM, Adam Barth wrote: > On Sat, May 12, 2012 at 11:17 AM, Maciej Stachowiak wrote: >> >> On May 12, 2012, at 6:28 AM, Mathew Marquis wrote: >> >>> While that information may be available at the time the img tag is parsed, >>> I don’t believe it will be available at the time of prefetching — I’m happy >>> to research this further and report back with citations. I’m sure I don’t >>> have to tell you that “disable prefetching on img tags just in case there >>> are matching sources” is going to be a hard sell to vendors that do >>> prefetch. If we’re left with a solution that fetches the original src >>> before applying any custom source logic, well, we’re no better off than we >>> would be with one of the scores of script-based solutions that have come >>> about in the past year. >>> >>> To your original point, though: as much as you can absolutely make a case >>> that a simpler implementation will benefit developers if inherently more >>> stable, you can’t convince me that `img set` suits the needs of developers >>> as well as `picture`. In fact, even if you were to convince me, it wouldn’t >>> matter. Picture is, for better or worse, what developers want and expect in >>> a “responsive images” element. There’s certainly no shortage of proof of >>> that, on this page alone: >>> http://www.w3.org/community/respimg/2012/05/11/respimg-proposal/ At the >>> moment, the Community Group server seems to be down due to excessive >>> traffic. >> >> The key to making the case for the element or something like it is >> to cite use cases. Most of the comments on that blog post just give >> opinions, without use cases backing them up. A lot more weight will be >> placed on explanations of *why* developers love something (e.g. it lets them >> do X where they otherwise couldn't, it lets them do Y more easily, etc) than >> just testimonials that they love it. >> >> Regards, >> Maciej >> >> >> P.S. Your examples in that blog post are not equivalent. Here are two >> examples that I believe would be equivalent for resolution adaptation only, >> presuming a 600x200 image and a 1200x400 scaled version: >> >> >> >> >> >> >> >> >> >> Other than more general verbosity, there are a few other other differences >> that show up: >> >> 1) The version has to repeat the alt text. >> 2) The version has to repeat the URL to the 1x asset. >> 3) The version has to explicitly set a width and height, because >> it does not have the built-in scaling semantics of srcset and so cannot rely >> on intrinsic size, since it will end up different between the two images. >> 4) The version has to use a specific order, while in the srcset >> version, order doesn't matter. > > Does the id="catface_picture" attribute play an essential role in this > example, or is it just extra clutter in the example? Extra clutter. I forgot to delete it after I switched that example from using externally declared
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sat, May 12, 2012 at 11:17 AM, Maciej Stachowiak wrote: > > On May 12, 2012, at 6:28 AM, Mathew Marquis wrote: > >> While that information may be available at the time the img tag is parsed, I >> don’t believe it will be available at the time of prefetching — I’m happy to >> research this further and report back with citations. I’m sure I don’t have >> to tell you that “disable prefetching on img tags just in case there are >> matching sources” is going to be a hard sell to vendors that do prefetch. If >> we’re left with a solution that fetches the original src before applying any >> custom source logic, well, we’re no better off than we would be with one of >> the scores of script-based solutions that have come about in the past year. >> >> To your original point, though: as much as you can absolutely make a case >> that a simpler implementation will benefit developers if inherently more >> stable, you can’t convince me that `img set` suits the needs of developers >> as well as `picture`. In fact, even if you were to convince me, it wouldn’t >> matter. Picture is, for better or worse, what developers want and expect in >> a “responsive images” element. There’s certainly no shortage of proof of >> that, on this page alone: >> http://www.w3.org/community/respimg/2012/05/11/respimg-proposal/ At the >> moment, the Community Group server seems to be down due to excessive traffic. > > The key to making the case for the element or something like it is > to cite use cases. Most of the comments on that blog post just give opinions, > without use cases backing them up. A lot more weight will be placed on > explanations of *why* developers love something (e.g. it lets them do X where > they otherwise couldn't, it lets them do Y more easily, etc) than just > testimonials that they love it. > > Regards, > Maciej > > > P.S. Your examples in that blog post are not equivalent. Here are two > examples that I believe would be equivalent for resolution adaptation only, > presuming a 600x200 image and a 1200x400 scaled version: > > > > > > > > > > Other than more general verbosity, there are a few other other differences > that show up: > > 1) The version has to repeat the alt text. > 2) The version has to repeat the URL to the 1x asset. > 3) The version has to explicitly set a width and height, because it > does not have the built-in scaling semantics of srcset and so cannot rely on > intrinsic size, since it will end up different between the two images. > 4) The version has to use a specific order, while in the srcset > version, order doesn't matter. Does the id="catface_picture" attribute play an essential role in this example, or is it just extra clutter in the example? Adam
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On May 12, 2012, at 6:28 AM, Mathew Marquis wrote: > While that information may be available at the time the img tag is parsed, I > don’t believe it will be available at the time of prefetching — I’m happy to > research this further and report back with citations. I’m sure I don’t have > to tell you that “disable prefetching on img tags just in case there are > matching sources” is going to be a hard sell to vendors that do prefetch. If > we’re left with a solution that fetches the original src before applying any > custom source logic, well, we’re no better off than we would be with one of > the scores of script-based solutions that have come about in the past year. > > To your original point, though: as much as you can absolutely make a case > that a simpler implementation will benefit developers if inherently more > stable, you can’t convince me that `img set` suits the needs of developers as > well as `picture`. In fact, even if you were to convince me, it wouldn’t > matter. Picture is, for better or worse, what developers want and expect in a > “responsive images” element. There’s certainly no shortage of proof of that, > on this page alone: > http://www.w3.org/community/respimg/2012/05/11/respimg-proposal/ At the > moment, the Community Group server seems to be down due to excessive traffic. The key to making the case for the element or something like it is to cite use cases. Most of the comments on that blog post just give opinions, without use cases backing them up. A lot more weight will be placed on explanations of *why* developers love something (e.g. it lets them do X where they otherwise couldn't, it lets them do Y more easily, etc) than just testimonials that they love it. Regards, Maciej P.S. Your examples in that blog post are not equivalent. Here are two examples that I believe would be equivalent for resolution adaptation only, presuming a 600x200 image and a 1200x400 scaled version: Other than more general verbosity, there are a few other other differences that show up: 1) The version has to repeat the alt text. 2) The version has to repeat the URL to the 1x asset. 3) The version has to explicitly set a width and height, because it does not have the built-in scaling semantics of srcset and so cannot rely on intrinsic size, since it will end up different between the two images. 4) The version has to use a specific order, while in the srcset version, order doesn't matter.
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
On Sat, 12 May 2012, Boris Zbarsky wrote: On 5/12/12 9:28 AM, Mathew Marquis wrote: While that information may be available at the time the img tag is parsed, I don’t believe it will be available at the time of prefetching Which information? At least in Gecko, prefetching happens "when the tag is parsed". So in fact in Gecko the srcset approach would be much more amenable to prefetching than the approach. Yes, I should have mentioned that is also true in the various types of optimistic resource loading/parsing that Opera does e.g. the delayed script execution mode and the speculative tokenisation feature.
Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)
While that information may be available at the time the img tag is parsed, I don’t believe it will be available at the time of prefetching — I’m happy to research this further and report back with citations. I’m sure I don’t have to tell you that “disable prefetching on img tags just in case there are matching sources” is going to be a hard sell to vendors that do prefetch. If we’re left with a solution that fetches the original src before applying any custom source logic, well, we’re no better off than we would be with one of the scores of script-based solutions that have come about in the past year. To your original point, though: as much as you can absolutely make a case that a simpler implementation will benefit developers if inherently more stable, you can’t convince me that `img set` suits the needs of developers as well as `picture`. In fact, even if you were to convince me, it wouldn’t matter. Picture is, for better or worse, what developers want and expect in a “responsive images” element. There’s certainly no shortage of proof of that, on this page alone: http://www.w3.org/community/respimg/2012/05/11/respimg-proposal/ At the moment, the Community Group server seems to be down due to excessive traffic. I’m regret that picture would be more difficult to implement; I do. If there’s anyone that can understand that frustration, it’s another developer. But implementing a completely disparate method way of specifying media sources based on client-side information is in no way beneficial to the end author — and certainly holds no provable benefit to the end _user_. All things being equal for the end user, implementor convenience should not take priority over developer convenience. This is the syntax that the developer community almost whole-heartedly agrees upon — and when was the last time you saw that phrase used in reference to proposed standards? If implementor convenience does take priority over that of authors, well, here we are — but let’s not frame it as though it’s doing developers some thickly-veiled favor. They’ve voiced their preference, in no uncertain terms. Mat Marquis On May 11, 2012, at 2:52 PM, Simon Pieters wrote: > There seem to be two proposals for what syntax to use for the responsive > images use case: several elements vs. an attribute. > > I think an attribute is simpler to implement and thus likely to result in > fewer bugs in browsers, which in turn benefits Web developers. > > With , in the parser case, all the information is > available when the tag has been parsed. Excellent! In the scripted case, we > need to "await a stable state" when the src or srcset attributes are changed > (because the same script might also change the other), and then process the > attributes again. The spec already does this for src="" because of the > crossorigin="" attribute, so it's basically a solved problem already for > in the spec. > > With ..., in the parser case, > the browser can't know it has all the information is not available until the > picture element has been popped off the stack (since there might be futher > elements), which necessarily delays fetching the image. Moreover, > scripts might have changed things before then, because scripts can run > between the time that the "picture" element is added to the document and the > time it is popped off the stack, which is a potential source for bugs. In the > scripted case, we need to listen for changes to img elements' parents, > changes to img element's attributes, changes to source elements' parents, and > changes to source elements' attributes, and "await a stable state" and then > process the picture element's img and source elements' attributes (for both > the old and new picture element, if you moved an element between two picture > elements), which is more complex than the attribute case. > > -- > Simon Pieters > Opera Software