subject:"\[whatwg\] Implementation complexity with elements vs an attribute \(responsive images\)"

A common sentiment here seems to be that the two proposed responsive
image solutions solve two different use cases:

- img srcset for serving different resolutions of a content image
(for bandwidth and dpi)
- picture for serving different versions of a content image (for art
direction)

...and that neither solution can deal with both issues. I disagree. I
would describe it as a single, broad use case:
Serving different sources of an image based on properties of the
client. These properties could include:
- Viewport width/height
- Containing element width/height
- Device orientation
- Colour capability
- Old-fashioned media type (screen/print)
- Connection speed
- Pixel density
- Things we haven't thought about/aren't an issue yet

There are media queries already standardised and implemented for many
of the above. Others will surely follow. One of the main strengths of
picture is that is incorporates media queries (rather than its own
syntax for testing properties), which will be developed as time goes
on to include other client properties as they become relevant. This is
good DRY practise.

The differences in the sources could include:
- Resolution
- Compression
- Zoom
- Cropping
- Aspect ratio
- Colour/monochrome

This is all possible with picture. I also agree with many other
developers that picture is definitely the more developer-friendly
syntax, in terms of reading, writing, maintaining and scripting.

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

2012-05-13 Thread James Graham




On Sun, 13 May 2012, David Goss wrote:


A common sentiment here seems to be that the two proposed responsive
image solutions solve two different use cases:

- img srcset for serving different resolutions of a content image
(for bandwidth and dpi)
- picture for serving different versions of a content image (for art
direction)

...and that neither solution can deal with both issues. I disagree. I
would describe it as a single, broad use case:
Serving different sources of an image based on properties of the
client. These properties could include:
- Viewport width/height
- Containing element width/height
- Device orientation
- Colour capability
- Old-fashioned media type (screen/print)
- Connection speed
- Pixel density
- Things we haven't thought about/aren't an issue yet


Which of hese things are actual requirements that people need to meet and 
which are hypothetical? For example I think it is uncontroversial that 
viewport width/height is a real requirement. On the other hand, I have 
never heard of a site that switches assets based on display colour 
capability. Can you point to sites actually switching assets based on each 
property you listed?


Also note that there is a great difference in implementation complexity 
between various properties above. For example, viewport width/height is 
rather easy to work with because one can assume it won't change between 
prefetching and layout, so one can prefetch the right asset. On the other 
hand switching based on containing element width/height requires layout to 
happen before the right asset can be selected, so it has to be loaded 
late. This will significantly decrease the perceived responsiveness of the 
site.


Other properties like connection speed are very difficult to work with 
because they can have high temporal variability e.g. due to sharing of one 
connection by many consumers, due to temporary environmental conditions 
(train goes into a tunnel) or due to switching transports (wifi to 3G, for 
example). My suspicion is that trying to write a solution for switching 
based on connection speed would lead to people getting the wrong assets 
much of the time.


Note that these concerns argue, to a certian extent, *against* reusing a 
very general syntax that can express constraints that aren't relevant to 
the actual use cases, or that provide an attractive nuisance that 
encourages developers to do things that can't be implemented in a 
performant way.

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

2012-05-13 Thread Jason Grigsby

On May 13, 2012, at 9:51 AM, David Goss wrote:

 A common sentiment here seems to be that the two proposed responsive
 image solutions solve two different use cases:

After skyping with Mat (@wilto) last night, I think I may be the only one who 
didn’t fully grok that the mediaqueries in picture could be used to address 
both use cases. 

It is still unclear to me if img srcset would address both.

 I also agree with many other
 developers that picture is definitely the more developer-friendly
 syntax, in terms of reading, writing, maintaining and scripting.


Edward’s original img srcset proposal was pretty straight forward, but as it 
has grown to try to address more use cases, the syntax has become more 
convoluted[1]. I read the latest proposal multiple times last night and still 
couldn’t figure out how it would work.

It may be that the proposal is written in language that implementors understand 
and that it needs to be rewritten to make it clearer for authors how it would 
work. Or it could be an indication that the syntax is too terse and confusing 
for authors (which is currently the feedback the community group is receiving).

-Jason

[1] http://junkyard.damowmow.com/507

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, May 13, 2012 at 10:36 AM, Jason Grigsby ja...@cloudfour.com wrote:
 It may be that the proposal is written in language that implementors 
 understand and that it needs to be rewritten to make it clearer for authors 
 how it would work. Or it could be an indication that the syntax is too terse 
 and confusing for authors (which is currently the feedback the community 
 group is receiving).

Some people seem to think the filenames are part of the syntax. Using
filenames that don't duplicate information in the syntax might
actually be less confusing:

  img src=a.jpg alt=
set=a.jpg 600w 200h 1x,
   b.jpg 600w 200h 2x,
   c.jpg 200w 200h

Perhaps changing the syntax to avoid confusion with units might help too:

  img src=a.jpg alt=
set=a.jpg 600x200 1x,
b.jpg 600x200 2x,
c.jpg 200x200

--
Benjamin Hawkes-Lewis

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On 13 May 2012 10:14, James Graham jgra...@opera.com wrote:

On Sun, 13 May 2012, David Goss wrote:

A common sentiment here seems to be that the two proposed responsive
image solutions solve two different use cases:

- img srcset for serving different resolutions of a content image
(for bandwidth and dpi)
- picture for serving different versions of a content image (for art
direction)

...and that neither solution can deal with both issues. I disagree. I
would describe it as a single, broad use case:
Serving different sources of an image based on properties of the
client. These properties could include:
- Viewport width/height
- Containing element width/height
- Device orientation
- Colour capability
- Old-fashioned media type (screen/print)
- Connection speed
- Pixel density
- Things we haven't thought about/aren't an issue yet

Which of hese things are actual requirements that people need to meet and
which are hypothetical? For example I think it is uncontroversial that
viewport width/height is a real requirement. On the other hand, I have never
heard of a site that switches assets based on display colour capability. Can
you point to sites actually switching assets based on each property you
listed?

Before going on, I'll note that it's difficult to find many sites
actually doing these things right now, the main reason being that the
costs (wasted downloads, very hacky methods) outweigh the benefits
much of the time and stop it being worthwhile to do.

Viewport width/height
As you said, viewport width/height is the obvious one. The Boston
Globe (http://bostonglobe.com/) is an example of a site doing this now
(and using a necessarily hacky method), as described in Mat's article
(http://www.alistapart.com/articles/responsive-images-how-they-almost-worked-and-what-we-need/).

Containing element width/height:
This is kind of hypothetical, at least right now. But authors would
like to be able to apply different CSS (and different image sources)
based on the current context of the target (e.g. a module could be
used multiple times on a site in different places, with various widths
depending on where they are in the layout) - see
http://blog.andyhume.net/responsive-containers/. I appreciate it would
be difficult in terms of syntax and browser implementation. I'm sure
it's no coincidence that no such media query exists yet. But this is
the point: the picture syntax allows us to use future media queries
if/when they arrive. As I understand it, the img srcset syntax would
have to keep getting extended every time we wanted to test a different
property.

Device orientation
This is essentially a part of viewport width/height, as it effectively
denotes is the viewport wider than it is tall, or vice versa.
Perhaps I shouldn't have listed it separately.

Pixel density
Higher pixel density displays are getting more common. The new iPad is
a good example. If you have a large, prominent image (especially a
photo) on a web page that takes up the majority of the screen at once,
you want to take advantage of the extra pixel density by serving a
high resolution image, but it'd be irresponsible to just serve an
img with the high res source to all users, making them wait longer
for the download even though they can't see the extra quality on their
screen. So, you serve the image at an ordinary resolution by default,
or serve the high res version if the pixel density is over a certain
number. Apple are doing this with a JS method, as documented by Jason
Grigsby
(http://blog.cloudfour.com/how-apple-com-will-serve-retina-images-to-new-ipads/),
and in cases where they do serve the high res image, the low res one
is also downloaded. This is one of the problems picture solves:
picture alt=The new iPad
source src=lores.jpg
source src=hires.jpg media=min-resolution: 300dpi
img src=lores.jpg alt=The new iPad
/picture

Connection speed
As an extension of the iPad example above, it would also be
irresponsible to serve the high res image to users that do have a high
pixel density display but are not on a fast internet connection for
whatever reason. So you might write:
picture alt=The new iPad
source src=lores.jpg
source src=hires.jpg media=(min-resolution: 300dpi) and
(min-connection-speed: 1mbps)
img src=lores.jpg alt=The new iPad
/picture

Also note that there is a great difference in implementation complexity
between various properties above. For example, viewport width/height is
rather easy to work with because one can assume it won't change between
prefetching and layout, so one can prefetch the right asset. On the other
hand switching based on containing element width/height requires layout to
happen before the right asset can be selected, so it has to be loaded late.
This will significantly decrease the perceived responsiveness of the site.

Other properties like connection speed are very difficult to work with
because they can have high temporal variability e.g. due to sharing of one

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

2012-05-13 Thread Ashley Sheridan


 
  Also note that there is a great difference in implementation complexity
  between various properties above. For example, viewport width/height is
  rather easy to work with because one can assume it won't change between
  prefetching and layout, so one can prefetch the right asset. On the other
  hand switching based on containing element width/height requires layout to
  happen before the right asset can be selected, so it has to be loaded late.
  This will significantly decrease the perceived responsiveness of the site.
 
  Other properties like connection speed are very difficult to work with
  because they can have high temporal variability e.g. due to sharing of one
  connection by many consumers, due to temporary environmental conditions
  (train goes into a tunnel) or due to switching transports (wifi to 3G, for
  example). My suspicion is that trying to write a solution for switching
  based on connection speed would lead to people getting the wrong assets
  much of the time.
 
 So, if someone's connection speed changes dramatically when the image
 request has already started, they'll either get:
 1 - a high res image, but slowly
 2 - a low res image (relative to what their device can display)
 
 I'm not an expert on internet connections. I'd like to know,
 statistically, how likely it is that a user's connection speed will
 stay consistent (say +/- 20%) for a ten second period. Does anyone
 know of any research like this? If wireless internet connections were
 constantly all over the place it would be a concern, but as someone
 that uses them all the time it doesn't feel that way to me.
 
 I do appreciate that standardised connection speed testing is going to
 be hard. But if and when it comes, it should surely come in the form
 of a media query (as well as a JS API) in which case picture would
 support it automatically.
 
  Note that these concerns argue, to a certian extent, *against* reusing a
  very general syntax that can express constraints that aren't relevant to the
  actual use cases, or that provide an attractive nuisance that encourages
  developers to do things that can't be implemented in a performant way.
 
 Dumping picture as it is won't prevent this, as it's already in the
 door in the form of video.

This may not be as much of a problem as it seems though. A browser could
quite easily keep track of the connection speed over time and determine
a logical average. It could take into account things like the type of
network connection (e.g. a laptop may switch between wireless, tethered
phone or wired, and may move about between various networks of differing
speeds).

This may even be available at the OS level (I believe a Mac allows for
various network configurations based on the network it's attached to)
which would allow for a better experience for anyone using multiple
browsers often.

Thanks,
Ash

http://www.ashleysheridan.co.uk

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, May 13, 2012 at 12:26 PM, David Goss dvdg...@gmail.com wrote:
 As I understand it, the img srcset syntax would
 have to keep getting extended every time we wanted to test a different
 property.

It doesn't test anything. It tells the UA metadata about the image
set it would otherwise have to obtain by downloading them all.

UAs can pick the right image based on this metadata plus what they
know about dimensions and orientation of some container, pixel
density, and connectivity.

It would need to get extended if we learnt of further image metadata
we needed to expose to UAs to help them pick the optimal download
(maybe MIME type).

Media queries test client properties, but this syntax is attempting to
describe an image. If we were trying to avoid reinventing wheels at
all costs, we'd want to look at an image description vocabulary like
EXIF:

http://www.w3.org/2003/12/exif/

--
Benjamin Hawkes-Lewis

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On 13 May 2012, at 13:19, Benjamin Hawkes-Lewis bhawkesle...@googlemail.com 
wrote:

 On Sun, May 13, 2012 at 12:26 PM, David Goss dvdg...@gmail.com wrote:
 As I understand it, the img srcset syntax would
 have to keep getting extended every time we wanted to test a different
 property.
 
 It doesn't test anything. It tells the UA metadata about the image
 set it would otherwise have to obtain by downloading them all.
 
 UAs can pick the right image based on this metadata plus what they
 know about dimensions and orientation of some container, pixel
 density, and connectivity.
 
 It would need to get extended if we learnt of further image metadata
 we needed to expose to UAs to help them pick the optimal download
 (maybe MIME type).
 
 Media queries test client properties, but this syntax is attempting to
 describe an image. If we were trying to avoid reinventing wheels at
 all costs, we'd want to look at an image description vocabulary like
 EXIF:
 
http://www.w3.org/2003/12/exif/
 
 --
 Benjamin Hawkes-Lewis

Yes, you're right, I should have referred to including more metadata not 
testing more properties when referring to the set syntax.

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

2012-05-13 Thread Boris Zbarsky


On 5/13/12 7:26 AM, David Goss wrote:

but it'd be irresponsible to just serve an
img  with the high res source to all users, making them wait longer
for the download even though they can't see the extra quality on their
screen.


Except when they can, e.g. by printing or moving the display to another 
screen.


Basically, in this case not sending the high-res image is optimizing a 
bit for one common use case while degrading the user experience in other 
use cases.  Maybe that's OK, but that requires a careful analysis of 
user behavior for the particular content involved.  And in particular, 
I'm not sure what I think of labeling content that aims to be usable by 
the user in the widest variety of circumstances as irresponsible. 
Past examples of us doing that sort of thing led to it being considered 
irresponsible to allow website layouts to scale above or below certain 
widths, amongst other current usability disasters.



Note that these concerns argue, to a certian extent, *against* reusing a
very general syntax that can express constraints that aren't relevant to the
actual use cases, or that provide an attractive nuisance that encourages
developers to do things that can't be implemented in a performant way.


Dumpingpicture  as it is won't prevent this, as it's already in the
door in the form ofvideo.


video has quite different dynamic behavior from img and quite 
different author expectations (e.g. authors don't expect videos to be 
preloaded in their entirety, by default; there are other significant 
expectation differences too).


-Boris

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

2012-05-13 Thread Odin Hørthe Omdal

Jason Grigsby wrote:
 David Goss wrote:
 A common sentiment here seems to be that the two proposed responsive
 image solutions solve two different use cases:
 
 After skyping with Mat (@wilto) last night, I think I may be the only
 one who didn’t fully grok that the mediaqueries in picture could be
 used to address both use cases. 

 It is still unclear to me if img srcset would address both.

Oh but they *do* solve two different use cases. Mediaqueries in
picture _may_ be able to address both, but img srcset is not, and
never will. It's simply not designed for it.

So, why then do I prefer srcset when it comes to solving the «save
bandwidth»/use correct weight of resource use case?


Because it's design is browser choose instead of web author choose.
It puts these decisions into the browsers control.  The easiest path for
the developers, is also what will make the browser be able to be a good
agent to its user, and decide what it shall download.

The srcset attribute is also much simpler, which makes me think it'll
be used more and also in the correct way. There's strong correlation
between simple and much used.

picture for solving this use case is/will be over engineered, just
because it can be done, doesn't mean it should. For the other use case,
adapting the image for different content, it might be a good candidate
though. source inside picture should also get a srcset attribute
then, so that it would be possible to choose different qualities if
they exist. For relatively advanced sites, it will look like a monster
though, so that's something to be looked more into.


David Goss wrote:
 Connection speed
 As an extension of the iPad example above, it would also be
 irresponsible to serve the high res image to users that do have a high
 pixel density display but are not on a fast internet connection for
 whatever reason. So you might write:
 picture alt=The new iPad
 source src=lores.jpg
 source src=hires.jpg media=(min-resolution: 300dpi) and
 (min-connection-speed: 1mbps)
 img src=lores.jpg alt=The new iPad
 /picture

As I said, this is one of my big gripes with this proposal. I don't
think this'll work satisfactorily, it puts the burden on figuring out
what is correct for that user into the page author's control. That model
skews the responsibility so that instead of only a few browser engines
having to get it right, millions of web page authors have to get it
right.

AND they have to update their sites and mediaqueries when we get
something new to optimize for. I don't think they will do that, based on
how extremely big the problem with -webkit-prefixes are.

I've seen enough of the web to be sceptical.


What if a the author doesn't write that min-connection-speed query
there? And who is the author of a page to decide such things anyway?
What about latency?  Should there be a max-latency: 10ms, there as well?
What about cost? I have a fast line, but it costs money to download
stuff, so I'd like the smaller pictures. What about if I have slow
internet connection, but I'd want to print the page, thus getting the
best version of the image? Or if I'm sitting with my new fancy hires KDE
Tablet and loading your page in a background-tab, letting it render and
be finished while I continue to read the page I'm on (the browser might
very well want to load the highres picture then, even though the
connection is slow, but with MQ, not that lucky).

 (... containing element width/height)
 As I understand it, the img srcset syntax would have to keep getting
 extended every time we wanted to test a different property.

No. It wouldn't, because it only describes the images, nothing more.

Given:

  img src=hero.jpg
    srcset=hero.jpg 800w 400h, hero-lo.jpg 200w 100h, hero-hi.jpg 2x

Say if you're in a browser optimizing for low bandwidth usage, and some
quality at the cost of speed.  The viewport is 800x600.  In the normal
case, the browser would choose hero.jpg because it fits well with its
resource algorithm. However, since being in the special mode, it defers
the prefetch of the image and waits for layout, where it can see that
this picture lies inside a 150px wide box - so it fetches hero-lo.jpg
because it doesn't need more.

With the MediaQueries proposal, you'd need loads of advanced information
to do the same. The browser could not just infer this on its own and
change its algorithms to do it.

Bandwidth might be expensive even though it's fast, so althought they
have over 1mbit speed, they want to get low pictures. With media queries
there's just so many variables, and so much to choose from.


srcset only chooses between different *qualities* of the same image
whereas who knows what mediaqueries does? It's not possible to not do
anything the web page author hasn't told you about. That's why srcset is
so much more powerful for its use case.


The browser is in a better position to decide what quality of image it'd
like to fetch.

The content author, however, is in a better position to lay out the

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

2012-05-13 Thread Mathew Marquis


On May 13, 2012, at 9:42 AM, Odin Hørthe Omdal wrote:
 
 Connection speed
 As an extension of the iPad example above, it would also be
 irresponsible to serve the high res image to users that do have a high
 pixel density display but are not on a fast internet connection for
 whatever reason. So you might write:
 picture alt=The new iPad
 source src=lores.jpg
 source src=hires.jpg media=(min-resolution: 300dpi) and
 (min-connection-speed: 1mbps)
 img src=lores.jpg alt=The new iPad
 /picture
 
 As I said, this is one of my big gripes with this proposal. I don't
 think this'll work satisfactorily, it puts the burden on figuring out
 what is correct for that user into the page author's control. That model
 skews the responsibility so that instead of only a few browser engines
 having to get it right, millions of web page authors have to get it
 right.

We’re speaking in theoreticals here, which is what makes it hard for me to wrap 
my head around it being anyone’s major gripe with either proposal. Maybe a 
bandwidth media query isn’t the solution? Maybe, when the time comes, we do 
rely on the UA to make that decision. Maybe there’s a user preference that 
could be set. This line of reasoning doesn’t speak for or against either 
approach, however.

 
 AND they have to update their sites and mediaqueries when we get
 something new to optimize for. I don't think they will do that, based on
 how extremely big the problem with -webkit-prefixes are.
 
 I've seen enough of the web to be sceptical.

The amount of “developers can never be trusted with this” sentiment I’ve heard 
from the members of this group is incredibly depressing.

When we get “something new to optimize for,” we start adding that thing going 
forward. The evolution of media queries—or, say, HTML5—hasn’t led to a need to 
constantly revisit every piece of client work a developer has ever produced. 
We’re speaking in terms of optimization because that’s what it is: incremental 
improvements to the way assets are loaded. As more become available, requests 
become more tailored.

 
 
 What if a the author doesn't write that min-connection-speed query
 there? And who is the author of a page to decide such things anyway?
 What about latency?  Should there be a max-latency: 10ms, there as well?
 What about cost? I have a fast line, but it costs money to download
 stuff, so I'd like the smaller pictures. What about if I have slow
 internet connection, but I'd want to print the page, thus getting the
 best version of the image? Or if I'm sitting with my new fancy hires KDE
 Tablet and loading your page in a background-tab, letting it render and
 be finished while I continue to read the page I'm on (the browser might
 very well want to load the highres picture then, even though the
 connection is slow, but with MQ, not that lucky).

I think this has has forked well into “should we have bandwidth available in a 
media query” territory. It’s a worthwhile topic for sure, but apart from being 
a minor and—again—purely speculative factor in considering an adaptive images 
approach, it is another conversation.


Mat Marquis

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, May 13, 2012 at 5:21 PM, Mathew Marquis m...@matmarquis.com wrote:
 AND they have to update their sites and mediaqueries when we get
 something new to optimize for. I don't think they will do that, based on
 how extremely big the problem with -webkit-prefixes are.

 I've seen enough of the web to be sceptical.

 The amount of “developers can never be trusted with this” sentiment I’ve 
 heard from the members of this group is incredibly depressing.

That it depresses you does not mean that taking a more optimistic
viewpoint will produce specifications that will result in better
end-user experiences.

 When we get “something new to optimize for,” we start adding that thing going 
 forward. The evolution of media queries—or, say, HTML5—hasn’t led to a need 
 to constantly revisit every piece of client work a developer has ever 
 produced.

In the case of webkit prefixes, authors have needed to update their
work, have failed to do so, and now user agents are having to support
webkit prefixes.

The key problem about designing a responsive images solution around
user agent characteristics not image characteristics is that authors
will inevitably make more false assumptions about what images match
what user agent characteristics than user agents will. As a result,
user agents may be forced to misinterpret media queries in order to
provide their users with better user experiences.

What authors _can_ do and user agents _cannot_ do is describe their
images. Such metadata never needs to be misinterpreted and allows user
agents to iterate and improve the end-user experience even when the
author either does not care about them, or has moved on, or is long
dead. Is authors not caring about user agents a real problem?
Certainly is - witness Opera's failed efforts to get authors to
support more than Webkit, witness the widespread inaccessibility of
web services, etc.

Depressing but true.

--
Benjamin Hawkes-Lewis

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, 13 May 2012 18:01:12 +0100, Benjamin Hawkes-Lewis  
bhawkesle...@googlemail.com wrote:



What authors _can_ do and user agents _cannot_ do is describe their
images. Such metadata never needs to be misinterpreted and allows user
agents to iterate and improve the end-user experience even when the
author either does not care about them, or has moved on, or is long
dead.


Indeed. An interesting use-case has come up on this list — use  
highest-resolution image when user chooses Save As.


While it's not an earth-shattering feature, it's a nice touch a browser  
could do having information about image. It wouldn't be possible/sensible  
to do with a media query.


Even if there was support for something like source  
media=user-action:save-as, I can bet that less than 0.5% of pages would  
use it and do it correctly.


--
regards, Kornel Lesiński

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, 13 May 2012 10:36:37 +0100, Jason Grigsby ja...@cloudfour.com  
wrote:



On May 13, 2012, at 9:51 AM, David Goss wrote:


A common sentiment here seems to be that the two proposed responsive
image solutions solve two different use cases:


After skyping with Mat (@wilto) last night, I think I may be the only  
one who didn’t fully grok that the mediaqueries in picture could be  
used to address both use cases.


It is still unclear to me if img srcset would address both.


I think layout (media queries) and optimisation cases are orthogonal and  
it would be a mistake to do both with the same mechanism.


Adaptation of images to the layout is page-specific. Adaptation of images  
to bandwidth/screen is UA/device-specific.


Author is in the best position to adapt image to page layout. User-agent  
is in the best position to determine speed/quality trade-offs.


Media queries MUST be interpreted exactly as author specified them.  
User-agents need freedom to choose image resolution based on open set of  
factors, many of which are details authors should not have to think about  
(presence in cache, cost of bandwidth, available memory, external  
displays, etc.)



So even though both proposals are about selecting right images, their  
purposes and requirements are very different.


--
regards, Kornel Lesiński

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

2012-05-13 Thread Mathew Marquis


On May 13, 2012, at 1:01 PM, Benjamin Hawkes-Lewis 
bhawkesle...@googlemail.com wrote:

 On Sun, May 13, 2012 at 5:21 PM, Mathew Marquis m...@matmarquis.com wrote:
 AND they have to update their sites and mediaqueries when we get
 something new to optimize for. I don't think they will do that, based on
 how extremely big the problem with -webkit-prefixes are.
 
 I've seen enough of the web to be sceptical.
 
 The amount of “developers can never be trusted with this” sentiment I’ve 
 heard from the members of this group is incredibly depressing.
 
 That it depresses you does not mean that taking a more optimistic
 viewpoint will produce specifications that will result in better
 end-user experiences.
 
 When we get “something new to optimize for,” we start adding that thing 
 going forward. The evolution of media queries—or, say, HTML5—hasn’t led to a 
 need to constantly revisit every piece of client work a developer has ever 
 produced.
 
 In the case of webkit prefixes, authors have needed to update their
 work, have failed to do so, and now user agents are having to support
 webkit prefixes.
 
 The key problem about designing a responsive images solution around
 user agent characteristics not image characteristics is that authors
 will inevitably make more false assumptions about what images match
 what user agent characteristics than user agents will. As a result,
 user agents may be forced to misinterpret media queries in order to
 provide their users with better user experiences.

Correct me if I’m wrong, but wouldn’t this same reasoning apply to layouts that 
depend heavily on media queries? 

The primary characteristics in play are screen size and resolution. I doubt any 
UAs will be forced to misinterpret common media queries because they haven’t 
been accounted for. I doubt UAs could, in fact—or would care. It wouldn’t be a 
matter of neglecting to support their product. It would be a matter of 
neglecting to supporting a screen at 310px wide.

This is to say nothing of the fact that the objective of a responsive layout 
_is_ to properly accommodate the widest possible range of displays. The things 
you’re trying to prevent are the things we’re trying to avoid anyway.

 
 What authors _can_ do and user agents _cannot_ do is describe their
 images. Such metadata never needs to be misinterpreted and allows user
 agents to iterate and improve the end-user experience even when the
 author either does not care about them, or has moved on, or is long
 dead. Is authors not caring about user agents a real problem?
 Certainly is - witness Opera's failed efforts to get authors to
 support more than Webkit, witness the widespread inaccessibility of
 web services, etc.
 
 Depressing but true.
 
 --
 Benjamin Hawkes-Lewis

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, 13 May 2012 01:33:25 +0100, Mathew Marquis m...@matmarquis.com  
wrote:


I worry that, when faced with this markup, developers will simply opt to  
serve the largest possible image in a src. In fairness, that approach  
works with far less headache.


In the long term that may be a very sensible approach. Selection of 1x/2x  
images is relevant only as long as we have 100dpi screens and slow  
connections, and both will disappear over time.


Perhaps we should think about making syntax for 200dpi+ images with  
intrinsic dimensions easy?



Selection between images for different screen sizes/orientations isn't a  
problem that will go away any time soon, so I think picture is a good  
idea regardless of solution for DPI/bandwidth problem.
As long as HTML claims to be independent of CSS there is no solution for  
that (i.e authors shouldn't be adding multiple img and showing one of  
them with CSS).




How about that:

picture
	source src=narrow_low-quality srcset=narrow_hi-quality 2x  
media=max-width:4in

source src=wide_low-quality srcset=wide_hi-quality 2x

img src=fallback alt=alt
/picture


Instead of srcset it could be src2x or another attribute that specifies  
image for higher screen density and/or bandwidth. The point is that  
media= would allow author to choose image version adapted to page  
layout, and another mechanism connected to source would allow UA to  
choose image resolution.


--
regards, Kornel Lesiński

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, 13 May 2012 20:20:54 +0100, Mathew Marquis m...@matmarquis.com  
wrote:



The key problem about designing a responsive images solution around
user agent characteristics not image characteristics is that authors
will inevitably make more false assumptions about what images match
what user agent characteristics than user agents will. As a result,
user agents may be forced to misinterpret media queries in order to
provide their users with better user experiences.


Correct me if I’m wrong, but wouldn’t this same reasoning apply to  
layouts that depend heavily on media queries?


No, those are very different situations.

Layout is under author's control and is known to the author when the page  
is authored. UA characteristics are not, and different UAs will have  
different requirements.


Conversely from UA perspective: layouts vary wildly between pages, so UA  
can't really mess with them. Bandwidth/DPI requirements are known to UA  
and there can be same set of UA-specific rules that works for all pages.


--
regards, Kornel Lesiński

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, May 13, 2012 at 6:01 PM, Benjamin Hawkes-Lewis
bhawkesle...@googlemail.com wrote:
 On Sun, May 13, 2012 at 5:21 PM, Mathew Marquis m...@matmarquis.com wrote:
 When we get ?something new to optimize for,? we start adding that thing 
 going forward. The evolution of media queries?or, say, HTML5?hasn?t led to a 
 need to constantly revisit every piece of client work a developer has ever 
 produced.

 In the case of webkit prefixes, authors have needed to update their
 work, have failed to do so, and now user agents are having to support
 webkit prefixes.

Agreed, but I don't think it's a fair comparison - we're not talking
about deprecating img with either new proposal. Existing work isn't
becoming invalid. It'll be something we can use to do a better job
from now on.

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

2012-05-13 Thread Bjartur Thorlacius

On 5/13/12, Kornel Lesiński kor...@geekhood.net wrote:
 I think layout (media queries) and optimisation cases are orthogonal and
 it would be a mistake to do both with the same mechanism.

My knee-jerk reaction to the above thought is that layout should be
done using CSS and any optimizations left up to the UA. A bandwidth
constrained UA could request a downsized thumbnail that fits the size
of the object/img/video poster/a element, or render an
appropriately sized bitmap from a SVG.

The problem with that, though, is that then bandwidth constraints
can't affect layout. Users should be able to configure UAs to use
downsized images even given a large viewport, if only to save
bandwidth and reserve a larger fraction of the viewport for text
columns.

 Adaptation of images to the layout is page-specific. Adaptation of images
 to bandwidth/screen is UA/device-specific.

Quite. But the latter just might affect the layout.

 Author is in the best position to adapt image to page layout. User-agent
 is in the best position to determine speed/quality trade-offs.

But low-res images usually don't look too good when upscaled. Thus few
pixels should mean small image, UAs mustn't default to pixelation.

 Media queries MUST be interpreted exactly as author specified them.
Thus we mustn't force UAs to pretend to render to small viewports to
find low-res images. That would have unwieldy side-effects.

 User-agents need freedom to choose image resolution based on open set of
 factors, many of which are details authors should not have to think about
 (presence in cache, cost of bandwidth, available memory, external
 displays, etc.)

But the chosen image resolution might be a factor for choosing layout.

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, May 13, 2012 at 8:55 PM, Bjartur Thorlacius
svartma...@gmail.com wrote:
 But the chosen image resolution might be a factor for choosing layout.

Maybe we should think of a way to expose _that_ information to CSS,
rather than going in the other direction.

section
  img src=a.jpg alt=
   set=a.jpg 600x200 1x,
   b.jpg 600x200 2x,
   c.jpg 200x200
/section

section { /* generic style rules */ }
section! img:intrinsic-aspect-ratio(2) { /* specific overrides for
section when the UA picks the narrow image */ }

--
Benjamin Hawkes-Lewis

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

2012-05-13 Thread Odin Hørthe Omdal

Kornel Lesiński wrote:
 Selection of 1x/2x images is relevant only as long as we have 100dpi
 screens and slow connections, and both will disappear over time.

Well, the web is a huge place. I'm quite sure that'll take ages and
ages if it ever happens at all (I don't think it'll ever disappear).
Might get irrelevant for some small and specific markets/segments though.

 How about that:

  picture
   source src=narrow_low-quality srcset=narrow_hi-quality 2x  
  media=max-width:4in
   source src=wide_low-quality srcset=wide_hi-quality 2x
 
   img src=fallback alt=alt
  /picture

 Instead of srcset it could be src2x or another attribute that specifies  
 image for higher screen density and/or bandwidth. The point is that  
 media= would allow author to choose image version adapted to page  
 layout, and another mechanism connected to source would allow UA to  
 choose image resolution.

Seeing it here in code it's actually not such a monster that I'd said
it'd be. So I like it even more, and it's the obvious way for these to
interact.

I think it'd be a mistake to call it src2x though, -- it feels very
specific. You can scale up to double then, but you can't necessarily go
beyond that: going down for e.g. mobile.

OTOH, 2x will be the most common usage at least as far as I can tell.

  img src=dog.jpg src2x=d...@2.jpg

  vs.

  img src=dog.jpg srcset=d...@2.jpg 2x

is not really all that different, but the second should be more
flexible. Also downscaling:

  img src=dog.jpg srcset=d...@2.jpg 2x, dog-lo.jpg 0.5x

Actually, for this to work, the user agent needs to know the size of the
standard image. So:

  img src=dog.jpg width=960
srcset=d...@2.jpg 2x, dog-lo.jpg 500w

So if you've got the smartphone held in portrait, it's 250 css pixels
wide, and so 500 real pixels, it could opt to show dog-lo.jpg rather
than dog.jpg.

-- 
Odin Hørthe Omdal, Core, Opera Software

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, 13 May 2012 20:55:08 +0100, Bjartur Thorlacius  
svartma...@gmail.com wrote:



The problem with that, though, is that then bandwidth constraints
can't affect layout. Users should be able to configure UAs to use
downsized images even given a large viewport, if only to save
bandwidth and reserve a larger fraction of the viewport for text
columns.


That wasn't my assumption.

For optimisation case expected images to vary only in compression strength  
or DPI (low-res JPEG or 2x image for Retina display), but never in their  
dimension in CSS pixels.



I think page should not be allowed to change layout based on bandwidth  
availability, because of browser caches. Imagine scenario:


1. User visits page on high-bandwidth connection and UA caches some high  
quality images.

2. Connection changes to slower one.
3. User visits the page again. HTML was non-cacheable, but images were  
cacheable.


Now the page may have mix of high-bandwidth and low-bandwidth content.  
It's entirely possible to have mixed quality with images if they have same  
CSS pixel size (and use of higher quality images whenever possible is  
desirable), but a page cannot contain mix of multiple different layouts at  
the same time.


If bandwidth was a media query, then entire page may have to be downgraded  
to low-bandwidth version even if UA had a lot of high-bandwidth content  
cached.


Adaptation of images to the layout is page-specific. Adaptation of  
images to bandwidth/screen is UA/device-specific.



Quite. But the latter just might affect the layout.


I think optimisation case should be forbidden from affecting the layout.


Author is in the best position to adapt image to page layout. User-agent
is in the best position to determine speed/quality trade-offs.


But low-res images usually don't look too good when upscaled. Thus few
pixels should mean small image, UAs mustn't default to pixelation.


I don't understand why UA would default to pixelation?

The whole point of declaring available image versions is giving UA  
possibility to default to whatever it deems best.


When user is on GPRS or roaming connection then pixelation is a great  
outcome compared to not being able to download images at all due to  
prohibitive size/price.


When user is on broadband connection then UA can select high resolution  
images and avoid pixelation when network/hardware allows it.


And when image properties are specified declaratively, a smart UA can even  
do a mix of both, e.g. download pixelated images on mobile connections and  
give user option to selectively download higher-res images, or download  
low-res first for fast initial display and then download high-res in the  
background.



Media queries MUST be interpreted exactly as author specified them.

Thus we mustn't force UAs to pretend to render to small viewports to
find low-res images. That would have unwieldy side-effects.


Indeed!


User-agents need freedom to choose image resolution based on open set of
factors, many of which are details authors should not have to think  
about (presence in cache, cost of bandwidth, available memory, external

displays, etc.)


But the chosen image resolution might be a factor for choosing layout.


By resolution I mean pixel density (regular vs Retina display), so this  
doesn't affect layout.



I can imagine layout complexity being tied to bandwidth (an image-rich  
design vs minimalistic text-only design), but I'm not sure how that would  
work in practice given that cache has infinite bandwidth, and network  
speed can change any second on mobile connections.


It would be weird if page design changed when you moved between cell  
towers or left/entered a cafe that had public WiFi. And if bandwidth media  
query was defined to be fixed, then you'd sometimes end up stuck with  
wrong design that was chosen based on a temporary network state.


There is no such problem if only same-CSS-pixel-size images are swapped  
in-place.


--
regards, Kornel Lesiński

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

On Sun, 13 May 2012 21:23:58 +0100, Odin Hørthe Omdal odi...@opera.com  
wrote:



 picture
  source src=narrow_low-quality srcset=narrow_hi-quality 2x
 media=max-width:4in
  source src=wide_low-quality srcset=wide_hi-quality 2x

  img src=fallback alt=alt
 /picture

Instead of srcset it could be src2x or another attribute that specifies
image for higher screen density and/or bandwidth. The point is that
media= would allow author to choose image version adapted to page
layout, and another mechanism connected to source would allow UA to
choose image resolution.


Seeing it here in code it's actually not such a monster that I'd said
it'd be. So I like it even more, and it's the obvious way for these to
interact.

I think it'd be a mistake to call it src2x though, -- it feels very
specific. You can scale up to double then, but you can't necessarily go
beyond that: going down for e.g. mobile.

OTOH, 2x will be the most common usage at least as far as I can tell.

  img src=dog.jpg src2x=d...@2.jpg

  vs.

  img src=dog.jpg srcset=d...@2.jpg 2x

is not really all that different, but the second should be more
flexible. Also downscaling:

  img src=dog.jpg srcset=d...@2.jpg 2x, dog-lo.jpg 0.5x


Yes, good point.


Actually, for this to work, the user agent needs to know the size of the
standard image. So:

  img src=dog.jpg width=960
srcset=d...@2.jpg 2x, dog-lo.jpg 500w

So if you've got the smartphone held in portrait, it's 250 css pixels
wide, and so 500 real pixels, it could opt to show dog-lo.jpg rather
than dog.jpg.


But still displayed at 960 CSS pixels or course? That'd be fine (and the  
UA could even download dog@2x when user zooms in).


--
regards, Kornel Lesiński

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)

2012-05-13 Thread Bjartur Thorlacius

On 5/13/12, Kornel Lesiński kor...@geekhood.net wrote:
 By resolution I mean pixel density (regular vs Retina display), so this
 doesn't affect layout.

Ah, I must have misunderstood you.

 I can imagine layout complexity being tied to bandwidth (an image-rich
 design vs minimalistic text-only design), but I'm not sure how that would
 work in practice given that cache has infinite bandwidth, and network
 speed can change any second on mobile connections.

The layout would not depend directly on bandwidth, but on an upper limit on
graphic-heaviness.

 It would be weird if page design changed when you moved between cell
 towers or left/entered a cafe that had public WiFi. And if bandwidth media

 query was defined to be fixed, then you'd sometimes end up stuck with
 wrong design that was chosen based on a temporary network state.

There's no question that bandwidth media queries would be a bad idea.
How would you measure bandwidth anyway? My thinnest downlink is a few
Mb/s, but I'm charged an increasing amount for every 10Gb/mo of
international downloads. Authors should of course not evaluate how
important their graphics are to users against bandwidth scarcity
(artificial or real).

 There is no such problem if only same-CSS-pixel-size images are swapped
 in-place.

True.

I've got a hunch I'm over-thinking this, but might
bandwidth-constrained users not prefer miniatures instead of huge
pixelated images?

Re: [whatwg] Implementation complexity with elements vs an attribute (responsive images)