Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Thu, Apr 16, 2015 at 7:07 AM, Justin Novosad ju...@google.com wrote:

 In the interest of moving forward with this, I began an experimental
 implementation of the renderedPixelWidth/Height propsal (
 https://wiki.whatwg.org/wiki/CanvasRenderedPixelSize) in Blink.  I ran
 into
 some issues, which I documented in the issues section of the proposal.  I
 would like to draw your attentions to them at this point:

 1. Hard to determine a rendered size when the canvas is not attached to the
 DOM. Perhaps in that case the current intrinsic size should be returned?


I think in this case you should estimate it based on the current zoom level
of the document, i.e. just assume no CSS transforms will be applied to the
element.


 2. After a layout change that affects rendered pixel size, there is no
 guarantee that the size change event will be handled before the layout
 change is propagated to screen, so the content may be temporarily displayed
 in an inconsistent state. Note: the same issue exists with existing methods
 that may be based on mutation observers or window.onresize, for example.
 Though it is not the stated objective of this proposal to solve this
 synchronization problem, there may be an opportunity to do so.
 3. Accessing rendered pixel size is layout-inducing. To avoid layout
 thrashing, we should consider making this an asynchronous getter (e.g.
 asyncGetBoundignClientRect). This would also prevent renderedsizechanged
 events from firing from within the evaluation of the
 renderedPixelWidth/Height
 attributes, which is weird.


renderedsizechange isn't exactly the same as resize. For example during
zooming (any kind of zooming), an element with a fixed CSS px size does not
need to receive a resize event, but it should probably receive a
renderedsizechange.

I think we should keep renderedsizechange as a separate event, but we do
need to adjust how it fires. I think we should flush renderedsizechange
events before painting, just like we do for onscroll events.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] High-density canvases

[Elliott Sprehn]

On Wed, Apr 15, 2015 at 12:07 PM, Justin Novosad ju...@google.com wrote:

 In the interest of moving forward with this, I began an experimental
 implementation of the renderedPixelWidth/Height propsal (
 https://wiki.whatwg.org/wiki/CanvasRenderedPixelSize) in Blink.  I ran
 into
 some issues, which I documented in the issues section of the proposal.  I
 would like to draw your attentions to them at this point:

 1. Hard to determine a rendered size when the canvas is not attached to the
 DOM. Perhaps in that case the current intrinsic size should be returned?
 2. After a layout change that affects rendered pixel size, there is no
 guarantee that the size change event will be handled before the layout
 change is propagated to screen, so the content may be temporarily displayed
 in an inconsistent state. Note: the same issue exists with existing methods
 that may be based on mutation observers or window.onresize, for example.
 Though it is not the stated objective of this proposal to solve this
 synchronization problem, there may be an opportunity to do so.


MutationObservers fire in the micro task, which is always before the next
paint (actually before the next task). window.onresize runs at
requestAnimationFrame time in Chrome which also means it never misses the
next paint.

3. Accessing rendered pixel size is layout-inducing. To avoid layout
 thrashing, we should consider making this an asynchronous getter (e.g.
 asyncGetBoundignClientRect). This would also prevent renderedsizechanged
 events from firing from within the evaluation of the
 renderedPixelWidth/Height
 attributes, which is weird.


renderedsizechanged feels like it's related to the general per element
resize event problem. It'd be unfortunate to add an event to the browser
specifically for canvas instead of solving the general problem. In that
case authors will start resorting to hacks where they insert canvases into
the page to listen for resizes (we've seen this with other events like
overflowchanged).

Perhaps we should add per element resize events and spec that they fire
after the layout, but before the paint, during normal frame creation. That
does mean you can cause an infinite loop if your resize handler keeps
mutating the page, but there's so many other ways to cause that already.

- E

Re: [whatwg] High-density canvases

[Dean Jackson]

 On 16 Apr 2015, at 6:42 am, Elliott Sprehn espr...@chromium.org wrote:
 
 3. Accessing rendered pixel size is layout-inducing. To avoid layout
 thrashing, we should consider making this an asynchronous getter (e.g.
 asyncGetBoundignClientRect). This would also prevent renderedsizechanged
 events from firing from within the evaluation of the renderedPixelWidth/Height
 attributes, which is weird.
 
 renderedsizechanged feels like it's related to the general per element resize 
 event problem. It'd be unfortunate to add an event to the browser 
 specifically for canvas instead of solving the general problem. In that case 
 authors will start resorting to hacks where they insert canvases into the 
 page to listen for resizes (we've seen this with other events like 
 overflowchanged).
 
 Perhaps we should add per element resize events and spec that they fire after 
 the layout, but before the paint, during normal frame creation. That does 
 mean you can cause an infinite loop if your resize handler keeps mutating the 
 page, but there's so many other ways to cause that already.

+1

Dean

Re: [whatwg] High-density canvases

[Justin Novosad]

In the interest of moving forward with this, I began an experimental
implementation of the renderedPixelWidth/Height propsal (
https://wiki.whatwg.org/wiki/CanvasRenderedPixelSize) in Blink.  I ran into
some issues, which I documented in the issues section of the proposal.  I
would like to draw your attentions to them at this point:

1. Hard to determine a rendered size when the canvas is not attached to the
DOM. Perhaps in that case the current intrinsic size should be returned?
2. After a layout change that affects rendered pixel size, there is no
guarantee that the size change event will be handled before the layout
change is propagated to screen, so the content may be temporarily displayed
in an inconsistent state. Note: the same issue exists with existing methods
that may be based on mutation observers or window.onresize, for example.
Though it is not the stated objective of this proposal to solve this
synchronization problem, there may be an opportunity to do so.
3. Accessing rendered pixel size is layout-inducing. To avoid layout
thrashing, we should consider making this an asynchronous getter (e.g.
asyncGetBoundignClientRect). This would also prevent renderedsizechanged
events from firing from within the evaluation of the renderedPixelWidth/Height
attributes, which is weird.

Re: [whatwg] High-density canvases

[Kenneth Russell]

On Wed, Apr 15, 2015 at 1:46 PM, Dean Jackson d...@apple.com wrote:

 On 16 Apr 2015, at 6:42 am, Elliott Sprehn espr...@chromium.org wrote:

 3. Accessing rendered pixel size is layout-inducing. To avoid layout

 thrashing, we should consider making this an asynchronous getter (e.g.
 asyncGetBoundignClientRect). This would also prevent renderedsizechanged
 events from firing from within the evaluation of the
 renderedPixelWidth/Height
 attributes, which is weird.


 renderedsizechanged feels like it's related to the general per element
 resize event problem. It'd be unfortunate to add an event to the browser
 specifically for canvas instead of solving the general problem. In that case
 authors will start resorting to hacks where they insert canvases into the
 page to listen for resizes (we've seen this with other events like
 overflowchanged).

 Perhaps we should add per element resize events and spec that they fire
 after the layout, but before the paint, during normal frame creation. That
 does mean you can cause an infinite loop if your resize handler keeps
 mutating the page, but there's so many other ways to cause that already.


 +1

If that's a viable option let's do that. It would solve many
longstanding problems.

-Ken

Re: [whatwg] High-density canvases

[Justin Novosad]

On Fri, Jun 27, 2014 at 6:42 PM, Robert O'Callahan rob...@ocallahan.org
wrote:

 On Sat, Jun 28, 2014 at 3:41 AM, Justin Novosad ju...@google.com wrote:

 Example of a problematic renderedsizechange event listener:

 canvas.width = Math.floor(canvas.renderedPixelWidth * myPixelScale);
 canvas.height = Math.floor(canvas.renderedPixelHeight * myPixelScale);


 I don't know why anyone would actually want to do this.


Could be performance, saving memory, retro game pixelated look, who knows.
For sure people are going to end up doing stuff like this if the API allows
it. The scary part is that web-developpers may test their page and
everything will seem OK, but then some clients that have a full page zoom
at a specific ratio or

If we don't want to bother supporting cases like this, then we should
constrain the API. For example we could implement an auto-resize option
instead of exposing renderedPixelWidth/Height, that way the browser stays
in charge of the resizing.


 This behavior seems sound at first glance, but because that arithmetic may
 induce a change to the intrinsic aspect ratio due to rounding, step 3) may
 not jump out of the loop during the second iteration, thus allowing the
 feedback loop to keep spinning.


 What are the specified values of CSS 'width' and 'height' on the canvas in
 your example?


Anything but 'auto'.



 Rob
 --
 Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
 le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
 stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
 'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
 waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Thu, Jul 3, 2014 at 3:03 AM, Justin Novosad ju...@google.com wrote:

 On Fri, Jun 27, 2014 at 6:42 PM, Robert O'Callahan rob...@ocallahan.org
 wrote:

 This behavior seems sound at first glance, but because that arithmetic may
  induce a change to the intrinsic aspect ratio due to rounding, step 3)
 may
  not jump out of the loop during the second iteration, thus allowing the
  feedback loop to keep spinning.
 
 
  What are the specified values of CSS 'width' and 'height' on the canvas
 in
  your example?
 

 Anything but 'auto'.


If both 'width' and 'height' are not 'auto' then changing the canvas width
and height will not trigger a change in the canvas content-box, so
renderedPixelWidth/Height should not change, so there's no loop here.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Justin Novosad]

On Thu, Jun 26, 2014 at 6:25 PM, Robert O'Callahan rob...@ocallahan.org
wrote:

 On Fri, Jun 27, 2014 at 2:00 AM, Justin Novosad ju...@google.com wrote:

 Hadn't thought of that. object-fit seems smells dangerous. I think we may
 need to define explicit behaviors for renderedPixelWidth/Height for the
 different object fit modes.


 I don't think so. Given renderedPixelWidth/Height returns the size of the
 content box, and the element's CSS width and height are not 'auto', then
 renderedPixelWidth/Height are not affected by object-fit or the intrinsic
 size, so there is no feedback loop.


I agree there is no feedback loop in the browser, but the event listener
may create one. See below.



 For example, with 'object-fit: contains', will the
 renderedPixelWidth/Height
 be computed in a way to fill the element's content area, or will it
 preserve the aspect ratio of the original intrinsic size?


 The former.


I agree that makes most sense, but because the canvas width and height are
integers, there will be some numerical instability and the aspect ratio may
drift over time as the canvas gets resized multiple times in succession.
This is because the intrinsic size used to compute the the new
renderedPixelWidth/Height is presumably the renderedPixelWidth/Height from
the previous resize iteration and so-on.




 Also, with object fit triggering a renderedsizechange event, the event
 listener will presumably change the intrinsic size of the canvas, which
 will invalidate style (because the object-fit computation depends on the
 intrinsic size), and that causes a style invalidation feedback loop.


 We don't implement object-fit in Gecko yet, but when we do the change to
 the intrinsic size will trigger a relayout that will end up not change
 anything so there is no feedback loop.


Whether or not there is a feedback loop is not entirely up to the browser.
It depends what the event listener does.

Control flow:
1) CSS size of the canvas is modified
2) layout - renderedPixelWidth/Height are recomputed based on the new
content box dimensions
3) if renderedPixelWidth/Height are unchanged jump to step 7
4) renderedsizechange event is fired
5) renderedsizechange event handler changes the canvas's intrinsic size.
6) if the canvas's intrinsic size changed and the element uses object-fit,
then jump to step 2. (feedback loop)
7) Paint page content

For this not to loop indefinitely depends on the event listener having
stable behavior. If the listener just sets the canvas size to
renderedPixelWidth/Height, I am pretty sure this will always be fine (two
pass layout). But in the case where the event listener does something like
this, it may take several more layout iterations for things to stabilize,
or it may oscillate indefinitely.

Example of a problematic renderedsizechange event listener:

canvas.width = Math.floor(canvas.renderedPixelWidth * myPixelScale);
canvas.height = Math.floor(canvas.renderedPixelHeight * myPixelScale);

This behavior seems sound at first glance, but because that arithmetic may
induce a change to the intrinsic aspect ratio due to rounding, step 3) may
not jump out of the loop during the second iteration, thus allowing the
feedback loop to keep spinning.

   -Justin



 Rob
 --
 Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
 le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
 stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
 'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
 waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Sat, Jun 28, 2014 at 3:41 AM, Justin Novosad ju...@google.com wrote:

 Example of a problematic renderedsizechange event listener:

 canvas.width = Math.floor(canvas.renderedPixelWidth * myPixelScale);
 canvas.height = Math.floor(canvas.renderedPixelHeight * myPixelScale);


I don't know why anyone would actually want to do this.

This behavior seems sound at first glance, but because that arithmetic may
 induce a change to the intrinsic aspect ratio due to rounding, step 3) may
 not jump out of the loop during the second iteration, thus allowing the
 feedback loop to keep spinning.


What are the specified values of CSS 'width' and 'height' on the canvas in
your example?

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Justin Novosad]

Hadn't thought of that. object-fit seems smells dangerous. I think we may
need to define explicit behaviors for renderedPixelWidth/Height for the
different object fit modes.
For example, with 'object-fit: contains', will the renderedPixelWidth/Height
be computed in a way to fill the element's content area, or will it
preserve the aspect ratio of the original intrinsic size?
Etc.

Also, with object fit triggering a renderedsizechange event, the event
listener will presumably change the intrinsic size of the canvas, which
will invalidate style (because the object-fit computation depends on the
intrinsic size), and that causes a style invalidation feedback loop.  If
the event listener just sets the intrinsic size to renderedPixelWidth/Height,
it's not so bad because the computation will stabilize after the first
iteration, so we just end up with a two-pass style computation. But if
there are tweaks beyond that, we could end-up in an oscillating state, and
the style invalidation feedback loop that would hang the browser.

-Justin


On Wed, Jun 25, 2014 at 6:52 PM, Robert O'Callahan rob...@ocallahan.org
wrote:

 On Thu, Jun 26, 2014 at 10:13 AM, Robert O'Callahan rob...@ocallahan.org
 wrote:

 To clarify, I think a version of option #1 would be the best way to go.
 renderedPixelWidth would return the current canvas buffer width when the
 canvas element's CSS specified value for 'width' is 'auto', and similar for
 height.


 Actually, that would break some use-cases involving object-fit, so instead
 renderedPixelWidth/Height should return the current canvas buffer width
 only when *both* 'width' and 'height' are 'auto'.

 Rob
 --
 Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
 le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
 stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
 'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
 waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Fri, Jun 27, 2014 at 2:00 AM, Justin Novosad ju...@google.com wrote:

 Hadn't thought of that. object-fit seems smells dangerous. I think we may
 need to define explicit behaviors for renderedPixelWidth/Height for the
 different object fit modes.


I don't think so. Given renderedPixelWidth/Height returns the size of the
content box, and the element's CSS width and height are not 'auto', then
renderedPixelWidth/Height are not affected by object-fit or the intrinsic
size, so there is no feedback loop.

For example, with 'object-fit: contains', will the renderedPixelWidth/Height
 be computed in a way to fill the element's content area, or will it
 preserve the aspect ratio of the original intrinsic size?


The former.


 Also, with object fit triggering a renderedsizechange event, the event
 listener will presumably change the intrinsic size of the canvas, which
 will invalidate style (because the object-fit computation depends on the
 intrinsic size), and that causes a style invalidation feedback loop.


We don't implement object-fit in Gecko yet, but when we do the change to
the intrinsic size will trigger a relayout that will end up not change
anything so there is no feedback loop.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Justin Novosad]

I was wondering if there is anything we can do with this feature to prevent
web apps from creating layout feedback loops.  What I mean is that if the
event listener for renderedsizechange changes the layout of the page in a
way that does not preserve renderedPixelWidth/Height, we can get into a lot
of trouble. Even if the resizing arithmetic in the event listener is sound,
if the rounding logic does not match exactly what the browser does,
fractional scales may cause instabilities.

The main source of of the problem is that intrinsic size affects CSS size
when width or height are set to 'auto', which is the default.  Some ideas
that come to mind for breaking the feedback loop:
1) renderedPixelWidth/Height return the intrinsic size when CSS
width/height are set to 'auto'
2) renderedPixelWidth/Height do not change when CSS width/height are set to
'auto'
3) have an explicit way to enable this feature. When it is enabled, the
canvas follows non-replaced element layout rules.




On Wed, Jun 25, 2014 at 1:09 AM, Robert O'Callahan rob...@ocallahan.org
wrote:

 On Wed, Jun 25, 2014 at 3:30 PM, Rik Cabanier caban...@gmail.com wrote:

  Add a new event renderedsizechange to HTMLCanvasElement. This event does
  not bubble and is not cancelable. Whenever the value that would be
 returned
  by renderedPixelWidth orrenderedPixelHeight changes, queue a task to fire
  renderedsizechange at the HTMLCanvasElement if there is not already a
 task
  pending to fire such an event at that element.
 
 
  - If there's a transition or animation that affects the canvas element,
  should it receive resize events at 60fps?
 

 UA dependent. In practice I would change the renderedPixelWidth/Height
 properties whenever Gecko rerasterizes the surrounding CSS content with a
 new resolution.


  - will CSS 3D transforms affect the rendered canvas size? If so, what
 would
  the optimal resolution be if there's a rotation
 

 UA dependent. Again, I would change the renderedPixelWidth/Height
 properties whenever Gecko rerasterizes the surrounding CSS content with a
 new resolution, based on that resolution.


  - what happens if the canvas element is not in the document? Will it just
  return the pixel width/height?
 

 UA dependent. I'd probably just leave it unchanged while it's not in the
 document (and assume 1 CSS pixel per device pixel if it's never been in a
 document with a screen presentation).

 Rob
 --
 Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
 le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
 stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
 'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
 waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Thu, Jun 26, 2014 at 2:49 AM, Justin Novosad ju...@google.com wrote:

 I was wondering if there is anything we can do with this feature to prevent
 web apps from creating layout feedback loops.  What I mean is that if the
 event listener for renderedsizechange changes the layout of the page in a
 way that does not preserve renderedPixelWidth/Height, we can get into a lot
 of trouble. Even if the resizing arithmetic in the event listener is sound,
 if the rounding logic does not match exactly what the browser does,
 fractional scales may cause instabilities.

 The main source of of the problem is that intrinsic size affects CSS size
 when width or height are set to 'auto', which is the default.  Some ideas
 that come to mind for breaking the feedback loop:
 1) renderedPixelWidth/Height return the intrinsic size when CSS
 width/height are set to 'auto'
 2) renderedPixelWidth/Height do not change when CSS width/height are set to
 'auto'


We could do this and I think it would work. It's arguable whether the
complexity is justified, but I'd be OK with doing it.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Thu, Jun 26, 2014 at 10:10 AM, Robert O'Callahan rob...@ocallahan.org
wrote:

 On Thu, Jun 26, 2014 at 2:49 AM, Justin Novosad ju...@google.com wrote:

 I was wondering if there is anything we can do with this feature to
 prevent
 web apps from creating layout feedback loops.  What I mean is that if the
 event listener for renderedsizechange changes the layout of the page in a
 way that does not preserve renderedPixelWidth/Height, we can get into a
 lot
 of trouble. Even if the resizing arithmetic in the event listener is
 sound,
 if the rounding logic does not match exactly what the browser does,
 fractional scales may cause instabilities.

 The main source of of the problem is that intrinsic size affects CSS size
 when width or height are set to 'auto', which is the default.  Some ideas
 that come to mind for breaking the feedback loop:
 1) renderedPixelWidth/Height return the intrinsic size when CSS
 width/height are set to 'auto'
 2) renderedPixelWidth/Height do not change when CSS width/height are set
 to
 'auto'


 We could do this and I think it would work. It's arguable whether the
 complexity is justified, but I'd be OK with doing it.


To clarify, I think a version of option #1 would be the best way to go.
renderedPixelWidth would return the current canvas buffer width when the
canvas element's CSS specified value for 'width' is 'auto', and similar for
height.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Thu, Jun 26, 2014 at 10:13 AM, Robert O'Callahan rob...@ocallahan.org
wrote:

 To clarify, I think a version of option #1 would be the best way to go.
 renderedPixelWidth would return the current canvas buffer width when the
 canvas element's CSS specified value for 'width' is 'auto', and similar for
 height.


Actually, that would break some use-cases involving object-fit, so instead
renderedPixelWidth/Height should return the current canvas buffer width
only when *both* 'width' and 'height' are 'auto'.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Kenneth Russell]

On Mon, Jun 23, 2014 at 6:06 PM, Robert O'Callahan rob...@ocallahan.org wrote:
 On Tue, Jun 24, 2014 at 12:27 PM, Robert O'Callahan rob...@ocallahan.org
 wrote:

 I'll do that now.


 Done.
 http://wiki.whatwg.org/wiki/CanvasRenderedPixelSize

Fantastic. Thanks Rob. That looks great. Filed crbug.com/388532 about
implementing them in Chrome.

-Ken

Re: [whatwg] High-density canvases

[Rik Cabanier]

On Mon, Jun 23, 2014 at 6:06 PM, Robert O'Callahan rob...@ocallahan.org
wrote:

 On Tue, Jun 24, 2014 at 12:27 PM, Robert O'Callahan rob...@ocallahan.org
 wrote:

 I'll do that now.


 Done.
 http://wiki.whatwg.org/wiki/CanvasRenderedPixelSize


The wiki states:

Add a new event renderedsizechange to HTMLCanvasElement. This event does
not bubble and is not cancelable. Whenever the value that would be returned
by renderedPixelWidth orrenderedPixelHeight changes, queue a task to fire
renderedsizechange at the HTMLCanvasElement if there is not already a task
pending to fire such an event at that element.


- If there's a transition or animation that affects the canvas element,
should it receive resize events at 60fps?
- will CSS 3D transforms affect the rendered canvas size? If so, what would
the optimal resolution be if there's a rotation
- what happens if the canvas element is not in the document? Will it just
return the pixel width/height?

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Wed, Jun 25, 2014 at 3:30 PM, Rik Cabanier caban...@gmail.com wrote:

 Add a new event renderedsizechange to HTMLCanvasElement. This event does
 not bubble and is not cancelable. Whenever the value that would be returned
 by renderedPixelWidth orrenderedPixelHeight changes, queue a task to fire
 renderedsizechange at the HTMLCanvasElement if there is not already a task
 pending to fire such an event at that element.


 - If there's a transition or animation that affects the canvas element,
 should it receive resize events at 60fps?


UA dependent. In practice I would change the renderedPixelWidth/Height
properties whenever Gecko rerasterizes the surrounding CSS content with a
new resolution.


 - will CSS 3D transforms affect the rendered canvas size? If so, what would
 the optimal resolution be if there's a rotation


UA dependent. Again, I would change the renderedPixelWidth/Height
properties whenever Gecko rerasterizes the surrounding CSS content with a
new resolution, based on that resolution.


 - what happens if the canvas element is not in the document? Will it just
 return the pixel width/height?


UA dependent. I'd probably just leave it unchanged while it's not in the
document (and assume 1 CSS pixel per device pixel if it's never been in a
document with a screen presentation).

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Kenneth Russell]

On Thu, Jun 19, 2014 at 4:20 PM, Robert O'Callahan rob...@ocallahan.org wrote:
 On Fri, Jun 20, 2014 at 6:06 AM, Kenneth Russell k...@google.com wrote:

 It is a little unfortunate that a canvas-specific solution is needed.
 Is it known whether document.getBoxQuads solves this problem in
 Firefox?


 It doesn't.

 Gecko (and I assume other engines) typically snaps CSS box edges to device
 pixel edges at render time, so that CSS backgrounds and borders look crisp.
 So if a canvas has a CSS background exactly filling its content box, our
 snapping of the content box is what determines the ideal device pixel size
 of the canvas buffer.

 Authors can estimate the canvas device pixel size using getBoxQuads to get
 the canvas content box in fractional CSS pixels and multiplying by
 devicePixelRatio (assuming no CSS transforms or other effects present). But
 if that estimate results in a non-integral device pixel size, there's no way
 for authors to know how we will snap that to an integral size.

 We could move these attributes up the element hierarchy, or better still to
 a new generic API that works on all elements (e.g. some variant of
 getBoxQuads). (CSS fragmentation might split an element into multiple boxes
 with different sizes.) I don't know of any good use-cases for that, but
 maybe there are some?

It's hard to predict. A more general API would be better than a
canvas-specific one, assuming it solves the problem. getBoxQuads can
return different types of CSS boxes (content, padding, border, margin)
-- will it be obvious to web developers which to use, and is it more
likely all implementations will handle them all correctly rather than
a couple of new properties on the canvas?

Could you suggest a name for the new API? getBoxQuadsInDevicePixels?
getDevicePixelBoxQuads?

-Ken

Re: [whatwg] High-density canvases

[Kenneth Russell]

On Mon, Jun 23, 2014 at 4:25 PM, Robert O'Callahan rob...@ocallahan.org wrote:
 On Tue, Jun 24, 2014 at 8:54 AM, Kenneth Russell k...@google.com wrote:

 It's hard to predict. A more general API would be better than a
 canvas-specific one, assuming it solves the problem. getBoxQuads can
 return different types of CSS boxes (content, padding, border, margin)
 -- will it be obvious to web developers which to use, and is it more
 likely all implementations will handle them all correctly rather than
 a couple of new properties on the canvas?

 Could you suggest a name for the new API? getBoxQuadsInDevicePixels?
 getDevicePixelBoxQuads?


 AFAIK all we need from this API is device pixel sizes. Extending getBoxQuads
 with an option to return geometry in device pixels doesn't really make sense
 to me since it doesn't make sense to ask what is element A's quad relative
 to element B in device pixels. So a general, dedicated API would probably
 look something like:
   interface DOMSize {
 readonly attribute unrestricted double width;
 readonly attribute unrestricted double height;
   };
   dictionary BoxSizeOptions {
 CSSBoxType box = border;
   };
   sequenceDOMSize getBoxDevicePixelSizes(optional BoxSizeOptions options);

 So renderedPixelWidth/Height on HTMLCanvasElement is definitely simpler. I
 have no strong opinion about which way to go, but I lean towards the
 attributes unless someone has a good use-case for the general API. We could
 add the general API later if we need to, the duplication would not be a big
 deal.

If there's no concern about potential duplicated functionality then
let's add the attributes to the canvas element. They'll be easier for
developers to understand and easier to verify as obviously correct.

How should we proceed? Would you add an entry to
http://wiki.whatwg.org/wiki/Category:Proposals ?

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Tue, Jun 24, 2014 at 11:44 AM, Kenneth Russell k...@google.com wrote:

 If there's no concern about potential duplicated functionality then
 let's add the attributes to the canvas element. They'll be easier for
 developers to understand and easier to verify as obviously correct.

 How should we proceed? Would you add an entry to
 http://wiki.whatwg.org/wiki/Category:Proposals ?


I'll do that now.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Tue, Jun 24, 2014 at 12:27 PM, Robert O'Callahan rob...@ocallahan.org
wrote:

 I'll do that now.


Done.
http://wiki.whatwg.org/wiki/CanvasRenderedPixelSize

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Justin Novosad]

On Thu, Jun 19, 2014 at 7:03 PM, Robert O'Callahan rob...@ocallahan.org
wrote:

 On Fri, Jun 20, 2014 at 7:32 AM, Justin Novosad ju...@google.com wrote:

 +1 from me too. But I have one concern in terms of future proofing,
 because
 I would like to keep the door open for auto-resizing as a possible future
 improvement. In an eventual auto-resize proposal, we will want to make
 the preferredsizechange
 event cancelable.


 Why?


Because apps may want to use the auto resizing feature to delegate the
resize math to the browser (and possibly the re-rendering too, in the case
of a command-buffer backed implementation), but still retain some control
over when the auto-resizing kicks in.  A simple way to provide that control
is to allow the app to inhibit an auto-resize by cancelling the
preferredsizechange event.  It could be done differently, but I think
making the event cancelable would be the most natural and succinct way to
expose that control.

Examples of how this might be used:
* If the CSS size of the canvas is being animated, the app may want to only
allow the backing store to be auto-resized at the end of the animation, so
that the animation stays fluid.
* The app may want to allow auto-resize to kick-in only when the preferred
vs actual size ratio exceeds a certain threshold.

-Justin


 Rob
 --
 Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
 le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
 stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
 'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
 waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Mark Callow]

On 19/06/2014 00:30, Justin Novosad wrote:
 My main point is, there is potentially significant progress in terms
 of HD canvas rendering that can be achieved without any changes to the
 spec (other than perhaps an opt-in flag). If it works out well without
 an explicit opt-in, legacy websites will benefit.
Neat as it sounds, it will do nothing to help WebGL canvases. In WebGL
applications have full control of the GL viewport which means they need
a way to determine the actual h/w (screen) pixel size of a canvas.
Current methods have many issues some of which are documented in the bug
I referenced in my previous message.

Regards

-Mark

-- 
注意：この電子メールには、株式会社エイチアイの機密情報が含まれている場合
が有ります。正式なメール受信者では無い場合はメール複製、 再配信または情
報の使用を固く禁じております。エラー、手違いでこのメールを受け取られまし
たら削除を行い配信者にご連絡をお願いいたし ます.

NOTE: This electronic mail message may contain confidential and
privileged information from HI Corporation. If you are not the intended
recipient, any disclosure, photocopying, distribution or use of the
contents of the received information is prohibited. If you have received
this e-mail in error, please notify the sender immediately and
permanently delete this message and all related copies.

Re: [whatwg] High-density canvases

[Justin Novosad]

On Thu, Jun 19, 2014 at 12:01 AM, Robert O'Callahan rob...@ocallahan.org
wrote:

 On Thu, Jun 19, 2014 at 11:52 AM, Robert O'Callahan rob...@ocallahan.org
 wrote:

 On Thu, Jun 19, 2014 at 3:30 AM, Justin Novosad ju...@google.com wrote:

 I am currently trying an experimental approach where canvases that are
 drawn to, but never read from (no toDataURL or getImageData calls) have
 their contents stored as a command buffer, rather than a pixel buffer.


 There must be a cliff with that approach, where if you issue a sufficient
 number of drawing commands without clearing the canvas you have to
 rasterize, otherwise memory usage will grow without bound. That seems like
 a problem for authors. I think it's reasonably easy for authors to
 understand and control canvas memory usage at the moment, and I'd like to
 not make it worse.


Yes there needs to be a threshold where the command buffer gets flushed and
discarded. We ran into that problem already with the deferred rasterization
mechanism already shipping in Chrome. Right now, it is not a big deal
because canvases are not auto-resized. With auto-resizing enabled, flushing
the command buffer will mean locking in a resolution.


 Also, you would have the problem of how correctly size temporary offscreen
 canvases and variable-resolution image assets that Ian's approach had. And
 without adding new API, there remains the problem of getImageData needing
 to return 1 pixel per canvas coordinate unit for compatibility, thus
 requiring some kind of getImageDataHD for authors who want more ... thus
 requiring new API.


Or the command buffer could be played back into a temporary buffer with 1:1
canvas pixel ratio.  But that is not ideal either, because there would be
loss of resolution through a getImageData/putImageData round trip.


 So while I applaud experimentation with retaining canvas command buffers,
 I don't want it to block consideration of alternatives for explicit canvas
 buffer sizing.


I agree, the more I think about and discuss this with others, the clearer
it seems that auto-resizing needs to be explicitly controlled and opt-in
behavior because the are too many problematic edge cases that would regress
existing usage.  That being said, the concept of retaining a command
buffer, but without the auto-resizing behavior, is still beneficial and can
be implemented without changes to the spec. Once we add an new HD API, the
command buffer implementation will not be a requirement, but may be quite
useful for providing an efficient resizing implementation.  With that in
mind, I would like us to make sure that any new HD canvas API could
accommodate such an implementation.

Looking at your proposal, the preferredsizechange event is the tricky
part. That event would be used as a signal to resize and  redraw canvas
contents for web pages that choose to resize their canvases dynamically.
However, if the browser is in a state where it is able to redraw the canvas
contents from a command buffer, then we may not want to redraw contents in
JavaScript and delegate the resize work to the browser. In that case, we
still need to let the web application know that the preferred size has
changed, just in case the application needs that information to do
something special. For example, when resizing beyond a certain threshold,
the app may decide to redraw in JS anyways in order to switch to assets
(like sprites) of a different resolution.

-Justin

Re: [whatwg] High-density canvases

[Stephen White]

On Thu, Jun 12, 2014 at 11:42 PM, Robert O'Callahan rob...@ocallahan.org
wrote:

 I think I'd rather not take control of canvas resizing away from
 applications, even opt-in. That leads to complexity such as extra API for
 slaving other canvases. I also think we'd be better off sticking to the
 invariant that the units of canvas coordinate space are single pixels in
 the canvas bitmap; I think that simplifies things for implementers and
 authors.


I agree.


 Here's an alternative proposal which I think is a bit simpler and more
 flexible:
 Expose two new DOM attributes on HTMLCanvasElement:
 readonly attribute long preferredWidth;
 readonly attribute long preferredHeight;
 These attributes are the UA's suggested canvas size for optimizing output
 quality. It's basically what Ian's proposal would have set as the automatic
 size. We would also add a preferredsizechange event when those attributes
 change.

 Applications that want DPI-aware canvases would read those attributes and
 use them to set the size, just once if they only want to paint once, during
 each requestAnimationFrame callback for games and other animations, and in
 the preferredsizechange event handler if they are willing to paint multiple
 times but aren't actually animating. The application would be responsible
 for scaling its output to the new canvas coordinate space size. Giving the
 application control of the size change simplifies things for the browser
 and gives applications maximum flexibility, e.g. resizing ancillary
 resources as needed, or imposing constraints on the chosen size.


I like this new proposal. It works for both the canvas and WebGL use cases,
and it puts the app in control of behaviour in a way that makes sense for
immediate-mode APIs.

I assume that the size change event would fire:

   - on browser page zoom
   - on pinch-zoom
   - when a CSS animation (e.g., scale) changes the canvas size in CSS
   pixels

For browsers that implement the latter two off the main thread, perhaps
they should only fire at end-of-gesture or end-of-animation, to avoid the
rendered size being out-of-sync with scaled size by the time the canvas
gets composited.

I agree with Mark that the names need work. How about something that
incorporates device pixel in some way, to reflect that this is roughly
dpr * css scale * size?

devicePixelWidth
widthInDevicePixels
pixelExactWidth
exactPixelWidth
pixelWidth
pixelRatioExactWidth
unscaledWidth
unscaledPixelWidth
nativeWidth
nativePixelWidth

Stephen



 Rob
 --
 Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
 le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
 stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
 'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
 waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Kenneth Russell]

On Thu, Jun 19, 2014 at 7:54 AM, Stephen White senorbla...@chromium.org wrote:
 On Thu, Jun 12, 2014 at 11:42 PM, Robert O'Callahan rob...@ocallahan.org
 wrote:

 I think I'd rather not take control of canvas resizing away from
 applications, even opt-in. That leads to complexity such as extra API for
 slaving other canvases. I also think we'd be better off sticking to the
 invariant that the units of canvas coordinate space are single pixels in
 the canvas bitmap; I think that simplifies things for implementers and
 authors.


 I agree.


 Here's an alternative proposal which I think is a bit simpler and more
 flexible:
 Expose two new DOM attributes on HTMLCanvasElement:
 readonly attribute long preferredWidth;
 readonly attribute long preferredHeight;
 These attributes are the UA's suggested canvas size for optimizing output
 quality. It's basically what Ian's proposal would have set as the automatic
 size. We would also add a preferredsizechange event when those attributes
 change.

 Applications that want DPI-aware canvases would read those attributes and
 use them to set the size, just once if they only want to paint once, during
 each requestAnimationFrame callback for games and other animations, and in
 the preferredsizechange event handler if they are willing to paint multiple
 times but aren't actually animating. The application would be responsible
 for scaling its output to the new canvas coordinate space size. Giving the
 application control of the size change simplifies things for the browser
 and gives applications maximum flexibility, e.g. resizing ancillary
 resources as needed, or imposing constraints on the chosen size.


 I like this new proposal. It works for both the canvas and WebGL use cases,
 and it puts the app in control of behaviour in a way that makes sense for
 immediate-mode APIs.

 I assume that the size change event would fire:

- on browser page zoom
- on pinch-zoom
- when a CSS animation (e.g., scale) changes the canvas size in CSS
pixels

 For browsers that implement the latter two off the main thread, perhaps
 they should only fire at end-of-gesture or end-of-animation, to avoid the
 rendered size being out-of-sync with scaled size by the time the canvas
 gets composited.

 I agree with Mark that the names need work. How about something that
 incorporates device pixel in some way, to reflect that this is roughly
 dpr * css scale * size?

 devicePixelWidth
 widthInDevicePixels
 pixelExactWidth
 exactPixelWidth

widthInDevicePixels and exactPixelWidth both sound clear.
renderedPixelWidth per
https://bugzilla.mozilla.org/show_bug.cgi?id=1024493 also sounds like
a good option. It would be great to spec these and their associated
change event. I'd be interested in adding support to Chrome.

It is a little unfortunate that a canvas-specific solution is needed.
Is it known whether document.getBoxQuads solves this problem in
Firefox?

-Ken


 pixelWidth
 pixelRatioExactWidth
 unscaledWidth
 unscaledPixelWidth
 nativeWidth
 nativePixelWidth

 Stephen



 Rob
 --
 Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
 le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
 stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
 'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
 waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Dean Jackson]

 On 20 Jun 2014, at 12:54 am, Stephen White senorbla...@chromium.org wrote:
 
 On Thu, Jun 12, 2014 at 11:42 PM, Robert O'Callahan rob...@ocallahan.org 
 wrote:
 I think I'd rather not take control of canvas resizing away from 
 applications, even opt-in. That leads to complexity such as extra API for 
 slaving other canvases. I also think we'd be better off sticking to the 
 invariant that the units of canvas coordinate space are single pixels in the 
 canvas bitmap; I think that simplifies things for implementers and authors.
 
 I agree.

+1 from me too and for the rest of Robert's proposal.

Dean

Re: [whatwg] High-density canvases

[Justin Novosad]

+1 from me too. But I have one concern in terms of future proofing, because
I would like to keep the door open for auto-resizing as a possible future
improvement. In an eventual auto-resize proposal, we will want to make
the preferredsizechange
event cancelable.  If we make the event non-cancelable initially, does that
lock us in forever in terms of spec backward compatibility? If the answer
is no, then +1 to Robert's proposal as is.

In terms of accommodating the possibility of auto-resizing with a command
buffer backed implementation, we should be able to add the necessary
signals for short-circuiting a full JS redraw by adding event attributes
and/or new event types on top of the current proposal.


On Thu, Jun 19, 2014 at 3:05 PM, Dean Jackson d...@apple.com wrote:


  On 20 Jun 2014, at 12:54 am, Stephen White senorbla...@chromium.org
 wrote:
 
  On Thu, Jun 12, 2014 at 11:42 PM, Robert O'Callahan 
 rob...@ocallahan.org wrote:
  I think I'd rather not take control of canvas resizing away from
 applications, even opt-in. That leads to complexity such as extra API for
 slaving other canvases. I also think we'd be better off sticking to the
 invariant that the units of canvas coordinate space are single pixels in
 the canvas bitmap; I think that simplifies things for implementers and
 authors.
 
  I agree.

 +1 from me too and for the rest of Robert's proposal.

 Dean

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Fri, Jun 20, 2014 at 7:32 AM, Justin Novosad ju...@google.com wrote:

 +1 from me too. But I have one concern in terms of future proofing, because
 I would like to keep the door open for auto-resizing as a possible future
 improvement. In an eventual auto-resize proposal, we will want to make
 the preferredsizechange
 event cancelable.


Why?

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Fri, Jun 20, 2014 at 6:06 AM, Kenneth Russell k...@google.com wrote:

 It is a little unfortunate that a canvas-specific solution is needed.
 Is it known whether document.getBoxQuads solves this problem in
 Firefox?


It doesn't.

Gecko (and I assume other engines) typically snaps CSS box edges to device
pixel edges at render time, so that CSS backgrounds and borders look crisp.
So if a canvas has a CSS background exactly filling its content box, our
snapping of the content box is what determines the ideal device pixel size
of the canvas buffer.

Authors can estimate the canvas device pixel size using getBoxQuads to get
the canvas content box in fractional CSS pixels and multiplying by
devicePixelRatio (assuming no CSS transforms or other effects present). But
if that estimate results in a non-integral device pixel size, there's no
way for authors to know how we will snap that to an integral size.

We could move these attributes up the element hierarchy, or better still to
a new generic API that works on all elements (e.g. some variant of
getBoxQuads). (CSS fragmentation might split an element into multiple boxes
with different sizes.) I don't know of any good use-cases for that, but
maybe there are some?

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Justin Novosad]

In the previous incarnation of High density canvases (i.e. getImageDataHD
and friends), we worked under the assumption that it was okay to have a
backing store with a pixel density that is higher than CSS pixel density.
And I think that was perfectly acceptable.

If I recall correctly, the feature failed because some websites were
already using CSS hacks to boost their pixel density, which led to 16x
(rather than 4x) memory consumption for canvas pixel buffers on devices
with a 2:1 device to CSS pixel ratio.  I think that failure could have been
avoided by making the feature smarter by dynamically adjusting the HD ratio
applied to the canvas to prevent canvas pixel density from being boosted
beyond the display's pixel density.

I am currently trying an experimental approach where canvases that are
drawn to, but never read from (no toDataURL or getImageData calls) have
their contents stored as a command buffer, rather than a pixel buffer. This
way, the contents can be painted at any resolution (à la SVG).  This
approach also allows canvases to be rasterized asynchronously, it allows
contents to change pixel density without redrawing in JS (in reaction to a
page zoom, for example), and it can support arbitrarily large canvas sizes.
In theory, it would be possible to inject this behavior without any changes
to the spec, but some side effects may be hard to resolve and/or live with.
 Having an experimental implementation will help us discover and iron-out
the issues. In the end, it may have to ship as opt-in behavior, but that
remains to be determined.

My main point is, there is potentially significant progress in terms of HD
canvas rendering that can be achieved without any changes to the spec
(other than perhaps an opt-in flag). If it works out well without an
explicit opt-in, legacy websites will benefit.

-Justin


On Tue, Jun 17, 2014 at 11:06 PM, Mark Callow callow.m...@artspark.co.jp
wrote:

 On 13/06/2014 12:42, Robert O'Callahan wrote:
  Here's an alternative proposal which I think is a bit simpler and more
  flexible:
  Expose two new DOM attributes on HTMLCanvasElement:
  readonly attribute long preferredWidth;
  readonly attribute long preferredHeight;
  These attributes are the UA's suggested canvas size for optimizing output
  quality. It's basically what Ian's proposal would have set as the
 automatic
  size. We would also add a preferredsizechange event when those
 attributes
  change.
 I like the functionality but these names really don't convey that
 functionality. The names you originally proposed over in *Bug 1024493*
 https://bugzilla.mozilla.org/show_bug.cgi?id=1024493 at mozilla.org,
 renderedPixelWidth/Height, while not perfect, convey it much better.

 Regards

 -Mark

 --
 注意：この電子メールには、株式会社エイチアイの機密情報が含まれている場合
 が有ります。正式なメール受信者では無い場合はメール複製、 再配信または情
 報の使用を固く禁じております。エラー、手違いでこのメールを受け取られまし
 たら削除を行い配信者にご連絡をお願いいたし ます.

 NOTE: This electronic mail message may contain confidential and
 privileged information from HI Corporation. If you are not the intended
 recipient, any disclosure, photocopying, distribution or use of the
 contents of the received information is prohibited. If you have received
 this e-mail in error, please notify the sender immediately and
 permanently delete this message and all related copies.

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Thu, Jun 19, 2014 at 3:30 AM, Justin Novosad ju...@google.com wrote:

 I am currently trying an experimental approach where canvases that are
 drawn to, but never read from (no toDataURL or getImageData calls) have
 their contents stored as a command buffer, rather than a pixel buffer.


There must be a cliff with that approach, where if you issue a sufficient
number of drawing commands without clearing the canvas you have to
rasterize, otherwise memory usage will grow without bound. That seems like
a problem for authors. I think it's reasonably easy for authors to
understand and control canvas memory usage at the moment, and I'd like to
not make it worse.

I can see that the command buffer approach would be useful for many
applications though.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Rik Cabanier]

On Wed, Jun 18, 2014 at 8:30 AM, Justin Novosad ju...@google.com wrote:

 In the previous incarnation of High density canvases (i.e. getImageDataHD
 and friends), we worked under the assumption that it was okay to have a
 backing store with a pixel density that is higher than CSS pixel density.
 And I think that was perfectly acceptable.

 If I recall correctly, the feature failed because some websites were
 already using CSS hacks to boost their pixel density, which led to 16x
 (rather than 4x) memory consumption for canvas pixel buffers on devices
 with a 2:1 device to CSS pixel ratio.  I think that failure could have been
 avoided by making the feature smarter by dynamically adjusting the HD ratio
 applied to the canvas to prevent canvas pixel density from being boosted
 beyond the display's pixel density.

 I am currently trying an experimental approach where canvases that are
 drawn to, but never read from (no toDataURL or getImageData calls) have
 their contents stored as a command buffer, rather than a pixel buffer. This
 way, the contents can be painted at any resolution (à la SVG).  This
 approach also allows canvases to be rasterized asynchronously, it allows
 contents to change pixel density without redrawing in JS (in reaction to a
 page zoom, for example), and it can support arbitrarily large canvas sizes.
 In theory, it would be possible to inject this behavior without any changes
 to the spec, but some side effects may be hard to resolve and/or live with.
  Having an experimental implementation will help us discover and iron-out
 the issues. In the end, it may have to ship as opt-in behavior, but that
 remains to be determined.

 My main point is, there is potentially significant progress in terms of HD
 canvas rendering that can be achieved without any changes to the spec
 (other than perhaps an opt-in flag). If it works out well without an
 explicit opt-in, legacy websites will benefit.


This should be an explicit opt-in, otherwise applications that call
'getImageData', will suddenly get a different rendition.
Also, they might *want* to see the pixels in which case upscaling is
undesired.

To make this work, you might have to store potentially an infinite amount
of commands which is problematic.


 On Tue, Jun 17, 2014 at 11:06 PM, Mark Callow callow.m...@artspark.co.jp
 wrote:

 On 13/06/2014 12:42, Robert O'Callahan wrote:
  Here's an alternative proposal which I think is a bit simpler and more
  flexible:
  Expose two new DOM attributes on HTMLCanvasElement:
  readonly attribute long preferredWidth;
  readonly attribute long preferredHeight;
  These attributes are the UA's suggested canvas size for optimizing
 output
  quality. It's basically what Ian's proposal would have set as the
 automatic
  size. We would also add a preferredsizechange event when those
 attributes
  change.
 I like the functionality but these names really don't convey that
 functionality. The names you originally proposed over in *Bug 1024493*
 https://bugzilla.mozilla.org/show_bug.cgi?id=1024493 at mozilla.org,

 renderedPixelWidth/Height, while not perfect, convey it much better.

 Regards

 -Mark

 --
 注意：この電子メールには、株式会社エイチアイの機密情報が含まれている場合
 が有ります。正式なメール受信者では無い場合はメール複製、 再配信または情
 報の使用を固く禁じております。エラー、手違いでこのメールを受け取られまし
 たら削除を行い配信者にご連絡をお願いいたし ます.

 NOTE: This electronic mail message may contain confidential and
 privileged information from HI Corporation. If you are not the intended
 recipient, any disclosure, photocopying, distribution or use of the
 contents of the received information is prohibited. If you have received
 this e-mail in error, please notify the sender immediately and
 permanently delete this message and all related copies.

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Thu, Jun 19, 2014 at 11:52 AM, Robert O'Callahan rob...@ocallahan.org
wrote:

 On Thu, Jun 19, 2014 at 3:30 AM, Justin Novosad ju...@google.com wrote:

 I am currently trying an experimental approach where canvases that are
 drawn to, but never read from (no toDataURL or getImageData calls) have
 their contents stored as a command buffer, rather than a pixel buffer.


 There must be a cliff with that approach, where if you issue a sufficient
 number of drawing commands without clearing the canvas you have to
 rasterize, otherwise memory usage will grow without bound. That seems like
 a problem for authors. I think it's reasonably easy for authors to
 understand and control canvas memory usage at the moment, and I'd like to
 not make it worse.


Also, you would have the problem of how correctly size temporary offscreen
canvases and variable-resolution image assets that Ian's approach had. And
without adding new API, there remains the problem of getImageData needing
to return 1 pixel per canvas coordinate unit for compatibility, thus
requiring some kind of getImageDataHD for authors who want more ... thus
requiring new API.

So while I applaud experimentation with retaining canvas command buffers, I
don't want it to block consideration of alternatives for explicit canvas
buffer sizing.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Mark Callow]

On 13/06/2014 12:42, Robert O'Callahan wrote:
 Here's an alternative proposal which I think is a bit simpler and more
 flexible:
 Expose two new DOM attributes on HTMLCanvasElement:
 readonly attribute long preferredWidth;
 readonly attribute long preferredHeight;
 These attributes are the UA's suggested canvas size for optimizing output
 quality. It's basically what Ian's proposal would have set as the automatic
 size. We would also add a preferredsizechange event when those attributes
 change.
I like the functionality but these names really don't convey that
functionality. The names you originally proposed over in *Bug 1024493*
https://bugzilla.mozilla.org/show_bug.cgi?id=1024493 at mozilla.org,
renderedPixelWidth/Height, while not perfect, convey it much better.

Regards

-Mark

-- 
注意：この電子メールには、株式会社エイチアイの機密情報が含まれている場合
が有ります。正式なメール受信者では無い場合はメール複製、 再配信または情
報の使用を固く禁じております。エラー、手違いでこのメールを受け取られまし
たら削除を行い配信者にご連絡をお願いいたし ます.

NOTE: This electronic mail message may contain confidential and
privileged information from HI Corporation. If you are not the intended
recipient, any disclosure, photocopying, distribution or use of the
contents of the received information is prohibited. If you have received
this e-mail in error, please notify the sender immediately and
permanently delete this message and all related copies.

Re: [whatwg] High-density canvases

[Robert O'Callahan]

[Resurrecting old thread]

On Tue, Sep 10, 2013 at 12:00 PM, Ian Hickson i...@hixie.ch wrote:

 It would be nice to fix these all at once, and I think we can, by
 introducing a configuration option on getContext(), in the style of WebGL:

getContext('2d', { density: 'autosize' });

 This would trigger the following behaviour: When the context is created,
 and subsequently when the canvas changes size (e.g. due to being sized
 with CSS relative units and the element they're relative to changing), or
 when the display density changes size (e.g. due to page zoom), then:

- the width and height of the canvas bitmaps get updated to match the
  new native size of the canvas, at native density.

- the coordinate space of the canvas (context.width/context.height)
  gets updated to match the size of the canvas in CSS pixel units.

- a 'resize' event gets fired at the canvas.

 We would dump the *HD versions of the methods, and make the regular ones
 go back to returning the actual raw pixels, since that would now work fine
 and still provide HD-quality content everywhere it's available.

 What do people think?


I think I'd rather not take control of canvas resizing away from
applications, even opt-in. That leads to complexity such as extra API for
slaving other canvases. I also think we'd be better off sticking to the
invariant that the units of canvas coordinate space are single pixels in
the canvas bitmap; I think that simplifies things for implementers and
authors.

Here's an alternative proposal which I think is a bit simpler and more
flexible:
Expose two new DOM attributes on HTMLCanvasElement:
readonly attribute long preferredWidth;
readonly attribute long preferredHeight;
These attributes are the UA's suggested canvas size for optimizing output
quality. It's basically what Ian's proposal would have set as the automatic
size. We would also add a preferredsizechange event when those attributes
change.

Applications that want DPI-aware canvases would read those attributes and
use them to set the size, just once if they only want to paint once, during
each requestAnimationFrame callback for games and other animations, and in
the preferredsizechange event handler if they are willing to paint multiple
times but aren't actually animating. The application would be responsible
for scaling its output to the new canvas coordinate space size. Giving the
application control of the size change simplifies things for the browser
and gives applications maximum flexibility, e.g. resizing ancillary
resources as needed, or imposing constraints on the chosen size.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

[Ian Hickson]

On Tue, 10 Sep 2013, Ian Hickson wrote:
 On Tue, 10 Sep 2013, Ian Hickson wrote:
  
  It would be nice to fix these all at once, and I think we can, by 
  introducing a configuration option on getContext(), in the style of 
  WebGL:
  
 getContext('2d', { density: 'autosize' });
  
  This would trigger the following behaviour: When the context is 
  created, and subsequently when the canvas changes size (e.g. due to 
  being sized with CSS relative units and the element they're relative 
  to changing), or when the display density changes size (e.g. due to 
  page zoom), then:
  
 - the width and height of the canvas bitmaps get updated to match the
   new native size of the canvas, at native density.
  
 - the coordinate space of the canvas (context.width/context.height) 
   gets updated to match the size of the canvas in CSS pixel units.
  
 - a 'resize' event gets fired at the canvas.
  
  We would dump the *HD versions of the methods, and make the regular 
  ones go back to returning the actual raw pixels, since that would now 
  work fine and still provide HD-quality content everywhere it's 
  available.
  
  What do people think?
 
 Rik pointed out on IRC that it's common to have out-of-DOM canvases, 
 e.g. for storing textures, which would need to track the density of the 
 one in the DOM. I'd have to think about it in more detail, but in 
 principle it seems some sort of slaving mechanism should be able to 
 handle that too.

Proposed slaving mechanism:

   var context1 = pageCanvas.getContext('2d', { density: 'autosize' });
   var context2 = hiddenCanvas.getContext('2d', { density: pageCanvas });

Assuming we go with http://wiki.whatwg.org/wiki/WorkerCanvas :

If the 'density' value isn't the string 'autosize', it has to be a canvas 
element or, if this is a worker, a WorkerCanvas object. Either way, that 
object has to have had a context already created for it, using density set 
to 'autosize'. Note that this means there's no way to have this magic 
behaviour make a worker's canvases change size to match a canvas being 
drawn on on the main thread. When using a WorkerCanvas, the browser takes 
care of forwarding density changes from the main thread to the worker (I'm 
not sure I understand the proposal properly, but I assume it autocommits 
when the worker goes idle, even if the main thread is busy).

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] High-density canvases

[Stephen White]

On Fri, Sep 27, 2013 at 5:51 PM, Ian Hickson i...@hixie.ch wrote:

 On Tue, 10 Sep 2013, Stephen White wrote:
 
  For posterity, here were our objections to the original high-DPI canvas
  spec:
 - It doesn't scale well to non-integer devicePixelRatios

 Can you elaborate on this? I don't see why the new proposal would have
 this problem, but I also don't see why the old one would, so I don't know
 if it's because I don't understand the problem or if it's because I'm
 missing something from the old proposal.


IIRC, the concern was that calling getImageData() would return surprising
results when resampling by non-integer ratios, where the returned result
very much depends on the resampling mode, and the border treatment. At any
rate, since getImageData() doesn't resample in the new proposal, this won't
be a problem.

Stephen

 One question: now that some browsers are including browser zoom (page
  zoom) in window.devicePixelRatio, will/should the new proposal
  automatically cause a resize callback on page zoom, in order to preserve
  1:1 device pixels?

 My intent was to do so, yes. In practice I presume it'd be up to the
 browser to decide how often to actually do this (e.g. if page zoom is
 being smoothly animated, you may wish to only do it every few frames).


  (Note that I think this is a problem with current JS-based
  implementations of canvas auto-scale as well, although perhaps there's a
  DOM event for this that you can listen to; I might just be showing my
  ignorance here.)

 Currently page zoom should trigger a 'resize' event, but I expect few
 pages check.


 On Tue, 10 Sep 2013, Justin Novosad wrote:
 
  There is another closely related issue that's been discussed before:
  adding a redraw callback to 2d canvas.  In the past we discussed this
  for solving the problem of recoverring from a gpu context loss, but it
  seems there may be better reasons to consider adding a redraw callback
  such as freeing memory consumed by canvas backing stores that are in
  background tabs, and re-building the content when needed. This
  discussion was revived in the past few days on the chromium graphics-dev
  mailing list:
 
 https://groups.google.com/a/chromium.org/forum/?fromgroups#!topic/graphics-dev/CQJXpXxO6dk
 
  The idea is still embryonic and we're brainstorming in this chromium
  issue: crbug.com/287823
 
  I think that discussion should be merged with this thread because a
  resize event is another case where one may want to redraw. It would be
  great to solve all of these issues together.

 I think it would make eminent sense to also fire the event ('resize', I
 guess) if the context was lost or if the canvas was about to be made
 visible again after the browser dropped the rendering, yes.

 --
 Ian Hickson   U+1047E)\._.,--,'``.fL
 http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
 Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] High-density canvases

[Ian Hickson]

On Mon, 9 Sep 2013, Rik Cabanier wrote:
 On Mon, Sep 9, 2013 at 5:00 PM, Ian Hickson i...@hixie.ch wrote:
 
  So my understanding is that the reason this feature failed is that 
  there's existing content that assumes a 1:1 ratio, and having an 
  automatic high-density mode was making some pages end up with canvases 
  with four canvas pixels per CSS pixel (linearly) -- two from the 
  browser making a native canvas, times two from the page scaling the 
  canvas for high DPI displays. This is a factor of sixteen over a 1:1 
  canvas, a factor of four more than it should be for high DPI, and a 
  big waste of resources.
 
  As much as sites do this manually, though, it's a huge pain in the 
  neck to have to worry about pixel density when you're creating your 
  canvas and drawing on it, especially if you're not drawing sprites on 
  it.
 
  While we're talking about annoying things, there's also the annoyance 
  that canvases tend to not take zoom into account (either 
  density-affecting zoom like page zoom on desktop, or transparent 
  zoom like pinch-zoom on mobile for non-mobile-optimised sites, which 
  the site isn't supposed to know about): you have to remember to listen 
  for onresize, and then manually blow away your canvas and recreate it 
  at the right density and then squeeze it into place so that the 
  coordinate space matches what your code is expecting while the 
  canvas is actually sized for the display.
 
  There's also the issue of full-bleed canvases where every time the 
  container changes, you have to remember to re-update the canvas 
  coordinate space and repaint because otherwise your pretty page gets 
  all warped.
 
  It would be nice to fix these all at once, and I think we can, by 
  introducing a configuration option on getContext(), in the style of 
  WebGL:
 
 getContext('2d', { density: 'autosize' });
 
  This would trigger the following behaviour: When the context is 
  created, and subsequently when the canvas changes size (e.g. due to 
  being sized with CSS relative units and the element they're relative 
  to changing), or when the display density changes size (e.g. due to 
  page zoom), then:
 
 - the width and height of the canvas bitmaps get updated to match the
   new native size of the canvas, at native density.
 
 - the coordinate space of the canvas (context.width/context.height)
   gets updated to match the size of the canvas in CSS pixel units.
 
 - a 'resize' event gets fired at the canvas.
 
 Will this workflow introduce flickering?

So long as the resize event (and thus the redrawing) is synchronous with 
the clearing, I don't see why it would.


 I'm unsure if these calls are really needed. An author can already do 
 this today with a couple of lines of JS.

It's a heck of a lot more than a couple of lines to do it right today.


 By doing the actual resizing in the event handler, I believe flickering 
 is avoided.

Not sure what you mean here.


 I share Dean's worry that it will lead to bad redrawing if people don't 
 test all scenarios that causes resizing.

Can you be more specific? What scenarios do you think would result in bad 
redrawing?


 (Temporary canvases would also have to be resized manually)

We'd have to provide a way to link canvases so that they maintain the same 
density. It's on my list of things to make sure I handle if I spec this.


On Tue, 10 Sep 2013, Stephen White wrote:

 For posterity, here were our objections to the original high-DPI canvas 
 spec:
 
- The API feels like a short-term hack to automagically do something
that the developer may or may not want done (e.g., if the game/app was
tuned for particular resolution, or for pixel-exact rendering) that we'll
be stuck with in the web platform long after its short-term usefulness has
expired

This is avoided in the new proposal by making it opt-in.


- It doesn't scale well to non-integer devicePixelRatios

Can you elaborate on this? I don't see why the new proposal would have 
this problem, but I also don't see why the old one would, so I don't know 
if it's because I don't understand the problem or if it's because I'm 
missing something from the old proposal.


- It is easy for developers to implement the above behaviour in JS if
desired

I wouldn't say easy.


 I think the new proposal addresses the first point, since it's opt-in. I 
 don't think the second point is a problem, since [get|put]ImageData() 
 will be back to manipulating exact backing store pixels, so no 
 non-integer resizing will be required. The third point becomes moot.

Agreed.


 One question: now that some browsers are including browser zoom (page 
 zoom) in window.devicePixelRatio, will/should the new proposal 
 automatically cause a resize callback on page zoom, in order to preserve 
 1:1 device pixels?

My intent was to do so, yes. In practice I presume it'd be up to the 
browser to decide how often to actually do this (e.g. if page zoom is 

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Thu, Sep 12, 2013 at 10:42 AM, Ian Hickson i...@hixie.ch wrote:

 On Thu, 12 Sep 2013, Robert O'Callahan wrote:
  On Wed, Sep 11, 2013 at 11:24 AM, Ian Hickson i...@hixie.ch wrote:
   On Wed, 11 Sep 2013, Robert O'Callahan wrote:
Pinch-zoom is hard because we don't want to trigger reflows or other
expensive behavior on pinch-zoom. I'd leave pinch-zoom out of it for
now.
  
   Unless I'm missing something fundamental, changing the pixel density
   doesn't cause a layout, it's changing the width that causes a layout.
 
  Changing pixel density does cause a layout in Gecko. For example we
  round CSS border widths to a whole number of device pixels. (This
  ensures that when page zoom is applied, all borders with the same
  specified width are rendered with the same visual width.) On some
  platforms (Windows), changing pixel density affects text hinting which
  affects text layout.

 So how do you do pinch-zoom under the hood on mobile devices?


We don't change layout, we just rerender with transformed coordinates. This
means that pinch-zoom can break apparent border-widths and you can get ugly
hinting on platforms that use pinch-zoom and have fonts with aggressive
hinting.

We could continue to not change layout, and still change the pixel density
of canvases during or after pinch-zoom. We could even change the pixel
density of canvases inside scaling CSS transforms.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Wed, Sep 11, 2013 at 11:24 AM, Ian Hickson i...@hixie.ch wrote:

 On Wed, 11 Sep 2013, Robert O'Callahan wrote:
  Pinch-zoom is hard because we don't want to trigger reflows or other
  expensive behavior on pinch-zoom. I'd leave pinch-zoom out of it for
  now.

 Unless I'm missing something fundamental, changing the pixel density
 doesn't cause a layout, it's changing the width that causes a layout.


Changing pixel density does cause a layout in Gecko. For example we round
CSS border widths to a whole number of device pixels. (This ensures that
when page zoom is applied, all borders with the same specified width are
rendered with the same visual width.) On some platforms (Windows), changing
pixel density affects text hinting which affects text layout.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] High-density canvases

[Dean Jackson]

On 11 Sep 2013, at 3:20 am, Robert O'Callahan rob...@ocallahan.org wrote:

 On Tue, Sep 10, 2013 at 1:26 PM, Rik Cabanier caban...@gmail.com wrote:
 There's a thread on www-style:
 http://lists.w3.org/Archives/Public/www-style/2012Nov/0144.html
 It's been in firefox for a while and blink is going to ship it soon:
 https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/RyUSi3zdumQ
 
 devicePixelRatio has reflected page zoom in Firefox ever since we first 
 exposed it (because the thing it maps to internally has always been affected 
 by page zoom).

OK. I can accept that it makes sense for page zoom (I was only thinking about 
pinch zoom... which is a bad name). Still, we've (Apple) been telling 
developers for a long time that devicePixelRatio is (basically) constant. I 
guess we'll have to either change or be non-conformant.

  
   Actually what I really think we should do is also change the
   window.devicePixelRatio for pinch zoom. Combined with the suggestions for
   canvas, that would allow (as Rik pointed out on IRC) for high-quality
   canvas all the way zoomed in, and for cheap canvases when zoomed out
 
 I believe pinch zoom should not change devicePixelRatio. It would be nice
 if there was a standard way to determine pinch zoom though and if the
 canvas could automatically resize.
 
 Pinch-zoom is hard because we don't want to trigger reflows or other 
 expensive behavior on pinch-zoom. I'd leave pinch-zoom out of it for now.

Agreed.

Dean

Re: [whatwg] High-density canvases

[Ian Hickson]

On Thu, 12 Sep 2013, Robert O'Callahan wrote:
 On Wed, Sep 11, 2013 at 11:24 AM, Ian Hickson i...@hixie.ch wrote:
  On Wed, 11 Sep 2013, Robert O'Callahan wrote:
   Pinch-zoom is hard because we don't want to trigger reflows or other 
   expensive behavior on pinch-zoom. I'd leave pinch-zoom out of it for 
   now.
 
  Unless I'm missing something fundamental, changing the pixel density 
  doesn't cause a layout, it's changing the width that causes a layout.
 
 Changing pixel density does cause a layout in Gecko. For example we 
 round CSS border widths to a whole number of device pixels. (This 
 ensures that when page zoom is applied, all borders with the same 
 specified width are rendered with the same visual width.) On some 
 platforms (Windows), changing pixel density affects text hinting which 
 affects text layout.

So how do you do pinch-zoom under the hood on mobile devices?

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Tue, Sep 10, 2013 at 10:46 PM, Tab Atkins Jr. jackalm...@gmail.comwrote:

 High-end laptops have high-dpi screens (the Pixel I'm using right now
 has one), and they're slowly spreading down the price scale.


On Mac there are Retinas of course, and on Windows Firefox defaults to a
non-1.0 devicePixelRatio on many systems now (anything where the default
Windows font DPI is not 96).

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] High-density canvases

[Robert O'Callahan]

On Tue, Sep 10, 2013 at 1:26 PM, Rik Cabanier caban...@gmail.com wrote:

 There's a thread on www-style:
 http://lists.w3.org/Archives/Public/www-style/2012Nov/0144.html
 It's been in firefox for a while and blink is going to ship it soon:

 https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/RyUSi3zdumQ


devicePixelRatio has reflected page zoom in Firefox ever since we first
exposed it (because the thing it maps to internally has always been
affected by page zoom).


   Actually what I really think we should do is also change the
   window.devicePixelRatio for pinch zoom. Combined with the suggestions
 for
   canvas, that would allow (as Rik pointed out on IRC) for high-quality
   canvas all the way zoomed in, and for cheap canvases when zoomed out

 I believe pinch zoom should not change devicePixelRatio. It would be nice
 if there was a standard way to determine pinch zoom though and if the
 canvas could automatically resize.


Pinch-zoom is hard because we don't want to trigger reflows or other
expensive behavior on pinch-zoom. I'd leave pinch-zoom out of it for now.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] High-density canvases

[Tab Atkins Jr.]

On Wed, Sep 11, 2013 at 1:20 AM, Robert O'Callahan rob...@ocallahan.org wrote:
   Actually what I really think we should do is also change the
   window.devicePixelRatio for pinch zoom. Combined with the suggestions
 for
   canvas, that would allow (as Rik pointed out on IRC) for high-quality
   canvas all the way zoomed in, and for cheap canvases when zoomed out

 I believe pinch zoom should not change devicePixelRatio. It would be nice
 if there was a standard way to determine pinch zoom though and if the
 canvas could automatically resize.


 Pinch-zoom is hard because we don't want to trigger reflows or other
 expensive behavior on pinch-zoom. I'd leave pinch-zoom out of it for now.

Yes.  Somewhere (HTML? CSSOM? CSSOM-View?) needs to define
layout-affecting zoom and visual zoom, define what goes where, and
then we can just hook things to layout-affecting zoom and be done
with it.

(I think dbaron had some name suggestions, or at least some name
*objections*, related to this stuff.)

~TJ

Re: [whatwg] High-density canvases

[Ian Hickson]

On Wed, 11 Sep 2013, Robert O'Callahan wrote:
 
 Pinch-zoom is hard because we don't want to trigger reflows or other 
 expensive behavior on pinch-zoom. I'd leave pinch-zoom out of it for 
 now.

Unless I'm missing something fundamental, changing the pixel density 
doesn't cause a layout, it's changing the width that causes a layout.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] High-density canvases

[Rik Cabanier]

On Wed, Sep 11, 2013 at 11:24 AM, Ian Hickson i...@hixie.ch wrote:

 On Wed, 11 Sep 2013, Robert O'Callahan wrote:
 
  Pinch-zoom is hard because we don't want to trigger reflows or other
  expensive behavior on pinch-zoom. I'd leave pinch-zoom out of it for
  now.

 Unless I'm missing something fundamental, changing the pixel density
 doesn't cause a layout, it's changing the width that causes a layout.


It will do a layout if the width is directly or indirectly dependent on the
CSS pixelratio.
For instance, notice how the text boxes reflow if you change the page zoom:
http://jsfiddle.net/BtAK7/1/

Re: [whatwg] High-density canvases

[Ian Hickson]

On Wed, 11 Sep 2013, Rik Cabanier wrote:
 On Wed, Sep 11, 2013 at 11:24 AM, Ian Hickson i...@hixie.ch wrote:
  On Wed, 11 Sep 2013, Robert O'Callahan wrote:
  
   Pinch-zoom is hard because we don't want to trigger reflows or other 
   expensive behavior on pinch-zoom. I'd leave pinch-zoom out of it for 
   now.
 
  Unless I'm missing something fundamental, changing the pixel density 
  doesn't cause a layout, it's changing the width that causes a layout.
 
 It will do a layout if the width is directly or indirectly dependent on 
 the CSS pixelratio. For instance, notice how the text boxes reflow if 
 you change the page zoom: http://jsfiddle.net/BtAK7/1/

Sure. But pinch-zoom doesn't change the width. (Page zoom does.)

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] High-density canvases

[Stephen White]

For posterity, here were our objections to the original high-DPI canvas
spec:

   - The API feels like a short-term hack to automagically do something
   that the developer may or may not want done (e.g., if the game/app was
   tuned for particular resolution, or for pixel-exact rendering) that we'll
   be stuck with in the web platform long after its short-term usefulness has
   expired
   - It doesn't scale well to non-integer devicePixelRatios
   - It is easy for developers to implement the above behaviour in JS if
   desired

I think the new proposal addresses the first point, since it's opt-in. I
don't think the second point is a problem, since [get|put]ImageData() will
be back to manipulating exact backing store pixels, so no non-integer
resizing will be required. The third point becomes moot.

One question: now that some browsers are including browser zoom (page zoom)
in window.devicePixelRatio, will/should the new proposal automatically
cause a resize callback on page zoom, in order to preserve 1:1 device
pixels? (Note that I think this is a problem with current JS-based
implementations of canvas auto-scale as well, although perhaps there's a
DOM event for this that you can listen to; I might just be showing my
ignorance here.)

Stephen


On Mon, Sep 9, 2013 at 8:00 PM, Ian Hickson i...@hixie.ch wrote:

 On Wed, 17 Jul 2013, Rik Cabanier wrote:
  Ian wrote:
  
   The density aspect of this might be pointless, given the failure of
   getImageDataHD(); if we're dropping that one, I'll drop this one at
   the same time.
 
  Yes, please drop it since the HD methods are going away from the one
  implementation.

 On Tue, 9 Jul 2013, Stephen White wrote:
 
  Conversely, if it helps to bring the spec closer to the implementations,
  one thing we do not intend to implement in Chrome is the automatic
  high-DPI canvas scaling (ie., auto-doubling of backing stores,
  getImageDataHD(), putImageDataHD(), etc).
 
  I believe Apple has also announced that they are dropping support for
  this in Safari 7.

 So my understanding is that the reason this feature failed is that there's
 existing content that assumes a 1:1 ratio, and having an automatic
 high-density mode was making some pages end up with canvases with four
 canvas pixels per CSS pixel (linearly) -- two from the browser making a
 native canvas, times two from the page scaling the canvas for high DPI
 displays. This is a factor of sixteen over a 1:1 canvas, a factor of four
 more than it should be for high DPI, and a big waste of resources.

 As much as sites do this manually, though, it's a huge pain in the neck to
 have to worry about pixel density when you're creating your canvas and
 drawing on it, especially if you're not drawing sprites on it.

 While we're talking about annoying things, there's also the annoyance that
 canvases tend to not take zoom into account (either density-affecting zoom
 like page zoom on desktop, or transparent zoom like pinch-zoom on mobile
 for non-mobile-optimised sites, which the site isn't supposed to know
 about): you have to remember to listen for onresize, and then manually
 blow away your canvas and recreate it at the right density and then
 squeeze it into place so that the coordinate space matches what your code
 is expecting while the canvas is actually sized for the display.

 There's also the issue of full-bleed canvases where every time the
 container changes, you have to remember to re-update the canvas coordinate
 space and repaint because otherwise your pretty page gets all warped.

 It would be nice to fix these all at once, and I think we can, by
 introducing a configuration option on getContext(), in the style of WebGL:

getContext('2d', { density: 'autosize' });

 This would trigger the following behaviour: When the context is created,
 and subsequently when the canvas changes size (e.g. due to being sized
 with CSS relative units and the element they're relative to changing), or
 when the display density changes size (e.g. due to page zoom), then:

- the width and height of the canvas bitmaps get updated to match the
  new native size of the canvas, at native density.

- the coordinate space of the canvas (context.width/context.height)
  gets updated to match the size of the canvas in CSS pixel units.

- a 'resize' event gets fired at the canvas.

 We would dump the *HD versions of the methods, and make the regular ones
 go back to returning the actual raw pixels, since that would now work fine
 and still provide HD-quality content everywhere it's available.

 What do people think?

 --
 Ian Hickson   U+1047E)\._.,--,'``.fL
 http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
 Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] High-density canvases

[Boris Zbarsky]

On 9/10/13 2:55 PM, Dean Jackson wrote:

Ouch. Who is doing this and why?


Any browser in which zoom changes the size of a CSS pixel, and because 
that's the definition of devicePixelRatio?


-Boris

Re: [whatwg] High-density canvases

[Dean Jackson]

On 11 Sep 2013, at 5:32 am, Ian Hickson i...@hixie.ch wrote:

 On Wed, 11 Sep 2013, Dean Jackson wrote:
 On 11 Sep 2013, at 12:14 am, Stephen White senorbla...@chromium.org 
 wrote:
 
 now that some browsers are including browser zoom (page zoom) in 
 window.devicePixelRatio
 
 Ouch. Who is doing this and why?
 
 Why ouch?

Because we’ve always operated under the assumption that devicePixelRatio is the 
mapping from CSS px to device pixels at 1:1 scale/zoom, and that it is constant 
for a particular display. The only change content would see is if the window is 
dragged to/from a high resolution screen.

There are other ways to query the page zoom.

I fear this will break existing content. I don’t think we’d ever want to change 
the behaviour.

Dean

 
 Actually what I really think we should do is also change the 
 window.devicePixelRatio for pinch zoom. Combined with the suggestions for 
 canvas, that would allow (as Rik pointed out on IRC) for high-quality 
 canvas all the way zoomed in, and for cheap canvases when zoomed out.

Re: [whatwg] High-density canvases

[Ian Hickson]

On Wed, 11 Sep 2013, Dean Jackson wrote:
 On 11 Sep 2013, at 5:32 am, Ian Hickson i...@hixie.ch wrote:
  On Wed, 11 Sep 2013, Dean Jackson wrote:
  On 11 Sep 2013, at 12:14 am, Stephen White senorbla...@chromium.org 
  wrote:
  
  now that some browsers are including browser zoom (page zoom) in 
  window.devicePixelRatio
  
  Ouch. Who is doing this and why?
  
  Why ouch?
 
 Because we’ve always operated under the assumption that devicePixelRatio 
 is the mapping from CSS px to device pixels at 1:1 scale/zoom, and that 
 it is constant for a particular display. The only change content would 
 see is if the window is dragged to/from a high resolution screen.
 
 There are other ways to query the page zoom.

That seems broken... why would you want to hide the page zoom?


 I fear this will break existing content. I don’t think we’d ever want to 
 change the behaviour.

I guess we'll just have to treat devicePixelRatio as legacy and introduce 
a new value that's the real device:pixel ratio, then.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] High-density canvases

[Dean Jackson]

On 11 Sep 2013, at 6:13 am, Ian Hickson i...@hixie.ch wrote:

 On Wed, 11 Sep 2013, Dean Jackson wrote:
 On 11 Sep 2013, at 5:32 am, Ian Hickson i...@hixie.ch wrote:
 On Wed, 11 Sep 2013, Dean Jackson wrote:
 On 11 Sep 2013, at 12:14 am, Stephen White senorbla...@chromium.org 
 wrote:
 
 now that some browsers are including browser zoom (page zoom) in 
 window.devicePixelRatio
 
 Ouch. Who is doing this and why?
 
 Why ouch?
 
 Because we’ve always operated under the assumption that devicePixelRatio 
 is the mapping from CSS px to device pixels at 1:1 scale/zoom, and that 
 it is constant for a particular display. The only change content would 
 see is if the window is dragged to/from a high resolution screen.
 
 There are other ways to query the page zoom.
 
 That seems broken... why would you want to hide the page zoom?

We don’t want to hide page zoom.

I think there are two separate things a developer might want:

- the number of actual pixels that correspond to 1 CSS px without zoom
- the page zoom

If you merge the two, then an unsuspecting developer might think that the user 
has zoomed in by 2x on an iPhone, and decide to make things smaller. Yes, 
that’s not necessarily great content, but it makes some sense. For most page 
content you don’t really care about the device:pixel ratio - text looks fine! 
For images, you should use the mechanisms we’ve developed (srcset and 
image-set).

Now, I completely agree that there are going to be cases where you might want 
the image selected for srcset and image-set to respond to zoom. After all, 
there is no point loading a huge image if it is going to be 1x1 cm^2. But I 
think that’s separate from changing devicePixelRatio.

 I fear this will break existing content. I don’t think we’d ever want to 
 change the behaviour.
 
 I guess we'll just have to treat devicePixelRatio as legacy and introduce 
 a new value that's the real device:pixel ratio, then.

Indeed. I’m not opposed to exposing this. I am reluctant to change something 
that has behaved a particular way for a number of years.

Meanwhile, back to canvas, why don’t we change canvas to be purely callback 
based, and pass in all the info a developer needs to decide what the best 
output should be? Put something like requestAnimationFrame into the canvas spec 
(although that is a bad name - it only sometimes is related to animation). That 
way you could even imagine a future where a single canvas could have multiple 
render targets (when printing you get an even higher resolution).

Dean

Re: [whatwg] High-density canvases

[Rik Cabanier]

On Tue, Sep 10, 2013 at 12:45 PM, Dean Jackson d...@apple.com wrote:


 On 11 Sep 2013, at 5:32 am, Ian Hickson i...@hixie.ch wrote:

  On Wed, 11 Sep 2013, Dean Jackson wrote:
  On 11 Sep 2013, at 12:14 am, Stephen White senorbla...@chromium.org
  wrote:
 
  now that some browsers are including browser zoom (page zoom) in
  window.devicePixelRatio
 
  Ouch. Who is doing this and why?
 
  Why ouch?

 Because we’ve always operated under the assumption that devicePixelRatio
 is the mapping from CSS px to device pixels at 1:1 scale/zoom, and that it
 is constant for a particular display. The only change content would see is
 if the window is dragged to/from a high resolution screen.

 There are other ways to query the page zoom.

 I fear this will break existing content. I don’t think we’d ever want to
 change the behaviour.


There's a thread on www-style:
http://lists.w3.org/Archives/Public/www-style/2012Nov/0144.html
It's been in firefox for a while and blink is going to ship it soon:
https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/RyUSi3zdumQ




 
  Actually what I really think we should do is also change the
  window.devicePixelRatio for pinch zoom. Combined with the suggestions for
  canvas, that would allow (as Rik pointed out on IRC) for high-quality
  canvas all the way zoomed in, and for cheap canvases when zoomed out


I believe pinch zoom should not change devicePixelRatio. It would be nice
if there was a standard way to determine pinch zoom though and if the
canvas could automatically resize.

Re: [whatwg] High-density canvases

[Ian Hickson]

On Wed, 11 Sep 2013, Dean Jackson wrote:
 
 I think there are two separate things a developer might want:
 
 - the number of actual pixels that correspond to 1 CSS px without zoom
 - the page zoom

Why? Exposing page zoom separately from device density seems like a 
fundamental abstraction failure. What's the difference between page zoom 
and native density?


 If you merge the two, then an unsuspecting developer might think that 
 the user has zoomed in by 2x on an iPhone, and decide to make things 
 smaller.

That's precisely why we should never make page zoom detectable. The entire 
point of zooming is that the author should _not_ make things smaller. 
Zooming should make CSS pixels bigger.

Authors should always treat CSS pixels as the core unit, so if something 
should be 10 CSS pixels wide, then that's what the author should do. If 
the user happens to have zoomed in so that those 10 pixels are 30cm 
across, then that's what the user wants -- why would the author be able to 
override that?


 Yes, that’s not necessarily great content, but it makes some sense.

What sense does it make?


 For most page content you don’t really care about the device:pixel ratio 
 - text looks fine! For images, you should use the mechanisms we’ve 
 developed (srcset and image-set).

Well the context here is canvas, right?


 Now, I completely agree that there are going to be cases where you might 
 want the image selected for srcset and image-set to respond to zoom.

Not edge cases, they're pretty core cases. srcset= should definitely be 
influenced by page zoom, there's no question there. Otherwise, users who 
zoom in will get low quality images, which defeats the whole point.


 After all, there is no point loading a huge image if it is going to be 
 1x1 cm^2. But I think that’s separate from changing devicePixelRatio.

Why?



  I fear this will break existing content. I don’t think we’d ever want 
  to change the behaviour.
  
  I guess we'll just have to treat devicePixelRatio as legacy and 
  introduce a new value that's the real device:pixel ratio, then.
 
 Indeed. I’m not opposed to exposing this. I am reluctant to change 
 something that has behaved a particular way for a number of years.
 
 Meanwhile, back to canvas, why don’t we change canvas to be purely 
 callback based, and pass in all the info a developer needs to decide 
 what the best output should be? Put something like requestAnimationFrame 
 into the canvas spec (although that is a bad name - it only sometimes is 
 related to animation). That way you could even imagine a future where a 
 single canvas could have multiple render targets (when printing you get 
 an even higher resolution).

I'm not really understanding what you're proposing here. Can you 
elaborate? What's the problem this is trying to solve? How would this work 
in a multi-worker environment?

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] High-density canvases

[Dean Jackson]

On 11 Sep 2013, at 12:14 am, Stephen White senorbla...@chromium.org wrote:

 now that some browsers are including browser zoom (page zoom) in 
 window.devicePixelRatio

Ouch. Who is doing this and why?

Dean

Re: [whatwg] High-density canvases

[Kornel Lesiński]

On Tue, 10 Sep 2013 21:22:51 +0100, Dean Jackson d...@apple.com wrote:


I think there are two separate things a developer might want:

- the number of actual pixels that correspond to 1 CSS px without zoom
- the page zoom

If you merge the two, then an unsuspecting developer might think that  
the user has zoomed in by 2x on an iPhone, and decide to make things  
smaller.


Do you have an example of a page that does make things smaller to counter  
the zoom? Are you referring to some iPhone-specific workarounds (like  
position:fixed elements being problematic for zoom?)


I assumed that sites which don't like being zoomed in would just block it  
via meta viewport.


--
regards, Kornel

Re: [whatwg] High-density canvases

[Ian Hickson]

On Wed, 11 Sep 2013, Dean Jackson wrote:
 On 11 Sep 2013, at 12:14 am, Stephen White senorbla...@chromium.org 
 wrote:
  
  now that some browsers are including browser zoom (page zoom) in 
  window.devicePixelRatio
 
 Ouch. Who is doing this and why?

Why ouch?

Actually what I really think we should do is also change the 
window.devicePixelRatio for pinch zoom. Combined with the suggestions for 
canvas, that would allow (as Rik pointed out on IRC) for high-quality 
canvas all the way zoomed in, and for cheap canvases when zoomed out.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] High-density canvases

[Boris Zbarsky]

On 9/10/13 4:13 PM, Ian Hickson wrote:

I guess we'll just have to treat devicePixelRatio as legacy and introduce
a new value that's the real device:pixel ratio, then.


I would be interested in some data on what different UAs do with 
devicePixelRatio...


-Boris

Re: [whatwg] High-density canvases

[Boris Zbarsky]

On 9/10/13 4:22 PM, Dean Jackson wrote:

I think there are two separate things a developer might want:

- the number of actual pixels that correspond to 1 CSS px without zoom


What are the use cases for this, as opposed to number of actual pixels 
that correspond to 1 CSS px?


And are you talking about the sort of zoom that changes layout (because 
it changes the number of CSS px per device px and hence changes font 
rasterization), or the sort that does not (which basically does a zoom 
without changing font metrics)?



- the page zoom


Which one?


Indeed. I’m not opposed to exposing this. I am reluctant to change something 
that has behaved a particular way for a number of years.


Shipping UAs already disagree on how devicePixelRatio works with zoom. 
They also disagree on how zoom behaves in general.


-Boris

Re: [whatwg] High-density canvases

[Glenn Maynard]

On Mon, Sep 9, 2013 at 7:31 PM, Ian Hickson i...@hixie.ch wrote:

 Right, resetting the context would definitely be part of the deal. This
 mode would be specifically defined as a mode where you had to listen to
 onresize or your canvas would almost certainly get cleared sooner or
 later. In fact, we could go further, and say that canvases that aren't
 getting rendered at all (e.g. display:none, off-screen, background tab)
 can get cleared, with the deal being that next time you need to show the
 canvas you immediately get an onresize.


It would be better if the resize didn't happen until the page is actually
ready to re-render.  That way, the canvas doesn't flicker if it's rendered
before the page actually does render (eg. it may need to reload resources
to render).  Rendering a blurry canvas briefly is better than rendering a
blank canvas.

For example, add a method resizeToCurrentDPI(), and don't do it
automatically at all.  Fire an event when calling the method *would* cause
a change of canvas size.  The page can then load resources asynchronously
as needed, and call the method when it's ready to redraw, avoiding any
period where a blank canvas might be composited.


 Yeah, my suggestion, if we do this, would be to not do it until high
 density displays are even more widely available than now. This is mostly a
 convenience and performance-improving API, not a critical feature add.


High-DPI displays are already widespread in mobile (all Apple devices
except for the iPad Mini; the Kindle Fire HD), and by contrast there's no
sign of them for desktops, so I think we're either there now or we won't be
for a long time.

-- 
Glenn Maynard

Re: [whatwg] High-density canvases

[Tab Atkins Jr.]

On Tue, Sep 10, 2013 at 6:24 PM, Glenn Maynard gl...@zewt.org wrote:
 Yeah, my suggestion, if we do this, would be to not do it until high
 density displays are even more widely available than now. This is mostly a
 convenience and performance-improving API, not a critical feature add.

 High-DPI displays are already widespread in mobile (all Apple devices
 except for the iPad Mini; the Kindle Fire HD), and by contrast there's no
 sign of them for desktops, so I think we're either there now or we won't be
 for a long time.

High-end laptops have high-dpi screens (the Pixel I'm using right now
has one), and they're slowly spreading down the price scale.

~TJ

Re: [whatwg] High-density canvases

[Tab Atkins Jr.]

On Mon, Sep 9, 2013 at 5:00 PM, Ian Hickson i...@hixie.ch wrote:
 It would be nice to fix these all at once, and I think we can, by
 introducing a configuration option on getContext(), in the style of WebGL:

getContext('2d', { density: 'autosize' });

 This would trigger the following behaviour: When the context is created,
 and subsequently when the canvas changes size (e.g. due to being sized
 with CSS relative units and the element they're relative to changing), or
 when the display density changes size (e.g. due to page zoom), then:

- the width and height of the canvas bitmaps get updated to match the
  new native size of the canvas, at native density.

- the coordinate space of the canvas (context.width/context.height)
  gets updated to match the size of the canvas in CSS pixel units.

- a 'resize' event gets fired at the canvas.

 We would dump the *HD versions of the methods, and make the regular ones
 go back to returning the actual raw pixels, since that would now work fine
 and still provide HD-quality content everywhere it's available.

 What do people think?

Sounds reasonable to me.  Firing a resize directly at the canvas is
nice and convenient.

~TJ

Re: [whatwg] High-density canvases

[Dean Jackson]

On 10 Sep 2013, at 10:00 am, Ian Hickson i...@hixie.ch wrote:

 On Wed, 17 Jul 2013, Rik Cabanier wrote:
 Ian wrote:
 
 The density aspect of this might be pointless, given the failure of 
 getImageDataHD(); if we're dropping that one, I'll drop this one at 
 the same time.
 
 Yes, please drop it since the HD methods are going away from the one 
 implementation.
 
 On Tue, 9 Jul 2013, Stephen White wrote:
 
 Conversely, if it helps to bring the spec closer to the implementations, 
 one thing we do not intend to implement in Chrome is the automatic 
 high-DPI canvas scaling (ie., auto-doubling of backing stores, 
 getImageDataHD(), putImageDataHD(), etc).
 
 I believe Apple has also announced that they are dropping support for 
 this in Safari 7.
 
 So my understanding is that the reason this feature failed is that there's 
 existing content that assumes a 1:1 ratio, and having an automatic 
 high-density mode was making some pages end up with canvases with four 
 canvas pixels per CSS pixel (linearly) -- two from the browser making a 
 native canvas, times two from the page scaling the canvas for high DPI 
 displays. This is a factor of sixteen over a 1:1 canvas, a factor of four 
 more than it should be for high DPI, and a big waste of resources.

It wasn’t just that. A lot of existing code did something like this:

var pixels = ctx.getImageData(0, 0, canvas.width, canvas.height);
for (var i = 0; i  canvas.width; i++) {
  for (var j = 0; j  canvas.height; j++) {
pixels[j * canvas.width + i][0] = newRedValue;
pixels[j * canvas.width + i][1] = newGreenValue;
pixels[j * canvas.width + i][2] = newBlueValue;
  }
}
ctx.putImageData(…)

i.e. no one read the spec that says you should check the width and height
of the returned ImageData.

Then there is toDataURL(). Should that return a full-sized bitmap? Then you
need to be sure what you’re uploading it to will handle the larger size.

Basically, there was pretty unanimous support from Web developers to
“just give us the damn pixels we asked for”, as well as Chrome and Mozilla
suggesting they didn’t want to implement the auto-doubling.

We (Safari) were in a position where we were requiring developers to
write extra code just for us, as well as potentially breaking some
(existing) content.

 
 As much as sites do this manually, though, it's a huge pain in the neck to 
 have to worry about pixel density when you're creating your canvas and 
 drawing on it, especially if you're not drawing sprites on it.
 
 While we're talking about annoying things, there's also the annoyance that 
 canvases tend to not take zoom into account (either density-affecting zoom 
 like page zoom on desktop, or transparent zoom like pinch-zoom on mobile 
 for non-mobile-optimised sites, which the site isn't supposed to know 
 about): you have to remember to listen for onresize, and then manually 
 blow away your canvas and recreate it at the right density and then 
 squeeze it into place so that the coordinate space matches what your code 
 is expecting while the canvas is actually sized for the display.

Yes, but I think the developer is the one who best knows what size/quality
her content requires.

A great developer will do exactly as you suggest: constantly examine the
rendered size of the canvas (taking into account pixel density and 
viewport/zoom).

 
 There's also the issue of full-bleed canvases where every time the 
 container changes, you have to remember to re-update the canvas coordinate 
 space and repaint because otherwise your pretty page gets all warped.
 
 It would be nice to fix these all at once, and I think we can, by 
 introducing a configuration option on getContext(), in the style of WebGL:
 
   getContext('2d', { density: 'autosize' });
 
 This would trigger the following behaviour: When the context is created, 
 and subsequently when the canvas changes size (e.g. due to being sized 
 with CSS relative units and the element they're relative to changing), or 
 when the display density changes size (e.g. due to page zoom), then:
 
   - the width and height of the canvas bitmaps get updated to match the
 new native size of the canvas, at native density.
 
   - the coordinate space of the canvas (context.width/context.height) 
 gets updated to match the size of the canvas in CSS pixel units.

Note that this would reset the context, which would throw away the contents
and context state. This might be exactly what you want though - put
this attribute on a full-bleed canvas and resize your window - everything
disappears unless you implement the ‘resize’ event handler.

 
   - a 'resize' event gets fired at the canvas.
 
 We would dump the *HD versions of the methods, and make the regular ones 
 go back to returning the actual raw pixels, since that would now work fine 
 and still provide HD-quality content everywhere it's available.
 
 What do people think?

This seems ok to me. I still worry we’ll get into a situation where developers
will add 

Re: [whatwg] High-density canvases

[Rik Cabanier]

On Mon, Sep 9, 2013 at 5:00 PM, Ian Hickson i...@hixie.ch wrote:

 On Wed, 17 Jul 2013, Rik Cabanier wrote:
  Ian wrote:
  
   The density aspect of this might be pointless, given the failure of
   getImageDataHD(); if we're dropping that one, I'll drop this one at
   the same time.
 
  Yes, please drop it since the HD methods are going away from the one
  implementation.

 On Tue, 9 Jul 2013, Stephen White wrote:
 
  Conversely, if it helps to bring the spec closer to the implementations,
  one thing we do not intend to implement in Chrome is the automatic
  high-DPI canvas scaling (ie., auto-doubling of backing stores,
  getImageDataHD(), putImageDataHD(), etc).
 
  I believe Apple has also announced that they are dropping support for
  this in Safari 7.

 So my understanding is that the reason this feature failed is that there's
 existing content that assumes a 1:1 ratio, and having an automatic
 high-density mode was making some pages end up with canvases with four
 canvas pixels per CSS pixel (linearly) -- two from the browser making a
 native canvas, times two from the page scaling the canvas for high DPI
 displays. This is a factor of sixteen over a 1:1 canvas, a factor of four
 more than it should be for high DPI, and a big waste of resources.

 As much as sites do this manually, though, it's a huge pain in the neck to
 have to worry about pixel density when you're creating your canvas and
 drawing on it, especially if you're not drawing sprites on it.

 While we're talking about annoying things, there's also the annoyance that
 canvases tend to not take zoom into account (either density-affecting zoom
 like page zoom on desktop, or transparent zoom like pinch-zoom on mobile
 for non-mobile-optimised sites, which the site isn't supposed to know
 about): you have to remember to listen for onresize, and then manually
 blow away your canvas and recreate it at the right density and then
 squeeze it into place so that the coordinate space matches what your code
 is expecting while the canvas is actually sized for the display.

 There's also the issue of full-bleed canvases where every time the
 container changes, you have to remember to re-update the canvas coordinate
 space and repaint because otherwise your pretty page gets all warped.

 It would be nice to fix these all at once, and I think we can, by
 introducing a configuration option on getContext(), in the style of WebGL:

getContext('2d', { density: 'autosize' });

 This would trigger the following behaviour: When the context is created,
 and subsequently when the canvas changes size (e.g. due to being sized
 with CSS relative units and the element they're relative to changing), or
 when the display density changes size (e.g. due to page zoom), then:

- the width and height of the canvas bitmaps get updated to match the
  new native size of the canvas, at native density.

- the coordinate space of the canvas (context.width/context.height)
  gets updated to match the size of the canvas in CSS pixel units.

- a 'resize' event gets fired at the canvas.


Will this workflow introduce flickering?



 We would dump the *HD versions of the methods, and make the regular ones
 go back to returning the actual raw pixels, since that would now work fine
 and still provide HD-quality content everywhere it's available.

 What do people think?


I'm unsure if these calls are really needed. An author can already do this
today with a couple of lines of JS.
By doing the actual resizing in the event handler, I believe flickering is
avoided.

I share Dean's worry that it will lead to bad redrawing if people don't
test all scenarios that causes resizing. (Temporary canvases would also
have to be resized manually)

Re: [whatwg] High-density canvases

[Ian Hickson]

On Tue, 10 Sep 2013, Ian Hickson wrote:
 
 It would be nice to fix these all at once, and I think we can, by 
 introducing a configuration option on getContext(), in the style of 
 WebGL:
 
getContext('2d', { density: 'autosize' });
 
 This would trigger the following behaviour: When the context is created, 
 and subsequently when the canvas changes size (e.g. due to being sized 
 with CSS relative units and the element they're relative to changing), 
 or when the display density changes size (e.g. due to page zoom), then:
 
- the width and height of the canvas bitmaps get updated to match the
  new native size of the canvas, at native density.
 
- the coordinate space of the canvas (context.width/context.height) 
  gets updated to match the size of the canvas in CSS pixel units.
 
- a 'resize' event gets fired at the canvas.
 
 We would dump the *HD versions of the methods, and make the regular ones 
 go back to returning the actual raw pixels, since that would now work 
 fine and still provide HD-quality content everywhere it's available.
 
 What do people think?

Rik pointed out on IRC that it's common to have out-of-DOM canvases, e.g. 
for storing textures, which would need to track the density of the one in 
the DOM. I'd have to think about it in more detail, but in principle it 
seems some sort of slaving mechanism should be able to handle that too.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'