Re: [whatwg] Grouping in canvas 2d

[Jürg Lehni]

Here another simple example:

When both filling and stroking a path and then drawing it with with an opacity 
of less than 100%, the path will be rendered differently than in an SVG (a 
large stroke width will make the issue more apparent):

- In Canvas, both the fill and the stroke will be rendered with the given 
opacity, and the fill will shine through the inner half of the stroke.

- In SVG, the stroke will cover the fill, and the fill will not shine through 
the inner half of the stroke, regardless of the opacity.

If you'd like to emulate the SVG behavior in Canvas (which we happen to do in 
Paper.js), then the only way to do so currently is to draw the path's fill and 
stroke at 100% opacity into a separate canvas, and then blit the whole thing 
over with the given opacity.

This is *much* slower than directly drawing into the Canvas, and happens to be 
one of the worst bottlenecks in Paper.js

I would really appreciate a solution to this problem.

Jürg

On Mar 14, 2014, at 19:09 , Ian Hickson i...@hixie.ch wrote:

 Can you elaborate on what precisely the performance bottleneck is? I was 
 looking through this thread but I can't find a description of the use 
 cases it addresses, so it's hard to evaluate the proposals.

Re: [whatwg] Grouping in canvas 2d

[Ian Hickson]

On Wed, 4 Dec 2013, Jürg Lehni wrote:
 
 Implementing [layering/grouping] would help us greatly to optimize 
 aspects of Paper.js, as double buffering into separate canvases is very 
 slow and costly.

Can you elaborate on what precisely the performance bottleneck is? I was 
looking through this thread but I can't find a description of the use 
cases it addresses, so it's hard to evaluate the proposals.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] Grouping in canvas 2d

[Rik Cabanier]

On Fri, Mar 14, 2014 at 11:09 AM, Ian Hickson i...@hixie.ch wrote:

 On Wed, 4 Dec 2013, Jürg Lehni wrote:
 
  Implementing [layering/grouping] would help us greatly to optimize
  aspects of Paper.js, as double buffering into separate canvases is very
  slow and costly.

 Can you elaborate on what precisely the performance bottleneck is? I was
 looking through this thread but I can't find a description of the use
 cases it addresses, so it's hard to evaluate the proposals.


Let's say you're drawing a scene and there is a bunch of artwork that you
want to apply a multiply effect or opacity to.
With today's code, it would look something like this:

var bigcanvas = document.getElementById(c)l

var ctx = bigcanvas.getContext(2d);

ctx.moveto(); // drawing underlying scene

var c = document.createElement(canvas);
ctx = c.getContext(2d);

ctx.moveto(); // drawing scene that needs the effect

ctx = bigcanvas.getContext(2d);

ctx,globalAlpha(.5);

ctx.drawImage(c, 0, 0);

With layers, it would become:

var bigcanvas = document.getElementById(c)l
var ctx = bigcanvas.getContext(2d);
ctx.moveto(); // drawing underlying scene

ctx,globalAlpha(.5);
ctx.beginLayer();
ctx.moveto(); // drawing scene that needs the effect

ctx.endLayer();

So, with layers you
- avoid creating (expensive) DOM elements
- simplify the drawing (especially when there's a transformation)

Re: [whatwg] Grouping in canvas 2d

[Rik Cabanier]

I


On Wed, Dec 4, 2013 at 11:41 AM, Jürg Lehni li...@scratchdisk.com wrote:

 This discussion seems to have stalled.

 Implementing this would help us greatly to optimize aspects of Paper.js,
 as double buffering into separate canvases is very slow and costly. Is this
 still on the map?

 There appears to be a library that emulates this functionality, using the
 same approach (and having a similar impact on performance) as we do   :

 https://github.com/evanmoran/composite

 Perhaps this can serve as a base for further discussion?


I would love if this was implemented. Last attempt stalled because Simon
Fraser objected. Simon, do you still object?


 On Jun 18, 2013, at 18:14 , Rik Cabanier caban...@gmail.com wrote:

  On Tue, Jun 18, 2013 at 7:25 AM, Justin Novosad ju...@google.com
 wrote:
 
 
 
 
  On Fri, Jun 14, 2013 at 2:34 PM, Rik Cabanier caban...@gmail.com
 wrote:
 
 
  I think so. If you leave a layer 'open', what would you display.
  It wouldn't just be for requestAnimationFrame, you would also need to
  define what happens if you read pixels with getImageData inside a
  beginLayer.
 
 
 
  I would like to suggest that all layers left 'open' should be
  automatically closed at the end of a script execution task.  Otherwise,
 it
  would be too easy to leak layers and get OOM crashes. This would also
  guarantee that the canvas contents are in a ready to display state.
 
 
  That sounds reasonable. Does this happen with save/restore as well?
  If you want to draw incrementally, you wouldn't want this to happen
 either.
 
 
 
  For the getImageData question I think the options that would make most
  sense are either:
  a) Read from the root layer (up to date except for contents of currently
  open layers),
  b) Read from the current (top most) layer, or
  c) Fail
  These options will never require the compositing of open (potentially
  unfinished) layers, which I think is important for implementations to be
  efficient.
  I am not sure which of the above makes more sense though.
 
 
  This is probably a case where we need to check with the implementors.
  Safari is the only browser that bolts the canvas implementation straight
  onto the OS calls so we should probably spec out what they do since other
  browser can probably emulate it.

Re: [whatwg] Grouping in canvas 2d

[Jürg Lehni]

This discussion seems to have stalled.

Implementing this would help us greatly to optimize aspects of Paper.js, as 
double buffering into separate canvases is very slow and costly. Is this still 
on the map?

There appears to be a library that emulates this functionality, using the same 
approach (and having a similar impact on performance) as we do   :

https://github.com/evanmoran/composite

Perhaps this can serve as a base for further discussion?

Jürg

On Jun 18, 2013, at 18:14 , Rik Cabanier caban...@gmail.com wrote:

 On Tue, Jun 18, 2013 at 7:25 AM, Justin Novosad ju...@google.com wrote:
 
 
 
 
 On Fri, Jun 14, 2013 at 2:34 PM, Rik Cabanier caban...@gmail.com wrote:
 
 
 I think so. If you leave a layer 'open', what would you display.
 It wouldn't just be for requestAnimationFrame, you would also need to
 define what happens if you read pixels with getImageData inside a
 beginLayer.
 
 
 
 I would like to suggest that all layers left 'open' should be
 automatically closed at the end of a script execution task.  Otherwise, it
 would be too easy to leak layers and get OOM crashes. This would also
 guarantee that the canvas contents are in a ready to display state.
 
 
 That sounds reasonable. Does this happen with save/restore as well?
 If you want to draw incrementally, you wouldn't want this to happen either.
 
 
 
 For the getImageData question I think the options that would make most
 sense are either:
 a) Read from the root layer (up to date except for contents of currently
 open layers),
 b) Read from the current (top most) layer, or
 c) Fail
 These options will never require the compositing of open (potentially
 unfinished) layers, which I think is important for implementations to be
 efficient.
 I am not sure which of the above makes more sense though.
 
 
 This is probably a case where we need to check with the implementors.
 Safari is the only browser that bolts the canvas implementation straight
 onto the OS calls so we should probably spec out what they do since other
 browser can probably emulate it.

Re: [whatwg] Grouping in canvas 2d

[Justin Novosad]

On Fri, Jun 14, 2013 at 2:34 PM, Rik Cabanier caban...@gmail.com wrote:


 I think so. If you leave a layer 'open', what would you display.
 It wouldn't just be for requestAnimationFrame, you would also need to
 define what happens if you read pixels with getImageData inside a
 beginLayer.



I would like to suggest that all layers left 'open' should be automatically
closed at the end of a script execution task.  Otherwise, it would be too
easy to leak layers and get OOM crashes. This would also guarantee that the
canvas contents are in a ready to display state.

For the getImageData question I think the options that would make most
sense are either:
a) Read from the root layer (up to date except for contents of currently
open layers),
b) Read from the current (top most) layer, or
c) Fail
These options will never require the compositing of open (potentially
unfinished) layers, which I think is important for implementations to be
efficient.
I am not sure which of the above makes more sense though.

Re: [whatwg] Grouping in canvas 2d

[Rik Cabanier]

On Tue, Jun 18, 2013 at 7:25 AM, Justin Novosad ju...@google.com wrote:




 On Fri, Jun 14, 2013 at 2:34 PM, Rik Cabanier caban...@gmail.com wrote:


 I think so. If you leave a layer 'open', what would you display.
 It wouldn't just be for requestAnimationFrame, you would also need to
 define what happens if you read pixels with getImageData inside a
 beginLayer.



 I would like to suggest that all layers left 'open' should be
 automatically closed at the end of a script execution task.  Otherwise, it
 would be too easy to leak layers and get OOM crashes. This would also
 guarantee that the canvas contents are in a ready to display state.


That sounds reasonable. Does this happen with save/restore as well?
If you want to draw incrementally, you wouldn't want this to happen either.



 For the getImageData question I think the options that would make most
 sense are either:
 a) Read from the root layer (up to date except for contents of currently
 open layers),
 b) Read from the current (top most) layer, or
 c) Fail
 These options will never require the compositing of open (potentially
 unfinished) layers, which I think is important for implementations to be
 efficient.
 I am not sure which of the above makes more sense though.


This is probably a case where we need to check with the implementors.
Safari is the only browser that bolts the canvas implementation straight
onto the OS calls so we should probably spec out what they do since other
browser can probably emulate it.

Re: [whatwg] Grouping in canvas 2d

[Jürg Lehni]

I am very much interested in this feature. Of course you can use separate 
canvases to achieve the same, which is what we currently do in Paper.js, but it 
appears to be rather slow to do so. I think performance could improve a lot 
which this alternative API proposal.

Jürg

On Jun 13, 2013, at 20:57 , Rik Cabanier caban...@gmail.com wrote:

 Last year, I requested if grouping could be added to canvas.
 The API would look like this:
 
 void beginLayer();
 void beginLayer(unsigned long x, unsigned long y, unsigned long w, unsigned
 long h);
 void endLayer();
 
 
 When you call beginLayer, you inherit everything from the graphics state
 except shadows, opacity and the current compositing mode. Those will reset
 to their initial value.
 At endLayer time, the graphics state will be restored and you will draw the
 content of the group.
 
 so for instance, if you want multiply blend with opacity of .5
 
 ctx.globalAlpha = .5;
 ctx.globalCompositeOperation = multiply;
 ctx.beginLayer();
 ... // drawing
 ctx.endLayer();
 
 This would cause everything between begin/endLayer to be drawn normally.
 This result is then blended and composited at endLayer time.
 
 Last year, people argued that you could achieve the same effect by using a
 temporary canvas so this API is not really needed.
 I would like to see if people are more open to this API now.
 Reasons to have it:
 - it is easily achievable with existing graphics libraries.
 - it is a very common idiom. You can cut down on the amount of JS needed.
 - if you want to avoid antialiasing issue, you need to carefully set the
 CTM on the temporary canvas and remove the CTM from the current canvas
 - creating a temporary canvas has a higher overhead than simply starting a
 layer.
 - creating lots of temporary canvas objects creates memory overhead
 - is polyfillable for older canvas implementations
 
 Rik

Re: [whatwg] Grouping in canvas 2d

[Rik Cabanier]

On Fri, Jun 14, 2013 at 6:04 AM, Brian Salomon bsalo...@chromium.orgwrote:

 As an implementor, we would prefer the layer approach. This would have
 lower overhead in Chromium/Skia. We can make better decisions about caching
 and deferred rendering. It also seems like a really handy API for devs,
 especially the ability to inherit the graphics state. Would the spec have
 anything to say about beginLayer()/endLayer() balancing, especially with
 respect to RAF?


I think so. If you leave a layer 'open', what would you display.
It wouldn't just be for requestAnimationFrame, you would also need to
define what happens if you read pixels with getImageData inside a
beginLayer.




 On Thu, Jun 13, 2013 at 11:57 PM, Rik Cabanier caban...@gmail.com wrote:

 Last year, I requested if grouping could be added to canvas.
 The API would look like this:

 void beginLayer();
 void beginLayer(unsigned long x, unsigned long y, unsigned long w,
 unsigned long h);
 void endLayer();


 When you call beginLayer, you inherit everything from the graphics state
 except shadows, opacity and the current compositing mode. Those will reset
 to their initial value.
 At endLayer time, the graphics state will be restored and you will draw
 the content of the group.

 so for instance, if you want multiply blend with opacity of .5

 ctx.globalAlpha = .5;
 ctx.globalCompositeOperation = multiply;
 ctx.beginLayer();
 ... // drawing
 ctx.endLayer();

 This would cause everything between begin/endLayer to be drawn normally.
 This result is then blended and composited at endLayer time.

 Last year, people argued that you could achieve the same effect by using
 a temporary canvas so this API is not really needed.
 I would like to see if people are more open to this API now.
 Reasons to have it:
 - it is easily achievable with existing graphics libraries.
 - it is a very common idiom. You can cut down on the amount of JS needed.
 - if you want to avoid antialiasing issue, you need to carefully set the
 CTM on the temporary canvas and remove the CTM from the current canvas
 - creating a temporary canvas has a higher overhead than simply starting
 a layer.
 - creating lots of temporary canvas objects creates memory overhead
 - is polyfillable for older canvas implementations

 Rik

[whatwg] Grouping in canvas 2d

[Rik Cabanier]

Last year, I requested if grouping could be added to canvas.
The API would look like this:

void beginLayer();
void beginLayer(unsigned long x, unsigned long y, unsigned long w, unsigned
long h);
void endLayer();


When you call beginLayer, you inherit everything from the graphics state
except shadows, opacity and the current compositing mode. Those will reset
to their initial value.
At endLayer time, the graphics state will be restored and you will draw the
content of the group.

so for instance, if you want multiply blend with opacity of .5

ctx.globalAlpha = .5;
ctx.globalCompositeOperation = multiply;
ctx.beginLayer();
... // drawing
ctx.endLayer();

This would cause everything between begin/endLayer to be drawn normally.
This result is then blended and composited at endLayer time.

Last year, people argued that you could achieve the same effect by using a
temporary canvas so this API is not really needed.
I would like to see if people are more open to this API now.
Reasons to have it:
- it is easily achievable with existing graphics libraries.
- it is a very common idiom. You can cut down on the amount of JS needed.
- if you want to avoid antialiasing issue, you need to carefully set the
CTM on the temporary canvas and remove the CTM from the current canvas
- creating a temporary canvas has a higher overhead than simply starting a
layer.
- creating lots of temporary canvas objects creates memory overhead
- is polyfillable for older canvas implementations

Rik