Re: [whatwg] Canvas: clarification of compositing operations needed

[carol.szabo]

HTML5 - Canvas.

I have read this thread (from and of July 2010) and I happen to agree that the 
Safari/Chromium implementation is more intuitive, and likely less expensive to 
implement, therefore I offer these 2 proposals for changing the spec's Drawing 
model section to match what I perceive to be the preference of most people in 
that discussion.

Of note: According to the current Drawing model, an opaque shape's shadow would 
be erased as part of step 6 when drawn with source-in composite Mode if 
globalAlpha is 1, which is probably not the intended behavior.

Version 1:

4.8.11.1.13 Drawing model


When a shape or image is painted, user agents must follow these steps, in the 
order given (or act as if they do):

   1. Render the shape or image onto an infinite transparent black bitmap, 
creating image M1, as described in the previous sections except that for the 
purpose of this step every pixel of the image will be considered to be fully 
opaque white and the current fillStyle will be considered to be solid fully 
opaque white and the strokeStyle will be considered fullyOpaque white as well.

   2. When shadows are drawn, render the shadow from image M1, using a fully 
opaque white shadow color as described in the shadows section, creating image 
M2.

   3. Let C1 be a region obtained by intersecting the canvas's cliping region 
with the set of pixels in the canvas that correspond to pixels in M1 (by having 
the same coordinates) that are not fully transparent.

   4. Let C2 be a region obtained by intersecting the canvas's cliping region 
with the set of pixels in the canvas that correspond to pixels in M2 (by having 
the same coordinates) that are not fully transparent.

   5. Render the shape or image onto an infinite transparent black bitmap, 
creating image A, as described in the previous sections. For shapes, the 
current fill, stroke, and line styles must be honored, and the stroke must 
itself also be subjected to the current transformation matrix.

   6. When shadows are drawn, render the shadow from image A, using the current 
shadow styles, creating image B.

   7. When shadows are drawn, multiply the alpha component of every pixel in B 
by globalAlpha.

   8. When shadows are drawn, composite B with the current canvas bitmap, 
cliping results to region C2 defined above, using the current composition 
operator.

   9. Multiply the alpha component of every pixel in A by globalAlpha.

  10. Composite A with the current canvas bitmap, cliping results to region C1 
defined above, using the current composition operator. 


Version 2:

4.8.11.1.13 Drawing model


When a shape or image is painted, user agents must follow these steps, in the 
order given (or act as if they do):

   1. Render the shape or image onto an infinite transparent black bitmap, 
creating image A, as described in the previous sections. For shapes, the 
current fill, stroke, and line styles must be honored, and the stroke must 
itself also be subjected to the current transformation matrix.

   2. When shadows are drawn, render the shadow from image A, using the current 
shadow styles, creating image B.

   3. When shadows are drawn, multiply the alpha component of every pixel in B 
by globalAlpha.

   4. When shadows are drawn, composite, using the current composition 
operator, B with the current canvas bitmap, cliping results to the cliping 
region of the canvas and to the pixels that would have taken the shadow's color 
if the composition operator would have been source-over and the shadow would 
have been fully opaque and the globalAlpha would have been 1.
   5. Multiply the alpha component of every pixel in A by globalAlpha.
   6. Composite, using the current composition operator, A with the current 
canvas bitmap, cliping results to the cliping region of the canvas and to the 
pixels that would have taken the shape's or image's pixel color if the 
composition operator would have been source-over and the image would have been 
fully opaque, the fillStyle and strokeStyle would have been a solid fully 
opaque color, and the globalAlpha would have been 1. 


I also suggest, this change to section 4.8.11.1.3 Compositing:

Instead of:

The operators in the above list must be treated as described by the Porter-Duff 
operator given at the start of their description (e.g. A over B). They are to 
be applied as part of the drawing model, at which point the clipping region is 
also applied. (Without a clipping region, these operators act on the whole 
bitmap with every operation.) [PORTERDUFF]

Say:
The operators in the above list must be treated as described by the Porter-Duff 
operator [PORTERDUFF] given at the start of their description (e.g. A over B). 
They are to be applied as part of the drawing model, which also defines the 
pixels in the canvas that are to be composited.

Re: [whatwg] Canvas: clarification of compositing operations needed

[Robert O'Callahan]

On Thu, Jul 29, 2010 at 10:09 AM, Oliver Hunt  wrote:

> This is the way the webkit canvas implementation has always worked, firefox
> implemented this incorrectly, and the spec was based off of that
> implementation.
>

I don't think "the spec was based off of that implementation" is true, since
Firefox never matched the spec (certain operators such as "copy" were
treated as source-bounded, because cairo does it that way, just like CG)
until I fixed that relatively recently.

Additionally the webkit behaviour is more powerful than the spec behaviour
> as the spec behaviour can be emulated trivially on top of the webkit model,
> but vice versa is much harder and much more expensive.
>

They're both pretty easy to emulate in terms of the other. But I agree that
emulating an unbounded operator in terms of the source-bounded operator has
a higher performance penalty, since the easy implementation is to use a
temporary surface, where as to emulate source-bounded using unbounded you
just do some extra clipping.

As it happens, I'm expecting to see a proposal on public-canvas-api sometime
soon that makes all composition source-bounded and has acceptable text
defining the shape affected by the composition operation. It's tricky
though, especially regarding shadows. But since I agree source-boundedness
is more intuitive for authors, I'm open to changing Firefox to support it.

Rob
-- 
"Now the Bereans were of more noble character than the Thessalonians, for
they received the message with great eagerness and examined the Scriptures
every day to see if what Paul said was true." [Acts 17:11]

Re: [whatwg] Canvas: clarification of compositing operations needed

[Tab Atkins Jr.]

On Thu, Jul 29, 2010 at 10:07 AM, Gregg Tavares (wrk)  wrote:
> #1) Get rid of the "infinite transparent black bitmap" stuff and change it
> to something that say only pixels inside the shape/image are effected

Like Ian said before, this is fine as soon as someone defines
precisely what pixels are "inside" of a shape/image.

~TJ

Re: [whatwg] Canvas: clarification of compositing operations needed

[Gregg Tavares (wrk)]

Even Firefox's implementation is inconsistent.

drawShape uses the "infinite transparent black bitmap" but drawImage does
not.

I believe even many at Mozilla would like Firefox to switch to the
Chrome/Safari method because it's more easily GPU accelerated.

In that direction it would be nice if 2 things in the spec changed

#1) Get rid of the "infinite transparent black bitmap" stuff and change it
to something that say only pixels inside the shape/image are effected

#2) Change the globalCompositingOperation spec from referencing PORTER-DUFF
to referencing OpenGL

source-over
   glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA);

source-in
   glBlendFunc(GL_DST_ALPHA, GL_ZERO);

source-out
   glBlendFunc(GL_ONE_MINUS_DST_ALPHA, GL_ZERO);

source-atop
   glBlendFunc(GL_DST_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

destination-over
   glBlendFunc(GL_ONE_MINUS_DST_ALPHA, GL_ONE);

destination-in
   glBlendFunc(GL_ZERO, GL_SRC_ALPHA)

destination-out
   glBlendFunc(GL_ZERO, GL_ONE_MINUS_SRC_ALPHA);

destination-atop
   glBlendFunc(GL_ONE_MINUS_DST_ALPHA, GL_SRC_ALPHA);

lighter
   glBlendFunc(GL_ONE, GL_ONE);

darker
   deprecated

copy
   glBlendFunc(GL_ONE, GL_ZERO);

xor
   glBlendFunc(GL_ONE_MINUS_DST_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

Re: [whatwg] Canvas: clarification of compositing operations needed

[David Flanagan]

Tab Atkins Jr. wrote:

On Wed, Jul 28, 2010 at 11:39 PM, David Flanagan
 wrote:

James Robinson wrote:

For example, I think
drawing a 20x20 image into a 500x500 canvas without scaling with a
globalCompositeOperation of 'copy' should result in only the 20x20 region
being cleared out, not the entire canvas.

Yikes!  It hadn't occurred to me that copy should behave that way.  But
you're right that that is what the spec requires.  Opera does it that way.
 Firefox, thankfully, does not.

Perhaps independently of the debate over infinite bitmap vs. shape extents,
we can agree that "copy" is a special value that means "do not perform
compositing"


That value already exists - it's called "source-over". 


You've lost me.  Are we disagreeing over the meaning of "composite".  It 
seems to me that "source-over" is clearly doing compositing: the result 
pixel is a blend of the source and destination pixels.



 "copy" does

some special compositing stuff no matter whether you do "global" or
"local" compositing - try using 'copy' when the source has .1 opacity.


The "copy" operation does not blend pixels: the result pixel is just the 
source pixel.  So when try what you suggest and draw with almost 
transparent pixels using the copy operation, the result is almost 
transparent pixels, regardless of what was under them.  That, to me, 
means that no compositing is being done.  What am I missing here?  What 
kind of "special stuff" is going on with "copy" compositing?


I'd argue that "copy" is the 2nd most important compositing operation 
after "source-over". Everyone but Opera violates the spec and treats it 
as a "local" operation.  If I understand correctly, the reason that the 
spec can't be changed to define compositing as a local operation is that 
vendors can't agree on what "local" means w.r.t. antialiasing, shadows, 
etc.


But since "copy" is a really important value, I propose that we sidestep 
the larger issue and explicitly state that when globalCompositeOperation 
is set to "copy" it means "just draw the damn pixels like we used to do 
in the olden days before we got all fancy with alpha channels and 
stuff".  A refinement would be to add a new value "none" and make "copy" 
a synonym for "none".


David

Re: [whatwg] Canvas: clarification of compositing operations needed

[Tab Atkins Jr.]

On Thu, Jul 29, 2010 at 7:17 AM, Leonardo Dutra  wrote:
> But this is not the point.
> "Composite A within the clipping region over the current canvas bitmap using
> the current composition operator."
> The word "clipping" is the point.

No, clipping is not the point either.  The difference between the
Firefox/spec behavior and the Webkit behavior has nothing to do with
clipping.  ("clipping region" refers specifically to the region you
create by calling the clip() function.)

~TJ

Re: [whatwg] Canvas: clarification of compositing operations needed

[Tab Atkins Jr.]

On Wed, Jul 28, 2010 at 11:39 PM, David Flanagan
 wrote:
> James Robinson wrote:
>> For example, I think
>> drawing a 20x20 image into a 500x500 canvas without scaling with a
>> globalCompositeOperation of 'copy' should result in only the 20x20 region
>> being cleared out, not the entire canvas.
>
> Yikes!  It hadn't occurred to me that copy should behave that way.  But
> you're right that that is what the spec requires.  Opera does it that way.
>  Firefox, thankfully, does not.
>
> Perhaps independently of the debate over infinite bitmap vs. shape extents,
> we can agree that "copy" is a special value that means "do not perform
> compositing"

That value already exists - it's called "source-over".  "copy" does
some special compositing stuff no matter whether you do "global" or
"local" compositing - try using 'copy' when the source has .1 opacity.

~TJ

Re: [whatwg] Canvas: clarification of compositing operations needed

[Leonardo Dutra]

People have a very strong feeling about Firefox. This is beautiful, but not
good to it. Firefox is now, over the Windows 7 and Mac, the slowest browser.
Just pick some JavaScripts and test. XUL is the most hard way for builting
an extension and Firefox eats CPU when we run any game built using HTML
Elements.

But this is not the point.

*"**Composite **A** within the clipping region over the current canvas
bitmap using the current composition operator."*
*
*
The word "clipping" is the point. I am very sad that input type="color" does
not cite the need of a color picker. But This part, is completely bright.

That's why Microsoft agree with Chrome, Chrome with Safari (they don't agree
sometimes) and Opera presents that way too.

A hug.

2010/7/29 David Flanagan 

> James Robinson wrote:
>
>> On Wed, Jul 28, 2010 at 2:46 PM, Tab Atkins Jr. > jackalm...@gmail.com>> wrote:
>>
>>On Wed, Jul 28, 2010 at 2:43 PM, David Flanagan
>>mailto:da...@davidflanagan.com>> wrote:
>> > Firefox and Chrome disagree about the implementation of the
>> > destination-atop, source-in, destination-in, and source-out
>>compositing
>> > operators.  Test code is attached.
>>
>>
>> I don't think your attachment made it through.
>> https://developer.mozilla.org/samples/canvas-tutorial/6_1_canvas_composite.htmlshows
>>  some of the differences, although it does not cover all cases.
>>
>
> You didn't miss much.  My example was very similar to the one you link to.
>
>
>
>> The spec is certainly clear but that does not make the behavior it
>> specifies good.  I find the spec's behavior pretty bizarre and Microsoft has
>> expressed a preference for the Safari/Chrome interpretation:
>> http://lists.w3.org/Archives/Public/public-canvas-api/2010AprJun/0046.html- 
>> although that thread did not get much discussion.
>>
>
> Thanks for that link.  The thread you reference refers back to an earlier
> thread on this list.  My apologies for not finding it and reading it before
> posting again.
>
>
> For example, I think
>
>> drawing a 20x20 image into a 500x500 canvas without scaling with a
>> globalCompositeOperation of 'copy' should result in only the 20x20 region
>> being cleared out, not the entire canvas.
>>
>
> Yikes!  It hadn't occurred to me that copy should behave that way.  But
> you're right that that is what the spec requires.  Opera does it that way.
>  Firefox, thankfully, does not.
>
> Perhaps independently of the debate over infinite bitmap vs. shape extents,
> we can agree that "copy" is a special value that means "do not perform
> compositing"
>
>David
>

Re: [whatwg] Canvas: clarification of compositing operations needed

[David Flanagan]

James Robinson wrote:
On Wed, Jul 28, 2010 at 2:46 PM, Tab Atkins Jr. > wrote:


On Wed, Jul 28, 2010 at 2:43 PM, David Flanagan
mailto:da...@davidflanagan.com>> wrote:
 > Firefox and Chrome disagree about the implementation of the
 > destination-atop, source-in, destination-in, and source-out
compositing
 > operators.  Test code is attached.


I don't think your attachment made it through. 
 https://developer.mozilla.org/samples/canvas-tutorial/6_1_canvas_composite.html shows 
some of the differences, although it does not cover all cases.


You didn't miss much.  My example was very similar to the one you link to.



The spec is certainly clear but that does not make the behavior it 
specifies good.  I find the spec's behavior pretty bizarre and Microsoft 
has expressed a preference for the Safari/Chrome 
interpretation: http://lists.w3.org/Archives/Public/public-canvas-api/2010AprJun/0046.html - 
although that thread did not get much discussion.  


Thanks for that link.  The thread you reference refers back to an 
earlier thread on this list.  My apologies for not finding it and 
reading it before posting again.


For example, I think
drawing a 20x20 image into a 500x500 canvas without scaling with a 
globalCompositeOperation of 'copy' should result in only the 20x20 
region being cleared out, not the entire canvas.


Yikes!  It hadn't occurred to me that copy should behave that way.  But 
you're right that that is what the spec requires.  Opera does it that 
way.  Firefox, thankfully, does not.


Perhaps independently of the debate over infinite bitmap vs. shape 
extents, we can agree that "copy" is a special value that means "do not 
perform compositing"


David

Re: [whatwg] Canvas: clarification of compositing operations needed

[David Flanagan]

Tab Atkins Jr. wrote:

On Wed, Jul 28, 2010 at 2:43 PM, David Flanagan  wrote:

Firefox and Chrome disagree about the implementation of the
destination-atop, source-in, destination-in, and source-out compositing
operators.  Test code is attached.

Chrome doesn't touch any destination pixels that are not underneath the
source pixels.  Firefox, on the other hand, treats the entire canvas (inside
the clipping region) as the destination and if you use the destination-in
operator, for example, will erase any pixels outside of whatever you are
drawing.

I suspect, based on the reference to an "infinite transparent black bitmap"
in 4.8.11.1.13 Drawing model that Firefox gets this right and Chrome gets it
wrong, but it would be nice to have that confirmed.

I suggest clarifying 4.8.11.1.3 Compositing to mention that the compositing
operation takes place on all pixels within the clipping region, and that
some compositing operators clear large portions of the canvas.


The spec is completely clear on this matter - Firefox is right,
Chrome/Safari are wrong.  They do it wrongly because that's how
CoreGraphics, their graphics library, does things natively.

~TJ



Thanks for the confirmation and the explanation of why webkit gets it 
wrong.


I disagree that the spec is completely clear, however.  In order to 
understand it you have to skip from section 3 on compositing operators 
to section 13 which mentions the infinite bitmap and compositing within 
the clipping region. There is no mention of the relevance of the 
clipping region in the section on compositing which seems like an oversight.


Also, descriptions of the operators cover only transparent and opaque 
pixels; they don't explain how compositing is done for translucent 
pixels, except by oblique reference to the Porter-Duff paper.


David

Re: [whatwg] Canvas: clarification of compositing operations needed

[James Robinson]

On Wed, Jul 28, 2010 at 2:46 PM, Tab Atkins Jr. wrote:

> On Wed, Jul 28, 2010 at 2:43 PM, David Flanagan 
> wrote:
> > Firefox and Chrome disagree about the implementation of the
> > destination-atop, source-in, destination-in, and source-out compositing
> > operators.  Test code is attached.
>

I don't think your attachment made it through.
https://developer.mozilla.org/samples/canvas-tutorial/6_1_canvas_composite.html
shows
some of the differences, although it does not cover all cases.

>
> > Chrome doesn't touch any destination pixels that are not underneath the
> > source pixels.  Firefox, on the other hand, treats the entire canvas
> (inside
> > the clipping region) as the destination and if you use the destination-in
> > operator, for example, will erase any pixels outside of whatever you are
> > drawing.
> >
> > I suspect, based on the reference to an "infinite transparent black
> bitmap"
> > in 4.8.11.1.13 Drawing model that Firefox gets this right and Chrome gets
> it
> > wrong, but it would be nice to have that confirmed.
> >
> > I suggest clarifying 4.8.11.1.3 Compositing to mention that the
> compositing
> > operation takes place on all pixels within the clipping region, and that
> > some compositing operators clear large portions of the canvas.
>
> The spec is completely clear on this matter - Firefox is right,
> Chrome/Safari are wrong.  They do it wrongly because that's how
> CoreGraphics, their graphics library, does things natively.
>

The spec is certainly clear but that does not make the behavior it specifies
good.  I find the spec's behavior pretty bizarre and Microsoft has expressed
a preference for the Safari/Chrome interpretation:
http://lists.w3.org/Archives/Public/public-canvas-api/2010AprJun/0046.html -
although that thread did not get much discussion.  For example, I think
drawing a 20x20 image into a 500x500 canvas without scaling with a
globalCompositeOperation of 'copy' should result in only the 20x20 region
being cleared out, not the entire canvas.

In informal discussions I got the impression that most folks would be happy
to standardize on something closer to the Safari/Chrome model if it could be
specified exactly.  In particular, there has to be a precise definition of
what region the compositing operation should apply in.

- James


>
> ~TJ
>

Re: [whatwg] Canvas: clarification of compositing operations needed

[Oliver Hunt]

On Jul 28, 2010, at 2:46 PM, Tab Atkins Jr. wrote:

> On Wed, Jul 28, 2010 at 2:43 PM, David Flanagan  
> wrote:
>> Firefox and Chrome disagree about the implementation of the
>> destination-atop, source-in, destination-in, and source-out compositing
>> operators.  Test code is attached.
>> 
>> Chrome doesn't touch any destination pixels that are not underneath the
>> source pixels.  Firefox, on the other hand, treats the entire canvas (inside
>> the clipping region) as the destination and if you use the destination-in
>> operator, for example, will erase any pixels outside of whatever you are
>> drawing.
>> 
>> I suspect, based on the reference to an "infinite transparent black bitmap"
>> in 4.8.11.1.13 Drawing model that Firefox gets this right and Chrome gets it
>> wrong, but it would be nice to have that confirmed.
>> 
>> I suggest clarifying 4.8.11.1.3 Compositing to mention that the compositing
>> operation takes place on all pixels within the clipping region, and that
>> some compositing operators clear large portions of the canvas.
> 
> The spec is completely clear on this matter - Firefox is right,
> Chrome/Safari are wrong.  They do it wrongly because that's how
> CoreGraphics, their graphics library, does things natively.

This is the way the webkit canvas implementation has always worked, firefox 
implemented this incorrectly, and the spec was based off of that 
implementation.  

Additionally the webkit behaviour is more powerful than the spec behaviour as 
the spec behaviour can be emulated trivially on top of the webkit model, but 
vice versa is much harder and much more expensive.

--Oliver


> 
> ~TJ

Re: [whatwg] Canvas: clarification of compositing operations needed

[Tab Atkins Jr.]

On Wed, Jul 28, 2010 at 2:43 PM, David Flanagan  wrote:
> Firefox and Chrome disagree about the implementation of the
> destination-atop, source-in, destination-in, and source-out compositing
> operators.  Test code is attached.
>
> Chrome doesn't touch any destination pixels that are not underneath the
> source pixels.  Firefox, on the other hand, treats the entire canvas (inside
> the clipping region) as the destination and if you use the destination-in
> operator, for example, will erase any pixels outside of whatever you are
> drawing.
>
> I suspect, based on the reference to an "infinite transparent black bitmap"
> in 4.8.11.1.13 Drawing model that Firefox gets this right and Chrome gets it
> wrong, but it would be nice to have that confirmed.
>
> I suggest clarifying 4.8.11.1.3 Compositing to mention that the compositing
> operation takes place on all pixels within the clipping region, and that
> some compositing operators clear large portions of the canvas.

The spec is completely clear on this matter - Firefox is right,
Chrome/Safari are wrong.  They do it wrongly because that's how
CoreGraphics, their graphics library, does things natively.

~TJ

[whatwg] Canvas: clarification of compositing operations needed

[David Flanagan]

Firefox and Chrome disagree about the implementation of the 
destination-atop, source-in, destination-in, and source-out compositing 
operators.  Test code is attached.


Chrome doesn't touch any destination pixels that are not underneath the 
source pixels.  Firefox, on the other hand, treats the entire canvas 
(inside the clipping region) as the destination and if you use the 
destination-in operator, for example, will erase any pixels outside of 
whatever you are drawing.


I suspect, based on the reference to an "infinite transparent black 
bitmap" in 4.8.11.1.13 Drawing model that Firefox gets this right and 
Chrome gets it wrong, but it would be nice to have that confirmed.


I suggest clarifying 4.8.11.1.3 Compositing to mention that the 
compositing operation takes place on all pixels within the clipping 
region, and that some compositing operators clear large portions of the 
canvas.


David