Bill Spitzak <spit...@gmail.com> writes:
>> - source: 0xff (8 bits)
>> - source alpha: 0x01 (8 bits)
>> - mask alpha: 0x7b (8 bits)
>> - dest: 0x00 (8 bits)
>> - dest alpha: 0xff (8 bits)
>>
>> When (src IN mask) is computed in 8 bits, the resulting alpha channel
>> is 0 due to rounding:
>>
>> floor ((0x01 * 0x7b) / 255.0 + 0.5) = floor (0.9823) = 0
>>
>> which means that since Render defines any division by zero as
>> infinity, the Fa and Fb for this operator end up as follows:
>>
>> Fa = max (1 - (1 - 1) / 0, 0) = 0
>>
>> Fb = min (1, (1 - 0) / 1) = 1
>>
>> and so since dest is 0x00, the overall result is 0.
>>
>> However, when computed in full precision, the alpha value no longer
>> rounds to 0, and so Fa ends up being
>>
>> Fa = max (1 - (1 - 1) / 0.0001, 0) = 1
>>
>> and so the result is now
>>
>> s * ma * Fa + d * Fb
>>
>> = (1.0 * (0x7b / 255.0) * 1) + d * 0
>>
>> = 0x7b / 255.0
>>
>> = 0.4823
>
> This example could be fixed by treating 0/0 as returning 0.
That doesn't work in general though. For example CONJOINT_ATOP which has
Fb = 1 - sa/da
Suppose sa is zero when computed as an integer, but non-zero when
computed as floating point. If da is zero, then Fb would be 1 with
integers, but 0 with floating point.
Søren
_______________________________________________
Pixman mailing list
Pixman@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/pixman
