Thanks for looking into this deeper.
Agg requires image buffers to be premultiplied, as described in the
third bullet point here. (It's not exactly clear, to say the least, but
that's what I take it to mean, and also from reading the code).
http://www.antigrain.com/news/release_notes/v22.agdoc.html
The bug is that in _image.cpp the input buffers are not declared as
premultiplied in _image.cpp. Arguably it is a bug that agg doesn't
reject filtering unmultiplied images, since the note states that the
assumption is that they are premultiplied by the time they get to the
filters.
I have attached a patch that fixes this. Would you mind testing it and
let me know how it works for you?
On 10/20/2011 10:29 PM, Daniel Hyams wrote:
I've looked all over the place through both the Python and C code, and
I don't see any premultiplication of alphas at any stage before the
pixels are passed off to agg, and neither can I find any place where
the alphas are "unmultiplied" on the way back from agg to the backend
for rendering.
matplotlib's support for alpha blending of images is basically by
accident, so it's not surprising the details aren't right.
I think it is a bug that after reading images in we don't premultiply
them before sending them to Agg. That bug has existed for a long time
in matplotlib because no one is really using alpha images a great deal.
(Masked images, yes, but that implies alpha is strictly 0 or 255 and
thus these issues don't come into play.)
Unmultiplying is not always necessary. Many of the GUI backends also
expect premultiplied alpha (Qt for example). However, there is
certainly a bug in writing PNG files (where the file format specifies
unmultiplied).
It's very possible that I missed it, but I would have to miss it in
two places (premultiply and the unmultiply). It looks to me like the
output from agg ends up getting passed on directly to the renderer,
which as far as I know, just uses straight alpha. The WxAgg renderer,
for example, just creates a wx.Bitmap out of the pixels and blits it.
Which means that any image going through agg's filters will not be
correct if it has any pixels with alpha != 0 or != 255.
[Using PIL images because they are simple to talk about...but the PIL
image could alternatively be an image.py image]
As far as I can tell, the image pixels current go through a pipeline
like the following:
[1] PIL Image -> _image image -> agg operations -> modified and/or
resized _image image -> renderer
If agg expects premultiplied alpha, the procedure should look something like:
[2] PIL Image -> _image image -> premultiply alphas ->agg options ->
unmultiply alphas -> modified and/or resized _image image -> renderer
I personally don't like pipeline [2] because picture detail is lost in
the "unmultiply alphas" stage. Better to use straight alpha all the
way through.
I think what needed is:
[3] PIL Image (or _png.cpp) -> premultiply alphas -> _image image -> agg
options ->
-> modified and/or resized _image image -> renderer -> (unmultiply alphas)?
-> GUI library
That is -- all image data should be kept premultiplied internally in all
buffers for efficiency and because this is what Agg is designed for.
Can you explain what you mean by "picture detail is lost in the
unmultiply alphas stage". There is the usual problem that by
premultiplying you lose any color data where alpha = 0 (and you lose
resolution everywhere else, but not resolution you can actually see
after compositing).
So long as matplotlib is using only a subset of agg algorithms that
work no matter whether the alphas are premultiplied or not, I would
think that the most reasonable route was the one that I took; to
always pass straight alphas (sticking with pipeline [1]), and modify
the agg source slightly to fit matplotlib's approach (i.e., remove the
clipping there).
I'd be really wary of modifying agg like this. Those things become hard
to maintain. I think this instead a bug in matplotlib and should be
fixed there.
I've put an issue in the issue tracker here:
https://github.com/matplotlib/matplotlib/issues/545
Cheers,
Mike
I hope that I'm not way off base (I have a sneaking feeling that I am
:O ), and hope this helps. I've verified on both Linux and Windows
that removing the alpha-clip lines from agg_span_image_filter_rgba.h,
rebuilding matplotlib, and replacing _image.so/_image.pyd and
_backend_agg.so/_backend_agg.pyd does the trick (along with passing
straight alphas). So far, I've seen no ill effects on any of my
plots, but I'm also not in a position to run the pixel-by-pixel
comparison matplotlib tests.
On Wed, Oct 19, 2011 at 7:26 PM, Daniel Hyams<dhy...@gmail.com> wrote:
There has to be something else in play here. I'll try to keep this
short, but the summary is this: I can get the transparency to look
right, but only if 1) I put "straight" alpha in image, not
premultiplied, and 2) I hack agg to remove specificially every
instance of the lines of code that you refer to above.
Why this is, I don't know. Hopefully I'm still misusing something.
However, it behaves as if the clipping of alpha in the agg library is
corrupting the alpha channel. I also submit that I could have broken
some other transparency capabilities of matplotlib, because I don't
know what other routines use what I hacked....I did check a few
transparent polygons and such though, and everything seemed to be
fine.
I know that the agg library has been around for quite a long time, so
that also means that such a basic bug is unlikely.
I've reattached the (slightly modified) script that reproduces the
problem, along with a sample image that it uses. The only change to
the script is right at the top, where a different image is read, a
quick statement is placed to add an alpha channel if there is not
already one, and I'm attempting to use premultiplied alphas. I've
also attached a screenshot of the output. Notice that in this case,
both "transparent" images look wrong.
Now, if I 1) hack agg to remove the alpha clipping, and 2) modify the
one line in the attached python script so that I use straight alpha,
everything looks right. Specifically, I removed every instance of the
code below from xxxx, rebuilt all of the matplotlib .so's, and
specifically replaced _image.so and _backend_agg.so in my matplotlib
distribution.
if(fg[order_type::A]> base_mask) fg[order_type::A]
= base_mask;
if(fg[order_type::R]> fg[order_type::A])
fg[order_type::R] = fg[order_type::A];
if(fg[order_type::G]> fg[order_type::A])
fg[order_type::G] = fg[order_type::A];
if(fg[order_type::B]> fg[order_type::A])
fg[order_type::B] = fg[order_type::A];
On Wed, Oct 19, 2011 at 2:34 PM, Daniel Hyams<dhy...@gmail.com> wrote:
Ah, thanks so much Michael! That explanation helps a great deal; I
was always considering things in "straight alpha" format, not even
knowing that there was alternative.
I'll play with this tonight; I don't see any problem getting the thing
working, though, now that I know what agg expects to see...
And yes, alpha support in the image class would be very helpful ;)
On Wed, Oct 19, 2011 at 2:16 PM, Michael Droettboom<md...@stsci.edu> wrote:
You are right that Agg is doing the resizing here. Agg expects
premultiplied alpha. See [1] for information about what that means.
[1] http://en.wikipedia.org/wiki/Alpha_compositing
After Agg interpolates the pixel values, to prevent oversaturation it
truncates all values to be less than alpha (which makes sense if everything
is assumed to be premultiplied alpha). Arguably, the bug here is that
nearest neighbor (which doesn't have to do any blending) doesn't perform the
truncation step -- then both would look "wrong".
It happens in this code snippet in span_image_filter_rgba: (base_mask is
255)
if(fg[order_type::A]> base_mask) fg[order_type::A]
= base_mask;
if(fg[order_type::R]> fg[order_type::A]) fg[order_type::R]
= fg[order_type::A];
if(fg[order_type::G]> fg[order_type::A]) fg[order_type::G]
= fg[order_type::A];
if(fg[order_type::B]> fg[order_type::A]) fg[order_type::B]
= fg[order_type::A];
So, the solution to make a partially transparent image is to not do:
pix[:,:,3] = 127
but instead, do
pix *= 0.5
Of course, the real fix here is to support alpha blending properly in the
image class, then the user wouldn't have to deal with such details. A bug
should probably be filed in the matplotlib issue tracker for this.
Mike
On 10/19/2011 12:23 PM, Daniel Hyams wrote:
[Sorry, I keep getting tripped up with HTML mail....resent in ascii,
and resaved one of the attachment png's to make it smaller.]
Example script attached (PIL required). Basically, if I impose a
specific value into an image's alpha channel and use any interpolation
scheme other than 'nearest', there appears gray all where the figure
didn't have any color to begin with. I've also attached a screenshot
of the output of the script on my machine.
Hopefully I'm doing something wrongly?
I chased the problem and managed to hack in a solution that fixes the
problem, but it's extremely inefficient...basically, in matplotlib's
image.py, routine BboxImage.make_image, you can create two images
there....one with no alpha channel (call it imRGB) and one with (call
it imRGBA). Go through all of the routine, doing exactly the same
things to both of the images *except* for the interpolation, which is
set to 'nearest' for imRGBA. Then, rip the colors out of imRGB, the
alpha channel off of imRGBA, and put them together....go through all
of the routine again with this composited image, and it works. I
know...I told you it was bad ;)
The problem seems to be in the "resize" call in that routine...resize,
which calls into C code, does not appear to handle things correctly
when the alpha is anything other than 255's across the board. It
might be a problem in the agg routines, but hopefully it is just maybe
a misuse of the agg routines.
The behavior seems to be backend independent as far as I could test (I
tried with wxagg and tk backends). I am using mpl 1.0.0 on Windows if
it matters.
--
Daniel Hyams
dhy...@gmail.com
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Ciosco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
--
Daniel Hyams
dhy...@gmail.com
--
Daniel Hyams
dhy...@gmail.com
diff --git a/src/_image.cpp b/src/_image.cpp
index 3278b6c..6bb202a 100644
--- a/src/_image.cpp
+++ b/src/_image.cpp
@@ -33,7 +33,7 @@
#include "mplutils.h"
-typedef agg::pixfmt_rgba32 pixfmt;
+typedef agg::pixfmt_rgba32_pre pixfmt;
typedef agg::renderer_base<pixfmt> renderer_base;
typedef agg::span_interpolator_linear<> interpolator_type;
typedef agg::rasterizer_scanline_aa<agg::rasterizer_sl_clip_dbl> rasterizer;
------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
