[matplotlib-devel] bug in path simplification
Mike,
In looking into the handling of inf and nan, I think I have found some
complexities and inefficiencies that are easily eliminated (and I have
committed some such changes; others are pending), but in the process I
have also found what I am fairly sure is a bug in the path
simplification code. It is illustrated by the attached modification of
nan_test.py. With 128 or more points in the data set, so that the
simplification is invoked, the moveto command that should jump across
the gap is getting changed to a lineto. This can be seen most easily by
applying the attached patch, which includes additional debugging
statements to pin down the incorrect command yielded by the
simplification, as well as pending changes to unify the handling of
masked arrays, nans, and infs. The bug shows up with or without this
patch, however. With the patch, it is also triggered by masked_demo.py,
which is how I first found it. (The non-debugging, or substantive,
parts of the patch are included here for your review or discussion as a
separate matter.)
The middle part of the extra debugging output with the patch applied
when running the nan_test.py looks like this:
2 214.726000 395.178372
3 return cmd: 2
2 218.012000 387.824331
4 skip: 2 218.012000 387.824331
1 359.31 396.688044
3 return cmd: 2
2 362.596000 403.422341
3 return cmd: 2
The line starting with "1" is the moveto command and coordinates yielded
by the c++ path iterator; the following line is showing that the
corresponding command yielded by the simplification code is instead "2",
and that it is being returned at a location I have called "3". All this
will make sense only when you look at the patched code.
Eric
#!/usr/bin/env python
"""
Example: simple line plots with NaNs inserted.
"""
import matplotlib
#matplotlib.use('pdf')
from pylab import *
import numpy as np
t = arange(0.0, 1.0+0.01, 0.00795)
nt = len(t)
sl1 = slice(nt/2-nt/6, nt/2, 1)
sl2 = slice(nt/2, nt/2+nt/6, 1)
s = cos(2*2*pi*t)
t[sl1] = NaN
t[sl2] = np.inf
#t = np.ma.masked_invalid(t)
subplot(2,1,1)
plot(t, s, 'o', lw=4)
xlabel('time (s)')
ylabel('voltage (mV)')
title('A sine wave with a gap of NaNs between 0.4 and 0.6')
grid(True)
subplot(2,1,2)
t[0] = NaN
t[-1] = NaN
plot(t, s, '-', lw=4)
title('Also with NaN in first and last point')
xlabel('time (s)')
ylabel('more nans')
grid(True)
print 'nt = ', nt
#print t.count()
show()
Index: src/agg_py_path_iterator.h
===
--- src/agg_py_path_iterator.h (revision 6015)
+++ src/agg_py_path_iterator.h (working copy)
@@ -116,7 +116,7 @@
agg::path_cmd_end_poly | agg::path_flags_close
};
-#define DEBUG_SIMPLIFY 0
+#define DEBUG_SIMPLIFY 1
template
class SimplifyPath
@@ -194,6 +194,7 @@
*y = front.y;
#if DEBUG_SIMPLIFY
printf((cmd == agg::path_cmd_move_to) ? "|" : "-");
+fprintf(stderr, "1 return cmd: %d\n", cmd);
#endif
return cmd;
}
@@ -218,6 +219,9 @@
// array on each draw.
while ((cmd = m_source->vertex(x, y)) != agg::path_cmd_stop)
{
+#if DEBUG_SIMPLIFY
+fprintf(stderr, "%d %f %f\n", (int) cmd, *x, *y);
+#endif
// Do any quantization if requested
if (m_quantize && agg::is_vertex(cmd))
{
@@ -236,6 +240,8 @@
#if DEBUG_SIMPLIFY
m_pushed++;
printf("|");
+fprintf(stderr, "2 return cmd: %d\n", cmd);
+
#endif
return agg::path_cmd_move_to;
}
@@ -245,6 +251,8 @@
{
#if DEBUG_SIMPLIFY
m_skipped++;
+fprintf(stderr, "1 skip: %d %f %f\n", (int) cmd, *x, *y);
+
#endif
continue;
}
@@ -262,6 +270,8 @@
m_clipped = true;
#if DEBUG_SIMPLIFY
m_skipped++;
+fprintf(stderr, "2 skip: %d %f %f\n", (int) cmd, *x, *y);
+
#endif
continue;
}
@@ -295,6 +305,8 @@
m_lastWrittenY = m_minY = m_lasty;
#if DEBUG_SIMPLIFY
m_skipped++;
+fprintf(stderr, "3 skip: %d %f %f\n", (int) cmd, *x, *y);
+
#endif
continue;
}
@@ -361,6 +373,8 @@
m_lasty = *y;
#if DEBUG_SIMPLIFY
m_skipped++;
+fprintf(stderr, "4 skip: %d %f %f\n", (int) cmd, *x, *y);
+
#endif
continue;
}
@@ -434,6 +448,8 @@
*y = front.y;
#if DEBUG_SIMPLIFY
printf((cmd == agg::path_cmd_move_to) ? "|" : "-");
+fprintf(stderr, "3 return cmd: %d\n", cmd);
+
#endif
return cmd;
}
Index: lib/matplotlib/path.py
===
--- lib/matplotlib/path.py (revision 6015)
+++ lib/matplotlib/path.py (working copy)
@@ -84,43 +84,22 @@
dimension.
If *codes* is None, *vert
Re: [matplotlib-devel] bug in path simplification
The simplification code was written with the assumption that all of the codes are LINETO. That is, it has no MOVETOs or CURVEs. There is code in backend_agg.h that tries to make sure not to run simplification when this is the case (see should_simplify -- it returns false whenever there is a codes array). This all worked before NaN support was added to the path iterator. However, we now have a case where this limitation of path simplification was not explicitly documented, and is now interacting with a new, more efficient, way to handle skipping that wasn't anticipated. So the easy fix is to turn off simplification when the array contains NaNs (and bonus points if we can cache that so we don't have to run through the list to find NaNs ahead of time). The harder fix is to rewrite the simplification code (which is rather opaque) to handle MOVETOs, or perhaps to handle NaNs directly (whichever is easier). I may not get to this before SciPy, however. Cheers, Mike Eric Firing wrote: > Mike, > > In looking into the handling of inf and nan, I think I have found some > complexities and inefficiencies that are easily eliminated (and I have > committed some such changes; others are pending), but in the process I > have also found what I am fairly sure is a bug in the path > simplification code. It is illustrated by the attached modification > of nan_test.py. With 128 or more points in the data set, so that the > simplification is invoked, the moveto command that should jump across > the gap is getting changed to a lineto. This can be seen most easily > by applying the attached patch, which includes additional debugging > statements to pin down the incorrect command yielded by the > simplification, as well as pending changes to unify the handling of > masked arrays, nans, and infs. The bug shows up with or without this > patch, however. With the patch, it is also triggered by > masked_demo.py, which is how I first found it. (The non-debugging, or > substantive, parts of the patch are included here for your review or > discussion as a separate matter.) > > The middle part of the extra debugging output with the patch applied > when running the nan_test.py looks like this: > > 2 214.726000 395.178372 > 3 return cmd: 2 > 2 218.012000 387.824331 > 4 skip: 2 218.012000 387.824331 > 1 359.31 396.688044 > 3 return cmd: 2 > 2 362.596000 403.422341 > 3 return cmd: 2 > > The line starting with "1" is the moveto command and coordinates > yielded by the c++ path iterator; the following line is showing that > the corresponding command yielded by the simplification code is > instead "2", and that it is being returned at a location I have called > "3". All this will make sense only when you look at the patched code. > > Eric - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] bug in path simplification
Michael Droettboom wrote: > So the easy fix is to turn off simplification when the array contains > NaNs (and bonus points if we can cache that so we don't have to run > through the list to find NaNs ahead of time). > On further thought, this shouldn't be too difficult -- so I'll go ahead and implement this... > The harder fix is to rewrite the simplification code (which is rather > opaque) to handle MOVETOs, or perhaps to handle NaNs directly (whichever > is easier). > ...and defer this until I have a good long chunk of time. Cheers, Mike - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] bug in path simplification
Simplification is now turned off whenever there are nonfinite elements in the vertices array. The "should_simplify" determination is now made in Python (to make it easier to tweak and cache). I also committed your patch to handle masked arrays in the same way as arrays-with-nonfinite values (which IMHO is rather elegant -- it gets rid of the "more than one way to do it" problem, and should be faster all around.) Now -- anyone want to improve the simplification algorithm? Cheers, Mike Eric Firing wrote: > Mike, > > In looking into the handling of inf and nan, I think I have found some > complexities and inefficiencies that are easily eliminated (and I have > committed some such changes; others are pending), but in the process I > have also found what I am fairly sure is a bug in the path > simplification code. It is illustrated by the attached modification > of nan_test.py. With 128 or more points in the data set, so that the > simplification is invoked, the moveto command that should jump across > the gap is getting changed to a lineto. This can be seen most easily > by applying the attached patch, which includes additional debugging > statements to pin down the incorrect command yielded by the > simplification, as well as pending changes to unify the handling of > masked arrays, nans, and infs. The bug shows up with or without this > patch, however. With the patch, it is also triggered by > masked_demo.py, which is how I first found it. (The non-debugging, or > substantive, parts of the patch are included here for your review or > discussion as a separate matter.) > > The middle part of the extra debugging output with the patch applied > when running the nan_test.py looks like this: > > 2 214.726000 395.178372 > 3 return cmd: 2 > 2 218.012000 387.824331 > 4 skip: 2 218.012000 387.824331 > 1 359.31 396.688044 > 3 return cmd: 2 > 2 362.596000 403.422341 > 3 return cmd: 2 > > The line starting with "1" is the moveto command and coordinates > yielded by the c++ path iterator; the following line is showing that > the corresponding command yielded by the simplification code is > instead "2", and that it is being returned at a location I have called > "3". All this will make sense only when you look at the patched code. > > Eric - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] irregularly spaced grids and imshow....
On Fri, 2008-08-08 at 16:05 +0200, Grégory Lielens wrote:
> Hello everybody,
>
> I have sent this message to the user group, but thinking of it, it may be more
> relevant to the development mailing list...so here it is again.
>
>
>
> We are looking for the best way to plot a waterfall diagram in
> Matplotlib. The 2 functions which could be used
> to do that are (as far as I have found) imshow and pcolormesh. Here is a
> small script that use both to compare the output:
>
> -
>
> from pylab import *
>
>
> delta = 0.2
> x = arange(-3.0, 3.0, delta)
> y = arange(-2.0, 2.0, delta)
> X, Y = meshgrid(x, y)
> Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
> Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
> # difference of Gaussians
> Z = 10.0 * (Z2 - Z1)
> figure(1)
> im = imshow(Z,extent=(-3,3,-2,2))
> CS = contour(X, -Y, Z, 6,
> colors='k', # negative contours will be dashed by default
> )
> clabel(CS, fontsize=9, inline=1)
> title('Using imshow')
> figure(2)
> im = pcolormesh(X,-Y,Z)
> CS = contour(X, -Y, Z, 6,
> colors='k', # negative contours will be dashed by default
> )
> clabel(CS, fontsize=9, inline=1)
> title('Using pcolormesh')
> show()
>
> -
>
>
> The problem is that we need some of the flexibility of pcolormesh (which
> is able to map the matrix of value on any deformed mesh), while
> we would like to use the interpolations available in imshow (which
> explain why the imshow version is much "smoother" than the pcolormesh
> one).
>
> In fact, what would be needed is not the full flexibility of pcolormesh
> (which can map the grid to any kind of shape), we "only" have to deal
> with rectangular grids where x- and y- graduations are irregularly spaced.
>
> Is there a drawing function in Matplotlib which would be able to work
> with such a rectangular non-uniform grid?
> And if not (and a quick look at the example and the code make me think
> that indeed the capability is currently not present),
> what about an extension of imshow which would work as this:
>
> im = imshow(Z,x_gridpos=x, y_gridpos=y) #specify the
> position of the grid's nodes, instead of giving the extend and assuming
> uniform spacing.
>
> Longer term, would a pcolormesh accepting interpolation be possible? The
> current behavior, averaging the color of the grids node to get a uniform
> cell color,
> is quite rough except for a large number of cells...And even then, it
> soon shows when you zoom in...
>
> The best would be to allow the same interpolations as in imshow (or a
> subset of it), and also allows to use interpolation before colormap
> lookup (or after),
> like in Matlab. Indeed, Matlab allows to finely tune interpolation by
> specifying Gouraud (interpolation after color
> lookup)/Phong(interpolation before color lookup, i.e. for each pixel).
> Phong is usually much better but also more CPU intensive. Phong is
> especially when using discrete colormap, producing banded colors
> equivalent to countour lines, while Gouraud does not work in those
> cases.
>
> Of course, the performance will be impacted by some of those
> interpolation options, which would degrade performance in animations for
> example but I think that having the different options available
> would be very useful, it allows to have the highest map quality, or have
> a "quick and dirty" map depending on situation (grid spacing, type of
> map, animation or not, ...).
>
> Best regards,
>
> Greg.
I have found a method which implement the proposed extension to imshow:
NonUniformImage...
However, this image instance support only nearest neighbor
interpolation. Trying to set the interpolation (using the
set_interpolation method)
to something akin imshow throw a "NotImplementedError: Only nearest
neighbor supported" exception
So basically I am still stuck, it seems that currently there is no way
in matplotlib to plot interpolated
colormap on irregular rectangular grid, and even less on arbitrarily
mapped grid...
Is there any plans to add support for more interpolation in
NonUniformImage in the future? Or maybe there is another
drawing function that I did not find yet, with this ability?
Best regards,
Greg.
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] Gcc, Solaris and putchar() problems
Michael, I'm going to see if there is a fixincludes route to fixing this, but in case there isn't, is adding an additional name to the TTStreamWriter, eg. TTStreamWriter::put_char, out of the question? Thanks, -Peter On Thu, Jul 31, 2008 at 09:08:18AM -0400, Michael Droettboom wrote: > (Sorry for the delay, just back from vacation). > > I don't have access to that version of gcc on Solaris to test. However, > if you find a preprocessor workaround that works for you, I'm happy to > test it on Linux/Mac/Windows and commit. > > Cheers, > Mike > > Peter C. Norton wrote: > >On solaris with gcc-4.3.1, the macro __cplusplus is not defined to > >199711L. This is a long-standing issue with solaris. This doesn't much > >matter on some platforms, but for matplotlib, what is happening to me > >is that through a series of defines and conditionals, the method > >putchar, as used here: > > > >./ttconv/ttutil.cpp:void TTStreamWriter::putchar(int val) > > > >gets tokenized by the preprocessor into putc by > >/usr/include/iso/stdio_iso.h, here on line 350: > > > >#define putchar(x) putc((x), stdout) > > > >This only happens if __cplusplus is < 199711L. I'm avoiding this by > >building using the studio C compiler, but I thought I'd mention this > >in case it's possible to detect this during the configuration stage, > >and perhaps wrap with something like: > > > >#ifdef > >#if __cplusplus <= 199711L > >#undef putchar > >#endif > >#endif > > > >However, I haven't tested this yet. Has anyone else thought about > >this? > > > >Thanks, > > > >-Peter > > > > > > > > > > > > -- > Michael Droettboom > Science Software Branch > Operations and Engineering Division > Space Telescope Science Institute > Operated by AURA for NASA > -- The 5 year plan: In five years we'll make up another plan. Or just re-use this one. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
