Re: [Matplotlib-users] plotting with missing data?
Giorgio F. Gilestro wrote: import numpy as np a = ['','','',1.1,2.2] mask_a = [i == '' for i in a] b = np.ma.MaskedArray(a, mask=mask_a) Not very efficient, though, is it? cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting with missing data?
Pierre GM wrote: Could you send me an example of the kind of data you're using ? It's basically performance and volume data for a high-volume website. Unfortunately, the data is gappy in places due to data collection errors in the past... (it's important the gaps are shown, rather than trying to interpolate them away, however) As it seems you're dealing with series indexed in time, you may want to try scikits.timeseries, a package Matt Knox and myself implemented for that very reason. How would this help me here and where can I find out about it? cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting with missing data?
import numpy as np a = ['','','',1.1,2.2] mask_a = [i == '' for i in a] b = np.ma.MaskedArray(a, mask=mask_a) Chris Withers wrote: Eric Firing wrote: Chris, Use masked arrays. See masked_demo.py in the mpl examples subdirectory. Hi Eric, I took a look at that, but it uses: import matplotlib.numerix.npyma as ma ...and matplotlib.numerix isn't listed in the API reference. Where are the docs for this? Specifically, what I have is an array like so: ['','','',1.1,2.2] I want to mask the strings out so I don't get ValueErrors raised when I call plot functions with that array. How should I do that? cheers, Chris -- [EMAIL PROTECTED] http://www.cafelamarck.it - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting with missing data?
Eric Firing wrote: Specifically, what I have is an array like so: ['','','',1.1,2.2] Try something like this: import numpy.ma as ma from pylab import * aa = [3.4, 2.5, '','','',1.1,2.2] def to_num(arg): if arg == '': return .0 return arg aanum = array([to_num(arg) for arg in aa]) aamasked = ma.masked_where(aanum==.0, aanum) plot(aamasked) show() What I ended up doing was getting my array to look like: from numpy import nan aa = [3.4,2.5,nan,nan,nan,1.1,2.2] values = numpy.array(aa) values = numpy.ma.masked_equal(values,nan) I only wish that masked_equal didn't blow up when aa contains datetime objects :-( cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting with missing data?
Chris Withers wrote: Eric Firing wrote: Specifically, what I have is an array like so: ['','','',1.1,2.2] Try something like this: import numpy.ma as ma from pylab import * aa = [3.4, 2.5, '','','',1.1,2.2] def to_num(arg): if arg == '': return .0 return arg aanum = array([to_num(arg) for arg in aa]) aamasked = ma.masked_where(aanum==.0, aanum) plot(aamasked) show() What I ended up doing was getting my array to look like: from numpy import nan aa = [3.4,2.5,nan,nan,nan,1.1,2.2] values = numpy.array(aa) values = numpy.ma.masked_equal(values,nan) This is not doing what you think it is, because any logical operation with a Nan returns False: In [4]:nan == nan Out[4]:False You should use numpy.masked_where(numpy.isnan(aa), aa). In some places in mpl, nans are treated as missing values, but this is not uniformly true, so it is better not to count on it. Your values array is not actually getting masked at the nans: In [7]:aa = array([1,nan,2]) In [8]:aa Out[8]:array([ 1., NaN, 2.]) In [9]:values = ma.masked_equal(aa, nan) In [10]:values Out[10]: masked_array(data = [1.0 nan 2.0], mask = [False False False], fill_value=1e+20) Eric I only wish that masked_equal didn't blow up when aa contains datetime objects :-( cheers, Chris - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting with missing data?
On Tuesday 18 March 2008 16:17:08 Eric Firing wrote: Chris Withers wrote: Eric Firing wrote: You should use numpy.masked_where(numpy.isnan(aa), aa). or use masked_invalid directly (shortcut to masked_where((isnan(aa) | isinf(aa)) I only wish that masked_equal didn't blow up when aa contains datetime objects :-( Could you send me an example of the kind of data you're using ? As it seems you're dealing with series indexed in time, you may want to try scikits.timeseries, a package Matt Knox and myself implemented for that very reason. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting with missing data?
Pierre GM wrote: On Tuesday 18 March 2008 16:17:08 Eric Firing wrote: Chris Withers wrote: Eric Firing wrote: You should use numpy.masked_where(numpy.isnan(aa), aa). (I meant numpy.ma.masked_where(...)) or use masked_invalid directly (shortcut to masked_where((isnan(aa) | isinf(aa)) I don't see it in numpy.ma, with numpy from svn. In any case, the fastest method is masked_where(~numpy.isfinite(aa), aa): In [1]:import numpy In [2]:xx = numpy.random.rand(1) In [3]:xx[xx0.8] = numpy.nan In [6]:timeit numpy.ma.masked_where(~numpy.isfinite(xx), xx) 1 loops, best of 3: 83.9 µs per loop In [7]:timeit numpy.ma.masked_where(numpy.isnan(xx), xx) 1 loops, best of 3: 119 µs per loop In [9]:timeit numpy.ma.masked_where((numpy.isnan(xx)|numpy.isinf(xx)), xx) 1000 loops, best of 3: 260 µs per loop So, wherever you do have masked_invalid defined, you might want to use the faster implementation with ~isfinite. Eric I only wish that masked_equal didn't blow up when aa contains datetime objects :-( Could you send me an example of the kind of data you're using ? As it seems you're dealing with series indexed in time, you may want to try scikits.timeseries, a package Matt Knox and myself implemented for that very reason. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting with missing data?
Eric Firing wrote: Chris, Use masked arrays. See masked_demo.py in the mpl examples subdirectory. Hi Eric, I took a look at that, but it uses: import matplotlib.numerix.npyma as ma ...and matplotlib.numerix isn't listed in the API reference. Where are the docs for this? Specifically, what I have is an array like so: ['','','',1.1,2.2] I want to mask the strings out so I don't get ValueErrors raised when I call plot functions with that array. How should I do that? cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting with missing data?
Chris Withers wrote: Eric Firing wrote: Chris, Use masked arrays. See masked_demo.py in the mpl examples subdirectory. Hi Eric, I took a look at that, but it uses: import matplotlib.numerix.npyma as ma ...and matplotlib.numerix isn't listed in the API reference. Where are the docs for this? numerix is obsolete, and numerix.npyma was a temporary method to provide access to either of two masked array implementations. It is probably time for me to remove it from the examples. Substitute import numpy.ma as ma The ma module is documented as part of numpy. Specifically, what I have is an array like so: ['','','',1.1,2.2] Try something like this: import numpy.ma as ma from pylab import * aa = [3.4, 2.5, '','','',1.1,2.2] def to_num(arg): if arg == '': return .0 return arg aanum = array([to_num(arg) for arg in aa]) aamasked = ma.masked_where(aanum==.0, aanum) plot(aamasked) show() Eric I want to mask the strings out so I don't get ValueErrors raised when I call plot functions with that array. How should I do that? cheers, Chris - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
[Matplotlib-users] plotting with missing data?
Hi All, Say I have data that looks like: date x y z 2008-01-01 10 2008-01-02 21 11 2008-01-02 32 15 5 How can I plot it such that all three lines are plotted by that it's apparent two of them are missing some data? (I know I could just sub in zeros for the missing values, but I'd like the point not to be there, not just down the bottom of the graph...) cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] plotting with missing data?
Chris, Use masked arrays. See masked_demo.py in the mpl examples subdirectory. Eric Chris Withers wrote: Hi All, Say I have data that looks like: date x y z 2008-01-01 10 2008-01-02 21 11 2008-01-02 32 15 5 How can I plot it such that all three lines are plotted by that it's apparent two of them are missing some data? (I know I could just sub in zeros for the missing values, but I'd like the point not to be there, not just down the bottom of the graph...) cheers, Chris - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
