Re: [matplotlib-devel] default float format in mlab.rec2csv
OK, I added unit/mlab_unit.py to svn. This checks that double precision floats can round-trip through rec2csv and csv2rec. To pass requires not only svn matplotlib (due to John's change), but also svn numpy (to be included with 1.0.5). Also, it doesn't seem to me that rec2csv should close a file handle passed to it. This prevents use with StringIO, for example. So, I added a test for that, too. John, if it's not going to break anything for you, I'll go ahead and fix that. John Hunter wrote: > On Dec 15, 2007 3:13 PM, Andrew Straw <[EMAIL PROTECTED]> wrote: > >> mlab.defaultformatd sets the default float formatter for rec2csv() to be >> something that doesn't keep the full representation of a floating point >> number. Obviously, I can pass in my own formatd argument to rec2csv(), >> but I wonder if there's any reason why defaultformatd shouldn't use >> repr() for numpy.float32 and numpy.float64? >> > > I'll be happy to make repr the default. I was using %g because I > mistakenly though this provided the appropriate number of significant > digits. I changed this to %r in the csvformat_factory in svn. > > JDJ > > - > SF.Net email is sponsored by: > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services > for just about anything Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace > ___ > Matplotlib-devel mailing list > Matplotlib-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > - SF.Net email is sponsored by: Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] default float format in mlab.rec2csv
On Dec 16, 2007 1:33 PM, Andrew Straw <[EMAIL PROTECTED]> wrote: > OK, I added unit/mlab_unit.py to svn. This checks that double precision > floats can round-trip through rec2csv and csv2rec. To pass requires not > only svn matplotlib (due to John's change), but also svn numpy (to be > included with 1.0.5). > > Also, it doesn't seem to me that rec2csv should close a file handle > passed to it. This prevents use with StringIO, for example. So, I added > a test for that, too. John, if it's not going to break anything for you, > I'll go ahead and fix that. Yes, please do. You might want to modify cbook.to_filehandle to optionally return a tuple specifying whether the returned fh is owned by the caller. Something like: fh, owner = cbook.to_filehandle(fname, returnowner=True) and then later if owner: fh.close() JDH - SF.Net email is sponsored by: Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] namespace packages in toolkits?
On Thursday 13 December 2007 2:22:13 pm Robert Kern wrote: > John Hunter wrote: > > Do we need namespace packages in toolkits? I recently added gtktools > > and exceltools to toolkits, and got a very hard to debug error: > > > > In [1]: import matplotlib > > > > In [2]: matplotlib.__file__ > > Out[2]: > > '/home/titan/johnh/dev/lib/python2.4/site-packages/matplotlib/__init__.py > >c' > > > > In [3]: matplotlib.rcParams['axes.axisbelow'] > > Out[3]: False > > > > In [4]: import matplotlib.toolkits.gtktools > > > > In [5]: matplotlib.__file__ > > Out[5]: > > '/opt/app/g++lib6/python-2.4/lib/python2.4/site-packages/matplotlib/__ini > >t__.pyc' > > > > In [6]: matplotlib.rcParams['axes.axisbelow'] > > > > Traceback (most recent call last): > > File "", line 1, in ? > > KeyError: 'axes.axisbelow' > > > > Notice that the matplotlib module which was previously imported got > > changed by the import of the toolkit which declared itself a namespace > > package. Now this may be the result of mpl not using namespace > > packages correctly (eg matplotlib/__init__.py should be empty save for > > the namespace package or something like that) > > Probably. In setuptools 0.7, they can have stuff in them, but all of them > must be identical. How would one learn about this new feature? According to setuptools.txt from the 0.7 svn checkout, the namespace package's __init__.py must still be empty save the namespace declaration. > If you are using an unmodified matplotlib/__init__.py in > the main matplotlib package, then the "if 0:" is preventing the declaration > of the namespace package, and that may be contributing. > > Also, neither matplotlib nor matplotlib.toolkits is named as a > namespace_package in matplotlib's setup.py, and both have to be. Similarly, > basemap (for example) only lists matplotlib.toolkits but not matplotlib. If I understand correctly, even after fixing these two problems, namespace packages would not work since matplotlib/__init__.py contains lots of meaningful code. Based on the remarks at http://mail.python.org/pipermail/distutils-sig/2007-October/008346.html and the setuptools svn commit log, it looks like development on version 0.7 is not a priority and a release in the next year is unlikely. So is it fair to say that for the foreseeable future, matplotlib is not compatible with namespace packages unless we cleaned out __init__.py? Darren - SF.Net email is sponsored by: Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] default float format in mlab.rec2csv
Done in r4748. I added the kwarg "return_opened" to cbook.to_filehandle(). John Hunter wrote: > On Dec 16, 2007 1:33 PM, Andrew Straw <[EMAIL PROTECTED]> wrote: >> OK, I added unit/mlab_unit.py to svn. This checks that double precision >> floats can round-trip through rec2csv and csv2rec. To pass requires not >> only svn matplotlib (due to John's change), but also svn numpy (to be >> included with 1.0.5). >> >> Also, it doesn't seem to me that rec2csv should close a file handle >> passed to it. This prevents use with StringIO, for example. So, I added >> a test for that, too. John, if it's not going to break anything for you, >> I'll go ahead and fix that. > > Yes, please do. You might want to modify cbook.to_filehandle to > optionally return a tuple specifying whether the returned fh is owned > by the caller. Something like: > > fh, owner = cbook.to_filehandle(fname, returnowner=True) > > and then later > > if owner: > fh.close() > > JDH > > - > SF.Net email is sponsored by: > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services > for just about anything Open Source. > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace > ___ > Matplotlib-devel mailing list > Matplotlib-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel - SF.Net email is sponsored by: Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] [Fwd: Nabble - War on trailing whitespace]
On Wednesday 28 November 2007 1:12:00 pm John Hunter wrote: > On Nov 28, 2007 11:46 AM, Eric Firing <[EMAIL PROTECTED]> wrote: > > I assume there is also some sort of C and/or C++ mode, and similar hook > > additions are needed for them--correct? Or is there a hierarchy of > > modes, in which case a single hook could go at a higher level? > > I don't know about a hierarchy, but there is a c++-mode-hook and a > c-mode-hook. I've updated the CODING_GUIDE to advise emacs users to > set these hooks. Soon your nightmare of trailing whitespaces will be > nothing more than a nasty memory For those who use the eric4 IDE, the version 4.1 snapshot that was released today includes support for stripping whitespace when saving, the same as the emacs hook does. Darren - SF.Net email is sponsored by: Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
[matplotlib-devel] funky strings in rec2csv
As a followup to the work on floats, I fixed rec2csv to deal with funky strings (strings with commas and quotes). I've checked this is as r4749, but I thought I'd announce here in case someone (John, in particular) was depending on some peculiar aspect of the old implementation. If there is something you depend on, can you update the unittests in unit/mlab_unit.py to check for the behavior you need? Also, John, I didn't see mlab.FormatString being used anywhere, but I didn't want to change it, either. So I made FormatString2 and used it. But if there's no known use of FormatString, lets kill the original and more FormatString2 into its place. -Andrew - SF.Net email is sponsored by: Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] funky strings in rec2csv
On Dec 16, 2007 5:26 PM, Andrew Straw <[EMAIL PROTECTED]> wrote: > As a followup to the work on floats, I fixed rec2csv to deal with funky > strings (strings with commas and quotes). I've checked this is as r4749, > but I thought I'd announce here in case someone (John, in particular) > was depending on some peculiar aspect of the old implementation. If > there is something you depend on, can you update the unittests in > unit/mlab_unit.py to check for the behavior you need? > > Also, John, I didn't see mlab.FormatString being used anywhere, but I > didn't want to change it, either. So I made FormatString2 and used it. > But if there's no known use of FormatString, lets kill the original and > more FormatString2 into its place. I am happy to leave it as is, and fix it if I bump into anything. There was a use case at work that made me quote all the strings with " but I am not sure what it is right now so I will test with your version tomorrow and see if I can find any problems. There is one more thing I'd like to clean up in this code, and that is the handling of date and datetime. The current implementation will return datetime regardless of whether the string looks like a date or a datetime. Thus it would fail a roundtrip tests if you had a date field. I may make some changes tomorrow to support date or datetime. What do you think of making checkrows=0 the default for csv2rec? It is slower, but mostly guaranteed to do the right thing, whereas checkrows=5 which is the current default can often fail if the first 5 rows match some type but later rows do not, and may lead to confusion among new users. JDH - SF.Net email is sponsored by: Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] funky strings in rec2csv
On Dec 16, 2007 5:26 PM, Andrew Straw <[EMAIL PROTECTED]> wrote: > As a followup to the work on floats, I fixed rec2csv to deal with funky > strings (strings with commas and quotes). I've checked this is as r4749, > but I thought I'd announce here in case someone (John, in particular) > was depending on some peculiar aspect of the old implementation. If > there is something you depend on, can you update the unittests in > unit/mlab_unit.py to check for the behavior you need? OK, I found one problem with repr. When adding support for date vs datetime to the csv2rec roundtrip, and adding this to the unit test, I notice that repr is probably not what we want for datetime; str makes more sense here: In [1]: import datetime In [2]: d = datetime.date.today() In [3]: repr(d) Out[3]: 'datetime.date(2007, 12, 16)' In [4]: str(d) Out[4]: '2007-12-16' The latter is more natural in the CSV file, and the repr version is not supported by dateutil, at least not the one we are shipping: In [5]: import dateutil.parser In [6]: dateutil.parser.parse(repr(d)) --- ValueErrorTraceback (most recent call last) /Users/jdhunter/python/svn/matplotlib/examples/ in () ...snip the rest of the traceback... In [7]: dateutil.parser.parse(str(d)) Out[7]: datetime.datetime(2007, 12, 16, 0, 0) So I changed FormatObject to use str, pending further discussion. At least for my common use cases, the only obj types I have in my record arrays are dates and datetimes, and I find this to be a pretty compelling use case since it is the type least likely to be supported by other persistence methods (tostring and pickle both fail or do not behave as expected with datetimes in the recarray). But there is an oddity in the parsing of milliseconds which is causing the updated unit test to fail; the code below illustrates the problem: In [3]: import dateutil.parser In [4]: import datetime In [5]: s = '2007-12-18 22:29:34.924122' In [6]: dateutil.parser.parse(s) Out[6]: datetime.datetime(2007, 12, 18, 22, 29, 34, 924121) Thoughts? JDH - SF.Net email is sponsored by: Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] funky strings in rec2csv
On Dec 16, 2007 9:10 PM, John Hunter <[EMAIL PROTECTED]> wrote: > On Dec 16, 2007 5:26 PM, Andrew Straw <[EMAIL PROTECTED]> wrote: > > > As a followup to the work on floats, I fixed rec2csv to deal with funky > > strings (strings with commas and quotes). I've checked this is as r4749, > > but I thought I'd announce here in case someone (John, in particular) > > was depending on some peculiar aspect of the old implementation. If > > there is something you depend on, can you update the unittests in > > unit/mlab_unit.py to check for the behavior you need? > > > > Also, John, I didn't see mlab.FormatString being used anywhere, but I > > didn't want to change it, either. So I made FormatString2 and used it. > > But if there's no known use of FormatString, lets kill the original and > > more FormatString2 into its place. > > I am happy to leave it as is, and fix it if I bump into anything. > There was a use case at work that made me quote all the strings with " > but I am not sure what it is right now so I will test with your > version tomorrow and see if I can find any problems. > > There is one more thing I'd like to clean up in this code, and that is > the handling of date and datetime. The current implementation will > return datetime regardless of whether the string looks like a date or > a datetime. Thus it would fail a roundtrip tests if you had a date > field. I may make some changes tomorrow to support date or datetime. I'm CCing Chris Burns, from Berkeley here, but it would be great if you post a note of these changes on the numpy list. One of the things to come out of the sprint was a lot of work on numpy i/o, and that includes looking into integrating much of the MPL facilities. So it would be great if you keep the numpy team posted, to make sure they get your latest code. Cheers. > > What do you think of making checkrows=0 the default for csv2rec? It > is slower, but mostly guaranteed to do the right thing, whereas > checkrows=5 which is the current default can often fail if the first 5 > rows match some type but later rows do not, and may lead to confusion > among new users. - SF.Net email is sponsored by: Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
