Re: [Numpy-discussion] Numpy on Python3
Pauli Virtanen skrev: > XXX: 3K: numpy.random is disabled for now, uses PyString_* > XXX: 3K: numpy.ma is disabled for now -- some issues > I thought numpy.random uses Cython? Is it just a matter of recompiling the pyx-file? > I remember Dag was working on this a bit: how far did it go? > > Cython's include file numpy.pxd has an ndarray class that extend PyArrayObject with PEP 3118 buffer compatibility. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy on Python3
On Nov 23, 2009, at 12:35 AM, Pauli Virtanen wrote: > http://github.com/pv/numpy-work/tree/py3k > > $ mkdir -p $PWD/dist/lib/python3.1/site-packages > $ python3 setup.py install --prefix=$PWD/dist > $ cd $PWD/dist/lib/python3.1/site-packages && python3 > Python 3.1.1+ (r311:74480, Oct 11 2009, 20:22:16) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. import numpy > XXX: 3K: numpy.random is disabled for now, uses PyString_* > XXX: 3K: numpy.ma is disabled for now -- some issues What are the issues ? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Correlation filter
On Fri, Nov 20, 2009 at 4:27 PM, wrote: > On Fri, Nov 20, 2009 at 2:28 PM, Keith Goodman wrote: >> On Fri, Nov 20, 2009 at 11:17 AM, wrote: >>> On Fri, Nov 20, 2009 at 1:51 PM, Keith Goodman wrote: def corr3(x, y): x = x - x.mean() x /= x.std() nx = x.size one = np.ones(nx) xy = lfilter(x, 1, y) sy = lfilter(one, 1, y) sy2 = lfilter(one, 1, y*y) d = xy / np.sqrt(nx * sy2 - sy * sy) return d >>> >>> Is this correct? xy uses the original y and not a demeaned y. >> >> I wouldn't be surprised if I made mistakes. But I don't think I need >> to demean y. One way to write the numerator of the correlation >> coefficent is >> >> N*sum(xy) - sum(x)*sum(y) >> >> but sum(x) is zero, so it becomes >> >> N*sum(xy) >> >> So I think I only need to demean x. But I'd better test the code. Plus >> I have no idea how lfilter will handle my missing values (NaN). > > I think nans fully propagate, trick might be to fill with zero and use > another convolution/correlation with the non-nan mask to get the > number of observations included (I think I saw it as a trick by Pierre > for autocorrelation with missing observations). Just an update, I used this for moving correlation between x and y of equal shape, just posted to the scipy-user mailing list. In the test case it works correctly. Josef > > Here is a moving slope function which estimate the slope coefficient > for a linear trend if x is arange() > I compared the ranking with your moving correlation and it is quite different. > > > def moving_slope(x,y) > '''estimate moving slope coefficient of regression of y on x > filters along axis=1, returns valid observations > Todo: axis and lag options > ''' > xx = np.column_stack((np.ones(x.shape), x)) > pinvxx = np.linalg.pinv(xx)[1:,:] > windsize = len(x) > lead = windsize//2 - 1 > return signal.correlate(y, pinvxx, 'full' > )[:,windsize-lead:-(windsize+1*lead-2)] > > > Josef > >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy on Python3
On Sun, Nov 22, 2009 at 10:35 PM, Pauli Virtanen wrote: > http://github.com/pv/numpy-work/tree/py3k > > $ mkdir -p $PWD/dist/lib/python3.1/site-packages > $ python3 setup.py install --prefix=$PWD/dist > $ cd $PWD/dist/lib/python3.1/site-packages && python3 > Python 3.1.1+ (r311:74480, Oct 11 2009, 20:22:16) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import numpy > XXX: 3K: numpy.random is disabled for now, uses PyString_* > XXX: 3K: numpy.ma is disabled for now -- some issues > >>> numpy.array([1., 2, 3, 4]) > array([ 1., 2., 3., 4.]) > >>> _ + 10 > array([ 11., 12., 13., 14.]) > >>> numpy.ones((4,), dtype=complex)/4 > array([ 0.25+0.j, 0.25+0.j, 0.25+0.j, 0.25+0.j]) > >>> numpy.array([object(), object()]) > array([, ], > dtype=b'object') > >*** > > Things were fairly straightforward this far, just many tiny changes. > What's left is then sorting out the bigger problems :) > > This is still far from being complete: > > - Most use of PyString_* needs auditing (note e.g. the b'object' in the > dtype print above...). > > I simply added convenience wrappers for PyString -> PyBytes, > but this is not the correct choice at all points. > > - Also, should dtype='S' be Bytes or Unicode? I chose Bytes for now. > > Yeah, this needs deciding. I think compatibility with old files requires Bytes, we can't change file formats and keep compatibility. BTW, I made some changes for py3k that depend on the macro NPY_PY3K being defined, but didn't actually define it anywhere. It needs to go in one of the include files. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy on Python3
On Mon, Nov 23, 2009 at 2:35 PM, Pauli Virtanen wrote: > It might be nice to have this merged in at some point after 1.4.0 (after > the most obvious glaring bugs have been fixed), so that we could perhaps > start aiming for Python3 compatibility in Numpy 1.5.0. One thing I have on my end is a numpy.distutils which runs under both python 2 and 3, so that you don't have to run 2to3 everytime you make a change. I did not put it in the trunk because I did not want to modify numpy.distutils for 1.4.0 at this point, but I will include the changes as soon as I branch the trunk into 1.4.0, David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Numpy on Python3
On Sun, Nov 22, 2009 at 10:35 PM, Pauli Virtanen wrote: > http://github.com/pv/numpy-work/tree/py3k > > $ mkdir -p $PWD/dist/lib/python3.1/site-packages > $ python3 setup.py install --prefix=$PWD/dist > $ cd $PWD/dist/lib/python3.1/site-packages && python3 > Python 3.1.1+ (r311:74480, Oct 11 2009, 20:22:16) > [GCC 4.4.1] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import numpy > XXX: 3K: numpy.random is disabled for now, uses PyString_* > XXX: 3K: numpy.ma is disabled for now -- some issues > >>> numpy.array([1., 2, 3, 4]) > array([ 1., 2., 3., 4.]) > >>> _ + 10 > array([ 11., 12., 13., 14.]) > >>> numpy.ones((4,), dtype=complex)/4 > array([ 0.25+0.j, 0.25+0.j, 0.25+0.j, 0.25+0.j]) > >>> numpy.array([object(), object()]) > array([, ], > dtype=b'object') > >*** > > Things were fairly straightforward this far, just many tiny changes. > What's left is then sorting out the bigger problems :) > > This is still far from being complete: > > - Most use of PyString_* needs auditing (note e.g. the b'object' in the > dtype print above...). > > I simply added convenience wrappers for PyString -> PyBytes, > but this is not the correct choice at all points. > > - Also, should dtype='S' be Bytes or Unicode? I chose Bytes for now. > > - Whether to inherit Numpy ints from PyLong_* needs some thinking, > as they are quite different objects. Now, I dropped the inheritance, > but I wonder if this will break something. > > Maybe. But it was always a hassle because it behaved differently than the other integer types. Now onto float ;) > - PyFile_AsFile has disappeared, and AsFileDescriptor+fdopen doesn't > seem to cut it -- don't know exactly what's wrong here. > > - Integer -> String formatting does not seem to work > > - Larger-than-long-long Python ints probably cause problems > > We used a python call which would raise an error if the number was too large. If that call is still valid, things should work. > - The new buffer interface needs to be implemented -- currently there > are just error-raising stubs. > > I remember Dag was working on this a bit: how far did it go? > > - Relative imports + 2to3 is a bit of a pain. A pity we can't have > them in the mainline code because of python2.4. > > - I didn't check for semantic changes in tp_* interface functions. > This we need still to do. > > I left some notes in the src folder. If you discover anything new put it in there. > - And probably many other issues lurking. > > We do need to look at the initialization of the type math stuff in the ufuncobject module. Yeah, its a bit of a circular dependency, one reason it would be nice to have ufuncs operate on buffer objects instead of ndarrays would be to break the mutual dependence. > > Also, I didn't yet try checking how far the test suite passes on > Python3. (It still passes completely on Python2, so at least I didn't > break that part.) > > It might be nice to have this merged in at some point after 1.4.0 (after > the most obvious glaring bugs have been fixed), so that we could perhaps > start aiming for Python3 compatibility in Numpy 1.5.0. > > If you want to see real suffering, look at the programmer notes in the hacked CRU files. Some poor sucker was stuck with fixing up the g*dawful code while also needing to determine what data was in undocumented binary files, some with the same names but containing different data. Folks, don't let that happen to you. The conversion to Py3k is going to be a breeze by comparison. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Numpy on Python3
http://github.com/pv/numpy-work/tree/py3k $ mkdir -p $PWD/dist/lib/python3.1/site-packages $ python3 setup.py install --prefix=$PWD/dist $ cd $PWD/dist/lib/python3.1/site-packages && python3 Python 3.1.1+ (r311:74480, Oct 11 2009, 20:22:16) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy XXX: 3K: numpy.random is disabled for now, uses PyString_* XXX: 3K: numpy.ma is disabled for now -- some issues >>> numpy.array([1., 2, 3, 4]) array([ 1., 2., 3., 4.]) >>> _ + 10 array([ 11., 12., 13., 14.]) >>> numpy.ones((4,), dtype=complex)/4 array([ 0.25+0.j, 0.25+0.j, 0.25+0.j, 0.25+0.j]) >>> numpy.array([object(), object()]) array([, ], dtype=b'object') *** Things were fairly straightforward this far, just many tiny changes. What's left is then sorting out the bigger problems :) This is still far from being complete: - Most use of PyString_* needs auditing (note e.g. the b'object' in the dtype print above...). I simply added convenience wrappers for PyString -> PyBytes, but this is not the correct choice at all points. - Also, should dtype='S' be Bytes or Unicode? I chose Bytes for now. - Whether to inherit Numpy ints from PyLong_* needs some thinking, as they are quite different objects. Now, I dropped the inheritance, but I wonder if this will break something. - PyFile_AsFile has disappeared, and AsFileDescriptor+fdopen doesn't seem to cut it -- don't know exactly what's wrong here. - Integer -> String formatting does not seem to work - Larger-than-long-long Python ints probably cause problems - The new buffer interface needs to be implemented -- currently there are just error-raising stubs. I remember Dag was working on this a bit: how far did it go? - Relative imports + 2to3 is a bit of a pain. A pity we can't have them in the mainline code because of python2.4. - I didn't check for semantic changes in tp_* interface functions. This we need still to do. - And probably many other issues lurking. Also, I didn't yet try checking how far the test suite passes on Python3. (It still passes completely on Python2, so at least I didn't break that part.) It might be nice to have this merged in at some point after 1.4.0 (after the most obvious glaring bugs have been fixed), so that we could perhaps start aiming for Python3 compatibility in Numpy 1.5.0. Cheers, Pauli ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] neighborhood iterator
On 22-Nov-09, at 12:50 PM, Nadav Horesh wrote: > > I wonder if the neighbourhood iterator can be used as a more > efficient replacement for ndimage.generic_filter. Is there a way to > use it from cython? Yes, using the NumPy C API, called like any other C function is from Cython. Something like: ## import numpy as np cimport numpy as np cdef extern from "numpy/arrayobject.h": object PyArray_NeighborhoodIterNew(object iter, np.npy_intp bounds, int mode, object, np.ndarray fill_value) int PyArrayNeighborhoodIter_Next(object iter) int PyArrayNeighborhoodIter_Reset(object iter) ## should do the trick. Note that you'll need to call np.import_array() before using any of these functions to initialize the C API, I think. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Installing numpy under cygwin
Hello,
I attempted to install Numpy for my Cygwin python again, by simply executing:
>python setup.py install
However, I now get the following:
>File "numpy/core/setup.py", line 253, in check_mathlib
> raise EnvironmentError("math library missing; rerun "
>EnvironmentError: math library missing; rerun setup.py after setting the
MATHLIB
>env variable
I have a math library from cygwin (libm.a), but I have not managed to set it.
How should I set this?
Thank you for the help,
Olivia
- Original Message
> From: David Cournapeau
> To: Discussion of Numerical Python
> Sent: Wed, November 18, 2009 9:01:06 AM
> Subject: Re: [Numpy-discussion] Installing numpy under cygwin
>
> Olivia Cheronet wrote:
> > Hello.
> >
> > I am currently trying to install the latest version of NumPy for my cygwin
> Python, and I am having problems...
> >
> > I have downloaded the source, and unzipped and untarred it in my home
> directory.
> > Subsequently, I included the following in the site.cfg file:
> >
> >> [DEFAULT]
> >> library_dirs = /cygdrive/c/cygwin/lib
> >> include_dirs = /cygdrive/c/cygwin/usr/local/include
> >> [blas_opt]
> >> libraries = f77blas, cblas, atlas
> >> [lapack_opt]
> >> libraries = lapack, f77blas, cblas, atlas
> >>
> >
> > In the Cygwin bash shell, after going to my home directory, I have
> > executed:
> > python setup.py config --compiler=mingw32 build --compiler=mingw32 install
> > as instructed in the "Installing SciPy/Windows" page.
> >
>
> If you use cygwin, you should not follow the windows instructions. For
> most purposes, cygwin works as unix. In particular, you don't think you
> should not use mingw32 build on cygwin. The -mno-cygwin flag appended to
> gcc is most likely coming from there, and this looks very wrong for a
> cygwin build.
>
> David
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 2to3c: 2to3 for C extensions
su, 2009-11-22 kello 19:28 +0100, René Dudfield kirjoitti: [clip] > I've been working on this too... see the previous py3k thread for > details of my plan. See http://github.com/illume/numpy3k/tree/work > for my (not very much) progress. > > I don't have time for about a week to do any more work on my tree. > Maybe your tree is ahead anyway... I am taking the approach we used > with pygame so that the tree can work with both python 2.x and 3.x. I > got up to getting a fair amount of the setup code working... but there > are still some issues. Ok, I forgot that you did also some work on this. I think my tree is currently a bit more along the way than yours -- it seems to have the same fixes as in yours, and it proceeds until it fails to compile multiarraymodule. The main difference seems to be that I'm using the 2to3 tool to automatically convert the Python code before the actual build -- this could be a major manual work saver. Pauli ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 2to3c: 2to3 for C extensions
On Sun, Nov 22, 2009 at 7:28 PM, René Dudfield wrote: > On Sun, Nov 22, 2009 at 7:16 PM, Pauli Virtanen wrote: >> la, 2009-11-21 kello 20:00 -0500, David Warde-Farley kirjoitti: >>> Guido posted a link to this on Twitter, it might be of interest to >>> people interested in the NumPy Python 3 transition: >>> >>> http://dmalcolm.livejournal.com/3935.html >>> >>> It's doubtful this is a magic bullet (at least, not yet) but it may >>> still help reduce the amount of busywork involved. >> >> Interesting -- though at the moment it doesn't seem to solve very many >> of our problems. >> >> >> I did some cleanup of numpy.testing, distutils, and core so that using >> 2to3 the build_ext in numpy.core can at least be started: >> >> http://github.com/pv/numpy-work/tree/py3k >> >> python3 tools/py3test testing distutils core >> >> Of course, the build currently fails with zillion errors in >> multiarraymodule. >> >> Pauli >> > > hi, > > I've been working on this too... see the previous py3k thread for > details of my plan. See http://github.com/illume/numpy3k/tree/work > for my (not very much) progress. > > I don't have time for about a week to do any more work on my tree. > Maybe your tree is ahead anyway... I am taking the approach we used > with pygame so that the tree can work with both python 2.x and 3.x. I > got up to getting a fair amount of the setup code working... but there > are still some issues. > > The 2to3 script for C code does look useful. However it will not work > in a backwards compatible manner. However maybe it can be modified to > work in such a way. > hi again, I was wrong about it not generating 2 and 3 compatible source... the tool does do things like this: #if PY_MAJOR_VERSION >= 3 foo(); #else bar(); #endif Very cool!!! cheers, ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] failure building trunk with mingw
On Sun, Nov 22, 2009 at 6:28 AM, wrote: > On Sun, Nov 22, 2009 at 2:35 AM, David Cournapeau > wrote: > > On Wed, Nov 18, 2009 at 6:10 PM, David Cournapeau > wrote: > >> On Wed, Nov 18, 2009 at 12:38 AM, wrote: > >> > >>> > >>> Now numpy builds without problems. > >>> When I run the tests I get 16 failures mostly nan related. I have no > idea > >>> whether they are real or if there is still something screwed up in my > >>> setup. See below. > >> > >> I can reproduce those. I did not see those because they are python 2.5 > >> specific (which is puzzling). > > > > I fix most of those, which were related to an atan2 bug in MS C > > library. Could you test whether this fixes it for you ? > > > > David > > ___ > > NumPy-Discussion mailing list > > NumPy-Discussion@scipy.org > > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > Thanks, > > I have 4 failures remaining. full ouput below > > Josef > > >python > Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit > (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import numpy > >>> numpy.test() > Running unit tests for numpy > NumPy version 1.4.0.dev7758 > NumPy is installed in C:\Josef\_progs\Subversion\numpy-trunk\dist\ > numpy-1.4.0.de > v7758.win32\Programs\Python25\Lib\site-packages\numpy > Python version 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit > (Int > el)] > nose version 0.11.1 > > > > > > > > > > > > > > > > > > ...K.FF. > > K..K.F.. > > > > > > > > > > > > .C:\Josef\_progs\Subversion\numpy-trunk\dist > > \numpy-1.4.0.dev7758.win32\Programs\Python25\Lib\site-packages\numpy\lib\io.py:1 > 324: ConversionWarning: Some errors were detected ! >Line #2 (got 4 columns instead of 5) >Line #12 (got 4 columns instead of 5) >Line #22 (got 4 columns instead of 5) >Line #32 (got 4 columns instead of 5) >Line #42 (got 4 columns instead of 5) > warnings.warn(errmsg, ConversionWarning) > > .C:\Josef\_progs\Subversion\numpy-trunk\dist\numpy-1.4.0.dev7758.win32\Programs\ > Python25\Lib\site-packages\numpy\lib\io.py:1324: ConversionWarning: Some > errors > were detected ! >Line #2 (got 4 columns instead of 2) >Line #12 (got 4 columns instead of 2) >Line #22 (got 4 columns instead of 2) >Line #32 (got 4 columns instead of 2) >Line #42 (got 4 columns instead of 2) > warnings.warn(errmsg, ConversionWarning) > > ..KKF... > > > > > > > > > > > > ..S. > > > > > > > > > > > > > ... > == > FAIL: test_umath.test_nextafter >
Re: [Numpy-discussion] 2to3c: 2to3 for C extensions
On Sun, Nov 22, 2009 at 7:16 PM, Pauli Virtanen wrote: > la, 2009-11-21 kello 20:00 -0500, David Warde-Farley kirjoitti: >> Guido posted a link to this on Twitter, it might be of interest to >> people interested in the NumPy Python 3 transition: >> >> http://dmalcolm.livejournal.com/3935.html >> >> It's doubtful this is a magic bullet (at least, not yet) but it may >> still help reduce the amount of busywork involved. > > Interesting -- though at the moment it doesn't seem to solve very many > of our problems. > > > I did some cleanup of numpy.testing, distutils, and core so that using > 2to3 the build_ext in numpy.core can at least be started: > > http://github.com/pv/numpy-work/tree/py3k > > python3 tools/py3test testing distutils core > > Of course, the build currently fails with zillion errors in > multiarraymodule. > > Pauli > hi, I've been working on this too... see the previous py3k thread for details of my plan. See http://github.com/illume/numpy3k/tree/work for my (not very much) progress. I don't have time for about a week to do any more work on my tree. Maybe your tree is ahead anyway... I am taking the approach we used with pygame so that the tree can work with both python 2.x and 3.x. I got up to getting a fair amount of the setup code working... but there are still some issues. The 2to3 script for C code does look useful. However it will not work in a backwards compatible manner. However maybe it can be modified to work in such a way. anyway... I'll look at your tree again next time I have time to work on this in a week or so. cheers! ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] failure building trunk with mingw
On Sun, Nov 22, 2009 at 8:37 AM, wrote: > On Sun, Nov 22, 2009 at 10:25 AM, David Cournapeau > wrote: > > On Mon, Nov 23, 2009 at 12:14 AM, wrote: > >> On Sun, Nov 22, 2009 at 10:01 AM, David Cournapeau > wrote: > >>> On Sun, Nov 22, 2009 at 11:45 PM, Charles R Harris > >>> wrote: > > Might be nice to print out the actual values of np.spacing and > np.nextafter > here. > > >>> > >>> Yes, I should add some utilities to print those for this kind of test. > >>> But in this case, I know the problem: mingw gcc use 80 bits long > >>> double but it uses the MS runtime which considers double and long > >>> double to be the same (8 bytes). > >>> > >>> I think the real fix is to force npy_longdouble to be double on mingw, > >>> but I don't want to make that change now for 1.4.0. > >> > >> adding the failing type in the test to the failure message would also > >> be helpful > > > > Yes, you're right. I also used the nose facility for using generators > > for complex corner cases, but with retrospect, it is not so useful, > > because you don't get a name when you have a failure (or maybe I am > > using it wrong). > > I don't know what the policy for the use of assert in numpy is, > but if you use the function np.testing.assert_ then you can add a > failure message with eg. repr(t) > > They should all use assert_ as it is designed to work even in optimized environments. However, this is not enforced, official policy ;) Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 2to3c: 2to3 for C extensions
la, 2009-11-21 kello 20:00 -0500, David Warde-Farley kirjoitti: > Guido posted a link to this on Twitter, it might be of interest to > people interested in the NumPy Python 3 transition: > > http://dmalcolm.livejournal.com/3935.html > > It's doubtful this is a magic bullet (at least, not yet) but it may > still help reduce the amount of busywork involved. Interesting -- though at the moment it doesn't seem to solve very many of our problems. I did some cleanup of numpy.testing, distutils, and core so that using 2to3 the build_ext in numpy.core can at least be started: http://github.com/pv/numpy-work/tree/py3k python3 tools/py3test testing distutils core Of course, the build currently fails with zillion errors in multiarraymodule. Pauli ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] neighborhood iterator
I wonder if the neighbourhood iterator can be used as a more efficient replacement for ndimage.generic_filter. Is there a way to use it from cython? Nadav. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] failure building trunk with mingw
On Sun, Nov 22, 2009 at 10:47 AM, wrote: > On Sun, Nov 22, 2009 at 10:37 AM, wrote: >> On Sun, Nov 22, 2009 at 10:25 AM, David Cournapeau >> wrote: >>> On Mon, Nov 23, 2009 at 12:14 AM, wrote: On Sun, Nov 22, 2009 at 10:01 AM, David Cournapeau wrote: > On Sun, Nov 22, 2009 at 11:45 PM, Charles R Harris > wrote: >> >> Might be nice to print out the actual values of np.spacing and >> np.nextafter >> here. >> > > Yes, I should add some utilities to print those for this kind of test. > But in this case, I know the problem: mingw gcc use 80 bits long > double but it uses the MS runtime which considers double and long > double to be the same (8 bytes). > > I think the real fix is to force npy_longdouble to be double on mingw, > but I don't want to make that change now for 1.4.0. adding the failing type in the test to the failure message would also be helpful >>> >>> Yes, you're right. I also used the nose facility for using generators >>> for complex corner cases, but with retrospect, it is not so useful, >>> because you don't get a name when you have a failure (or maybe I am >>> using it wrong). >> >> I don't know what the policy for the use of assert in numpy is, >> but if you use the function np.testing.assert_ then you can add a >> failure message with eg. repr(t) >> >> With "yield check_a_function arguments" nose prints the arguments >> in the test description, putting information into the arguments gets >> then displayed. This is also useful for arguments that the check >> function doesn't really need. > > example: > > I change the test to this, then the new test failure output is as below > > def test_spacing(): > for t in [np.float32, np.float64, np.longdouble]: > one = t(1) > eps = np.finfo(t).eps > nan = t(np.nan) > inf = t(np.inf) > assert_(np.spacing(one) == eps, repr(t)) > assert_(np.isnan(np.spacing(nan)), repr(t)) > assert_(np.isnan(np.spacing(inf)), repr(t)) > assert_(np.isnan(np.spacing(-inf)), repr(t)) > > > == > FAIL: test_umath.test_spacing > -- > Traceback (most recent call last): > File > "c:\programs\python25\lib\site-packages\nose-0.11.1-py2.5.egg\nose\case.p > y", line 183, in runTest > self.test(*self.arg) > File > "C:\Josef\_progs\Subversion\numpy-trunk\dist\numpy-1.4.0.dev7758.win32\Pr > ograms\Python25\Lib\site-packages\numpy\core\tests\test_umath.py", line 863, > in > test_spacing > assert_(np.spacing(one) == eps, repr(t)) > File "\Programs\Python25\Lib\site-packages\numpy\testing\utils.py", line 33, > i > n assert_ > AssertionError: or something like this to also get the values of the failing assertion def test_spacing(): for t in [np.float32, np.float64, np.longdouble]: one = t(1) eps = np.finfo(t).eps nan = t(np.nan) inf = t(np.inf) msg = '%r %r == %r' % (t, np.spacing(one), eps) assert_(np.spacing(one) == eps, msg) #assert_(np.spacing(one) == eps, repr(t)+ repr(np.spacing(one))+repr(eps)) assert_(np.isnan(np.spacing(nan)), repr(t)+ repr(np.spacing(nan))) assert_(np.isnan(np.spacing(inf)), repr(t)+ repr(np.spacing(inf))) assert_(np.isnan(np.spacing(-inf)), repr(t)+ repr(np.spacing(-inf))) Josef > > Josef > >> Josef >> >>> >>> David >>> ___ >>> NumPy-Discussion mailing list >>> NumPy-Discussion@scipy.org >>> http://mail.scipy.org/mailman/listinfo/numpy-discussion >>> >> > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] failure building trunk with mingw
On Sun, Nov 22, 2009 at 10:37 AM, wrote: > On Sun, Nov 22, 2009 at 10:25 AM, David Cournapeau wrote: >> On Mon, Nov 23, 2009 at 12:14 AM, wrote: >>> On Sun, Nov 22, 2009 at 10:01 AM, David Cournapeau >>> wrote: On Sun, Nov 22, 2009 at 11:45 PM, Charles R Harris wrote: > > Might be nice to print out the actual values of np.spacing and > np.nextafter > here. > Yes, I should add some utilities to print those for this kind of test. But in this case, I know the problem: mingw gcc use 80 bits long double but it uses the MS runtime which considers double and long double to be the same (8 bytes). I think the real fix is to force npy_longdouble to be double on mingw, but I don't want to make that change now for 1.4.0. >>> >>> adding the failing type in the test to the failure message would also >>> be helpful >> >> Yes, you're right. I also used the nose facility for using generators >> for complex corner cases, but with retrospect, it is not so useful, >> because you don't get a name when you have a failure (or maybe I am >> using it wrong). > > I don't know what the policy for the use of assert in numpy is, > but if you use the function np.testing.assert_ then you can add a > failure message with eg. repr(t) > > With "yield check_a_function arguments" nose prints the arguments > in the test description, putting information into the arguments gets > then displayed. This is also useful for arguments that the check > function doesn't really need. example: I change the test to this, then the new test failure output is as below def test_spacing(): for t in [np.float32, np.float64, np.longdouble]: one = t(1) eps = np.finfo(t).eps nan = t(np.nan) inf = t(np.inf) assert_(np.spacing(one) == eps, repr(t)) assert_(np.isnan(np.spacing(nan)), repr(t)) assert_(np.isnan(np.spacing(inf)), repr(t)) assert_(np.isnan(np.spacing(-inf)), repr(t)) == FAIL: test_umath.test_spacing -- Traceback (most recent call last): File "c:\programs\python25\lib\site-packages\nose-0.11.1-py2.5.egg\nose\case.p y", line 183, in runTest self.test(*self.arg) File "C:\Josef\_progs\Subversion\numpy-trunk\dist\numpy-1.4.0.dev7758.win32\Pr ograms\Python25\Lib\site-packages\numpy\core\tests\test_umath.py", line 863, in test_spacing assert_(np.spacing(one) == eps, repr(t)) File "\Programs\Python25\Lib\site-packages\numpy\testing\utils.py", line 33, i n assert_ AssertionError: Josef > Josef > >> >> David >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] failure building trunk with mingw
On Sun, Nov 22, 2009 at 10:25 AM, David Cournapeau wrote: > On Mon, Nov 23, 2009 at 12:14 AM, wrote: >> On Sun, Nov 22, 2009 at 10:01 AM, David Cournapeau >> wrote: >>> On Sun, Nov 22, 2009 at 11:45 PM, Charles R Harris >>> wrote: Might be nice to print out the actual values of np.spacing and np.nextafter here. >>> >>> Yes, I should add some utilities to print those for this kind of test. >>> But in this case, I know the problem: mingw gcc use 80 bits long >>> double but it uses the MS runtime which considers double and long >>> double to be the same (8 bytes). >>> >>> I think the real fix is to force npy_longdouble to be double on mingw, >>> but I don't want to make that change now for 1.4.0. >> >> adding the failing type in the test to the failure message would also >> be helpful > > Yes, you're right. I also used the nose facility for using generators > for complex corner cases, but with retrospect, it is not so useful, > because you don't get a name when you have a failure (or maybe I am > using it wrong). I don't know what the policy for the use of assert in numpy is, but if you use the function np.testing.assert_ then you can add a failure message with eg. repr(t) With "yield check_a_function arguments" nose prints the arguments in the test description, putting information into the arguments gets then displayed. This is also useful for arguments that the check function doesn't really need. Josef > > David > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] failure building trunk with mingw
On Mon, Nov 23, 2009 at 12:14 AM, wrote: > On Sun, Nov 22, 2009 at 10:01 AM, David Cournapeau wrote: >> On Sun, Nov 22, 2009 at 11:45 PM, Charles R Harris >> wrote: >>> >>> Might be nice to print out the actual values of np.spacing and np.nextafter >>> here. >>> >> >> Yes, I should add some utilities to print those for this kind of test. >> But in this case, I know the problem: mingw gcc use 80 bits long >> double but it uses the MS runtime which considers double and long >> double to be the same (8 bytes). >> >> I think the real fix is to force npy_longdouble to be double on mingw, >> but I don't want to make that change now for 1.4.0. > > adding the failing type in the test to the failure message would also > be helpful Yes, you're right. I also used the nose facility for using generators for complex corner cases, but with retrospect, it is not so useful, because you don't get a name when you have a failure (or maybe I am using it wrong). David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] failure building trunk with mingw
On Sun, Nov 22, 2009 at 10:01 AM, David Cournapeau wrote: > On Sun, Nov 22, 2009 at 11:45 PM, Charles R Harris > wrote: >> >> Might be nice to print out the actual values of np.spacing and np.nextafter >> here. >> > > Yes, I should add some utilities to print those for this kind of test. > But in this case, I know the problem: mingw gcc use 80 bits long > double but it uses the MS runtime which considers double and long > double to be the same (8 bytes). > > I think the real fix is to force npy_longdouble to be double on mingw, > but I don't want to make that change now for 1.4.0. adding the failing type in the test to the failure message would also be helpful from test_umath test_spacing(): >>> for t in [np.float32, np.float64, np.longdouble]: ... one = t(1) ... eps = np.finfo(t).eps ... nan = t(np.nan) ... inf = t(np.inf) ... print t ... print np.spacing(one), eps ... print np.isnan(np.spacing(nan)) ... print np.isnan(np.spacing(inf)) ... print np.isnan(np.spacing(-inf)) ... 1.19209e-007 1.19209e-007 True True True 2.22044604925e-016 2.22044604925e-016 True True True 1.08420217249e-019 2.22044604925e-016 True True True Josef > > David > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] failure building trunk with mingw
On Sun, Nov 22, 2009 at 11:45 PM, Charles R Harris wrote: > > Might be nice to print out the actual values of np.spacing and np.nextafter > here. > Yes, I should add some utilities to print those for this kind of test. But in this case, I know the problem: mingw gcc use 80 bits long double but it uses the MS runtime which considers double and long double to be the same (8 bytes). I think the real fix is to force npy_longdouble to be double on mingw, but I don't want to make that change now for 1.4.0. David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] failure building trunk with mingw
On Sun, Nov 22, 2009 at 6:28 AM, wrote: > On Sun, Nov 22, 2009 at 2:35 AM, David Cournapeau > wrote: > > On Wed, Nov 18, 2009 at 6:10 PM, David Cournapeau > wrote: > >> On Wed, Nov 18, 2009 at 12:38 AM, wrote: > >> > >>> > >>> Now numpy builds without problems. > >>> When I run the tests I get 16 failures mostly nan related. I have no > idea > >>> whether they are real or if there is still something screwed up in my > >>> setup. See below. > >> > >> I can reproduce those. I did not see those because they are python 2.5 > >> specific (which is puzzling). > > > > I fix most of those, which were related to an atan2 bug in MS C > > library. Could you test whether this fixes it for you ? > > > > David > > ___ > > NumPy-Discussion mailing list > > NumPy-Discussion@scipy.org > > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > Thanks, > > I have 4 failures remaining. full ouput below > > Josef > > >python > Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit > (Intel)] on > win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import numpy > >>> numpy.test() > Running unit tests for numpy > NumPy version 1.4.0.dev7758 > NumPy is installed in C:\Josef\_progs\Subversion\numpy-trunk\dist\ > numpy-1.4.0.de > v7758.win32\Programs\Python25\Lib\site-packages\numpy > Python version 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit > (Int > el)] > nose version 0.11.1 > > > > > > > > > > > > > > > > > > ...K.FF. > > K..K.F.. > > > > > > > > > > > > .C:\Josef\_progs\Subversion\numpy-trunk\dist > > \numpy-1.4.0.dev7758.win32\Programs\Python25\Lib\site-packages\numpy\lib\io.py:1 > 324: ConversionWarning: Some errors were detected ! >Line #2 (got 4 columns instead of 5) >Line #12 (got 4 columns instead of 5) >Line #22 (got 4 columns instead of 5) >Line #32 (got 4 columns instead of 5) >Line #42 (got 4 columns instead of 5) > warnings.warn(errmsg, ConversionWarning) > > .C:\Josef\_progs\Subversion\numpy-trunk\dist\numpy-1.4.0.dev7758.win32\Programs\ > Python25\Lib\site-packages\numpy\lib\io.py:1324: ConversionWarning: Some > errors > were detected ! >Line #2 (got 4 columns instead of 2) >Line #12 (got 4 columns instead of 2) >Line #22 (got 4 columns instead of 2) >Line #32 (got 4 columns instead of 2) >Line #42 (got 4 columns instead of 2) > warnings.warn(errmsg, ConversionWarning) > > ..KKF... > > > > > > > > > > > > ..S. > > > > > > > > > > > > > ... > == > FAIL: test_umath.test_nextafter >
Re: [Numpy-discussion] failure building trunk with mingw
On Sun, Nov 22, 2009 at 2:35 AM, David Cournapeau wrote: > On Wed, Nov 18, 2009 at 6:10 PM, David Cournapeau wrote: >> On Wed, Nov 18, 2009 at 12:38 AM, wrote: >> >>> >>> Now numpy builds without problems. >>> When I run the tests I get 16 failures mostly nan related. I have no idea >>> whether they are real or if there is still something screwed up in my >>> setup. See below. >> >> I can reproduce those. I did not see those because they are python 2.5 >> specific (which is puzzling). > > I fix most of those, which were related to an atan2 bug in MS C > library. Could you test whether this fixes it for you ? > > David > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > Thanks, I have 4 failures remaining. full ouput below Josef >python Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.test() Running unit tests for numpy NumPy version 1.4.0.dev7758 NumPy is installed in C:\Josef\_progs\Subversion\numpy-trunk\dist\numpy-1.4.0.de v7758.win32\Programs\Python25\Lib\site-packages\numpy Python version 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Int el)] nose version 0.11.1 ...K.FF. K..K.F.. .C:\Josef\_progs\Subversion\numpy-trunk\dist \numpy-1.4.0.dev7758.win32\Programs\Python25\Lib\site-packages\numpy\lib\io.py:1 324: ConversionWarning: Some errors were detected ! Line #2 (got 4 columns instead of 5) Line #12 (got 4 columns instead of 5) Line #22 (got 4 columns instead of 5) Line #32 (got 4 columns instead of 5) Line #42 (got 4 columns instead of 5) warnings.warn(errmsg, ConversionWarning) .C:\Josef\_progs\Subversion\numpy-trunk\dist\numpy-1.4.0.dev7758.win32\Programs\ Python25\Lib\site-packages\numpy\lib\io.py:1324: ConversionWarning: Some errors were detected ! Line #2 (got 4 columns instead of 2) Line #12 (got 4 columns instead of 2) Line #22 (got 4 columns instead of 2) Line #32 (got 4 columns instead of 2) Line #42 (got 4 columns instead of 2) warnings.warn(errmsg, ConversionWarning) ..KKF... ..S. ... == FAIL: test_umath.test_nextafter -- Traceback (most recent call last): File "c:\programs\python25\lib\site-packages\nose-0.11.1-py2.5.egg\nose\case.p y", line 183, in runTest self.test(*self.arg) File "C:\Josef\_progs\Subversion\numpy-trunk\dist\numpy-1.4.0.dev7758.win32\Pr o
