Re: [Numpy-discussion] NumPy-Discussion Digest, Vol 84, Issue 19
Hi again Julian, its not np.any that is slow in this case its np.array([A, B, C]) Not really. The 4 unique alternative implementations provided all work quickly and correctly as-is, without a need to jump through a strange idiomatic hoop, so the problem does indeed lie with the implementation approach used in the current numpy.any(). It is unreasonable to expect users to know and always prepare their data using the idiom you propose if they might want to use the .any() function. If quirky idioms are needed to fix the performance of .any(), they should be placed in the implementation of .any(). this has the same performance as your code: a = empty([3] list(A.shape) How do you know? This isn't valid Python, it won't run. Please can you send me a working version I can benchmark and compare? Graeme. On Sep 5, 2013, at 8:52 PM, numpy-discussion-requ...@scipy.org wrote: hi, its not np.any that is slow in this case its np.array([A, B, C]) np.dstack([A, B, C]) is better but writing it like this has the same performance as your code: a = empty([3] list(A.shape) a[0] = A5; a[1] = B2; a[2] = A10; np.any(a, 0) I'll check if creating an array from a sequence can be improved for this case. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy-Discussion Digest, Vol 84, Issue 18
This is good stuff, but I can't help thinking that if I needed to do an any/all test on a number of arrays with common and/or combos -- I'd probably write a Cython function to do it. Hi Chris, I agree, if you want the best performance and have no constraints on implementation, then using something like cython might be the way to do it. However, not everyone has the option of using cython. I'm using a GIS program which accepts a single numpy expression and performs it across every cell. What I'm doing has to a) be compatible with the standard release b) should 'just work' for all users, not just me and preferably c) shouldn't be complicated to implement/test. I don't know if GDAL and SWIG will work out of the box with a hybrid numpy/Cython approach. From what I've read, I believe it probably won't be practical for most users, particularly on Windows. Whereas python based or core numpy improvements are drop-in upgrades to performance. I suppose this leads to a more general point that using cython instead of improving numpy performance during projects, is locally good, but globally bad, rather like the situation with idiomatic workarounds for performance as opposed to fixing the implementation. Graeme. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Funny business with 'is' operator?
On Fri, Sep 6, 2013 at 6:50 PM, James Bergstra bergs...@iro.umontreal.ca wrote: Thanks, this is exactly what I was looking for. I'll look into what this Diophantine equation is. Let's say we have two arrays with shape tuples `shape0` and `shape1`, stride tuples `stride0` and `stride1` and memory offsets `offset0` and `offset1`. Without loss of generality, let's assume that itemsize=1 since it is trivial to convert the general case to itemsize=1. Now, if you will permit Einstein summation notation, you can generate the memory address of every item in the first array like so: index0[i]*stride0[i] + offset0 0 = i len(shape0) 0 = index0[i] shape0[i] There exists an overlap between the two arrays iff there exists two tuples `index0` and `index1` such that index0[i]*stride0[i] + offset0 = index1[j]*stride0[j] + offset1 0 = i len(shape0) 0 = index0[i] shape0[i] 0 = j len(shape1) 0 = index1[j] shape1[j] This is a bounded linear Diophantine equation. Diophantine because the variables `index0[i]` and `index1[j]` are integer-valued, linear because the variables only appear multiplied by a constant, and bounded because each variable must stay within the size of its corresponding axis. Unbounded linear Diophantine equations can be solved using an extended version of the Euclidean GCD algorithm. Bounded linear Diophantine equations are NP-mumble to solve. With the right heuristics, a branch-and-bound approach could probably work well. Also, relatedly, a few months ago Julian Taylor at least wrote what was there in C, which made it faster, if not better. The main benefit of that was to make it available at the C level for other C-implemented routines, IIRC, not speed. -- Robert Kern ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: 1.8.0b2 release.
Charles R Harris wrote: Hi all, I'm happy to announce the second beta release of Numpy 1.8.0. This release should solve the Windows problems encountered in the first beta. Many thanks to Christolph Gohlke and Julian Taylor for their hard work in getting those issues settled. It would be good if folks running OS X could try out this release and report any issues on the numpy-dev mailing list. Unfortunately the files still need to be installed from source as dmg files are not avalable at this time. Source tarballs and release notes can be found at https://sourceforge.net/projects/numpy/files/NumPy/1.8.0b2/. The Windows and OS X installers will follow when the infrastructure issues are dealt with. Chuck Fedora 19 linux x86_64 mkl Package ID: l_mkl_11.0.3.163 MKL ERROR: Parameter 4 was incorrect on entry to DGETRF. ... FAIL: test_linalg.test_xerbla -- Traceback (most recent call last): File /usr/lib/python2.7/site-packages/nose/case.py, line 197, in runTest self.test(*self.arg) File /home/nbecker/.local/lib/python2.7/site- packages/numpy/testing/decorators.py, line 146, in skipper_func return f(*args, **kwargs) File /home/nbecker/.local/lib/python2.7/site- packages/numpy/linalg/tests/test_linalg.py, line 925, in test_xerbla assert_(False) File /home/nbecker/.local/lib/python2.7/site- packages/numpy/testing/utils.py, line 44, in assert_ raise AssertionError(msg) AssertionError -- Ran 5271 tests in 56.843s FAILED (KNOWNFAIL=5, SKIP=13, failures=1) nose.result.TextTestResult run=5271 errors=0 failures=1 ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Bug (?) converting list to array
I'm trying to enter a 2-D array and np.array() is returning a 1-D array of lists. I'm using Python (x,y) on Windows 7 with numpy 1.7.1. Here's the code that is giving me issues. f1 = [[15.207, 15.266, 15.181, 15.189, 15.215, 15.198], [-45, -57, -62, -70, -72, -73.5, -77]] f1a = np.array(f1) f1a array([[15.207, 15.266, 15.181, 15.189, 15.215, 15.198], [-45, -57, -62, -70, -72, -73.5, -77]], dtype=object) What am I missing? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: 1.8.0b2 release.
Hi,
I checkout the dev version of numpy and it fail to compile with this error:
creating
build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray
compile options: '-DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include
-Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy
-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core
-Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath
-Inumpy/core/src/npysort -Inumpy/core/include
-I/opt/lisa/os/epd-7.1.2/include/python2.7 -c'
gcc: numpy/core/src/multiarray/sequence.c
gcc: numpy/core/src/multiarray/descriptor.c
gcc: numpy/core/src/multiarray/getset.c
gcc: numpy/core/src/multiarray/arrayobject.c
gcc: numpy/core/src/multiarray/methods.c
numpy/core/src/multiarray/methods.c: In function ‘array_partition’:
numpy/core/src/multiarray/methods.c:1199:38: error:
‘PyArray_SelectkindConverter’ undeclared (first use in this function)
numpy/core/src/multiarray/methods.c:1199:38: note: each undeclared
identifier is reported only once for each function it appears in
numpy/core/src/multiarray/methods.c: In function ‘array_argpartition’:
numpy/core/src/multiarray/methods.c:1316:38: error:
‘PyArray_SelectkindConverter’ undeclared (first use in this function)
numpy/core/src/multiarray/methods.c:1352:9: warning: assignment makes
pointer from integer without a cast
numpy/core/src/multiarray/methods.c: In function ‘array_partition’:
numpy/core/src/multiarray/methods.c:1199:38: error:
‘PyArray_SelectkindConverter’ undeclared (first use in this function)
numpy/core/src/multiarray/methods.c:1199:38: note: each undeclared
identifier is reported only once for each function it appears in
numpy/core/src/multiarray/methods.c: In function ‘array_argpartition’:
numpy/core/src/multiarray/methods.c:1316:38: error:
‘PyArray_SelectkindConverter’ undeclared (first use in this function)
numpy/core/src/multiarray/methods.c:1352:9: warning: assignment makes
pointer from integer without a cast
error: Command gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -O2 -fPIC
-DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include
-Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy
-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core
-Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath
-Inumpy/core/src/npysort -Inumpy/core/include
-I/opt/lisa/os/epd-7.1.2/include/python2.7 -c
numpy/core/src/multiarray/methods.c -o
build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/methods.o failed
with exit status 1
PyArray_SelectkindConverter is defined in
numpy/core/src/multiarray/conversion_utils.c. methods.c include
conversion_utils.h, but there is no fct declaration of this fct in this
file. Is that normal?
Fred
On Sun, Sep 8, 2013 at 8:55 PM, Charles R Harris
charlesr.har...@gmail.comwrote:
On Sun, Sep 8, 2013 at 6:36 PM, Charles R Harris
charlesr.har...@gmail.com wrote:
On Sun, Sep 8, 2013 at 3:45 PM, Christoph Gohlke cgoh...@uci.edu wrote:
On 9/8/2013 12:14 PM, Charles R Harris wrote:
Hi all,
I'm happy to announce the second beta release of Numpy 1.8.0. This
release should solve the Windows problems encountered in the first
beta.
Many thanks to Christolph Gohlke and Julian Taylor for their hard work
in getting those issues settled.
It would be good if folks running OS X could try out this release and
report any issues on the numpy-dev mailing list. Unfortunately the
files
still need to be installed from source as dmg files are not avalable at
this time.
Source tarballs and release notes can be found at
https://sourceforge.net/projects/numpy/files/NumPy/1.8.0b2/. The
Windows
and OS X installers will follow when the infrastructure issues are
dealt
with.
Chuck
Hello,
I tested numpy 1.8.0b2 with Visual Studio and Intel MKL on Python 2.7
and 3.3 for Windows, 32 and 64 bit. There's only a single test failure
on win-amd64-py3.3, which looks strange since the test expects a
TypeError to be raised.
==
ERROR: test_record_no_hash (test_multiarray.TestRecord)
--
Traceback (most recent call last):
File
X:\Python33\lib\site-packages\numpy\core\tests\test_multiarray.py,
line 2464, in test_record_no_hash
self.assertRaises(TypeError, hash, a[0])
File X:\Python33\lib\unittest\case.py, line 570, in assertRaises
return context.handle('assertRaises', callableObj, args, kwargs)
File X:\Python33\lib\unittest\case.py, line 135, in handle
callable_obj(*args, **kwargs)
File X:\Python33\lib\unittest\case.py, line 153, in __exit__
self.obj_name))
TypeError: unhashable type: 'writeable void-scalar'
Hmm, that *is* strange.
I don't know what to make of the scipy errors at first glance.
snip
I'm going to guess self.assertRaises tried to hash it again, raising the
error again, and we see the second one. The assertRaises
Re: [Numpy-discussion] Bug (?) converting list to array
The two lists are of different sizes. Had to count twice to catch that. Ben Root On Mon, Sep 9, 2013 at 9:46 AM, Chad Kidder cckid...@gmail.com wrote: I'm trying to enter a 2-D array and np.array() is returning a 1-D array of lists. I'm using Python (x,y) on Windows 7 with numpy 1.7.1. Here's the code that is giving me issues. f1 = [[15.207, 15.266, 15.181, 15.189, 15.215, 15.198], [-45, -57, -62, -70, -72, -73.5, -77]] f1a = np.array(f1) f1a array([[15.207, 15.266, 15.181, 15.189, 15.215, 15.198], [-45, -57, -62, -70, -72, -73.5, -77]], dtype=object) What am I missing? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Bug (?) converting list to array
One list has 6 entries and one has 7, so they can't be aligned into a single array. Possibly it would be better to raise an error here instead of returning an object array, but that's what's going on. -n On 9 Sep 2013 14:49, Chad Kidder cckid...@gmail.com wrote: I'm trying to enter a 2-D array and np.array() is returning a 1-D array of lists. I'm using Python (x,y) on Windows 7 with numpy 1.7.1. Here's the code that is giving me issues. f1 = [[15.207, 15.266, 15.181, 15.189, 15.215, 15.198], [-45, -57, -62, -70, -72, -73.5, -77]] f1a = np.array(f1) f1a array([[15.207, 15.266, 15.181, 15.189, 15.215, 15.198], [-45, -57, -62, -70, -72, -73.5, -77]], dtype=object) What am I missing? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Bug (?) converting list to array
Oh, so there was a bug in the user... On Mon, Sep 9, 2013 at 7:52 AM, Nathaniel Smith n...@pobox.com wrote: One list has 6 entries and one has 7, so they can't be aligned into a single array. Possibly it would be better to raise an error here instead of returning an object array, but that's what's going on. -n On 9 Sep 2013 14:49, Chad Kidder cckid...@gmail.com wrote: I'm trying to enter a 2-D array and np.array() is returning a 1-D array of lists. I'm using Python (x,y) on Windows 7 with numpy 1.7.1. Here's the code that is giving me issues. f1 = [[15.207, 15.266, 15.181, 15.189, 15.215, 15.198], [-45, -57, -62, -70, -72, -73.5, -77]] f1a = np.array(f1) f1a array([[15.207, 15.266, 15.181, 15.189, 15.215, 15.198], [-45, -57, -62, -70, -72, -73.5, -77]], dtype=object) What am I missing? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] PEP8
On Sat, Sep 7, 2013 at 7:56 PM, Charles R Harris charlesr.har...@gmail.comwrote: Hi All, I've been doing some PEP8 work using autopep8. One problem that has turned up is that the default behavior of autopep8 is version dependent. I'd like to put a script in numpy tools that runs autopep8 with some features disabled, namely 1. E226 -- puts spaces around arithmetic operators (+, -, *, /, **). 2. E241 -- allows only single spaces after ',' Something we have done in matplotlib is that we have made PEP8 a part of the tests. We are transitioning, but the idea is that eventually, with Travis, all pull requests will get PEP8 checked. I am very leary of automatic PEP8-ing. I would rather have the tests fail and let me manually fix it rather than have code automatically changed. The first leaves expression formatting in the hands of the coder and avoids things like 2 ** 3. The second allows array entries to be vertically aligned, which can be useful in clarifying the values used in tests. A few other things that might need decisions: 1. [:,:, 2] or [:, :, 2] 2. Blank line before first function after class Foo(): For the first one, I prefer spaces. For the second one, I prefer no blank lines. Cheers! Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] PEP8
Something we have done in matplotlib is that we have made PEP8 a part of the tests. In Iris and Cartopy we've also done this and it works well. While we transition we have an exclusion list (which is gradually getting shorter). We've had mixed experiences with automatic reformatting, so prefer to keep the human in the loop. Richard ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] PEP8
On Mon, Sep 9, 2013 at 8:12 AM, Richard Hattersley rhatters...@gmail.comwrote: Something we have done in matplotlib is that we have made PEP8 a part of the tests. In Iris and Cartopy we've also done this and it works well. While we transition we have an exclusion list (which is gradually getting shorter). We've had mixed experiences with automatic reformatting, so prefer to keep the human in the loop. I agree with keeping a human in the loop, the script would be intended to get things into the right neighborhood, the submitter would have to review the changes after. If the script isn't too strict there will be than one way to do some things and those bits would rely on the good taste of the coder. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Bug (?) converting list to array
On Mon, Sep 9, 2013 at 9:52 AM, Nathaniel Smith n...@pobox.com wrote: One list has 6 entries and one has 7, so they can't be aligned into a single array. Possibly it would be better to raise an error here instead of returning an object array, but that's what's going on. It did at some point (and I relied on the exception to catch bugs, since I'm still using mainly numpy 1.5) f1 = [[15.207, 15.266, 15.181, 15.189, 15.215, 15.198], [-45, -57, -62, -70, -72, -73.5, -77]] np.array(f1) Traceback (most recent call last): File stdin, line 1, in module ValueError: setting an array element with a sequence. np.__version__ '1.5.1' now we get object arrays (in scipy.stats, and I didn't know what to do with them) I don't remember any discussion on this. Josef -n On 9 Sep 2013 14:49, Chad Kidder cckid...@gmail.com wrote: I'm trying to enter a 2-D array and np.array() is returning a 1-D array of lists. I'm using Python (x,y) on Windows 7 with numpy 1.7.1. Here's the code that is giving me issues. f1 = [[15.207, 15.266, 15.181, 15.189, 15.215, 15.198], [-45, -57, -62, -70, -72, -73.5, -77]] f1a = np.array(f1) f1a array([[15.207, 15.266, 15.181, 15.189, 15.215, 15.198], [-45, -57, -62, -70, -72, -73.5, -77]], dtype=object) What am I missing? ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: 1.8.0b2 release.
On Mon, Sep 9, 2013 at 7:46 AM, Frédéric Bastien no...@nouiz.org wrote: Hi, I checkout the dev version of numpy and it fail to compile with this error: creating build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray compile options: '-DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/lisa/os/epd-7.1.2/include/python2.7 -c' gcc: numpy/core/src/multiarray/sequence.c gcc: numpy/core/src/multiarray/descriptor.c gcc: numpy/core/src/multiarray/getset.c gcc: numpy/core/src/multiarray/arrayobject.c gcc: numpy/core/src/multiarray/methods.c numpy/core/src/multiarray/methods.c: In function ‘array_partition’: numpy/core/src/multiarray/methods.c:1199:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1199:38: note: each undeclared identifier is reported only once for each function it appears in numpy/core/src/multiarray/methods.c: In function ‘array_argpartition’: numpy/core/src/multiarray/methods.c:1316:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1352:9: warning: assignment makes pointer from integer without a cast numpy/core/src/multiarray/methods.c: In function ‘array_partition’: numpy/core/src/multiarray/methods.c:1199:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1199:38: note: each undeclared identifier is reported only once for each function it appears in numpy/core/src/multiarray/methods.c: In function ‘array_argpartition’: numpy/core/src/multiarray/methods.c:1316:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1352:9: warning: assignment makes pointer from integer without a cast error: Command gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -O2 -fPIC -DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/lisa/os/epd-7.1.2/include/python2.7 -c numpy/core/src/multiarray/methods.c -o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/methods.o failed with exit status 1 PyArray_SelectkindConverter is defined in numpy/core/src/multiarray/conversion_utils.c. methods.c include conversion_utils.h, but there is no fct declaration of this fct in this file. Is that normal? No, it looks like a bug. What is strange is that it doesn't show up on my machine. What compiler flags are you using? Could you make a PR for this? snip Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: 1.8.0b2 release.
On Mon, Sep 9, 2013 at 8:51 AM, Charles R Harris charlesr.har...@gmail.comwrote: On Mon, Sep 9, 2013 at 7:46 AM, Frédéric Bastien no...@nouiz.org wrote: Hi, I checkout the dev version of numpy and it fail to compile with this error: creating build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray compile options: '-DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/lisa/os/epd-7.1.2/include/python2.7 -c' gcc: numpy/core/src/multiarray/sequence.c gcc: numpy/core/src/multiarray/descriptor.c gcc: numpy/core/src/multiarray/getset.c gcc: numpy/core/src/multiarray/arrayobject.c gcc: numpy/core/src/multiarray/methods.c numpy/core/src/multiarray/methods.c: In function ‘array_partition’: numpy/core/src/multiarray/methods.c:1199:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1199:38: note: each undeclared identifier is reported only once for each function it appears in numpy/core/src/multiarray/methods.c: In function ‘array_argpartition’: numpy/core/src/multiarray/methods.c:1316:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1352:9: warning: assignment makes pointer from integer without a cast numpy/core/src/multiarray/methods.c: In function ‘array_partition’: numpy/core/src/multiarray/methods.c:1199:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1199:38: note: each undeclared identifier is reported only once for each function it appears in numpy/core/src/multiarray/methods.c: In function ‘array_argpartition’: numpy/core/src/multiarray/methods.c:1316:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1352:9: warning: assignment makes pointer from integer without a cast error: Command gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -O2 -fPIC -DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/lisa/os/epd-7.1.2/include/python2.7 -c numpy/core/src/multiarray/methods.c -o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/methods.o failed with exit status 1 PyArray_SelectkindConverter is defined in numpy/core/src/multiarray/conversion_utils.c. methods.c include conversion_utils.h, but there is no fct declaration of this fct in this file. Is that normal? No, it looks like a bug. What is strange is that it doesn't show up on my machine. What compiler flags are you using? Could you make a PR for this? snip Wait a minute, it is in the API. Try a clean build and see what happens. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] PEP8
I think a good solution would to use add a git_hooks directory with a
pre-commit git hook along with an git hook installation script. And a note
should be added to DEV_README.txt suggesting installing the git hooks for
pep8 compatibility. I personally use this as a pre-commit
#!/bin/sh
FILES=$(git diff --cached --name-status | grep -v ^D | awk '$1 $2 { print
$2}' | grep -e .py$)
if [ -n $FILES ]; then
pep8 -r $FILES
fi
which is from here: https://gist.github.com/lentil/810399#comment-303703
On Mon, Sep 9, 2013 at 10:54 AM, Nathaniel Smith n...@pobox.com wrote:
On Mon, Sep 9, 2013 at 3:29 PM, Charles R Harris
charlesr.har...@gmail.com wrote:
On Mon, Sep 9, 2013 at 8:12 AM, Richard Hattersley
rhatters...@gmail.com
wrote:
Something we have done in matplotlib is that we have made PEP8 a part
of
the tests.
In Iris and Cartopy we've also done this and it works well. While we
transition we have an exclusion list (which is gradually getting
shorter).
We've had mixed experiences with automatic reformatting, so prefer to
keep
the human in the loop.
I agree with keeping a human in the loop, the script would be intended to
get things into the right neighborhood, the submitter would have to
review
the changes after. If the script isn't too strict there will be than one
way
to do some things and those bits would rely on the good taste of the
coder.
So if I understand right, the goal is to have some script that
developers can run before (or after) submitting a PR, like
tools/autopep8-my-changes numpy/
that will fix up their changes, but leave the rest of numpy alone?
And the proposed mechanism is to come up with a combination of changes
to the numpy source and an autopep8 configuration such that
autopep8 --our-config numpy/
becomes a no-op, and then we can use this as an implementation of
tools/autopep8-my-changes?
If that's right then my feeling is that the goal seems worthwhile but
the approach seems difficult and unlikely to survive for long. As soon
as someone overrides autopep8 once, we either have to disable the rule
for the whole project or keep overriding it manually forever. You're
already suggesting taking out the spaces-around-arithmetic rule, which
strikes me as one of the most useful -- sure, it gets things wrongs
sometimes, but I feel like we're constantly reviewing PRs where
all*the*(arithmetic+is)-written**like*this.
Maybe a better approach would be to spend that time hacking up some
script that uses git and autopep8 together to run autopep8 over all
and only those lines which the current branch has actually touched?
It's pretty easy to parse 'git diff' output to get a list of all line
numbers which have been modified, and then we could run autopep8 over
the modified files and pull out only those changes which touch those
lines.
-n
P.S.: definitely [:, :, 2]
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Bug (?) converting list to array
On Mon, Sep 9, 2013 at 11:35 AM, Nathaniel Smith n...@pobox.com wrote: On 9 Sep 2013 15:50, josef.p...@gmail.com wrote: On Mon, Sep 9, 2013 at 9:52 AM, Nathaniel Smith n...@pobox.com wrote: One list has 6 entries and one has 7, so they can't be aligned into a single array. Possibly it would be better to raise an error here instead of returning an object array, but that's what's going on. It did at some point (and I relied on the exception to catch bugs, since I'm still using mainly numpy 1.5) f1 = [[15.207, 15.266, 15.181, 15.189, 15.215, 15.198], [-45, -57, -62, -70, -72, -73.5, -77]] np.array(f1) Traceback (most recent call last): File stdin, line 1, in module ValueError: setting an array element with a sequence. np.__version__ '1.5.1' now we get object arrays (in scipy.stats, and I didn't know what to do with them) I don't remember any discussion on this. There may not have been any. Isn't it too late now? Feel free to submit a PR and we can argue about which way is better... (I also prefer the 1.5 approach personally.) I'm just a balcony muppet (and user) (and I lost the argument against object arrays in scipy.stats) Josef -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] PEP8
On Mon, Sep 9, 2013 at 3:29 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Sep 9, 2013 at 8:12 AM, Richard Hattersley rhatters...@gmail.com wrote: Something we have done in matplotlib is that we have made PEP8 a part of the tests. In Iris and Cartopy we've also done this and it works well. While we transition we have an exclusion list (which is gradually getting shorter). We've had mixed experiences with automatic reformatting, so prefer to keep the human in the loop. I agree with keeping a human in the loop, the script would be intended to get things into the right neighborhood, the submitter would have to review the changes after. If the script isn't too strict there will be than one way to do some things and those bits would rely on the good taste of the coder. So if I understand right, the goal is to have some script that developers can run before (or after) submitting a PR, like tools/autopep8-my-changes numpy/ that will fix up their changes, but leave the rest of numpy alone? And the proposed mechanism is to come up with a combination of changes to the numpy source and an autopep8 configuration such that autopep8 --our-config numpy/ becomes a no-op, and then we can use this as an implementation of tools/autopep8-my-changes? If that's right then my feeling is that the goal seems worthwhile but the approach seems difficult and unlikely to survive for long. As soon as someone overrides autopep8 once, we either have to disable the rule for the whole project or keep overriding it manually forever. You're already suggesting taking out the spaces-around-arithmetic rule, which strikes me as one of the most useful -- sure, it gets things wrongs sometimes, but I feel like we're constantly reviewing PRs where all*the*(arithmetic+is)-written**like*this. Maybe a better approach would be to spend that time hacking up some script that uses git and autopep8 together to run autopep8 over all and only those lines which the current branch has actually touched? It's pretty easy to parse 'git diff' output to get a list of all line numbers which have been modified, and then we could run autopep8 over the modified files and pull out only those changes which touch those lines. -n P.S.: definitely [:, :, 2] ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: 1.8.0b2 release.
On Mon, Sep 9, 2013 at 9:33 AM, Frédéric Bastien no...@nouiz.org wrote: I tried it and retried and it still fail. This is in an virtualenv $git show commit c9b06111227f7a4ec213571f97e1b8d19b9c23f5 Merge: 73fbfb2 8edccea Author: Charles Harris charlesr.har...@gmail.com Date: Sun Sep 8 19:47:21 2013 -0700 Merge pull request #3701 from cgohlke/patch-2 ENH: add support for Python 3.4 ast.NameConstant $rm -rf build ## Fail as there is no such directory $pip install . # fail with the same error $pip uninstall numpy $python setup.py install --user # fail with the same error $pip install . ## fail with the same error: $git grep PyArray_SelectkindConverter doc/release/1.8.0-notes.rst:* PyArray_SelectkindConverter numpy/core/code_generators/numpy_api.py: 'PyArray_SelectkindConverter': 298, numpy/core/src/multiarray/conversion_utils.c:PyArray_SelectkindConverter(PyObject *obj, NPY_SELECTKIND *selectkind) numpy/core/src/multiarray/methods.c: PyArray_SelectkindConverter, sortkind, numpy/core/src/multiarray/methods.c: PyArray_SelectkindConverter, sortkind, Here I don't see PyArray_SelectkindConverter in conversion_utils.h as you said it is present. Witch commit do you use? It's not there, it is part of the API. I've got a PR to add it to the *.h file. The question is why you are the only person (so far) to have a problem compiling. What are your CFLAGS? Chuck Fred On Mon, Sep 9, 2013 at 11:02 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Sep 9, 2013 at 8:51 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Sep 9, 2013 at 7:46 AM, Frédéric Bastien no...@nouiz.orgwrote: Hi, I checkout the dev version of numpy and it fail to compile with this error: creating build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray compile options: '-DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/lisa/os/epd-7.1.2/include/python2.7 -c' gcc: numpy/core/src/multiarray/sequence.c gcc: numpy/core/src/multiarray/descriptor.c gcc: numpy/core/src/multiarray/getset.c gcc: numpy/core/src/multiarray/arrayobject.c gcc: numpy/core/src/multiarray/methods.c numpy/core/src/multiarray/methods.c: In function ‘array_partition’: numpy/core/src/multiarray/methods.c:1199:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1199:38: note: each undeclared identifier is reported only once for each function it appears in numpy/core/src/multiarray/methods.c: In function ‘array_argpartition’: numpy/core/src/multiarray/methods.c:1316:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1352:9: warning: assignment makes pointer from integer without a cast numpy/core/src/multiarray/methods.c: In function ‘array_partition’: numpy/core/src/multiarray/methods.c:1199:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1199:38: note: each undeclared identifier is reported only once for each function it appears in numpy/core/src/multiarray/methods.c: In function ‘array_argpartition’: numpy/core/src/multiarray/methods.c:1316:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1352:9: warning: assignment makes pointer from integer without a cast error: Command gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -O2 -fPIC -DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/lisa/os/epd-7.1.2/include/python2.7 -c numpy/core/src/multiarray/methods.c -o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/methods.o failed with exit status 1 PyArray_SelectkindConverter is defined in numpy/core/src/multiarray/conversion_utils.c. methods.c include conversion_utils.h, but there is no fct declaration of this fct in this file. Is that normal? No, it looks like a bug. What is strange is that it doesn't show up on my machine. What compiler flags are you using? Could you make a PR for this? snip Wait a minute, it is in the API. Try a clean build and see what happens. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org
Re: [Numpy-discussion] ANN: 1.8.0b2 release.
I don't have CFLAGS defined. But I have iothers env variable that point to other python stuff like CPATH. But even in that case, I don't understand how other people could have compiled methods.c. The include aren't part of the env variable, but in the file. Anyway, I think your PR is the good fix. I checked our PR and now I have this new error: gcc: numpy/core/src/multiarray/multiarraymodule.c In file included from numpy/core/src/multiarray/multiarraymodule.c:3753:0: build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:303:18: error: ‘PyArray_Partition’ undeclared here (not in a function) build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:304:18: error: ‘PyArray_ArgPartition’ undeclared here (not in a function) In file included from numpy/core/src/multiarray/multiarraymodule.c:3753:0: build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:303:18: error: ‘PyArray_Partition’ undeclared here (not in a function) build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:304:18: error: ‘PyArray_ArgPartition’ undeclared here (not in a function) error: Command gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -O2 -fPIC -DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/lisa/os/epd-7.1.2/include/python2.7 -c numpy/core/src/multiarray/multiarraymodule.c -o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/multiarraymodule.o failed with exit status 1 So it seam we have the same problem with those 2 functions. They are defined in numpy/core/src/multiarray/item_selection.c, but not in the .h file. thanks Fred On Mon, Sep 9, 2013 at 11:44 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Sep 9, 2013 at 9:33 AM, Frédéric Bastien no...@nouiz.org wrote: I tried it and retried and it still fail. This is in an virtualenv $git show commit c9b06111227f7a4ec213571f97e1b8d19b9c23f5 Merge: 73fbfb2 8edccea Author: Charles Harris charlesr.har...@gmail.com Date: Sun Sep 8 19:47:21 2013 -0700 Merge pull request #3701 from cgohlke/patch-2 ENH: add support for Python 3.4 ast.NameConstant $rm -rf build ## Fail as there is no such directory $pip install . # fail with the same error $pip uninstall numpy $python setup.py install --user # fail with the same error $pip install . ## fail with the same error: $git grep PyArray_SelectkindConverter doc/release/1.8.0-notes.rst:* PyArray_SelectkindConverter numpy/core/code_generators/numpy_api.py: 'PyArray_SelectkindConverter': 298, numpy/core/src/multiarray/conversion_utils.c:PyArray_SelectkindConverter(PyObject *obj, NPY_SELECTKIND *selectkind) numpy/core/src/multiarray/methods.c: PyArray_SelectkindConverter, sortkind, numpy/core/src/multiarray/methods.c: PyArray_SelectkindConverter, sortkind, Here I don't see PyArray_SelectkindConverter in conversion_utils.h as you said it is present. Witch commit do you use? It's not there, it is part of the API. I've got a PR to add it to the *.h file. The question is why you are the only person (so far) to have a problem compiling. What are your CFLAGS? Chuck Fred On Mon, Sep 9, 2013 at 11:02 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Sep 9, 2013 at 8:51 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Sep 9, 2013 at 7:46 AM, Frédéric Bastien no...@nouiz.orgwrote: Hi, I checkout the dev version of numpy and it fail to compile with this error: creating build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray compile options: '-DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/lisa/os/epd-7.1.2/include/python2.7 -c' gcc: numpy/core/src/multiarray/sequence.c gcc: numpy/core/src/multiarray/descriptor.c gcc: numpy/core/src/multiarray/getset.c gcc: numpy/core/src/multiarray/arrayobject.c gcc: numpy/core/src/multiarray/methods.c numpy/core/src/multiarray/methods.c: In function ‘array_partition’: numpy/core/src/multiarray/methods.c:1199:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1199:38: note: each undeclared identifier is reported only once for each function it appears in numpy/core/src/multiarray/methods.c: In function ‘array_argpartition’: numpy/core/src/multiarray/methods.c:1316:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1352:9: warning: assignment makes pointer from integer
Re: [Numpy-discussion] ANN: 1.8.0b2 release.
On Mon, Sep 9, 2013 at 11:09 AM, Frédéric Bastien no...@nouiz.org wrote: I don't have CFLAGS defined. But I have iothers env variable that point to other python stuff like CPATH. But even in that case, I don't understand how other people could have compiled methods.c. The include aren't part of the env variable, but in the file. Anyway, I think your PR is the good fix. I checked our PR and now I have this new error: gcc: numpy/core/src/multiarray/multiarraymodule.c In file included from numpy/core/src/multiarray/multiarraymodule.c:3753:0: build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:303:18: error: ‘PyArray_Partition’ undeclared here (not in a function) build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:304:18: error: ‘PyArray_ArgPartition’ undeclared here (not in a function) In file included from numpy/core/src/multiarray/multiarraymodule.c:3753:0: build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:303:18: error: ‘PyArray_Partition’ undeclared here (not in a function) build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:304:18: error: ‘PyArray_ArgPartition’ undeclared here (not in a function) error: Command gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -O2 -fPIC -DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/lisa/os/epd-7.1.2/include/python2.7 -c numpy/core/src/multiarray/multiarraymodule.c -o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/multiarraymodule.o failed with exit status 1 So it seam we have the same problem with those 2 functions. They are defined in numpy/core/src/multiarray/item_selection.c, but not in the .h file. I'm going to guess that there is something special about your virtualenv. snip Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: 1.8.0b2 release.
On Mon, Sep 9, 2013 at 6:09 PM, Frédéric Bastien no...@nouiz.org wrote: I don't have CFLAGS defined. But I have iothers env variable that point to other python stuff like CPATH. But even in that case, I don't understand how other people could have compiled methods.c. The include aren't part of the env variable, but in the file. Anyway, I think your PR is the good fix. I checked our PR and now I have this new error: gcc: numpy/core/src/multiarray/multiarraymodule.c In file included from numpy/core/src/multiarray/multiarraymodule.c:3753:0: build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:303:18: error: ‘PyArray_Partition’ undeclared here (not in a function) build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:304:18: error: ‘PyArray_ArgPartition’ undeclared here (not in a function) In file included from numpy/core/src/multiarray/multiarraymodule.c:3753:0: build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:303:18: error: ‘PyArray_Partition’ undeclared here (not in a function) build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:304:18: error: ‘PyArray_ArgPartition’ undeclared here (not in a function) error: Command gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -O2 -fPIC -DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/lisa/os/epd-7.1.2/include/python2.7 -c numpy/core/src/multiarray/multiarraymodule.c -o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/multiarraymodule.o failed with exit status 1 So it seam we have the same problem with those 2 functions. They are defined in numpy/core/src/multiarray/item_selection.c, but not in the .h file. Something is going horribly wrong with your build process. The API generation code should automatically be putting prototypes for these functions into __multiarray_api.h. I don't understand how you could have a reference to these functions in your __multiarray_api.c without having them in your __multiarray_api.h, though... -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: 1.8.0b2 release.
On Mon, Sep 9, 2013 at 12:04 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Sep 9, 2013 at 11:09 AM, Frédéric Bastien no...@nouiz.org wrote: I don't have CFLAGS defined. But I have iothers env variable that point to other python stuff like CPATH. But even in that case, I don't understand how other people could have compiled methods.c. The include aren't part of the env variable, but in the file. Anyway, I think your PR is the good fix. I checked our PR and now I have this new error: gcc: numpy/core/src/multiarray/multiarraymodule.c In file included from numpy/core/src/multiarray/multiarraymodule.c:3753:0: build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:303:18: error: ‘PyArray_Partition’ undeclared here (not in a function) build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:304:18: error: ‘PyArray_ArgPartition’ undeclared here (not in a function) In file included from numpy/core/src/multiarray/multiarraymodule.c:3753:0: build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:303:18: error: ‘PyArray_Partition’ undeclared here (not in a function) build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:304:18: error: ‘PyArray_ArgPartition’ undeclared here (not in a function) error: Command gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -O2 -fPIC -DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/lisa/os/epd-7.1.2/include/python2.7 -c numpy/core/src/multiarray/multiarraymodule.c -o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/multiarraymodule.o failed with exit status 1 So it seam we have the same problem with those 2 functions. They are defined in numpy/core/src/multiarray/item_selection.c, but not in the .h file. I'm going to guess that there is something special about your virtualenv. snip The prototypes should be in `arrayobject.h` and `ndarrayobject.h`. Perhaps old versions are being used and cached somewhere. Are they precompiled somewhere? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] ANN: 1.8.0b2 release.
try rebuilding everything from scratch. setup.py dependency handling is a bit dodgy with the generated files. On 09.09.2013 19:09, Frédéric Bastien wrote: I don't have CFLAGS defined. But I have iothers env variable that point to other python stuff like CPATH. But even in that case, I don't understand how other people could have compiled methods.c. The include aren't part of the env variable, but in the file. Anyway, I think your PR is the good fix. I checked our PR and now I have this new error: gcc: numpy/core/src/multiarray/multiarraymodule.c In file included from numpy/core/src/multiarray/multiarraymodule.c:3753:0: build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:303:18: error: ‘PyArray_Partition’ undeclared here (not in a function) build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:304:18: error: ‘PyArray_ArgPartition’ undeclared here (not in a function) In file included from numpy/core/src/multiarray/multiarraymodule.c:3753:0: build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:303:18: error: ‘PyArray_Partition’ undeclared here (not in a function) build/src.linux-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.c:304:18: error: ‘PyArray_ArgPartition’ undeclared here (not in a function) error: Command gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -O2 -fPIC -DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/lisa/os/epd-7.1.2/include/python2.7 -c numpy/core/src/multiarray/multiarraymodule.c -o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/multiarraymodule.o failed with exit status 1 So it seam we have the same problem with those 2 functions. They are defined in numpy/core/src/multiarray/item_selection.c, but not in the .h file. thanks Fred On Mon, Sep 9, 2013 at 11:44 AM, Charles R Harris charlesr.har...@gmail.com mailto:charlesr.har...@gmail.com wrote: On Mon, Sep 9, 2013 at 9:33 AM, Frédéric Bastien no...@nouiz.org mailto:no...@nouiz.org wrote: I tried it and retried and it still fail. This is in an virtualenv $git show commit c9b06111227f7a4ec213571f97e1b8d19b9c23f5 Merge: 73fbfb2 8edccea Author: Charles Harris charlesr.har...@gmail.com mailto:charlesr.har...@gmail.com Date: Sun Sep 8 19:47:21 2013 -0700 Merge pull request #3701 from cgohlke/patch-2 ENH: add support for Python 3.4 ast.NameConstant $rm -rf build ## Fail as there is no such directory $pip install . # fail with the same error $pip uninstall numpy $python setup.py install --user # fail with the same error $pip install . ## fail with the same error: $git grep PyArray_SelectkindConverter doc/release/1.8.0-notes.rst:* PyArray_SelectkindConverter numpy/core/code_generators/numpy_api.py: 'PyArray_SelectkindConverter': 298, numpy/core/src/multiarray/conversion_utils.c:PyArray_SelectkindConverter(PyObject *obj, NPY_SELECTKIND *selectkind) numpy/core/src/multiarray/methods.c: PyArray_SelectkindConverter, sortkind, numpy/core/src/multiarray/methods.c: PyArray_SelectkindConverter, sortkind, Here I don't see PyArray_SelectkindConverter in conversion_utils.h as you said it is present. Witch commit do you use? It's not there, it is part of the API. I've got a PR to add it to the *.h file. The question is why you are the only person (so far) to have a problem compiling. What are your CFLAGS? Chuck Fred On Mon, Sep 9, 2013 at 11:02 AM, Charles R Harris charlesr.har...@gmail.com mailto:charlesr.har...@gmail.com wrote: On Mon, Sep 9, 2013 at 8:51 AM, Charles R Harris charlesr.har...@gmail.com mailto:charlesr.har...@gmail.com wrote: On Mon, Sep 9, 2013 at 7:46 AM, Frédéric Bastien no...@nouiz.org mailto:no...@nouiz.org wrote: Hi, I checkout the dev version of numpy and it fail to compile with this error: creating build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray compile options: '-DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private
Re: [Numpy-discussion] ANN: 1.8.0b2 release.
I tried it and retried and it still fail. This is in an virtualenv $git show commit c9b06111227f7a4ec213571f97e1b8d19b9c23f5 Merge: 73fbfb2 8edccea Author: Charles Harris charlesr.har...@gmail.com Date: Sun Sep 8 19:47:21 2013 -0700 Merge pull request #3701 from cgohlke/patch-2 ENH: add support for Python 3.4 ast.NameConstant $rm -rf build ## Fail as there is no such directory $pip install . # fail with the same error $pip uninstall numpy $python setup.py install --user # fail with the same error $pip install . ## fail with the same error: $git grep PyArray_SelectkindConverter doc/release/1.8.0-notes.rst:* PyArray_SelectkindConverter numpy/core/code_generators/numpy_api.py: 'PyArray_SelectkindConverter': 298, numpy/core/src/multiarray/conversion_utils.c:PyArray_SelectkindConverter(PyObject *obj, NPY_SELECTKIND *selectkind) numpy/core/src/multiarray/methods.c: PyArray_SelectkindConverter, sortkind, numpy/core/src/multiarray/methods.c: PyArray_SelectkindConverter, sortkind, Here I don't see PyArray_SelectkindConverter in conversion_utils.h as you said it is present. Witch commit do you use? Fred On Mon, Sep 9, 2013 at 11:02 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Sep 9, 2013 at 8:51 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Mon, Sep 9, 2013 at 7:46 AM, Frédéric Bastien no...@nouiz.org wrote: Hi, I checkout the dev version of numpy and it fail to compile with this error: creating build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray compile options: '-DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/lisa/os/epd-7.1.2/include/python2.7 -c' gcc: numpy/core/src/multiarray/sequence.c gcc: numpy/core/src/multiarray/descriptor.c gcc: numpy/core/src/multiarray/getset.c gcc: numpy/core/src/multiarray/arrayobject.c gcc: numpy/core/src/multiarray/methods.c numpy/core/src/multiarray/methods.c: In function ‘array_partition’: numpy/core/src/multiarray/methods.c:1199:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1199:38: note: each undeclared identifier is reported only once for each function it appears in numpy/core/src/multiarray/methods.c: In function ‘array_argpartition’: numpy/core/src/multiarray/methods.c:1316:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1352:9: warning: assignment makes pointer from integer without a cast numpy/core/src/multiarray/methods.c: In function ‘array_partition’: numpy/core/src/multiarray/methods.c:1199:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1199:38: note: each undeclared identifier is reported only once for each function it appears in numpy/core/src/multiarray/methods.c: In function ‘array_argpartition’: numpy/core/src/multiarray/methods.c:1316:38: error: ‘PyArray_SelectkindConverter’ undeclared (first use in this function) numpy/core/src/multiarray/methods.c:1352:9: warning: assignment makes pointer from integer without a cast error: Command gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -O2 -fPIC -DHAVE_NPY_CONFIG_H=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-2.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/lisa/os/epd-7.1.2/include/python2.7 -c numpy/core/src/multiarray/methods.c -o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/methods.o failed with exit status 1 PyArray_SelectkindConverter is defined in numpy/core/src/multiarray/conversion_utils.c. methods.c include conversion_utils.h, but there is no fct declaration of this fct in this file. Is that normal? No, it looks like a bug. What is strange is that it doesn't show up on my machine. What compiler flags are you using? Could you make a PR for this? snip Wait a minute, it is in the API. Try a clean build and see what happens. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] PEP8
I made a PR request about supplying a git hooks framework at
https://github.com/numpy/numpy/pull/200. I asked for it to be closed
because I couldn't easily figure our how to handle x-platform issues.
If you have an answer, what I was working on might be a start. But
your script is an example of the x-platform challenges since it will
only run on Windows that has a Linux environment installed (Mingw or
the like).
What I have started to use in my projects is 'tox', 'coverage', and
'flake8'. Really nice stuff. There is an issue though with the
'flake8' report. There are only very localized places in a few files
that I want to ignore some portion or other of PEP8, so I don' t
insist that 'flake8' pass, I just ignore the output where I need to
ignore it. Usually what I want to ignore is the spaces around
brackets and commas in order to line up array values as mentioned in
an earlier post within this thread. Here is an example of my project
setup based in part on 'cookiecutter' -
https://bitbucket.org/timcera/astronomia/src. Yes, I use bitbucket -
so sue me. Magic happens inside the 'tox.ini' file.
I suspect that for numpy, making something like 'flake8' part of the
tests would work better than 'autopep8'. 'autopep8' can be configured
to just report with the --list-fixes option, which would give people a
little more confidence to use it rather that it's primary mission as
an editor. Another plug for 'flake8'; it can be configured on a file
by file basis to ignore particular errors or warnings, and exits with
an error status if the file doesn't pass.
I do think the idea has merit, but this entire process would be a lot
of work and I am not stepping forward to do it. At this point I
simply have to say that I am a 'balcony muppet'. Much thanks to Josef
for the reminder about where I learned my curmudgeony ways. Those
guys made the show!
Kindest regards,
Tim
On Mon, Sep 9, 2013 at 12:08 PM, Blake Griffith
blake.a.griff...@gmail.com wrote:
I think a good solution would to use add a git_hooks directory with a
pre-commit git hook along with an git hook installation script. And a note
should be added to DEV_README.txt suggesting installing the git hooks for
pep8 compatibility. I personally use this as a pre-commit
#!/bin/sh
FILES=$(git diff --cached --name-status | grep -v ^D | awk '$1 $2 { print
$2}' | grep -e .py$)
if [ -n $FILES ]; then
pep8 -r $FILES
fi
which is from here: https://gist.github.com/lentil/810399#comment-303703
On Mon, Sep 9, 2013 at 10:54 AM, Nathaniel Smith n...@pobox.com wrote:
On Mon, Sep 9, 2013 at 3:29 PM, Charles R Harris
charlesr.har...@gmail.com wrote:
On Mon, Sep 9, 2013 at 8:12 AM, Richard Hattersley
rhatters...@gmail.com
wrote:
Something we have done in matplotlib is that we have made PEP8 a part
of
the tests.
In Iris and Cartopy we've also done this and it works well. While we
transition we have an exclusion list (which is gradually getting
shorter).
We've had mixed experiences with automatic reformatting, so prefer to
keep
the human in the loop.
I agree with keeping a human in the loop, the script would be intended
to
get things into the right neighborhood, the submitter would have to
review
the changes after. If the script isn't too strict there will be than one
way
to do some things and those bits would rely on the good taste of the
coder.
So if I understand right, the goal is to have some script that
developers can run before (or after) submitting a PR, like
tools/autopep8-my-changes numpy/
that will fix up their changes, but leave the rest of numpy alone?
And the proposed mechanism is to come up with a combination of changes
to the numpy source and an autopep8 configuration such that
autopep8 --our-config numpy/
becomes a no-op, and then we can use this as an implementation of
tools/autopep8-my-changes?
If that's right then my feeling is that the goal seems worthwhile but
the approach seems difficult and unlikely to survive for long. As soon
as someone overrides autopep8 once, we either have to disable the rule
for the whole project or keep overriding it manually forever. You're
already suggesting taking out the spaces-around-arithmetic rule, which
strikes me as one of the most useful -- sure, it gets things wrongs
sometimes, but I feel like we're constantly reviewing PRs where
all*the*(arithmetic+is)-written**like*this.
Maybe a better approach would be to spend that time hacking up some
script that uses git and autopep8 together to run autopep8 over all
and only those lines which the current branch has actually touched?
It's pretty easy to parse 'git diff' output to get a list of all line
numbers which have been modified, and then we could run autopep8 over
the modified files and pull out only those changes which touch those
lines.
-n
P.S.: definitely [:, :, 2]
___
NumPy-Discussion mailing list
Re: [Numpy-discussion] Bug (?) converting list to array
On 9 Sep 2013 15:50, josef.p...@gmail.com wrote: On Mon, Sep 9, 2013 at 9:52 AM, Nathaniel Smith n...@pobox.com wrote: One list has 6 entries and one has 7, so they can't be aligned into a single array. Possibly it would be better to raise an error here instead of returning an object array, but that's what's going on. It did at some point (and I relied on the exception to catch bugs, since I'm still using mainly numpy 1.5) f1 = [[15.207, 15.266, 15.181, 15.189, 15.215, 15.198], [-45, -57, -62, -70, -72, -73.5, -77]] np.array(f1) Traceback (most recent call last): File stdin, line 1, in module ValueError: setting an array element with a sequence. np.__version__ '1.5.1' now we get object arrays (in scipy.stats, and I didn't know what to do with them) I don't remember any discussion on this. There may not have been any. Feel free to submit a PR and we can argue about which way is better... (I also prefer the 1.5 approach personally.) -n ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
