[Numpy-discussion] Help building NumPy on Windows
Hi, I'm having some trouble building numpy on a 64-bit Windows 7 machine. I'm probably accidentally missing a step following the build process described at http://scipy.org/Installing_SciPy/Windows; it would be great if someone could spot what! Here's what I did: 1. installed python 2.7 from python.org 2. installed mingw32 from the link above (results in gcc 3.4.5) 3. added c:\mingw\bin to front of path 4. ran python setup.py build --compiler=mingw32 install --prefix=numpy-install I got the following error: [...] C:\Users\ceball\npslavetest\Windows7_x64_mingw32_3.4.5- py27\build\numpy\distutils\system_info.py:1409: UserWarning: Lapack (http://www.netlib.org/lapack/) sources not found. Directories to search for the sources can be specified in the numpy/distutils/site.cfg file (section [lapack_src]) or by setting the LAPACK_SRC environment variable. warnings.warn(LapackSrcNotFoundError.__doc__) customize GnuFCompiler gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler using config C compiler: gcc -mno-cygwin -O2 -Wall -Wstrict-prototypes compile options: '-DNPY_MINGW_USE_CUSTOM_MSVCR -D__MSVCRT_VERSION__=0x0900 - Inumpy\core\src\private -Inumpy\core\src -Inumpy\core -Inumpy\core\src\npymath - Inumpy\core\src\multiarray -Inumpy\core\src\umath -Inumpy\core\src\npysort - Inumpy\core\include -Ic:\Python27\include -Ic:\Python27\PC -c' gcc -mno-cygwin -O2 -Wall -Wstrict-prototypes -DNPY_MINGW_USE_CUSTOM_MSVCR - D__MSVCRT_VERSION__=0x0900 -Inumpy\core\src\private -Inumpy\core\src - Inumpy\core -Inumpy\core\src\npymath -Inumpy\core\src\multiarray - Inumpy\core\src\umath -Inumpy\core\src\npysort -Inumpy\core\include - Ic:\Python27\include -Ic:\Python27\PC -c _configtest.c -o _configtest.o Found executable c:\mingw\bin\gcc.exe g++ -mno-cygwin _configtest.o -lmsvcr90 -o _configtest.exe Found executable c:\mingw\bin\g++.exe c:\mingw\bin\..\lib\gcc\mingw32\3.4.5\..\..\..\..\mingw32\bin\ld.exe: cannot find -lmsvcr90 collect2: ld returned 1 exit status failure. removing: _configtest.exe.manifest _configtest.c _configtest.o Traceback (most recent call last): File setup.py, line 214, in module setup_package() File setup.py, line 207, in setup_package configuration=configuration ) File C:\Users\ceball\npslavetest\Windows7_x64_mingw32_3.4.5- py27\build\numpy\distutils\core.py, line 186, in setup return old_setup(**new_attr) File c:\Python27\lib\distutils\core.py, line 152, in setup dist.run_commands() File c:\Python27\lib\distutils\dist.py, line 953, in run_commands self.run_command(cmd) File c:\Python27\lib\distutils\dist.py, line 972, in run_command cmd_obj.run() File C:\Users\ceball\npslavetest\Windows7_x64_mingw32_3.4.5- py27\build\numpy\distutils\command\build.py, line 37, in run old_build.run(self) File c:\Python27\lib\distutils\command\build.py, line 127, in run self.run_command(cmd_name) File c:\Python27\lib\distutils\cmd.py, line 326, in run_command self.distribution.run_command(command) File c:\Python27\lib\distutils\dist.py, line 972, in run_command cmd_obj.run() File C:\Users\ceball\npslavetest\Windows7_x64_mingw32_3.4.5- py27\build\numpy\distutils\command\build_src.py, line 152, in run self.build_sources() File C:\Users\ceball\npslavetest\Windows7_x64_mingw32_3.4.5- py27\build\numpy\distutils\command\build_src.py, line 163, in build_sources self.build_library_sources(*libname_info) File C:\Users\ceball\npslavetest\Windows7_x64_mingw32_3.4.5- py27\build\numpy\distutils\command\build_src.py, line 298, in build_library_sources sources = self.generate_sources(sources, (lib_name, build_info)) File C:\Users\ceball\npslavetest\Windows7_x64_mingw32_3.4.5- py27\build\numpy\distutils\command\build_src.py, line 385, in generate_sources source = func(extension, build_dir) File numpy\core\setup.py, line 648, in get_mathlib_info raise RuntimeError(Broken toolchain: cannot link a simple C program) RuntimeError: Broken toolchain: cannot link a simple C program Can anyone see what I've missed from this? I'm not sure what the situation is for supporting building with Microsoft compilers, but on another Windows 7 machine (also 64 bits) with MS Visual Studio 9.0 installed, the build completes but I get test failures. You can see the full output of this here: https://jenkins.shiningpanda.com/scipy/job/NumPy-Windows7_x64- py27/26/consoleFull ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Possible test failure on Debian 6 with Python 2.6
Hi, I'm trying to set up various build machines. Some of these are with ShiningPanda.com, which provides a 64-bit Debian 6 machine (as well as Windows 7). This machine has multiple versions of Python installed. Using the build procedure below, I see a test failure with Python 2.6 (and 2.7) but not 2.5 (and 2.4). Any ideas? I guess other people are testing on Debian 6, and if they don't see the test failure with Python 2.6 maybe there's something unusual about the setup of this machine. Build procedure: $ python setup.py build install --prefix=./numpy-install $ cd numpy-install $ export PYTHONPATH=/path/to/numpy-install/lib/python2.6/site-packages $ python ../tools/test-installed-numpy.py --coverage -- --with-xunit Test error: == ERROR: test_multiarray.TestNewBufferProtocol.test_roundtrip -- Traceback (most recent call last): File /home/slave/jenkins/shiningpanda/jobs/51551925/virtualenvs/d41d8cd9/lib/python2 .6/site-packages/nose/case.py, line 197, in runTest self.test(*self.arg) File /home/slave/jenkins/workspace/NumPy-Debian6_x64-py26/numpy- install/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py, line 2537, in test_roundtrip assert_raises(ValueError, self._check_roundtrip, x) File /home/slave/jenkins/workspace/NumPy-Debian6_x64-py26/numpy- install/lib/python2.6/site-packages/numpy/testing/utils.py, line 1018, in assert_raises return nose.tools.assert_raises(*args,**kwargs) File /sp/lib/python/cpython-2.6/lib/python2.6/unittest.py, line 336, in failUnlessRaises callableObj(*args, **kwargs) File /home/slave/jenkins/workspace/NumPy-Debian6_x64-py26/numpy- install/lib/python2.6/site-packages/numpy/core/tests/test_multiarray.py, line 2471, in _check_roundtrip y = np.asarray(x) RuntimeWarning: tp_compare didn't return -1 or -2 for exception [...] Ran 4781 tests in 68.179s FAILED (KNOWNFAIL=4, SKIP=6, errors=1) The full build and test output is here: https://jenkins.shiningpanda.com/scipy/job/NumPy-Debian6_x64-py26/8/consoleFull For comparison, with Python 2.5: https://jenkins.shiningpanda.com/scipy/job/NumPy-Debian6_x64-py25/9/consoleFull ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Error building with Python 3.3 (was Re: Code Freeze for NumPy 1.7)
Stefan Krah stefan-usenet at bytereef.org writes: ... I wonder if this might be a blocker: Python-3.3 will be released in August and I don't think the issue is fixed yet: http://projects.scipy.org/numpy/ticket/2145 In case it helps, on a 64-bit Debian 6 machine where building with Python 3.1 and 3.2 seems fine, I think I see the same error as above. Here's the build log: https://jenkins.shiningpanda.com/scipy/job/NumPy-Debian6_x64-py33/7/consoleFull Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Option parsing: tox and test-installed-numpy.py
Hi, When calling tools/test-installed-numpy.py (https://github.com/numpy/numpy/blob/master/tools/test-installed-numpy.py), I can pass options to nose by supplying those options after --, eg: $ python tools/test-installed-numpy.py -- --with-xunit (which passes --with-xunit to nose). NumPy's tox.ini (https://github.com/numpy/numpy/blob/master/tox.ini) uses tools/test-installed-numpy.py to run the tests. To pass options to test-installed-numpy.py when calling tox, I can pass the options after --, eg: $ tox -- -v (which passes -v to test-installed-numpy.py). However, what I want to do is supply an option to tox that gets all the way through to nose! Is there a way I can do that, or do I need to edit tools/test-installed-numpy.py to have an option corresponding to nose's option? I hope that makes sense to someone! Thanks, Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Option parsing: tox and test-installed-numpy.py
On Sun, Jul 8, 2012 at 6:52 PM, Nathaniel Smith n...@pobox.com wrote: On Sun, Jul 8, 2012 at 6:44 PM, Chris Ball s0454...@sms.ed.ac.uk wrote: Hi, When calling tools/test-installed-numpy.py (https://github.com/numpy/numpy/blob/master/tools/test-installed-numpy.py), I can pass options to nose by supplying those options after --, eg: $ python tools/test-installed-numpy.py -- --with-xunit (which passes --with-xunit to nose). NumPy's tox.ini (https://github.com/numpy/numpy/blob/master/tox.ini) uses tools/test-installed-numpy.py to run the tests. To pass options to test-installed-numpy.py when calling tox, I can pass the options after --, eg: $ tox -- -v (which passes -v to test-installed-numpy.py). However, what I want to do is supply an option to tox that gets all the way through to nose! Is there a way I can do that, or do I need to edit tools/test-installed-numpy.py to have an option corresponding to nose's option? tox -- -- --with-xunit ? Thanks - I'd tried that (was just guessing), but it didn't work: $ tox -e py26 -- -- --with-xunit ... [TOX] numpy/.tox/py26$ bin/python numpy/tools/test-installed-numpy.py --with-xunit Usage: test-installed-numpy.py [options] -- [nosetests options] test-installed-numpy.py: error: no such option: --with-xunit [TOX] ERROR: InvocationError: 'bin/python numpy/tools/test-installed-numpy.py --with-xunit' If the double -- is standard (is it?), maybe it's a problem with tox? Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Buildbot status
Stéfan van der Walt stefan at sun.ac.za writes: ... I'd like to find out what the current status of continuous integration is for numpy. I'm aware of: a) http://buildbot.scipy.org -- used by Ralf for testing releases? b) http://travis-ci.org -- connected via GitHub c) http://184.73.247.160:8111 -- dedicated Amazon EC2 with TeamCity d) http://build.pydata.org:8111/ -- dedicated Rackspace instance with TeamCity e) https://jenkins.shiningpanda.com/numpy/ -- python 2.4 on Debian There's also: f) https://jenkins.shiningpanda.com/scipy/ -- python 2.4, 2.5, 2.6, 2.7 on Debian 6 Could easily add: Windows 7 slave, and various other versions of python (3, pypy, etc.). Could also produce graphical test and coverage reports. g) buildbot on the EC2 Currently only has one old, temporary linux test slave. Should work with any available platform and python versions, but - in contrast to shiningpanda - each new addition is a machine (or VM) that someone has to volunteer and look after. Volunteer slaves would require tox and virtualenv (in addition to numpy's requirements). I propose that we following a simple migration path for now: move the current buildbot onto the EC2 instance, redirect buildbot.scipy.org, and then connect the nipy build slaves. This should require minimal effort, but provide us with fairly wide coverage until you can invest more time in Jenkins etc. I'm happy to help connect slaves to the EC2 buildbot if someone sends connection details to me. Another thing to consider is where to send test results. Travis-CI already comments on pull requests, which is great; the other solutions could be set up to email a list (or even the individuals potentially responsible for causing test failures). Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] How to run NumPy's tests with coverage?
Hi, I'm trying to figure out how to run NumPy's tests with coverage enabled (i.e. numpy.test(coverage=True) ). I can run the tests successfully like this: $ git clone git://github.com/numpy/numpy.git [...] $ cd numpy/ $ python setup.py build_ext -i [...] $ cd .. # (avoid running from source directory) $ export PYTHONPATH=numpy/ $ python Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3] on linux2 Type help, copyright, credits or license for more information. python import numpy python numpy.test() Running unit tests for numpy NumPy version 1.7.0.dev-259fff8 NumPy is installed in [...]/numpy Python version 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3] nose version 0.11.1 [...] Ran 3710 tests in 27.654s OK (KNOWNFAIL=3, SKIP=6) nose.result.TextTestResult run=3710 errors=0 failures=0 However, if I try to run the tests with coverage, I get lots of errors (and seven more tests are run than without coverage): python numpy.test(coverage=True) Running unit tests for numpy NumPy version 1.7.0.dev-259fff8 NumPy is installed in [...]/numpy Python version 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3] nose version 0.11.1 Could not locate executable icc Could not locate executable ecc [...]/numpy/numarray/alter_code2.py:12: UserWarning: numpy.numarray.alter_code2 is not working yet. warnings.warn(numpy.numarray.alter_code2 is not working yet.) [...]/numpy/oldnumeric/alter_code2.py:26: UserWarning: numpy.oldnumeric.alter_code2 is not working yet. warnings.warn(numpy.oldnumeric.alter_code2 is not working yet.) [...] == ERROR: Failure: ImportError (No module named waflib.Configure) -- Traceback (most recent call last): File /usr/lib/pymodules/python2.6/nose/loader.py, line 379, in loadTestsFromName addr.filename, addr.module) File /usr/lib/pymodules/python2.6/nose/importer.py, line 39, in importFromPath return self.importFromDir(dir_path, fqname) File /usr/lib/pymodules/python2.6/nose/importer.py, line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File [...]/numpy/build_utils/waf.py, line 4, in module import waflib.Configure ImportError: No module named waflib.Configure == ERROR: Failure: ImportError (No module named numscons.numdist) -- Traceback (most recent call last): File /usr/lib/pymodules/python2.6/nose/loader.py, line 379, in loadTestsFromName addr.filename, addr.module) File /usr/lib/pymodules/python2.6/nose/importer.py, line 39, in importFromPath return self.importFromDir(dir_path, fqname) File /usr/lib/pymodules/python2.6/nose/importer.py, line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File [...]/numpy/core/scons_support.py, line 21, in module from numscons.numdist import process_c_str as process_str ImportError: No module named numscons.numdist == ERROR: Failure: ImportError (No module named numscons) -- Traceback (most recent call last): File /usr/lib/pymodules/python2.6/nose/loader.py, line 379, in loadTestsFromName addr.filename, addr.module) File /usr/lib/pymodules/python2.6/nose/importer.py, line 39, in importFromPath return self.importFromDir(dir_path, fqname) File /usr/lib/pymodules/python2.6/nose/importer.py, line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File [...]/numpy/core/setupscons.py, line 8, in module from numscons import get_scons_build_dir ImportError: No module named numscons == ERROR: test_multiarray.TestNewBufferProtocol.test_roundtrip -- Traceback (most recent call last): File /usr/lib/pymodules/python2.6/nose/case.py, line 183, in runTest self.test(*self.arg) File [...]/numpy/core/tests/test_multiarray.py, line 2233, in test_roundtrip assert_raises(ValueError, self._check_roundtrip, x) File [...]/numpy/testing/utils.py, line 1053, in assert_raises return nose.tools.assert_raises(*args,**kwargs) File /usr/lib/python2.6/unittest.py, line 336, in failUnlessRaises callableObj(*args, **kwargs) File [...]/numpy/core/tests/test_multiarray.py, line 2167, in _check_roundtrip y = np.asarray(x) File [...]/numpy/core/tests/test_multiarray.py, line 2167, in _check_roundtrip y = np.asarray(x) File /usr/lib/python2.6/dist-packages/coverage.py, line 322, in t self.c[(f.f_code.co_filename, f.f_lineno)] = 1 RuntimeWarning: tp_compare didn't return -1 or -2 for exception
Re: [Numpy-discussion] Continuous Integration
David Froger david.froger at gmail.com writes: I've been working on setting up a new buildbot for NumPy. Unfortunately, I don't have much time to work on it, so it's slow going! ... Hi, If there are things one can contribute to help the development of the buildbot for NumPy, I would be happy to participate! Great! I've sent you a message off the list so we can coordinate further. Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Test failures - which dependencies am I missing?
Chris Ball ceball at gmail.com writes: Keith Hughitt keith.hughitt at gmail.com writes: Hi Chris, Try sudo apt-get build-dep python-numpy to install the dependencies for building NumPy. I believe it will install all of the optional dependencies as well. Thanks for that, but I'd already tried it and found the same failures. However, I also found that on my version of Ubuntu (10.04 LTS), which includes NumPy 1.3.0, running numpy.test(verbose=3) yielded the following: (Above, numpy is the Ubuntu-supplied numpy in case that wasn't clear.) nose.config: INFO: Excluding tests matching ['f2py_ext','f2py_f90_ext','gen_ext', 'pyrex_ext', 'swig_ext', 'array_from_pyobj'] I've discovered that numpy itself explicitly excludes these tests (in numpy/testing/nosetester.py): # Stuff to exclude from tests. These are from numpy.distutils excludes = ['f2py_ext', 'f2py_f90_ext', 'gen_ext', 'pyrex_ext', 'swig_ext'] So, all is explained now. Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Continuous Integration
Pauli Virtanen pav at iki.fi writes: 01.05.2012 11:14, David Froger kirjoitti: Excerpts from Travis Oliphant's message of mar. mai 01 01:39:26 +0200 2012: If you have particular reasons why we should choose a particular CI service, please speak up and let your voice be heard. There is still time to make a difference in what we are setting up. Hi all, What about buildbot? (http://trac.buildbot.net/) I'm using it currently, and like it because is GPL 2, configuration files are powerful Python scripts, and development team is active and dynamic. We're currently using it: http://buildbot.scipy.org Although, it hasn't been building automatically for some time now. I've been working on setting up a new buildbot for NumPy. Unfortunately, I don't have much time to work on it, so it's slow going! Right now I am still at the stage of getting NumPy to pass all its tests on the machines I'm using as test slaves. After that, I plan to transfer existing slaves to the new setup, and then maybe ask for new volunteer slave machines (if people think the buildbot setup is useful). One nice feature of the new buildbot setup is that it can build and test a pull request, reporting the results back via a comment on the pull request. It has the following shortcomings: - Apparently, for one master instance, only one project. As others have said, this is no longer true. - Last time I looked, Git integration was poor. Maybe this has improved since... Again, as others have said, I think this has been improved since last time you looked. Buildbot can receive github notifications or can poll a repository periodically. On the other hand, I've also been trying ShiningPanda (https:// jenkins.shiningpanda.com/scipy/) because I like the idea of a hosted CI solution. Again, though, I haven't yet got all the tests to pass on ShiningPanda's default Debian 6 setup (and again, that's because of a lack of time). So, I'm working on Buildbot and ShiningPanda from the community side, but am always ready to step aside if someone else has time :) Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Test failures - which dependencies am I missing?
Keith Hughitt keith.hughitt at gmail.com writes: Hi Chris, Try sudo apt-get build-dep python-numpy to install the dependencies for building NumPy. I believe it will install all of the optional dependencies as well. Thanks for that, but I'd already tried it and found the same failures. However, I also found that on my version of Ubuntu (10.04 LTS), which includes NumPy 1.3.0, running numpy.test(verbose=3) yielded the following: nose.config: INFO: Excluding tests matching ['f2py_ext','f2py_f90_ext','gen_ext', 'pyrex_ext', 'swig_ext', 'array_from_pyobj'] I'm not sure where this nose config comes from, but at least someone else knows about some of these failures; presumably they are not important. My next step was going to be searching some mailing lists/contacting the packager... Thanks, Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Test failures - which dependencies am I missing?
Hi, When I build NumPy and then run the tests on Ubuntu (10.04 LTS) and Debian (6.1), I always seem to get several failures. I guess most of these failures come from not having some dependencies installed, but I can't figure out which ones by reading e.g. http://docs.scipy.org/doc/numpy/user/install.html. It would be great if someone could tell me what I've likely missed! I remember Gael Varoquaux posted a few weeks back with some of the same errors (http://thread.gmane.org/gmane.comp.python.numeric.general/49032/). He was also using Ubuntu (though a newer version). Anyway, on Ubuntu here are the errors - other than known failures - after python setup.py build_ext -i (or python setup.py build_ext -i -- fcompiler=gnu) followed by nosetests: == ERROR: Failure: ImportError (cannot import name fib2) -- Traceback (most recent call last): File /usr/lib/pymodules/python2.6/nose/loader.py, line 379, in loadTestsFromName addr.filename, addr.module) File /usr/lib/pymodules/python2.6/nose/importer.py, line 39, in importFromPath return self.importFromDir(dir_path, fqname) File /usr/lib/pymodules/python2.6/nose/importer.py, line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File /scratch/ceball/numpy/numpy/distutils/tests/f2py_ext/tests/ test_fib2.py, line 3, in module from f2py_ext import fib2 ImportError: cannot import name fib2 == ERROR: Failure: ImportError (cannot import name foo) -- Traceback (most recent call last): File /usr/lib/pymodules/python2.6/nose/loader.py, line 379, in loadTestsFromName addr.filename, addr.module) File /usr/lib/pymodules/python2.6/nose/importer.py, line 39, in importFromPath return self.importFromDir(dir_path, fqname) File /usr/lib/pymodules/python2.6/nose/importer.py, line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File /scratch/ceball/numpy/numpy/distutils/tests/f2py_f90_ext/tests/ test_foo.py, line 3, in module from f2py_f90_ext import foo ImportError: cannot import name foo == ERROR: Failure: ImportError (cannot import name fib3) -- Traceback (most recent call last): File /usr/lib/pymodules/python2.6/nose/loader.py, line 379, in loadTestsFromName addr.filename, addr.module) File /usr/lib/pymodules/python2.6/nose/importer.py, line 39, in importFromPath return self.importFromDir(dir_path, fqname) File /usr/lib/pymodules/python2.6/nose/importer.py, line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File /scratch/ceball/numpy/numpy/distutils/tests/gen_ext/tests/ test_fib3.py, line 3, in module from gen_ext import fib3 ImportError: cannot import name fib3 == ERROR: Failure: ImportError (No module named primes) -- Traceback (most recent call last): File /usr/lib/pymodules/python2.6/nose/loader.py, line 379, in loadTestsFromName addr.filename, addr.module) File /usr/lib/pymodules/python2.6/nose/importer.py, line 39, in importFromPath return self.importFromDir(dir_path, fqname) File /usr/lib/pymodules/python2.6/nose/importer.py, line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File /scratch/ceball/numpy/numpy/distutils/tests/pyrex_ext/tests/ test_primes.py, line 3, in module from pyrex_ext.primes import primes ImportError: No module named primes == ERROR: Failure: ImportError (cannot import name example) -- Traceback (most recent call last): File /usr/lib/pymodules/python2.6/nose/loader.py, line 379, in loadTestsFromName addr.filename, addr.module) File /usr/lib/pymodules/python2.6/nose/importer.py, line 39, in importFromPath return self.importFromDir(dir_path, fqname) File /usr/lib/pymodules/python2.6/nose/importer.py, line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File /scratch/ceball/numpy/numpy/distutils/tests/swig_ext/tests/ test_example.py, line 3, in module from swig_ext import example ImportError: cannot import name example == ERROR: Failure: ImportError (cannot import name example2) -- Traceback (most recent call last): File /usr/lib/pymodules/python2.6/nose/loader.py, line 379, in
Re: [Numpy-discussion] Segmentation fault during tests with Python 2.7.2 on Debian 6?
Charles R Harris charlesr.harris at gmail.com writes: On Thu, Apr 12, 2012 at 8:13 PM, Charles R Harris charlesr.harris at gmail.com wrote: On Thu, Apr 12, 2012 at 7:41 PM, Charles R Harris charlesr.harris at gmail.com wrote: On Thu, Apr 12, 2012 at 2:05 AM, Chris Ball ceball at gmail.com wrote: Hi, I'm trying out various continuous integration options, so I happen to be testing NumPy on several platforms that I don't normally use. Recently, I've been getting a segmentation fault on Debian 6 (with Python 2.7.2): Linux debian6-amd64 2.6.32-5-amd64 #1 SMP Thu Mar 22 17:26:33 UTC 2012 x86_64 GNU/Linux (Debian GNU/Linux 6.0 \n \l) ... Segmentation fault is buried in console output of Jenkins:https:// jenkins.shiningpanda.com/scipy/job/NumPy/PYTHON=CPython-2.7/6/console The previous build was ok:https://jenkins.shiningpanda.com/scipy/job/NumPy/ PYTHON=CPython-2.7/5/console Changes that Jenkins claims are responsible:https://jenkins.shiningpanda.com/ scipy/job/NumPy/PYTHON=CPython-2.7/6/ changes#detail0 It seems that python2.7 is far, far, too recent to be part of Debian 6. I mean, finding python 2.7 in recent Debian stable would be like finding an atomic cannon in a 1'st dynasty Egyptian tomb. So it is in testing, but for replication I like to know where you got it. Python 2.7 from Debian testing works fine here. But ActiveState python (ucs2) segfaults with a = np.array(['0123456789'], 'U') aSegmentation faultThe string needs to be long for this to show.Chuck Sorry for the delay. I'll let you know about that as soon as I can (I didn't set up the machine, and although I can get ssh access, it's not straightforward). Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] YouTrack License
Travis Oliphant teoliphant at gmail.com writes: I just received word that NumPy has a license to use TeamCity and YouTrack for NumPy development. YouTrack is a really nice issue tracker: http://www.jetbrains.com/youtrack/ TeamCity is a really nice Continuous Integration system: http:// www.jetbrains.com/teamcity/ ShiningPanda's hosted Jenkins service was mentioned in a previous thread (Issue Tracking; http://thread.gmane.org/ gmane.comp.python.numeric.general/47965), along with the limitation that it supports only Debian 6. I emailed ShiningPanda asking for some more information about using the service with a bigger range of hardware/platforms, in case it helps for comparison with TeamCity. From the ShiningPanda team (with permission): It is possible to install a Jenkins server on external hardware, run the tests there, and bring back the results on the hosted instance of ShiningPanda. It is very easy to do using this plugin: https://wiki.jenkins-ci.org/display/JENKINS/ Build+Publisher+Plugin. [On the external hardware you] will need to install a Jenkins server, it's not much work. Besides Jenkins is quite self-contained so you could easily pre- configure one and distribute it. But it will need a JVM, which could be a problem. It is not in Shining Panda documentation yet. [...] The current gentleman agreement for this feature is: if you cannot build it on ShiningPanda, then you can build it outside. It will most certainly make it to our ToS at some point. We are adding support for Windows 7: it should be available next month. At first it will be a free beta, then it will become a paid option: around $0.60 ~ $0.70 per hour of build. MacOS X should follow later this year. Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Buildbot/continuous integration (was Re: Issue Tracking)
Ralf Gommers ralf.gommers at googlemail.com writes: ... While we're at it, our buildbot situation is much worse than our issue tracker situation. This also looks good (and free): http://www.jetbrains.com/teamcity/ I'd like to help with the NumPy Buildbot situation, and below I propose a plan for myself to do this. However, I realize there are people who know more about continuous integration than I do. So, if someone is already planning to do something, I'd be happy to help with a different plan instead! I know how to set up and run Buildbot (and how much effort that takes), but I'm not familiar with the alternatives, so I can only propose one concrete plan: I'll find a machine on which to run a build master, then start to add slaves (real machines or virtual machines). At first I'll focus on the NumPy master branch, (a) testing it over different operating systems and versions of Python and (b) reporting things such as test coverage. I'll keep the Buildbot configuration in a github project, along with documentation (in case I disappear...). After getting to this initial stage, I'll discuss about adding more features (such as testing pull requests, performance testing, building binaries on the different operating systems, etc). Also, if it's working well, this Buildbot setup could replace/be merged with the one at buildbot.scipy.org (I don't know who is currently running that). Buildbot is used by some big projects (e.g. Python, Chromium, and Mozilla), but I'm aware that several projects in the scientific/numeric Python ecosystem use Jenkins (including Cython, IPython, and SymPy), often using a hosted Jenkins solution such as Shining Panda. A difficult part of running a Buildbot service is finding hardware for the slaves and keeping them alive, so a hosted solution sounds wonderful (assuming hosted solutions offer an adequate range of operating systems etc). Also, earlier in the Issue Tracking thread some commercial packages were mentioned; I don't know anything about those. So, as I said at the beginning, if someone is already planning to do something (or wants to) I'd be happy to help with a different plan instead! Otherwise, I can proceed with the plan I suggested. Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] clip() with None as argument
Robert Kern robert.kern at gmail.com writes: On Wed, Sep 8, 2010 at 14:42, Chris Ball ceball at gmail.com wrote: Robert Kern robert.kern at gmail.com writes: ... a = numpy.array([1,2,3,4,5]) a.clip(2,None) array([2, 2, 2, 2, 2], dtype=object) I'm not sure why the returned array has a dtype of object The usual type-matching semantics. a.clip() tries to find a common type that fits `a`, 2 and None. Since None only fits in an object dtype, that's what you get. ... and also why the values are not like this: numpy.maximum(a,2) array([2, 2, 3, 4, 5]) clip(a, low, high) boils down to this in the C code itself (or at least the code path that it goes down when the input array needs to be upcast): maximum(minimum(a, high), low) Since all integer objects compare None based on the default Python 2.x implementation of comparison across types, you would just get the low value out of that. Thanks for explaining that. It all makes sense if clip() requires both bounds to be passed, which is what the documentation indicates, so there is not really a problem. I was just confused because of the post and associated commit from Travis a while back implying that he'd altered clip() to support only one bound (Allow clip method to have either min or max passed in, 4566). Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] clip() with None as argument
Robert Kern robert.kern at gmail.com writes: On Tue, Sep 7, 2010 at 15:12, Friedrich Romstedt friedrichromstedt at gmail.com wrote: Ah, no need to answer, I do this myself: Friedrich, would you please use numpy.inf and -numpy.inf. But if you have an integer array, you will run into the same problem. The result will be upcast to float. I think we would accept a patch that interprets None to be the appropriate extreme bound given the input datatype. This will be tricky, though, since it must be done in C (PyArray_Clip in numpy/core/src/multiarray/calculation.c). I've been a bit confused about what numpy's clip is supposed to support. The documentation (e.g. http://docs.scipy.org/doc/numpy/reference/generated/numpy.clip.html) doesn't make any mention of supporting only one of min or max, but I remember a message some time back from Travis Oliphant that seemed to suggest it does (http://thread.gmane.org/gmane.comp.python.numeric.general/17844/focus=17877). Right now, I'm surprised by two aspect's of clip's behavior, both demonstrated below... $ python Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3] on linux2 Type help, copyright, credits or license for more information. import numpy numpy.__version__ '2.0.0.dev8698' a = numpy.array([1,2,3,4,5]) a.clip(2,None) array([2, 2, 2, 2, 2], dtype=object) I'm not sure why the returned array has a dtype of object (although that can be avoided by using clip's out argument), and also why the values are not like this: numpy.maximum(a,2) array([2, 2, 3, 4, 5]) I've been using minimum() and maximum() instead of clip() to have clip upper and clip lower functions, so it's not really a problem for me, I'm just curious (and thought maybe this could be useful for others searching in the future). Thanks, Chris ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Problem accessing elements of an array of dtype=O from C
Hi,
I'm having some trouble accessing elements in an array of dtype=O
from C code; I hope someone on the list could give me some advice
(because I might be doing something stupid).
I have an array of simple objects, created as follows:
class CF(object):
def __init__(self,num=0.0):
self.num=num
from numpy import array
objs = array([[CF(0.0),CF(0.1),CF(0.2)],
[CF(1.0),CF(1.1),CF(1.2)]],dtype=object)
I'd like to loop through this array and access the 'num' attribute of
each CF object - but using C.
I have a C function (based on an example in the numpy book - 'Basic
Iteration', page 312):
double loop(PyObject* a_){
PyArrayIterObject *iter;
iter = (PyArrayIterObject *)PyArray_IterNew(a_);
while (iter-index iter-size) {
PyObject *cf = (PyObject *)(iter-dataptr);
PyObject *num_obj = PyObject_GetAttrString(cf,num);
PyArray_ITER_NEXT(iter);
}
return 0.0;
}
I get a segmentation fault when I try to call loop(objs). (Commenting
out the 'PyObject_GetAttrString' line, I do not get a segmentation
fault.) Using instead an array of floats and a very similar function,
I have no problems:
from numpy import array
objs = array([[1.0,2.0,3.0],[4.0,5.0,6.0]],dtype=float)
...
double loop(PyObject* a_){
PyArrayIterObject *iter;
iter = (PyArrayIterObject *)PyArray_IterNew(a_);
while (iter-index iter-size) {
double *num = (double *)(iter-dataptr);
printf(%f\\n,*num);
PyArray_ITER_NEXT(iter);
}
return 0.0;
}
I am calling this C code via Instant [1]. I have previously written to
the SciPy-Dev mailing list about a strange (to me) indexing problem
using arrays of dtype='O' with Weave [2]; I believe that this is the
same problem (and that it is either with numpy, or with my
understanding of numpy, rather than with Weave or Instant).
Please could someone help me out?
If you think you could help if I were to give you a runnable example
of the problem in your own preferred way of calling C code from
Python, please let me know, and I can probably provide one.
Thanks,
Chris
[1] http://www.fenics.org/wiki/Instant
[2] http://thread.gmane.org/gmane.comp.python.scientific.devel/7198/focus=7264
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 0/0 is 0 or nan?
Thank you both for your replies - the difference is clear to me now. Chris ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] 0/0 is 0 or nan?
Hi,
I noticed some behavior that seems inconsistent to me:
from numpy import divide, seterr
seterr(divide='ignore')
{'over': 'raise', 'divide': 'raise', 'invalid': 'raise', 'under': 'raise'}
seterr()
{'over': 'raise', 'divide': 'ignore', 'invalid': 'raise', 'under': 'raise'}
divide(0,0)
0
divide(0.0,0.0)
Traceback (most recent call last):
File stdin, line 1, in ?
FloatingPointError: invalid value encountered in divide
Please could someone tell me whether this behavior comes from numpy or
elsewhere?
Thanks,
Chris
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
