Re: [Numpy-discussion] isinf raises in inf
Testing with arithmetic can raise overflows and underflows. I think the correct isinf is to compare to NPY_INFINITY and -NPY_INFINITY. Patch is attached to #1500 - Alkuperäinen viesti - On Thu, Jul 15, 2010 at 6:42 PM, John Hunter jdh2...@gmail.com wrote: On Thu, Jul 15, 2010 at 7:27 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Thu, Jul 15, 2010 at 6:11 PM, John Hunter jdh2...@gmail.com wrote: On Thu, Jul 15, 2010 at 6:14 PM, Eric Firing efir...@hawaii.edu wrote: Is it certain that the Solaris compiler lacks isinf? Is it possible that it has it, but it is not being detected? Just to clarify, I'm not using the sun compiler, but gcc-3.4.3 on solaris x86 Might be related to this thread. What version of numpy are you using? svn HEAD (2.0.0.dev8480) After reading the thread you suggested, I tried forcing the CFLAGS=-DNPY_HAVE_DECL_ISFINITE flag to be set, but this is apparently a bad idea for my platform... File /home/titan/johnh/dev/lib/python2.4/site-packages/numpy/core/__init__.py, line 5, in ? import multiarray ImportError: ld.so.1: python: fatal: relocation error: file /home/titan/johnh/dev/lib/python2.4/site-packages/numpy/core/multiarray.so: symbol isfinite: referenced symbol not found so while I think my bug is related to that thread, I don't see anything in that thread to help me fix my problem. Or am I missing something? __ Can you try out this patch without David's fixes? diff --git a/numpy/core/include/numpy/npy_math.h b/numpy/core/include/numpy/npy_ index d53900e..341fb58 100644 --- a/numpy/core/include/numpy/npy_math.h +++ b/numpy/core/include/numpy/npy_math.h @@ -151,13 +151,13 @@ double npy_spacing(double x); #endif #ifndef NPY_HAVE_DECL_ISFINITE - #define npy_isfinite(x) !npy_isnan((x) + (-x)) + #define npy_isfinite(x) (((x) + (x)) != (x) (x) == (x)) #else #define npy_isfinite(x) isfinite((x)) #endif #ifndef NPY_HAVE_DECL_ISINF - #define npy_isinf(x) (!npy_isfinite(x) !npy_isnan(x)) + #define npy_isinf(x) (((x) + (x)) == (x) (x) != 0) #else #define npy_isinf(x) isinf((x)) #endif Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] isinf raises in inf
Thu, 15 Jul 2010 19:09:15 -0600, Charles R Harris wrote:
[clip]
PS, of course we should fix the macro also. Since the bit values of +/-
infinity are known we should be able to define them as constants using a
couple of ifdefs and unions.
We already do that, NPY_INFINITY and -NPY_INFINITY.
[clip]
int
(isinf)(double x)
{
if ((-1.0 x) (x 1.0))/* x is small, and thus finite */
return (0);
else if ((x + x) == x)/* only true if x == Infinity */
return (1);
else /* must be finite (normal or subnormal), or NaN */
return (0);
}
This function can generate overflows, for example for
x = 0.6 * np.finfo(np.float64).max
--
Pauli Virtanen
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [SciPy-User] Saving Complex Numbers
Thu, 15 Jul 2010 20:00:09 -0400, David Warde-Farley wrote:
(CCing NumPy-discussion where this really belongs)
On 2010-07-08, at 1:34 PM, cfra...@uci.edu wrote:
Need Complex numbers in the saved file.
Ack, this has come up several times according to list archives and no
one's been able to provide a real answer.
It seems that there is nearly no formatting support for complex numbers
in Python. for a single value, {0.real:.18e}{0.imag:+.18e}.format(val)
will get the job done, but because of the way numpy.savetxt creates its
format string this isn't a trivial fix.
Anyone else have ideas on how complex number format strings can be
elegantly incorporated in savetxt?
The bug here is that
z = np.complex64(1+2j)
'%.18e' % z
'1.00e+00'
float(np.complex64(1+2j))
1.0
This should raise an exception. I guess the culprit is at
scalarmathmodule.c.src:1023.
It should at least be changed to raise a ComplexWarning, which we now
anyway do in casting.
--
Pauli Virtanen
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Array concatenation performance
Christopher Barker skrev: However, Python lists hold python objects, so it's bit inefficient, at least in terms of memory use. I guess that is mainly in terms of memory use, as the Python (scalar) objects must be created for the call to append. np.array([]) can also be inefficient, as Anne explained yesterday, but an appendable ndarray would not have that problem. But I don't know how important it is to save a few milliseconds worth of cpu time for this. At least not enough to convince me to spend my time on it. The memory issue is more important though. But arrays with O(1) amortized append are memory inefficient on their own. So it might make more sence to save a list of equally long short buffers (e.g. ndarrays), and flatten to a contigous buffer when needed. At least that is what I have been doing when needing to gradually build up an ndarray. The worst case memory use will be like this, amortizing the Python object overhead for large N: list of ndarrays: N*elsizewhen building up 2*N*elsize when compacting N*elsizewhen used after compacting appendable array: 2*N*elsize when building up 2*N*elsize when used before compacting 3*N*elsize when compacting N*elsizewhen used after compacting Personally I am not convinced about an appendable array, neither speed nor memory wise, I just wanted to hear what other people think. And there is one more thing that makes me sceptical: We cannot use realloc() to grow an appendable array. That is because it would segfault if there are array views in use (the views might suddenly get a dangling data pointer). And prohibiting realloc(), the performance of an appendable array might not be fantastic (still O(1) on average though). But as Anne said, this has probably been discussed in details before. Sturla ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Question on Absoft 11 and f2py
Any good reason CompaqVisualFCompiler should raise an exception when I'm not using it? Using Enthought 6.2-2. Sturla D:\testf2py -c --fcompiler=absoft --compiler=msvc -m hello hello.f90 Forcing DISTUTILS_USE_SDK=1 Unexpected ValueError in C:\Python26\lib\site-packages\numpy\distutils\fcompiler \compaq.pyc Traceback (most recent call last): File C:\Python26\Scripts\f2py-script.py, line 24, in module main() File C:\Python26\lib\site-packages\numpy\f2py\f2py2e.py, line 557, in main run_compile() File C:\Python26\lib\site-packages\numpy\f2py\f2py2e.py, line 443, in run_co mpile fcompiler.load_all_fcompiler_classes() File C:\Python26\lib\site-packages\numpy\distutils\fcompiler\__init__.py, li ne 715, in load_all_fcompiler_classes __import__ (module_name) File C:\Python26\lib\site-packages\numpy\distutils\fcompiler\compaq.py, line 55, in module class CompaqVisualFCompiler(FCompiler): File C:\Python26\lib\site-packages\numpy\distutils\fcompiler\compaq.py, line 95, in CompaqVisualFCompiler raise e ValueError: [u'path', u'lib'] ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] isinf raises in inf
On Fri, Jul 16, 2010 at 2:27 AM, Pauli Virtanen p...@iki.fi wrote:
Thu, 15 Jul 2010 19:09:15 -0600, Charles R Harris wrote:
[clip]
PS, of course we should fix the macro also. Since the bit values of +/-
infinity are known we should be able to define them as constants using a
couple of ifdefs and unions.
We already do that, NPY_INFINITY and -NPY_INFINITY.
[clip]
int
(isinf)(double x)
{
if ((-1.0 x) (x 1.0))/* x is small, and thus finite */
return (0);
else if ((x + x) == x)/* only true if x == Infinity */
return (1);
else /* must be finite (normal or subnormal), or NaN */
return (0);
}
This function can generate overflows, for example for
x = 0.6 * np.finfo(np.float64).max
Yep, but the test still works correctly. Unless overflow can somehow be
flagged to raise an error. Beebe is usually pretty careful...
Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] How to use decorator numpy.vectorize for methods in class
Hi All,
I tried to use the decorator @numpy.vectorize for the class method
without success. Can any one point me to the correct way of using it?
I am able to use the numpy.vectorize() sucessfully.
Below is the code with did not work
import numpy
class test:
�...@numpy.vectorize #= Using decorator
def add(self,x,y):
return x+y
def operate(self,operation,x,y):
if operation =='add':
return self.add(x,y) #==trying to call
def test_test():
test_instance=test()
x=numpy.arange(1,10)
y=numpy.arange(11,20)
print test_instance.operate('add',x,y)
if __name__=='__main__':
test_test()
File test.py, line 10, in operate
return self.add(x,y)
File /usr/local/lib/python2.5/site-packages/numpy/lib/function_base.py,
line 1854, in __call__
raise ValueError, mismatch between python function inputs\
ValueError: mismatch between python function inputs and received arguments
-
But the following modification works
import numpy
class test:
def add(self,x,y):
return x+y
def operate(self,operation,x,y):
if operation =='add':
newfunc=numpy.vectorize(self.add) # Explicitly
vectorizing
return newfunc(x,y) # calling the vectorized func
def test_test():
test_instance=test()
x=numpy.arange(1,10)
y=numpy.arange(11,20)
print test_instance.operate('add',x,y)
if __name__=='__main__':
test_test()
What exactly is happening in case of using decorator?
Thanks,
Shailendra
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] How to use decorator numpy.vectorize for methods in class
On Fri, Jul 16, 2010 at 12:00, Shailendra shailendra.vi...@gmail.com wrote: Hi All, I tried to use the decorator @numpy.vectorize for the class method without success. Can any one point me to the correct way of using it? I am able to use the numpy.vectorize() sucessfully. Below is the code with did not work import numpy class test: �...@numpy.vectorize #= Using decorator def add(self,x,y): return x+y numpy.vectorize does not work on methods, just functions. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Array concatenation performance
Sturla Molden wrote: Christopher Barker skrev: However, Python lists hold python objects, so it's bit inefficient, at least in terms of memory use. I guess that is mainly in terms of memory use, as the Python (scalar) objects must be created for the call to append. np.array([]) can also be inefficient, as Anne explained yesterday, but an appendable ndarray would not have that problem. But I don't know how important it is to save a few milliseconds worth of cpu time for this. not very. At least not enough to convince me to spend my time on it. The memory issue is more important though. I agree. But arrays with O(1) amortized append are memory inefficient on their own. well, yes, but by the amount that you buffer them -- I've seen reference to methods that allocate twice as much memory as needed on each expansion, so that would be a maximum of a 2X hit. In poking around the python code, it looks like lists and array.arrays over allocate by 1/16 (once the arrays are large, anyway). In my code, It defaults to over allocating by 1/4. In my very limited testing, I didn't see a performance gain by allocating more than that. So it's a maximum of 25% memory hit -- and it's user configurable. So it might make more sence to save a list of equally long short buffers (e.g. ndarrays), and flatten to a contigous buffer when needed. I did that in a C extension I wrote a while back for reading big text files. I found that the size of the buffer hardly mattered to performance. In my newer code, I found it a lot easier to over-allocate, and even if you over allocate by 1/10 or 1/16, performance is still pretty good. Personally I am not convinced about an appendable array, neither speed nor memory wise, I just wanted to hear what other people think. I still think it's a good idea -- though clearly not important enough for me to have gone further with my code (though I am using it now). For me, the reasons are memory and the ability to deal natively with numpy data types that don't map directly to Python types. I think there could be a real benefit to loadtxt() for instance. there is one more thing that makes me sceptical: We cannot use realloc() to grow an appendable array. That is because it would segfault if there are array views in use (the views might suddenly get a dangling data pointer). And prohibiting realloc(), the performance of an appendable array might not be fantastic I used ndarray.resize, which I'm pretty sure uses realloc(). And that's why I didn't subclass ndarray, but rather have an ndarray as the buffer, which never gets a view on it. When slicing, a copy is returned. But you're right this limits the utility, as it's not a first class citizen of numpy -- you still use it like you do list -- accumulate in it, then convert to a regular numpy array for further processing. I'd love to see someone with more C chops than me pick this up -- but Sturla's right -- it's hard to find the motivation for the relatively small gains. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] build problem on windows: python26.dll not found
I am way out of my element trying to build numpy on Windows, but we
are having a replicable (though somewhat inconsistently on some
machines) problem using fmin_cg on Windows that I am trying to debug.
I'm on Windows 7 64-bit with Python 2.6.4
First, I try to build numpy. It complains about having no
C:\windows\system32\python26.dll, but note
C:\Users\skipperdir %windir%\system32\python26.dll
Volume in drive C has no label.
Volume Serial Number is 182E-23CE
Directory of C:\Windows\system32
10/26/2009 08:36 AM 2,764,288 python26.dll
1 File(s) 2,764,288 bytes
0 Dir(s) 72,496,721,920 bytes free
C:\Users\skipper\source\numpypython setup.py config --compiler=mingw32 build --
compiler=mingw32 bdist_wininst
Running from numpy source directory.Forcing DISTUTILS_USE_SDK=1
F2PY Version 2_8480
blas_opt_info:
blas_mkl_info:
libraries mkl,vml,guide not found in C:\Users\skipper\source\atlaslib32
NOT AVAILABLE
atlas_blas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in C:\Users\skipper\source\atlasli
b32
NOT AVAILABLE
atlas_blas_info:
FOUND:
libraries = ['f77blas', 'cblas', 'atlas']
library_dirs = ['C:\\Users\\skipper\\source\\atlaslib32']
language = c
C:\Users\skipper\source\numpy\numpy\distutils\command\config.py:397: Deprecation
Warning:
+
Usage of get_output is deprecated: please do not
use it anymore, and avoid configuration checks
involving running executable on the target machine.
+
DeprecationWarning)
No module named msvccompiler in numpy.distutils; trying from distutils
FOUND:
libraries = ['f77blas', 'cblas', 'atlas']
library_dirs = ['C:\\Users\\skipper\\source\\atlaslib32']
language = c
define_macros = [('ATLAS_INFO', '\\?.?.?\\')]
lapack_opt_info:
lapack_mkl_info:
mkl_info:
libraries mkl,vml,guide not found in C:\Users\skipper\source\atlaslib32
NOT AVAILABLE
NOT AVAILABLE
atlas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in C:\Users\skipper\source\atlasli
b32
libraries lapack_atlas not found in C:\Users\skipper\source\atlaslib32
numpy.distutils.system_info.atlas_threads_info
NOT AVAILABLE
atlas_info:
libraries lapack_atlas not found in C:\Users\skipper\source\atlaslib32
numpy.distutils.system_info.atlas_info
FOUND:
libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
library_dirs = ['C:\\Users\\skipper\\source\\atlaslib32']
language = f77
No module named msvccompiler in numpy.distutils; trying from distutils
FOUND:
libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
library_dirs = ['C:\\Users\\skipper\\source\\atlaslib32']
language = f77
define_macros = [('ATLAS_INFO', '\\?.?.?\\')]
non-existing path in '': 'site.cfg.example'
running config
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler opti
ons
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler opt
ions
running build_src
build_src
building py_modules sources
building library npymath sources
Looking for python26.dll
Building import library (arch=AMD64): C:\Python26\libs\libpython26.a (from C:\
Windows\system32\python26.dll)
C:\Windows\system32\python26.dll
C:\Python26\libs\python26.def
objdump.exe: 'C:\Windows\system32\python26.dll': No such file
Traceback (most recent call last):
File setup.py, line 210, in module
setup_package()
File setup.py, line 203, in setup_package
configuration=configuration )
File C:\Users\skipper\source\numpy\numpy\distutils\core.py, line 186, in set
up
return old_setup(**new_attr)
File C:\Python26\lib\distutils\core.py, line 152, in setup
dist.run_commands()
File C:\Python26\lib\distutils\dist.py, line 975, in run_commands
self.run_command(cmd)
File C:\Python26\lib\distutils\dist.py, line 995, in run_command
cmd_obj.run()
File C:\Users\skipper\source\numpy\numpy\distutils\command\build.py, line 37
, in run
old_build.run(self)
File C:\Python26\lib\distutils\command\build.py, line 134, in run
self.run_command(cmd_name)
File C:\Python26\lib\distutils\cmd.py, line 333, in run_command
self.distribution.run_command(command)
File C:\Python26\lib\distutils\dist.py, line 995, in run_command
cmd_obj.run()
File C:\Users\skipper\source\numpy\numpy\distutils\command\build_src.py, lin
e 152, in run
self.build_sources()
File C:\Users\skipper\source\numpy\numpy\distutils\command\build_src.py, lin
e 163, in build_sources
self.build_library_sources(*libname_info)
File C:\Users\skipper\source\numpy\numpy\distutils\command\build_src.py, lin
e 298, in build_library_sources
sources = self.generate_sources(sources, (lib_name, build_info))
File C:\Users\skipper\source\numpy\numpy\distutils\command\build_src.py, lin
e 385, in generate_sources
source =
