Re: [Numpy-discussion] numpy/Windows shared arrays between processes?
Thanks all: At 10:00 AM 10/10/2007, Robert Kern wrote: Something like the following should suffice (untested, though I've done similar things with ctypes before): I tested, successfully: nFromAddress.py def fromaddress(address, dtype, shape, strides=None): Create a numpy array from an integer address, a dtype or dtype string, a shape tuple, and possibly strides. import numpy # Make sure our dtype is a dtype, not just f or whatever. dtype = numpy.dtype(dtype) class Dummy(object): pass d = Dummy() d.__array_interface__ = dict( data = (address, False), typestr = dtype.str, descr = dtype.descr, shape = shape, strides = strides, version = 3, ) return numpy.asarray(d) ## Numeric example, with address kludge import Numeric, numpy, ctypes, string a0 = Numeric.zeros((1), Numeric.Int16) nAddress = int(string.split(repr(a0.__copy__))[-1][:-1], 16) tmp=(ctypes.c_long*1)(0) ctypes.memmove(tmp, nAddress+8, 4) nAddress = tmp[0] a1 = fromaddress(nAddress, numpy.int16, (1,)) ## explicit type a0[0] = 5 print a1[0] ## numpy example a2 = numpy.zeros(1, numpy.int16) nAddress = a2.__array_interface__['data'][0] nType = a2.__array_interface__['typestr'] nShape = a2.__array_interface__['shape'] a3 = fromaddress(nAddress, nType, nShape) a2[0] = 5 print a3[0] So, now with little effort the relevant info can be passed over pipes, shared memory, etc. and shares/views created in other processes, since they are not objects but ints and strings. David Cournapeau Wrote: Basically, you cannot expect file descriptors (or even file handles: the standard ones from C library fopen) to cross dll boundaries if the dll do not have exactly the same runtime. It sounds like there is a general dilemma: no one with Python 2.4 or 2.5 can reliably expect to compile extensions/modules if they did not install the 7.1 compiler in time. The 2.6 seems to use VC 2005 Express, I don't know about py3000(?), with associated upgrade issues. It would be nice if the build bots could also compile suggested modules/extentions. Thanks again, Ray ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] How does Python treat commons in fortran code?
Hello all This afternoon I introduced Python to my boss and the rest of my colleagues in the CFD laboratory (universidad politecnica de madrid, school of aeronautics). My boss wanted to know if it would be helpful to manage a quite complex parallel algorithm while keeping its performance. He has a long background in high performance computing, mainly in fortran. He asked me something i could not answer, in fact I just have no clue about it. We have some very tuned parallel subroutines to perform FFTs that are written in fortran 77; they have lots of _common_ structures that are used to manage large chunks of memory. Those subroutines have shown to be scalable to 2000 processors so it is very important that they perform as expected. Of course our intention is to use them and I had the idea of building a wrapper using f2py. His question was. ¿How does ctypes or f2py handle the _common_ structures present in fortran 77? ¿Are all they allocated at load? ¿Do they work exactly as they were called from a fortran executable and the main program is aware of all the _commons_ and allocates the storage when it is asked for? I told him that I see no difference between the python interpreter and any other executable but my knowledge of python is not that deep. I could not explain him successfully exactly what is stored in the interpreter stack and what is not neither. He has been programming for more than 40 years and I could not get him much into object oriented programing, however he found python+numpy+scipy a very promising tool. guillem ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] fortran 90 compiler problem
Hello,
I'm using the last svn version of numpy (.
I tried the solution found on the mail list:
python setup.py config_fc --fcompiler=gnu95 build
the two interesting part (at least for me) in the building process are:
customize GnuFCompiler Found executable /usr/bin/g77
gnu: no Fortran 90 compiler found
gnu: no Fortran 90 compiler found
customize GnuFCompiler
gnu: no Fortran 90 compiler found
gnu: no Fortran 90 compiler found
customize GnuFCompiler using config
and later:
customize G95FCompiler Found executable /usr/local/bin/g95
but I still have the same answer when I'm trying to do: f2py -c -m hello
hello.f90
customize GnuFCompiler Found executable /usr/bin/g77
gnu: no Fortran 90 compiler found gnu: no Fortran 90
compiler found customize GnuFCompiler
gnu: no Fortran 90 compiler found
gnu: no Fortran 90 compiler found
and so:
error: f90 not supported by GnuFCompiler needed for hello.f90
I can't compile my fortran module because of this. I'll attach the file obtain
with
python setup.py -v config_fc --fcompiler=gnu95 build
And I have the same result for g95.
thank you for numpy and f2py.
N.
non-existing path in 'numpy/distutils': 'site.cfg'
F2PY Version 2_4026
blas_opt_info:
blas_mkl_info:
( library_dirs = /usr/local/lib:/usr/lib )
( include_dirs = /usr/local/include:/usr/include )
(paths: )
(paths: )
(paths: )
(paths: )
(paths: )
(paths: )
libraries mkl,vml,guide not found in /usr/local/lib
(paths: )
(paths: )
(paths: )
(paths: )
(paths: )
(paths: )
libraries mkl,vml,guide not found in /usr/lib
NOT AVAILABLE
atlas_blas_threads_info:
Setting PTATLAS=ATLAS
( library_dirs = /usr/local/lib:/usr/lib )
(paths: )
(paths: /usr/lib/atlas,/usr/lib/sse2)
(paths: )
(paths: )
(paths: )
(paths: )
(paths: )
(paths: )
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
(paths: )
(paths: )
(paths: )
(paths: )
(paths: )
(paths: )
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/atlas
(paths: )
(paths: )
(paths: )
(paths: )
(paths: /usr/lib/sse2/libatlas.so)
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2
(paths: )
(paths: )
(paths: )
(paths: )
(paths: /usr/lib/libatlas.so)
libraries ptf77blas,ptcblas,atlas not found in /usr/lib
NOT AVAILABLE
atlas_blas_info:
( library_dirs = /usr/local/lib:/usr/lib )
(paths: )
(paths: /usr/lib/atlas,/usr/lib/sse2)
(paths: )
(paths: )
(paths: )
(paths: )
(paths: )
(paths: )
libraries f77blas,cblas,atlas not found in /usr/local/lib
(paths: )
(paths: )
(paths: )
(paths: )
(paths: )
(paths: )
libraries f77blas,cblas,atlas not found in /usr/lib/atlas
(paths: /usr/lib/sse2/libf77blas.so)
(paths: /usr/lib/sse2/libcblas.so)
(paths: /usr/lib/sse2/libatlas.so)
( include_dirs = /usr/local/include:/usr/include )
(paths: )
(paths:
/usr/include/atlas_type.h,/usr/include/atlas_misc.h,/usr/include/atlas_enum.h,/usr/include/atlas_aux.h)
(paths: /usr/include/cblas.h)
( library_dirs = /usr/local/lib:/usr/lib )
(paths: )
(paths: /usr/lib/atlas,/usr/lib/sse2)
FOUND:
libraries = ['f77blas', 'cblas', 'atlas']
library_dirs = ['/usr/lib/sse2']
language = c
include_dirs = ['/usr/include']
new_compiler returns distutils.unixccompiler.UnixCCompiler
customize GnuFCompiler
find_executable('g77')
Found executable /usr/bin/g77
gnu: no Fortran 90 compiler found
gnu: no Fortran 90 compiler found
exec_command(['/usr/bin/g77', '--version'],)
Retaining cwd: /home/gruel/tmp/Python/SCIPY/numpy
_preserve_environment([])
_update_environment(...)
_exec_command_posix(...)
Running os.system('( /usr/bin/g77 --version ; echo $? /tmp/tmphW9yyT/O1TTQZ )
/tmp/tmphW9yyT/TsbhKl 21')
_update_environment(...)
exec_command(['/usr/bin/g77', '-g', '-Wall', '-fno-second-underscore', '-fPIC',
'-O3', '-funroll-loops', '-march=pentium-m', '-mmmx', '-msse2', '-msse',
'-fomit-frame-pointer', '-print-libgcc-file-name'],)
Retaining cwd: /home/gruel/tmp/Python/SCIPY/numpy
_preserve_environment([])
_update_environment(...)
_exec_command_posix(...)
Running os.system('( /usr/bin/g77 -g -Wall -fno-second-underscore -fPIC -O3
-funroll-loops -march=pentium-m -mmmx -msse2 -msse -fomit-frame-pointer
-print-libgcc-file-name ; echo $? /tmp/tmphW9yyT/kZfve1 )
/tmp/tmphW9yyT/oZpGzj 21')
_update_environment(...)
customize GnuFCompiler
gnu: no Fortran 90 compiler found
gnu: no Fortran 90 compiler found
exec_command(['/usr/bin/g77', '--version'],)
Retaining cwd: /home/gruel/tmp/Python/SCIPY/numpy
_preserve_environment([])
_update_environment(...)
_exec_command_posix(...)
Running os.system('( /usr/bin/g77 --version ; echo $? /tmp/tmphW9yyT/yuFkkf )
/tmp/tmphW9yyT/J1E0P7 21')
_update_environment(...)
exec_command(['/usr/bin/g77', '-g', '-Wall', '-fno-second-underscore', '-fPIC',
'-O3', '-funroll-loops', '-march=pentium-m', '-mmmx', '-msse2', '-msse',
'-fomit-frame-pointer', '-print-libgcc-file-name'],)
Retaining cwd: /home/gruel/tmp/Python/SCIPY/numpy
_preserve_environment([])
_update_environment(...)
_exec_command_posix(...)
Running
Re: [Numpy-discussion] fortran 90 compiler problem
I have already gfortran and g95 installed and working fine for all the system
but numpy and scipy.
By the way I'm using linux not MacOSX. My problem is that when I'm installing
numpy gfortran (or g95) is detected but not at all the step and so f2py seem
to be unable to use it.
look at:
line 92 in my attach file:
find_executable('g77')
Found executable /usr/bin/g77
gnu: no Fortran 90 compiler found
gnu: no Fortran 90 compiler found
but line 469:
customize Gnu95FCompiler
find_executable('gfortran')
Found executable /usr/local/bin/gfortran
I don't understand the problem.
Thanks
N.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] fortran 90 compiler problem
[EMAIL PROTECTED] wrote: Hello, I'm using the last svn version of numpy (. I tried the solution found on the mail list: python setup.py config_fc --fcompiler=gnu95 build the two interesting part (at least for me) in the building process are: customize GnuFCompiler Found executable /usr/bin/g77 gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler using config and later: customize G95FCompiler Found executable /usr/local/bin/g95 but I still have the same answer when I'm trying to do: f2py -c -m hello hello.f90 customize GnuFCompiler Found executable /usr/bin/g77 gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found This is a correct message and does not imply that no Fortran 90 compiler exists. The GnuFCompiler (gnu) only looks at g77, which is not a Fortran 90 compiler. That message only applies to the gnu compiler, not any other. and so: error: f90 not supported by GnuFCompiler needed for hello.f90 I can't compile my fortran module because of this. I'll attach the file obtain with python setup.py -v config_fc --fcompiler=gnu95 build And I have the same result for g95. thank you for numpy and f2py. Please give us the build log that actually failed and the command that you used to obtain it. The one you show, for numpy, succeeded. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] fortran 90 compiler problem
This is the log I have when I'm trying to use f2py with a fortran 90 file. The last line seems to tell me that f2py cannot see gfortran and g95 even if it detected them when I installed f2py. thanks, N. gfortran and g95 are installed and I can use them: gfortran gfortran: no input files gfortran -c hello.f90 ls hello* hello.f90 hello.o g95 g95: no input files g95 -c hello.f90 ls hello* hello.f90 hello.o and now f2py -c -m hello hello.f90 running build running config_cc unifing config_cc, config, build_clib, build_ext, build commands --compiler opti ons running config_fc unifing config_fc, config, build_clib, build_ext, build commands --fcompiler opt ions running build_src building extension hello sources f2py options: [] f2py: /tmp/tmpg5exdv/src.linux-i686-2.5/hellomodule.c creating /tmp/tmpg5exdv creating /tmp/tmpg5exdv/src.linux-i686-2.5 Reading fortran codes... Reading file 'hello.f90' (format:free) Post-processing... Block: hello Block: foo Post-processing (stage 2)... Building modules... Building module hello... Constructing wrapper function foo... foo(a) Wrote C/API module hello to file /tmp/tmpg5exdv/src.linux-i686-2.5/he llomodule.c adding '/tmp/tmpg5exdv/src.linux-i686-2.5/fortranobject.c' to sources. adding '/tmp/tmpg5exdv/src.linux-i686-2.5' to include_dirs. copying /home/gruel/usr/lib/python2.5/site-packages/numpy/f2py/src/fortranobject .c - /tmp/tmpg5exdv/src.linux-i686-2.5 copying /home/gruel/usr/lib/python2.5/site-packages/numpy/f2py/src/fortranobject .h - /tmp/tmpg5exdv/src.linux-i686-2.5 running build_ext customize UnixCCompiler customize UnixCCompiler using build_ext customize GnuFCompiler Found executable /usr/bin/g77 gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler using build_ext building 'hello' extension compiling C sources C compiler: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O2 -Wall -Wstrict-pro totypes -fPIC creating /tmp/tmpg5exdv/tmp creating /tmp/tmpg5exdv/tmp/tmpg5exdv creating /tmp/tmpg5exdv/tmp/tmpg5exdv/src.linux-i686-2.5 compile options: '-I/tmp/tmpg5exdv/src.linux-i686-2.5 -I/home/gruel/usr/lib/pyth on2.5/site-packages/numpy/core/include -I/usr/include/python2.5 -c' gcc: /tmp/tmpg5exdv/src.linux-i686-2.5/hellomodule.c gcc: /tmp/tmpg5exdv/src.linux-i686-2.5/fortranobject.c compiling Fortran sources Fortran f77 compiler: /usr/bin/g77 -g -Wall -fno-second-underscore -fPIC -O3 -fu nroll-loops -march=pentium-m -mmmx -msse2 -msse -fomit-frame-pointer compile options: '-I/tmp/tmpg5exdv/src.linux-i686-2.5 -I/home/gruel/usr/lib/pyth on2.5/site-packages/numpy/core/include -I/usr/include/python2.5 -c' error: f90 not supported by GnuFCompiler needed for hello.f90 On Wednesday 10 October 2007 16:15:32 Robert Kern wrote: [EMAIL PROTECTED] wrote: Hello, I'm using the last svn version of numpy (. I tried the solution found on the mail list: python setup.py config_fc --fcompiler=gnu95 build the two interesting part (at least for me) in the building process are: customize GnuFCompiler Found executable /usr/bin/g77 gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler using config and later: customize G95FCompiler Found executable /usr/local/bin/g95 but I still have the same answer when I'm trying to do: f2py -c -m hello hello.f90 customize GnuFCompiler Found executable /usr/bin/g77 gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found This is a correct message and does not imply that no Fortran 90 compiler exists. The GnuFCompiler (gnu) only looks at g77, which is not a Fortran 90 compiler. That message only applies to the gnu compiler, not any other. and so: error: f90 not supported by GnuFCompiler needed for hello.f90 I can't compile my fortran module because of this. I'll attach the file obtain with python setup.py -v config_fc --fcompiler=gnu95 build And I have the same result for g95. thank you for numpy and f2py. Please give us the build log that actually failed and the command that you used to obtain it. The one you show, for numpy, succeeded. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] fortran 90 compiler problem
I had the same problem building on Linux. I found this on a wiki somewhere (can only find it in google cache now): (Note: Make sure that if you build with *gfortran* that *g77* is not installed on your system (or at least is not in your *PATH* when *numpy* is being built) as you need to link with the same compiler that you built lapack with when *numpy* builds. It will try and find *g77* first which will lead to linking errors if you have built lapack with *gfortran*). The trick for me was that step of removing g77 (or taking it out of the path) - then distutils seems to pick up gfortran fine, but otherwise it doesn't seem to work (even with --fcompiler=gnu95 etc) Cheers Robin ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] How does Python treat commons in fortran code?
Guillem Borrell i Nogueras wrote: ¿How does ctypes or f2py handle the _common_ structures present in fortran 77? I think that's covered (for f2py) by this: http://cens.ioc.ee/projects/f2py2e/usersguide/#common-blocks ¿Are all they allocated at load? ¿Do they work exactly as they were called from a fortran executable and the main program is aware of all the _commons_ and allocates the storage when it is asked for? I'm pretty sure that's the case. He has been programming for more than 40 years and I could not get him much into object oriented programing, however he found python+numpy+scipy a very promising tool. Good news -- you could save a lot of time by using FORTRAN only for what it's good for: the number crunching. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception [EMAIL PROTECTED] ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] fortran 90 compiler problem
[EMAIL PROTECTED] wrote: This is the log I have when I'm trying to use f2py with a fortran 90 file. The last line seems to tell me that f2py cannot see gfortran and g95 even if it detected them when I installed f2py. thanks, N. gfortran and g95 are installed and I can use them: gfortran gfortran: no input files gfortran -c hello.f90 ls hello* hello.f90 hello.o g95 g95: no input files g95 -c hello.f90 ls hello* hello.f90 hello.o and now f2py -c -m hello hello.f90 Tell f2py that you want it to use gfortran instead of the default. f2py --fcompiler=gnu95 -c -m hello hello.f90 -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] fortran 90 compiler problem
On 10/10/07, Robert Kern [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: Hello, I'm using the last svn version of numpy (. I tried the solution found on the mail list: python setup.py config_fc --fcompiler=gnu95 build the two interesting part (at least for me) in the building process are: customize GnuFCompiler Found executable /usr/bin/g77 gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler using config and later: customize G95FCompiler Found executable /usr/local/bin/g95 but I still have the same answer when I'm trying to do: f2py -c -m hello hello.f90 customize GnuFCompiler Found executable /usr/bin/g77 gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found This is a correct message and does not imply that no Fortran 90 compiler exists. The GnuFCompiler (gnu) only looks at g77, which is not a Fortran 90 compiler. That message only applies to the gnu compiler, not any other. I have always found these messages confusing. Perhaps they could be better worded to reflect the process, or, even better, silenced. All that really matters is success or failure and the reason, not every little dead end explored on the way. If they are needed for debugging, write the whole mess to a file in the build directory instead of the console. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] fortran 90 compiler problem
Charles R Harris wrote: On 10/10/07, *Robert Kern* [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Hello, I'm using the last svn version of numpy (. I tried the solution found on the mail list: python setup.py config_fc --fcompiler=gnu95 build the two interesting part (at least for me) in the building process are: customize GnuFCompiler Found executable /usr/bin/g77 gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler using config and later: customize G95FCompiler Found executable /usr/local/bin/g95 but I still have the same answer when I'm trying to do: f2py -c -m hello hello.f90 customize GnuFCompiler Found executable /usr/bin/g77 gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found customize GnuFCompiler gnu: no Fortran 90 compiler found gnu: no Fortran 90 compiler found This is a correct message and does not imply that no Fortran 90 compiler exists. The GnuFCompiler (gnu) only looks at g77, which is not a Fortran 90 compiler. That message only applies to the gnu compiler, not any other. I have always found these messages confusing. Perhaps they could be better worded to reflect the process, or, even better, silenced. All that really matters is success or failure and the reason, not every little dead end explored on the way. If they are needed for debugging, write the whole mess to a file in the build directory instead of the console. Go for it. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] For review: first milestone of scons support in numpy
Hi, (if you are not interested in numpy developement, you can stop now :) ). Following the discussion a few days ago on using scons to build extensions in numpy, I have reached a somewhat usable milestone, in the numpy.scons branch of numpy, and would like to hear some comments, remarks, critics, etc...: Where to get/see: - svn repository : http://svn.scipy.org/svn/numpy/branches/numpy.scons looking at the code: http://projects.scipy.org/scipy/numpy/browser/branches/numpy.scons Examples: - To see how it feels from the package developer point of view, I have put three really simple examples: - Building a python extension: http://projects.scipy.org/scipy/numpy/browser/branches/numpy.scons/numpy/scons_fake/pyext - Building a ctypes-based package: http://projects.scipy.org/scipy/numpy/browser/branches/numpy.scons/numpy/scons_fake/ctypesext - An example on how to check for libraries and symbols in them: http://projects.scipy.org/scipy/numpy/browser/branches/numpy.scons/numpy/scons_fake/pyext For the numpy user, this should be totally transparent (no difference when building/installing). What: - This first milestone implements the following: - adding a scons command to numpy.distutils - adding an add_sconscript function to numpy.distutils setup, for packages willing to use scons - two builders: one for ctypes extension, and one for python extension - a basic implementation to check for libraries (the paths can be overwritten exactly like with distutils, using site.cfg; I have not yet implemented overwriting with environment variables). I have been testing this on the following platforms: - linux with gcc - linux with icc - linux with suncc - windows with MS toolikit 2003 - solaris studio express with suncc - mac os X (tiger, x86) And now ? - As discussed previously, I think numpy would benefit from exclusively using scons to build compiled extensions. I have started working on fortran support for scons (separate project, since this may be useful to all scons users, not just numpy): https://launchpad.net/numpy.scons.support and I can already do some non trivial things, not possible with numpy.distutils (automatically figuring out fortran mangling, flags for linking with C, blas/lapack flags). As expected, this is much more robust than distutils approach of hardcoding everything: although I used g77 for development, it worked without any change with ifort, gfortran and sun fortran compiler (on linux). There are still some issues for sure, but I don't see big problems. I don't want to do the work for nothing, though, so I would like to know the feeling of numpy developers first on this direction, in particular which platforms should work before merging consideration, etc... cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy/Windows shared arrays between processes?
Ray S wrote: Thanks all: At 10:00 AM 10/10/2007, Robert Kern wrote: Something like the following should suffice (untested, though I've done similar things with ctypes before): I tested, successfully: nFromAddress.py def fromaddress(address, dtype, shape, strides=None): Create a numpy array from an integer address, a dtype or dtype string, a shape tuple, and possibly strides. import numpy # Make sure our dtype is a dtype, not just f or whatever. dtype = numpy.dtype(dtype) class Dummy(object): pass d = Dummy() d.__array_interface__ = dict( data = (address, False), typestr = dtype.str, descr = dtype.descr, shape = shape, strides = strides, version = 3, ) return numpy.asarray(d) ## Numeric example, with address kludge import Numeric, numpy, ctypes, string a0 = Numeric.zeros((1), Numeric.Int16) nAddress = int(string.split(repr(a0.__copy__))[-1][:-1], 16) tmp=(ctypes.c_long*1)(0) ctypes.memmove(tmp, nAddress+8, 4) nAddress = tmp[0] a1 = fromaddress(nAddress, numpy.int16, (1,)) ## explicit type a0[0] = 5 print a1[0] ## numpy example a2 = numpy.zeros(1, numpy.int16) nAddress = a2.__array_interface__['data'][0] nType = a2.__array_interface__['typestr'] nShape = a2.__array_interface__['shape'] a3 = fromaddress(nAddress, nType, nShape) a2[0] = 5 print a3[0] So, now with little effort the relevant info can be passed over pipes, shared memory, etc. and shares/views created in other processes, since they are not objects but ints and strings. David Cournapeau Wrote: Basically, you cannot expect file descriptors (or even file handles: the standard ones from C library fopen) to cross dll boundaries if the dll do not have exactly the same runtime. It sounds like there is a general dilemma: no one with Python 2.4 or 2.5 can reliably expect to compile extensions/modules if they did not install the 7.1 compiler in time. Well, in theory you could: 'just' recompile python. The problem is not the compiler as such, but the C runtime. I don't see any solution to this situation, unfortunately; if MS decides to ship a broken libc, it is hard to get around that in a portable way. For files (I don't know any other problems, but this certainly do not mean they do not exist), the only way I know is to use the win32 files handles. At least, it works in C (I had similar problems when dealing with tmp files on win32). To do it directly in python, you may need pywin32 specific functions (I cannot remember them on the top of my head). The 2.6 seems to use VC 2005 Express, I don't know about py3000(?), with associated upgrade issues. But what if the next MS compiler has again broken libc implementation ? (Incidently, VS2005 was not used for python2.5 for even more broken libc than in 2003): http://mail.python.org/pipermail/python-dev/2006-April/064126.html cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] appending extra items to arrays
Hi In some code I have, I need to append some extra data to a given array. At the moment I construct the data in a list, append the extra information I need and then convert the final list to an array. Is there a way that I can append extra information to an existing array thereby negating the need for the list, as the array object doesn't seem to have an append() method? Cheers Adam ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] appending extra items to arrays
Adam Mercer wrote: Hi In some code I have, I need to append some extra data to a given array. At the moment I construct the data in a list, append the extra information I need and then convert the final list to an array. Is there a way that I can append extra information to an existing array thereby negating the need for the list, as the array object doesn't seem to have an append() method? Fundamentally, numpy works because its underlying data is in a block of memory. This is incompatible with appending data at will, which is the exact thing lists are good at. Now, depending on the problem you are trying to solve, it may be better (read faster, more memory efficient, etc...) to use numpy arrays, and create new bigger ones when you need to append data. It all depends on your problem. cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] appending extra items to arrays
Adam Mercer wrote: Hi In some code I have, I need to append some extra data to a given array. At the moment I construct the data in a list, append the extra information I need and then convert the final list to an array. Is there a way that I can append extra information to an existing array thereby negating the need for the list, as the array object doesn't seem to have an append() method? Appending to a list then converting the list to an array is the most straightforward way to do it. If the performance of this isn't a problem, I recommend leaving it alone. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] appending extra items to arrays
On 11/10/2007, Robert Kern [EMAIL PROTECTED] wrote: Appending to a list then converting the list to an array is the most straightforward way to do it. If the performance of this isn't a problem, I recommend leaving it alone. Just a speculation: Python strings have a similar problem - they're immutable, and so are even more resistant to growth than numpy arrays. For those situations where you really really want to grow a srting, python provides StringIO, where you keep efficiently adding to the string, then finalize it to get the real string out. Would something analogous be interesting for arrays? Anne P.S. A (somewhat) slow python implementation would be fairly easy to write. -A ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] appending extra items to arrays
Anne Archibald wrote: On 11/10/2007, Robert Kern [EMAIL PROTECTED] wrote: Appending to a list then converting the list to an array is the most straightforward way to do it. If the performance of this isn't a problem, I recommend leaving it alone. Just a speculation: Python strings have a similar problem - they're immutable, and so are even more resistant to growth than numpy arrays. For those situations where you really really want to grow a srting, python provides StringIO, where you keep efficiently adding to the string, then finalize it to get the real string out. Would something analogous be interesting for arrays? The Python version of StringIO, at least, just keeps a list, too. The primary benefit of StringIO is that it exposes the file interface. For building up a string/array, using the list is just as convenient (and for arrays, probably more convenient given the variety of shapes one might want to build) and at least as readable. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] fortran 90 compiler problem
It's working fine thank you. I'm a little bit surprised because I was thinking that the --fcompiler when building numpy was the way to change the default compiler but anyway, that's work now. N. Tell f2py that you want it to use gfortran instead of the default. f2py --fcompiler=gnu95 -c -m hello hello.f90 ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
