[Numpy-discussion] f2py: optional parameters and present() bug?
in a F90 routine called from python the present() test (for optional parameters) always returns true whether the parameter is present in the function call or not. My F90 test code looks like this subroutine test(test1,test2,test3) integer,intent(in) :: test1 integer,intent(in),optional :: test2 integer,intent(inout),optional :: test3 write (*,*) test1,present(test2),present(test3) if (present(test2)) write (*,*) 'test2',test2 if (present(test3)) then write (*,*) 'test3',test3 test3=5 end if end subroutine test The output from python calls to this routine (its inside a module tsdtm.tsg ... just to explain why the function looks like it does). I have just created this with f2py automatically generating the interface with no hand modification of the interfaces. What this test shows is that present() always returns true no matter (1) the intent (2) whether the parameter is present in the call, or (3) the type of the parameter (real, integer ... though I haven't tested arrays etc) as well. Secondly I can't see how to get the returned value of 'test3'. It doesn't seem to be returned as a result of the function? The docs say f2py handles optional parameters and other than present () my testing suggests that optional parameters seem otherwise OK. I'm on Mac OSX and using numpy 1.0.3. Any thoughts? >>> print tsimdtm.tsg.test.__doc__ test - Function signature: test(test1,[test2,test3]) Required arguments: test1 : input int Optional arguments: test2 : input int test3 : in/output rank-0 array(int,'i') >>> tsimdtm.tsg.test(1) 1 T T test2 0 test3 0 >>> tsimdtm.tsg.test(1,2) 1 T T test2 2 test3 0 >>> tsimdtm.tsg.test(1,2,3) 1 T T test2 2 test3 3 >>> tsimdtm.tsg.test(1,test2=2) 1 T T test2 2 test3 0 >>> tsimdtm.tsg.test(1,test3=3) 1 T T test2 0 test3 3 >>> tsimdtm.tsg.test(1,test2=2,test3=3) 1 T T test2 2 test3 3 Prof Garry Willgoose, Australian Professorial Fellow in Environmental Engineering, Director, Centre for Climate Impact Management (C2IM), School of Engineering, The University of Newcastle, Callaghan, 2308 Australia. Centre webpage: www.c2im.org.au Phone: (International) +61 2 4921 6050 (Tues-Fri AM); +61 2 6545 9574 (Fri PM-Mon) FAX: (International) +61 2 4921 6991 (Uni); +61 2 6545 9574 (personal and Telluric) Env. Engg. Secretary: (International) +61 2 4921 6042 email: [EMAIL PROTECTED]; [EMAIL PROTECTED] email-for-life: [EMAIL PROTECTED] personal webpage: www.telluricresearch.com/garry "Do not go where the path may lead, go instead where there is no path and leave a trail" Ralph Waldo Emerson ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] mac osx10.5 and external library crash (possible f2py/numpy problem?)
I've just moved all my stuff from a Intel Imac on 10.4.11 to a Macpro on 10.5.2. On both machines I have the same universal activestate python (both 2.5.1 and 2.5.2.2 give the same problem). I have some codes in fortran from which I build a shared library using f2py from numpy. Now when I import the library as built on osx10.4 it all works fine but when I import library as built on 10.5 I get the following error message willgoose-macpro:system garrywillgoose$ python ActivePython 2.5.1.1 (ActiveState Software Inc.) based on Python 2.5.1 (r251:54863, May 1 2007, 17:40:00) [GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import tsimdtm Fatal Python error: Interpreter not initialized (version mismatch?) Abort trap Its only my libraries that are the problem. Now also (1) if I move this library back to my 10.4 machine it works fine, and (2) if I move the library built on my 10.4 machine over to 10.5 it also works fine. Its only the library built on 10.5 and run on 10.5 that is the problem. Re f2py it appears to be independent of fortran compiler (g95, gfortran and intel fortran all give same result). I upgraded to the latest version of activestate 2.5.2 but same problem. OSX 10.4 has xcode 2.4 installed, 10.5 has 3.0 installed. Reinstalled all the fortran compilers and the numpy support library ... still the same. Any ideas (1) what the error means in the first place and (2) what I should do? I've cross-posted this on the Python-mac and numpy discussion since it appears to be a OSX/numpy/f2py interaction. ======== Prof Garry Willgoose, Australian Professorial Fellow in Environmental Engineering, Director, Centre for Climate Impact Management (C2IM), School of Engineering, The University of Newcastle, Callaghan, 2308 Australia. Centre webpage: www.c2im.org.au Phone: (International) +61 2 4921 6050 (Tues-Fri AM); +61 2 6545 9574 (Fri PM-Mon) FAX: (International) +61 2 4921 6991 (Uni); +61 2 6545 9574 (personal and Telluric) Env. Engg. Secretary: (International) +61 2 4921 6042 email: [EMAIL PROTECTED]; [EMAIL PROTECTED] email-for-life: [EMAIL PROTECTED] personal webpage: www.telluricresearch.com/garry "Do not go where the path may lead, go instead where there is no path and leave a trail" Ralph Waldo Emerson ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] when will osx linker option -bundle be reflected in distutils
I was just wondering what plans there were to reflect the different linker options (i.e. -bundle instead of -shared) that are required on OSX in the fcompiler files within distutils. While its a minor thing it always catches the users of my software when they either install fresh or update numpy ... and sometimes on a bad day it even catches me ;-) Prof Garry Willgoose, Australian Professorial Fellow in Environmental Engineering, Director, Centre for Climate Impact Management (C2IM), School of Engineering, The University of Newcastle, Callaghan, 2308 Australia. Centre webpage: www.c2im.org.au Phone: (International) +61 2 4921 6050 (Tues-Fri AM); +61 2 6545 9574 (Fri PM-Mon) FAX: (International) +61 2 4921 6991 (Uni); +61 2 6545 9574 (personal and Telluric) Env. Engg. Secretary: (International) +61 2 4921 6042 email: [EMAIL PROTECTED]; [EMAIL PROTECTED] email-for-life: [EMAIL PROTECTED] personal webpage: www.telluricresearch.com/garry "Do not go where the path may lead, go instead where there is no path and leave a trail" Ralph Waldo Emerson ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] when will osx linker option -bundle be reflected in distutils
> > I was just wondering what plans there were to reflect the different
> > linker options (i.e. -bundle instead of -shared) that are required
> on
> > OSX in the fcompiler files within distutils. While its a minor thing
> > it always catches the users of my software when they either install
> > fresh or update numpy ... and sometimes on a bad day it even catches
> > me ;-)
>
> I'm sorry; I don't follow. What problems are you having? --
> -- Robert Kern
>
---
OK for example the distribution g95.py in distutils/fcompiler has the
following code
executables = {
'version_cmd' : ["g95", "--version"],
'compiler_f77' : ["g95", "-ffixed-form"],
'compiler_fix' : ["g95", "-ffixed-form"],
'compiler_f90' : ["g95"],
'linker_so': ["g95","-shared"],
'archiver' : ["ar", "-cr"],
'ranlib' : ["ranlib"]
}
For osx you need to modify it to
executables = {
'version_cmd' : ["g95", "--version"],
'compiler_f77' : ["g95", "-ffixed-form"],
'compiler_fix' : ["g95", "-ffixed-form"],
'compiler_f90' : ["g95"],
'linker_so': ["g95","-shared"],
'archiver' : ["ar", "-cr"],
'ranlib' : ["ranlib"]
}
import sys
if sys.platform.lower() == 'darwin':
executables[linker_so'] = ["g95","-Wall -bundle"]
The 'shared' option is not implemented in the osx linker. Not sure
what the underlying difference between 'shared' and 'bundle' is but
this substitution is necessary and this has been working for me for
the last year or so. You also need the -Wall but for reasons that
completely escape me.
The same goes for gfortran and intel (both of which I use) and I
assume the other compilers that are available for OSX.
Prof Garry Willgoose,
Australian Professorial Fellow in Environmental Engineering,
Director, Centre for Climate Impact Management (C2IM),
School of Engineering, The University of Newcastle,
Callaghan, 2308
Australia.
Centre webpage: www.c2im.org.au
Phone: (International) +61 2 4921 6050 (Tues-Fri AM); +61 2 6545 9574
(Fri PM-Mon)
FAX: (International) +61 2 4921 6991 (Uni); +61 2 6545 9574 (personal
and Telluric)
Env. Engg. Secretary: (International) +61 2 4921 6042
email: garry.willgo...@newcastle.edu.au; g.willgo...@telluricresearch.com
email-for-life: garry.willgo...@alum.mit.edu
personal webpage: www.telluricresearch.com/garry
"Do not go where the path may lead, go instead where there is no path
and leave a trail"
Ralph Waldo Emerson
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] f2py: sharing F90 module data between modules
I have a suite of fortran modules that I want to wrap with f2py independently (so they appear to python as seperate imports) but where each module has access to another fortran module (which contains global data that is shared between the suite of fortran modules). I currently compile all the fortran modules, including the global data, in a single f2py statement so all the fortran code gets imported in a single statement import tsim where tsim is the python name of the aggregated modules. What I would like to do is as below, where fm1, fm2 are two example modules that will be called from python, and fm3 has the global data shared between fm1, fm2. module fm1 contains subroutine modify(myvalue) use fm3 real :: myvalue value=myvalue end subroutine modify real function result() use fm3 result=value end function result end module fm1 module fm2 contains subroutine modify(myvalue) use fm3 real, intent(in) :: myvalue value=myvalue end subroutine modify real function result() use fm3 result=value end function result end module fm2 module fm3 real :: value=3.0 end module fm3 The result of running the following script is import fm1,fm2 result1=fm1.fm1.result() result2=fm2.fm2.result() print 'original',result1,result2 x=10.0 fm1.fm1.modify(x) result1=fm1.fm1.result() result2=fm2.fm2.result() print 'fm1 allocate',result1,result2 x=20.0 fm2.fm2.modify(x) result1=fm1.fm1.result() result2=fm2.fm2.result() print 'fm2 allocate',result1,result2 is original 3.0 3.0 fm1 allocate 10.0 3.0 fm2 allocate 10.0 20.0 which clearly shows that fm1, and fm2 have independent images of fm3 (I vaguely recall some time back finding docs that said that f2py/ python loaded each imported into seperate memory and common/modules couldn't be shared between python modules). As I mentioned if fm1,fm2,fm3 are all compiled in a single f2py statement (so they are in a single python module) then this works fine. I guess the question is there any way that I can get fm3 to be shared between fm1 and fm2? The reasons for wanting to do this are because I'm developing a plug-in like architecture for environmental modelling where the user can develop new fortran modules (suitably f2py'ed) that can just be dropped into the module search path but still have access to the global data (subject to fortran module interfaces, etc). ==== Prof Garry Willgoose, Australian Professorial Fellow in Environmental Engineering, Director, Centre for Climate Impact Management (C2IM), School of Engineering, The University of Newcastle, Callaghan, 2308 Australia. Centre webpage: www.c2im.org.au Phone: (International) +61 2 4921 6050 (Tues-Fri AM); +61 2 6545 9574 (Fri PM-Mon) FAX: (International) +61 2 4921 6991 (Uni); +61 2 6545 9574 (personal and Telluric) Env. Engg. Secretary: (International) +61 2 4921 6042 email: [EMAIL PROTECTED]; [EMAIL PROTECTED] email-for-life: [EMAIL PROTECTED] personal webpage: www.telluricresearch.com/garry "Do not go where the path may lead, go instead where there is no path and leave a trail" Ralph Waldo Emerson ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] f2py: sharing F90 module data between modules
Thanks for that. The docs suggest library dl is Unix only. Does that mean this solution will not work on Windows? Windows is on my implementation roadmap but I'm not quite there yet to test it. I guess I am now thinking maybe I can assemble (using f2py) an (aggregated) shared library on the fly from the individual module shared libraries when I open my modelling environment. I could check the aggregated library mod dates against all the .so files of the components and only rebuild the aggregated library if there was a newer component than the aggregated library. That appears to work (and is fast) except for one behaviour of f2py. If I give f2py a list of files that are ALL .so (i.e. no fortran files) then f2py quits without actually doing anything, even if all the component shared libraries all have perfectly fine pythons interfaces from previous f2py builds. I can give it a trivial fortran file (module .. end module) and it works fine. f2py -c --fcompiler=g95 --verbose -m stuff -L/Library/Frameworks/ Python.framework/Versions/Current/lib/python2.5/config -lpython2.5 aread8.so fglobal_data.so # doesn't work f2py -c --fcompiler=g95 --verbose -m stuff -L/Library/Frameworks/ Python.framework/Versions/Current/lib/python2.5/config -lpython2.5 aread8.so fglobal_data.so junk.f90# does work Why is that problem? I can envisage a user that just wants to use the environment without writing any additional fortran modules (and thus may not even have an installed fortran compiler) and if they screw up mod dates on the files (by say a ftp from one machine to another ... for instance on our cluster the compiler is only installed on one machine and only binaries are moved around the cluster) then the environment might want to reassemble (with f2py) the aggregated library because it (erroneously) thinks there is a newer component shared library. This will fail because f2py quits when asked to process ONLY .so files. If I have a trivial fortran file to force f2py then this forces users to have a fortran compiler on their machine, even if they do not want to actually compile a new fortran module component, simply because f2py will not operate unless it is offered at least one fortran file. Does this make sense or am I just being thick about this? Is there a way of making f2py merge a number of existing shared libraries into a single library without having to compile any fortran. I guess I could just invoke the linker directly in the case where there are no fortran files to compile but is nice being able to use distutils to get away from platform dependencies. >> according to which makes your goal unachivable because of how >> Python loads shared libraries *by default*, see below. > >> Try to use sys.setdlopenflags(...) before importing f2py generated >> extension modules and then reset the state using sys.setdlopenflags >> (0). > > I also had to do something similar for solving a different problem, > feel free to reuse the code here. This way, you have chances to make > it working in a many platforms. You can put this in a __init__.py, and > next import all your extensions inside the last try/finally block. > > http://projects.scipy.org/mpi4py/browser/mpi4py/trunk/src/_rtld.py > > > > > -- > Lisandro Dalcín > --- > Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) > Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) > Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) > PTLC - Güemes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 > ___ > Numpy-discussion mailing list > Numpy-discussion@scipy.org > http://projects.scipy.org/mailman/listinfo/numpy-discussion Prof Garry Willgoose, Australian Professorial Fellow in Environmental Engineering, Director, Centre for Climate Impact Management (C2IM), School of Engineering, The University of Newcastle, Callaghan, 2308 Australia. Centre webpage: www.c2im.org.au Phone: (International) +61 2 4921 6050 (Tues-Fri AM); +61 2 6545 9574 (Fri PM-Mon) FAX: (International) +61 2 4921 6991 (Uni); +61 2 6545 9574 (personal and Telluric) Env. Engg. Secretary: (International) +61 2 4921 6042 email: [EMAIL PROTECTED]; [EMAIL PROTECTED] email-for-life: [EMAIL PROTECTED] personal webpage: www.telluricresearch.com/garry "Do not go where the path may lead, go instead where there is no path and leave a trail" Ralph Waldo Emerson ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] f2py: sharing F90 module data between modules
Pearu, Ohh Pearu I'm not complaining about deficiencies in f2py ... its a great piece of work that makes what I'm doing possible at all. Just like most open source software (including my own ;-) there may be ways to tweak it to do things that are undocumented. >> Why is that problem? I can envisage a user that just wants to use the >> environment without writing any additional fortran modules (and thus >> may not even have an installed fortran compiler) and if they screw up >> mod dates on the files (by say a ftp from one machine to another ... >> for instance on our cluster the compiler is only installed on one >> machine and only binaries are moved around the cluster) then the >> environment might want to reassemble (with f2py) the aggregated >> library because it (erroneously) thinks there is a newer component >> shared library. This will fail because f2py quits when asked to >> process ONLY .so files. If I have a trivial fortran file to force >> f2py then this forces users to have a fortran compiler on their >> machine, even if they do not want to actually compile a new fortran >> module component, simply because f2py will not operate unless it is >> offered at least one fortran file. > > This is not a typical task for f2py. f2py is not a general purpose > linker. It's amazing that f2py could even be used for such a task, > so I don't think that the above demonstrates any bug of f2py. Indeed not typical ... as I recognise ... which is why I wondered if there was an undocumented way to tweak to do what I want to do (I get requests like this on my own software all the time;-). > > However, if you are worried about whether users have fortran compilers > installed then can you assume that they have a C compiler installed? > If so, then instead of trivial Fortran file try using the following > trivial .pyf file: > > python module dummy > interface > subroutine dummyfunc() > fortranname > callstatement ; > end subroutine dummyfunc > end interface > end python module dummy > > that should force f2py to build a shared library dummy.so > with no Fortran dependencies. Perfect ... > Prof Garry Willgoose, Australian Professorial Fellow in Environmental Engineering, Director, Centre for Climate Impact Management (C2IM), School of Engineering, The University of Newcastle, Callaghan, 2308 Australia. Centre webpage: www.c3im.org.au Phone: (International) +61 2 4921 6050 (Tues-Fri AM); +61 2 6545 9574 (Fri PM-Mon) FAX: (International) +61 2 4921 6991 (Uni); +61 2 6545 9574 (personal and Telluric) Env. Engg. Secretary: (International) +61 2 4921 6042 email: [EMAIL PROTECTED]; [EMAIL PROTECTED] email-for-life: [EMAIL PROTECTED] personal webpage: www.telluricresearch.com/garry "Do not go where the path may lead, go instead where there is no path and leave a trail" Ralph Waldo Emerson ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
