Re: [Numpy-discussion] For review: first milestone of scons support in numpy
Pearu Peterson wrote:
Yes. I was considering the situations where distutils in not used at
all. May be it will be relevant in future..
If distutils is not used at all, there will be more difficult problems
to solve than this one, then. I think it is safe to assume that
distutils will stay there for quite some time, no ?
Note that I try hard in numpy/scons to separate the scons specific from
the parts which are not specific to scons. If in the future, we decide
to go on the waf route, or whatever, I hope that most of the logic would
be easily reused: for example, parsing the fortran link output to guess
the link options for C/Fortran is totally independant from scons (this
has the nice advantage of making regression tests possible on all
platforms).
I would prefer the former solution where each package is totally
independent. Let's first see what is the acctual performance and then
maybe consider more complex solutions if the build is really slow.
I measured the initialization time of scons if called for every
subpackage in numpy/: this adds less than two percents of total build
time, so this is not as bad as I feared (specially since only a few
package would need scons: pure python ones do not need them). And scons
can also give some speed advantage, because it explicitely support in a
totally safe way parallel builds.
So basically, I went to the same conclusion than you. I have started
porting numpy.core to scons (by far the most difficult subpackage, at
least a priori) following this route.
These macros are fine. May be we need more such macros for F90 compilers
but we can discuss about these when the corresponding issues will emerge.
Oh, I forgot to say it, but the macro would be implemented for both F77
and F90 (and F2003 or whatever should we need it). As most of the logic
is independent of the version used, this is not a problem.
However, if you could also provide the macros
that f2py uses then everything would work with no change in f2py part.
f2py uses the following defintion of F_FUNC, for example:
I can of course change the name of the macro (this could be an option,
even). I guess it is better to follow f2py convention than autotools
here, in our case.
system_info has a feature of providing architecture specific
and optimization flags for fortran compilers. Could scons
support such a feature?
scons uses the concept of Environment objects: they have sensible
defaults (compiler/platform combination), but you can override anything
you want, and they are used for compilation.
env = Environment()
env.AppendUnique(F77FLAGS = -O3)
env.Program('hello', source = ['hello.f'])
Of course, in our case, we should provide some support such as instead
of adding -O3, we have something like a FULL_OPTIMIZED variable which
contains the optimization flags tweaked for a particular
compiler/platform. Better, we can check that a given flag is actually
supported. This is something I have not done yet, but that's the kind of
things scons does really well, so I don't see any problems here.
The main problem right now is to convert all the checks in
numpy/core/setup.py, which is the only non trivial setup.py file as far
as I can tell in numpy. This will take a few hours to get it right on
major platforms :)
cheers,
David
P.S If you have access to some fortran compiler you would like to check,
you can check my bzr branch on launchpad (this is done outside numpy
because some things may be merged to scons sources at some point):
https://code.launchpad.net/~david-ar/numpy.scons.support/dev
And test it by: scons CC=ccompiler F77=f77compiler F90=f90compiler. For
example: scons CC=icc F77=g77 F90=ifort.
(If you change compiler, you may need to add the option --config=force).
This is a fake sconscript which tests the following:
- does the fortran compiler works (e.g. can compile a dummy program)
- find the dummy main
- find the verbose flag (useful to find informations at runtime)
- find the name mangling
- find the necessary options for C/Fortran mix (only implemented for
F77 for now).
There is a pretty good chance that you will be able to break it (in
particular finding the options for C/Fortran: I have not yet implemented
all the necessary parsing), but in this case, please provides me with
the config.log (which logs all the configuration dones by scons).
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Use of PyArray FromAny() with a PyArrayInterface object
2007/10/12, Alan G Isaac [EMAIL PROTECTED]: On Fri, 12 Oct 2007, Matthieu Brucher apparently wrote: I'm trying to understand (but perhaps everything is in the numpy book in which case I'd rather buy the book immediately) how to use the PyArray_FromAny() function. This function is discussed in the NumPy Book. I see perhaps a dozen references to the function in the Complete API chapter. Even apart from that chapter, which is useful to you but not to me, I have found the book to be excellent. Excellent, thank you for the tip, the book is now ordered :) Matthieu ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Use of PyArray FromAny() wi th a PyArrayInterface object
On Fri, 12 Oct 2007, Matthieu Brucher apparently wrote: I'm trying to understand (but perhaps everything is in the numpy book in which case I'd rather buy the book immediately) how to use the PyArray_FromAny() function. This function is discussed in the NumPy Book. I see perhaps a dozen references to the function in the Complete API chapter. Even apart from that chapter, which is useful to you but not to me, I have found the book to be excellent. Cheers, Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Use of PyArray FromAny() with a PyArrayInterface object
I don't see the text string '__array' anywhere in this book. I saw something a subchapter with the PyArrayInterface, nothing around it with __array__ or __array_struct__ ? Matthieu (wating for the book :)) ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Use of PyArray FromAny() with a PyArrayInterface object
Matthieu Brucher wrote: 2007/10/12, Alan G Isaac [EMAIL PROTECTED]: On Fri, 12 Oct 2007, Matthieu Brucher apparently wrote: I'm trying to understand (but perhaps everything is in the numpy book in which case I'd rather buy the book immediately) how to use the PyArray_FromAny() function. This function is discussed in the NumPy Book. I see perhaps a dozen references to the function in the Complete API chapter. Even apart from that chapter, which is useful to you but not to me, I have found the book to be excellent. Excellent, thank you for the tip, the book is now ordered :) Matthieu I don't see the text string '__array' anywhere in this book. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Use of PyArray_FromAny() with a PyArrayInterface object
Hi, I'm trying to understand (but perhaps everything is in the numpy book in which case I'd rather buy the book immediately) how to use the PyArray_FromAny() function. Suppose I have a C object (created by Pyrex or SWIG or Boost.Python) that has the __array_struct__ attribute. Can I pass this object to the PyArray_FromAny() function to get a PyArrayObject ? If it is not the case, if I have a PyCObject created with a PyArrayInterface, is it possible ? I read on a mailing-list that there is the context argument that can be passed to an __array__ method. What is the purpose of this __array__ method ? I don't recollect anything regarding this method :( Matthieu ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Use of PyArray FromAny() with a PyArrayInterface object
Neal Becker wrote: Matthieu Brucher wrote: 2007/10/12, Alan G Isaac [EMAIL PROTECTED]: On Fri, 12 Oct 2007, Matthieu Brucher apparently wrote: I'm trying to understand (but perhaps everything is in the numpy book in which case I'd rather buy the book immediately) how to use the PyArray_FromAny() function. This function is discussed in the NumPy Book. I see perhaps a dozen references to the function in the Complete API chapter. Even apart from that chapter, which is useful to you but not to me, I have found the book to be excellent. Excellent, thank you for the tip, the book is now ordered :) Matthieu I don't see the text string '__array' anywhere in this book. Perhaps PDF searching cannot see bold text. But, take a look at page 74. -Travis ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] abstraction and interfaces
Hello there, I use NumPy/SciPy to process telemetry, producing some metrics that I plot with matplotlib. Well, I want to package my scripts into a sort of all-wrapping application, i.e, the user just calls the app, selects which metrics to produce and waits to obtain the plots, without being exposed to anything more dangerous than a zoom button.. The scripts that produce these metrics use Scipy/Numpy functions that operate on data conveniently converted to numpy arrays. They're quite specific, and I tend to produce/tweak a lot of them. So, to fit in this application someone suggested I programmed 'interfaces' (in java jargon) to them - that way I could develop the whole wrapper application without giving much thought to the actual number-crunching bits. Problem is, I learnt numerics using Fortran, and though I understand the idea of objects, I have quite some difficulty imagining the 'patterns' that allow OO to be so flexible. So I was wondering if anyone experienced with OO Numerical computing could provide some pointers. For context, here are my thoughts: 1. the core of the application is responsible for interaction with the outside world. Crunching blocks are provided data and return matplotlib objects that go into GTK. To write procedures that abide to these parameters is my first (fortranic) impulse. But then I need to change the core everytime I add a procedure, and I'm limited to numpy-IN/matplolib-OUT. 2. if I could force every analysis procedure to be a class that implements its own data retrieval and outputs matplolib objects, my core no longer controls the whole communication with the outside world. And i still have to edit the core everytime a new analysis is required. A very interesting thing is that with these classes I could create new objects with different attributes, rather than pass a host of parameters to a procedure.. So, I'm a bit at odds with this brave new world I discovered. I'd be quite grateful for pointers on literature or examples that more or less lead to a solution. So far my searches have justified the need for objects, but not helped address this modularity requirement... Cheers, Renato ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] abstraction and interfaces
On Fri, 12 Oct 2007, Renato Serodio apparently wrote: The scripts that produce these metrics use Scipy/Numpy functions that operate on data conveniently converted to numpy arrays. They're quite specific, and I tend to produce/tweak a lot of them. So, to fit in this application someone suggested I programmed 'interfaces' (in java jargon) to them - that way I could develop the whole wrapper application without giving much thought to the actual number-crunching bits. That sounds quite right. Check out URL:https://projects.scipy.org/scipy/scikits/browser/trunk/openopt/scikits/openopt/solvers/optimizers/optimizer URL:http://svn.scipy.org/svn/scipy/trunk/scipy/stats/models/ for examples that may be relevant to your project. Python does not have interfaces per se, but that does not stop you from designing interface-like classes and inheriting from them. fwiw, Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
