Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Simon Palmer
Pretty simple.  I don't do any transformations.  It is a euclidean distance
matrix between n vectors in my data space, so I use it for lookup of minima
and I recalculate portions of it during the processing of my algorithm.

It is the single largest limitation of my code, both in terms of performance
and scalability.  A fast and efficient solution to this issue would make a
huge difference to me.

On Wed, Jun 11, 2008 at 1:05 AM, Robert Kern [EMAIL PROTECTED] wrote:

 On Tue, Jun 10, 2008 at 18:53, Simon Palmer [EMAIL PROTECTED]
 wrote:
  Hi I have a problem which involves the creation of a large square matrix
  which is zero across its diagonal and symmetrical about the diagonal i.e.
  m[i,j] = m[j,i] and m[i,i] = 0.  So, in fact, it is a large triangular
  matrix.  I was wondering whether there is any way of easily handling a
  matrix of this shape without either incurring a memory penalty or a whole
  whack of proprietary code?
 
  To get through this I have implemented a 1D array which has ((n-1)^2)/2
  elements inside a wrapper class which manpulates the arguments of array
  accessors with some arithmetic to return the approriate value.  To be
 honest
  I'd love to throw this away, but I haven't yet come across a feasible
  alternative.
 
  Any ideas?

 What operations do you want to perform on this array?

 --
 Robert Kern

 I have come to believe that the whole world is an enigma, a harmless
 enigma that is made terrible by our own mad attempt to interpret it as
 though it had an underlying truth.
  -- Umberto Eco
 ___
 Numpy-discussion mailing list
 Numpy-discussion@scipy.org
 http://projects.scipy.org/mailman/listinfo/numpy-discussion

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Simon Palmer
clustering yes, hierarchical no.

On Wed, Jun 11, 2008 at 4:55 PM, Charles R Harris [EMAIL PROTECTED]
wrote:



 On Wed, Jun 11, 2008 at 9:33 AM, Simon Palmer [EMAIL PROTECTED]
 wrote:

 Pretty simple.  I don't do any transformations.  It is a euclidean
 distance matrix between n vectors in my data space, so I use it for lookup
 of minima and I recalculate portions of it during the processing of my
 algorithm.

 It is the single largest limitation of my code, both in terms of
 performance and scalability.  A fast and efficient solution to this issue
 would make a huge difference to me.


 So are you doing hierarchical clustering?

 Chuck



 ___
 Numpy-discussion mailing list
 Numpy-discussion@scipy.org
 http://projects.scipy.org/mailman/listinfo/numpy-discussion


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Peter Skomoroch
You could do something like this, where MyDIstanceFunc is written with weave
and some intelligent caching.  This only computes the upper diagonal
elements in a sparse matrix:

self.NumDatapoints = len(self.DataPoints)
self.Distances = sparse.lil_matrix((self.NumDatapoints,self.NumDatapoints))

for index, x in enumerate(self.DataPoints):
self.Distances[index,0:index] = array(map(lambda y:self.MyDistanceFunc(
x, y ), self.DataPoints[0:index] ))

If an approximation is ok, you can save space by only calculating entries
for random samples of the rows and doing a matrix factorization like NMF on
the matrix with missing elements.  Depending on the dataset and accuracy
needed, this can be a big space savings.





On Wed, Jun 11, 2008 at 8:55 AM, Charles R Harris [EMAIL PROTECTED]
wrote:



 On Wed, Jun 11, 2008 at 9:33 AM, Simon Palmer [EMAIL PROTECTED]
 wrote:

 Pretty simple.  I don't do any transformations.  It is a euclidean
 distance matrix between n vectors in my data space, so I use it for lookup
 of minima and I recalculate portions of it during the processing of my
 algorithm.

 It is the single largest limitation of my code, both in terms of
 performance and scalability.  A fast and efficient solution to this issue
 would make a huge difference to me.


 So are you doing hierarchical clustering?

 Chuck



 ___
 Numpy-discussion mailing list
 Numpy-discussion@scipy.org
 http://projects.scipy.org/mailman/listinfo/numpy-discussion




-- 
Peter N. Skomoroch
[EMAIL PROTECTED]
http://www.datawrangling.com
http://del.icio.us/pskomoroch
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Bruce Southey
Charles R Harris wrote:


 On Wed, Jun 11, 2008 at 9:33 AM, Simon Palmer [EMAIL PROTECTED] 
 mailto:[EMAIL PROTECTED] wrote:

 Pretty simple.  I don't do any transformations.  It is a euclidean
 distance matrix between n vectors in my data space, so I use it
 for lookup of minima and I recalculate portions of it during the
 processing of my algorithm. 

 It is the single largest limitation of my code, both in terms of
 performance and scalability.  A fast and efficient solution to
 this issue would make a huge difference to me.


 So are you doing hierarchical clustering?

 Chuck


 

 ___
 Numpy-discussion mailing list
 Numpy-discussion@scipy.org
 http://projects.scipy.org/mailman/listinfo/numpy-discussion
   
Lapack supports symmetric matrices (http://en.wikipedia.org/wiki/LAPACK) 
but, to my knowledge, these are not implemented in SciPy (and I am not 
sure if sparse will help).  So you have the memory penalty or 
performance penalty unless you create the bindings necessary that you need.

Bruce

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Keith Goodman
On Wed, Jun 11, 2008 at 9:47 AM, Simon Palmer [EMAIL PROTECTED] wrote:
 clustering yes, hierarchical no.

I think scipy-cluster [1] only stores the upper triangle of the
distance matrix.

[1] http://code.google.com/p/scipy-cluster/
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Anne Archibald
2008/6/10 Simon Palmer [EMAIL PROTECTED]:
 Hi I have a problem which involves the creation of a large square matrix
 which is zero across its diagonal and symmetrical about the diagonal i.e.
 m[i,j] = m[j,i] and m[i,i] = 0.  So, in fact, it is a large triangular
 matrix.  I was wondering whether there is any way of easily handling a
 matrix of this shape without either incurring a memory penalty or a whole
 whack of proprietary code?

 To get through this I have implemented a 1D array which has ((n-1)^2)/2
 elements inside a wrapper class which manpulates the arguments of array
 accessors with some arithmetic to return the approriate value.  To be honest
 I'd love to throw this away, but I haven't yet come across a feasible
 alternative.

 Any ideas?

Well, there's a risk of shooting yourself in the foot, but here's a
way to get at the elements quickly and conveniently: store the
coefficients in a 1D array (as you do already). Then if your matrix is
M by M, element (i,j) is at position (M-2)*j+i-1, assuming ij. Of
course, you can simply use this expression every time you want to look
an element up, as you do now, but that's no fun. So now create a
view A of this array having offset -1, shape (M,M), and strides
(1,(M-2)). (This kind of weird view can be created with the ndarray
constructor.) Then A[i,j] addresses the right element. A[j,i] is of
course totally wrong. But as long as you avoid ij, you can use all
numpy's magical indexing tricks. Unfortunately you can't really use
ufuncs, since they'll waste time operating on the useless lower half.

Personally, I would live with the factor-of-two memory hit.

Anne
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Simon Palmer
Actually that is very close to what I currently do, which is why I want to
throw it away.

The factor of two memory hit is not really the problem, it is the scaling
O(N^2) which is the limitation

I suspect I may end up using the 1-d array plus arithmetic as it is at least
efficient for retrieval.  I have just started working on a stack which sits
between the vector array and a lookup for a minimum, I think that could
improve performance a great deal.  Next up is a disk based memory map of
some kind which might free me from the constraints of physical memory.
Anyone know of an implementation of array() which automatically reads/writes
to disk?

To answer another suggestion, I'm reluctant to sample not because it is not
a good approach but rather because the commercial application at the end of
this is much harder to explain, justify and sell if I have to include
reasoning for not using all the data.  As scientists this is obvious, as a
lay person, and particularly a business person, it appears to be madness.
The fewer leaps of faith necessary the better.

On Wed, Jun 11, 2008 at 7:34 PM, Anne Archibald [EMAIL PROTECTED]
wrote:

 2008/6/10 Simon Palmer [EMAIL PROTECTED]:
  Hi I have a problem which involves the creation of a large square matrix
  which is zero across its diagonal and symmetrical about the diagonal i.e.
  m[i,j] = m[j,i] and m[i,i] = 0.  So, in fact, it is a large triangular
  matrix.  I was wondering whether there is any way of easily handling a
  matrix of this shape without either incurring a memory penalty or a whole
  whack of proprietary code?
 
  To get through this I have implemented a 1D array which has ((n-1)^2)/2
  elements inside a wrapper class which manpulates the arguments of array
  accessors with some arithmetic to return the approriate value.  To be
 honest
  I'd love to throw this away, but I haven't yet come across a feasible
  alternative.
 
  Any ideas?

 Well, there's a risk of shooting yourself in the foot, but here's a
 way to get at the elements quickly and conveniently: store the
 coefficients in a 1D array (as you do already). Then if your matrix is
 M by M, element (i,j) is at position (M-2)*j+i-1, assuming ij. Of
 course, you can simply use this expression every time you want to look
 an element up, as you do now, but that's no fun. So now create a
 view A of this array having offset -1, shape (M,M), and strides
 (1,(M-2)). (This kind of weird view can be created with the ndarray
 constructor.) Then A[i,j] addresses the right element. A[j,i] is of
 course totally wrong. But as long as you avoid ij, you can use all
 numpy's magical indexing tricks. Unfortunately you can't really use
 ufuncs, since they'll waste time operating on the useless lower half.

 Personally, I would live with the factor-of-two memory hit.

 Anne
 ___
 Numpy-discussion mailing list
 Numpy-discussion@scipy.org
 http://projects.scipy.org/mailman/listinfo/numpy-discussion

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Hoyt Koepke
 The factor of two memory hit is not really the problem, it is the scaling
 O(N^2) which is the limitation

Have you thought about using kd-trees or similar structures?  If you
are only concerned about the minimum distances between two points,
that is the ideal data structure.  It would reduce your running time /
memory usage for generating / using the data from O(n^2) to O(n log
n), with a O(log n) lookup time for the closest distance to each
point.

Without doing something like that, looking at the distance between
each pair of points is \Omega(n^2).

--Hoyt

-- 
+++
Hoyt Koepke
UBC Department of Computer Science
http://www.cs.ubc.ca/~hoytak/
[EMAIL PROTECTED]
+++
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Robert Kern
On Wed, Jun 11, 2008 at 15:14, Simon Palmer [EMAIL PROTECTED] wrote:
 Actually that is very close to what I currently do, which is why I want to
 throw it away.

 The factor of two memory hit is not really the problem, it is the scaling
 O(N^2) which is the limitation

 I suspect I may end up using the 1-d array plus arithmetic as it is at least
 efficient for retrieval.  I have just started working on a stack which sits
 between the vector array and a lookup for a minimum, I think that could
 improve performance a great deal.  Next up is a disk based memory map of
 some kind which might free me from the constraints of physical memory.
 Anyone know of an implementation of array() which automatically reads/writes
 to disk?

Well, if you have a decent OS, you should already be free of the
constraints of physical memory. The OS should swap data in and out as
necessary. But of course, we do have mmap support with numpy.memmap.
If you are using numpy 1.1.0, I would also recommend using
numpy.lib.format.open_memmap() to use the new NPY file format in order
to (hopefully) future-proof your data. Be careful with using mmap'ed
arrays, though. Disk seek time could kill your performance.

If this is the big hotspot in your program, you might want to consider
coding up exactly the behavior you need in C. It doesn't appea

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
 -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Switching to nose test framework (was: NumpyTest problem)

2008-06-11 Thread Barry Wark
Could this also be added as a comment in the  module docstring for
test modules? It seems that anyone hacking on test code would be most
likely to see it there...?

Barry

On Tue, Jun 10, 2008 at 2:57 AM, Stéfan van der Walt [EMAIL PROTECTED] wrote:
 2008/6/10 Alan McIntyre [EMAIL PROTECTED]:
 Is the stuff Robert pointed out on a wiki page somewhere? It would be
 nice to have a Welcome noob NumPy developer, here's how to do
 NumPy-specific development things, page. There may be such a page,
 but I just haven't stumbled across it yet.

 We could add this info the the `numpy` docstring; that way new users
 will have it available without having to search the web.

 http://sd-2116.dedibox.fr/pydocweb/doc/numpy/

 Regards
 Stéfan
 ___
 Numpy-discussion mailing list
 Numpy-discussion@scipy.org
 http://projects.scipy.org/mailman/listinfo/numpy-discussion

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Switching to nose test framework (was: NumpyTest problem)

2008-06-11 Thread Robert Kern
On Tue, Jun 10, 2008 at 04:57, Stéfan van der Walt [EMAIL PROTECTED] wrote:
 2008/6/10 Alan McIntyre [EMAIL PROTECTED]:
 Is the stuff Robert pointed out on a wiki page somewhere? It would be
 nice to have a Welcome noob NumPy developer, here's how to do
 NumPy-specific development things, page. There may be such a page,
 but I just haven't stumbled across it yet.

 We could add this info the the `numpy` docstring; that way new users
 will have it available without having to search the web.

Build instructions don't really belong in docstrings. Put it in the
README.txt or DEV_README.txt.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
 -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] array vs matrix

2008-06-11 Thread Ondrej Certik
On Sun, Jun 8, 2008 at 3:54 AM, Anne Archibald
[EMAIL PROTECTED] wrote:
 2008/6/7 Robert Kern [EMAIL PROTECTED]:
 On Sat, Jun 7, 2008 at 14:37, Ondrej Certik [EMAIL PROTECTED] wrote:
 Hi,

 what is the current plan with array and matrix with regard of calculating

 sin(A)

 ? I.e. elementwise vs sin(A) = Q*sin(D)*Q^T? Is the current approach
 (elementwise for array and Q*sin(D)*Q^T for matrix) the way to go?

 I don't believe we intend to make numpy.matrix any more featureful. I
 don't think it's a good idea for you to base sympy.Matrix's
 capabilities in lock-step with numpy.matrix, though. There are very
 different constraints at work. Please, do what you think is best for
 sympy.Matrix.

 Let me elaborate somewhat:

 We recently ran across some painful quirks in numpy's handling of
 matrices, and they spawned massive discussion. As it stands now, there
 is significant interest in reimplementing the matrix object from
 scratch, with different behaviour. So emulating its current behaviour
 is not a win.

 For consistency, it makes a certain amount of sense to have sin(A)
 compute a matrix sine, since A**n computes a matrix power. But looking
 at the matrix exponential, I see that we have several implementations,
 none of which is satisfactory for all matrices. I would expect the
 matrix sine to be similar - particularly when faced with complex
 matrices - so perhaps needing an explicit matrix sine is a good thing?

 Also worth noting is that this problem can be evaded with namespaces;
 matrix sin could be scipy.matrixmath.sin, abbreviated perhaps to
 mm.sin, as opposed to np.sin.

I see. Thanks Robert and Anne for the replies. That's all I needed to know.

Ondrej
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Switching to nose test framework (was: NumpyTest problem)

2008-06-11 Thread Alan McIntyre
On Wed, Jun 11, 2008 at 6:09 PM, Robert Kern [EMAIL PROTECTED] wrote:
 On Tue, Jun 10, 2008 at 04:57, Stéfan van der Walt [EMAIL PROTECTED] wrote:
 2008/6/10 Alan McIntyre [EMAIL PROTECTED]:
 Is the stuff Robert pointed out on a wiki page somewhere? It would be
 nice to have a Welcome noob NumPy developer, here's how to do
 NumPy-specific development things, page. There may be such a page,
 but I just haven't stumbled across it yet.

 We could add this info the the `numpy` docstring; that way new users
 will have it available without having to search the web.

 Build instructions don't really belong in docstrings. Put it in the
 README.txt or DEV_README.txt.

I agree; I'm far more likely to look at the READMEs than in module
docstrings for build/test instructions.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Switching to nose test framework (was: NumpyTest problem)

2008-06-11 Thread Stéfan van der Walt
2008/6/12 Robert Kern [EMAIL PROTECTED]:
 On Tue, Jun 10, 2008 at 04:57, Stéfan van der Walt [EMAIL PROTECTED] wrote:
 2008/6/10 Alan McIntyre [EMAIL PROTECTED]:
 Is the stuff Robert pointed out on a wiki page somewhere? It would be
 nice to have a Welcome noob NumPy developer, here's how to do
 NumPy-specific development things, page. There may be such a page,
 but I just haven't stumbled across it yet.

 We could add this info the the `numpy` docstring; that way new users
 will have it available without having to search the web.

 Build instructions don't really belong in docstrings. Put it in the
 README.txt or DEV_README.txt.

No, but running the tests is something every user should do before
using NumPy.  Unless you object, I'll add that bit to the docstring.

Cheers
Stéfan
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] Switching to nose test framework (was: NumpyTest problem)

2008-06-11 Thread Robert Kern
On Wed, Jun 11, 2008 at 19:51, Stéfan van der Walt [EMAIL PROTECTED] wrote:
 2008/6/12 Robert Kern [EMAIL PROTECTED]:
 On Tue, Jun 10, 2008 at 04:57, Stéfan van der Walt [EMAIL PROTECTED] wrote:
 2008/6/10 Alan McIntyre [EMAIL PROTECTED]:
 Is the stuff Robert pointed out on a wiki page somewhere? It would be
 nice to have a Welcome noob NumPy developer, here's how to do
 NumPy-specific development things, page. There may be such a page,
 but I just haven't stumbled across it yet.

 We could add this info the the `numpy` docstring; that way new users
 will have it available without having to search the web.

 Build instructions don't really belong in docstrings. Put it in the
 README.txt or DEV_README.txt.

 No, but running the tests is something every user should do before
 using NumPy.

Right, which means README.txt, not something you only see after you've
installed the thing. But what text exactly are you talking about?
Nothing I wrote about inplace builds (which is I *thought* was under
discussion) are relevant to running the tests in general but are
targeted specifically at developers.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
 -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


[Numpy-discussion] [numscons] Would having two mandatory scons script per package be acceptable ?

2008-06-11 Thread David Cournapeau
Hi,

Short story: it would make my life simpler to have two mandatory 
scons scripts per package instead of only one. Long story: for some 
reasons linked to scons' notion of build directory, having two scons 
scripts files per package (instead of one) would simplify quite a bit 
some parts of numscons, as well as simplifying the addition of custom 
builders (for swig, cython, etc...) by other people. One of the two 
files would basically be exactly the same as the current one, and the 
other one independent of the package (always the same 3 lines). This is 
a bit ugly, but less ugly than the actual contortions I have to do to 
make scons and distutils play nice together.
Is there any strong feeling against this ?

cheers,

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] [numscons] Would having two mandatory scons script per package be acceptable ?

2008-06-11 Thread Robert Kern
On Wed, Jun 11, 2008 at 23:35, David Cournapeau
[EMAIL PROTECTED] wrote:
 Hi,

Short story: it would make my life simpler to have two mandatory
 scons scripts per package instead of only one. Long story: for some
 reasons linked to scons' notion of build directory, having two scons
 scripts files per package (instead of one) would simplify quite a bit
 some parts of numscons, as well as simplifying the addition of custom
 builders (for swig, cython, etc...) by other people. One of the two
 files would basically be exactly the same as the current one, and the
 other one independent of the package (always the same 3 lines). This is
 a bit ugly, but less ugly than the actual contortions I have to do to
 make scons and distutils play nice together.
Is there any strong feeling against this ?

Can you give me a longer story? What are the three lines? Why are they
necessary in a separate, boilerplate file?

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
 -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] [numscons] Would having two mandatory scons script per package be acceptable ?

2008-06-11 Thread David Cournapeau
Robert Kern wrote:

 Can you give me a longer story? What are the three lines? Why are they
 necessary in a separate, boilerplate file?

Sure: scons builds everything from the sources in the source directory, 
and build everything in the build directory.  By default, both 
directories are the same as the directory where the scons script is.

Now, this is of course unacceptable to use with distutils, and scons has 
this notion of build directory to change this: depending on where the 
scons script is called in the tree, I change the source and build 
directories accordingly. But for this to work, you have to change how to 
refer to source and targets in builders (scons objects to build things 
like object code and libraries). Again, unacceptable, so I did remove 
most of those in the scons scripts using severals 'tricks' in numscons. 
But the more I did those tricks, the more I realized it was fragile, and 
could break in obscure cases.

The recommended way to do it in scons is to use a second scons script: 
http://www.scons.org/wiki/UnderstandingBuildDir. The 'problem' is also 
linked to the fact that instead of building everything with one scons 
project, I have one scons project per package (because that's how 
distutils worked, in particular, and for other reasons I can develop if 
you want). I asked about doing the same thing as scons does without two 
files, but this seems difficult (I was told it is 'magic').

http://scons.tigris.org/servlets/ReadMsg?list=usersmsgNo=13642

cheers,

David




___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] [numscons] Would having two mandatory scons script per package be acceptable ?

2008-06-11 Thread Robert Kern
On Thu, Jun 12, 2008 at 00:08, David Cournapeau
[EMAIL PROTECTED] wrote:
 Robert Kern wrote:

 Can you give me a longer story? What are the three lines? Why are they
 necessary in a separate, boilerplate file?

 Sure: scons builds everything from the sources in the source directory,
 and build everything in the build directory.  By default, both
 directories are the same as the directory where the scons script is.

 Now, this is of course unacceptable to use with distutils, and scons has
 this notion of build directory to change this: depending on where the
 scons script is called in the tree, I change the source and build
 directories accordingly. But for this to work, you have to change how to
 refer to source and targets in builders (scons objects to build things
 like object code and libraries). Again, unacceptable, so I did remove
 most of those in the scons scripts using severals 'tricks' in numscons.
 But the more I did those tricks, the more I realized it was fragile, and
 could break in obscure cases.

 The recommended way to do it in scons is to use a second scons script:
 http://www.scons.org/wiki/UnderstandingBuildDir. The 'problem' is also
 linked to the fact that instead of building everything with one scons
 project, I have one scons project per package (because that's how
 distutils worked, in particular, and for other reasons I can develop if
 you want). I asked about doing the same thing as scons does without two
 files, but this seems difficult (I was told it is 'magic').

 http://scons.tigris.org/servlets/ReadMsg?list=usersmsgNo=13642

Works for me.

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
 -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] [numscons] Would having two mandatory scons script per package be acceptable ?

2008-06-11 Thread David Cournapeau
Robert Kern wrote:
 Works for me.
   

Do you mean having two files/packages is ok ?

cheers,

David

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] [numscons] Would having two mandatory scons script per package be acceptable ?

2008-06-11 Thread Robert Kern
On Thu, Jun 12, 2008 at 00:28, David Cournapeau
[EMAIL PROTECTED] wrote:
 Robert Kern wrote:
 Works for me.

 Do you mean having two files/packages is ok ?

Yes. You had me at The recommended way to do it in scons 

-- 
Robert Kern

I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth.
 -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


[Numpy-discussion] [numscons] numscons build for numpy will be broken for a few hours

2008-06-11 Thread David Cournapeau
Hi,

Just to mention that building numpy and scipy with numscons will be 
broken for a few hours, the time to change some organizations.

Sorry for the trouble,

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion