Re: [Numpy-discussion] Deprecating PyDataMem_RENEW ?

2008-05-06 Thread David Cournapeau
Anne Archibald wrote: > > How much does this matter? I mean, what if you simply left all the > reallocs as-is? The arrays that resulted from reallocs would not > typically be aligned, but we cannot in any case expect all arrays to > be aligned. The problem would be the interaction between the alig

Re: [Numpy-discussion] Deprecating PyDataMem_RENEW ?

2008-05-06 Thread Anne Archibald
2008/5/5 David Cournapeau <[EMAIL PROTECTED]>: > Basically, what I have in mind is, in a first step (for numpy 1.2): > - define functions to allocate on a given alignement > - make PyMemData_NEW 16 byte aligned by default (to be compatible > with SSE and co). > > The problem was, and st

Re: [Numpy-discussion] Deprecating PyDataMem_RENEW ?

2008-05-06 Thread David Cournapeau
Robert Kern wrote: > > Since there are only 6 places where PyMemData_RENEW is used, all 6 > uses should be benchmarked. I would prefer a more targeted benchmark > so we know exactly what we are measuring. > Ok, I started a new branch for aligned allocator: http://projects.scipy.org/scipy/numpy/br

Re: [Numpy-discussion] Deprecating PyDataMem_RENEW ?

2008-05-05 Thread Robert Kern
On Mon, May 5, 2008 at 1:30 PM, David Cournapeau <[EMAIL PROTECTED]> wrote: > On Tue, May 6, 2008 at 2:11 AM, Robert Kern <[EMAIL PROTECTED]> wrote: > > > > I am in favor of at least trying this out. We will have to have a set > > of benchmarks to make sure we haven't hurt the current uses of

Re: [Numpy-discussion] Deprecating PyDataMem_RENEW ?

2008-05-05 Thread Anne Archibald
2008/5/5 Robert Kern <[EMAIL PROTECTED]>: > On Mon, May 5, 2008 at 7:44 AM, David Cournapeau > <[EMAIL PROTECTED]> wrote: > > > In numpy, we can always replace realloc by malloc/free, because we know > > the size of the old block: would deprecating PyMemData_RENEW and > > replacing them by Py

Re: [Numpy-discussion] Deprecating PyDataMem_RENEW ?

2008-05-05 Thread David Cournapeau
On Tue, May 6, 2008 at 2:11 AM, Robert Kern <[EMAIL PROTECTED]> wrote: > > I am in favor of at least trying this out. We will have to have a set > of benchmarks to make sure we haven't hurt the current uses of > PyMemData_RENEW which Tim points out. What would be a good stress test for PyArray_

Re: [Numpy-discussion] Deprecating PyDataMem_RENEW ?

2008-05-05 Thread David Cournapeau
On Tue, May 6, 2008 at 1:59 AM, Timothy Hochberg <[EMAIL PROTECTED]> wrote: > I don't think you would want to do this in the core of PyArray_FromIter; > presumably realloc can sometimes reuse the existing pointer and save on > allocating a new chunk of memory. Since there are lots of allocations i

Re: [Numpy-discussion] Deprecating PyDataMem_RENEW ?

2008-05-05 Thread Robert Kern
On Mon, May 5, 2008 at 7:44 AM, David Cournapeau <[EMAIL PROTECTED]> wrote: > In numpy, we can always replace realloc by malloc/free, because we know > the size of the old block: would deprecating PyMemData_RENEW and > replacing them by PyMemeData_NEW/PyMemData_FREE be possible, such as to > ma

Re: [Numpy-discussion] Deprecating PyDataMem_RENEW ?

2008-05-05 Thread Timothy Hochberg
On Mon, May 5, 2008 at 5:44 AM, David Cournapeau < [EMAIL PROTECTED]> wrote: > Hi, > >While working again on the fftpack module, to clean things up and > speed some backends (in particular fftw3, which is really sub-optimal > right now), I remembered how much unaligned data pointer in numpy ar

[Numpy-discussion] Deprecating PyDataMem_RENEW ?

2008-05-05 Thread David Cournapeau
Hi, While working again on the fftpack module, to clean things up and speed some backends (in particular fftw3, which is really sub-optimal right now), I remembered how much unaligned data pointer in numpy arrays hurt performances. So I would like to relaunch the discussion on aligned allo