On Mon, May 5, 2008 at 5:44 AM, David Cournapeau < [EMAIL PROTECTED]> wrote:
> Hi, > > While working again on the fftpack module, to clean things up and > speed some backends (in particular fftw3, which is really sub-optimal > right now), I remembered how much unaligned data pointer in numpy arrays > hurt performances. So I would like to relaunch the discussion on aligned > allocators and default alignement for numpy arrays : > > http://www.mail-archive.com/numpy-discussion@scipy.org/msg04005.html > > Basically, what I have in mind is, in a first step (for numpy 1.2): > - define functions to allocate on a given alignement > - make PyMemData_NEW 16 byte aligned by default (to be compatible > with SSE and co). > > The problem was, and still is, realloc. It is not possible to implement > realloc with malloc/free (in a portable way), and as such, it is not > possible to have an aligned realloc. > > In numpy, we can always replace realloc by malloc/free, because we know > the size of the old block: would deprecating PyMemData_RENEW and > replacing them by PyMemeData_NEW/PyMemData_FREE be possible, such as to > make all numpy arrays follow a default alignement ? There are only a few > of them in numpy (6 of them), 0 in scipy, and I guess extensions never > really used them ? I don't think you would want to do this in the core of PyArray_FromIter; presumably realloc can sometimes reuse the existing pointer and save on allocating a new chunk of memory. Since there are lots of allocations in fromiter, this could potentially be a big performance hit. (At least I think so, realloc has always been kinda voodoo to me). One could use PyMemData_NEW/PyMemData_FREE in the final allocation to make sure that the data is alligned, we allready do a realloc there to dump any extra space. Or, possibly better, one could choose which allocation strategy to use here depending on whether the data was alligned or not. > > > cheers, > > David > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion@scipy.org > http://projects.scipy.org/mailman/listinfo/numpy-discussion > -- . __ . |-\ . . [EMAIL PROTECTED]
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion