Stefan wrote: > > I wonder how to best deal with the new buffer interface. > > http://www.python.org/dev/peps/pep-3118/ >
I like your general idea. I'm weak in the "C side of things" but I'll comment on some other aspects. Some more background: IIRC, the buffers interface comes from repeated attempts by Travis Oliphant of NumPy to get some kind of native support for data exchange of arrays (prime example being loading images with PIL and manipulate the images with NumPy). It therefore looks *very* similar to the NumPy data structures. This means that this stuff is potentially relevant for my summer of code. (I.e., any multidimensional array indexing that I create for NumPy might potentially be made to work for these buffers as well, keep that in mind below.) > This would normally call for two special functions __getbuffer__ and > __releasebuffer__. To me, however, this looks like an extremely C-ish > interface that does not fit Cython at all. So I'm wondering what others > think > about this approach: > > 1) define a pseudo-builtin PyxBuffer extension type that mimics a > Py_buffer > struct without the ".internal" field > 2) support a new special method __getbuffer__(self, int flags) that allows > users to create, fill and return an instance of PyxBuffer or a subtype > (maybe the PyxBuffer class could also contain a public enum that > defines > the flags?) > 3) generate glue code for the bf_getbuffer() call that > a) calls __getbuffer__() and raises an exception if it does not return > a > PyxBuffer instance > b) copies the Py_buffer struct content over from the returned PyxBuffer > object to the Py_buffer struct that was passed into bf_getbuffer() > c) stores an incref-ed reference to the PyxBuffer object in the > ".internal" > field > 4) generate generic code for the bf_releasebuffer slot that decrefs the > reference in the ".internal" field (and sets it to NULL), thus > eventually > calling __dealloc__() on the PyxBuffer object. User defined subtypes of > PyxBuffer can then do The Right Thing here. I'd like to mention to variations on this (without implying I think they are better, I just think they should be mentioned): One is more native support for buffers. I'm not sure how it could be made to work, but perhaps some kind of union with "CEP 512 - C arrays as first class types"; i.e., the dynamically allocated/deallocated C arrays would support the buffer interface. (I haven't thought much about this and whether there would be any overhead). If so, __getbuffer__ could simply return such a primitive array object with native syntax support; or one could drop __getbuffer__ and use a more generic coercion operator overload (and have coercion to a native array type be your __getbuffer__). The other idea I had is the completely opposite: Do provide support for the direct, low-level C manipulation, but also create a pxd-file that ships with Cython with a base class that implement __getbuffer__ in the way stated above, using Cython code. (It looks like inlineable code in pxd files will be part of my summer of code in some form.) This could potentially make the Cython compiler less complex as well as provide some more flexibility. (But as I said, I don't know the C side of this and so this may be off mark.) Dag Sverre _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
