Hi, I wonder how to best deal with the new buffer interface.
http://www.python.org/dev/peps/pep-3118/ It's actually pretty simple, it exposes two straight forward functions through the PyBufferProcs struct using the old Py2 "tp_as_buffer" slot (but different content). typedef int (*getbufferproc)(PyObject *obj, Py_buffer *view, int flags); typedef void (*releasebufferproc)(PyObject *, Py_buffer *); typedef struct { getbufferproc bf_getbuffer; releasebufferproc bf_releasebuffer; } PyBufferProcs; struct bufferinfo { void *buf; Py_ssize_t len; int readonly; const char *format; int ndim; Py_ssize_t *shape; Py_ssize_t *strides; Py_ssize_t *suboffsets; Py_ssize_t itemsize; void *internal; } Py_buffer; This would normally call for two special functions __getbuffer__ and __releasebuffer__. To me, however, this looks like an extremely C-ish interface that does not fit Cython at all. So I'm wondering what others think about this approach: 1) define a pseudo-builtin PyxBuffer extension type that mimics a Py_buffer struct without the ".internal" field 2) support a new special method __getbuffer__(self, int flags) that allows users to create, fill and return an instance of PyxBuffer or a subtype (maybe the PyxBuffer class could also contain a public enum that defines the flags?) 3) generate glue code for the bf_getbuffer() call that a) calls __getbuffer__() and raises an exception if it does not return a PyxBuffer instance b) copies the Py_buffer struct content over from the returned PyxBuffer object to the Py_buffer struct that was passed into bf_getbuffer() c) stores an incref-ed reference to the PyxBuffer object in the ".internal" field 4) generate generic code for the bf_releasebuffer slot that decrefs the reference in the ".internal" field (and sets it to NULL), thus eventually calling __dealloc__() on the PyxBuffer object. User defined subtypes of PyxBuffer can then do The Right Thing here. This has several advantages: - it would only require a single special method - it matches the way GC and normal extension types work in Cython - it relieves the user from having to cast around with "internal" pointers possibly even to hand-refcounted Python objects (if used that way) - the PyxBuffer instance could easily be reused in __getbuffer__() and would only be garbage collected when all references are gone, leaving the user full control over the GC process - the "shapes", "strides" and "suboffsets" field could be normal Py_ssize_t values in PyxBuffer that the Py_buffer fields would just point to - all generated functions would be wrapped by a conditional "#if Py3 ..." - the user code would never rely on Py_buffer directly, thus nicely compiling to unused code in Py2 A disadvantage I see is that PyxBuffer cannot directly inherit the "official" struct from the Python header files, so changes to the struct would have to be reflected in Cython. But given the usual stability of Python APIs, I think that's a non-problem. Opinions? Stefan _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
