On 04/10/2012 02:11 AM, Travis Oliphant wrote:
> Hi all,
>
> Some of you are aware of Numba. Numba allows you to create the equivalent
> of C-function's dynamically from Python. One purpose of this system is to
> allow NumPy to take these functions and use them in operations like ufuncs,
> generalized ufuncs, file-reading, fancy-indexing, and so forth. There are
> actually many use-cases that one can imagine for such things.
>
> One question is how do you pass this function pointer to the C-side. On
> the Python side, Numba allows you to get the raw integer address of the
> equivalent C-function pointer that it just created out of the Python code.
> One can think of this as a 32- or 64-bit integer that you can cast to a
> C-function pointer.
>
> Now, how should this C-function pointer be passed from Python to NumPy? One
> approach is just to pass it as an integer --- in other words have an API in C
> that accepts an integer as the first argument that the internal function
> interprets as a C-function pointer.
>
> This is essentially what ctypes does when creating a ctypes function pointer
> out of:
>
> func = ctypes.CFUNCTYPE(restype, *argtypes)(integer)
>
> Of course the problem with this is that you can easily hand it integers which
> don't make sense and which will cause a segfault when control is passed to
> this "function"
>
> We could also piggy-back on-top of Ctypes and assume that a ctypes
> function-pointer object is passed in. This allows some error-checking at
> least and also has the benefit that one could use ctypes to access a
> c-function library where these functions were defined. I'm leaning towards
> this approach.
>
> Now, the issue is how to get the C-function pointer (that npy_intp integer)
> back and hand it off internally. Unfortunately, ctypes does not make it
> very easy to get this address (that I can see). There is no ctypes C-API,
> for example. There are two potential options:
>
> 1) Create an API for such Ctypes function pointers in NumPy and use the
> ctypes object structure. If ctypes were to ever change it's object structure
> we would have to adapt this API.
>
> Something like this is what is envisioned here:
>
> typedef struct {
> PyObject_HEAD
> char *b_ptr;
> } _cfuncptr_object;
>
> then the function pointer is:
>
> (*((void **)(((_sp_cfuncptr_object *)(obj))->b_ptr)))
>
> which could be wrapped-up into a nice little NumPy C-API call like
>
> void * Npy_ctypes_funcptr(obj)
>
>
> 2) Use the Python API of ctypes to do the same thing. This has the
> advantage of not needing to mirror the simple _cfuncptr_object structure in
> NumPy but it is *much* slower to get the address. It basically does the
> equivalent of
>
> ctypes.cast(obj, ctypes.c_void_p).value
>
>
> There is working code for this in the ctypes_callback branch of my
> scipy fork on github.
>
>
> I would like to propose two things:
>
> * creating a Npy_ctypes_funcptr(obj) function in the C-API of NumPy and
> * implement it with the simple pointer dereference above (option #1)
>
>
> Thoughts?
I really hope we can find some project-neutral common ground, so that
lots of tools (Cython, f2py, numba, C extensions in NumPy and SciPy) can
agree on how to "unbox callables".
A new extension type in NumPy would not fit this bill I feel. I've
created a specification for this; if a number of projects (the ones
mentioned above) agree on this or something similar and implement
support, we could propose a PEP and do it properly once it has proven
itself.
http://wiki.cython.org/enhancements/cep1000
In Cython, this may take the form
def call_callback(object func):
cdef double (*typed_func)(int)
typed_func = func
return typed_func(4)
...it would be awesome if passing a Numba-compiled function just worked
in this example.
Dag
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
