Dag, Thanks for the link to your CEP. This is the first time I've seen it. You probably referenced it before, but I hadn't seen it.
That CEP seems along the lines of what I was thinking of. We can make scipy follow that CEP and NumPy as well in places that it needs function pointers. I can certainly get behind it with Numba and recommend it to SciPy (and write the scipy.integrate.quad function to support it). Thanks for the CEP. -Travis On Apr 12, 2012, at 2:08 PM, Dag Sverre Seljebotn wrote: > On 04/12/2012 07:24 PM, Nathaniel Smith wrote: >> On Wed, Apr 11, 2012 at 10:23 PM, Travis Oliphant<teoliph...@gmail.com> >> wrote: >>>>> In the mean-time, I think we could do as Robert essentially suggested and >>>>> just use Capsule Objects around an agreed-upon simple C-structure: >>>>> >>>>> int id /* Some number that can be used as a "type-check" */ >>>>> void *func; >>>>> char *string; >>>>> >>>>> We can then just create some nice functions to go to and from this form >>>>> in NumPy ctypeslib and then use this while the Python PEP gets written >>>>> and adopted. >>>> >>>> What is not clear to me is how one get from the Python callable to the >>>> capsule. >>> >>> This varies substantially based on the tool. Numba would do it's work and >>> create the capsule object using it's approach. Cython would use a >>> different approach. >>> >>> I would also propose to have in NumPy some basic functions that go back-and >>> forth between this representation, ctypes, and any other useful >>> representations that might emerge. >>> >>>> >>>> Or do you simply intend to pass a non-callable capsule as an argument in >>>> place of the callback? >>> >>> I had simply intended to allow a non-callable capsule argument to be passed >>> in instead of another call-back to any SciPy or NumPy function that can >>> take a raw C-function pointer. >> >> If the cython folks are worried about type-checking overhead, then >> PyCapsule seems sub-optimal, because it's unnecessarily complicated to >> determine what sort of PyCapsule you have, and then extract the actual >> C struct. (At a minimum, it requires two calls to non-inlineable >> functions, plus an unnecessary pointer indirection.) > > I think this discussion is moot -- the way I reverse-engineer Travis is > that there's no time for a cross-project discussion about this now. > That's not too bad, Cython will go its own way (eventually), and perhaps > we can merge in the future... > > But for the entertainment value: > > In my CEP [1] I descripe two access mechanisms, one slow (for which I > think capsules is fine), and a faster one. > > Obviously, only the slow mechanism will be implemented first. > > So the only things I'd like changed in how Travis' want to do this is > > a) Storing the signature string data in the struct, rather than as a char*; > > void *func > char string[1]; // variable-size-allocated and null-terminated > > b) Allow for multiple signatures in the same capsule, i.e. "dd->d", > "ff->f", in the same capsule. > >> A tiny little custom class in a tiny little library that everyone can >> share might be better? (Bonus: a custom class could define a __call__ >> method that used ctypes to call the function directly, for interactive >> convenience/testing/etc.) > > Having NumPy and Cython depend on a common library, and getting that to > work for users, seems rather utopic to me. And if I propose that Cython > have a hard dependency of NumPy for a feature as basic as calli.ng a > callback object then certain people will be very angry. > > Anyway, in my CEP I went to great pains to avoid having to do this, with > a global registration mechanism for multiple such types. > > Regarding your idea for the __call__, that's the exact opposite of what > I'm doing in the CEP. I'm pretty sure that what I described is what we > want for Cython; we will never tell our users to pass capsules around. > What I want is this: > > @numba > def f(x): return 2 * x > > @cython.inline > def g(x): return 3 * x > > print f(3) > print g(3) > print scipy.integrate.quad(f, 0.2, 3) # fast! > print scipy.integrate.quad(g, 0.2, 3) # fast! > > # If you really want a capsule: > print f.__nativecall__ > > Dag > > [1] http://wiki.cython.org/enhancements/cep1000 > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
