On 04/12/2012 11:13 PM, Travis Oliphant wrote: > Dag, > > Thanks for the link to your CEP. This is the first time I've seen it. You > probably referenced it before, but I hadn't seen it. > > That CEP seems along the lines of what I was thinking of. We can make > scipy follow that CEP and NumPy as well in places that it needs function > pointers. > > I can certainly get behind it with Numba and recommend it to SciPy (and write > the scipy.integrate.quad function to support it). > > Thanks for the CEP.
Great. I'll pass this message on to the Cython list and see if anybody wants to provide input (but given the scope, it should be minor tweaks and easy to accommodate in whatever code you write). You will fill in more of the holes as you implement this in Numba and SciPy of course (my feeling is they will support it before Cython; let's say I hope this happens within the next year). Dag > > -Travis > > > > > > On Apr 12, 2012, at 2:08 PM, Dag Sverre Seljebotn wrote: > >> On 04/12/2012 07:24 PM, Nathaniel Smith wrote: >>> On Wed, Apr 11, 2012 at 10:23 PM, Travis Oliphant<teoliph...@gmail.com> >>> wrote: >>>>>> In the mean-time, I think we could do as Robert essentially suggested >>>>>> and just use Capsule Objects around an agreed-upon simple C-structure: >>>>>> >>>>>> int id /* Some number that can be used as a "type-check" */ >>>>>> void *func; >>>>>> char *string; >>>>>> >>>>>> We can then just create some nice functions to go to and from this form >>>>>> in NumPy ctypeslib and then use this while the Python PEP gets written >>>>>> and adopted. >>>>> >>>>> What is not clear to me is how one get from the Python callable to the >>>>> capsule. >>>> >>>> This varies substantially based on the tool. Numba would do it's work >>>> and create the capsule object using it's approach. Cython would use a >>>> different approach. >>>> >>>> I would also propose to have in NumPy some basic functions that go >>>> back-and forth between this representation, ctypes, and any other useful >>>> representations that might emerge. >>>> >>>>> >>>>> Or do you simply intend to pass a non-callable capsule as an argument in >>>>> place of the callback? >>>> >>>> I had simply intended to allow a non-callable capsule argument to be >>>> passed in instead of another call-back to any SciPy or NumPy function that >>>> can take a raw C-function pointer. >>> >>> If the cython folks are worried about type-checking overhead, then >>> PyCapsule seems sub-optimal, because it's unnecessarily complicated to >>> determine what sort of PyCapsule you have, and then extract the actual >>> C struct. (At a minimum, it requires two calls to non-inlineable >>> functions, plus an unnecessary pointer indirection.) >> >> I think this discussion is moot -- the way I reverse-engineer Travis is >> that there's no time for a cross-project discussion about this now. >> That's not too bad, Cython will go its own way (eventually), and perhaps >> we can merge in the future... >> >> But for the entertainment value: >> >> In my CEP [1] I descripe two access mechanisms, one slow (for which I >> think capsules is fine), and a faster one. >> >> Obviously, only the slow mechanism will be implemented first. >> >> So the only things I'd like changed in how Travis' want to do this is >> >> a) Storing the signature string data in the struct, rather than as a char*; >> >> void *func >> char string[1]; // variable-size-allocated and null-terminated >> >> b) Allow for multiple signatures in the same capsule, i.e. "dd->d", >> "ff->f", in the same capsule. >> >>> A tiny little custom class in a tiny little library that everyone can >>> share might be better? (Bonus: a custom class could define a __call__ >>> method that used ctypes to call the function directly, for interactive >>> convenience/testing/etc.) >> >> Having NumPy and Cython depend on a common library, and getting that to >> work for users, seems rather utopic to me. And if I propose that Cython >> have a hard dependency of NumPy for a feature as basic as calli.ng a >> callback object then certain people will be very angry. >> >> Anyway, in my CEP I went to great pains to avoid having to do this, with >> a global registration mechanism for multiple such types. >> >> Regarding your idea for the __call__, that's the exact opposite of what >> I'm doing in the CEP. I'm pretty sure that what I described is what we >> want for Cython; we will never tell our users to pass capsules around. >> What I want is this: >> >> @numba >> def f(x): return 2 * x >> >> @cython.inline >> def g(x): return 3 * x >> >> print f(3) >> print g(3) >> print scipy.integrate.quad(f, 0.2, 3) # fast! >> print scipy.integrate.quad(g, 0.2, 3) # fast! >> >> # If you really want a capsule: >> print f.__nativecall__ >> >> Dag >> >> [1] http://wiki.cython.org/enhancements/cep1000 >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
