On 04/12/2012 11:55 PM, Travis Oliphant wrote: > > On Apr 12, 2012, at 4:51 PM, Dag Sverre Seljebotn wrote: > >> On 04/12/2012 11:13 PM, Travis Oliphant wrote: >>> Dag, >>> >>> Thanks for the link to your CEP. This is the first time I've seen it. >>> You probably referenced it before, but I hadn't seen it. >>> >>> That CEP seems along the lines of what I was thinking of. We can make >>> scipy follow that CEP and NumPy as well in places that it needs function >>> pointers. >>> >>> I can certainly get behind it with Numba and recommend it to SciPy (and >>> write the scipy.integrate.quad function to support it). >>> >>> Thanks for the CEP. >> >> Great. I'll pass this message on to the Cython list and see if anybody >> wants to provide input (but given the scope, it should be minor tweaks >> and easy to accommodate in whatever code you write). >> >> You will fill in more of the holes as you implement this in Numba and >> SciPy of course (my feeling is they will support it before Cython; let's >> say I hope this happens within the next year). > > Very nice. This will help immensely I think. It's actually just what I > was looking for. Just to be clear, by " pad to sizeof(void*) alignment", > you mean that after the first 2 bytes there are (sizeof(void*) - 2) bytes > before the first function pointer in the memory block pointed to by the > PyCObject / Capsule?
You are only right if the starting address of the data is divisible by sizeof(void*). On 64-bit you would do something like func_ptr = (func_ptr_t*)((char*)descriptor & 0xfffffffffffffff7) + 8) Hmm, not sure if I like it any longer. I don't know a priori how much alignment really matters either on modern CPUs (but in the Cython case, we would like this to be somewhat competitive with compile-time binding, so it does merit checking I think). Dag > > Thanks, > > -Travis > > >> >> Dag >> >> >> >> >>> >>> -Travis >>> >>> >>> >>> >>> >>> On Apr 12, 2012, at 2:08 PM, Dag Sverre Seljebotn wrote: >>> >>>> On 04/12/2012 07:24 PM, Nathaniel Smith wrote: >>>>> On Wed, Apr 11, 2012 at 10:23 PM, Travis Oliphant<teoliph...@gmail.com> >>>>> wrote: >>>>>>>> In the mean-time, I think we could do as Robert essentially suggested >>>>>>>> and just use Capsule Objects around an agreed-upon simple C-structure: >>>>>>>> >>>>>>>> int id /* Some number that can be used as a "type-check" */ >>>>>>>> void *func; >>>>>>>> char *string; >>>>>>>> >>>>>>>> We can then just create some nice functions to go to and from this >>>>>>>> form in NumPy ctypeslib and then use this while the Python PEP gets >>>>>>>> written and adopted. >>>>>>> >>>>>>> What is not clear to me is how one get from the Python callable to the >>>>>>> capsule. >>>>>> >>>>>> This varies substantially based on the tool. Numba would do it's work >>>>>> and create the capsule object using it's approach. Cython would use a >>>>>> different approach. >>>>>> >>>>>> I would also propose to have in NumPy some basic functions that go >>>>>> back-and forth between this representation, ctypes, and any other useful >>>>>> representations that might emerge. >>>>>> >>>>>>> >>>>>>> Or do you simply intend to pass a non-callable capsule as an argument in >>>>>>> place of the callback? >>>>>> >>>>>> I had simply intended to allow a non-callable capsule argument to be >>>>>> passed in instead of another call-back to any SciPy or NumPy function >>>>>> that can take a raw C-function pointer. >>>>> >>>>> If the cython folks are worried about type-checking overhead, then >>>>> PyCapsule seems sub-optimal, because it's unnecessarily complicated to >>>>> determine what sort of PyCapsule you have, and then extract the actual >>>>> C struct. (At a minimum, it requires two calls to non-inlineable >>>>> functions, plus an unnecessary pointer indirection.) >>>> >>>> I think this discussion is moot -- the way I reverse-engineer Travis is >>>> that there's no time for a cross-project discussion about this now. >>>> That's not too bad, Cython will go its own way (eventually), and perhaps >>>> we can merge in the future... >>>> >>>> But for the entertainment value: >>>> >>>> In my CEP [1] I descripe two access mechanisms, one slow (for which I >>>> think capsules is fine), and a faster one. >>>> >>>> Obviously, only the slow mechanism will be implemented first. >>>> >>>> So the only things I'd like changed in how Travis' want to do this is >>>> >>>> a) Storing the signature string data in the struct, rather than as a char*; >>>> >>>> void *func >>>> char string[1]; // variable-size-allocated and null-terminated >>>> >>>> b) Allow for multiple signatures in the same capsule, i.e. "dd->d", >>>> "ff->f", in the same capsule. >>>> >>>>> A tiny little custom class in a tiny little library that everyone can >>>>> share might be better? (Bonus: a custom class could define a __call__ >>>>> method that used ctypes to call the function directly, for interactive >>>>> convenience/testing/etc.) >>>> >>>> Having NumPy and Cython depend on a common library, and getting that to >>>> work for users, seems rather utopic to me. And if I propose that Cython >>>> have a hard dependency of NumPy for a feature as basic as calli.ng a >>>> callback object then certain people will be very angry. >>>> >>>> Anyway, in my CEP I went to great pains to avoid having to do this, with >>>> a global registration mechanism for multiple such types. >>>> >>>> Regarding your idea for the __call__, that's the exact opposite of what >>>> I'm doing in the CEP. I'm pretty sure that what I described is what we >>>> want for Cython; we will never tell our users to pass capsules around. >>>> What I want is this: >>>> >>>> @numba >>>> def f(x): return 2 * x >>>> >>>> @cython.inline >>>> def g(x): return 3 * x >>>> >>>> print f(3) >>>> print g(3) >>>> print scipy.integrate.quad(f, 0.2, 3) # fast! >>>> print scipy.integrate.quad(g, 0.2, 3) # fast! >>>> >>>> # If you really want a capsule: >>>> print f.__nativecall__ >>>> >>>> Dag >>>> >>>> [1] http://wiki.cython.org/enhancements/cep1000 >>>> _______________________________________________ >>>> NumPy-Discussion mailing list >>>> NumPy-Discussion@scipy.org >>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion >>> >>> _______________________________________________ >>> NumPy-Discussion mailing list >>> NumPy-Discussion@scipy.org >>> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
