Sebastian Berg wrote > On Sun, 2021-01-10 at 09:59 -0700, zoj613 wrote: >> Hi all, >> >> I am looking for a way to use `np.PyArray_MultiIterNew2` in Cython to >> broadcast parameters of a function. The requirement is that the two >> arguments can be scalar and/or sequences. Using the usual >> `np.broadcast` >> function works well but is slow when iterating over the broadcasted >> input in >> a tight loop. I want to achieve the same using the C API. > > NB: There is also a different, newer, iterator (`NpyIter_New`). > Depending on what you do, that might be a lot faster and better > (although, I admit it tends to be more verbose too). > That iterator also does broadcasting, and one nice property is that it > can allocate a result array. Most importantly, it casts for you and > allows you to take charge of the inner-loop (a one dimensional loop) > for performance. (This is the core of how ufuncs work.) > >> Currently, if I used `(<double*>np.PyArray_MultiIter_DATA(bcast, >> i))[0]` to >> iterate over the input when one of them is a scalar, >> I get no errors, but I notice the output of the parent function >> returns an >> array of zeros, which implies this approach didn't work. After >> investigating, it seems that np.PyArray_MultiIter_DATA only accepts >> numpy >> arrays. > > I am honestly confused by this (did you mean a different command?). > `PyArray_MultiIter_DATA` returns a pointer to the raw data, i.e. in > cython you would probably do: > > cdef double *value_ptr = > <double *> > npc.PyArray_MultiIter_DATA(iter, 0) > value_ptr[0] = 3.1416 > > Do want to reference the original data? You could reference the > original data for scalars (read-only since scalars are immutable), but > for lists/tuples there is no original data to begin with (unless you > have Python objects inside, but it would seem strange if NumPy to > attempted to transparently expose that). > >> I could write a function to handle all combinations of >> scalar/array/list/tuple, and create temporary arrays to store the >> input > > The typical pattern is to convert everything to array first, but > `PyArray_MultiIter()` does that for you. Unless you want to optimize > that conversion away? > > Cheers, > > Sebastian > > >> data, but that seems daunting and error prone. Is there a way I can >> achieve >> this and have scalar arguments passed to np.PyArray_MultiIter_DATA be >> converted to same-element arrays without writing my own code from >> scratch? >> >> Any suggestions are welcome. >> >> >> >> -- >> Sent from: http://numpy-discussion.10968.n7.nabble.com/ >> _______________________________________________ >> NumPy-Discussion mailing list >>
> NumPy-Discussion@ >> https://mail.python.org/mailman/listinfo/numpy-discussion >> > > I think I wasn't clear enough in my original question. Here is what I > have: > a python function with the signature: def pyfunc(a, b). This function > calls another cython function internally such that > def pyfunc(a, b): > .... > return cfunc(a, b) > > I want the pyfunc to support scalar and array_like input for both > parameters, so that internally I can have: > def pyfunc(a, b): > .... > # some part of the function > for i, j in some_broadcasted_iterator: > output[indx] = cfunc(i, j) > > Broadcasting using python level functions allows me to do this but then > the iteration part is slow because it uses python python code, meaning I > can't release the gil. What I was thinking of doing is using the C API to > create the broadcasted iterator using PyArray_MultiIterNew2 then iterating > over that. I kept getting a zero array as output when I tested it so I > assumed that the function doesnt work with scalars. It turns out the bug > was caused by me using integer scalars when the C function expects doubles > (But I fixed that after creating this post). > > Now I'm left with figuring out how I can efficiently convert even > lists/tuple inputs into a broadcasted iterator (without requiring the user > to pass numpy arrays). When I pass a list for parameter a and a scalar for > parameter b, the program hangs, so Im not sure if the > PyArray_MultiIterNew2 function does it for me? > > Btw, do you have a link to the more recent NpyIter_New? I couldn't find it > reading the C-API page in the docs. > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@ > https://mail.python.org/mailman/listinfo/numpy-discussion > > > signature.asc (849 bytes) > <http://numpy-discussion.10968.n7.nabble.com/attachment/48862/0/signature.asc> -- Sent from: http://numpy-discussion.10968.n7.nabble.com/ _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
