On Sun, Nov 8, 2009 at 8:08 PM, David Goldsmith <d.l.goldsm...@gmail.com>wrote:
> On Sun, Nov 8, 2009 at 7:40 PM, Anne Archibald > <peridot.face...@gmail.com>wrote: > >> As Josef said, this is not correct. I think the key point of confusion is >> this: >> >> Do not pass choose two arrays. >> >> Pass it one array and a *list* of arrays. The fact that choices can be >> an array is a quirk we can't change, but you should think of the >> second argument as a list of arrays, > > > Fine, but as you say, one *can* pass choose an array as the second argument > and it doesn't raise an exception, so if someone is stupid/careless enough > to pass an array for `choices`, how is choose interpreting it as a list? Is > the first dimension "list converted" (so that, e.g., my (2,1,2) example is > interpreted as a two element list, each of whose elements is a (1,2) array)? > > >> possibly of different shapes. >> >> np.choose(np.ones((2,1,1)), [ np.ones((1,3,1)), np.ones((1,1,5)) ] ) >> > > >>> np.choose(np.ones((2,1,1)), [ np.ones((1,3,1)), np.ones((1,1,5)) ] ) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "C:\Python254\lib\site-packages\numpy\core\fromnumeric.py", line > 208, in > choose > return choose(choices, out=out, mode=mode) > TypeError: array cannot be safely cast to required type > Never mind, I recognized the problem from my experimenting: a needs to be integer valued, so you need, e.g., np.choose(np.ones((2,1,1),dtype=int), [ np.ones((1,3,1)), np.ones((1,1,5)) ] ) > > Here the first argument is an array of choices. The second argument is >> a *list* - if you cast it to an array you'll get an error or nonsense >> > > So the result of my example was nonsense? > > >> - of arrays to choose from. The broadcasting ensures the first >> argument and *each element of the list* are the same shape. The only >> constraint on the number of arrays in the list is that it be larger >> than the largest value in a. >> >> If you try to make the second argument into a single array, for one >> thing you are throwing away useful generality by forcing each choice >> to be the same shape (and real shape, not zero-strided fake shape), >> and for another the broadcasting becomes very hard to understand. >> > > Obviously, I find the broadcasting very hard to understand regardless. :-( > > DG > > >> >> Anne >> >> >> 2009/11/8 David Goldsmith <d.l.goldsm...@gmail.com>: >> > OK, now I'm trying to wrap my brain around broadcasting in choose when >> both >> > `a` *and* `choices` need to be (non-trivially) broadcast in order to >> arrive >> > at a common shape, e.g.: >> > >> >>>> c=np.arange(4).reshape((2,1,2)) # shape is (2,1,2) >> >>>> a=np.eye(2, dtype=int) # shape is (2,2) >> >>>> np.choose(a,c) >> > array([[2, 1], >> > [0, 3]]) >> > >> > (Unfortunately, the implementation is in C, so I can't easily insert >> print >> > statements to see intermediate results.) >> > >> > First, let me confirm that the above is indeed an example of what I >> think it >> > is, i.e., both `a` and `choices` are broadcast in order for this to >> work, >> > correct? (And if incorrect, how is one broadcast to the shape of the >> > other?) Second, both are broadcast to shape (2,2,2), correct? But how, >> > precisely, i.e., does c become >> > >> > [[[0, 1], [2, 3]], [[[0, 1], [0, 1]], >> > [[0, 1], [2, 3]]] or [[2, 3], [2, 3]]] >> > >> > and same question for a? Then, once a is broadcast to a (2,2,2) shape, >> > precisely how does it "pick and choose" from c to create a (2,2) result? >> > For example, suppose a is broadcast to: >> > >> > [[[1, 0], [0, 1]], >> > [[1, 0], [0, 1]]] >> > >> > (as indicated above, I'm uncertain at this point if this is indeed what >> a is >> > broadcast to); how does this create the (2,2) result obtained above? >> > (Obviously this depends in part on precisely how c is broadcast, I do >> > recognize that much.) >> > >> > Finally, a seemingly relevant comment in the C source is: >> > >> > /* Broadcast all arrays to each other, index array at the end.*/ >> > >> > This would appear to confirm that "co-broadcasting" is performed if >> > necessary, but what does the "index array at the end" phrase mean? >> > >> > Thanks for your continued patience and tutelage. >> > >> > DG >> > >> > On Sun, Nov 8, 2009 at 5:36 AM, <josef.p...@gmail.com> wrote: >> >> >> >> On Sun, Nov 8, 2009 at 5:00 AM, David Goldsmith < >> d.l.goldsm...@gmail.com> >> >> wrote: >> >> > On Sun, Nov 8, 2009 at 12:57 AM, Anne Archibald >> >> > <peridot.face...@gmail.com> >> >> > wrote: >> >> >> >> >> >> 2009/11/8 David Goldsmith <d.l.goldsm...@gmail.com>: >> >> >> > On Sat, Nov 7, 2009 at 11:59 PM, Anne Archibald >> >> >> > <peridot.face...@gmail.com> >> >> >> > wrote: >> >> >> >> >> >> >> >> 2009/11/7 David Goldsmith <d.l.goldsm...@gmail.com>: >> >> >> >> > So in essence, at least as it presently functions, the shape of >> >> >> >> > 'a' >> >> >> >> > *defines* what the individual choices are within 'choices`, and >> if >> >> >> >> > 'choices' >> >> >> >> > can't be parsed into an integer number of such individual >> choices, >> >> >> >> > that's >> >> >> >> > when an exception is raised? >> >> >> >> >> >> >> >> Um, I don't think so. >> >> >> >> >> >> >> >> Think of it this way: you provide np.choose with a selector >> array, >> >> >> >> a, >> >> >> >> and a list (not array!) [c0, c1, ..., cM] of choices. You >> construct >> >> >> >> an >> >> >> >> output array, say r, the same shape as a (no matter how many >> >> >> >> dimensions it has). >> >> >> > >> >> >> > Except that I haven't yet seen a working example with 'a' greater >> >> >> > than >> >> >> > 1-D, >> >> >> > Josef's last two examples notwithstanding; or is that what you're >> >> >> > saying >> >> >> > is >> >> >> > the bug. >> >> >> >> >> >> There's nothing magic about A being one-dimensional. >> >> >> >> >> >> C = np.random.randn(2,3,5) >> >> >> A = (C>-1).astype(int) + (C>0).astype(int) + (C>1).astype(int) >> >> >> >> >> >> R = np.choose(A, (-1, -C, C, 1)) >> >> > >> >> > OK, now I get it: np.choose(A[0,:,:], (-1,-C,C,-1)) and >> >> > np.choose(A[0,:,0].reshape((3,1)), (-1,-C,C,1)), e.g., also work, but >> >> > np.choose(A[0,:,0], (-1,-C,C,-1)) doesn't - what's necessary for >> >> > choose's >> >> > arguments is that both can be broadcast to a common shape (as you >> state >> >> > below), but choose won't reshape the arguments for you to make this >> >> > possible, you have to do so yourself first, if necessary. That does >> >> > appear >> >> > to be what's happening now; but do we want choose to be smarter than >> >> > that >> >> > (e.g., for np.choose(A[0,:,0], (-1,-C,C,-1)) to work, so that the >> user >> >> > doesn't need to include the .reshape((3,1)))? >> >> >> >> No, I don't think we want to be that smart. >> >> >> >> If standard broadcasting rules apply, as I think they do, then I >> wouldn't >> >> want >> >> any special newaxis or reshapes done automatically. It will be >> confusing, >> >> the function wouldn't know what to do if there are, e.g., as many rows >> as >> >> columns, and this looks like a big source of errors. >> >> Standard broadcasting is pretty nice (once I got the hang of it), and >> >> adding >> >> a lot of np.newaxis (or some reshapes) to the code is only a small >> price >> >> to pay. >> >> >> >> Josef >> >> >> >> >> >> >> >> > >> >> > DG >> >> > >> >> >> >> >> >> Requv = np.minimum(np.abs(C),1) >> >> >> >> >> >> or: >> >> >> >> >> >> def wedge(*functions): >> >> >> """Return a function whose value is the minimum of those of >> >> >> functions""" >> >> >> def wedgef(X): >> >> >> fXs = [f(X) for f in functions] >> >> >> A = np.argmin(fXs, axis=0) >> >> >> return np.choose(A,fXs) >> >> >> return wedgef >> >> >> >> >> >> so e.g. np.abs is -wedge(lambda X: X, lambda X: -X) >> >> >> >> >> >> This works no matter what shape of X the user supplies - so a wedged >> >> >> function can be somewhat ufunclike - by making A the same shape. >> >> >> >> >> >> >> The (i0, i1, ..., iN) element of the output array >> >> >> >> is obtained by looking at the (i0, i1, ..., iN) element of a, >> which >> >> >> >> should be an integer no larger than M; say j. Then r[i0, i1, ..., >> >> >> >> iN] >> >> >> >> = cj[i0, i1, ..., iN]. That is, each element of the selector >> array >> >> >> >> determines which of the choice arrays to pull the corresponding >> >> >> >> element from. >> >> >> > >> >> >> > That's pretty clear (thanks for doing my work for me). ;-), Yet, >> see >> >> >> > above. >> >> >> > >> >> >> >> For example, suppose that you are processing an array C, and have >> >> >> >> constructed a selector array A the same shape as C in which a >> value >> >> >> >> is >> >> >> >> 0, 1, or 2 depending on whether the C value is too small, okay, >> or >> >> >> >> too >> >> >> >> big respectively. Then you might do something like: >> >> >> >> >> >> >> >> C = np.choose(A, [-inf, C, inf]) >> >> >> >> >> >> >> >> This is something you might want to do no matter what shape A and >> C >> >> >> >> have. It's important not to require that the choices be an array >> of >> >> >> >> choices, because they often have quite different shapes (here, >> two >> >> >> >> are >> >> >> >> scalars) and it would be wasteful to broadcast them up to the >> same >> >> >> >> shape as C, just to stack them. >> >> >> > >> >> >> > OK, that's a pretty generic use-case, thanks; let me see if I >> >> >> > understand >> >> >> > it >> >> >> > correctly: A is some how created independently with a 0 everywhere >> C >> >> >> > is >> >> >> > too >> >> >> > small, a 1 everywhere C is OK, and a 2 everywhere C is too big; >> then >> >> >> > np.choose(A, [-inf, C, inf]) creates an array that is -inf >> everywhere >> >> >> > C >> >> >> > is >> >> >> > too small, inf everywhere C is too large, and C otherwise (and >> since >> >> >> > -inf >> >> >> > and inf are scalars, this implies broadcasting of these is taking >> >> >> > place). >> >> >> > This is what you're asserting *should* be the behavior. So, >> unless >> >> >> > there is >> >> >> > disagreement about this (you yourself said the opposite viewpoint >> >> >> > might >> >> >> > rationally be held) np.choose definitely presently has a bug, >> namely, >> >> >> > the >> >> >> > index array can't be of arbitrary shape. >> >> >> >> >> >> There seems to be some disagreement between versions, but both Josef >> >> >> and I find that the index array *can* be arbitrary shape. In numpy >> >> >> 1.2.1 I find that all the choose items must be the same shape as it, >> >> >> which I think is a bug. >> >> >> >> >> >> What I suggested might be okay was if the index array was not >> >> >> broadcasted, so that the outputs always had exactly the same shape >> as >> >> >> the index array. But upon reflection it's useful to be able to use a >> >> >> 1-d array to select rows from a set of matrices, so I now think that >> >> >> all of A and the elements of choose should be broadcast to the same >> >> >> shape. This seems to be what Josef observes in his version of numpy, >> >> >> so maybe there's nothing to do. >> >> >> >> >> >> Anne >> >> >> >> >> >> > DG >> >> >> > >> >> >> >> >> >> >> >> Anne >> >> >> >> _______________________________________________ >> >> >> >> NumPy-Discussion mailing list >> >> >> >> NumPy-Discussion@scipy.org >> >> >> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> >> >> > >> >> >> > >> >> >> > _______________________________________________ >> >> >> > NumPy-Discussion mailing list >> >> >> > NumPy-Discussion@scipy.org >> >> >> > http://mail.scipy.org/mailman/listinfo/numpy-discussion >> >> >> > >> >> >> > >> >> >> _______________________________________________ >> >> >> NumPy-Discussion mailing list >> >> >> NumPy-Discussion@scipy.org >> >> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> >> > >> >> > >> >> > _______________________________________________ >> >> > NumPy-Discussion mailing list >> >> > NumPy-Discussion@scipy.org >> >> > http://mail.scipy.org/mailman/listinfo/numpy-discussion >> >> > >> >> > >> >> _______________________________________________ >> >> NumPy-Discussion mailing list >> >> NumPy-Discussion@scipy.org >> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> > >> > >> > _______________________________________________ >> > NumPy-Discussion mailing list >> > NumPy-Discussion@scipy.org >> > http://mail.scipy.org/mailman/listinfo/numpy-discussion >> > >> > >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> > >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
