Re: [Numpy-discussion] numpy1.2 : make sorts unary ufuncs
On Sat, Apr 19, 2008 at 12:20 AM, Travis E. Oliphant <[EMAIL PROTECTED]> wrote: > Charles R Harris wrote: > > The signature for a ufunc is something like > > > > @[EMAIL PROTECTED]@kind@(char **args, intp *dimensions, intp *steps, void > > *func) > > > > Which contains all the info necessary to do a sort. Means and other > > such functions could also be implemented that way. > I dont' think the ufunc is the right place for this.Perhaps sorting > can be viewed however as a "general function" which I'd like to add. > Maybe, in that case the ufunc could be a special case of the "general > function" concept. But, that is the direction I would expect the > discussion to go. > > The ufunc object is a very particular kind of thing. The 1-d inner > loop signature is a key part of it, but there is more to it than that. > I don't see how the sorting function can be pushed into this concept. > Yes, but the inner loop is just something that uses the array values along that axis to produce another set of values, i.e., it is a vector valued function of vectors. So is a sort, so is argsort, so is the inner product, so on and so forth. That's what we have here: typedef void (*PyUFuncGenericFunction) (char **, npy_intp *, npy_intp *, void *); No difference that I can see. It is the call function in PyUFuncObject that matters. > But, there is a case to be made for perhaps figuring out how to merge > the data-type functions and the ufuncs into one idea that allows for > better code re-use. > I don't think we can get all of them, but I think we can get most. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Truth value of an array
On Sat, Apr 19, 2008 at 1:32 AM, Joe Harrington <[EMAIL PROTECTED]> wrote: > What wasn't obvious was how to cast an array to bool. Just like any other type: x.astype(bool) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Truth value of an array
> Just cast your arrays to booleans if you want to do boolean operations > on them. It turns out there's an even better way: logical_and() and its friends do boolean operations on arrays. IDL solves the problem exactly as numpy does, erroring on arrays in conditionals and short-circuiting boolean operators. They, too, provide logical_and() and friends to do guarranteed-to-execute-on-all- elements logical array operations, which was in fact why I thought to look for numpy's. I'm curious what Matlab does (I don't have access). What wasn't obvious was how to cast an array to bool. Doing that the obvious way doesn't work, as is the subject of this thread: numpy.bool(x) Traceback (most recent call last): File "", line 1, in ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() My next thought was: numpy.array(x, dtype=bool) array([ True, True, True, True, True, True, True, True, True, True], dtype=bool) This works, but is a lot to type, and it's pretty opaque to a new user that you'd have to do it this way. Then I discovered the numpy version of bool() (took 45 minutes): numpy.bool_(x) # note trailing underscore array([ True, True, True, True, True, True, True, True, True, True], dtype=bool) I just added bool_ and the other casting functions to the categorized list of functions on the web site. Each of these links points to the Numpy_Example_List_With_Doc page, where there is no example, however. I hope that's ok, for now. --jh-- ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy1.2 : make sorts unary ufuncs
Charles R Harris wrote: > The signature for a ufunc is something like > > @[EMAIL PROTECTED]@kind@(char **args, intp *dimensions, intp *steps, void > *func) > > Which contains all the info necessary to do a sort. Means and other > such functions could also be implemented that way. I dont' think the ufunc is the right place for this.Perhaps sorting can be viewed however as a "general function" which I'd like to add. Maybe, in that case the ufunc could be a special case of the "general function" concept. But, that is the direction I would expect the discussion to go. The ufunc object is a very particular kind of thing. The 1-d inner loop signature is a key part of it, but there is more to it than that. I don't see how the sorting function can be pushed into this concept. But, there is a case to be made for perhaps figuring out how to merge the data-type functions and the ufuncs into one idea that allows for better code re-use. -Travis ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy1.2 : make sorts unary ufuncs
On Fri, Apr 18, 2008 at 11:29 PM, Charles R Harris < [EMAIL PROTECTED]> wrote: > > > On Fri, Apr 18, 2008 at 11:23 PM, Robert Kern <[EMAIL PROTECTED]> > wrote: > > > On Sat, Apr 19, 2008 at 12:18 AM, Charles R Harris > > <[EMAIL PROTECTED]> wrote: > > > On Fri, Apr 18, 2008 at 11:02 PM, Robert Kern <[EMAIL PROTECTED]> > > wrote: > > > > On Fri, Apr 18, 2008 at 11:58 PM, Charles R Harris > > > > > > > > <[EMAIL PROTECTED]> wrote: > > > > > > > > > > On Fri, Apr 18, 2008 at 10:53 PM, Robert Kern < > > [EMAIL PROTECTED]> > > > wrote: > > > > > > > > > > > > On Fri, Apr 18, 2008 at 11:47 PM, Charles R Harris > > > > > > <[EMAIL PROTECTED]> wrote: > > > > > > > The signature for a ufunc is something like > > > > > > > > > > > > > > @[EMAIL PROTECTED]@kind@(char **args, intp *dimensions, intp > > > > > > > *steps, void > > > *func) > > > > > > > > > > > > > > Which contains all the info necessary to do a sort. Means and > > other > > > such > > > > > > > functions could also be implemented that way. > > > > > > > > > > > > axis? > > > > > > > > > > I believe Travis is already selecting an axis to get the best > > cache > > > usage, > > > > > so it just needs a tweak. Axis = None is the special case. > > > > > > > > So what is the benefit here? Why bother? > > > > > > Code reuse and a unified concept. Why duplicate all the complication > > as we > > > do now? Argsorts become binary ufuncs, etc. Besides, I'm trying to > > track > > > down a ufunc call bug, feeling annoyed, and it would be nice to have > > all the > > > crap in one location so we can work on cleaning it up. > > > > I have a suspicion that tweaking ufuncs to allow the special case of > > sorting will negate the benefits. It smells like an abuse of the > > machinery rather than a natural merging of similar functionality. > > > Don't think so, the ufunc inner loops are all over a given axis with the > other indices varying. I think anything that processes arrays along an axis > is a natural for the framework. And if we add bells and whistles, like > threads, it might be nice to keep everything in one place. > You can also do matrix multiplication in a ufunc. It's got two inputs and an output. Needs two axis specs, but other than that goes in quite nicely. IIRC, BLAS already has the strides built in. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy1.2 : make sorts unary ufuncs
On Fri, Apr 18, 2008 at 11:23 PM, Robert Kern <[EMAIL PROTECTED]> wrote: > On Sat, Apr 19, 2008 at 12:18 AM, Charles R Harris > <[EMAIL PROTECTED]> wrote: > > On Fri, Apr 18, 2008 at 11:02 PM, Robert Kern <[EMAIL PROTECTED]> > wrote: > > > On Fri, Apr 18, 2008 at 11:58 PM, Charles R Harris > > > > > > <[EMAIL PROTECTED]> wrote: > > > > > > > > On Fri, Apr 18, 2008 at 10:53 PM, Robert Kern <[EMAIL PROTECTED] > > > > wrote: > > > > > > > > > > On Fri, Apr 18, 2008 at 11:47 PM, Charles R Harris > > > > > <[EMAIL PROTECTED]> wrote: > > > > > > The signature for a ufunc is something like > > > > > > > > > > > > @[EMAIL PROTECTED]@kind@(char **args, intp *dimensions, intp > > > > > > *steps, void > > *func) > > > > > > > > > > > > Which contains all the info necessary to do a sort. Means and > other > > such > > > > > > functions could also be implemented that way. > > > > > > > > > > axis? > > > > > > > > I believe Travis is already selecting an axis to get the best cache > > usage, > > > > so it just needs a tweak. Axis = None is the special case. > > > > > > So what is the benefit here? Why bother? > > > > Code reuse and a unified concept. Why duplicate all the complication as > we > > do now? Argsorts become binary ufuncs, etc. Besides, I'm trying to track > > down a ufunc call bug, feeling annoyed, and it would be nice to have all > the > > crap in one location so we can work on cleaning it up. > > I have a suspicion that tweaking ufuncs to allow the special case of > sorting will negate the benefits. It smells like an abuse of the > machinery rather than a natural merging of similar functionality. Don't think so, the ufunc inner loops are all over a given axis with the other indices varying. I think anything that processes arrays along an axis is a natural for the framework. And if we add bells and whistles, like threads, it might be nice to keep everything in one place. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy1.2 : make sorts unary ufuncs
On Sat, Apr 19, 2008 at 12:18 AM, Charles R Harris <[EMAIL PROTECTED]> wrote: > On Fri, Apr 18, 2008 at 11:02 PM, Robert Kern <[EMAIL PROTECTED]> wrote: > > On Fri, Apr 18, 2008 at 11:58 PM, Charles R Harris > > > > <[EMAIL PROTECTED]> wrote: > > > > > > On Fri, Apr 18, 2008 at 10:53 PM, Robert Kern <[EMAIL PROTECTED]> > wrote: > > > > > > > > On Fri, Apr 18, 2008 at 11:47 PM, Charles R Harris > > > > <[EMAIL PROTECTED]> wrote: > > > > > The signature for a ufunc is something like > > > > > > > > > > @[EMAIL PROTECTED]@kind@(char **args, intp *dimensions, intp *steps, > > > > > void > *func) > > > > > > > > > > Which contains all the info necessary to do a sort. Means and other > such > > > > > functions could also be implemented that way. > > > > > > > > axis? > > > > > > I believe Travis is already selecting an axis to get the best cache > usage, > > > so it just needs a tweak. Axis = None is the special case. > > > > So what is the benefit here? Why bother? > > Code reuse and a unified concept. Why duplicate all the complication as we > do now? Argsorts become binary ufuncs, etc. Besides, I'm trying to track > down a ufunc call bug, feeling annoyed, and it would be nice to have all the > crap in one location so we can work on cleaning it up. I have a suspicion that tweaking ufuncs to allow the special case of sorting will negate the benefits. It smells like an abuse of the machinery rather than a natural merging of similar functionality. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy1.2 : make sorts unary ufuncs
On Fri, Apr 18, 2008 at 11:02 PM, Robert Kern <[EMAIL PROTECTED]> wrote: > On Fri, Apr 18, 2008 at 11:58 PM, Charles R Harris > <[EMAIL PROTECTED]> wrote: > > > > On Fri, Apr 18, 2008 at 10:53 PM, Robert Kern <[EMAIL PROTECTED]> > wrote: > > > > > > On Fri, Apr 18, 2008 at 11:47 PM, Charles R Harris > > > <[EMAIL PROTECTED]> wrote: > > > > The signature for a ufunc is something like > > > > > > > > @[EMAIL PROTECTED]@kind@(char **args, intp *dimensions, intp *steps, > > > > void > *func) > > > > > > > > Which contains all the info necessary to do a sort. Means and other > such > > > > functions could also be implemented that way. > > > > > > axis? > > > > I believe Travis is already selecting an axis to get the best cache > usage, > > so it just needs a tweak. Axis = None is the special case. > > So what is the benefit here? Why bother? > Code reuse and a unified concept. Why duplicate all the complication as we do now? Argsorts become binary ufuncs, etc. Besides, I'm trying to track down a ufunc call bug, feeling annoyed, and it would be nice to have all the crap in one location so we can work on cleaning it up. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy1.2 : make sorts unary ufuncs
On Fri, Apr 18, 2008 at 11:58 PM, Charles R Harris <[EMAIL PROTECTED]> wrote: > > On Fri, Apr 18, 2008 at 10:53 PM, Robert Kern <[EMAIL PROTECTED]> wrote: > > > > On Fri, Apr 18, 2008 at 11:47 PM, Charles R Harris > > <[EMAIL PROTECTED]> wrote: > > > The signature for a ufunc is something like > > > > > > @[EMAIL PROTECTED]@kind@(char **args, intp *dimensions, intp *steps, void > > > *func) > > > > > > Which contains all the info necessary to do a sort. Means and other such > > > functions could also be implemented that way. > > > > axis? > > I believe Travis is already selecting an axis to get the best cache usage, > so it just needs a tweak. Axis = None is the special case. So what is the benefit here? Why bother? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy1.2 : make sorts unary ufuncs
On Fri, Apr 18, 2008 at 10:53 PM, Robert Kern <[EMAIL PROTECTED]> wrote: > On Fri, Apr 18, 2008 at 11:47 PM, Charles R Harris > <[EMAIL PROTECTED]> wrote: > > The signature for a ufunc is something like > > > > @[EMAIL PROTECTED]@kind@(char **args, intp *dimensions, intp *steps, void > > *func) > > > > Which contains all the info necessary to do a sort. Means and other such > > functions could also be implemented that way. > > axis? > I believe Travis is already selecting an axis to get the best cache usage, so it just needs a tweak. Axis = None is the special case. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy1.2 : make sorts unary ufuncs
On Fri, Apr 18, 2008 at 11:47 PM, Charles R Harris <[EMAIL PROTECTED]> wrote: > The signature for a ufunc is something like > > @[EMAIL PROTECTED]@kind@(char **args, intp *dimensions, intp *steps, void > *func) > > Which contains all the info necessary to do a sort. Means and other such > functions could also be implemented that way. axis? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] numpy1.2 : make sorts unary ufuncs
The signature for a ufunc is something like @[EMAIL PROTECTED]@kind@(char **args, intp *dimensions, intp *steps, void *func) Which contains all the info necessary to do a sort. Means and other such functions could also be implemented that way. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] powerpc yellow dog linux port of numpy
On Fri, Apr 18, 2008 at 8:19 PM, Vincent Broman <[EMAIL PROTECTED]> wrote:
> I reported back on August 30 to this list,
> with some discussion following on September 4 and 5,
> about my attempt to build numpy on an ancient powerpc setup.
> I'm running yellow dog linux 2.1, gcc 2.95.3.20010111, on processors from
> Curtiss-Wright Controls.
> Don't tell me to just upgrade; this configuration will be
> fighting the good fight for several more years.
> I just retried with the latest numpy (svn yesterday) and gotten further than
> I did before.
>
> umathmodule.c gets many compiler errors from gcc, of two kinds.
>
> The simpler were like
>
> warning: conflicting types for built-in function `sinl'
>
> repeated for `cosl', `fabsl', and `sqrtl'.
> These seem to be caused by npy_longdouble being typedef'ed as double not
> long double,
> due to the latter two types having the same size.
> umathmodule.c defines its own sinl, sqrtl, etc. with npy_longdouble
> arguments and results,
> which then conflict with the builtin sinl, sqrtl provided by gcc that expect
> long double.
We check for the presence of expl() to determine if all of the rest
are provided and set the HAVE_LONGDOUBLE_FUNCS flag. It is possible
that you don't have expl() but do have these other functions.
> I worked around that by adding the "-fno-builtin" argument to the
> extra_compiler_args in setup.py.
This is not unreasonable.
> The other compiler complaints from the same file were:
>
> inconsistent operand constraints in an `asm'
>
> which came from every line that raised a division by zero exception,
> the code in each case being "feraiseexcept( FE_DIVBYZERO)" after
> preprocessing.
> That function is defined in fenv.h with a "__THROW" attribute,
> but I saw no sign of it being an inline asm or anything.
> I don't understand "__THROW".
>
> I'm afraid I would need to find the asm code involved, before I could
> see what "operand constraints" are "inconsistent".
> Any hints where to look?
> Any way to make the call go to a nice simple library instead?
In the file numpy/core/include/numpy/ufuncobject.h, there is a stanza
that looks like this:
#if defined(__GLIBC__) || defined(__APPLE__) || defined(__MINGW32__)
|| defined(__FreeBSD__)
#include
#elif defined(__CYGWIN__)
#include "fenv/fenv.c"
#endif
I assume that you have __GLIBC__ defined. You will have to find your
platform's fenv.[ch] file from your libc sources. You may want to
comment out all of that and use our included
#include "fenv/fenv.c"
Also, edit numpy/core/setup.py to include these files for your
platform in addition to Cygwin.
# Don't install fenv unless we need them.
if sys.platform == 'cygwin':
config.add_data_dir('include/numpy/fenv')
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
-- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] powerpc yellow dog linux port of numpy
I reported back on August 30 to this list, with some discussion following on September 4 and 5, about my attempt to build numpy on an ancient powerpc setup. I'm running yellow dog linux 2.1, gcc 2.95.3.20010111, on processors from Curtiss-Wright Controls. Don't tell me to just upgrade; this configuration will be fighting the good fight for several more years. I just retried with the latest numpy (svn yesterday) and gotten further than I did before. umathmodule.c gets many compiler errors from gcc, of two kinds. The simpler were like warning: conflicting types for built-in function `sinl' repeated for `cosl', `fabsl', and `sqrtl'. These seem to be caused by npy_longdouble being typedef'ed as double not long double, due to the latter two types having the same size. umathmodule.c defines its own sinl, sqrtl, etc. with npy_longdouble arguments and results, which then conflict with the builtin sinl, sqrtl provided by gcc that expect long double. I worked around that by adding the "-fno-builtin" argument to the extra_compiler_args in setup.py. The other compiler complaints from the same file were: inconsistent operand constraints in an `asm' which came from every line that raised a division by zero exception, the code in each case being "feraiseexcept( FE_DIVBYZERO)" after preprocessing. That function is defined in fenv.h with a "__THROW" attribute, but I saw no sign of it being an inline asm or anything. I don't understand "__THROW". I'm afraid I would need to find the asm code involved, before I could see what "operand constraints" are "inconsistent". Any hints where to look? Any way to make the call go to a nice simple library instead? Vincent Broman ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] [OT - IPython] Old 'broken terminal' bug finally fixed
[ Sorry for the cross-post, but I know this is something that has hit quite a few people on this list. If you have any questions on it, please ask on the ipython list, this is just an FYI ] Hi all, there's a very old, *extremely* annoying bug that multiple people have asked about (on list and in person) that is finally just fixed. The behavior was that at some point during a normal session, after a call to 'foo?', your terminal would be totally messed up, with no displayed input. You could (blindly) type !reset to issue the system terminal reset command, but that would only help until the next time foo? was called, and the problem would then return. Most of us would end up just quitting ipython and restarting, often losing useful session state. The problem with this is that we never knew how to reliably reproduce it... Anyway, it's fixed now in current bzr: http://bazaar.launchpad.net/~ipython/ipython/stable-dev/revision/79 I don't actually know how to trigger it, but it hit me during an important session where I really couldn't afford to lose what I was working on, and I managed to track it down to what I'm pretty sure is a curses bug. Basically curses.initscr() fails to correctly initialize the terminal sometimes (I still don't have a clue why), and after that it's all lost. But it turns out that via termios one can in fact reset the terminal state reliably , so now we unconditionally do that. Anyway, I figured this would be worth mentioning here, since I know the problem is one that quite often bites people in the middle of work sessions an it can be very, very annoying. Cheers, f back to what I was busy doing, with my terminal now fully functional again... :) ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Truth value of an array
On 18/04/2008, Robert Kern <[EMAIL PROTECTED]> wrote: > On Fri, Apr 18, 2008 at 3:33 PM, Joe Harrington <[EMAIL PROTECTED]> wrote: > > For that matter, is there a reason logical operations don't work on > > arrays other than booleans? What about: > > The keywords "and", "or", and "not" only work on bool objects; Python > tries to convert the operands using bool(). bool(some_array) raises > the exception just as we've been discussing. This is a bit misleading, as the actual value is returned: In [167]: "" or "A" Out[167]: 'A' In [168]: "A" and "B" Out[168]: 'B' In fact the problem is that "and" and "or" are short-circuiting, which means that "return X or Y" is equivalent to something like: if X: return X elif Y: return Y else: return False These are handled specially by syntax so that, for example, you can do "False and 1/0" and not raise a ZeroDivisionError. So "and" and "or" aren't even really operators, and it's not just impossible to change them but unlikely to ever become possible. Just cast your arrays to booleans if you want to do boolean operations on them. Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Truth value of an array
On Fri, 18 Apr 2008, Joe Harrington apparently wrote: > For that matter, is there a reason logical operations don't work on > arrays other than booleans? What about: > import numpy > x = numpy.ones((10), dtype='Bool') > y = numpy.ones((10), dtype='Bool') > y[6] = False > z = x and y # logical AND: this one fails with an error about arrays It's the same reason. (See the other comments in this thread.) Do this:: x = numpy.ones((10), dtype='Bool') y = numpy.ones((10), dtype='Bool') y[6] = False z= x&y hth, Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Truth value of an array
On Fri, Apr 18, 2008 at 3:33 PM, Joe Harrington <[EMAIL PROTECTED]> wrote: > For that matter, is there a reason logical operations don't work on > arrays other than booleans? What about: The keywords "and", "or", and "not" only work on bool objects; Python tries to convert the operands using bool(). bool(some_array) raises the exception just as we've been discussing. > Here, both x and y are completely True, but you can't trivially do a > logical AND, and the bitwise & isn't the one you want. You have to > cast to boolean manually and do the logical instead. > > Can this be fixed or is there a reason for the exception? No, it cannot be fixed. Python does not give us a way to overload the behavior of these keywords. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Truth value of an array
For that matter, is there a reason logical operations don't work on arrays other than booleans? What about: import numpy x = numpy.ones((10), dtype='Bool') y = numpy.ones((10), dtype='Bool') y[6] = False z = x and y # logical AND: this one fails with an error about arrays --- Traceback (most recent call last) /home/jh/ in () : The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() --- z = x & y # bitwise AND: this one succeeds print z [ True True True True True True False True True True] But, both operators should return the same result, since booleans have one bit. Where this really matters is ANDing ints: x = numpy.ones((10), dtype='uint8') y = numpy.ones((10), dtype='uint8') y[6] = 2 # still True this time: http://docs.python.org/ref/Booleans.html z = x and y # logical AND: this one fails with an error about arrays --- Traceback (most recent call last) /home/jh/ in () : The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() --- z = x & y # bitwise AND: this one succeeds...but it isn't the AND I want! print z [1 1 1 1 1 1 0 1 1 1] # the AND I want makes this array all True Here, both x and y are completely True, but you can't trivially do a logical AND, and the bitwise & isn't the one you want. You have to cast to boolean manually and do the logical instead. Can this be fixed or is there a reason for the exception? --jh-- ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Truth value of an array
On 18/04/2008, Olivier <[EMAIL PROTECTED]> wrote: > Let's restrict the discussion the case to boolean arrays (dtype bool), > since all the comparisons (A==B, A!=B, A arrays). > > So I have an array filled with booleans. Is there a reason not to map > "bool(A)" to "A.all()" but instead raise an exception? > > As far as I can see, "if A==B" is clear enough; one always means > "(A==B).all()". Isn't that so? > > I would like to see examples where there is clearly an ambiguity so > that I understand the benefit of having an exception raised for > boolean matrices instead of simply using all(). > > If there is no such clear example, then why not map "bool(A)" to > "A.all()" in numpy? In [1]: import numpy as np In [2]: A = np.array([0,1]) In [3]: B = np.array([0,0]) In [4]: (A==B).all() Out[4]: False In [5]: (A!=B).all() Out[5]: False One would expect A!=B to be the logical opposite of A==B, but with your proposed suggestion it is not. In math, when comparing functions, one can compare the functions as a whole, or one can compare them pointwise. numpy's implementation does pointwise comparison, for a variety of good reasons. As for why converting an array to a boolean doesn't automatically do all(), if you don't know you are dealing with an array and using all(), you will surely shoot yourself in the foot sooner rather than later (as the above example shows). If you do, simply wrapping it in all() is easy and clear. Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Truth value of an array
Hi, I must say, I agree with the other posters here, that it is not completely obvious to me that: a = np.array([True, False]) bool(a) should return False. Especially given: L = [True, False] bool(L) returns True. Given that it's not completely obvious, and a.all() is completely obvious, and readable, the current behavior seems right to me... Otherwise someone somewhere is going to assume that bool(a) does the same as a.any(), and run into problems. Explicit is better than implicit... Best, Matthew ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Truth value of an array
Let's restrict the discussion the case to boolean arrays (dtype bool), since all the comparisons (A==B, A!=B, A wrote: > On Fri, 18 Apr 2008, Olivier Verdier apparently wrote: > > What is ambiguous about "bool(A==B)"? > > A==B is an array. Compare: > > >>> bool([]) > False > >>> bool([0]) > True > > Even if you decide the second should be false, > what about [0,1]? (I.e., all or any?) > > Cheers, > Alan Isaac > > ___ > Numpy-discussion mailing list > [EMAIL PROTECTED]://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Truth value of an array
On Fri, 18 Apr 2008, Olivier Verdier apparently wrote: > What is ambiguous about "bool(A==B)"? A==B is an array. Compare: >>> bool([]) False >>> bool([0]) True Even if you decide the second should be false, what about [0,1]? (I.e., all or any?) Cheers, Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Truth value of an array
On Fri, Apr 18, 2008 at 8:43 AM, Matthieu Brucher <[EMAIL PROTECTED]> wrote: > > > 2008/4/18, Olivier Verdier <[EMAIL PROTECTED]>: > > I certainly didn't mean that "A==B" should return a boolean!! > > > > "A==B" should return an array of boolean as it does now. This is all > right. > > > > *However* "bool(A==B)" should return a boolean, *not* raise an > > exception. Why raise an exception? What is ambiguous about > > "bool(A==B)"?? > > > > This is what happens when you write "if A==B" because then > > "bool(A==B)" is somehow triggered and bang, an exception is raised. > > > > As I tried to explain, the default behaviour should be that "bool(A)" > > return "A.all()" but not an exception. Why is there an exception > > raising at all? > > Sometimes, you want .all(), sometimes .any(). It really depends on the > question you are asking. Even for bool(A), there is no easy answer. In some > case I want True if some elements are true, in other cases only if all > elements are true. > I would agree with you some years ago, but after using bool(A) = A.all(), I > started noticing that it brakes my coding instead of speeding it and does > not give me any benefit at all. Not only might reasonable people differ on want any or all semantics, it is also quite reasonable to want __nonzero__ to merely indicate whether or not the ndarray has any elements in the first dimension (i.e. len >0) like the built-in list and array. At least I think this is reasonable as this is the semantic that I often want! But seeing how there are multiple prior conceptions, perhaps the current behavior is best because it forces us to make sure what we think is being returned is in fact being returned? Regards, Alex ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] fast take patch
Hi Eric On 18/04/2008, Eric Firing <[EMAIL PROTECTED]> wrote: > The motivation for the patch is that in matplotlib color mapping, such as [...] Beautiful patch, good motivation. Thank you! Applied in r5044. Regards Stéfan ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Truth value of an array
2008/4/18, Olivier Verdier <[EMAIL PROTECTED]>: > > I certainly didn't mean that "A==B" should return a boolean!! > > "A==B" should return an array of boolean as it does now. This is all > right. > > *However* "bool(A==B)" should return a boolean, *not* raise an > exception. Why raise an exception? What is ambiguous about > "bool(A==B)"?? > > This is what happens when you write "if A==B" because then > "bool(A==B)" is somehow triggered and bang, an exception is raised. > > As I tried to explain, the default behaviour should be that "bool(A)" > return "A.all()" but not an exception. Why is there an exception > raising at all? Sometimes, you want .all(), sometimes .any(). It really depends on the question you are asking. Even for bool(A), there is no easy answer. In some case I want True if some elements are true, in other cases only if all elements are true. I would agree with you some years ago, but after using bool(A) = A.all(), I started noticing that it brakes my coding instead of speeding it and does not give me any benefit at all. Matthieu -- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Truth value of an array
I certainly didn't mean that "A==B" should return a boolean!! "A==B" should return an array of boolean as it does now. This is all right. *However* "bool(A==B)" should return a boolean, *not* raise an exception. Why raise an exception? What is ambiguous about "bool(A==B)"?? This is what happens when you write "if A==B" because then "bool(A==B)" is somehow triggered and bang, an exception is raised. As I tried to explain, the default behaviour should be that "bool(A)" return "A.all()" but not an exception. Why is there an exception raising at all? I hope that my question is clearer now... Thanks. On 18/04/2008, David Douard <[EMAIL PROTECTED]> wrote: > On Fri, Apr 18, 2008 at 01:11:37PM +0200, Olivier Verdier wrote: > > In mathematics, if I compare two function, it means that I compare on > > all its "coordinates". If I say "f < g" I mean "f(x) < g(x) for all > > x". > > > > The same holds for a vector, if I write "v == w" I mean "v[i] == w[i] > > for all i". > > > > How come this doesn't work in numpy? And why the message about the > > truth value of an array being ambiguous? What is ambiguous? > > > > In my opinion any boolean array should be cast with all() > > automatically so that people can simply write: > > if v == w: > > ... > > > No. A comparison operator on ndarray produce an ndarray of booleans. So > you have to write > > if numpy.alltrue(u == v): >blah > > This is a very good idea, since it allows you to also write things > like: > > u[v<0] = 0 > > Which you could definitely not do if the comparison returns a scalar > bool instead of a ndarray. > > More, you may also want to write things like: > > if numpy.sometrue(u<0): >blah > > Same remark. > > > > > > > > Or is there a good reason why this is not possible? > > > > Thank you! > > > ___ > > Numpy-discussion mailing list > > Numpy-discussion@scipy.org > > http://projects.scipy.org/mailman/listinfo/numpy-discussion > > > > > -- > David DouardLOGILAB, Paris (France), +33 1 45 32 03 > 12 > Formations Python, Zope, Debian : http://www.logilab.fr/formations > Développement logiciel sur mesure : http://www.logilab.fr/services > Informatique scientifique : http://www.logilab.fr/science > > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.6 (GNU/Linux) > > iD8DBQFICIwl15pXfQ0A2uMRAm/3AJ9nqNHSH7vY5NpIULCXjgXdqkCUUgCfdxyi > NIYA8A4grCPDp5lShTKhcoY= > =S6VB > -END PGP SIGNATURE- > > ___ > Numpy-discussion mailing list > Numpy-discussion@scipy.org > http://projects.scipy.org/mailman/listinfo/numpy-discussion > > > ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Truth value of an array
On Fri, Apr 18, 2008 at 01:11:37PM +0200, Olivier Verdier wrote: > In mathematics, if I compare two function, it means that I compare on > all its "coordinates". If I say "f < g" I mean "f(x) < g(x) for all > x". > > The same holds for a vector, if I write "v == w" I mean "v[i] == w[i] > for all i". > > How come this doesn't work in numpy? And why the message about the > truth value of an array being ambiguous? What is ambiguous? > > In my opinion any boolean array should be cast with all() > automatically so that people can simply write: > if v == w: > ... No. A comparison operator on ndarray produce an ndarray of booleans. So you have to write if numpy.alltrue(u == v): blah This is a very good idea, since it allows you to also write things like: u[v<0] = 0 Which you could definitely not do if the comparison returns a scalar bool instead of a ndarray. More, you may also want to write things like: if numpy.sometrue(u<0): blah Same remark. > > Or is there a good reason why this is not possible? > > Thank you! > ___ > Numpy-discussion mailing list > Numpy-discussion@scipy.org > http://projects.scipy.org/mailman/listinfo/numpy-discussion > -- David DouardLOGILAB, Paris (France), +33 1 45 32 03 12 Formations Python, Zope, Debian : http://www.logilab.fr/formations Développement logiciel sur mesure : http://www.logilab.fr/services Informatique scientifique : http://www.logilab.fr/science signature.asc Description: Digital signature ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Truth value of an array
In mathematics, if I compare two function, it means that I compare on all its "coordinates". If I say "f < g" I mean "f(x) < g(x) for all x". The same holds for a vector, if I write "v == w" I mean "v[i] == w[i] for all i". How come this doesn't work in numpy? And why the message about the truth value of an array being ambiguous? What is ambiguous? In my opinion any boolean array should be cast with all() automatically so that people can simply write: if v == w: ... Or is there a good reason why this is not possible? Thank you! ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] fast take patch
Stefan, (or anyone else who might be interested)
Since you committed my fast putmask patch many months ago, I thought you
might like to deal with my fast take patch. Attached is the diff
relative to 5043, ignoring whitespace. (Yes, those pesky whitespace
anomalies are still cropping up.)
The motivation for the patch is that in matplotlib color mapping, such
as when displaying an image or pcolor plot, the time-limiting factor can
be a single numpy operation of indexing into the color map.
(http://www.mail-archive.com/[EMAIL PROTECTED]/msg06482.html)
It was being done with fancy indexing. I changed that to use the take
method, which sped it up by a factor of two, but I found that take
itself is slower than it needs to be by a factor of 2-3. As with
putmask, the culprit is memmove, so I used the same strategy to
substitute direct variable assignment when possible.
The patch includes tests for the take method. I did not find any such
pre-existing tests.
Thanks for taking a look. Let me know what needs to be changed.
Eric
Index: numpy/core/include/numpy/ndarrayobject.h
===
--- numpy/core/include/numpy/ndarrayobject.h(revision 5043)
+++ numpy/core/include/numpy/ndarrayobject.h(working copy)
@@ -1052,6 +1052,10 @@
void *max, void *out);
typedef void (PyArray_FastPutmaskFunc)(void *in, void *mask, npy_intp n_in,
void *values, npy_intp nv);
+typedef int (PyArray_FastTakeFunc)(void *dest, void *src, npy_intp *indarray,
+ npy_intp nindarray, npy_intp n_outer,
+ npy_intp m_middle, npy_intp nelem,
+ NPY_CLIPMODE clipmode);
typedef struct {
npy_intp *ptr;
@@ -1130,6 +1134,7 @@
PyArray_FastClipFunc *fastclip;
PyArray_FastPutmaskFunc *fastputmask;
+PyArray_FastTakeFunc *fasttake;
} PyArray_ArrFuncs;
#define NPY_ITEM_REFCOUNT 0x01 /* The item must be reference counted
Index: numpy/core/src/multiarraymodule.c
===
--- numpy/core/src/multiarraymodule.c (revision 5043)
+++ numpy/core/src/multiarraymodule.c (working copy)
@@ -3824,11 +3824,13 @@
PyArray_TakeFrom(PyArrayObject *self0, PyObject *indices0, int axis,
PyArrayObject *ret, NPY_CLIPMODE clipmode)
{
+PyArray_FastTakeFunc *func;
PyArrayObject *self, *indices;
-intp nd, i, j, n, m, max_item, tmp, chunk;
+intp nd, i, j, n, m, max_item, tmp, chunk, nelem;
intp shape[MAX_DIMS];
char *src, *dest;
int copyret=0;
+int err;
indices = NULL;
self = (PyAO *)_check_axis(self0, &axis, CARRAY);
@@ -3892,10 +3894,14 @@
}
max_item = self->dimensions[axis];
+nelem = chunk;
chunk = chunk * ret->descr->elsize;
src = self->data;
dest = ret->data;
+func = self->descr->f->fasttake;
+if (func == NULL) {
+
switch(clipmode) {
case NPY_RAISE:
for(i=0; idata),
+max_item, n, m, nelem, clipmode);
+if (err) goto fail;
+}
PyArray_INCREF(ret);
Index: numpy/core/src/arraytypes.inc.src
===
--- numpy/core/src/arraytypes.inc.src (revision 5043)
+++ numpy/core/src/arraytypes.inc.src (working copy)
@@ -2179,6 +2179,89 @@
#define OBJECT_fastputmask NULL
+
+/
+ * Fast take functions
+ */
+
+/**begin repeat
+#name=BOOL,BYTE, UBYTE, SHORT, USHORT, INT, UINT, LONG, ULONG, LONGLONG,
ULONGLONG, FLOAT, DOUBLE, LONGDOUBLE,CFLOAT, CDOUBLE, CLONGDOUBLE#
+#type= Bool, byte, ubyte, short, ushort, int, uint, long, ulong, longlong,
ulonglong, float, double, longdouble,cfloat, cdouble, clongdouble#
+*/
+static int
[EMAIL PROTECTED]@_fasttake(@type@ *dest, @type@ *src, intp *indarray,
+intp nindarray, intp n_outer,
+intp m_middle, intp nelem,
+NPY_CLIPMODE clipmode)
+{
+intp i, j, k, tmp;
+
+switch(clipmode) {
+case NPY_RAISE:
+for(i=0; i= nindarray)) {
+PyErr_SetString(PyExc_IndexError,
+"index out of range "\
+"for array");
+return 1;
+}
+if (nelem == 1) *dest++ = *(src+tmp);
+else {
+for(k=0; k= nindarray)
+while (tmp >= nindarray)
+tmp -= nindarray;
+if (nelem == 1) *dest++ = *(src+tmp);
+else {
+for(k=0; k= nindarray)
+tmp = nindarray-1;
+if (nelem == 1) *dest++ = *(src+tmp);
+else {
+for(k=0; ki4','f8'), ('z', '__
