[Numpy-discussion] Review of issue 825

[Neil Muller]

Could someone review the patch at
http://scipy.org/scipy/numpy/ticket/825, please?

The issue prevents the test suite running successfully on Sparc, so
I'd like to see it fixed.

-- 
Neil Muller
[EMAIL PROTECTED]
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Request for clarification from Travis

[Charles R Harris]

On Tue, Jun 24, 2008 at 8:40 PM, Travis E. Oliphant <[EMAIL PROTECTED]>
wrote:

> Charles R Harris wrote:
> >
> >
> > On Tue, Jun 24, 2008 at 1:41 PM, Travis E. Oliphant
> > <[EMAIL PROTECTED] > wrote:
> >
> >
> > > The problem is that numpy has a *bug*. I am trying to fix it and I
> > > want your help figuring out how to do so while preserving the
> > behavior
> > > you had in mind. So if you could point out specific cases you were
> > > thinking of it would help me with this.
> > This is a hack to support matrices and other subclasses of ndarray
> > like
> > masked arrays.
> >
> >
> > That's what I was thinking. Do you have a test case? One that failed
> > without the NotImplemented return would be very helpful.
> There should be some in the test suite, but I'm not sure.  Perhaps the
> masked array test suite also has test cases.
>
> The NotImplemented is being returned so that the other operation will
> try and use it if it can.
>

Yes, but it is directed at the interpreter, which will raise a TypeError if
needed. But the interpreter doesn't know that some generic function might
return NotImplemented and wouldn't know what to do if it did. What the user
should see when they call something like right_shift(a,b) and it fails is an
exception, they shouldn't have to check the return value.

I think what you want is that subtypes of ndarray can override some ufuncs
so that, for instance, right_shift() should interpret itself as a call to
arg2.__rrshift__ if it exists, even if it isn't called through the
interpreter friendly slot function >>. Is that correct? If so, I would
rather have an explicit overload flag in the subtype so that the ufunc can
easily do the right thing.

I can't find __array_priority__ in your book, what exactly does it imply?

Chuck







> -Travis
>
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Request for clarification from Travis

[Travis E. Oliphant]

Charles R Harris wrote:
>
>
> On Tue, Jun 24, 2008 at 1:41 PM, Travis E. Oliphant 
> <[EMAIL PROTECTED] > wrote:
>
>
> > The problem is that numpy has a *bug*. I am trying to fix it and I
> > want your help figuring out how to do so while preserving the
> behavior
> > you had in mind. So if you could point out specific cases you were
> > thinking of it would help me with this.
> This is a hack to support matrices and other subclasses of ndarray
> like
> masked arrays.
>
>
> That's what I was thinking. Do you have a test case? One that failed 
> without the NotImplemented return would be very helpful.
There should be some in the test suite, but I'm not sure.  Perhaps the 
masked array test suite also has test cases.

The NotImplemented is being returned so that the other operation will 
try and use it if it can. 

-Travis

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Request for clarification from Travis

[Charles R Harris]

On Tue, Jun 24, 2008 at 1:41 PM, Travis E. Oliphant <[EMAIL PROTECTED]>
wrote:

>
> > The problem is that numpy has a *bug*. I am trying to fix it and I
> > want your help figuring out how to do so while preserving the behavior
> > you had in mind. So if you could point out specific cases you were
> > thinking of it would help me with this.
> This is a hack to support matrices and other subclasses of ndarray like
> masked arrays.
>

That's what I was thinking. Do you have a test case? One that failed without
the NotImplemented return would be very helpful.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Request for clarification from Travis

[Travis E. Oliphant]

> The problem is that numpy has a *bug*. I am trying to fix it and I 
> want your help figuring out how to do so while preserving the behavior 
> you had in mind. So if you could point out specific cases you were 
> thinking of it would help me with this.
This is a hack to support matrices and other subclasses of ndarray like 
masked arrays.

-Travis

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Request for clarification from Travis

[Charles R Harris]

On Tue, Jun 24, 2008 at 12:46 PM, Travis E. Oliphant <[EMAIL PROTECTED]>
wrote:

> Charles R Harris wrote:
> > Hi Travis,
> >
> > Could you expand on your thinking  concerning NotImplemented? The
> > relevant code is:
> >
> > /*
> >  * FAIL with NotImplemented if the other object has
> >  * the __r__ method and has __array_priority__ as
> >  * an attribute (signalling it can handle ndarray's)
> >  * and is not already an ndarray
> >  */
> > if ((arg_types[1] == PyArray_OBJECT) && \
> > (loop->ufunc->nin==2) && (loop->ufunc->nout == 1)) {
> > PyObject *_obj = PyTuple_GET_ITEM(args, 1);
> > if (!PyArray_CheckExact(_obj) &&\
> > PyObject_HasAttrString(_obj, "__array_priority__") && \
> > _has_reflected_op(_obj, loop->ufunc->name)) {
> > loop->notimplemented = 1;
> > return nargs;
> > }
> > }
> >
> > And the check is made after the needed promotions have been determined
> > and stored in arg_types. Consequently if a type is promoted to an
> > object array, as happens for the shift operations with floating
> > scalars, the rest of the conditions will be checked and pass because
> > numpy scalar types aren't arrays. I wonder if that is what you
> > intended here?
> I'm not sure, I don't understand what problem you are trying to solve.
>

1) NotImplemented is associated to specific types, hence the CHECKABLE flag
on the type.
2) It is a message to the python interpreter, not to the user.
3) It is associated with the slot functions of numeric types, not with
general functions.

When python encounters something like a + b, it looks at the CHECKABLE flag
on a. If set, it calls a.__add__ with b as is, not attempting to promote the
type. If that fails, a.__add__ returns NotImplemented, telling the
interpreter to do something, probably try the b.__radd__. If that fails it
raises an error.

If the *user* ever sees NotImplemented, it is a bug. Some of the numpy
ufuncs exhibit this bug and we should fix it.


> The problem is that NumPy is very aggressive and tries to handle
> everything so if I let the code get past this point, it would run
> successfully as an object array which is not what is necessarily wanted
>

The problem is that numpy has a *bug*. I am trying to fix it and I want your
help figuring out how to do so while preserving the behavior you had in
mind. So if you could point out specific cases you were thinking of it would
help me with this.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ndarray methods vs numpy module functions

[Sebastian Haase]

On Tue, Jun 24, 2008 at 7:40 PM, Robert Kern <[EMAIL PROTECTED]> wrote:
> On Tue, Jun 24, 2008 at 02:43, Sebastian Haase <[EMAIL PROTECTED]> wrote:
>> On Tue, Jun 24, 2008 at 3:11 AM, Robert Kern <[EMAIL PROTECTED]> wrote:
>>> On Mon, Jun 23, 2008 at 19:35, Ryan May <[EMAIL PROTECTED]> wrote:
 Robert Kern wrote:
> On Mon, Jun 23, 2008 at 18:10, Sebastian Haase <[EMAIL PROTECTED]> wrote:
>> On Mon, Jun 23, 2008 at 10:31 AM, Bob Dowling <[EMAIL PROTECTED]> wrote:
>>> [ I'm new here and this has the feel of an FAQ but I couldn't find
>>> anything at http://www.scipy.org/FAQ .  If I should have looked
>>> somewhere else a URL would be gratefully received. ]
>>>
>>>
>>> What's the reasoning behind functions like sum() and cumsum() being
>>> provided both as module functions (numpy.sum(data, axis=1)) and as
>>> object methods (data.sum(axis=1)) but other functions - and I stumbled
>>> over diff() - only being provided as module functions?
>>>
>>>
>> Hi Bob,
>> this is a very good question.
>> I think the answers are
>> a) historical reasons AND, more importantly, differing personal 
>> preferences
>> b) I would file  the missing data.diff() as a bug.
>
> It's not.
>
 Care to elaborate?
>>>
>>> There is not supposed to be a one-to-one correspondence between the
>>> functions in numpy and the methods on an ndarray. There is some
>>> duplication between the two, but that is not a reason to make more
>>> duplication.
>>>
>> Are you saying the duplication is "just random" ?
>
> No. If you want a clearer (but still imperfect) dividing line is that
> all of the methods are implemented in C. numpy.diff(), for example, is
> not. A lot of the C functions in Numeric's core (but not FFT, SVD,
> etc.) got moved into methods, partly for implementation reasons
> (ndarray is subclassable now, so methods are easier to make generic),
> and partly for "it seemed like a good idea at the time." We couldn't
> remove the functions, then, if we wanted any kind of continuity with
> Numeric, and we certainly can't now.
>
>> It would be better
>> -- as in: principle of minimum surprise -- if there would be some sort
>> "reasonable set" of duplicates 
>> If there are only a handful functions missing, why not try to make it 
>> complete.
>
> There aren't.
>
>> ( Again, a list of functions vs. methods on the wiki would clarify
>> what we are talking about )
>
> Go ahead.
>
>> Just thinking loudly of course. Don't take this as an offense .
>
> It's not that you're being offensive, but you are kicking up dust on
> an old argument that was settled before 1.0, when it actually
> mattered.

Just for the record: I like it the way it is. I did follow the
discussion at the time, and while there was a real (historical) need
to keep functions, there were arguments for supporting a two paradigms
and thus also having methods. "Numpy serves both people coming from
the Matlab/IDL community and people coming from (strict /
method-based) OO programming".

Regarding the OP, if he stumbled over this, like probably many other
"newcomers", it should go into the FAQ section. (at
http://www.scipy.org/FAQ )
("For implementation reasons some operations are only available as
functions or as methods respectively")

Cheers,
- Sebastian Haase
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] embedded arrays- still an issue

[Thomas Hrabe]

Hi all, 


I finally managed to come back to my memory troubles which I have described 
recently to this group (see below). Here is the requested backtrace of my 
program:

#0  0x94659b88 in strlen ()
#1  0x004437b8 in PyString_FromFormatV (format=0x4f2360 "'%.50s' object has no 
attribute '%.400s'", vargs=0xbfffee00 "") at 
/Users/ronald/r252/Objects/stringobject.c:202
#2  0x004a57e8 in PyErr_Format (exception=0x505fb4, format=0xbfffee00 "") at 
/Users/ronald/r252/Python/errors.c:522
#3  0x00432430 in PyObject_GetAttr (v=0x80ef40, name=0xc5278) at 
/Users/ronald/r252/Objects/object.c:1130
#4  0x004324b8 in PyObject_GetAttrString (v=0x80ef40, name=0x0) at 
/Users/ronald/r252/Objects/object.c:1069
#5  0x00b47660 in PyArray_FromStructInterface ()
#6  0x00b47938 in PyArray_FromAny ()
#7  0x00b72fb4 in array_richcompare ()
#8  0x004341f0 in PyObject_RichCompare (v=0x80ef40, w=0x8b9440, op=2) at 
/Users/ronald/r252/Objects/object.c:874
#9  0x00490d20 in PyEval_EvalFrameEx (f=0x8b9090, throwflag=9146828) at 
/Users/ronald/r252/Python/ceval.c:3989
#10 0x00493e50 in PyEval_EvalCodeEx (co=0xf4d578, globals=0xbfffee00, 
locals=0x4437b8, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, 
defcount=0, closure=0x0) at /Users/ronald/r252/Python/ceval.c:2836
#11 0x00493ff0 in PyEval_EvalCode (co=0x0, globals=0xbfffee00, locals=0x0) at 
/Users/ronald/r252/Python/ceval.c:494
#12 0x004b9b24 in PyRun_InteractiveOneFlags (fp=0x0, filename=0x4f88f0 
"", flags=0xb448) at /Users/ronald/r252/Python/pythonrun.c:1273
#13 0x004b9d30 in PyRun_InteractiveLoopFlags (fp=0xa08bf374, filename=0x4f88f0 
"", flags=0xb448) at /Users/ronald/r252/Python/pythonrun.c:723
#14 0x004ba3f0 in PyRun_AnyFileExFlags (fp=0xa08bf374, filename=0x4f88f0 
"", closeit=0, flags=0xb448) at 
/Users/ronald/r252/Python/pythonrun.c:692
#15 0x004c9a9c in Py_Main (argc=1, argv=0xb5d8) at 
/Users/ronald/r252/Modules/main.c:523
#16 0x19bc in ?? ()
#17 0x16c0 in ?? ()
(gdb) 
in /Users/ronald/r252/Objects/stringobject.c

Please let me know if there are any ways how to go on from this point. I am new 
into this kind of debugging...

Thank you in advance for your help,
Thomas

-Ursprüngliche Nachricht-
Von: [EMAIL PROTECTED] im Auftrag von Robert Kern
Gesendet: Fr 06.06.2008 15:27
An: Discussion of Numerical Python
Betreff: Re: [Numpy-discussion] embedded arrays
 
On Fri, Jun 6, 2008 at 17:10, Thomas Hrabe <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> while writing a extension module in C++ for python & numpy, I find a strange
> error.
>
> I can send and retrieve numpy arrays to and from my module.
> But python stops if I do the following:
>
> a = numpy.array([[1.1,2,3],[4,5,6]])
> PM.put(a,'a')   //send a to the module
> b = PM.get('a') //get a identical copy from the module
> print b
> array([[ 1.1,  2. ,  3. ],
>[ 4. ,  5. ,  6. ]])
>
> b.__class__
> Out[36]: 
>
>
> perfect, until
> a == b
>
> the interpreter does not continue from here...
> I can add values to to b, everything, but a == b simply crashes ...?
>
> Does anybody have a clue for this problem?

Not really. It probably depends on some details with your interfacing.
Since we don't have access to your code, we don't have much to go on.
You might have buggy reference counting or perhaps you gave the numpy
ndarray ownership of the array's memory when it actually shouldn't.
Memory issues can be a bit finicky where everything will work for a
while, then crash.

Try running your program under a C debugger like gdb so we can get a
backtrace. That might give us some more ideas about exactly where
problems are occurring.




<>___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Keywords in wrapped functions

[Travis E. Oliphant]

Pierre GM wrote:
> All,
> Sorry to bumpt the post, accept my apologies for my rudeness, but I'm 
> curious...
> So, let me rephrase my question:
>
> Many numpy functions (min, max, sum...) based on ndarray methods have a 
> construction of the style
> #---
> def amin(a, axis=None, out=None):
> try:
> amin = a.min
> except AttributeError:
> return _wrapit(a, 'min', axis, out)
> return amin(axis, out)
> #---
>
> I'm wondering why the initial keywords arguments are transformed into 
> positional arguments in the return. In the previous example, I would expect a 
> line like
> return amin(axis=axis, out=out)
>
> [Actually, I would even prefer a 
> return amin(axis=axis, out=out, **other_optional_parameters), which would 
> permit to add some extra parameters such as a "fill_value", when dealing with 
> masked arrays ?]
>
> Is this for efficiency purposes ?
>
>
>   
The keyword arguments to the methods were added later and so positional 
argument were initially necessary.  I don't think they are now, but I 
have not verified that.

-Travis

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Request for clarification from Travis

[Travis E. Oliphant]

Charles R Harris wrote:
> Hi Travis,
>
> Could you expand on your thinking  concerning NotImplemented? The 
> relevant code is:
>
> /*
>  * FAIL with NotImplemented if the other object has
>  * the __r__ method and has __array_priority__ as
>  * an attribute (signalling it can handle ndarray's)
>  * and is not already an ndarray
>  */
> if ((arg_types[1] == PyArray_OBJECT) && \
> (loop->ufunc->nin==2) && (loop->ufunc->nout == 1)) {
> PyObject *_obj = PyTuple_GET_ITEM(args, 1);
> if (!PyArray_CheckExact(_obj) &&\
> PyObject_HasAttrString(_obj, "__array_priority__") && \
> _has_reflected_op(_obj, loop->ufunc->name)) {
> loop->notimplemented = 1;
> return nargs;
> }
> }
>
> And the check is made after the needed promotions have been determined 
> and stored in arg_types. Consequently if a type is promoted to an 
> object array, as happens for the shift operations with floating 
> scalars, the rest of the conditions will be checked and pass because 
> numpy scalar types aren't arrays. I wonder if that is what you 
> intended here? 
I'm not sure, I don't understand what problem you are trying to solve.

> The reason I ask is that this determination should be associated with 
> the standard slot functions when the call is made and I don't want to 
> worry about type promotions. Further, I wonder why you want this 
> return before the attempt to run the function with the current 
> arguments is made. So it would be very helpful if you could clarify 
> what you aim to achieve here.
This is to ensure that object arrays get a chance to run their own code 
if they have the required right-hand special name (e.g. __radd__).

The problem is that NumPy is very aggressive and tries to handle 
everything so if I let the code get past this point, it would run 
successfully as an object array which is not what is necessarily wanted

-Travis

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Keywords in wrapped functions

[Pierre GM]

On Tuesday 24 June 2008 14:11:12 Robert Kern wrote:
> On Tue, Jun 24, 2008 at 13:03, Pierre GM <[EMAIL PROTECTED]> wrote:

> > Is this for efficiency purposes ?
>
> Most likely, the author just didn't realize that it might matter. Go
> ahead with your changes.

OK, great. Thanks a lot!
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Keywords in wrapped functions

[Robert Kern]

On Tue, Jun 24, 2008 at 13:03, Pierre GM <[EMAIL PROTECTED]> wrote:
> All,
> Sorry to bumpt the post, accept my apologies for my rudeness, but I'm
> curious...
> So, let me rephrase my question:
>
> Many numpy functions (min, max, sum...) based on ndarray methods have a
> construction of the style
> #---
> def amin(a, axis=None, out=None):
>try:
>amin = a.min
>except AttributeError:
>return _wrapit(a, 'min', axis, out)
>return amin(axis, out)
> #---
>
> I'm wondering why the initial keywords arguments are transformed into
> positional arguments in the return. In the previous example, I would expect a
> line like
> return amin(axis=axis, out=out)
>
> [Actually, I would even prefer a
> return amin(axis=axis, out=out, **other_optional_parameters), which would
> permit to add some extra parameters such as a "fill_value", when dealing with
> masked arrays ?]
>
> Is this for efficiency purposes ?

Most likely, the author just didn't realize that it might matter. Go
ahead with your changes.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Request for clarification from Travis

[Charles R Harris]

Hi Travis,

Could you expand on your thinking  concerning NotImplemented? The relevant
code is:

/*
 * FAIL with NotImplemented if the other object has
 * the __r__ method and has __array_priority__ as
 * an attribute (signalling it can handle ndarray's)
 * and is not already an ndarray
 */
if ((arg_types[1] == PyArray_OBJECT) && \
(loop->ufunc->nin==2) && (loop->ufunc->nout == 1)) {
PyObject *_obj = PyTuple_GET_ITEM(args, 1);
if (!PyArray_CheckExact(_obj) &&\
PyObject_HasAttrString(_obj, "__array_priority__") && \
_has_reflected_op(_obj, loop->ufunc->name)) {
loop->notimplemented = 1;
return nargs;
}
}

And the check is made after the needed promotions have been determined and
stored in arg_types. Consequently if a type is promoted to an object array,
as happens for the shift operations with floating scalars, the rest of the
conditions will be checked and pass because numpy scalar types aren't
arrays. I wonder if that is what you intended here? The reason I ask is that
this determination should be associated with the standard slot functions
when the call is made and I don't want to worry about type promotions.
Further, I wonder why you want this return before the attempt to run the
function with the current arguments is made. So it would be very helpful if
you could clarify what you aim to achieve here.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Keywords in wrapped functions

[Pierre GM]

All,
Sorry to bumpt the post, accept my apologies for my rudeness, but I'm 
curious...
So, let me rephrase my question:

Many numpy functions (min, max, sum...) based on ndarray methods have a 
construction of the style
#---
def amin(a, axis=None, out=None):
try:
amin = a.min
except AttributeError:
return _wrapit(a, 'min', axis, out)
return amin(axis, out)
#---

I'm wondering why the initial keywords arguments are transformed into 
positional arguments in the return. In the previous example, I would expect a 
line like
return amin(axis=axis, out=out)

[Actually, I would even prefer a 
return amin(axis=axis, out=out, **other_optional_parameters), which would 
permit to add some extra parameters such as a "fill_value", when dealing with 
masked arrays ?]

Is this for efficiency purposes ?




___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ndarray methods vs numpy module functions

[Robert Kern]

On Tue, Jun 24, 2008 at 02:43, Sebastian Haase <[EMAIL PROTECTED]> wrote:
> On Tue, Jun 24, 2008 at 3:11 AM, Robert Kern <[EMAIL PROTECTED]> wrote:
>> On Mon, Jun 23, 2008 at 19:35, Ryan May <[EMAIL PROTECTED]> wrote:
>>> Robert Kern wrote:
 On Mon, Jun 23, 2008 at 18:10, Sebastian Haase <[EMAIL PROTECTED]> wrote:
> On Mon, Jun 23, 2008 at 10:31 AM, Bob Dowling <[EMAIL PROTECTED]> wrote:
>> [ I'm new here and this has the feel of an FAQ but I couldn't find
>> anything at http://www.scipy.org/FAQ .  If I should have looked
>> somewhere else a URL would be gratefully received. ]
>>
>>
>> What's the reasoning behind functions like sum() and cumsum() being
>> provided both as module functions (numpy.sum(data, axis=1)) and as
>> object methods (data.sum(axis=1)) but other functions - and I stumbled
>> over diff() - only being provided as module functions?
>>
>>
> Hi Bob,
> this is a very good question.
> I think the answers are
> a) historical reasons AND, more importantly, differing personal 
> preferences
> b) I would file  the missing data.diff() as a bug.

 It's not.

>>> Care to elaborate?
>>
>> There is not supposed to be a one-to-one correspondence between the
>> functions in numpy and the methods on an ndarray. There is some
>> duplication between the two, but that is not a reason to make more
>> duplication.
>>
> Are you saying the duplication is "just random" ?

No. If you want a clearer (but still imperfect) dividing line is that
all of the methods are implemented in C. numpy.diff(), for example, is
not. A lot of the C functions in Numeric's core (but not FFT, SVD,
etc.) got moved into methods, partly for implementation reasons
(ndarray is subclassable now, so methods are easier to make generic),
and partly for "it seemed like a good idea at the time." We couldn't
remove the functions, then, if we wanted any kind of continuity with
Numeric, and we certainly can't now.

> It would be better
> -- as in: principle of minimum surprise -- if there would be some sort
> "reasonable set" of duplicates 
> If there are only a handful functions missing, why not try to make it 
> complete.

There aren't.

> ( Again, a list of functions vs. methods on the wiki would clarify
> what we are talking about )

Go ahead.

> Just thinking loudly of course. Don't take this as an offense .

It's not that you're being offensive, but you are kicking up dust on
an old argument that was settled before 1.0, when it actually
mattered.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ndarray methods vs numpy module functions

[Robert Kern]

On Tue, Jun 24, 2008 at 02:33, Bob Dowling <[EMAIL PROTECTED]> wrote:
>> There is not supposed to be a one-to-one correspondence between the
>> functions in numpy and the methods on an ndarray. There is some
>> duplication between the two, but that is not a reason to make more
>> duplication.
>
> I would make a plea for consistency, to start with.

It's way too late to make changes like this.

> Those of us who write in an OO style are required to switch backwards
> and forwards between OO and not-OO, or to abandon OO altogether in our
> NumPy code.  Neither is an attractive option.

OO does not mean "always use methods."

> The reason I tripped over this is that I am currently writing a course
> which introduces students to NumPy.  I am going to be asked this
> question from the audience.  As yet I don't have any answer except
> "history".

Well, "history," usually along with "it seemed like a good idea at the
time," are valid reasons for things to continue to exist in any
nontrivial software project with a userbase. Your students will need
to learn this if they use software.

If you want a slightly better answer, the implementation of many of
the C functions were somewhat easier to do as methods on ndarray than
separate functions particularly since numpy.ndarray has subclasses.
The functions could then be implemented similar to the following:

  def myfunc(a):
  return asanyarray(a).myfunc()

One thing you will notice about numpy.diff() is that it is a pure
Python function rather than a C function, so it's certainly not going
to be a method on ndarray.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] _md5 module?

[Charles Doutriaux]

Hi Charles,

Yes.. unfortunately we have to be able to build out of the box, we can't 
rely on much anything (Except compiler and X) to be there already...

It would be much easier for CDAT to simply say getpython 2.5 and numpy 
1.1 and all the externals, and if you ever get it together come back see 
us :)

Unfortunately i don't think we'd have much success that way :)

I'll keep looking... I don't understand why python didn't build with 
md5. Never happened before...

I guess it's more of a python issue than numpy... But a syou said it's 
deprecated so you might as well want to replace it.

C.

Charles R Harris wrote:
>
>
> On Tue, Jun 24, 2008 at 10:07 AM, Charles Doutriaux 
> <[EMAIL PROTECTED] > wrote:
>
> Hello on redhat enterprise 5 i get this with numpy 1.1 and python 2.5
>
> Any idea if we can get around tihs? short of rebuilding python
> with md5 support i guess.
> Shouldn't numpy catch that before building?
>
> C.
>
>
> 
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/__init__.py",
> line 42, in 
>import add_newdocs
>  File
> 
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/add_newdocs.py",
> line 5, in 
>from lib import add_newdoc
>  File
> 
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/lib/__init__.py",
> line 18, in 
>from io import *
>  File
> 
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/lib/io.py",
> line 16, in 
>from _datasource import DataSource
>  File
> 
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/lib/_datasource.py",
> line 42, in 
>from urllib2 import urlopen, URLError
>  File
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/urllib2.py",
> line 91, in 
>import hashlib
>  File
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/hashlib.py",
> line 133, in 
>md5 = __get_builtin_constructor('md5')
>  File
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/hashlib.py",
> line 60, in __get_builtin_constructor
>import _md5
> ImportError: No module named _md5
>
>
> Python2.5 should come with md5, at least it does on fedora 8: 
> /usr/lib/python2.5/lib-dynload/_md5module.so. It is also deprecated as 
> of 2.5, so we might need some modifications anyway.
>
> Are you using the python built by CDAT? I found the easiest way to get 
> CDAT 4.3 working was to copy over the relevant site-packages from the 
> CDAT built python to the system python (of the same version). You 
> could maybe go the other way also.
>
> Chuck
>
>
> 
>
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>   

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] _md5 module?

[Charles R Harris]

On Tue, Jun 24, 2008 at 10:07 AM, Charles Doutriaux <[EMAIL PROTECTED]>
wrote:

> Hello on redhat enterprise 5 i get this with numpy 1.1 and python 2.5
>
> Any idea if we can get around tihs? short of rebuilding python with md5
> support i guess.
> Shouldn't numpy catch that before building?
>
> C.
>
>
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/__init__.py",
> line 42, in 
>import add_newdocs
>  File
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/add_newdocs.py",
> line 5, in 
>from lib import add_newdoc
>  File
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/lib/__init__.py",
> line 18, in 
>from io import *
>  File
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/lib/io.py",
> line 16, in 
>from _datasource import DataSource
>  File
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/lib/_datasource.py",
> line 42, in 
>from urllib2 import urlopen, URLError
>  File
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/urllib2.py",
> line 91, in 
>import hashlib
>  File
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/hashlib.py",
> line 133, in 
>md5 = __get_builtin_constructor('md5')
>  File
> "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/hashlib.py",
> line 60, in __get_builtin_constructor
>import _md5
> ImportError: No module named _md5
>
>
Python2.5 should come with md5, at least it does on fedora 8:
/usr/lib/python2.5/lib-dynload/_md5module.so. It is also deprecated as of
2.5, so we might need some modifications anyway.

Are you using the python built by CDAT? I found the easiest way to get CDAT
4.3 working was to copy over the relevant site-packages from the CDAT built
python to the system python (of the same version). You could maybe go the
other way also.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Sparse Matrices in Numpy -- (with eigenvalue algorithms if possible)

[Charles R Harris]

On Tue, Jun 24, 2008 at 9:45 AM, Dan Yamins <[EMAIL PROTECTED]> wrote:

>
>
> On Tue, Jun 24, 2008 at 2:12 AM, Charles R Harris <
> [EMAIL PROTECTED]> wrote:
>
>> Hi Dan,
>>
>> Did you finally get numpy installed as 64 bits?
>>
>> Chuck
>>
>> Hey, thanks for asking.   I did in fact get it installed -- at least, I
> think so.
>
> First, I had to built python in 64bit.  I did this using exactly the
> instructions that mabshoff gave (they should be accessible in an email in
> the archives of this mailinglist).  Then, when it came to installing numpy I
> ran into a problem that at first I couldn't get numpy to import properly --
> the multiarray.so module was build for 32-bit, not 64.  But then I followed
> mabshoff's instructions again and deleted the build folder from the numpy
> download; this time when I ran setup.py, multiarray.so appeared to have been
> built properly.  I _think_ at least it's working -- when I run
>numpy.dtype(numpy.uintp).itemsize
> the result is '8'.  Moreover, I can run numpy array functions efficienctly
> on objects of size >> 3 GB, so it looks like it's working.
>
> Btw, I had to do the same to get scipy to install properly as 64-bit -- at
> least, I think it's been installed properly.(Again,
> scipy.dtype(scipy.uintp).itemsize returns 8 --  I assume that's diagnostic
> here as well?)
>

Looks like you got it right. The reason you needed to delete the build
folder was that setup didn't know that python had changed and, since nothing
else had changed either, it just copied over the old build.


> It might be useful to have a repository of 64-bit builds and/or build
> instructions.
>

We should probably put mabshoff's instructions for building 64-bit python on
the MAC somewhere in the wiki.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] _md5 module?

[Charles Doutriaux]

Hello on redhat enterprise 5 i get this with numpy 1.1 and python 2.5

Any idea if we can get around tihs? short of rebuilding python with md5 support 
i guess.
Shouldn't numpy catch that before building?

C.


"/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/__init__.py",
 line 42, in 
import add_newdocs
  File
"/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/add_newdocs.py",
 line 5, in 
from lib import add_newdoc
  File
"/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/lib/__init__.py",
 line 18, in 
from io import *
  File
"/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/lib/io.py",
 line 16, in 
from _datasource import DataSource
  File
"/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/lib/_datasource.py",
 line 42, in 
from urllib2 import urlopen, URLError
  File
"/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/urllib2.py",
line 91, in 
import hashlib
  File
"/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/hashlib.py",
line 133, in 
md5 = __get_builtin_constructor('md5')
  File
"/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/hashlib.py",
line 60, in __get_builtin_constructor
import _md5
ImportError: No module named _md5

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Sparse Matrices in Numpy -- (with eigenvalue algorithms if possible)

[Dan Yamins]

On Tue, Jun 24, 2008 at 2:12 AM, Charles R Harris <[EMAIL PROTECTED]>
wrote:

> Hi Dan,
>
> Did you finally get numpy installed as 64 bits?
>
> Chuck
>
> Hey, thanks for asking.   I did in fact get it installed -- at least, I
think so.

First, I had to built python in 64bit.  I did this using exactly the
instructions that mabshoff gave (they should be accessible in an email in
the archives of this mailinglist).  Then, when it came to installing numpy I
ran into a problem that at first I couldn't get numpy to import properly --
the multiarray.so module was build for 32-bit, not 64.  But then I followed
mabshoff's instructions again and deleted the build folder from the numpy
download; this time when I ran setup.py, multiarray.so appeared to have been
built properly.  I _think_ at least it's working -- when I run
   numpy.dtype(numpy.uintp).itemsize
the result is '8'.  Moreover, I can run numpy array functions efficienctly
on objects of size >> 3 GB, so it looks like it's working.

Btw, I had to do the same to get scipy to install properly as 64-bit -- at
least, I think it's been installed properly.(Again,
scipy.dtype(scipy.uintp).itemsize returns 8 --  I assume that's diagnostic
here as well?)

It might be useful to have a repository of 64-bit builds and/or build
instructions.

Thanks,
Dan
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ndarray methods vs numpy module functions

[Stéfan van der Walt]

Hi Bob

2008/6/24 Bob Dowling <[EMAIL PROTECTED]>:
> I would make a plea for consistency, to start with.
>
> Those of us who write in an OO style are required to switch backwards
> and forwards between OO and not-OO, or to abandon OO altogether in our
> NumPy code.  Neither is an attractive option.

There are an infinite number of operations to be performed on arrays,
so the question becomes: which of those belong as members of the
class?  In my opinion, none do; we have them simply for backward
compatibility.

In general, my feeling (not the status quo) is to:

a) Use array methods for in-place operations and operations pertaining
specifically to ndarrays.  This would include `sort`, but not `sum` or
`dump`.
b) Use numpy functions for operations that copy the object, or do
calculations that yield new objects.

Even if you subscribe to such a rule, having x.sum() at hand is
convenient, so many people use it.  There's bound to be a big outcry
if we try to remove them now.  I'm not even sure most people would
agree with these guidelines.

Regards
Stéfan
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ndarray methods vs numpy module functions

[David Cournapeau]

Sebastian Haase wrote:
> Are you saying the duplication is "just random" ?  It would be better
> -- as in: principle of minimum surprise -- if there would be some sort
> "reasonable set" of duplicates 

Yes it would be better. But how do you do it without breaking other
people code and avoiding duplication ? That's a trade-off: if we remove
say numpy.sum, people will complain that numpy.sum does not exist.
Adding all the functions in numpy arrays to be able to say a.foo instead
of foo(a) is not that great either. I think we can spend our time on
more interesting/important problems.

> If there are only a handful functions missing, why not try to make it 
> complete.
Duplication is bad, and should be avoided as much as possible. a.foo vs
foo(a) does not sound like a good case to introduce more duplication ot me.

cheers,

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ndarray methods vs numpy module functions

[Anne Archibald]

2008/6/24 Bob Dowling <[EMAIL PROTECTED]>:
>> There is not supposed to be a one-to-one correspondence between the
>> functions in numpy and the methods on an ndarray. There is some
>> duplication between the two, but that is not a reason to make more
>> duplication.
>
> I would make a plea for consistency, to start with.
>
> Those of us who write in an OO style are required to switch backwards
> and forwards between OO and not-OO, or to abandon OO altogether in our
> NumPy code.  Neither is an attractive option.
>
> The reason I tripped over this is that I am currently writing a course
> which introduces students to NumPy.  I am going to be asked this
> question from the audience.  As yet I don't have any answer except
> "history".

As a rule, I personally avoid methods whenever reasonable. The primary
reason is that the function versions generally work fine on lists,
giving my functions some extra genericity for free. I generally make
an exception only for attributes - X.shape, for example, rather than
np.shape(X).

It's true, it's a mess, but I'd just set down some simple rules that
work, and mention that some functions exist in other forms. There's
something to be said for rationalizing numpy's rules - or at least
writing them down! - but there's no need to use every version of every
function. And I believe they're all accessible as module-level
functions.

Anne
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ndarray methods vs numpy module functions

[Sebastian Haase]

On Tue, Jun 24, 2008 at 3:11 AM, Robert Kern <[EMAIL PROTECTED]> wrote:
> On Mon, Jun 23, 2008 at 19:35, Ryan May <[EMAIL PROTECTED]> wrote:
>> Robert Kern wrote:
>>> On Mon, Jun 23, 2008 at 18:10, Sebastian Haase <[EMAIL PROTECTED]> wrote:
 On Mon, Jun 23, 2008 at 10:31 AM, Bob Dowling <[EMAIL PROTECTED]> wrote:
> [ I'm new here and this has the feel of an FAQ but I couldn't find
> anything at http://www.scipy.org/FAQ .  If I should have looked
> somewhere else a URL would be gratefully received. ]
>
>
> What's the reasoning behind functions like sum() and cumsum() being
> provided both as module functions (numpy.sum(data, axis=1)) and as
> object methods (data.sum(axis=1)) but other functions - and I stumbled
> over diff() - only being provided as module functions?
>
>
 Hi Bob,
 this is a very good question.
 I think the answers are
 a) historical reasons AND, more importantly, differing personal preferences
 b) I would file  the missing data.diff() as a bug.
>>>
>>> It's not.
>>>
>> Care to elaborate?
>
> There is not supposed to be a one-to-one correspondence between the
> functions in numpy and the methods on an ndarray. There is some
> duplication between the two, but that is not a reason to make more
> duplication.
>
Are you saying the duplication is "just random" ?  It would be better
-- as in: principle of minimum surprise -- if there would be some sort
"reasonable set" of duplicates 
If there are only a handful functions missing, why not try to make it complete.

( Again, a list of functions vs. methods on the wiki would clarify
what we are talking about )

Just thinking loudly of course. Don't take this as an offense .
-Sebastian
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] ndarray methods vs numpy module functions

[Bob Dowling]

> There is not supposed to be a one-to-one correspondence between the
> functions in numpy and the methods on an ndarray. There is some
> duplication between the two, but that is not a reason to make more
> duplication.

I would make a plea for consistency, to start with.

Those of us who write in an OO style are required to switch backwards 
and forwards between OO and not-OO, or to abandon OO altogether in our 
NumPy code.  Neither is an attractive option.

The reason I tripped over this is that I am currently writing a course 
which introduces students to NumPy.  I am going to be asked this 
question from the audience.  As yet I don't have any answer except 
"history".
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion