Re: [Numpy-discussion] matlab vs. python question

[belinda thom]

Thank you so much! When I finally get a moment to take a break, I'll  
look in more detail into using your suggestion.

--b


On Apr 26, 2007, at 12:47 AM, Pauli Virtanen wrote:

> belinda thom kirjoitti:
>> On Apr 25, 2007, at 12:46 PM, Bill Baxter wrote:
>>
>> Agree w/most of what you've said, but will add one other thing that
>> drives me nuts in python that hasn't been a problem in Matplotlib:
>>
>> In Python, if interacting w/the interpreter as your primary IDE, and
>> if you've got multiple files that depend on one another that you're
>> modifying, then you need to restart the interpreter frequently b/c
>> otherwise things in the interpreter can be stale; IOW, changes to
>> several interdependent files aren't easy to import so that everything
>> in your interpreted environment reflects the latest code. Yeah,
>> there's reload tricks, but doing them in the right order and the
>> right number of times can be a pain when dependencies are cyclic.
>>
>> I realize in general that this issue of stale code is just difficult,
>> that its not inherently a Python problem per se, for automatic
>> percolation of code changes backwards in time is difficult in
>> general, but I've never had the problem bite me when I was developing
>> in Matlab. I just save whatever file, and it appears that whatever's
>> latest on disk is what's executed. (Friends who know more about PL
>> than I tell me I've been lucky.)
>
> I've been using the attached autoreload code (by Thomas Heller,
> different versions are probably floating around the net) rather
> successfully for a more Matlabish feel to IPython. It reloads modules
> every time their files have been changed.
>
> Of course, this doesn't solve all staleness problems you describe, but
> only eliminates manual work involved in solving the most common ones.
> Reloading modules does have pitfalls [1], but at least in my use these
> haven't really mattered much in practice.
>
> I think having autoreload functionality bundled with IPython (maybe
> turned off per default) would be quite useful: although in some rare
> cases autoreloading doesn't work in the ideal way, it's very  
> convenient
> not to have to type reload(foo) after every change, especially when
> tweaking for example plotting scripts.
>
>   Pauli
>
> [1] For example, instances of classes created are not updated on  
> reload,
> but continue using the old code. Also, from foo import * imports  
> are not
> updated, so you'll have to manually touch files that do this if 'foo'
> has been changed.
> """
>
> autoreload.py - automatically reload changed source
> code into a running program
>
> You might want to add the following to your ~/.ipython/ipythonrc
>
> import_mod sys
> execute sys.path.append('path/where/this/file/resides')
> import_mod autoreload
> execute autoreload.run()
>
> or adding the following
>
> import sys
> sys.path.append('path/where/this/file/resides')
> import autoreload
> autoreload.run()
>
> to some startup file.
>
>
> Created: Thomas Heller, 2000-04-17
> Modified: Pauli Virtanen, 2006
> """
>
> # $Id: autoreload.py 3117 2006-09-27 20:28:46Z pauli $
> #
> # $Log: autoreload.py,v $
> # Revision 1.9  2001/11/15 18:41:18  thomas
> # Cleaned up and made working again before posting to c.l.p.
> # Added code to update bound (or unbound) methods as suggested
> # by Just van Rossum. Thanks!
> #
> # ...
> #
> # Revision 1.1  2001/10/04 16:54:04  thomas
> # Discovered this old module on my machine, it didn't work too well,
> # but seems worth to continue with it...
> # Checked in as a first step.
>
>
> __version__ = "$Revision: 1.9 $".split()[1]
>
> # ToDo:
> #
> #  Cannot reload __main__ - explain why this cannot work,
> #  and explain a workaround.
> #
> #  Optimize - the number of watches objects (in old_objects)
> #  grows without limits. Think if this is really necessary...
>
>
> import time, os, threading, sys, types, imp, inspect, traceback
>
> def _get_compiled_ext():
> for ext, mode, typ in imp.get_suffixes():
> if typ == imp.PY_COMPILED:
> return ext
>
> # the official way to get the extension of compiled files (.pyc  
> or .pyo)
> PY_COMPILED_EXT = _get_compiled_ext()
>
> class ModuleWatcher:
> running = 0
> def __init__(self):
> # If we don't do this, there may be tracebacks
> # when shutting down python.
> import atexit
> atexit.register(self.stop)
>
> def run(self):
> if self.running:
> print "# autoreload already running"
> return
> print "# starting autoreload"
> self.running = 1
> self.thread = threading.Thread(target=self._check_modules)
> self.thread.setDaemon(1)
> self.thread.start()
>
> def stop(self):
> if not self.running:
> print "# autoreload not running"
> return
> self.running = 0
> self.thread.join()
> #print "# autoreload stopped"
>
> def _c

Re: [Numpy-discussion] numpy endian question

[Travis Oliphant]

> I really want to know if the array is in big-endian order (I don't care 
> whether it's native). This is for sending arrays to the ds9 image viewer 
> via xpa (and communicating with ds9 is not easy). To do this reliably I 
> need to indicate the byte order of the data.
>   

Thanks for the clarification and the original question.By the way, 
are you aware of the  numpy.numarray module

numpy.numarray.isBigEndian  works as expected.


-Travis

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] numpy endian question

[Christopher Hanley]

Russell,

This should work as a consistent test for bigendian:

-> isBigEndian = (obj.dtype.str[0] == '>')

Also, I have ported numarray's numdisplay to numpy if you would like to 
directly display an array in DS9.  We haven't done an official release 
yet (sometime soon) but I can forward you a copy if you are interested.

Chris

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] numpy endian question

[Russell E. Owen]

In article <[EMAIL PROTECTED]>,
 Travis Oliphant <[EMAIL PROTECTED]> wrote:

> Russell E. Owen asked how best to translate some numarray code that 
> determines if an array is in big-endian order...
>
> What are you trying to test for (where a data-type is big-endian or not?).

I really want to know if the array is in big-endian order (I don't care 
whether it's native). This is for sending arrays to the ds9 image viewer 
via xpa (and communicating with ds9 is not easy). To do this reliably I 
need to indicate the byte order of the data.

I suppose there are alternatives such as making a "native byte order" 
copy of the data and sending that without any flag. But I think it's 
easier to just send the flag.

> To directly answer your question, however.
> 
> On page 42 of the sample pages (online for free) of my book, the 
> recommended translation of numarray's arr.isbyteswapped() is  (not 
> arr.dtype.isnative)

I found that (I own your book).

> The translation of numarray.isBigEndian is (not numpy.little_endian)

I missed that one (and I did look for it).

>...
> isBigEndian = arr.dtype.isnative ^ numpy.little_endian

That should be much clearer than my translation. Thanks!

-- Russell

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] sort bug

[tan2]

Just doing argsort() on the whole array is faster (up until about 1e6
elements)
because it does everything in C whereas heapq will create a lot of Python
objects because it is treating the array as a general Python container.




That's a good point. I wasn't thinking about the efficiency issue.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] sort bug

[Robert Kern]

tan2 wrote:
> 
> On 4/26/07, * Anton Sherwood* <[EMAIL PROTECTED]
> > wrote:
> 
> All I really need is the four highest eigenvalues and their
> vectors.
> I'll have a look in "Python Cookbook" to see if there's a more
> efficient way to do that partial sort.
> 
> Maybe "heapq.nlargest ()" is what you want?
> http://www.python.org/doc/2.4/lib/module-heapq.html

Just doing argsort() on the whole array is faster (up until about 1e6 elements)
because it does everything in C whereas heapq will create a lot of Python
objects because it is treating the array as a general Python container.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] numpy endian question

[Travis Oliphant]

Russell E. Owen wrote:
> In converting some code from numarray to numpy I had this:
> isBigendian = (arr.isbyteswapped() != numarray.isBigEndian)
>
> The only numpy version I've come up with is:
> isBigEndian = (arr.dtype.descr[0][1][0] == '>')
>
> which is short but very obscure. Has anyone got a suggestion for a 
> clearer test? I found lots of *almost* useful flags and methods.
>   

How about looking at  arr.dtype.byteorder?

What are you trying to test for (where a data-type is big-endian or not?).

sys.byteorder is either "little" or "big"
numpy.little_endian is True if you are on a little endian system.

arr.dtype.byteorder gives you either "=" for native or ">" for 
big-endian or "<" for little-endian or "|" for it doesn't matter.

To directly answer your question, however.

On page 42 of the sample pages (online for free) of my book, the 
recommended translation of numarray's arr.isbyteswapped() is  (not 
arr.dtype.isnative)

The translation of numarray.isBigEndian is (not numpy.little_endian)

Thus a direct translation of your code is:

isBigEndian = ((not arr.dtype.isnative) != (not numpy.little_endian))

which is equivalent to

isBigEndian = (arr.dtype.isnative != numpy.little_endian)

and because != on boolean items is equivalent to XOR this is the same as

isBigEndian = arr.dtype.isnative ^ numpy.little_endian


-Travis


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] sort bug

[tan2]

On 4/26/07, Anton Sherwood <[EMAIL PROTECTED]> wrote:
>
>
> All I really need is the four highest eigenvalues and their vectors.
> I'll have a look in "Python Cookbook" to see if there's a more
> efficient way to do that partial sort.





Maybe "heapq.nlargest()" is what you want?
http://www.python.org/doc/2.4/lib/module-heapq.html
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] sort bug

[Anne Archibald]

On 26/04/07, Anton Sherwood <[EMAIL PROTECTED]> wrote:

> All I really need is the four highest eigenvalues and their vectors.

It's not really very efficient, but sorting time is going to be tiny
compared to however you get your eigenvalues, so you could argsort the
eigenvalue array and use the result to sort the eigenvector array.

You can also accelerate the process by quickly checking whether the
eigenvalue array is already sorted - usually it is.

Anne
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] sort bug

[Charles R Harris]

On 4/26/07, Anton Sherwood <[EMAIL PROTECTED]> wrote:


Travis Oliphant wrote:
> cmp(x,y) must return -1, 0, or 1 which doesn't work on arrays with more
> than 1 element because it is ambiguous.  Thus you get this error.

Ah.  Since lists *can* be compared, I assumed arrays would inherit
that property.

> The operation is undefined.  What do you actually want to do when you
> have equal-valued eigenvalues?

Don't care.

All I really need is the four highest eigenvalues and their vectors.
I'll have a look in "Python Cookbook" to see if there's a more
efficient way to do that partial sort.



I don't think there are any partial sorts available. I think the best
approach in this case an argsort on the eigenvalues, followed by a take
using the last four resulting indexes would be the way to go.

Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Questions about converting to numpy

[Robert Kern]

Russell E. Owen wrote:

> But I still don't understand why one shouldn't just use dtype=float or 
> numpy.float. Does that result in an array with a different type of float 
> than numpy.float_ (float64)? Or does it just somehow speed up numpy 
> because it doesn't have to convert the python type into a numpy dtype.

For specifying dtype=float in functions that take such an argument, go ahead and
use dtype=float. There isn't really a reason to use anything else if you don't
want to. However, the scalar type objects are used in other places, so float_
should exist.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] sort bug

[Travis Oliphant]

Anton Sherwood wrote:
> Travis Oliphant wrote:
>   
>> cmp(x,y) must return -1, 0, or 1 which doesn't work on arrays with more
>> than 1 element because it is ambiguous.  Thus you get this error.
>> 
>
> Ah.  Since lists *can* be compared, I assumed arrays would inherit
> that property.
>
>   
>> The operation is undefined.  What do you actually want to do when you
>> have equal-valued eigenvalues?
>> 
>
> Don't care.
>   

One approach is to use argsort to create an index list of sorted 
eigenvalues and then sort the eig and eigvector arrays before zipping 
them together in a list of tuples.


eig, val = numpy.linalg.eig(a)

indx = eig.argsort()
eig = eig.take(indx)
val = val.take(indx, axis=1)
master = zip(eig, val.T)

Will do what you want.


-Travis

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] numpy endian question

[Francesc Altet]

El dj 26 de 04 del 2007 a les 11:38 -0700, en/na Russell E. Owen va
escriure:
> In converting some code from numarray to numpy I had this:
> isBigendian = (arr.isbyteswapped() != numarray.isBigEndian)
> 
> The only numpy version I've come up with is:
> isBigEndian = (arr.dtype.descr[0][1][0] == '>')

isBigEndian = (arr.dtype.str[0] == '>')

is a little bit shorter. A more elegant approach could be:

isBigEndian = arr.dtype.isnative ^ numpy.little_endian

Cheers,

-- 
Francesc Altet|  Be careful about using the following code --
Carabos Coop. V.  |  I've only proven that it works, 
www.carabos.com   |  I haven't tested it. -- Donald Knuth

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Questions about converting to numpy

[Russell E. Owen]

In article <[EMAIL PROTECTED]>,
 Robert Kern <[EMAIL PROTECTED]> wrote:

> Christopher Barker wrote:
> 
> > I can only help with one:
> >> - Even after reading the book I'm not really clear on why one would use 
> >> numpy.float_ instead of numpy.float or float
> > 
> > They float and numpy.float are the same, and numpy.float_ is the same as 
> > numpy.float64:
> > 
> >  >>> import numpy
> >  >>> float is numpy.float
> > True
> >  >>> numpy.float64 is numpy.float64
> > True
> >  >>>
> > 
> > float was added to the numpy namespace so that we could write consistent 
> > code like:
> > 
> > a = array(object, numpy.float32)
> > b = array(object, numpy.float)
> > 
> > i.e. have it all in the same namespace.
> > 
> > I'm not sure why float_ is an alias for float64, though I'm guessing 
> > it's possible that on some platforms they are not the same.
> 
> Rather, numpy.float used to be an alias for numpy.float64; however, it 
> overrode
> the builtin float() when "from numpy import *" was used at the interactive
> prompt. Consequently, we renamed it numpy.float_ and specifically imported 
> the
> builtin float as numpy.float such that we didn't break code that had already
> started using "numpy.float".

But I still don't understand why one shouldn't just use dtype=float or 
numpy.float. Does that result in an array with a different type of float 
than numpy.float_ (float64)? Or does it just somehow speed up numpy 
because it doesn't have to convert the python type into a numpy dtype.

Anyway, thank you all for the helpful answers! I'm glad numpy throws the 
standard MemoryError since I'm already testing for that.

-- Russell

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[David Goldsmith]

Guido van Rossum wrote:
> Jeffrey, is there any way you can drop the top of the tree and going
> straight from Number to Complex -> Real -> Rational -> Integer? These
> are the things that everyone with high school math will know.
>   
Having taught mathematics at a community college for a little while 
(hell, having taught mathematics at a few Universities - I won't name 
names), I would suggest that this is not a valid premise.  I was going 
to qualify that by adding that hopefully it is a valid premise for 
would-be mathematics-package-using programmers, but I recall a fellow 
student in an Intro to C class I was once in (not teaching, however) 
who, even though the class was not supposed to be an Intro to 
Programming course (this was back in the day when those were still 
taught in Pascal and those taking the C class were supposed to have had 
successfully completed the Pascal course first), had to be handheld not 
simply through the implementation of algorithms in C, but in the very 
development of algorithms from a problem statement.  Even without having 
to resort to such an extreme anecdote, I can envision situations where 
knowledgeable, skilled, experienced, but "deficient"-in-math programmers 
are called upon (implicitly or explicitly) by management to use numpy/scipy.

All this said, I'm certainly of the opinion that we can't "dumb down" 
numpy *too* much (after all, by some definition, it is an "advanced" 
mathematics package, at least IMO), and to my mind the sequence above, 
i.e., X->C->R->Q->Z, is a more than reasonable "line in the sand".

DG
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] numpy endian question

[Russell E. Owen]

In converting some code from numarray to numpy I had this:
isBigendian = (arr.isbyteswapped() != numarray.isBigEndian)

The only numpy version I've come up with is:
isBigEndian = (arr.dtype.descr[0][1][0] == '>')

which is short but very obscure. Has anyone got a suggestion for a 
clearer test? I found lots of *almost* useful flags and methods.

-- Russell

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] sort bug

[Anton Sherwood]

Travis Oliphant wrote:
> cmp(x,y) must return -1, 0, or 1 which doesn't work on arrays with more
> than 1 element because it is ambiguous.  Thus you get this error.

Ah.  Since lists *can* be compared, I assumed arrays would inherit
that property.

> The operation is undefined.  What do you actually want to do when you
> have equal-valued eigenvalues?

Don't care.

All I really need is the four highest eigenvalues and their vectors.
I'll have a look in "Python Cookbook" to see if there's a more
efficient way to do that partial sort.

-- 
Anton Sherwood, http://www.ogre.nu/
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Jean-Paul Calderone]

On Thu, 26 Apr 2007 10:11:27 PDT, Bill Janssen <[EMAIL PROTECTED]> wrote:
>> Are the base number operations in Python all that difficult to
>> understand?
>
>Well, they're a little tricky.
>
>But the basic problem with "number" support, in almost every
>programming language, is that they are too low-level an abstraction.
>A number in a program is never *just* an integer or a float.  It's
>always an integer or float (or complex, OK) deployed in support of a
>very specific purpose.  It's a loop index, or a weight in kilos, or
>the number of calendar entries for a particular day.  It often has
>units, and a fixed range of values or fixed set of values, neither of
>which are taken into account in most programming languages.
>
>Strings really suffer from the same lack of focus.  A string is never
>just a string.  It's a book title, or a filename, or a person's
>surname, or an email address.  It's usually got a particular human
>language (or computer OS) associated with it.  Each of these usages
>has particular limitations and patterns which limit both its range of
>values, and the operations which can be successfully applied to it.

I definitely agree with this, and resolving it would probably address
a large class of mistakes made when doing any programming involving
numbers and strings (anyone done much of that lately? :).

Does the introduction of a lot of base classes for `int' and `float'
do anything to address this, though?  What you really want is a bunch
of new subclasses  of them, the opposite.

Jean-Paul
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Guido van Rossum]

On 4/25/07, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote:
> On Wed, 25 Apr 2007 20:40:14 -0400, "Phillip J. Eby" <[EMAIL PROTECTED]> 
> wrote:
> >At 06:40 PM 4/25/2007 -0400, Jean-Paul Calderone wrote:
> >>On Wed, 25 Apr 2007 18:10:23 -0400, Jim Jewett <[EMAIL PROTECTED]>
> >>wrote:
> >> >The current ABC proposal is to use isinstance as the test; Jeffrey
> >> >Yaskin's numbers PEP highlighted the weakness there with a concrete
> >> >example.
> >> >
> >> >If you need to an abstraction less powerful than an existing ABC,
> >> >you're out of luck; you can't just assert that the existing class is
> >> >already sufficient, nor can you expect everyone else to use multiple
> >> >annotations.
> >>
> >>I'm sure everyone is already aware of the behavior of the classImplements
> >>and directlyProvides functions available in zope.interface, which exactly
> >>satisfy this use-case in the interface world.
> >
> >I'm either misunderstanding Jim or you, because I don't see the relationship
> >here.  If I understand Jim correctly, he's actually asking for something
> >like PyProtocols' "protocolIsSubsetOf" declaration -- something that would
> >be like dynamically adding a superclass to an existing interface in
> >zope.interface.  Whereas the features you're talking about sound like
> >declaring that a class object itself implements an interface -- something
> >apparently unrelated to the question at hand.
> >
>
> Ugh, you're at least half right.  I didn't read the implementation of
> ABC.meets carefully enough to notice that it's actually an adapter
> registry, not just a predicate (meets seems like a funny name for that
> method).
>
> classImplements would have satisfied the adapterless use-case by letting
> Jim declare that someone else's class definition actually implements the
> feature set that some unit of code he wants to use that class with is
> expecting.
>
> With the adapter registry, I don't see why this isn't just PEP 246 adapt()
> (if I am reading one of your later mails correctly though, you think it is
> too).
>
> I'm not particularly familiar with protocolIsSubsetOf, but it wouldn't
> surprise me at all if it could also satisfy this use-case.  There are
> lots of ways to do this, many of them fairly simple.  Using inheritance
> is the part that makes it hard, since inheritance is controlled by the
> wrong party in most cases and comes with unrelated features that are,
> at best, irrelevant to the particular use case and at worst actively
> detrimental.
>
> I'm sure a way around this can be invented, I just don't see why it
> should be.

JP, would you mind dropping this and instead considering how we could
turn isinstance and issubclass into (poor-man's) GFs by allowing
classes to define class methods __isinstance__ and __issubclass__?
(Though the arguments are reversed so to avoid confusion perhaps the
names should be different, similar as with the 'in' operator and the
__contains__ magic method.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Phillip J. Eby]

At 06:40 PM 4/25/2007 -0400, Jean-Paul Calderone wrote:
>On Wed, 25 Apr 2007 18:10:23 -0400, Jim Jewett <[EMAIL PROTECTED]> wrote:
> >The current ABC proposal is to use isinstance as the test; Jeffrey
> >Yaskin's numbers PEP highlighted the weakness there with a concrete
> >example.
> >
> >If you need to an abstraction less powerful than an existing ABC,
> >you're out of luck; you can't just assert that the existing class is
> >already sufficient, nor can you expect everyone else to use multiple
> >annotations.
>
>I'm sure everyone is already aware of the behavior of the classImplements
>and directlyProvides functions available in zope.interface, which exactly
>satisfy this use-case in the interface world.

I'm either misunderstanding Jim or you, because I don't see the 
relationship here.  If I understand Jim correctly, he's actually asking for 
something like PyProtocols' "protocolIsSubsetOf" declaration -- something 
that would be like dynamically adding a superclass to an existing interface 
in zope.interface.  Whereas the features you're talking about sound like 
declaring that a class object itself implements an interface -- something 
apparently unrelated to the question at hand.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Phillip J. Eby]

At 06:10 PM 4/25/2007 -0400, Jim Jewett wrote:
>I suspect Phillip will say that we really need to make the ABCs
>generic functions...

Nope; they're *already* generic functions: hash(), len(), iter(), various 
operator.* functions, and I believe we're adding next().  I've got nothing 
to add to that list.  :)

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Guido van Rossum]

On 4/25/07, Jeffrey Yasskin <[EMAIL PROTECTED]> wrote:
> On 4/25/07, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> > Jeffrey, is there any way you can drop the top of the tree and going
> > straight from Number to Complex -> Real -> Rational -> Integer? These
> > are the things that everyone with high school math will know.
>
> I think yes, if you can confirm that the
> import numbers
> class Ring(AdditiveGroup): ...
> numbers.Complex.__bases__ = (Ring,) + numbers.Complex.__bases__
> hack I mention under IntegralDomain will make isinstance(int, Ring) return 
> True.

Hm... That would require built-in types to be mutable (by mere
mortals) which would kill the idea of multiple interpreters. AFAIK the
only user of multiple interpreters is mod_python, which is a pretty
popular Apache module... (Google claims 1.8M hits, vs. 5.66M for
mod_perl, the market leader.)

I guess the alternative would be to keep the mathematical ones in but
make it so that mere mortals (among which I count myself :) never need
to know about them. Sort of like metaclasses, for most people.

But this still presents a bit of a problem. If we insist that built-in
types are immutable, whenever a mathematician needs a new kind of
algebraic category (you mention a few in the PEP), they have no way to
declare that int or float are members of that category. They could
write things like

class MyFloat(float, DivisionRing): pass

but this would be fairly painful. I would like to let users modify the
built-ins, but such changes ought to be isolated from other
interpreters running in the same address space. I don't want to be the
one to tell the mod_python community they won't be able to upgrade to
3.0...

> Do you want "Number" to be equivalent to Ring+Hashable (or a
> Haskell-like Ring+Hashable+__abs__+__str__)? I don't think a "Number"
> class is strictly necessary since people can check for Complex or Real
> directly, and one of those two is probably what they'll expect after
> knowing that something's a number.

Fair enough, you could start with Complex.

> Also, I don't see much point in putting Rational between Real and
> Integer. The current hierarchy is Real (int, float, Decimal, rational)
> :> Integer (int) and Real :> FractionalReal (float, Decimal,
> rational), but Integer and FractionalReal are just siblings.

Why? Why not have a properfraction() on Integer that always returns 0
for the fraction? The current math.modf() works just fine on ints
(returning a tuple of floats).

And why distinguish between Real and FractionalReal? What's the use
case for reals that don't know how to compute their fractional part?

> I'm wondering how many people are writing new numeric types who don't
> know the abstract algebra.

It's easy to know the algebra without recalling the terminologies.
Lots of programmers (myself included!) are largely self-taught.

> I think we should be able to insulate
> people who are just using the types from the complexities underneath,
> and the people writing new types will benefit. Or will seeing "Ring"
> in a list of superclasses be enough to confuse people?

Shouldn't only mathematicians be tempted to write classes deriving
from Ring? Everyone else is likely to subclass one of the more
familiar types, from Complex onwards.

PS. I've checked your PEP in, as PEP3141. We might as well keep a
fairly complete record in svn, even if we decide to cut out the
algebraic foundational classes.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Jeffrey Yasskin]

On 4/25/07, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> Jeffrey, is there any way you can drop the top of the tree and going
> straight from Number to Complex -> Real -> Rational -> Integer? These
> are the things that everyone with high school math will know.

I think yes, if you can confirm that the
import numbers
class Ring(AdditiveGroup): ...
numbers.Complex.__bases__ = (Ring,) + numbers.Complex.__bases__
hack I mention under IntegralDomain will make isinstance(int, Ring) return True.

Do you want "Number" to be equivalent to Ring+Hashable (or a
Haskell-like Ring+Hashable+__abs__+__str__)? I don't think a "Number"
class is strictly necessary since people can check for Complex or Real
directly, and one of those two is probably what they'll expect after
knowing that something's a number.

Also, I don't see much point in putting Rational between Real and
Integer. The current hierarchy is Real (int, float, Decimal, rational)
:> Integer (int) and Real :> FractionalReal (float, Decimal,
rational), but Integer and FractionalReal are just siblings.


I'm wondering how many people are writing new numeric types who don't
know the abstract algebra. I think we should be able to insulate
people who are just using the types from the complexities underneath,
and the people writing new types will benefit. Or will seeing "Ring"
in a list of superclasses be enough to confuse people?

-- 
Namasté,
Jeffrey Yasskin
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Guido van Rossum]

On 4/25/07, Collin Winter <[EMAIL PROTECTED]> wrote:
> The following things mean absolutely nothing to me:
>
> - Monoid
> - MonoidUnderPlus
> - Group
> - Ring
> - Semiring
> - Field
>
> So, most of the terminology in the PEP.
>
> I can see use-cases for this level of formalism, but I'm a strong -1
> on making any part of the stdlib effectively off-limits for people
> without advanced math degrees. Why can't this be shipped as a
> third-party module?

As a math major I have no excuse, but I have to confess I'm also
really rusty on these things. (Nothing a quick look at wikipedia can't
refresh though.)

Jeffrey, is there any way you can drop the top of the tree and going
straight from Number to Complex -> Real -> Rational -> Integer? These
are the things that everyone with high school math will know.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Jeffrey Yasskin]

[bcc'ing numpy-discussion. Comments should probably try to stay on the
python-3000 list.]

On 4/25/07, Josiah Carlson <[EMAIL PROTECTED]> wrote:
> Are the base number operations in Python all that difficult to
> understand?  Do we really need to add mathematical formalism into
> Python's type system before people understand the meaning of X * Y?
> Because all I really see this doing is confusing the hell out of people
> who aren't mathematicians; I'm a theoretical computer scientist and I
> had to try to think for a moment to verify that all of those were really
> necessary to separate the cases.
>
> I really do understand wanting to be able to ask "is operation X
> supported for values Y and Z", but is this really necessary for numbers?

If you don't want to use the full power of the system, nothing forces
you to. If someone doesn't know what a Ring or a complex number is,
they can just check for Real and rely on the same operations, at the
small cost of overconstraining their inputs. One reason I'm suggesting
splitting the classes into the "algebra" and "numbers" modules is so
that people who don't know or care about the algebra can rely on just
"numbers" and still get nearly all of the benefits.

However, I think it's important to provide a framework for people who
want to do more powerful things. If you write a function that can take
an arbitrary ring, it should Just Work on a python int, or any other
Complex subclass, without you having to check for a lot of operations
separately. The Haskell98 numerical hierarchy actually does start at
Num, ignoring the algebraic structures underneath, and it draws a lot
of criticism for it.

That does mean that the names and documentation in the numbers module
need to be as clear as we can make them for non-mathematical
audiences. I'm certainly open to suggestions on that front.

-- 
Namasté,
Jeffrey Yasskin
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Collin Winter]

The following things mean absolutely nothing to me:

- Monoid
- MonoidUnderPlus
- Group
- Ring
- Semiring
- Field

So, most of the terminology in the PEP.

I can see use-cases for this level of formalism, but I'm a strong -1
on making any part of the stdlib effectively off-limits for people
without advanced math degrees. Why can't this be shipped as a
third-party module?

Collin Winter
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[Pauli Virtanen]

belinda thom kirjoitti:
> On Apr 25, 2007, at 12:46 PM, Bill Baxter wrote:
> 
> Agree w/most of what you've said, but will add one other thing that  
> drives me nuts in python that hasn't been a problem in Matplotlib:
> 
> In Python, if interacting w/the interpreter as your primary IDE, and  
> if you've got multiple files that depend on one another that you're  
> modifying, then you need to restart the interpreter frequently b/c  
> otherwise things in the interpreter can be stale; IOW, changes to  
> several interdependent files aren't easy to import so that everything  
> in your interpreted environment reflects the latest code. Yeah,  
> there's reload tricks, but doing them in the right order and the  
> right number of times can be a pain when dependencies are cyclic.
> 
> I realize in general that this issue of stale code is just difficult,  
> that its not inherently a Python problem per se, for automatic  
> percolation of code changes backwards in time is difficult in  
> general, but I've never had the problem bite me when I was developing  
> in Matlab. I just save whatever file, and it appears that whatever's  
> latest on disk is what's executed. (Friends who know more about PL  
> than I tell me I've been lucky.)

I've been using the attached autoreload code (by Thomas Heller,
different versions are probably floating around the net) rather
successfully for a more Matlabish feel to IPython. It reloads modules
every time their files have been changed.

Of course, this doesn't solve all staleness problems you describe, but
only eliminates manual work involved in solving the most common ones.
Reloading modules does have pitfalls [1], but at least in my use these
haven't really mattered much in practice.

I think having autoreload functionality bundled with IPython (maybe
turned off per default) would be quite useful: although in some rare
cases autoreloading doesn't work in the ideal way, it's very convenient
not to have to type reload(foo) after every change, especially when
tweaking for example plotting scripts.

Pauli

[1] For example, instances of classes created are not updated on reload,
but continue using the old code. Also, from foo import * imports are not
updated, so you'll have to manually touch files that do this if 'foo'
has been changed.
"""

autoreload.py - automatically reload changed source
code into a running program

You might want to add the following to your ~/.ipython/ipythonrc

import_mod sys
execute sys.path.append('path/where/this/file/resides')
import_mod autoreload
execute autoreload.run()

or adding the following

import sys
sys.path.append('path/where/this/file/resides')
import autoreload
autoreload.run()

to some startup file.


Created: Thomas Heller, 2000-04-17
Modified: Pauli Virtanen, 2006
"""

# $Id: autoreload.py 3117 2006-09-27 20:28:46Z pauli $
#
# $Log: autoreload.py,v $
# Revision 1.9  2001/11/15 18:41:18  thomas
# Cleaned up and made working again before posting to c.l.p.
# Added code to update bound (or unbound) methods as suggested
# by Just van Rossum. Thanks!
#
# ...
#
# Revision 1.1  2001/10/04 16:54:04  thomas
# Discovered this old module on my machine, it didn't work too well,
# but seems worth to continue with it...
# Checked in as a first step.


__version__ = "$Revision: 1.9 $".split()[1]

# ToDo:
#
#  Cannot reload __main__ - explain why this cannot work,
#  and explain a workaround.
#
#  Optimize - the number of watches objects (in old_objects)
#  grows without limits. Think if this is really necessary...


import time, os, threading, sys, types, imp, inspect, traceback

def _get_compiled_ext():
for ext, mode, typ in imp.get_suffixes():
if typ == imp.PY_COMPILED:
return ext

# the official way to get the extension of compiled files (.pyc or .pyo)
PY_COMPILED_EXT = _get_compiled_ext()

class ModuleWatcher:
running = 0
def __init__(self):
# If we don't do this, there may be tracebacks
# when shutting down python.
import atexit
atexit.register(self.stop)

def run(self):
if self.running:
print "# autoreload already running"
return
print "# starting autoreload"
self.running = 1
self.thread = threading.Thread(target=self._check_modules)
self.thread.setDaemon(1)
self.thread.start()

def stop(self):
if not self.running:
print "# autoreload not running"
return
self.running = 0
self.thread.join()
#print "# autoreload stopped"

def _check_modules(self):
skipped = {}
while self.running:
time.sleep(0.01)
for m in sys.modules.values():
if not hasattr(m, '__file__'):
continue
if m.__name__ == '__main__':

# we cannot reload(__main__) First I thought we
# could use mod = imp.load_modu

Re: [Numpy-discussion] [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Jean-Paul Calderone]

On Wed, 25 Apr 2007 20:40:14 -0400, "Phillip J. Eby" <[EMAIL PROTECTED]> wrote:
>At 06:40 PM 4/25/2007 -0400, Jean-Paul Calderone wrote:
>>On Wed, 25 Apr 2007 18:10:23 -0400, Jim Jewett <[EMAIL PROTECTED]> 
>>wrote:
>> >The current ABC proposal is to use isinstance as the test; Jeffrey
>> >Yaskin's numbers PEP highlighted the weakness there with a concrete
>> >example.
>> >
>> >If you need to an abstraction less powerful than an existing ABC,
>> >you're out of luck; you can't just assert that the existing class is
>> >already sufficient, nor can you expect everyone else to use multiple
>> >annotations.
>>
>>I'm sure everyone is already aware of the behavior of the classImplements
>>and directlyProvides functions available in zope.interface, which exactly
>>satisfy this use-case in the interface world.
>
>I'm either misunderstanding Jim or you, because I don't see the relationship 
>here.  If I understand Jim correctly, he's actually asking for something 
>like PyProtocols' "protocolIsSubsetOf" declaration -- something that would 
>be like dynamically adding a superclass to an existing interface in 
>zope.interface.  Whereas the features you're talking about sound like 
>declaring that a class object itself implements an interface -- something 
>apparently unrelated to the question at hand.
>

Ugh, you're at least half right.  I didn't read the implementation of
ABC.meets carefully enough to notice that it's actually an adapter
registry, not just a predicate (meets seems like a funny name for that
method).

classImplements would have satisfied the adapterless use-case by letting
Jim declare that someone else's class definition actually implements the
feature set that some unit of code he wants to use that class with is
expecting.

With the adapter registry, I don't see why this isn't just PEP 246 adapt()
(if I am reading one of your later mails correctly though, you think it is
too).

I'm not particularly familiar with protocolIsSubsetOf, but it wouldn't
surprise me at all if it could also satisfy this use-case.  There are
lots of ways to do this, many of them fairly simple.  Using inheritance
is the part that makes it hard, since inheritance is controlled by the
wrong party in most cases and comes with unrelated features that are,
at best, irrelevant to the particular use case and at worst actively
detrimental.

I'm sure a way around this can be invented, I just don't see why it
should be.

Jean-Paul
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Jean-Paul Calderone]

On Wed, 25 Apr 2007 18:10:23 -0400, Jim Jewett <[EMAIL PROTECTED]> wrote:
>The current ABC proposal is to use isinstance as the test; Jeffrey
>Yaskin's numbers PEP highlighted the weakness there with a concrete
>example.
>
>If you need to an abstraction less powerful than an existing ABC,
>you're out of luck; you can't just assert that the existing class is
>already sufficient, nor can you expect everyone else to use multiple
>annotations.

I'm sure everyone is already aware of the behavior of the classImplements
and directlyProvides functions available in zope.interface, which exactly
satisfy this use-case in the interface world.

Jean-Paul
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Jim Jewett]

On 4/25/07, Jeffrey Yasskin <[EMAIL PROTECTED]> wrote:
> class MonoidUnderPlus(Abstract):

Is this useful?  Just because two things are both Monoid instances
doesn't mean I can add them -- they have to be part of the same
Monoid.  By the time you do

assert isinstance(a, MonoidUnderPlus)
assert isinstance(b, MonoidUnderPlus)
assert isinstance(a, b.__class__) or isinstance(b, a.__class__)

I'm not sure how much you've really saved.

> **Open issue:** Do we want to give people a choice of which of the
> following to define, or should we pick one arbitrarily?::

> def __neg__(self):
> """Must define this or __sub__()."""
> return self.zero() - self

> def __sub__(self, other):
> """Must define this or __neg__()."""
> return self + -other

Probably better to pick one;  then it is clear that they have to
override something, and there won't be as many accidental infinite
loops.  If they really want to define the other, they can just do

def __neg__(self):
super(_this_class__, self).__neg__()

The decorator doesn't know it isn't a real override.

-jJ
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Jeffrey Yasskin]

[bcc'ing numpy-discussion. Comments should probably try to stay on the
python-3000 list.]

On 4/25/07, Josiah Carlson <[EMAIL PROTECTED]> wrote:
> Are the base number operations in Python all that difficult to
> understand?  Do we really need to add mathematical formalism into
> Python's type system before people understand the meaning of X * Y?
> Because all I really see this doing is confusing the hell out of people
> who aren't mathematicians; I'm a theoretical computer scientist and I
> had to try to think for a moment to verify that all of those were really
> necessary to separate the cases.
>
> I really do understand wanting to be able to ask "is operation X
> supported for values Y and Z", but is this really necessary for numbers?

If you don't want to use the full power of the system, nothing forces
you to. If someone doesn't know what a Ring or a complex number is,
they can just check for Real and rely on the same operations, at the
small cost of overconstraining their inputs. One reason I'm suggesting
splitting the classes into the "algebra" and "numbers" modules is so
that people who don't know or care about the algebra can rely on just
"numbers" and still get nearly all of the benefits.

However, I think it's important to provide a framework for people who
want to do more powerful things. If you write a function that can take
an arbitrary ring, it should Just Work on a python int, or any other
Complex subclass, without you having to check for a lot of operations
separately. The Haskell98 numerical hierarchy actually does start at
Num, ignoring the algebraic structures underneath, and it draws a lot
of criticism for it.

That does mean that the names and documentation in the numbers module
need to be as clear as we can make them for non-mathematical
audiences. I'm certainly open to suggestions on that front.

-- 
Namasté,
Jeffrey Yasskin
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Josiah Carlson]

"Jeffrey Yasskin" <[EMAIL PROTECTED]> wrote:
> Here's a draft of the numbers ABCs PEP. The most up to date version
> will live in the darcs repository at
> http://jeffrey.yasskin.info/darcs/PEPs/pep-3141.txt (unless the number
> changes) for now. Naming a PEP about numbers 3.141 seems cute, but of
> course, I don't get to pick the number. :) This is my first PEP, so
> apologies for any obvious mistakes.
> 
> I'd particularly like the numpy people's input on whether I've gotten
> floating-point support right.
> 
> Thanks,
> Jeffrey Yasskin
> 
> ---
> PEP: 3141
> Title: A Type Hierarchy for Numbers (and other algebraic entities)
> Version: $Revision: 54928 $
> Last-Modified: $Date: 2007-04-23 16:37:29 -0700 (Mon, 23 Apr 2007) $
> Author: Jeffrey Yasskin <[EMAIL PROTECTED]>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 23-Apr-2007
> Post-History: Not yet posted
> 
> 
> Abstract
> 
> 
> This proposal defines a hierarchy of Abstract Base Classes (ABCs)
> [#pep3119] to represent numbers and other algebraic entities similar
> to numbers.  It proposes:
[snip]

Are the base number operations in Python all that difficult to
understand?  Do we really need to add mathematical formalism into
Python's type system before people understand the meaning of X * Y? 
Because all I really see this doing is confusing the hell out of people
who aren't mathematicians; I'm a theoretical computer scientist and I
had to try to think for a moment to verify that all of those were really
necessary to separate the cases.

I really do understand wanting to be able to ask "is operation X
supported for values Y and Z", but is this really necessary for numbers?


 - Josiah

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Jeffrey Yasskin]

Here's a draft of the numbers ABCs PEP. The most up to date version
will live in the darcs repository at
http://jeffrey.yasskin.info/darcs/PEPs/pep-3141.txt (unless the number
changes) for now. Naming a PEP about numbers 3.141 seems cute, but of
course, I don't get to pick the number. :) This is my first PEP, so
apologies for any obvious mistakes.

I'd particularly like the numpy people's input on whether I've gotten
floating-point support right.

Thanks,
Jeffrey Yasskin

---
PEP: 3141
Title: A Type Hierarchy for Numbers (and other algebraic entities)
Version: $Revision: 54928 $
Last-Modified: $Date: 2007-04-23 16:37:29 -0700 (Mon, 23 Apr 2007) $
Author: Jeffrey Yasskin <[EMAIL PROTECTED]>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 23-Apr-2007
Post-History: Not yet posted


Abstract


This proposal defines a hierarchy of Abstract Base Classes (ABCs)
[#pep3119] to represent numbers and other algebraic entities similar
to numbers.  It proposes:

* A hierarchy of algebraic concepts, including monoids, groups, rings,
  and fields with successively more operators and constraints on their
  operators. This will be added as a new library module named
  "algebra".

* A hierarchy of specifically numeric types, which can be converted to
  and from the native Python types. This will be added as a new
  library module named "numbers".

Rationale
=

Functions that take numbers as arguments should be able to determine
the properties of those numbers, and if and when overloading based on
types is added to the language, should be overloadable based on the
types of the arguments. This PEP defines some abstract base classes
that are useful in numerical calculations. A function can check that
variable is an instance of one of these classes and then rely on the
properties specified for them. Of course, the language cannot check
these properties, so where I say something is "guaranteed", I really
just mean that it's one of those properties a user should be able to
rely on.

This PEP tries to find a balance between providing fine-grained
distinctions and specifying types that few people will ever use.

Specification
=

Although this PEP uses terminology from PEP3119, the hierarchy is
meaningful for any systematic method of defining sets of
classes. **Todo:** link to the Interfaces PEP when it's ready. I'm
also using the extra notation from [#pep3107] (annotations) to specify
some types.

Object oriented systems have a general problem in constraining
functions that take two arguments. To take addition as an example,
``int(3) + int(4)`` is defined, and ``vector(1,2,3) + vector(3,4,5)``
is defined, but ``int(3) + vector(3,4,5)`` doesn't make much sense. So
``a + b`` is not guaranteed to be defined for any two instances of
``AdditiveGroup``, but it is guaranteed to be defined when ``type(a)
== type(b)``. On the other hand, ``+`` does make sense for any sorts
of numbers, so the ``Complex`` ABC refines the properties for plus so
that ``a + b`` is defined whenever ``isinstance(a,Complex) and
isinstance(b,Complex)``, even if ``type(a) != type(b)``.


Monoids (http://en.wikipedia.org/wiki/Monoid) consist of a set with an
associative operation, and an identity element under that
operation. **Open issue**: Is a @classmethod the best way to define
constants that depend only on the type?::

class MonoidUnderPlus(Abstract):
"""+ is associative but not necessarily commutative and has an
identity given by plus_identity().

Subclasses follow the laws:

  a + (b + c) === (a + b) + c
  a.plus_identity() + a === a === a + a.plus_identity()

Sequences are monoids under plus (in Python) but are not
AdditiveGroups.
"""
@abstractmethod
def __add__(self, other):
raise NotImplementedError

@classmethod
@abstractmethod
def plus_identity(cls):
raise NotImplementedError

I skip ordinary non-commutative groups here because I don't have any
common examples of groups that use ``+`` as their operator but aren't
commutative. If we find some, the class can be added later.::

class AdditiveGroup(MonoidUnderPlus):
"""Defines a commutative group whose operator is +, and whose inverses
are produced by -x.
See http://en.wikipedia.org/wiki/Abelian_group.

Where a, b, and c are instances of the same subclass of
AdditiveGroup, the operations should follow these laws, where
'zero' is a.__class__.zero().

  a + b === b + a
(a + b) + c === a + (b + c)
   zero + a === a
   a + (-a) === zero
  a - b === a + -b

Some abstract subclasses, such as Complex, may extend the
definition of + to heterogenous subclasses, but AdditiveGroup only
guarantees it's defined on arguments of exactly the same types.

Vectors are AdditiveGroups but are no

Re: [Numpy-discussion] matlab vs. python question

[Christopher Barker]

Brian Blais wrote:
> I agree with Sturla here, up until Pyrex.  Looking at the Python API, swig, 
> f2py,
> etc. I found mex files far more straightforward.  Mostly this is because of 
> the
> (mostly) single datatype in Matlab: the double matrix.

well, that's not really fair. I've written extensions to Numeric (but 
the same would be true for numpy) that only work on 2-d double arrays -- 
that was easy. Generalizing to other types and shapes is a pain, but you 
don't even have that option with Matlab.

> Having to do reference counting in the Python API is a drag.

Yes, it is. But every tool, other than writing to the C API directly, 
takes care of that for you.

> Now, with Pyrex, that's a different matter entirely.  Pyrex makes extension 
> writing
> so incredibly easy,

Yes, very cool. I haven't used it for anything real, but f2py looks very 
easy too, if you're working with Fortran.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

[EMAIL PROTECTED]
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] PEP 31XX: A Type Hierarchy for Numbers (and other algebraic entities)

[Dan Christensen]

[sage list added]

"Travis E. Oliphant" <[EMAIL PROTECTED]> writes:

> The SAGE people may be interested in this, but I doubt there will more 
> than a handful of users of these algebraic base classes.

SAGE has quite a sophisticated type hierarchy, and a sophisticated set
of coercion methods.  What is done in SAGE should definitely be
consulted, since it is probably the most complex set of mathematical
types yet written in python.  The SAGE tutorial is at

  http://www.sagemath.org/doc/html/tut/tut.html

and Section 2.2 gives a brief introduction to numbers:

  http://www.sagemath.org/doc/html/tut/node9.html

The SAGE reference manual is at

  http://www.sagemath.org/doc/html/ref/index.html

Chapter 20:

  http://www.sagemath.org/doc/html/ref/node198.html

and nearby chapters are quite relevant for this discussion.

> For general purpose Python, I would do something like
>
> Complex
>Rational_Complex
>  Integer_Complex
>Floating_Complex  # uses hardware float
>Decimal_Complex
>Real
>  Rational
>Integer
>  Floating# uses hardware float
>  Decimal

Note also that double-precision reals are a subset of the rationals,
since each double precision real is exactly representable as a
rational number, but many rational numbers are not exactly
representable as double precision reals.  Not sure if this means
that reals should be a subclass of the rationals.

I believe that in SAGE these relationships aren't expressed using
subclassing.

Dan

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] [ANN] PyTables 2.0rc1 released

[Ivan Vilata i Balaguer]

Hi all,

The development of the second major release of PyTables is steadily
approaching its goal!  Today we want to announce the *first release
candidate version of PyTables 2.0*, i.e. PyTables 2.0rc1.  This version
settles the API and file format for the following PyTables 2.0 series.
No more features will be added until the final version of 2.0 is
released, so we do enter a time for exhaustive platform testing and
fixing the last remaining bugs, as well as updating any outdated
documentation.

Your collaboration is very important in this stage of development, so we
encourage you to download PyTables, test it, and report any problems you
find or any suggestions you have.  Thank you!

And now, the official announcement:



 Announcing PyTables 2.0rc1


PyTables is a library for managing hierarchical datasets and designed to
efficiently cope with extremely large amounts of data with support for
full 64-bit file addressing.  PyTables runs on top of the HDF5 library
and NumPy package for achieving maximum throughput and convenient use.

You can download a source package of the version 2.0rc1 with
generated PDF and HTML docs and binaries for Windows from
http://www.pytables.org/download/preliminary/

For an on-line version of the manual, visit:
http://www.pytables.org/docs/manual-2.0rc1
Please have in mind that some sections in the manual can be obsolete
(specially the "Optimization tips" chapter).  Other chapters should be
fairly up-to-date though (although still a bit in state of flux).

In case you want to know more in detail what has changed in this
version, have a look at ``RELEASE_NOTES.txt``.  Find the HTML version
for this document at:
http://www.pytables.org/moin/ReleaseNotes/Release_2.0rc1

If you are a user of PyTables 1.x, probably it is worth for you to look
at ``MIGRATING_TO_2.x.txt`` file where you will find directions on how
to migrate your existing PyTables 1.x apps to the 2.0 version.  You can
find an HTML version of this document at
http://www.pytables.org/moin/ReleaseNotes/Migrating_To_2.x

Keep reading for an overview of the most prominent improvements in
PyTables 2.0 series.


New features of PyTables 2.0


- NumPy is finally at the core!  That means that PyTables no longer
  needs numarray in order to operate, although it continues to be
  supported (as well as Numeric).  This also means that you should be
  able to run PyTables in scenarios combining Python 2.5 and 64-bit
  platforms (these are a source of problems with numarray/Numeric
  because they don't support this combination as of this writing).

- Most of the operations in PyTables have experimented noticeable
  speed-ups (sometimes up to 2x, like in regular Python table
  selections).  This is a consequence of both using NumPy internally and
  a considerable effort in terms of refactorization and optimization of
  the new code.

- Combined conditions are finally supported for in-kernel selections.
  So, now it is possible to perform complex selections like::

  result = [ row['var3'] for row in
 table.where('(var2 < 20) | (var1 == "sas")') ]

  or::

  complex_cond = '((%s <= col5) & (col2 <= %s)) ' \
 '| (sqrt(col1 + 3.1*col2 + col3*col4) > 3)'
  result = [ row['var3'] for row in
 table.where(complex_cond % (inf, sup)) ]

  and run them at full C-speed (or perhaps more, due to the cache-tuned
  computing kernel of Numexpr, which has been integrated into PyTables).

- Now, it is possible to get fields of the ``Row`` iterator by
  specifying their position, or even ranges of positions (extended
  slicing is supported).  For example, you can do::

  result = [ row[4] for row in table# fetch field #4
 if row[1] < 20 ]
  result = [ row[:] for row in table# fetch all fields
 if row['var2'] < 20 ]
  result = [ row[1::2] for row in   # fetch odd fields
 table.iterrows(2, 3000, 3) ]

  in addition to the classical::

  result = [row['var3'] for row in table.where('var2 < 20')]

- ``Row`` has received a new method called ``fetch_all_fields()`` in
  order to easily retrieve all the fields of a row in situations like::

  [row.fetch_all_fields() for row in table.where('column1 < 0.3')]

  The difference between ``row[:]`` and ``row.fetch_all_fields()`` is
  that the former will return all the fields as a tuple, while the
  latter will return the fields in a NumPy void type and should be
  faster.  Choose whatever fits better to your needs.

- Now, all data that is read from disk is converted, if necessary, to
  the native byteorder of the hosting machine (before, this only
  happened with ``Table`` objects).  This should help to accelerate
  applications that have to do computations with data generated in
  platforms with a byteorder different than the user machine.

- The modification of values in ``*Array`` obje

Re: [Numpy-discussion] matlab vs. python question

[Fernando Perez]

On 4/26/07, Gael Varoquaux <[EMAIL PROTECTED]> wrote:

> Anyway, IANAL, and I am not to sure if releasing a preprint on a mailing
> list renders the article ineligible for CiSE or not but I just put a
> version on
> http://gael-varoquaux.info/computers/agile_computer_control_of_an_experiment.pdf

You're safe: there was a thread yesterday on the MPL list precisely on
this, the CiSE policy (IEEE/AIP in reality) allows you to do this, as
long as you replace it with their official PDF once/if it gets
published and you clearly indicate their copyright terms.  But hosting
a preprint on your own site is allowed by their policies.

I can hunt down the specific references if you need them.

Cheers,

f
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[Gael Varoquaux]

On Thu, Apr 26, 2007 at 12:06:56PM +0200, Zdeněk Hurák wrote:
> But what makes Matlab difficult to be replaced is that lots of other
> projects (commercial: Mathematica, Maple, ... and free: octave, maxima,
> scipy, ...) only offer computation and visualization, while engineers in my
> field also need INTERACTION OF THE SYSTEM WITH EXTERNAL WORLD. That is,
> compatibility with a real-time operating system and MOST available
> input-output (AD-DA) cards. Being able to acquire measurement data from an
> external industrial system, process them computationally (for instance,
> solving some Riccati matrix differential equations), visualize the data and
> put the computed results back to the real system, this is what we need.

I am very surprised that you think Matlab is more suited for such a task.
I have had the opposite experience. 

I work in an experimental lab where we do rather heavy experimental
physics (Bose Einstein condensation). I am building an experiment from
scratch and had to build a control framework for the experiment. It is
fully automated and with need some reasonably sophisticated logics to
scan parameters, do on the fly processing of the data, store it on the
disk, display all this in a useful interface for the user, and feed all
this back to the experiment.

Due to legacy reasons I built the software in Matlab. It was a bad
experience. 

First of all linking to C libraries to control hardware is really a pain.
Writing Mex files is not that much fun and the memory management of
MatLab is flawed. I have had segfaults for no obvious reasons, and the
problems where neither in my code nor in the drivers (I worked with the
engineer who wrote the driver to diagnose this). Matlab is single
threaded so all your calls to the hardware are blocking (unless you are
willing to add threads in you mex file, but this is very dangerous as you
are playing with a loop-hole in matlab's mex loader which keeps the mex
in the memory after it is executed). Finally the lack of proper object
oriented programming and separated namespaces make it hard to write good
code to control instruments.

The lack of object oriented programming, passing by reference, threads,
and proper control of the GUI event-loop also makes it very hard to write
a complex interactive GUI.

Python provides all this. What it does not provide are pre-linked
libraries to control hardware, but if you are trying to control exotic
hardware, all you have is a C or C++ SDK. It is even worse when the
hardware is homemade, as you have to write the library yourself, and
trust me, I'd much better write a hardware control library that speak to
my custom made electronic over a bus in Python than in Matlab (GPIB,
ethernet, serial, Python has bindings for all that).

I have had a second choice to build the computer control framework of a
similar experiment. I took my chance to build it in Python (it was a
different lab, and I was replacing an existing C software that had become
impossible to maintain). I never regretted it. Python has really been
very good at it due to all the different libraries available, the ease to
link to C, and the possibility to do proper OOP.

I have written an article about this, that I submitted a month ago to CiSE.
I have no news from them. The article is very poorly written (I had no
hindsight about these matters when I started writing it, submitted it
because I was tired to see it dragging along, and now I wish I had
reworked it a bit), but I think I raise some points about what you need
to build a control framework for an experiment, and how python can
address those needs.

Anyway, IANAL, and I am not to sure if releasing a preprint on a mailing
list renders the article ineligible for CiSE or not but I just put a
version on
http://gael-varoquaux.info/computers/agile_computer_control_of_an_experiment.pdf
. I hope it can help understanding what is needed to control an
experiment from Python, what can be improved, and what already exists and
is so incredibly useful.

I am interested in comments. Keep in mind that I am working in a field
where nobody is interested in computing and I sure could use some advice.
I have written quite a few softwares to control experiments in different
labs with different languages, and I think I have acquired some
experience, and made a lot of progress, but I also have had to reinvent
the wheel more than once. We all know that good software engineering
books or article readable by some one who nows little about computing are
hard to find. Let us address this issue !

Gaël
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[Sturla Molden]

On 4/26/2007 2:42 PM, David Cournapeau wrote:

 > You are true for the semantics, but wrong for the consequences on 
copying, as matlab is using COW, and this works well in matlab.

It works well only if you don't change your input arguments. Never try 
to write to a matrix received as an argument in a function call. If you 
do, the memory expenditure may grow very rapidly.

But as long a s NumPy does not use lazy evaluation the difference is not 
as striking as it could be.


S.M.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[Sturla Molden]

On 4/26/2007 2:47 PM, Neal Becker wrote:

> That's interesting.  How did you find this information?

What information?

Matlab's pass-by-value semantics is well known to anyone who has ever 
used Matlab.

The Mathwork's employees have numerous times stated that Matlab uses 
copy-on-write to optimize function calls. I first learned of it in the 
usenet group comp.soft-sys.matlab.


Sturla Molden




___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[Neal Becker]

Sturla Molden wrote:

> On 4/26/2007 2:19 PM, Steve Lianoglou wrote:
> 
>>> Beside proper programing paradigm Python easily scales to large-
>>> scale number crunching: You can run large-matrices calculations
>>> with about 1/2 to 1/4 of memory consumption comparing to Matlab.
>> 
>> Is that really true? (The large-matrix number crunching, not the
>> proper programming paradigm ;-)
>> 
>> By no scientific means of evaluation, I was under the impression that
>> the opposite was true to a smaller degree.
> 
> Matlab have pass-by-value semantics, so you have to copy your data in
> and copy your data out for every function call. You can achieve the same
> result in Python by pickling and unpickling arguments and return values,
> e.g. using this function decorator:
> 
> 
> import cPickle as pickle
> 
> def Matlab_Semantics(f):
> 
> '''
> Emulates Matlab's pass-by-value semantics,
> objects are serialized in and serialized out.
> 
> Example:  @Matlab_Semantics
>   def foo(bar):
>   pass
> '''
> 
> func = f
> return wrapper
> 
> def wrapper(*args,**kwargs):
>args_in = pickle.loads(pickle.dumps(args))
>kwargs_in = {}
>for k in kwargs:
>   kwargs_in[k] = pickle.loads(pickle.dumps(kwargs[k]))
>args_out = func(*args_in,**kwargs_in)
>args_out = pickle.loads(pickle.dumps(args_out))
>return args_out
> 
> Imagine using this horrible semantics in several layers of function
> calls. That is exactly what Matlab does. Granted, Matlab optimizes
> function calls by using copy-on-write, so it will be efficient in some
> cases, excessive cycles og copy-in and copy-out is usually what you get.
> 
> 
That's interesting.  How did you find this information?

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[David Cournapeau]

Sturla Molden wrote:
> On 4/26/2007 2:19 PM, Steve Lianoglou wrote:
>
>>> Beside proper programing paradigm Python easily scales to large- 
>>> scale number crunching: You can run large-matrices calculations  
>>> with about 1/2 to 1/4 of memory consumption comparing to Matlab.
>> Is that really true? (The large-matrix number crunching, not the  
>> proper programming paradigm ;-)
>>
>> By no scientific means of evaluation, I was under the impression that  
>> the opposite was true to a smaller degree.
>
> Matlab have pass-by-value semantics, so you have to copy your data in 
> and copy your data out for every function call. 
You are true for the semantics, but wrong for the consequences on 
copying, as matlab is using COW, and this works well in matlab. I have 
never noticed a big difference between matlab and python + numpy for 
memory consumption; I fail to see any reason why it would be 
significantly different (except the fact that numpy does not have to use 
double, whereas matlab had to for a long time, and double is still the 
default on matlab).

David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[Zdeněk Hurák]

Steve Lianoglou wrote:

>> Beside proper programing paradigm Python easily scales to large-
>> scale number crunching: You can run large-matrices calculations
>> with about 1/2 to 1/4 of memory consumption comparing to Matlab.
> 
> Is that really true? (The large-matrix number crunching, not the
> proper programming paradigm ;-)
> 
> By no scientific means of evaluation, I was under the impression that
> the opposite was true to a smaller degree.
> 
> Also, a lot of the times that I'm dealing with extremely large
> matrices, they're of the sparse variety, which matlab currently
> handles with a bit more ease (and transparency) to the user.
> 
> -steve

As Matlab is linking to Lapack, Atlas and similar libraries for (at least a
part of) linear algebra computations, if you link to these freely available
libraries from whatever other language, you should get about the same
results.

Zdenek

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[Sturla Molden]

On 4/26/2007 2:19 PM, Steve Lianoglou wrote:

>> Beside proper programing paradigm Python easily scales to large- 
>> scale number crunching: You can run large-matrices calculations  
>> with about 1/2 to 1/4 of memory consumption comparing to Matlab.
> 
> Is that really true? (The large-matrix number crunching, not the  
> proper programming paradigm ;-)
> 
> By no scientific means of evaluation, I was under the impression that  
> the opposite was true to a smaller degree.

Matlab have pass-by-value semantics, so you have to copy your data in 
and copy your data out for every function call. You can achieve the same 
result in Python by pickling and unpickling arguments and return values, 
e.g. using this function decorator:


import cPickle as pickle

def Matlab_Semantics(f):

'''
Emulates Matlab's pass-by-value semantics,
objects are serialized in and serialized out.

Example:  @Matlab_Semantics
  def foo(bar):
  pass
'''

func = f
return wrapper

def wrapper(*args,**kwargs):
   args_in = pickle.loads(pickle.dumps(args))
   kwargs_in = {}
   for k in kwargs:
  kwargs_in[k] = pickle.loads(pickle.dumps(kwargs[k]))
   args_out = func(*args_in,**kwargs_in)
   args_out = pickle.loads(pickle.dumps(args_out))
   return args_out

Imagine using this horrible semantics in several layers of function 
calls. That is exactly what Matlab does. Granted, Matlab optimizes 
function calls by using copy-on-write, so it will be efficient in some 
cases, excessive cycles og copy-in and copy-out is usually what you get.


Sturla Molden
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[Steve Lianoglou]

> Beside proper programing paradigm Python easily scales to large- 
> scale number crunching: You can run large-matrices calculations  
> with about 1/2 to 1/4 of memory consumption comparing to Matlab.

Is that really true? (The large-matrix number crunching, not the  
proper programming paradigm ;-)

By no scientific means of evaluation, I was under the impression that  
the opposite was true to a smaller degree.

Also, a lot of the times that I'm dealing with extremely large  
matrices, they're of the sparse variety, which matlab currently  
handles with a bit more ease (and transparency) to the user.

-steve


___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] numexpr efficency depends on the size of the computing kernel

[Francesc Altet]

Just a quick followup about this issue:

After a bit of investigation, I discovered that the responsible for the 
difference in performance between the original numexpr and their PyTables 
counterpart (see the message below) was due *only* to the different flags 
used for compiling (and not to a cache instruction overload in the CPU).

It turns out that the original numexpr always add the '-O2 -funroll-all-loops' 
flag (GCC compiler), while I compiled the PyTables instance with the python 
default (-O3).  After recompiling the latter using the same flags as original 
numexpr, then I get exactly the same results on either version of numexpr , 
even with a processor with as a small secondary cache as 64 KB (AMD Duron)
(i.e. the '-funroll-all-loops' flag seems to be *very* effective for 
optimizing the computing kernel of numexpr, at least with CPUs with small 
caches).

So, at least, this leads to the conclusion that the numexpr's virtual machine 
is still far away from getting overloaded, most specially with nowadays 
processors with 512KB of secondary cache or more.

Cheers,

A Dimecres 14 Març 2007 22:05, Francesc Altet escrigué:
> Hi,
>
> Now that I'm commanding my old AMD Duron machine, I've made some
> benchmarks just to prove that the numexpr computing is not influenced by
> the size of the CPU cache, but I failed miserably (and Tim was right:
> there is a dependency of the numexpr efficency on CPU cache size).
>
> Provided that the pytables instance of the computing kernel of numexpr
> is quite larger (it supports more datatypes) than the original,
> comparing the performance of both versions can be a good way to check
> the influence of CPU cache on the computing efficency.
>
> The attached benchmark is a small modification of the timing.py that
> comes with the numexpr package (the modification was needed to allow the
> numexpr version of pytables to run all the cases). Basically, the
> expressions tested operations with arrays of 1 million of elements, with
> a mix of contiguous and strided arrays (no unaligned arrays are present
> here). See the code in benchmark for the details.
>
> The speed-ups of numexpr over plain numpy on a AMD Duron machine (64 +
> 64 KB L1 cache, 64 KB L2 cache) are:
>
> For the original numexpr package:
>
> 2.14, 2.21, 2.21  (these represent averages for 3 complete runs)
>
> For the modified pytables version (enlarged computing kernel):
>
> 1.32, 1.34, 1.37
>
> So, with a CPU with a very small cache, the original numexpr kernel is
> 1.6x faster than the pytables one.
>
> However, using a AMD Opteron which has a much bigger L2 cache (64 + 64
> KB L1 cache, 1 MB L2 cache), the speed-ups are quite similar:
>
> For the original numexpr package:
>
> 3.10, 3.35, 3.35
>
> For the modified pytables version (enlarged computing kernel):
>
> 3.37, 3.50, 3.45
>
> So, there is effectively a dependency on the CPU cache size. It would be
> nice to run the benchmark with other CPUs with a L2 cache in the range
> between 64 KB and 1 MB so as to find the point where the performance
> starts to be similar (this should be a good guess on the size of the
> computing kernel).
>
> Meanwhile, the lesson learned is that Tim worries were correct: one
> should be very careful on adding more opcodes (at least, until CPUs with
> a very small L2 cache are in use).  With this, perhaps we will have to
> reduce the opcodes in the numexpr version for pytables to a bare
> minimum :-/
>
> Cheers,

-- 
>0,0<   Francesc Altet     http://www.carabos.com/
V   V   Cárabos Coop. V.   Enjoy Data
 "-"
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[Zdeněk Hurák]

Travis Oliphant wrote:

[...]
> I would love to see some good contributions in the area of Simulink-like
> work. There are several things out there that are good starts.

Even though I praised Simulink highly in the previous contribution, I don't
think that it would be a good way to mimic it. That way (the way that, for
instance, Scilab/Scicos is going) you will only be SECOND.

What I like (still as a newcomer) about Scipy/Numpy is that it does not try
to be Matlab for poor. You (I mean the community) can take an inspiration
BY THE USE, but please, don't look too much on Matlab. Let's create
somethink better even at the cost of not being compatible and perhaps
requiring some learing effort from Matlab users. I keep my fingers crossed
for Octave project, which is kind of free (GNU GPL) Matlab clone, but at
the same moment I can recognize their limits in being JUST A CLONE. The
leaders of Octave project do realize this danger but user kind of dictate
compatibility with Matlab.

To be finally more specific, there is some discussion going on in the
systems&control community as to "block-diagram vs. equation-based
modelling". I am not a guru in dynamic systems modeling but in my humble
opinion it is worth considering something like Modelica language
http://www.modelica.org/ as the ground for modeling in Python. There is a
young free implementation called OpenModelica
http://www.ida.liu.se/~pelab/modelica/OpenModelica.html. A commercial
implementation is produced by Dynasim and is called Dymola
http://www.dynasim.com/ and can at least give an inspiration, showing that
from a user point of view, it is also just graphical blocks that are being
connected to build a model.

To make it clear, I am not a proponent of the above mentioned tools, I just
came across these a couple of days ago when searching for an alternative to
Simulink (commercial of free) and I found the whole Modelica movement
interesting and perhaps the only one with significant development effort
investment.

>> But what makes Matlab difficult to be replaced is that lots of other
>> projects (commercial: Mathematica, Maple, ... and free: octave, maxima,
>> scipy, ...) only offer computation and visualization, while engineers in
>> my field also need INTERACTION OF THE SYSTEM WITH EXTERNAL WORLD. That
>> is, compatibility with a real-time operating system and MOST available
>> input-output (AD-DA) cards.
> The only way to solve this is to get more users interested in making
> these kinds of things happen. Or to start a company that does this and
> charges for a special build of Python compatible with your favorite
> hardware.

I think that it would not be necessary to start from the scratch. There are
some free projects for interfacing PC to these cards like COMEDI
http://www.comedi.org/. There are also projects with real-time adaptations
of commong operating systems, for Linux, I know of two: RTAI
https://www.rtai.org/ and RTLinux-GPL  http://www.rtlinux-gpl.org/. I am 
not an expert in these domains but it appears that there should be no
problems to interface to these tools. The only think is the investment of
developers effort, of course...:-)

Best regards,

Zdenek

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[Travis Oliphant]

Zdeněk Hurák wrote:
> Coming from the field of control engineering, I don't think that at this
> moment there is any replacement for their graphical interface to solvers
> for nonlinear differential/difference equations called Simulink. 

This is correct. This is the one thing that Python needs improvements 
in. Many of my colleagues use simulink to design and model a digital 
signal processing element which they can then download to an FPGA using 
a third-party tool from XiLINX. There is no way they could replace that 
with Python/SciPy at this point (although of course it "could" be done).

I would love to see some good contributions in the area of Simulink-like 
work. There are several things out there that are good starts.

> But what makes Matlab difficult to be replaced is that lots of other
> projects (commercial: Mathematica, Maple, ... and free: octave, maxima,
> scipy, ...) only offer computation and visualization, while engineers in my
> field also need INTERACTION OF THE SYSTEM WITH EXTERNAL WORLD. That is,
> compatibility with a real-time operating system and MOST available
> input-output (AD-DA) cards.
The only way to solve this is to get more users interested in making 
these kinds of things happen. Or to start a company that does this and 
charges for a special build of Python compatible with your favorite 
hardware.

>  Being able to acquire measurement data from an
> external industrial system, process them computationally (for instance,
> solving some Riccati matrix differential equations), visualize the data and
> put the computed results back to the real system, this is what we need.
>   
This is doable in many respects already (look at what Andrew Straw has 
done for example), but it of course could be made "easier" to do. But, 
I'm not sure it will happen without the work of a company. It would be 
great if hardware manufacturers used Python and made sure their stuff 
worked right with it, but this requires many more users.
> I am absolutely sure that Python (and Scipy and Numpy) project has a
> potential to fulfill also these needs (and by starting using these and
> sharing my own code I would like to contribute), but it is not the case at
> the moment. Without being negative and discouraging, I think it is fair to
> say that currently for some people it would be very difficult to switch
> completely to Python libraries. 
>   
Yes, this is fair to say. As you have indicated, the situation is 
improving.


-Travis

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[Zdeněk Hurák]

Matthew Brett wrote:
[...]
> I agree that matlab has many attractions as a teaching tool and for
> small numeric processing scripts, but if you are writing a large to
> medium-sized application, I really don't think there is any
> comparison...
[...]

Matthew, there are also other (engineering) computational needs than just
neuroimaging. 

Coming from the field of control engineering, I don't think that at this
moment there is any replacement for their graphical interface to solvers
for nonlinear differential/difference equations called Simulink. Its
extension for discrete-event systems called Stateflow is also impressive. I
know on no other tool (commercial or free) that would offer this high
productivity. 

But what makes Matlab difficult to be replaced is that lots of other
projects (commercial: Mathematica, Maple, ... and free: octave, maxima,
scipy, ...) only offer computation and visualization, while engineers in my
field also need INTERACTION OF THE SYSTEM WITH EXTERNAL WORLD. That is,
compatibility with a real-time operating system and MOST available
input-output (AD-DA) cards. Being able to acquire measurement data from an
external industrial system, process them computationally (for instance,
solving some Riccati matrix differential equations), visualize the data and
put the computed results back to the real system, this is what we need.
Well, to make this complete, I should say that Scilab project is starting
to be a competitor. But it is not completely free and the user comfort is
not too high at the moment. (Perhaps joining efforts with Openmodelica
developers is the way to go for Python community?)

I am absolutely sure that Python (and Scipy and Numpy) project has a
potential to fulfill also these needs (and by starting using these and
sharing my own code I would like to contribute), but it is not the case at
the moment. Without being negative and discouraging, I think it is fair to
say that currently for some people it would be very difficult to switch
completely to Python libraries. 

But surely this is improving steadily.

Zdenek Hurak

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[Zdeněk Hurák]

Nadav Horesh wrote:

> Matlab added classes in a fairly recent version, ...

Just wanted to correct this. Classes were introduced to Matlab something
like ten years ago. I guess that it was in version 5.

Zdenek

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] matlab vs. python question

[Matthew Brett]

Well - these threads always go on for a long time, but...

I've used matlab heavily for 10 years.  I found that I had to use perl
and C fairly heavily to get things done that matlab could not do well.
 Now I've switched to numpy, scipy, matplotlib, there is really
nothing I miss in matlab.  We would not attempt what we are doing now:

http://neuroimaging.scipy.org/

in matlab - it's just not the right tool for a large scale programming effort.

I agree that matlab has many attractions as a teaching tool and for
small numeric processing scripts, but if you are writing a large to
medium-sized application, I really don't think there is any
comparison...

Matthew
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] sort bug

[Travis Oliphant]

Anton Sherwood wrote:
> This code --
>
>   adj = [ [eval(y) for y in x.split()] for x in infile ]
>   val,vec = numpy.linalg.eig(adj)
>   master = zip( val, vec.transpose() )
>   master.sort()
>
> *sometimes* gives this error:
>
>   Traceback (most recent call last):
> File "3work.py", line 14, in 
>   master.sort()
>   ValueError: The truth value of an array with more
>   than one element is ambiguous. Use a.any() or a.all()
>
> What does sort() care about truth values?!
>   

The sort method on lists basically uses cmp(x,y) to determine sort 
order.  In this case, I suspect you are getting errors whenever you have 
equal-valued eigenvalues so the comparison has to go to the second item 
in the tuple (which is the array).   

cmp(x,y) must return -1, 0, or 1 which doesn't work on arrays with more 
than 1 element because it is ambiguous.  Thus you get this error.

The operation is undefined.  What do you actually want to do when you 
have equal-valued eigenvalues?



-Travis



___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion