Hi all,
much delayed, but here it is, finally. The doc regarding our
discussion about PEP 225 is attached, and I'm keeping a public copy
for viewing convenience (with html) here:
https://cirl.berkeley.edu/fperez/static/numpy-pep225/
Note that, as indicated in the link above, the real doc is in bzr, so
you can check it out to generate patches or a branch (the preferred
formats for big changes).
This is just a first cut, going from memory and notes. I'd appreciate
any feedback, corrections, etc. I'm giving a talk at the BayPiggies
group Nov 13 about SciPy and will take the opportunity to bring this
document up, since that user group has a lot of python people,
including some core developers.
Since this is from memory and written by me, it's likely pretty
biased. But I really want to clarify that I'm trying to act as a
scribe here, not to push my own agenda (I do have one :). So please
bring up anything you feel is missing/misstated here; I'd really like
to have a doc that people feel is a fair representation of the
community's views to bring to the core python team. That way we
either get our operators or not once and for all, but this issue can
be put to rest in the language for good (as of 2.7/3.1, of course).
So let's have Nov 11 or so as a wrapup deadline on this discussion, so
I can have the summary ready for the Nov 13 talk.
Best,
f
=================================================================
Discussion regarding possible new operators in Python (PEP 225)
=================================================================
.. Author: Fernando Perez
.. Contact: [EMAIL PROTECTED]
.. Time-stamp: "2008-10-22 12:29:24 fperez"
.. Copyright: this document has been placed in the public domain.
.. contents::
..
1 Introduction
2 Summary
3 Arguments neutral towards or against the PEP
4 Other considerations and suggestions
4.1 Operator form for logical_X functions
4.2 Alternate comparisons
4.3 Why just go one step?
4.4 Alternate syntax
4.5 Unicode operators
Introduction
============
In the Python-dev mailing lists, there were recently two threads regarding the
possibility of adding to the language new multiplication operators (amongst
others). This would allow one to define things like an element-wise and a
matrix product for numpy arrays, for example. The original python-dev threads
are here:
http://mail.python.org/pipermail/python-dev/2008-July/081508.html
http://mail.python.org/pipermail/python-dev/2008-July/081551.html
And the actual PEP that discusses this at length is here:
http://www.python.org/dev/peps/pep-0225/
In order to provide feedback from the scientific/numerical community, a
discussion was held on the numpy mailing list. This document is a brief
summary of this discussion, in addition to some issues that were brought up
during a BOF session that was held during the SciPy'08 conference. The point
of this document is to provide the Python development team (and ultimately
Guido) with feedback from a community that would be likely an important target
of the features discussed in PEP 225, hoping that a final decision can be made
on the PEP, either as-is or with modifications arising from this feedback.
The full discussion thread in the numpy list can be found here:
http://projects.scipy.org/pipermail/numpy-discussion/2008-August/036769.html
http://projects.scipy.org/pipermail/numpy-discussion/2008-August/036858.html
Summary
=======
Overall the response to the pep seems to range from positive to very positive.
Participants expressed that it would solve real-world, practical problems for
them and enable them to use Python more effectively in teaching and research
involving numerical work, in particular (but not limited to) topics involving
linear algebra.
It is probably useful, for the sake of the Python dev team, to provide a bit of
background regarding the array/matrix situation in Numpy, which is at the heart
of much of this. The numpy library provides, in addition to the basic arrays
that support element-wise operations::
In [2]: a = array([[1,2],[3,4]])
In [3]: b = array([[10,20],[30,40]])
In [4]: a*b
Out[4]:
array([[ 10, 40],
[ 90, 160]])
also an object called ``matrix`` that implements classic matrix multiplication
semantics for the ``*`` operator::
In [5]: aM = matrix(a)
In [6]: bM = matrix(b)
In [7]: aM*bM
Out[7]:
matrix([[ 70, 100],
[150, 220]])
The existence of two almost-but-not-quite identical objects at the core of
numpy has been the source of a large amount of discussion and effort to provide
smoother integration. Yet, despite much work it is the opinion of many that
the situation is still unsatisfactory, as many pitfalls continue to exist. It
is very easy to pass a matrix to some routine that does numerical manipulations
on its inputs and forgets to verify that a matrix was received, and to end up
with an array instead afterwards. While much work has gone into making the
core numpy and scipy libraries 'matrix-proof', arbitrary third party code may
or may not be matrix aware, and thus this problem is likely to remain an open
issue.
Multiplication is perhaps the main need for matrices, and the proposed PEP
could provide a viable solution for this problem. Exponentiation is also
useful and covered by the PEP, and the ``~/`` division could perhaps be an
acceptable way of implementing matlab's `slash operator`_.
.. _slash operator:
http://www.mathworks.com/access/helpdesk/help/techdoc/index.html?/access/helpdesk/help/techdoc/ref/mrdivide.html
It is felt that the PEP could lower the notational barrier to using Python for
tasks where languages like Matlab and IDL are typically the default choice.
Expressing common algorithms, especially in the domain of linear algebra, would
be easier and more readable if numpy could provide operator-level support for
matrix operations without introducing new top-level objects.
Arguments neutral towards or against the PEP
============================================
While the response was overall positive, there were a few differing opinions:
* Some expressed a neutral position: they felt there was really no overwhelming
need for this and that people could get by with the tools offered today by
Python).
* Some felt that perhaps the time is not right for this, and that one should
wait instead a few years for Unicode support to become more pervasive, and
implement this idea via Unicode instead (see below).
Other considerations and suggestions
====================================
Here we list some additional considerations that were presented. A number of
participants were interested in seeing the PEP, if adopted, extended to cover
logical and comparison operators as well, so that object-wise semantics could
be provided in parallel to element-wise semantics for all operations, not only
the few arithmetic ones originally discussed in the PEP.
Operator form for logical_X functions
-------------------------------------
Since the meaning of the logical, short-circuiting operators ``and`` and ``or``
in Python can't be redefined (they are keywords of the language), Numpy
implements its own versions as functions: ``logical_and`` and ``logical_or``
(as well as ``not`` and ``xor`` versions). It would be good to extend the PEP
to cover type-defined implementations of
Alternate comparisons
---------------------
It could be useful to extend the original PEP not only to cover the basic
arithmetic operators, but also comparisons. Having for a given object
alternate comparison mechanisms would allow one to declare, for example,
special orderings in fields for objects which can be ordered in different ways
depending on the structure in which they are considered to live.
[XXX - more justification for the field ordering example? This was suggested at
scipy, but I don't recall all the details]
Why just go one step?
---------------------
Instead, it would be nicer to have user-definable multiplication operators such
that for example::
a ~1* b
a ~2* b
etc... are all valid. The user could thus define as many ~N* operators as
desired, for arbitrary integer values of N. The proponents of this idea argue
that this provides a general solution to the problem of user-definable
operators for complex objects while maintaining the spirit of the PEP. There's
the concern that if one new family of operators is allowed, we'll want more in
the future and it is thus wortwhile solving the general problem once and for
all. The R language is cited as a precedent with a similar syntax, where
``%op%`` allows for arbitrary operators to be declared.
Alternate syntax
----------------
The only other syntax to receive much support was to use ``!`` instead of ``~``
as the special character.
Unicode operators
-----------------
A number of proposals were made to consider Unicode operators instead of ascii
2-character combinations. While this was favored by one group, it was also
strongly opposed by another who feared difficulty of editing with standard
tools, font problems and readability problems. Several people had problems
displaying the provided examples in their browsers.
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
