Matthew,
Arron has provided a fantastic summary of the issues involve. The
summary is that even if you subclass Add/Mul/Pow, sympy will end up
creating expressions that don't use your subclasses everywhere. This
can't be fixed without changing the core, which is outside the scope
of your project. I know I sound like a broken record, but you should
resist the temptation to define your own Add/Mul/Pow classes.
Cheers,
Brian
On Tue, Jul 12, 2011 at 10:25 PM, Aaron Meurer <asmeu...@gmail.com> wrote:
> One thing that I have noticed with regard to overriding __mul__ and
> __rmul__ (for example) is that you can never completely control what
> happens to your class because of association. For example, suppose
> that A and B are MatrixExprs and x is some Expr object (say, a
> Symbol). Suppose that A*B should give a ShapeError, and that x is a
> scalar, which does not matter where it is in the expression. Then you
> have to program it so that all of the following give ShapeError:
>
> 1. x*(A*B)
> 2. (x*A)*B
> 3. (A*x)*B
> 4. A*(x*B)
> 5. (A*B)*x
> 6. A*(B*x)
>
> Let us look at these. Obviously, 1 and 5 will work, since you have
> complete control over A.__mul__(B). Similarly, in 4 and 6, you will
> end up with A.__mul__(Mul(x, B)), but this is not a problem, since you
> can make A.__mul__ check for the left-most noncommutative in a Mul.
>
> But what about 2 and 3? Here, we have __mul__ being called by a Expr
> type, namely, Mul(x, A). You want to get B.__rmul__(Mul(x, A)), since
> you have implemented shape checking logic in MatrixExpr.__(r)mul__.
> But b.__rmul__(a) is called with a*b only if a.__mul__(b) returns
> NotImplemented. So, basically, we need Mul.__mul__(MatrixExpr) to
> return NotImplemented. Well, let us look at the code for Mul.__mul__
> (actually Expr.__mul__):
>
> @_sympifyit('other', NotImplemented)
> @call_highest_priority('__rmul__')
> def __mul__(self, other):
> return Mul(self, other)
>
> Well, this is not very helpful, because the logic is buried in some
> decorators. So let us look at the code for those decorators. I will
> not bore you by pasting the entire code here (you can see it in
> sympy/core/decorators.py), but basically, _sympifyit makes tries to
> sympify other, and makes __mul__ return NotImplemented if it fails. I
> am not delving into this, because sympify would not fail for
> MatrixExpr (otherwise, we would actually have a problem doing any form
> of x*A*B).
>
> So call_highest_priority seems to be a better bet. This decorator I
> will also not paste here, but it basically lets you define
> ._op_priority on your object, and if it is greater than Expr's (which
> is quite arbitrarily set to 10.0), then it will call
> other.__rmul__(self).
>
> But wait, there's more. _op_priority can make the above cases work,
> but * is not the only way that SymPy multiplies things. You will also
> have to deal with variations on Mul(x, A, B). Mul completely ignores
> _op_priority. Unfortunately, even if this may seem like a more
> esoteric way to multiply things that you can just recommend users
> avoid, it is used internally a lot, because Mul(*args) is more
> efficient than reduce(operator.mul, args).
>
> Thus, you see that it is quite impossible to make things work 100% of
> the time without modifying the core. And actually, because of the Mul
> thing that would not work at all and that is called by so many core
> functions and methods, you will not even get something like things
> working 90% of the time, but instead things will break when used a
> certain way, and you will have hard to track bugs.
>
> Aaron Meurer
>
> On Tue, Jul 12, 2011 at 3:08 PM, Matthew Rocklin <mrock...@gmail.com> wrote:
>> Subclassing Expr has some issues as well. This is what Brian was referring
>> to. Within all of our code we use Add and Mul and don't check if instead we
>> should use some subclass of Add or subclass of Mul. If I feed a matrix
>> expression into these objects then the special matrix structure is lost.
>> This happens if, for example, you call simplify on a matrix expression.
>> I think I can get around this though with a few well placed "matrixify"
>> functions. Matrixify is a function which goes through the expression tree
>> and makes appropriate fixes. I've had good success so far with a very very
>> basic Matrixify function.
>> Brian, did you have particular horror stories trying to subclass Expr? I'm
>> enthusiastic about my approach but you seemed to have a bad experience. Can
>> you suggest difficult test cases that I should think about?
>>
>>
>> On Fri, Jul 8, 2011 at 3:22 PM, Aaron Meurer <asmeu...@gmail.com> wrote:
>>>
>>> On Fri, Jul 8, 2011 at 4:00 AM, SherjilOzair <sherjiloz...@gmail.com>
>>> wrote:
>>> > There is something I'm doing as part of my project which maybe be
>>> > useful. I'm implementing a Matrix_ wrapper class which will wrap over
>>> > low-level matrices. Its being written to replace the current Matrix
>>> > class. Algorithmic code and user-level code is being separated into
>>> > different classes. Currently, we have three internal matrices, the
>>> > DenseMatrix( a modified form of the current Matrix class), DOKMatrix
>>> > and LILMatrix. These three are essentially Data internal matrices.
>>> > The MatrixSymbol can be another object that Matrix_ (later Matrix)
>>> > would use as internal.
>>> >
>>> > Of what I understand from this discussion about the Matrix Symbol, we
>>> > need to have an object that will be treated like a Matrix everywhere,
>>> > but without the internal data. Space is being made in the Matrix
>>> > module for such an object, but for it work nicely it needs to interact
>>> > nicely with Expr objects. If its a problem subclassing Expr and
>>> > friends, I'm +1 about making a separate codebase for matrix
>>> > expressions. It is, after all, a different algebra.
>>> >
>>> > I'm not very familiar with Expr, Add, etc. but I think a separate
>>> > algebra MatrixExpr, MatrixAdd, etc. could be developed by copy-pasting
>>> > some code from Expr, Add, and modifying according to need. I say 'copy-
>>> > paste' and not subclassing as the code would be need to be modified
>>> > much before it can work on matrices.
>>>
>>> As has been discussed before, this is a bad idea. Just subclass Expr.
>>>
>>> Aaron Meurer
>>>
>>> >
>>> > Presently, I'm quite busy with this new encapsulation class Matrix_
>>> > and can only think of working fulltime on MatrixSymbol after I've
>>> > merged in my work on DOKMatrix, LILMatrix and the ongoing work on
>>> > Matrix_.
>>> >
>>> > Matthew, you could set up a wiki page on 'matrix expressions' with
>>> > some example code in it. 'symbolic matrices' can mean a few number of
>>> > things, and is somewhat vague. It would be good if everyone could see
>>> > what sort of concrete things are expected from matrix expressions.
>>> > Approaches to implement Matrix Algebra, with pros and cons listed,
>>> > could be added there for discussion, suggestion and improvement.
>>> >
>>> > Regards,
>>> > Sherjil Ozair
>>> >
>>> > --
>>> > You received this message because you are subscribed to the Google
>>> > Groups "sympy" group.
>>> > To post to this group, send email to sympy@googlegroups.com.
>>> > To unsubscribe from this group, send email to
>>> > sympy+unsubscr...@googlegroups.com.
>>> > For more options, visit this group at
>>> > http://groups.google.com/group/sympy?hl=en.
>>> >
>>> >
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "sympy" group.
>>> To post to this group, send email to sympy@googlegroups.com.
>>> To unsubscribe from this group, send email to
>>> sympy+unsubscr...@googlegroups.com.
>>> For more options, visit this group at
>>> http://groups.google.com/group/sympy?hl=en.
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "sympy" group.
>> To post to this group, send email to sympy@googlegroups.com.
>> To unsubscribe from this group, send email to
>> sympy+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/sympy?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To post to this group, send email to sympy@googlegroups.com.
> To unsubscribe from this group, send email to
> sympy+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/sympy?hl=en.
>
>
--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgran...@calpoly.edu and elliso...@gmail.com
--
You received this message because you are subscribed to the Google Groups
"sympy" group.
To post to this group, send email to sympy@googlegroups.com.
To unsubscribe from this group, send email to
sympy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sympy?hl=en.
