William,
<grumpy o' pa>
There is a strong technical issue that needs to be resolved, and for
which I explicitly take the blame. And I understand your frustration
when fighting with zillions of technical points. However I perceived
your e-mail as unfairly bashing the sage-combinat community, which
is just putting me off. Now, I should just step back and answer in a
couple hours, but on the other hand this discussion needs to get
going on the technical side.
Deep breath.
</grumpy o' pa>
On Mon, Jul 26, 2010 at 11:53:59AM -0700, William Stein wrote:
> I'm talking about the __getattr__ that you added. Simply commenting
> out the __getattr__ function leads to a significant speedup. This
> has been replicated by several other people, and this thread
> contains a deep investigation into why this is happening (though
> things still aren't clear).
Then, do you have an explanation for why this does not show up above?
Can you reproduce my timings?
> >> A few months ago some code was added to Sage by the combinat folks
> >> called "categories". I recently learned ObjectiveC, and found out
> >> that this idea of "categories" goes back to smalltalk at least, and
> >> was fully implemented in ObjectiveC in the mid 1980s. It has little
> >> to do with category theory in the sense of mathematics, though is
> >> related. Basically, it is a way to attach a bunch of generic
> >> methods to a class without using inheritance.
> >
> > That's a very reductive view. And actually quite inaccurate: most of
> > Sage's parents and elements *do* get their generic methods from the
> > categories, through class inheritance. There is indeed one exception:
> > extension types (i.e. Cython classes), which currently use the getattr
> > hack (because Cython does not support yet any form of multiple
> > inheritance).
>
> Do you know anything about Objective C or smalltalk or "categories" in
> those languages?
I haven't played with them directly. But from looking around this very
much looks like what "extends" do in other languages I played with
(Aldor, ...). And which I would love to have in Python as well (I am
actually planning to make a crude implementation of it, for plain
Python classes, for use with my external experimental code; it should
just be a couple lines of code).
So yes, the getattr hack is some sort of dynamic implementation of
categories (in smalltalk sense), which is a building block for the
implementation of "real" categories (in the Sage sense) for extension
types.
> I don't really care about blame; I just want to find a way to
> resolve this problem. I've slowed down/messed up etc., so many
> things in Sage over the years, it isn't even funny.
<grumpy o' pa>
I am glad to hear that, because that did not show up in your e-mail.
</grumpy o' pa>
> I think the impact depends a lot on how many python def'd methods the
> class has. E.g., I was testing with the huge number field classes,
> and the slowdown is much greater than with testing with a naked class
> that just derives from Element. I did not notice any real difference
> between otherwise identical Cython and Python classes.
I just had a look at the Python sources (in Object/typeobject.c), and
it's all about the difference between slot_tp_getattro and
slot_tp_getattr_hook, the later being a bit more complicated than the
former. I did not see obvious reasons for having it slower for large
classes, except for the lookup of the __getattr__ method. From
glancing at it, it sounds like this could be optimized, and in
particular that the getattr lookup could be delayed until it is really
used (if at all). This does not tell anything about Cython I guess.
Also it sounds like __getattr__ could be set to None if it is not
really needed, and then on the next call getattro_hook would switch
tp_getattro to the fast simple implementation slot_tp_getattroback. At
least this could solve the issue for Python classes. I'll give it a
shot.
> > 2 Since only extension types need the getattr hack, make sure that
> > __getattr__ is only defined for those. I would love it. Anyone a
> > good idea on how to achieve this? The problem is that all parents
> > and elements classes derive from Parent and Element, which are the
> > natural places to put the getattr hack.
>
> I'm dubious that this is not an issue for Cython classes.
Could someone lookup the sources to know this for sure?
> > 3 Implement partial multiple inheritance in Cython
> >
> > We discussed Option 3 quite some with Robert during the categories
> > brainstorm in Nancy (October 2008). As far as I remember from the
> > discussions, full multiple inheritance (including merging of data
> > structure and method lookup table) was impossible; on the other hand,
> > there was no technical obstructions for allowing a Cython class to
> > inherit from multiple purely abstract classes besides the main base
> > class branch, if one accepts the price of a virtual lookup for the
> > methods defined there. Essentially, this would boil down to
> > implementing a variant of the getattr trick, but within the Cython
> > compiler, and with full control so as to make sure that standard
> > attribute and method lookups would not be affected.
> That sounds like implementing what is called "categories" in
> ObjectiveC. ObjectiveC is an extension to C that was done in the
> late 1980s to add features like one has in Smalltalk. It's built on
> C structs, like Cython is, and does *not* have multiple inheritance.
> Categories make up for this.
Yeah, with the extra thing that this would have to be done
dynamically, and for parents possibly on a parent by parent basis,
since in many cases one does not know statically the category.
Cheers,
Nicolas
--
Nicolas M. Thiéry "Isil" <nthi...@users.sf.net>
http://Nicolas.Thiery.name/
--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org
