[Tutor] New Style Classes, __getitem__ and iteration
Hi Tutors and Tutees, I've been teaching Python quite a while and a brilliant student asked a question that made me realize a big hole in my understanding. I think it is really magical that, when I define __getitem__ on classic classes, the built-in iterator uses it. But, when I make the same class as a new style class, I lose this behavior. I didn't realize that something was lost in defining a new style class. Maybe it's something else that I'm missing. Thank you for your insights. Marilyn p.s. Here's some code that demonstrates my confusion: #!/usr/bin/env python class Circle: def __init__(self, data, times): Put the 'data' in a circle that goes around 'times' times. self.data = data self.times = times def __getitem__(self, i): circle[i] -- Circle.__getitem__(circle, i). l_self = len(self) if i = self.times * l_self: raise IndexError, \ Error raised: Circle object only goes around %d times\ % self.times return self.data[i % l_self] def __len__(self): return len(self.data) class NewCircle(list): def __init__(self, data, times): list.__init__(self, data) self.times = times def __getitem__(self, i): l_self = len(self) if i = self.times * l_self: raise IndexError, \ Error raised: NewCircle object only goes around %d times\ % self.times return list.__getitem__(self, i % l_self) def main(): circle = Circle(around, 3) print sorted(circle) new_circle = NewCircle(around, 3) print sorted(new_circle) main() OUTPUT: $ ./circle_question.py ['a', 'a', 'a', 'd', 'd', 'd', 'n', 'n', 'n', 'o', 'o', 'o', 'r', 'r', 'r', 'u', 'u', 'u'] ['a', 'd', 'n', 'o', 'r', 'u'] $ ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] New Style Classes, __getitem__ and iteration
I think it is really magical that, when I define __getitem__ on classic classes, the built-in iterator uses it. But, when I make the same class as a new style class, I lose this behavior. I didn't realize that something was lost in defining a new style class. Maybe it's something else that I'm missing. marilyn, the problem isn't related to you using a new-style class. if you ported the classic class directly, it works just fine: class Circle2(object): def __init__(self, data, times): Put the 'data' in a circle that goes around 'times' times. self.data = data self.times = times def __getitem__(self, i): circle[i] -- Circle.__getitem__(circle, i). l_self = len(self) print 'i, self.times, l_self:', i, self.times, l_self if i = self.times * l_self: raise IndexError, \ Error raised: Circle object only goes around %d times\ % self.times return self.data[i % l_self] def __len__(self): return len(self.data) $ ./circle_question.py ['a', 'a', 'a', 'd', 'd', 'd', 'n', 'n', 'n', 'o', 'o', 'o', 'r', 'r', 'r', 'u', 'u', 'u'] to really start to debug your code, change the call from sorted() to list(), as in: circle = Circle2(around, 3) print list(circle) the output will change to: $ ./circle_question.py ['a', 'r', 'o', 'u', 'n', 'd', 'a', 'r', 'o', 'u', 'n', 'd', 'a', 'r', 'o', 'u', 'n', 'd'] when you call sorted() or list() on your object, it attempts to build a new list consisting of the return values of each getitem() call. the difference is that with your original class, it doesn't end until the count has surpassed the number of times you want it (3) and the number of elements total (6), meaning it stops when you go beyond 18. i put in a tiny print statement in the __getitem__() code in all 3 classes. in the original classic class and my port to new-style classes, that print statement told me everything i just said above. the thing that is different about your new-style class is that it subclasses list. in your class that derives from list, it did not execute your __getitem__() method even once. that clues me in on my belief that the reason why it doesn't work is because it is *already* a list. calling list() or sorted() on it attempts to create a new list, but since the original data structure is a list, it merely just copies all of its references without using getitem(). without looking at the source code, i can't confirm this. can anyone else finish the sentence i started? hope this partially helps! -- wesley - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Core Python Programming, Prentice Hall, (c)2007,2001 http://corepython.com wesley.j.chun :: wescpy-at-gmail.com python training and technical consulting cyberweb.consulting : silicon valley, ca http://cyberwebconsulting.com ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] New Style Classes, __getitem__ and iteration
On Mon, May 19, 2008 at 5:21 PM, Marilyn Davis [EMAIL PROTECTED] wrote: Hi Tutors and Tutees, I've been teaching Python quite a while and a brilliant student asked a question that made me realize a big hole in my understanding. I think it is really magical that, when I define __getitem__ on classic classes, the built-in iterator uses it. But, when I make the same class as a new style class, I lose this behavior. There are a couple of possible explanations, I'm not sure which is right without looking into the implementation of sorted(). Wesley may be right, that sorted() sees that its argument is a list and uses a lower-level access than __getitem__() to copy the list for sorting. In general, list operations are not implemented in terms of __getitem__(). In other words, you can't subclass list(), override __getitem__() and expect to change how all list operations work. __getitem__() is not a bottleneck procedure. It's also possible that sorted() just expects a sequence and uses the iterator protocol to access the values of its argument. This is certainly the case with your Circle class. This class doesn't define an __iter__() method so Python uses the older iteration protocol based on __getitem__(). With NewCircle, even if sorted() is accessing it with an iterator, your __getitem__() will not be called because list implements __iter__() so iteration will use the modern iteration protocol. -- OK, I looked at the source a bit. builtin_sorted(), in bltinmodule.c, calls PySequence_List() to create a list to sort. PySequence_List(), in abstract.c, creates a new list, then calls _PyList_Extend() to add the new items. _PyList_Extend(), in listobject.c, calls listextend() which is the implementation of list.extend() listextend() is clearly optimized for lists and tuples. I didn't dig farther than that. For other data types it iterates over the argument. A good discussion of the iterator protocol is here: http://www.python.org/doc/2.2.3/whatsnew/node4.html Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] New-style classes
Hi, after updating a program to use new-style classes, I am a bit confused. My setup looks like this (simplified): #Show.py import Data class Base(object): ... class Page(Data.Page, Base): ... class Author(Data.Author, Base): ... #Data.py class Base: ... class Page(Base): ... class Author(Base): ... The module Show.py provides methods, Data.py provides data attributes. I use the property method in Data.Base for a certain attribute - but why does this work? Data.Base has no base classes, so it is not based on a built-in type: How can it be a new-style class? Thanks for any hints, Jan -- There are 10 kinds of people: those who understand binary, and those who don't ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] New-style classes
Jan Eden wrote: Hi, after updating a program to use new-style classes, I am a bit confused. My setup looks like this (simplified): #Show.py import Data class Base(object): ... class Page(Data.Page, Base): ... class Author(Data.Author, Base): ... #Data.py class Base: ... class Page(Base): ... class Author(Base): ... The module Show.py provides methods, Data.py provides data attributes. I use the property method in Data.Base for a certain attribute - but why does this work? Data.Base has no base classes, so it is not based on a built-in type: How can it be a new-style class? Data.Base is not a new-style class, but the classes in Show are. And properties may appear to work in old-style classes. This document http://www.python.org/2.2.3/descrintro.html#property says, Properties do not work for classic classes, but you don't get a clear error when you try this. Your get method will be called, so it appears to work, but upon attribute assignment, a classic class instance will simply set the value in its __dict__ without calling the property's set method, and after that, the property's get method won't be called either. Here is an old-style class that defines a property: class OldStyle: ... def getA(self): ... print getA ... return self.__a ... def setA(self, value): ... print setA ... self.__a = value ... a = property(getA, setA) ... o=OldStyle() OldStyle.getA() is called the first time I access o.a: o.a getA Traceback (most recent call last): File stdin, line 1, in ? File stdin, line 4, in getA AttributeError: OldStyle instance has no attribute '_OldStyle__a' Setting a value for a wipes out the property, now it behaves like a normal attribute: o.a = 3 o.a 3 A class that inherits from object and OldStyle is a new-style class and the property works as expected: class NewStyle(object, OldStyle): ... pass ... n=NewStyle() n.a getA Traceback (most recent call last): File stdin, line 1, in ? File stdin, line 4, in getA AttributeError: 'NewStyle' object has no attribute '_OldStyle__a' n.a=3 setA n.a getA 3 Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] New-style classes
Hi Kent, Kent Johnson wrote on 29.09.2005: Data.Base has no base classes, so it is not based on a built-in type: How can it be a new-style class? Data.Base is not a new-style class, but the classes in Show are. And properties may appear to work in old-style classes. This document http://www.python.org/2.2.3/descrintro.html#property says, Properties do not work for classic classes, but you don't get a clear error when you try this. Your get method will be called, so it appears to work, but upon attribute assignment, a classic class instance will simply set the value in its __dict__ without calling the property's set method, and after that, the property's get method won't be called either. Thanks for your quick reply. I read the document you mentioned before, that's why I was confused. My actual code looks like this: class Base: def GetOwnType(self): try: return self._own_type except: return self.child_type def SetOwnType(self, value): self._own_type = value own_type = property(GetOwnType, SetOwnType) For some of the subclasses of Base, the attribute own_type is defined, the others should use child_type. For both groups of subclasses, this works fine - if own_type has not been set somewhere else, self.child_type is returned when calling self.own_type. When checking Data.Base.__mro__, I get an error, so it is not a new-style class by itself. On the other hand, every time I use the own_type attribute, I do so via instances of new-style classes (Show.Page, Show.Author etc). Could it be that the nature of these classes makes the code in Data.Base behave according to the new-style rules? Thanks, Jan -- I'd never join any club that would have the likes of me as a member. - Groucho Marx ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
