Re: Class variable inheritance
Terry Reedy wrote:
Lie Ryan wrote:
Note that when the python interpreter meets this statement:
class B(P):
def foo(self):
print('ab')
X = 'f'
the compiler sees a class statement - create a new blank class
- assign P as the new class' parent
No, it saves the name 'B' and bases tuple P, and create a new *dict*,
call it d here though it is anonymous as far as the class body is
concerned.
Neat, I'd never thought that it creates the .__dict__ before the class
itself.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
Lie Ryan lie.1...@gmail.com wrote:
Terry Reedy wrote:
Lie Ryan wrote:
Note that when the python interpreter meets this statement:
class B(P):
def foo(self):
print('ab')
X = 'f'
the compiler sees a class statement - create a new blank class
- assign P as the new class'
parent
No, it saves the name 'B' and bases tuple P, and create a new *dict*,
call it d here though it is anonymous as far as the class body is
concerned.
Neat, I'd never thought that it creates the .__dict__ before the
class itself.
It has to be that way: some of the internal methods cannot be modified
after the initial creation of the class, so you need to use a namespace
that exists before the class itself exists.
The situation changes slightly in Python 3 where the metaclass can hook
into the creation of the dict and instead create any kind of object which
exposes a dict-like interface. That opens the door to classes where the
order in which the order attributes are defined is significant or even
where you can give multiple definitions for the same attribute (e.g. to
support overloaded methods).
--
Duncan Booth http://kupuguy.blogspot.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
Duncan Booth wrote:
Lie Ryan lie.1...@gmail.com wrote:
Terry Reedy wrote:
Lie Ryan wrote:
Note that when the python interpreter meets this statement:
class B(P): def foo(self): print('ab') X = 'f'
the compiler sees a class statement - create a new blank class
- assign P as the new class' parent
No, it saves the name 'B' and bases tuple P, and create a new
*dict*, call it d here though it is anonymous as far as the class
body is concerned.
Neat, I'd never thought that it creates the .__dict__ before the
class itself.
It has to be that way: some of the internal methods cannot be
modified after the initial creation of the class, so you need to use
a namespace that exists before the class itself exists.
The situation changes slightly in Python 3 where the metaclass can
hook into the creation of the dict and instead create any kind of
object which exposes a dict-like interface. That opens the door to
classes where the order in which the order attributes are defined is
significant or even where you can give multiple definitions for the
same attribute (e.g. to support overloaded methods).
The documentation for this, with an example, is in RefMan 3.3.3,
Customizing Class Creation. See the metaclass .__prepare__ method.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
http://docs.python.org/reference/datamodel.html#new-style-and-classic... - search for 'method resolution order' for other hits in that document. First of all, that's the LR for Python 2, I'm with Python 3. Second of all, there's one single passage containing the phrase method resolution order in the Python 3 LR, and it's one that I've already quoted -- and discarded -- previously in this thread. -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
And by the way, the reason I've come across this problem at all is because I have something like this: class A: class X: n = 'a' x = X() class B(A): class X(A.X): n = 'b' # x = X() The line commented out was originally not there, but I found out I had to add it if I wanted B().x.n to be 'b' instead of 'a'. I might be wrong, but aren't there some (non-obscure) OOP language in which the equivalent code (without the out-commented line) would have made B().x an object of type B.X instead of A.X? -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
On Tue, 08 Sep 2009 13:14:42 -0700, HPJ wrote: I could, but I will let you read and find what it says about class attributes. You think I would have asked specifically about the Language Reference if I hadn't read it and failed to find what I was looking for? You must be new to the Internet *wink* Of course people ask without having made the effort themselves. Please sir, will you do my work for me? is practically the norm on Internet forums. Only without the please. The closest thing I was able to find was section 3.2. The standard type hierarchy, subsection Custom classes, where it says: When the attribute name is not found [in its namespace dictionary], the attribute search continues in the base classes. This search of the base classes uses the C3 method resolution order [...] That tells me how the lookup works, which I already knew. But it doesn't tell me what happens when a class is inherited. The inheriting class has its base class set, and the MRO (method resolution order), and that's pretty much it as far as I can tell. Seems to me that the documentation is a little incomplete in this regard. It looks to me like one of those language lawyer details that needs fleshing out. Out of curiosity, are there languages where inheritance means copy? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
On Sep 8, 8:51 pm, HPJ henrypija...@gmail.com wrote: Conceptually, Python checks for the presence of B.foo, and if it's not there it checks for foo's presence in the base classes. Yes, I have no problem believing you guys that this is what Python does. Still, my question remains about where in the Language Reference this is specified. And if the answer is nowhere, than the LR needs to be amended, for obviously the way inheritance is done is no small matter and its understanding should not be left to the user's own intuition. Well, I don't know if this detail alone is all that grave an omission-- it'll matter to very few users--but it does seem the LRM could use a subsection on inheritance in general. I'm sure the doc maintainers would welcome a contribution. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
On Sep 8, 11:05 pm, HPJ henrypija...@gmail.com wrote: And by the way, the reason I've come across this problem at all is because I have something like this: class A: class X: n = 'a' x = X() class B(A): class X(A.X): n = 'b' # x = X() You've nested classes here, that's a whole new layer of complexity. I would strongly recommend against nesting classes in Python, unless the inner class is a smallish class (such as a custom exception type) that doesn't interact with the outer class. It's just that nested classes don't ever seem to behave like anyone expects them to. Also you can't take advantage of Python's lexical scoping with nested classes (unlike, for instance, Java) so there really isn't much benefit to nesting them. The line commented out was originally not there, but I found out I had to add it if I wanted B().x.n to be 'b' instead of 'a'. Hm, even if class B did get copies of class A's attributes, B().x.n would still return 'a'. It seems as if you expect class B to re- execute A's statements in B's context, or something. That's not how it works at all. (Now if you define self.x=X() inside of __init__, that's a different story.) I might be wrong, but aren't there some (non-obscure) OOP language in which the equivalent code (without the out-commented line) would have made B().x an object of type B.X instead of A.X? Maybe. For some languages this might be an obvious behavior, but Python would have different circumstances so it isn't so obvious (or possible) there. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
On Tue, Sep 8, 2009 at 11:30 PM, Steven D'Apranoste...@remove.this.cybersource.com.au wrote: snip Out of curiosity, are there languages where inheritance means copy? I think some prototype-based ones handle it that way... Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
Maybe. For some languages this might be an obvious behavior, but Python would have different circumstances so it isn't so obvious (or possible) there. Which means this topic definitely needs to be handled in detail by the LR, and preferably also in the Tutorial. Otherwise there is no way anyone coming to Python could know which of the many ways of inheritance Python follows. -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
On Sep 9, 3:32 am, HPJ henrypija...@gmail.com wrote: Maybe. For some languages this might be an obvious behavior, but Python would have different circumstances so it isn't so obvious (or possible) there. Which means this topic definitely needs to be handled in detail by the LR, and preferably also in the Tutorial. As I said, I'm sure the doc maintainers would likey welcome your contributions. Otherwise there is no way anyone coming to Python could know which of the many ways of inheritance Python follows. Mo way? You are definitely overstating things here. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
Makes sense to me. To step through what's happening: A.n, B.n (0, 0) Here, the lookup on B.n fails (that is, B itself has no variable n), and thus falls back to A.n See, this is what tripped me up, right at the beginning. I thought B would inherit (as in copy) the variable n from A. Can you point out to me which part (or parts) of the Language Reference says this is the way it's supposed to be? -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
HPJ wrote: Makes sense to me. To step through what's happening: A.n, B.n (0, 0) Here, the lookup on B.n fails (that is, B itself has no variable n), and thus falls back to A.n See, this is what tripped me up, right at the beginning. I thought B would inherit (as in copy) the variable n from A. Python does not copy objects unless asked to. Inheritance is a link request, not a copy request. Can you point out to me which part (or parts) of the Language Reference says this is the way it's supposed to be? I could, but I will let you read and find what it says about class attributes. tjr -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
would you expect the B class to have a copy of the foo method? Sorta. I would expect B to have a copy of the foo attribute, which then refers to the same method as A.foo. So the method itself will be be copied, but its address stored separately in A.foo and B.foo. -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
On Sep 8, 4:50 pm, HPJ henrypija...@gmail.com wrote: would you expect the B class to have a copy of the foo method? Sorta. I would expect B to have a copy of the foo attribute, which then refers to the same method as A.foo. So the method itself will be be copied, but its address stored separately in A.foo and B.foo. No, I'm afraid not. Here is what happens. Conceptually, Python checks for the presence of B.foo, and if it's not there it checks for foo's presence in the base classes. (In actuality Python premaps attributes to the approprirate base class, so only two dict lookups are necessary.) Python is a very dynamic langauge which allows you to modify class objects at runtime. So if you inherit from a class, then later modify that class, what should happen? class A(object): foo = 1 class B(A): pass A.foo = 2 print B.foo # what should this print, 1 or 2? You could argue that copying class attributes (so that B.foo would be 1) is a reasonable way to do inheritance, but IMO the referencing attributes in base classes reflects the underlying concept of inheritance better. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
Conceptually, Python checks for the presence of B.foo, and if it's not there it checks for foo's presence in the base classes. Yes, I have no problem believing you guys that this is what Python does. Still, my question remains about where in the Language Reference this is specified. And if the answer is nowhere, than the LR needs to be amended, for obviously the way inheritance is done is no small matter and its understanding should not be left to the user's own intuition. -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
On 9/09/2009 1:51 PM, HPJ wrote: Conceptually, Python checks for the presence of B.foo, and if it's not there it checks for foo's presence in the base classes. Yes, I have no problem believing you guys that this is what Python does. Still, my question remains about where in the Language Reference this is specified. And if the answer is nowhere, than the LR needs to be amended, for obviously the way inheritance is done is no small matter and its understanding should not be left to the user's own intuition. http://docs.python.org/reference/datamodel.html#new-style-and-classic-classes - search for 'method resolution order' for other hits in that document. HTH, Mark -- http://mail.python.org/mailman/listinfo/python-list
Class variable inheritance
I've just found out that a subclass shares the class variables of its superclass until it's instantiated for the first time, but not any more afterwards: Python 3.1 (r31:73574, Jun 26 2009, 20:21:35) [MSC v.1500 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. class A: ... n = 0 ... def __init__(self): ... type(self).n += 1 class B(A): ... pass A.n, B.n (0, 0) (A().n, A.n, B.n), (A().n, A.n, B.n), (B().n, A.n, B.n) ((1, 1, 1), (2, 2, 2), (3, 2, 3)) (A().n, A.n, B.n), (A().n, A.n, B.n), (B().n, A.n, B.n) ((3, 3, 3), (4, 4, 3), (4, 4, 4)) (A().n, A.n, B.n), (A().n, A.n, B.n), (B().n, A.n, B.n) ((5, 5, 4), (6, 6, 4), (5, 6, 5)) This makes no sense to me at all. Could it possibly be a bug? -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
On Mon, 07 Sep 2009 19:21:28 -0700, Henry 'Pi' James wrote: I've just found out that a subclass shares the class variables String variables are strings. Int variables are ints. Float variables are floats. List variables are lists. Class variables are classes. Classes are first-class objects in Python. Perhaps you mean class attributes? of its superclass until it's instantiated for the first time, but not any more afterwards: ... This makes no sense to me at all. Could it possibly be a bug? You have misinterpreted what you have seen, and if there's a bug, it's in your code. When you retrieve an attribute, Python first looks for instance attributes, then class attributes, then attributes attached to superclasses. When you assign to an attribute, Python conceptually uses the exact same search path, except that instance assignment always succeeds. (Well, almost -- but if it fails, you get an exception.) So in practice: x = obj.whatever may return the contents of an instance attribute whatever, a class attribute, or an attribute of a superclass. But: obj.whatever = x always attempts to store x as an instance attribute, because there's no way for Python to read your mind and know that you mean a class attribute unless you say so explicitly. This attempt will either succeed, or it will raise an exception. Python doesn't try writing further along the hierarchy of instance/class/superclass(es). If you wish to write to a class attribute, you have to explicitly say so: obj.__class__.whatever = x Your other misunderstanding relates to augmented assignment: x += 1 does not modify x in place, it is *exactly* equivalent to: x = x + 1 Given the rules of attribute access, obj.whatever += 1 is exactly equivalent to: obj.whatever = obj.whatever + 1 The right hand attribute access finds a class attribute, and the left hand one sets an instance attribute. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Class variable inheritance
On Mon, Sep 7, 2009 at 7:21 PM, Henry 'Pi' Jameshenrypija...@gmail.com wrote: I've just found out that a subclass shares the class variables of its superclass until it's instantiated for the first time, but not any more afterwards: Python 3.1 (r31:73574, Jun 26 2009, 20:21:35) [MSC v.1500 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. class A: ... n = 0 ... def __init__(self): ... type(self).n += 1 class B(A): ... pass snip This makes no sense to me at all. Could it possibly be a bug? Makes sense to me. To step through what's happening: A.n, B.n (0, 0) Here, the lookup on B.n fails (that is, B itself has no variable n), and thus falls back to A.n Thus, at this point, the expressions `A.n` and `B.n` are equivalent. (A().n, A.n, B.n) (1, 1, 1) A.__init__() gets called, incrementing A.n; again, B.n falls back to A.n (A().n, A.n, B.n) (2, 2, 2), same thing (B().n, A.n, B.n) (3, 2, 3) A.__init__() gets called since B did not define one of its own and this inherited A's. Therein, type(self) evaluates to B (not A as before). B.n += 1 is in this case equivalent to: B.n = B.n +1 Evaluating the right side, B.n falls back A.n once again, and we add 1. Now the assignment takes place, creating a new variable B.n, *separate* from A.n From hereon in, lookups of B.n don't fall back to A.n since B now has its own variable 'n'. Thus, the values of A.n and B.n differ and the expressions now refer to 2 distinct variables. The rest falls out from this and is left as an exercise for the reader. Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Class Variable Inheritance
It became clear to me that mastervar inside of class a is a static
variable, and is associated with all instances of classes that extend
class a. To get around this, I implemented a seperate mapping class:
class mapper:
def __init__(self):
self.mastermap = []
def add(self, map):
self.mastermap.append(map)
def get(self):
return self.mastermap
class a(object):
def __init__(self):
self.map = mapper()
print 'called a'
class b(a):
def __init__(self):
self.map = mapper()
print 'called b'
self.mapvar()
def mapvar(self):
self.map.add('b')
class c(b):
def __init__(self):
self.map = mapper()
print 'called c'
self.mapvar()
def mapvar(self):
super(c, self).mapvar()
self.map.add('c')
if __name__ == '__main__':
a1 = a()
a2 = a()
b1 = b()
c1 = c()
d1 = c() # Call C again
print a1.map.get()
print a1.map.get()
print b1.map.get()
print c1.map.get()
print d1.map.get()
Brian Jones wrote:
I'm sure the solution may be obvious, but this problem is driving me
mad. The following is my code:
--
http://mail.python.org/mailman/listinfo/python-list
Re: Class Variable Inheritance
Brian bojo Jones wrote:
It became clear to me that mastervar inside of class a is a static
variable, and is associated with all instances of classes that extend
class a.
Yeah, that's basically what's happening. AFAICT, a variable declared at
class level is shared with all subclasses (and is available to all
instances unless hidden by an instance variable). You can simulate the
kind of behavior you want using descriptors:
import copy
class subclass_copied(object):
... def __init__(self, initial_value):
... self.initial_value = initial_value
... self.instances = {}
... def __get__(self, instance, owner):
... if owner not in self.instances:
... self.instances[owner] = copy.copy(self.initial_value)
... return self.instances[owner]
...
class A(object):
... x = subclass_copied([])
...
class B(A):
... pass
...
A.x.append(1)
A().x.append(2)
A.x
[1, 2]
B.x
[]
B.x.append(3)
B().x.append(4)
B.x
[3, 4]
A.x
[1, 2]
Basically, the subclass_copied descriptor returns a different object for
each class by keeping a type - object dict. If you wanted to be
thorough with this, you would probably define a __set__ method too; see:
http://docs.python.org/ref/descriptors.html
Steve
Steve
--
http://mail.python.org/mailman/listinfo/python-list
Re: Class Variable Inheritance
On Thu, 2004-12-09 at 08:55, Brian Jones wrote: I'm sure the solution may be obvious, but this problem is driving me mad. The following is my code: class a(object): mastervar = [] def __init__(self): print 'called a' class b(a): def __init__(self): print 'called b' self.mapvar() def mapvar(self): self.mastervar.append(['b']) class c(b): def __init__(self): print 'called c' self.mapvar() def mapvar(self): super(c, self).mapvar() self.mastervar.append(['c']) if __name__ == '__main__': a1 = a() b1 = b() c1 = c() d1 = c() # Call C again print a1.mastervar print b1.mastervar print c1.mastervar print d1.mastervar What I don't understand is why mastervar gets modified by each _seperate instance_ of classes that happen to extend the base class 'a'. Shouldn't mastervar be contained within the scope of the inheriting classes? Why is it being treated like a global variable and being modified by the other instances? A variable declared in a class definition is shared by all instances of the class. Python uses, essentially, a search path for names that goes from most specific to least specific scope. In your example, a lookup in c1 for mastervar will search for mastervar in: c1.__dict__ c1.__class__.dict (ie b.__dict__) a.__dict__ Note that class variables are not _copied_, they're just looked up in the class object if not found in the instance. Remember, the class declaration is only executed _once_ - then __init__ is executed for each new instance. If you want a per-instance copy of a variable, you need to generate a new copy in the __init__ method of the instance (or the parent class and call the parent's __init__ with super() ). If you actually assigned c1.mastervar, rather than modifying the dict, you would get the per-instance dictionary you expected. I strongly recommend a read of the book Learning Python if you want to really _understand_ this stuff (and they do a much better job explaining it than I ever could in my overcomplicated and incoherent way). I think the Python behaviour is counter-intuitive to C++ and Java programmers, but makes good sense in the context of the way Python classes, inheritance, and namespaces work. It's also extremely consistent with the way namespaces and inheritance work in the rest of Python - there's no special magic for objects and classes. A good rule might be If you want Java-style instance variables, create instances variables in __init__ not in the class declaration. This might help explain things - ore might just confuse even more: class A(object): ... ina = [in A] ... class B(A): ... inb = [in B] ... class C(A): ... ina = [in C] ... inc = [in C] ... # examine the class dictionaries ... A.__dict__.keys() ['ina', ...] B.__dict__.keys() ['inb', ...] C.__dict__.keys() ['ina', 'inc', ...] # Now look up some variables to demonstrate the namespace ... # search in class inheritance ... A.ina ['in A'] B.ina ['in A'] C.ina # remember, we redefined this in C ['in C'] B.inb ['in B'] C.inc ['in C'] # This should help explain things ... B.ina is A.ina # True, because B.ina just looks up A.ina True C.ina is A.ina # False, because C.ina is found first False # Now modify B.ina. Because asking for B.ina searches B, then A, ... # for ina, we'll actually end up modifying A.ina ... B.ina.append(blah) A.ina ['in A', 'blah'] B.ina ['in A', 'blah'] # but if we do the same to C.ina, which is redefined in C, ... # a.ina won't be modified ... C.ina.append(change) A.ina ['in A', 'blah'] C.ina ['in C', 'change'] # Now we're going to assign to B.ina, rebinding the name, ... # instead of just modifying the existing mutable object. ... B.ina = fred B.ina fred B.__dict__.keys() ['inb', 'ina'] # Note that there's now a new value for ina in B's dictionary. It ... # is found by the search BEFORE looking for A.ina, and used. A.ina ... # is not modified. ... A.ina ['in A', 'blah'] What you can see happening here is the combination of a couple of principles: - Name lookups in classes happen as a search through the class dictionary then all its parent classes' dictionaries for a name. - Name assignments to a class are made directly in its dictionary. - A modification of a mutable value is a lookup of a name followed by the modification of an object, not an assignment to a name. The principle is pretty similar for instances and classes - a variable defined in a class is a single object shared between all instances of a class, and if mutable can be modified by all of them. Example: class D(object): ... cvar = [] ... def __init__(self, name): ... self.cvar.append(name) ... print I am , name ... a = D(fred) I am fred b =
