Re: Descriptor: class name precedence over instance name
Ian Kelly wrote: > On Sat, Jul 2, 2016 at 3:34 AM, Veek. M wrote: >> So essentially from what Ian said: >> data_descriptor_in_instance -> instance_attribute -> non- >> data_descriptor_in_instance -->__mro__ >> >> is how the search takes place. Correct? > > Close, but I would write it as: > data_descriptor_in_class_including_mro -> instance_attribute -> > non_data_descriptor_or_class_attribute_including_mro > > In either case the class dict search is in __mro__ order. You can find > the actual implementation here: > > https://hg.python.org/cpython/file/30099abdb3a4/Objects/object.c#l1326 > > Roughly, the algorithm is this: > > tp = type(obj) > > # This call searches the MRO. > descr = _PyType_Lookup(tp, name) > > if descr != NULL and is_data_descriptor(descr): > return descr.__get__(obj, tp) > > if name in obj.__dict__: > return obj.__dict__[name] > > if descr != NULL and is_non_data_descriptor(descr): > return descr.__get__(obj, tp) > > if descr != NULL: > return descr > > raise AttributeError(name) > >> -- >> >> Regarding part2 of the Q, :) Ian hasn't explained it, so I'm not sure >> how to explain it better :) but i'll try given that he has clarified >> part of the answer. >> >> Basically Beazley has a TypedProperty descriptor class, and in class >> Foo he instantiates: >> name = TypedProperty >> Then he does f = Foo() >> >> Thanks to Ian, we now know that any lookup on 'f' eg: f.name would > > Not *any* lookup, only one for which there is a declared descriptor. > So in this example f.name or f.num would resolve according to the > descriptor's __get__ method, but any other attribute lookup would just > return whatever value is set on the instance (if any). > >> cause.. well.. the f.name(TypedProperty-descriptor) to gain >> precedence thus hiding the f.name attribute! Therefore he needs to >> name-decorate or in this example append an '_' for whatever stupid >> reason. > > Right, if the "name" descriptor tried to store the value in the > unmangled "name" instance attribute, then its getattr call would just > recur back to the descriptor and you'd have an infinite loop. > > In my opinion the mangling should be a bit heavier than what's done in > this example, to avoid collisions between the descriptor and the class > that's using it. I like to follow the pattern of __foo name mangling, > so I would have instead used: > > self.name = "_TypedProperty__" + name Ouch - that's above my skill level (Python types, especially PyType_Lookup has no docs but i did find https://docs.python.org/3/c-api/index.html) and I got side tracked into reading the PCIe spec so today went down the drain. I'm putting this on hold as totally not understood and requiring much more reading. I'll drag it up later if everyones okay with it. -- https://mail.python.org/mailman/listinfo/python-list
Re: Descriptor: class name precedence over instance name
On Sat, Jul 2, 2016 at 3:34 AM, Veek. M wrote: > So essentially from what Ian said: > data_descriptor_in_instance -> instance_attribute -> non- > data_descriptor_in_instance -->__mro__ > > is how the search takes place. Correct? Close, but I would write it as: data_descriptor_in_class_including_mro -> instance_attribute -> non_data_descriptor_or_class_attribute_including_mro In either case the class dict search is in __mro__ order. You can find the actual implementation here: https://hg.python.org/cpython/file/30099abdb3a4/Objects/object.c#l1326 Roughly, the algorithm is this: tp = type(obj) # This call searches the MRO. descr = _PyType_Lookup(tp, name) if descr != NULL and is_data_descriptor(descr): return descr.__get__(obj, tp) if name in obj.__dict__: return obj.__dict__[name] if descr != NULL and is_non_data_descriptor(descr): return descr.__get__(obj, tp) if descr != NULL: return descr raise AttributeError(name) > -- > > Regarding part2 of the Q, :) Ian hasn't explained it, so I'm not sure > how to explain it better :) but i'll try given that he has clarified > part of the answer. > > Basically Beazley has a TypedProperty descriptor class, and in class Foo > he instantiates: > name = TypedProperty > Then he does f = Foo() > > Thanks to Ian, we now know that any lookup on 'f' eg: f.name would Not *any* lookup, only one for which there is a declared descriptor. So in this example f.name or f.num would resolve according to the descriptor's __get__ method, but any other attribute lookup would just return whatever value is set on the instance (if any). > cause.. well.. the f.name(TypedProperty-descriptor) to gain precedence > thus hiding the f.name attribute! Therefore he needs to name-decorate or > in this example append an '_' for whatever stupid reason. Right, if the "name" descriptor tried to store the value in the unmangled "name" instance attribute, then its getattr call would just recur back to the descriptor and you'd have an infinite loop. In my opinion the mangling should be a bit heavier than what's done in this example, to avoid collisions between the descriptor and the class that's using it. I like to follow the pattern of __foo name mangling, so I would have instead used: self.name = "_TypedProperty__" + name -- https://mail.python.org/mailman/listinfo/python-list
Re: Descriptor: class name precedence over instance name
Ben Finney wrote: > "Veek. M" writes: > >> Trying to make sense of this para: > > At the risk of being ruse, I am trying to make sense of some > paragraphs in the messages you write here. Could you take a little > more time to write clearly, as a way of communicating in this forum? > >> Is he say that Descriptors are a special case where Foo is checked >> first, then what - Base classes..? or does he hop back to look in the >> instance? How is C3 Linearization altered? > > I really have no idea what this paragraph means. Can you please write > again assuming we don't know already what you are trying to say? > Sorry about that, I found it hard to read too (when I came back to it). I was trying to figure out the order in which attributes are looked up. Beazley's a great book, but sometimes he kills me and mounts my head on a pike - page 127 - Descriptors section, last para. He says that descriptor-attribute-names in a class, take precedence in a attribute lookup wrt instance attributes. When you do an x.a, python goes on a hunt for 'a' - the whole binding idea; typically, that is, Instance Name Space -> Class NS -> BaseClasses (C3 Linearization algorithm) Therefore, I was wondering how he could start the search at the instance-Class, instead of the instance. When you print __mro__ you get a list of classes that are traversed but there's no explicit mention within the __mro__ that the instance is searched first. So I wanted to know if there was any implications to C3/__mro__ So essentially from what Ian said: data_descriptor_in_instance -> instance_attribute -> non- data_descriptor_in_instance -->__mro__ is how the search takes place. Correct? -- Regarding part2 of the Q, :) Ian hasn't explained it, so I'm not sure how to explain it better :) but i'll try given that he has clarified part of the answer. Basically Beazley has a TypedProperty descriptor class, and in class Foo he instantiates: name = TypedProperty Then he does f = Foo() Thanks to Ian, we now know that any lookup on 'f' eg: f.name would cause.. well.. the f.name(TypedProperty-descriptor) to gain precedence thus hiding the f.name attribute! Therefore he needs to name-decorate or in this example append an '_' for whatever stupid reason. I think i've got the jist down pat so :p Here's the image: http://storage1.static.itmages.com/i/16/0702/h_1467451175_7972040_b5037f6b46.png (thanks Ian) -- https://mail.python.org/mailman/listinfo/python-list
Re: Descriptor: class name precedence over instance name
"Veek. M" writes: > Trying to make sense of this para: At the risk of being ruse, I am trying to make sense of some paragraphs in the messages you write here. Could you take a little more time to write clearly, as a way of communicating in this forum? > Is he say that Descriptors are a special case where Foo is checked > first, then what - Base classes..? or does he hop back to look in the > instance? How is C3 Linearization altered? I really have no idea what this paragraph means. Can you please write again assuming we don't know already what you are trying to say? -- \ “[I]t is impossible for anyone to begin to learn that which he | `\thinks he already knows.” —Epictetus, _Discourses_ | _o__) | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: Descriptor: class name precedence over instance name
On Fri, Jul 1, 2016 at 10:27 PM, Veek. M wrote:
> Trying to make sense of this para:
>
> --
> Also, the attribute name used by the class to hold a descriptor takes
> prece- dence over attributes stored on instances.
>
> In the previous example,
> this is why the descriptor object takes a name parameter and why the
> name is changed slightly by inserting a leading underscore. In order
> for the descriptor to store a value on the instance, it has to pick a
> name that is different than that being used by the descriptor itself
> -
> Under normal circumstances, when I do an attribute lookup:
> x = Foo()
> x.a
> he will first check 'x' then Foo.
>
> Is he say that Descriptors are a special case where Foo is checked
> first,
It depends whether it's a "data descriptor" or not. A data descriptor
is one that defines at least one of __set__ or __delete__, not just
__get__. Data descriptors take precendence over instance attributes.
Instance attributes take precedence over non-data descriptors.
> then what - Base classes..?
Checking base classes is part of checking the class. If a base class
has a data descriptor, that will likewise take precedence over the
instance attribute, which will likewise take precedence over non-data
descriptors in the base class.
> or does he hop back to look in the
> instance? How is C3 Linearization altered?
It's not.
> Additionally,
> class Foo:
>def __init__(self, name, value):
> self.name = name
>
> cannot be done because
> x = Foo('bar', 10)
>
> x.bar will..? attribute in class takes precedence.. great, isn't that
> what we want?
I don't understand what you're asking here or what this example has to
do with descriptors.
--
https://mail.python.org/mailman/listinfo/python-list
Descriptor: class name precedence over instance name
Trying to make sense of this para:
--
Also, the attribute name used by the class to hold a descriptor takes
prece- dence over attributes stored on instances.
In the previous example,
this is why the descriptor object takes a name parameter and why the
name is changed slightly by inserting a leading underscore. In order
for the descriptor to store a value on the instance, it has to pick a
name that is different than that being used by the descriptor itself
-
Under normal circumstances, when I do an attribute lookup:
x = Foo()
x.a
he will first check 'x' then Foo.
Is he say that Descriptors are a special case where Foo is checked
first, then what - Base classes..? or does he hop back to look in the
instance? How is C3 Linearization altered?
Additionally,
class Foo:
def __init__(self, name, value):
self.name = name
cannot be done because
x = Foo('bar', 10)
x.bar will..? attribute in class takes precedence.. great, isn't that
what we want?
--
https://mail.python.org/mailman/listinfo/python-list
