Re: [Tutor] class decorator question

[Albert-Jan Roskam]

- Original Message -

> From: Steven D'Aprano 
> To: tutor@python.org
> Cc: 
> Sent: Sunday, October 6, 2013 4:52 AM
> Subject: Re: [Tutor] class decorator question
> 
> On Sat, Oct 05, 2013 at 12:26:14PM -0700, Albert-Jan Roskam wrote:
> 
>>  >> On http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ I 
> saw 
>>  >> a very cool and useful example of a class decorator. It 
> (re)implements 
>>  >> __str__ and __unicode__ in case Python 2 is used. For Python 3, 
> the 
>>  >> decorator does nothing. I wanted to generalize this decorator so 
> the 
>>  >> __str__ method under Python 2 encodes the string to an arbitrary 
>>  >> encoding. This is what I've created: 
> http://pastebin.com/vghD1bVJ.
>>  >> 
>>  >> It works, but the code is not very easy to understand, I am 
> affraid. 
>>  >
>>  >It's easy to understand, it's just doing it the wrong way. It 
> creates 
>>  >and subclass of your class, which it shouldn't do. 
>> 
>>  Why not? Because it's an unusual coding pattern? Or is it ineffecient?
> 
> It is both of those things. (Well, the inefficiency is minor.) My 
> main objection is that it is inelegant, like using a screwdriver as 
> a chisel instead of using a chisel -- even when it's "good 
> enough", 
> it's not something you want other people to see you doing if you 
> care about looking like a craftsman :-)

or use a shoe to hammer a nail in the wall... ;-)

> Another issue is to do with naming. In your example, you decorate Test. 
> What that means in practice is that you create a new class, Klass(Test), 
> throw away Test, and bind Klass to the top-level name Test. So in effect 
> you're doing this:
> 
> class Test # The undecorated version.
> 
> class Klass(Test)  # Subclass it inside the decorator.
> 
> Test = Klass  # throw away the original and re-use the variable name.
> 
> But classes, like functions, have *two* names. They have the name they 
> are bound to, the variable name (*usually* one of these, but sometimes 
> zero or two or more). And they have their own internal name:
> 
> Test.__name__
> => returns "Klass"
> 
> 
> This will make debugging unneccesarily confusing. If you use your 
> decorator three times:
> 
> @implements_to_string
> class Spam
> 
> @implements_to_string
> class Eggs
> 
> @implements_to_string
> class Cheese
> 
> 
> instances of all three of Spam, Eggs and Cheese will claim to be 
> instances of "Klass".

That would indeed be *very* confusing. 

> Now there is a simple work-around for this: inside the decorator, call
> 
> Klass.__name__ = cls.__name__ 
> 
> before returning. But that leads to another issue, where instances of 
> the parent, undecorated, class (if any!) and instances of the child, 
> decorated, class both claim to be from the same "Test" class. This is 
> more of theoretical concern, since you're unlikely to be instantiating 
> the undecorated parent class.
> 
> 
>>  I subclassed because I needed the encoding value in the decorator.
>>  But subclassing may indeed have been overkill.
> 
> Yes :-)
> 
> The encoding value isn't actually defined until long after the decorator 
> has finished doing its work, after the class is decorated, and an 
> instance is defined. So there is no encoding value used in the decorator 
> itself. The decorator can trivially refer to the encoding value, so long 
> as that doesn't actually get executed until after an instance is 
> created:
> 
> def decorate(cls):
>     def spam(self):
>         print(self.encoding)
>     cls.spam = spam
>     return cls
> 
> works fine without subclassing.

waah, why didn't I think of this? I've been making this way more complicated 
than needed. self.__dict__["encoding"] = self.encoding (see also below) was 
another way I considered to pass the encoding value from the class to its 
decorator. I even considered making a class decorator with arguments. All 
unnecesary. 
 
> 
>>  >Here's a better 
>>  >approach: inject the appropriate methods into the class directly. 
> Here's 
>>  >a version for Python 3:
> [...]
>>  >This avoids overwriting __str__ if it is already defined, and likewise 
>>  >for __bytes__.
>> 
>>  Doesn't a class always have __str__ implementation?
> 
> No. Where is the __str__ implementation here?
> 
> class X:
>     pass
> 
> This class defines no methods at all. Its *superclass*, object in Python 
> 3, defines methods such as __

Re: [Tutor] class decorator question

[Mark Lawrence]

On 06/10/2013 03:58, Steven D'Aprano wrote:

On Sun, Oct 06, 2013 at 01:06:18AM +0100, Alan Gauld wrote:

On 05/10/13 20:26, Albert-Jan Roskam wrote:


General question: I am using pastebin now. Is that okay,


For code as short as this it's probably best kept with the message.
But once you get to 100+ lines its more debatable and if you get
to 200+ lines I'd definitely say a pastebin is better.


If somebody is tempted to post 200+ lines here, they probably shouldn't.
Instead, they should read this:

http://sscce.org/




I totally agree.

--
Roses are red,
Violets are blue,
Most poems rhyme,
But this one doesn't.

Mark Lawrence

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] class decorator question

[Steven D'Aprano]

On Sun, Oct 06, 2013 at 01:06:18AM +0100, Alan Gauld wrote:
> On 05/10/13 20:26, Albert-Jan Roskam wrote:
> 
> >General question: I am using pastebin now. Is that okay,
> 
> For code as short as this it's probably best kept with the message.
> But once you get to 100+ lines its more debatable and if you get
> to 200+ lines I'd definitely say a pastebin is better.

If somebody is tempted to post 200+ lines here, they probably shouldn't. 
Instead, they should read this:

http://sscce.org/


-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] class decorator question

[Steven D'Aprano]

On Sat, Oct 05, 2013 at 12:26:14PM -0700, Albert-Jan Roskam wrote:

> >> On http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ I saw 
> >> a very cool and useful example of a class decorator. It (re)implements 
> >> __str__ and __unicode__ in case Python 2 is used. For Python 3, the 
> >> decorator does nothing. I wanted to generalize this decorator so the 
> >> __str__ method under Python 2 encodes the string to an arbitrary 
> >> encoding. This is what I've created: http://pastebin.com/vghD1bVJ.
> >> 
> >> It works, but the code is not very easy to understand, I am affraid. 
> >
> >It's easy to understand, it's just doing it the wrong way. It creates 
> >and subclass of your class, which it shouldn't do. 
> 
> Why not? Because it's an unusual coding pattern? Or is it ineffecient?

It is both of those things. (Well, the inefficiency is minor.) My 
main objection is that it is inelegant, like using a screwdriver as 
a chisel instead of using a chisel -- even when it's "good enough", 
it's not something you want other people to see you doing if you 
care about looking like a craftsman :-)

Another issue is to do with naming. In your example, you decorate Test. 
What that means in practice is that you create a new class, Klass(Test), 
throw away Test, and bind Klass to the top-level name Test. So in effect 
you're doing this:

class Test # The undecorated version.

class Klass(Test)  # Subclass it inside the decorator.

Test = Klass  # throw away the original and re-use the variable name.

But classes, like functions, have *two* names. They have the name they 
are bound to, the variable name (*usually* one of these, but sometimes 
zero or two or more). And they have their own internal name:

Test.__name__
=> returns "Klass"


This will make debugging unneccesarily confusing. If you use your 
decorator three times:

@implements_to_string
class Spam

@implements_to_string
class Eggs

@implements_to_string
class Cheese


instances of all three of Spam, Eggs and Cheese will claim to be 
instances of "Klass".

Now there is a simple work-around for this: inside the decorator, call

Klass.__name__ = cls.__name__ 

before returning. But that leads to another issue, where instances of 
the parent, undecorated, class (if any!) and instances of the child, 
decorated, class both claim to be from the same "Test" class. This is 
more of theoretical concern, since you're unlikely to be instantiating 
the undecorated parent class.


> I subclassed because I needed the encoding value in the decorator.
> But subclassing may indeed have been overkill.

Yes :-)

The encoding value isn't actually defined until long after the decorator 
has finished doing its work, after the class is decorated, and an 
instance is defined. So there is no encoding value used in the decorator 
itself. The decorator can trivially refer to the encoding value, so long 
as that doesn't actually get executed until after an instance is 
created:

def decorate(cls):
def spam(self):
print(self.encoding)
cls.spam = spam
return cls

works fine without subclassing.



> >Here's a better 
> >approach: inject the appropriate methods into the class directly. Here's 
> >a version for Python 3:
[...]
> >This avoids overwriting __str__ if it is already defined, and likewise 
> >for __bytes__.
> 
> Doesn't a class always have __str__ implementation?

No. Where is the __str__ implementation here?

class X:
pass

This class defines no methods at all. Its *superclass*, object in Python 
3, defines methods such as __str__. But you'll notice that I didn't call 

hasattr(cls, '__str__') 

since that will return True, due to object having a __str__ method. I 
called

'__str__' in cls.__dict__

which only returns True if cls explicitly defines a __str__ method.


> Nice, thanks Steven. I made a couple of versions after reading your 
> advise. The main change that I still had to somehow retrieve the 
> encoding value from the class to be decorated (decoratee?). I simply 
> stored it in __dict__. Here is the second version that I created: 
> http://pastebin.com/te3Ap50C. I tested it in Python 2 and 3. 

Not sufficiently :-) Your test class has problems. See below.



> The Test 
> class contains __str__ and __unicode__ which are renamed and redefined 
> by the decorator if Python 3 (or 4, or..) is used.
> 
> 
> General question: I am using pastebin now. Is that okay, given that 
> this is not part of the "memory" of the Python Tutor archive? It might 
> be annoying if people search the archives and get 404s if they try to 
> follow these links. Just in case I am also pasting the code below:

In my opinion, no it's not okay, particularly if your code is short 
enough to be posted here.

Just because a pserson has access to this mailing list doesn't 
necessarily mean they have access to pastebin. It might be blocked. The 
site might be down. They might object to websites that require 
Javascript (pastebin doesn't *require* it, but it's 

Re: [Tutor] class decorator question

[Alan Gauld]

On 05/10/13 20:26, Albert-Jan Roskam wrote:


General question: I am using pastebin now. Is that okay,


For code as short as this it's probably best kept with the message.
But once you get to 100+ lines its more debatable and if you get
to 200+ lines I'd definitely say a pastebin is better.


from __future__ import print_function
import sys

def decorate(cls):
 print("decorate called")
 if sys.version_info[0] > 2:
 cls.__dict__["__str__"].__name__ = '__bytes__'
 cls.__dict__["__unicode__"].__name__ = '__str__'
 cls.__bytes__ = cls.__dict__["__str__"]
 cls.__str__ = cls.__dict__["__unicode__"]
 return cls

@decorate
class Test(object):

 def __init__(self):
 self.__dict__["encoding"] = self.encoding

 def __str__(self):
 return "str called".encode(self.encoding)

 def __unicode__(self):
 return "unicode called"

 @property
 def encoding(self):
 """In reality this method extracts the encoding from a file"""
 return "utf-8" # rot13 no longer exists in Python3

if __name__ == "__main__":
 t = Test()
 if sys.version_info[0] == 2:
 print(unicode(t))
 print(str(t))


--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.flickr.com/photos/alangauldphotos

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] class decorator question

[Albert-Jan Roskam]

___

> From: Steven D'Aprano 
>To: tutor@python.org 
>Sent: Saturday, October 5, 2013 3:14 PM
>Subject: Re: [Tutor] class decorator question
>
>On Sat, Oct 05, 2013 at 05:33:46AM -0700, Albert-Jan Roskam wrote:
>> Hi,
>> 
>> On http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ I saw 
>> a very cool and useful example of a class decorator. It (re)implements 
>> __str__ and __unicode__ in case Python 2 is used. For Python 3, the 
>> decorator does nothing. I wanted to generalize this decorator so the 
>> __str__ method under Python 2 encodes the string to an arbitrary 
>> encoding. This is what I've created: http://pastebin.com/vghD1bVJ.
>> 
>> It works, but the code is not very easy to understand, I am affraid. 
>
>It's easy to understand, it's just doing it the wrong way. It creates 
>and subclass of your class, which it shouldn't do. 



Why not? Because it's an unusual coding pattern? Or is it ineffecient?
I subclassed because I needed the encoding value in the decorator.
But subclassing may indeed have been overkill.



Here's a better 
>approach: inject the appropriate methods into the class directly. Here's 
>a version for Python 3:
>
>
>def decorate(cls):
>    if '__str__' not in cls.__dict__:
>        # inject __str__ method
>        def __str__(self):
>            ...
>        cls.__str__ = __str__
>
>    if '__bytes__' not in cls.__dict__:
>        # like above
>
>    return cls
>
>
>This avoids overwriting __str__ if it is already defined, and likewise 
>for __bytes__.


Doesn't a class always have __str__ implementation?

>>> class Foo(object): pass
>>> f = Foo()
>>> f.__str__

>>> Foo.__str__



>>> 
>> Or is it? And I have no idea how to call the class Klass. Maybe 
>> reimplements_Test? Is this a good approach, or can this be done in an 
>> easier way? I would *really* like keep statements "if 
>> sys.version_info[0] == 3..." separate from the "main" code. Also, 
>> learning about class decorators is cool ;-). So the code below... 
>> mehhh  no sir, I don't like it.

Btw, that was a quote: http://www.youtube.com/watch?v=dQ3acvz5LfI ;-)

>> 
>> 
>> def __str__(self):
>> 
>>     if sys.version_info[0] == 3:
>>         blah
>>     else:
>>         bleh
>>   
>> if sys.version_info[0] == 2:
>>     def __unicode__(self):
>>     blh
>
>
>That performs the version check every time the __str__ method is called. 


Good point.

>Better would be something like this:
>
>if sys.version_info[0] == 2:
>    def __str__(self):
>        ...
>
>    def __unicode__(self):
>        ...
>
>else:
>    def __bytes__(self):
>        ...
>
>    def __str__(self):
>        ...
>
>If you don't like repeating the code twice, once for version 2 and once 
>for version 3, you may be able to define the methods once, then rename 
>them, something like this:
>
># Assume version 3
>def __str__(self):
>    ...
>
>def __bytes__(self):
>    ...
>
>if sys.version_info[0] == 2:
>    __str__.__name__ = '__unicode__'
>    __bytes.__name__ = '__str__'
>    # Inject into the class, as above.
>    cls.__unicode__ = __str__
>    cls.__str__ = __bytes__
>
>else:
>    cls.__str__ = __str__
>    cls.__bytes__ = __bytes__
>
>
>Combining this with the decorator is left for you :-)

Nice, thanks Steven. I made a couple of versions after reading your advise. The 
main change that I still had to somehow retrieve the encoding value from the 
class to be decorated (decoratee?). I simply stored it in __dict__. Here is the 
second version that I created: http://pastebin.com/te3Ap50C. I tested it in 
Python 2 and 3. The Test class contains __str__ and __unicode__ which are 
renamed and redefined by the decorator if Python 3 (or 4, or..) is used.


General question: I am using pastebin now. Is that okay, given that this is not 
part of the "memory" of the Python Tutor archive? It might be annoying if 
people search the archives and get 404s if they try to follow these links. Just 
in case I am also pasting the code below:

from __future__ import print_function
import sys
    
def decorate(cls):
    print("decorate called")
    if sys.version_info[0] > 2:
    cls.__dict__["__str__"].__name__ = '__bytes__'
    cls.__dict__["__unicode__"].__name__ = '__str__'
    cls.__bytes__ = cls.__dict__["__str__"]
    cls.__str__ = cls.__dict__["__unicode__"]  
    return cls

@decorate
cl

Re: [Tutor] class decorator question

[Steven D'Aprano]

On Sat, Oct 05, 2013 at 05:33:46AM -0700, Albert-Jan Roskam wrote:
> Hi,
> 
> On http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ I saw 
> a very cool and useful example of a class decorator. It (re)implements 
> __str__ and __unicode__ in case Python 2 is used. For Python 3, the 
> decorator does nothing. I wanted to generalize this decorator so the 
> __str__ method under Python 2 encodes the string to an arbitrary 
> encoding. This is what I've created: http://pastebin.com/vghD1bVJ.
> 
> It works, but the code is not very easy to understand, I am affraid. 

It's easy to understand, it's just doing it the wrong way. It creates 
and subclass of your class, which it shouldn't do. Here's a better 
approach: inject the appropriate methods into the class directly. Here's 
a version for Python 3:


def decorate(cls):
if '__str__' not in cls.__dict__:
# inject __str__ method
def __str__(self):
...
cls.__str__ = __str__

if '__bytes__' not in cls.__dict__:
# like above

return cls


This avoids overwriting __str__ if it is already defined, and likewise 
for __bytes__.


> Or is it? And I have no idea how to call the class Klass. Maybe 
> reimplements_Test? Is this a good approach, or can this be done in an 
> easier way? I would *really* like keep statements "if 
> sys.version_info[0] == 3..." separate from the "main" code. Also, 
> learning about class decorators is cool ;-). So the code below... 
> mehhh  no sir, I don't like it.
> 
> 
> def __str__(self):
> 
>     if sys.version_info[0] == 3:
>         blah
>     else:
>         bleh
>   
> if sys.version_info[0] == 2:
>     def __unicode__(self):
>     blh


That performs the version check every time the __str__ method is called. 
Better would be something like this:

if sys.version_info[0] == 2:
def __str__(self):
...

def __unicode__(self):
...

else:
def __bytes__(self):
...

def __str__(self):
...



If you don't like repeating the code twice, once for version 2 and once 
for version 3, you may be able to define the methods once, then rename 
them, something like this:


# Assume version 3
def __str__(self):
...

def __bytes__(self):
...

if sys.version_info[0] == 2:
__str__.__name__ = '__unicode__'
__bytes.__name__ = '__str__'
# Inject into the class, as above.
cls.__unicode__ = __str__
cls.__str__ = __bytes__

else:
cls.__str__ = __str__
cls.__bytes__ = __bytes__


Combining this with the decorator is left for you :-)



-- 
Steven
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor