Re: __slots__
Op 2006-03-25, John J. Lee schreef [EMAIL PROTECTED]: [EMAIL PROTECTED] (Alex Martelli) writes: [...] you should be using pychecker or pylint [...] I'm curious, as somebody who doesn't regularly use these tools: How do they fit into your workflow? Do you run them every few hours, every day, every time you run functional tests, every release, every so often, on gut feeling about when it's likely to catch problems...? How do you cope with spurious warnings? Does it involve tweaking code to quell warnings? Keeping suppression lists? Maintaining special invocations of pychecker / pylint per-project? Do they cope well with other people's code who do not use these tools? IMO they don't cope well unless you adapt to what they expect. I just tested the two. pylint gave me countless warnings because I use two space to indent instead of four. It also didn't like that I used uppercase letters in my identifiers. That there was no space after a comma or around an operator was a problem too. That I use the same name for the index variable of for loops in different scopes is also warned about. Pychecker also warns about these shadowed variable. Pychecker goes even so far to warn you that a local variable in an imported module is shadowing a global variable of yours. Like the following line: /usr/lib/python2.3/random.py:247: Local variable (i) shadows global defined on line 43 in file demo1a.py So it seems pychecker thinks I shouldn't use something like for i in lst: ... in my code at the global level because some module in the standard library has a function with a local i. Pychecker also froze on my system. I don't recommend the use of these tools. -- Antoon Pardon -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__
Antoon Pardon wrote: Op 2006-03-25, John J. Lee schreef [EMAIL PROTECTED]: [EMAIL PROTECTED] (Alex Martelli) writes: [...] you should be using pychecker or pylint [...] I'm curious, as somebody who doesn't regularly use these tools: How do they fit into your workflow? Do you run them every few hours, every day, every time you run functional tests, every release, every so often, on gut feeling about when it's likely to catch problems...? How do you cope with spurious warnings? Does it involve tweaking code to quell warnings? Keeping suppression lists? Maintaining special invocations of pychecker / pylint per-project? Do they cope well with other people's code who do not use these tools? IMO they don't cope well unless you adapt to what they expect. I just tested the two. pylint gave me countless warnings because I use two space to indent instead of four. It also didn't like that I used uppercase letters in my identifiers. That there was no space after a comma or around an operator was a problem too. Pylint knows different types of issues, these are code style issues and can be - turned off - configured to what you like. They help getting your coding style consistent and default to PEP8 style. That I use the same name for the index variable of for loops in different scopes is also warned about. Well, everything that _could_ be problematic _or_ confuse a reader is warned about. Pretty good in my eyes. Pychecker also warns about these shadowed variable. Pychecker goes even so far to warn you that a local variable in an imported module is shadowing a global variable of yours. Like the following line: /usr/lib/python2.3/random.py:247: Local variable (i) shadows global defined on line 43 in file demo1a.py That's odd. How are the two modules related? So it seems pychecker thinks I shouldn't use something like for i in lst: ... in my code at the global level because some module in the standard library has a function with a local i. Pychecker also froze on my system. Pychecker imports the modules. Thus these things can happen when a module expects not to be imported as-is. I don't recommend the use of these tools. Well, then I don't recommend anyone reading your code wink Georg -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__
Op 2006-03-27, Georg Brandl schreef [EMAIL PROTECTED]: Antoon Pardon wrote: Op 2006-03-25, John J. Lee schreef [EMAIL PROTECTED]: [EMAIL PROTECTED] (Alex Martelli) writes: [...] you should be using pychecker or pylint [...] I'm curious, as somebody who doesn't regularly use these tools: How do they fit into your workflow? Do you run them every few hours, every day, every time you run functional tests, every release, every so often, on gut feeling about when it's likely to catch problems...? How do you cope with spurious warnings? Does it involve tweaking code to quell warnings? Keeping suppression lists? Maintaining special invocations of pychecker / pylint per-project? Do they cope well with other people's code who do not use these tools? IMO they don't cope well unless you adapt to what they expect. I just tested the two. pylint gave me countless warnings because I use two space to indent instead of four. It also didn't like that I used uppercase letters in my identifiers. That there was no space after a comma or around an operator was a problem too. Pylint knows different types of issues, these are code style issues and can be - turned off - configured to what you like. They help getting your coding style consistent and default to PEP8 style. I have been on their website in the documentation section and found nothing that could easily help me here. That I use the same name for the index variable of for loops in different scopes is also warned about. Well, everything that _could_ be problematic _or_ confuse a reader is warned about. Pretty good in my eyes. No it isn't. If most of these line are in fact not problematic or confusing at all then people will in general no longer take notice of those kind of lines. If I use a specific variable like i only as an index in for loops, there is no reason to think this is problematic or confusing even if those loops are in different scopes. And all those lines reporting something that is not a problem will detract from the few lines that may be a real problem. Pychecker also warns about these shadowed variable. Pychecker goes even so far to warn you that a local variable in an imported module is shadowing a global variable of yours. Like the following line: /usr/lib/python2.3/random.py:247: Local variable (i) shadows global defined on line 43 in file demo1a.py That's odd. How are the two modules related? the program demo1a.py contains the following line: from random import Random, randint, sample So it seems pychecker thinks I shouldn't use something like for i in lst: ... in my code at the global level because some module in the standard library has a function with a local i. Pychecker also froze on my system. Pychecker imports the modules. Thus these things can happen when a module expects not to be imported as-is. I still find it unacceptable. -- Antoon Pardon -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__
David wrote: 3. What is a simple example of a Pythonic use of __slots__ that does NOT involved the creation of **many** instances. mu. Your question presupposes the existence of such an example. -Mike -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__
Aahz [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Because __slots__ breaks with inheritance. I believe that was the point of Ziga's example, which I acknowledged as a good one in my reply. So there still appears to be this single reason, which applies if your class may be subclassed. Does this beg the question of whether __slots__ *should* break with inheritance? One other question I did not get answered: is there any simple example of a Pythonic use of __slots__ that does NOT involve the creation of **many** instances. Thanks, Alan Isaac -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__
David Isaac [EMAIL PROTECTED] wrote: ... Does this beg the question of whether __slots__ *should* break with inheritance? How would you expect the following code to behave: class Base(object): def __init__(self): self.x = 23 class Derived(Base): __slots__ = 'y', ? I would expect it to work (as it does in Python), therefore I think it's fit and proper that __slots__ breaks with inheritance, meaning it basically has no effect unless every class in the inheritance DAG has slots. One other question I did not get answered: is there any simple example of a Pythonic use of __slots__ that does NOT involve the creation of **many** instances. Since the only benefit of __slots__ is saving a few bytes per instance, it's not worth the bother unless there are many instances -- so, the answer is 'no'. Alex -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__
In article [EMAIL PROTECTED], [EMAIL PROTECTED] (Alex Martelli) wrote: One other question I did not get answered: is there any simple example of a Pythonic use of __slots__ that does NOT involve the creation of **many** instances. Since the only benefit of __slots__ is saving a few bytes per instance, it's not worth the bother unless there are many instances -- so, the answer is 'no'. I can think of at least two other benefits to using __slots__: 1. If you have a typo in an attribute assignment you get an exception instead of a latent downstream bug. 2. Implicit documentation. rg -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__
Ron Garret [EMAIL PROTECTED] wrote: In article [EMAIL PROTECTED], [EMAIL PROTECTED] (Alex Martelli) wrote: One other question I did not get answered: is there any simple example of a Pythonic use of __slots__ that does NOT involve the creation of **many** instances. Since the only benefit of __slots__ is saving a few bytes per instance, it's not worth the bother unless there are many instances -- so, the answer is 'no'. I can think of at least two other benefits to using __slots__: 1. If you have a typo in an attribute assignment you get an exception instead of a latent downstream bug. If your unittests are so feeble that they won't catch such typos, you have far bigger problems -- and you should be using pychecker or pylint anyway, as they'll catch far more typos than __slots__ ever will (far from all, of course -- a simple typo of + vs - can still kill you -- which is why they can't *substitute* for unittests in any case). 2. Implicit documentation. If you don't document what the sundry variables are FOR, you're really not documenting your code at all -- just listing the names of some attributes is far too weak. If the existence of such listing can in any way give the programmer an excuse to NOT do real documentation (as your classifying it as implicit documentation strongly suggests), then the net effet is not a benefit, but a serious detriment to code quality. Alex -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__
Ron Garret wrote: [EMAIL PROTECTED] (Alex Martelli) wrote: One other question I did not get answered: is there any simple example of a Pythonic use of __slots__ that does NOT involve the creation of **many** instances. Since the only benefit of __slots__ is saving a few bytes per instance, it's not worth the bother unless there are many instances -- so, the answer is 'no'. I can think of at least two other benefits to using __slots__: 1. If you have a typo in an attribute assignment you get an exception instead of a latent downstream bug. Only if all classes in the inheritance define __slots__, and don't include __dict__ in their slots. In particular, it means you can never rely on this in any code you write which inherits from a library class over which you have no control. In other words, it is such a risky thing to depend on that you would be much better never to rely on it. Try writing some unit tests instead. 2. Implicit documentation. Explicit is better than implicit. -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__
[EMAIL PROTECTED] (Alex Martelli) writes: [...] you should be using pychecker or pylint [...] I'm curious, as somebody who doesn't regularly use these tools: How do they fit into your workflow? Do you run them every few hours, every day, every time you run functional tests, every release, every so often, on gut feeling about when it's likely to catch problems...? How do you cope with spurious warnings? Does it involve tweaking code to quell warnings? Keeping suppression lists? Maintaining special invocations of pychecker / pylint per-project? Do they cope well with other people's code who do not use these tools? John -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__
John J. Lee [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] (Alex Martelli) writes: [...] you should be using pychecker or pylint [...] I'm curious, as somebody who doesn't regularly use these tools: How do they fit into your workflow? Do you run them every few hours, every day, every time you run functional tests, every release, every so often, on gut feeling about when it's likely to catch problems...? I use them before mailing code off for review (which in our workflow is mandatory before the code is submitted into the codebase) -- not quite as often as unittests, which I run with just about every keystroke in my editor;-). I'd rather have such a tool run automatically, as a presubmit check, but that depends on convincing every colleague that it's worth it (probably hopeless for pychecker as long as it needs to i How do you cope with spurious warnings? Does it involve tweaking code to quell warnings? Keeping suppression lists? Maintaining special invocations of pychecker / pylint per-project? Do they cope well with other people's code who do not use these tools? If said other people's code, for example, does pointless (but hopefully innocuous, otherwise unittests would have caught that;-) things such as importing modules it never uses, the warning tools complain. I do not consider such complaints spurious, and prefer to fix those things (sending the brief codereview to the original author with advice to start using the checking tools -- it usually works;-). Very occasionally, there _are_ warnings which are indeed spurious: the typical example is a function which, to comply with a certain API, MUST take three arguments named exactly x, y, and z (no more, no less, not with any different names) but only needs x; in such cases, you do need to add, e.g., __pychecker__ = 'unusednames=y,z' I pay the price gladly, since this will also serve to reassure human readers that my code's anomalous behavior (accepting arguments y and z but ignoring them) is by design, and not an accident or mistake. I do just about the same thing with the lint variant we use for C++, btw. Alex -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__
John J. Lee wrote: [EMAIL PROTECTED] (Alex Martelli) writes: you should be using pychecker or pylint I'm curious, as somebody who doesn't regularly use these tools: How do they fit into your workflow? Do you run them every few hours, every day, every time you run functional tests, every release, every so often, on gut feeling about when it's likely to catch problems...? I use pychecker when I have finished a chunk of work that is supposed to be syntactically correct, but before I try any unit testing. Think of it as an optional step that takes the place of running the compiler in statically-typed language development. I use pytchecker again before I release anything, just to be sure. How do you cope with spurious warnings? Does it involve tweaking code to quell warnings? Keeping suppression lists? Suppression lists work for me. From http://pychecker.sourceforge.net/ *** begin quote *** You can also define suppressions in your code by doing: __pychecker__ = 'no-namedargs maxreturns=0 unusednames=foo,bar' *** end quote *** Do they cope well with other people's code who do not use these tools? pychecker complains a lot when used on code that is poorly written and includes bad practices such as using builtin function names as local variable names. I consider pychecker's complaints in these cases to be a *feature*. -- Ben Caradoc-Davies [EMAIL PROTECTED] http://wintersun.org/ Those who deny freedom to others deserve it not for themselves. - Abraham Lincoln -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__
In article [EMAIL PROTECTED], David Isaac [EMAIL PROTECTED] wrote: Ziga Seilnacht [EMAIL PROTECTED] wrote: If you want to restrict attribute asignment, you should use the __setattr__ special method, see: http://docs.python.org/ref/attribute-access.html That should is what I am asking about. If I understand, in the simplest case, you want me to say something like def __setattr__(self,name,value): if name in myattrlist: object.__setattr__(self,name,value) else: raise AttributeError instead just saying __slots__ = myattrlist I understand that this *is* the prevailing advice. But why? Because __slots__ breaks with inheritance. -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ Look, it's your affair if you want to play with five people, but don't go calling it doubles. --John Cleese anticipates Usenet -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__
David Isaac wrote:
1. Without a __dict__ variable,
instances cannot be assigned new variables not listed in the __slots__
definition.
So this seemed an interesting restriction to impose in some instances,
but I've noticed that this behavior is being called by some a side effect
the reliance on which is considered unPythonic. Why?
If you want to restrict attribute asignment, you should use the
__setattr__ special method, see:
http://docs.python.org/ref/attribute-access.html
2. What is a simple example where use of slots has caused subtle problems,
as some claim it will?
The first point is true only if all bases use __slots__:
class A(object):
... pass
...
class B(A):
... __slots__ = ('spam',)
...
b = B()
b.eggs = 1
b.eggs
1
3. What is a simple example of a Pythonic use of __slots__ that does NOT
involved the creation of **many** instances.
Thanks,
Alan Isaac
Ziga
--
http://mail.python.org/mailman/listinfo/python-list
Re: __slots__
Ziga Seilnacht [EMAIL PROTECTED] wrote:
If you want to restrict attribute asignment, you should use the
__setattr__ special method, see:
http://docs.python.org/ref/attribute-access.html
That should is what I am asking about. If I understand,
in the simplest case, you want me to say something like
def __setattr__(self,name,value):
if name in myattrlist:
object.__setattr__(self,name,value)
else:
raise AttributeError
instead just saying
__slots__ = myattrlist
I understand that this *is* the prevailing advice. But why?
class A(object):
... pass
...
class B(A):
... __slots__ = ('spam',)
...
b = B()
b.eggs = 1
b.eggs
1
A good example of something that could be easily
missed, and possibly an answer to my question above.
(Although not I think if I am subclassing object.)
Thanks,
Alan Isaac
--
http://mail.python.org/mailman/listinfo/python-list
Re: __slots__ in derived class
Schüle Daniel wrote: Hello, consider this code class A(object): ... def __init__(self): ... self.a = 1 ... self.b = 2 ... class B(A): ... __slots__ = [x,y] ... b=B() b.a 1 b.b 2 b.x = 100 b.y = 100 b.z = 100 no exception here does __slots__ nothing when used in derived classes? class Z(object): ... __slots__ = [x,y] ... z=Z() z.x = 100 z.y = 100 z.z = 100 Traceback (most recent call last): File stdin, line 1, in ? AttributeError: 'Z' object has no attribute 'z' here it works like expected Regards, Daniel I would expect that A has to define its own __slots__ too. The following code should work as expected and makes also sense with the memory optimization considerations that motivated introduction of the __slots__ variable. class A(object): __slots__ = [a,b] def __init__(self): self.a = 1 self.b = 2 class B(A): __slots__ = [x,y] Kay -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__ in derived class
Schüle Daniel wrote: consider this code class A(object): ... def __init__(self): ... self.a = 1 ... self.b = 2 ... class B(A): ... __slots__ = [x,y] ... b=B() b.a 1 b.b 2 b.x = 100 b.y = 100 b.z = 100 no exception here does __slots__ nothing when used in derived classes? __slots__ is intended as a way to reduce memory consumption. It was never intended as a protection mechanism. The slots which are available in a class only add to the attributes available in the base class. You can hide base class slots by defining a slot of the same name, but you cannot remove them. Your base class has a __dict__ attribute and therefore all instances of the base class or any derived classes also have a __dict__ attribute. -- http://mail.python.org/mailman/listinfo/python-list
Don't use __slots__ (was Re: __slots__ in derived class)
In article [EMAIL PROTECTED], =?ISO-8859-1?Q?Sch=FCle_Daniel?= [EMAIL PROTECTED] wrote: does __slots__ nothing when used in derived classes? Short answer: don't use __slots__ until you're comfortable writing metaclasses and decorators. __slots__ are a performance hack strictly for advanced users, and if you think you need them, you probably don't. -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ 19. A language that doesn't affect the way you think about programming, is not worth knowing. --Alan Perlis -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__ and copy again: why does it work?
I should've mentioned this was tested on Python 2.4.2. fortepianissimo wrote: I remember from painful experience that copy.copy() won't really copy __slots__ members. But I have trouble explaning why the following code works: --- START--- #!/usr/bin/env python import copy class Foo (object): __slots__ = 'i' def __init__ (self): self.i = 10 class Bar (Foo): __slots__ = 'j' def __init__ (self): self.j = 20 f1 = Foo() f2 = copy.copy(f1) print f2.i # why does it work? b1 = Bar() b2 = copy.copy(b1) print b2.j # why does it work? print b2.i # doesn't work, as expected --- END--- Any insight is welcome! -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__ and copy again: why does it work?
More weird observations: the following code does not work until you change the name of the member 'longer' to a one-char name, for example, 'j': --- START --- #!/usr/bin/env python import copy class Foo (object): __slots__ = 'i' class Bar (Foo): __slots__ = 'longer' #__slots__ = 'j' b1 = Bar() b1.longer = 22 #b1.j = 22 b2 = copy.copy(b1) # doesn't work in Python 2.4.2 # BUT if 'longer' is changed to 'j' in the entire file, then it works! print b2.longer #print b2.j --- END --- I've tried different names and concluded that as long as I used one character the code works. Anything longer than one character bombs. Why? -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__ and copy again: why does it work?
Mystery solved - when there's only one slot I should've used __slots__
= ('i', ). Duh!
So in short, __slots__ and copy.copy() work fine in Python 2.4.2.
--
http://mail.python.org/mailman/listinfo/python-list
Re: __slots__ and copy again: why does it work?
26 Dec 2005 20:33:35 -0800, fortepianissimo [EMAIL PROTECTED]:
Mystery solved - when there's only one slot I should've used __slots__
= ('i', ). Duh!
So in short, __slots__ and copy.copy() work fine in Python 2.4.2.
Hard works, but very useful :)
--
I like python!
My Blog: http://www.donews.net/limodou
NewEdit Maillist: http://groups.google.com/group/NewEdit
--
http://mail.python.org/mailman/listinfo/python-list
Re: __slots__ and copy again: why does it work?
To be complete, the first code snippet, when modified as follows, works
fine in Python 2.4.2:
--- START ---
#!/usr/bin/env python
import copy
class Foo (object):
__slots__ = ('i', )
def __init__ (self):
self.i = 10
class Bar (Foo):
__slots__ = ('j', )
def __init__ (self):
super(Bar, self).__init__()
self.j = 20
f1 = Foo()
f2 = copy.copy(f1)
print f2.i # works
b1 = Bar()
b2 = copy.copy(b1)
print b2.j # works
print b2.i # works
--- END ---
The last line didn't work last time because b2.i was never initialized.
--
http://mail.python.org/mailman/listinfo/python-list
Re: __slots__ and class attributes
Steven Bethard wrote: But why do you want a class level attribute with the same name as an instance level attribute? I would have written your class as: class A(object): __slots__ = ['value'] def __init__(self, value=1): self.value = value where the default value you put in the class is simply expressed as a default value to the __init__ parameter. Thanks for your explanation. The reason why I was doing it was to have class-level defaults, so that one can easily adjust how new instances will be made. I'm doing it now with capitilized class attribute names to avoid the name clash. -- -- Ewald -- http://mail.python.org/mailman/listinfo/python-list
Re: __slots__ and class attributes
Ewald R. de Wit wrote: I'm running into a something unexpected for a new-style class that has both a class attribute and __slots__ defined. If the name of the class attribute also exists in __slots__, Python throws an AttributeError. Is this by design (if so, why)? class A( object ): __slots__ = ( 'value', ) value = 1 def __init__( self, value = None ): self.value = value or A.value a = A() print a.value Traceback (most recent call last): File t1.py, line 8, in ? a = A() File t1.py, line 6, in __init__ self.value = value or A.value AttributeError: 'A' object attribute 'value' is read-only Check the documentation on __slots__[1]: __slots__ are implemented at the class level by creating descriptors (3.3.2) for each variable name. As a result, class attributes cannot be used to set default values for instance variables defined by __slots__; otherwise, the class attribute would overwrite the descriptor assignment. I agree that the error you get is a bit confusing. I think this has to do with how the descriptor machinery works. When you write something like a.value where a is a class instance, Python tries to invoke something like: type(a).value.__get__(a) Here's an example of that, working normallly: py class A(object): ... __slots__ = ['value'] ... def __init__(self): ... self.value = 1 ... py a = A() py type(a).value member 'value' of 'A' objects py type(a).value.__get__ method-wrapper object at 0x0129A1B0 py type(a).value.__get__(a) 1 Now when you add a class attribute called 'value', you overwrite the descriptor. So when Python tries to do the same thing (because your definition of __slots__ makes it assume that 'value' is a descriptor), the descriptor machinery raises an AttributeError: py class A(object): ... __slots__ = ['value'] ... value = 1 ... py a = A() py type(a).value 1 py type(a).value.__get__ Traceback (most recent call last): File interactive input, line 1, in ? AttributeError: 'int' object has no attribute '__get__' This AttributeError must be somehow caught by the __slots__ machinery and interpreted to mean that you tried to write to a read-only attribute. The resulting error message is probably not what you want, but I don't know the source well enough to figure out whether or not a better error message could be given. But why do you want a class level attribute with the same name as an instance level attribute? I would have written your class as: class A(object): __slots__ = ['value'] def __init__(self, value=1): self.value = value where the default value you put in the class is simply expressed as a default value to the __init__ parameter. Steve [1]http://docs.python.org/ref/slots.html -- http://mail.python.org/mailman/listinfo/python-list
