Re: Pylint false positives
On Wed, Aug 22, 2018 at 10:39 AM, Steven D'Aprano wrote: > On Wed, 22 Aug 2018 03:58:29 +1000, Chris Angelico wrote: > >> On Wed, Aug 22, 2018 at 2:38 AM, Marko Rauhamaa >> wrote: >>> Gregory Ewing : >>> Marko Rauhamaa wrote: > Lexically, there is special access: > >class C: >def __init__(self, some, arg): >c = self >class D: >def method(self): >access(c) >access(some) >access(arg) [...] you can do that without creating a new class every time you want an instance. You just have to be *slightly* more explicit about the link between the inner and outer instances. >>> >>> By "*slightly* more explicit," do you mean more syntactic clutter? >>> >>> >> No, he actually means "explicit" in the normal English sense. You're >> trying to use it in the python-ideas sense of "code that I like", and >> since you don't like it, you want to call it "implicit" instead, but it >> obviously isn't that, so you call it "syntactic clutter". > > That's an incredible insight into Marko's internal mental state you have > there. And you get that all from the words "syntactic clutter"? I thought > he just meant that it was cluttered code. How naive was that? > > *wink* > Hah. I was just picking up on a few specific parts of the conversation: 1) It's possible to create a class with a number of methods, then instantiate that class, and have all of its methods be closures. Lexical containment grants implicit access to the variable "c". 2) It's also possible to create the class at a broader scope, then instantiate that class, passing it a reference to whatever is in "c". 3) Passing an explicit reference is, if I'm understanding Marko's scoffing correctly, "syntactic clutter". 4) Clutter is, by implication, unnecessary. In simple assignment "x = 1", we don't consider the equals sign to be "clutter". My point is that passing the reference truly is "explicit" and relying on lexical containment truly is "implicit", unlike the usual uses of those words. Maybe I misjudged Marko's internal mental state. All I have is his outputs, so it's like trying to deduce Mersenne's internal state from its outputs - not fundamentally impossible, but not particularly easy either. But if I have, he is, as always, most welcome to correct me. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Wed, 22 Aug 2018 03:58:29 +1000, Chris Angelico wrote: > On Wed, Aug 22, 2018 at 2:38 AM, Marko Rauhamaa > wrote: >> Gregory Ewing : >> >>> Marko Rauhamaa wrote: Lexically, there is special access: class C: def __init__(self, some, arg): c = self class D: def method(self): access(c) access(some) access(arg) >>> >>> [...] >>> >>> you can do that without creating a new class every time you want an >>> instance. You just have to be *slightly* more explicit about the link >>> between the inner and outer instances. >> >> By "*slightly* more explicit," do you mean more syntactic clutter? >> >> > No, he actually means "explicit" in the normal English sense. You're > trying to use it in the python-ideas sense of "code that I like", and > since you don't like it, you want to call it "implicit" instead, but it > obviously isn't that, so you call it "syntactic clutter". That's an incredible insight into Marko's internal mental state you have there. And you get that all from the words "syntactic clutter"? I thought he just meant that it was cluttered code. How naive was that? *wink* > But this is actually a case of explicit vs implicit. To be honest, I don't even understand Greg's comment. With no inner class, what is this "inner instance" he refers to here? "you can do that without creating a new class every time you want an instance. You just have to be *slightly* more explicit about the link between the inner and outer instances." Marko wants to use closures. So how do you close over per-instance variables if you create the closures before the instances are created? If we only needed *one* function, there would be no problem: class Outer: def __init__(self, some, arg): c = self def closure(): access(c) access(some) access(arg) # then do something useful with closure But as soon as you have a lot of them, its natural to want to wrap them up in a namespace, and the only solution we have for that is to use a class. Its a truism that anything you can do with a closure, you can do with a class (or vise versa) so I dare say there are alternative designs which avoids closures altogether but we don't know the full requirements here and its hard to judge from the outside on why Marko picked the design he has and whether its a good idea. It could be a case of "ooh, closures are a shiny new hammer, this problem must be a nail!" but let's give him the benefit of the doubt and assume he has good reasons, not just reasons. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Wed, Aug 22, 2018 at 2:38 AM, Marko Rauhamaa wrote: > Gregory Ewing : > >> Marko Rauhamaa wrote: >>> Lexically, there is special access: >>> >>>class C: >>>def __init__(self, some, arg): >>>c = self >>>class D: >>>def method(self): >>>access(c) >>>access(some) >>>access(arg) >> >> [...] >> >> you can do that without creating a new class every time you want an >> instance. You just have to be *slightly* more explicit about the link >> between the inner and outer instances. > > By "*slightly* more explicit," do you mean more syntactic clutter? > No, he actually means "explicit" in the normal English sense. You're trying to use it in the python-ideas sense of "code that I like", and since you don't like it, you want to call it "implicit" instead, but it obviously isn't that, so you call it "syntactic clutter". But this is actually a case of explicit vs implicit. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Gregory Ewing : > Marko Rauhamaa wrote: >> Lexically, there is special access: >> >>class C: >>def __init__(self, some, arg): >>c = self >>class D: >>def method(self): >>access(c) >>access(some) >>access(arg) > > [...] > > you can do that without creating a new class every time you want an > instance. You just have to be *slightly* more explicit about the link > between the inner and outer instances. By "*slightly* more explicit," do you mean more syntactic clutter? Because of course you replace inner classes and closures with top-level classes and methods of top-level classes. And of course, I would prefer not to create a class for a singleton object: class C: def __init__(self, some, arg): c = self self.d = object: def method(self): access(c) access(some) access(arg) Unfortunately, there is no such syntax in Python. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Tue, 21 Aug 2018 00:36:56 +, Dan Sommers wrote: [...] (Not that I do this using "inner classes", but I do often want to use a class as a container for functions, without caring about "self" or wrapping everything in staticmethod.) >>> >>> Isn't that what modules are for? (I suspect that I'm missing >>> something, because I also suspect that you knew/know that.) >> >> What's the syntax for creating an inner module...? > > Why does it have to be an inner anything? An ordinary, top-level, > "outer" module is a perfectly good "container for functions, without > caring about "self."" And what if you want to subdivide those functions (or other objects) into categories that are finer than the module, without introducing a package structure? We can design the structure of our program into *outward* hierarchies, by adding packages with subpackages and sub-subpackages: import spam.eggs.cheese.tomato.aardvark So using the file system and packages, we can logically nest modules inside modules inside modules 'til the cows come home. But that's a fairly heavyweight solution, in the sense that it requires separate directory for each level of the hierarchy. Sometimes a package is too much. I want a single module file, but still want to pull out a collection of related functions and other objects and put them in their own namespace, but without creating a new module. The Zen says: Namespaces are one honking great idea -- let's do more of those! but Python's namespaces are relatively impoverished. We have packages, modules, classes and instances, and that's it. Classes and instances come with inheritance, self etc which is great if you want a class, but if you just want a simple module-like namespace without the extra file, classes are a pretty poor alternative. But they're all we've got. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Mon, 20 Aug 2018 22:55:26 +0300, Marko Rauhamaa wrote: > Dan Sommers : > >> On Mon, 20 Aug 2018 14:39:38 +, Steven D'Aprano wrote: >>> I have often wished Python had proper namespaces, so I didn't have to >>> abuse classes as containers in this way :-( >>> >>> (Not that I do this using "inner classes", but I do often want to use >>> a class as a container for functions, without caring about "self" or >>> wrapping everything in staticmethod.) >> >> Isn't that what modules are for? (I suspect that I'm missing something, >> because I also suspect that you knew/know that.) > > What's the syntax for creating an inner module...? Why does it have to be an inner anything? An ordinary, top-level, "outer" module is a perfectly good "container for functions, without caring about "self."" The Python "math" module, for example. As a long time, non-native speaker of Object Oriented, I must admit that I still don't understand the obsession with classes (a certain level of usefulness and arguably appropriateness for some kinds of problems, sure, but not the obsession). Dan -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Mon, 20 Aug 2018 22:55:26 +0300, Marko Rauhamaa wrote:
> Dan Sommers :
>
>> On Mon, 20 Aug 2018 14:39:38 +, Steven D'Aprano wrote:
>>> I have often wished Python had proper namespaces, so I didn't have to
>>> abuse classes as containers in this way :-(
>>>
>>> (Not that I do this using "inner classes", but I do often want to use
>>> a class as a container for functions, without caring about "self" or
>>> wrapping everything in staticmethod.)
>>
>> Isn't that what modules are for? (I suspect that I'm missing
>> something, because I also suspect that you knew/know that.)
>
> What's the syntax for creating an inner module...?
from types import ModuleType
m = ModuleType('m')
m.one = 1
m.a = 'a'
m.b = lambda x: x + one
except that not only doesn't it look nice, but it doesn't work because
the m.b function doesn't pick up the m.one variable, but a global
variable instead.
--
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson
--
https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Marko Rauhamaa wrote: Lexically, there is special access: class C: def __init__(self, some, arg): c = self class D: def method(self): access(c) access(some) access(arg) That's only because the definition of method() is lexically inside the definition of __init__(). It has nothing to do with nesting of the *class* statements. Inner classes in Java have some special magic going on that doesn't happen with nested classes in Python. the reason to use a class is that there is no handier way to create a method dispatch or a singleton object. That's perfectly fine, but you can do that without creating a new class every time you want an instance. You just have to be *slightly* more explicit about the link between the inner and outer instances. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Dan Sommers : > On Mon, 20 Aug 2018 14:39:38 +, Steven D'Aprano wrote: >> I have often wished Python had proper namespaces, so I didn't have to >> abuse classes as containers in this way :-( >> >> (Not that I do this using "inner classes", but I do often want to use >> a class as a container for functions, without caring about "self" or >> wrapping everything in staticmethod.) > > Isn't that what modules are for? (I suspect that I'm missing something, > because I also suspect that you knew/know that.) What's the syntax for creating an inner module...? Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Steven D'Aprano : > On Mon, 20 Aug 2018 11:40:16 +0300, Marko Rauhamaa wrote: > >>class C: >>def __init__(self, some, arg): >>c = self >>class D: >>def method(self): >>access(c) >>access(some) >>access(arg) >> >> IOW, inner class D is a container for a group of interlinked closure >> functions. > > If a class' methods don't use self, it probably shouldn't be a class. > > I have often wished Python had proper namespaces, so I didn't have to > abuse classes as containers in this way :-( > > (Not that I do this using "inner classes", but I do often want to use a > class as a container for functions, without caring about "self" or > wrapping everything in staticmethod.) Yes, the reason to use a class is that there is no handier way to create a method dispatch or a singleton object. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Tue, Aug 21, 2018 at 2:12 AM, Dan Sommers wrote: > On Mon, 20 Aug 2018 14:39:38 +, Steven D'Aprano wrote: > >> If a class' methods don't use self, it probably shouldn't be a class. > > Agreed. > >> I have often wished Python had proper namespaces, so I didn't have to >> abuse classes as containers in this way :-( >> >> (Not that I do this using "inner classes", but I do often want to use >> a class as a container for functions, without caring about "self" or >> wrapping everything in staticmethod.) > > Isn't that what modules are for? (I suspect that I'm missing something, > because I also suspect that you knew/know that.) Yes, except that sometimes you want submodules without going for a package with modules inside it. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Mon, 20 Aug 2018 14:39:38 +, Steven D'Aprano wrote: > If a class' methods don't use self, it probably shouldn't be a class. Agreed. > I have often wished Python had proper namespaces, so I didn't have to > abuse classes as containers in this way :-( > > (Not that I do this using "inner classes", but I do often want to use > a class as a container for functions, without caring about "self" or > wrapping everything in staticmethod.) Isn't that what modules are for? (I suspect that I'm missing something, because I also suspect that you knew/know that.) Dan -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Mon, 20 Aug 2018 11:40:16 +0300, Marko Rauhamaa wrote: >class C: >def __init__(self, some, arg): >c = self >class D: >def method(self): >access(c) >access(some) >access(arg) > > IOW, inner class D is a container for a group of interlinked closure > functions. If a class' methods don't use self, it probably shouldn't be a class. I have often wished Python had proper namespaces, so I didn't have to abuse classes as containers in this way :-( (Not that I do this using "inner classes", but I do often want to use a class as a container for functions, without caring about "self" or wrapping everything in staticmethod.) -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Mon, 20 Aug 2018 15:58:57 +0300, Marko Rauhamaa wrote: [...] >> The point is that creating a class object every time you want a closure >> is pointlessly wasteful. There is *no benefit whatsoever* in doing >> that. If you think there is, then it's probably because you're trying >> to write Java programs in Python. > > The benefit, as in using closures in general, is in the idiom. Yes, but you get closures by nesting functions, not by nesting classes. We don't *inherently* have to duplicate Java style nested classes in order to get the container of closures you mentioned earlier. We don't have to duplicate the idiom exactly, if it doesn't match the execution model of Python. On the other hand, there's no need to optimize this if it isn't critical code in your application. As I have often said, the question isn't whether Python is fast or not, but whether it is *fast enough*. If you are aware of the potential pitfalls, and the code is fast enough, and refactoring it to something faster and less pitfall-y is too difficult (or not a priority), then that's fine too. >>> But now I'm thinking the original Java approach (anonymous inner >>> classes) is probably the most versatile of them all. A single function >>> rarely captures behavior. That's the job of an object with its >>> multiple methods. In in order to create an ad-hoc object in Python, >>> you will need an ad-hoc class. >> >> An important difference between Python and Java here is that in Python >> the class statement is an *executable* statement, whereas in Java it's >> just a declaration. So putting a class statement inside a Python >> function incurs a large runtime overhead that you don't get with a Java >> inner class. > > The same is true for inner def statements. Indeed. Inner def statements are not very useful (in my opinion, although I believe Tim Peters disagrees) unless they are used as closures. The biggest problem with the idea of using inner functions in the Pascal sense is that you can't test them since they aren't visible from the outside. > I don't see how creating a class would be fundamentally slower to > execute than, say, adding two integers. Well, fundamentally adding two integers could be as quick as a single machine instruction to add two fixed-width ints. CPUs are pretty much optimized to do that *really quickly*. Creating a new class requires allocating a chunk of memory (about 500 bytes in Python), calling the appropriate metaclass, setting a bunch of fields, possibly even executing arbitrary metaclass methods. Its not as expensive as (say) listing the first trillion digits of pi but it surely is going to be more costly than adding two ints. [...] > Anyway, in practice on my laptop it takes 7 µs to execute a class > statement, which is clearly worse than executing a def statement (0.1 > µs) or integer addition (0.05 µs). However, 7 microseconds is the least > of my programming concerns. And fair enough... premature optimization and all that. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Gregory Ewing : > Marko Rauhamaa wrote: >> Some of these chores are heavier, some of them are lighter. But where >> I have used Python, performance hasn't been a bottleneck. It it were, >> I'd choose different approaches of implementation. > > The point is that creating a class object every time you want a > closure is pointlessly wasteful. There is *no benefit whatsoever* in > doing that. If you think there is, then it's probably because you're > trying to write Java programs in Python. The benefit, as in using closures in general, is in the idiom. >> But now I'm thinking the original Java approach (anonymous inner >> classes) is probably the most versatile of them all. A single >> function rarely captures behavior. That's the job of an object with >> its multiple methods. In in order to create an ad-hoc object in >> Python, you will need an ad-hoc class. > > An important difference between Python and Java here is that in Python > the class statement is an *executable* statement, whereas in Java it's > just a declaration. So putting a class statement inside a Python > function incurs a large runtime overhead that you don't get with a > Java inner class. The same is true for inner def statements. I don't see how creating a class would be fundamentally slower to execute than, say, adding two integers. It may be the CPython has not been optimized with the inner class use case. And it may be that Python's data model has painted CPython into a corner, which then would call the data model into question. Anyway, in practice on my laptop it takes 7 µs to execute a class statement, which is clearly worse than executing a def statement (0.1 µs) or integer addition (0.05 µs). However, 7 microseconds is the least of my programming concerns. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Marko Rauhamaa wrote: Some of these chores are heavier, some of them are lighter. But where I have used Python, performance hasn't been a bottleneck. It it were, I'd choose different approaches of implementation. The point is that creating a class object every time you want a closure is pointlessly wasteful. There is *no benefit whatsoever* in doing that. If you think there is, then it's probably because you're trying to write Java programs in Python. But now I'm thinking the original Java approach (anonymous inner classes) is probably the most versatile of them all. A single function rarely captures behavior. That's the job of an object with its multiple methods. In in order to create an ad-hoc object in Python, you will need an ad-hoc class. An important difference between Python and Java here is that in Python the class statement is an *executable* statement, whereas in Java it's just a declaration. So putting a class statement inside a Python function incurs a large runtime overhead that you don't get with a Java inner class. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Mon, Aug 20, 2018 at 7:05 PM, Gregory Ewing wrote: > Marko Rauhamaa wrote: >> >> Chris Angelico : >> >>> 3) Every invocation of method() has to execute the class body, which >>> takes time. >> >> >> That's what happens with every method invocation in Python regardless. > > > No, it doesn't! Invoking a method involves creating a bound method > object, which is very small and lightweight. Executing a class > statement creates a class object, which is enormous by comparison, > and quite expensive to initialise. Additionally, "creating a bound method object" is an action which can be optimized, or even completely optimized out. There have been proposals to peephole-optimize "x.method(arg)" to a single opcode, while still maintaining the proper semantics for "f = x.method; f(arg)", so the creation of the bound method would be skipped where it's unnecessary. (I think PyPy was looking at something like this? Not sure.) Even if the bound method is created, proper use of free lists can make it pretty efficient. Creating a new class object, on the other hand, MUST be performed sequentially, executing each of the statements inside it. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Marko Rauhamaa wrote:
Chris Angelico :
3) Every invocation of method() has to execute the class body, which
takes time.
That's what happens with every method invocation in Python regardless.
No, it doesn't! Invoking a method involves creating a bound method
object, which is very small and lightweight. Executing a class
statement creates a class object, which is enormous by comparison,
and quite expensive to initialise.
A quick test of your Outer class vs. Chris Angelico's version
suggests that yours is about 12 times slower at creating instances
of Inner.
from timeit import timeit
class Outer1:
def method(self):
outer = self
class Inner:
def spam(self, a, b):
outer.quarantine(a, b)
return Inner()
def quarantine(self, a, b):
pass
def test1():
x = Outer1()
for i in range(10):
y = x.method()
y.spam(1, 2)
class Outer2:
class Inner:
def __init__(self, outer):
self.outer = outer
def spam(self, a, b):
self.outer.quarantine(a, b)
def method(self):
return self.Inner(self)
def quarantine(self, a, b):
pass
def test2():
x = Outer2()
for i in range(10):
y = x.method()
y.spam(1, 2)
t1 = timeit(test1, number = 1)
print("Nested class:", t1)
t2 = timeit(test2, number = 1)
print("Non-nested class:", t2)
print("Ratio =", t1 / t2)
--
Results:
--
Nested class: 1.89952481718
Non-nested class: 0.15806536600030086
Ratio = 12.01733729573816
--
Greg
--
https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Gregory Ewing : > Marko Rauhamaa wrote: >> At least some of the methods of inner classes are closures (or there >> would be no point to an inner class). > > In Python there is no such thing as an "inner class" in the Java > sense. You can nest class statements, but that just gives you > a class that happens to be an attribute of another class. > Nothing in the nested class has any special access to anything > in the containing class. Lexically, there is special access: class C: def __init__(self, some, arg): c = self class D: def method(self): access(c) access(some) access(arg) IOW, inner class D is a container for a group of interlinked closure functions. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Marko Rauhamaa wrote: At least some of the methods of inner classes are closures (or there would be no point to an inner class). In Python there is no such thing as an "inner class" in the Java sense. You can nest class statements, but that just gives you a class that happens to be an attribute of another class. Nothing in the nested class has any special access to anything in the containing class. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Chris Angelico : > On Sun, Aug 19, 2018 at 11:54 PM, Marko Rauhamaa wrote: >>> 3) Every invocation of method() has to execute the class body, which >>> takes time. >> >> That's what happens with every method invocation in Python regardless. > > No. You have to execute the *class body*. Every method invocation has > to execute a function body. Yours includes a class definition. > Remember: The 'class' statement is NOT a declaration. It is an > executable statement. Sorry, I was being imprecise. Obviously, whenever you execute a def statement, you have to execute a def statement. Similarly, whenever you execute a class statement, you have to execute a class statement. And executing a class statement involves certain chores. Intantiating an object involves a different set of chores. And invoking a method involves yet another set of chores. Some of these chores are heavier, some of them are lighter. But where I have used Python, performance hasn't been a bottleneck. It it were, I'd choose different approaches of implementation. > Oh, even easier then. You don't need any of this locality. Which means > the inner class is buying you a whole lot of nothing. When I first ran into Java's anonymous inner classes, I wondered why they hadn't simply introduced anonymous functions. In fact, the whole debacle ended up in a schism where C# forked out of Java and introduced delegates, which are a neat concept and should have been there in C++ from the get-go. In their arms race both C# and Java finally introduced lambdas. But now I'm thinking the original Java approach (anonymous inner classes) is probably the most versatile of them all. A single function rarely captures behavior. That's the job of an object with its multiple methods. In in order to create an ad-hoc object in Python, you will need an ad-hoc class. > Hmm... so it's fine to create a class at run time, but it's not okay > to define its methods at run time. Yes. > I'm seriously confused here as to what you gain by that. How is it of > value to create a new class at run time, but to require that all its > methods and attributes be hand-written in the source code, and thus > completely fixed? Not completely fixed because the methods of the inner class can refer to the variables of the outer scope. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Sun, Aug 19, 2018 at 11:54 PM, Marko Rauhamaa wrote: > Chris Angelico : >> 2) You can't identify these objects as being of the same type (since >> they're not). > > That's a feature, not a bug. Type membership checking goes against > duck-typing. Oh, so it would be better for Python if every integer object were an instance of a unique type, all of them called "int"? Duck typing does NOT mean "having lots of identical type objects". You can argue that it's an insignificant flaw to have the duplicate types, but it's certainly not a virtue. >> 3) Every invocation of method() has to execute the class body, which >> takes time. > > That's what happens with every method invocation in Python regardless. No. You have to execute the *class body*. Every method invocation has to execute a function body. Yours includes a class definition. Remember: The 'class' statement is NOT a declaration. It is an executable statement. >> At the price of a very small amount of encapsulation, you can make it >> actually an inner class: >> >> class Outer: >> class Inner: >> def __init__(self, outer): >> self.spam = outer.quarantine >> >> def method(self): >> return self.Inner(self) > > Sure, there are ways to avoid closures, but the expressive price is > usually higher than the supposed performance gain. > >> Now all instances of Inner are actually instances of the same type. >> It's still local to the class, just not local to that method. > > Locality to the class is usually not worth the trouble. It's good enough > to have names local to the module. Oh, even easier then. You don't need any of this locality. Which means the inner class is buying you a whole lot of nothing. >> None of this explains your aversion to creating functions in a loop at >> class scope, while still being perfectly happy doing so at function >> scope. > > It had to do with populating a namespace programmatically using strings > as field/method names (which, generally, you shouldn't be doing). > Defining functions and classes dynamically during runtime is perfectly > ok. Hmm... so it's fine to create a class at run time, but it's not okay to define its methods at run time. I'm seriously confused here as to what you gain by that. How is it of value to create a new class at run time, but to require that all its methods and attributes be hand-written in the source code, and thus completely fixed? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Chris Angelico : > On Sun, Aug 19, 2018 at 10:28 PM, Marko Rauhamaa wrote: >> The most useful use of inner classes is something like this: >> >> class Outer: >> def method(self): >> outer = self >> >> class Inner: >> def spam(self, a, b): >> outer.quarantine(a, b) >> >> return Inner() > > That's pretty inefficient. Hasn't become an issue for me. > I'm not sure what you gain by having Inner be local to that method, It's a practical way of implementing the state pattern (https://en.wikipedia.org/wiki/State_pattern>). Your outer object behaves differently in different states, and the inner object encapsulates the behavior differences. Java listeners have for ever used the pattern to implement event listeners: http://www.fredosaurus.com/notes-java/GUI/events/anonym ous_listener.html>. > but here's what you lose: > > 1) Each instance of Inner carries with it a large class object. Again, that hasn't been an issue for me in practice. > 2) You can't identify these objects as being of the same type (since > they're not). That's a feature, not a bug. Type membership checking goes against duck-typing. > 3) Every invocation of method() has to execute the class body, which > takes time. That's what happens with every method invocation in Python regardless. > At the price of a very small amount of encapsulation, you can make it > actually an inner class: > > class Outer: > class Inner: > def __init__(self, outer): > self.spam = outer.quarantine > > def method(self): > return self.Inner(self) Sure, there are ways to avoid closures, but the expressive price is usually higher than the supposed performance gain. > Now all instances of Inner are actually instances of the same type. > It's still local to the class, just not local to that method. Locality to the class is usually not worth the trouble. It's good enough to have names local to the module. > None of this explains your aversion to creating functions in a loop at > class scope, while still being perfectly happy doing so at function > scope. It had to do with populating a namespace programmatically using strings as field/method names (which, generally, you shouldn't be doing). Defining functions and classes dynamically during runtime is perfectly ok. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Sun, Aug 19, 2018 at 10:28 PM, Marko Rauhamaa wrote: > Steven D'Aprano : > >> On Sun, 19 Aug 2018 11:43:44 +0300, Marko Rauhamaa wrote: >>> At least some of the methods of inner classes are closures (or there >>> would be no point to an inner class). >> >> [...] >> >> (2) Whether or not the methods of an inner class are closures depends on >> the methods, not the fact that it is an inner class. There are no >> closures here: >> >> class Outer: >> class Inner: >>... >> >> no matter what methods Inner has. Nor is this a closure: >> >> class Outer: >> def method(self): >> class Inner: >> def spam(self): >> return self.eggs >> return Inner > > The most useful use of inner classes is something like this: > > class Outer: > def method(self): > outer = self > > class Inner: > def spam(self, a, b): > outer.quarantine(a, b) > > return Inner() That's pretty inefficient. I'm not sure what you gain by having Inner be local to that method, but here's what you lose: 1) Each instance of Inner carries with it a large class object. 2) You can't identify these objects as being of the same type (since they're not). 3) Every invocation of method() has to execute the class body, which takes time. At the price of a very small amount of encapsulation, you can make it actually an inner class: class Outer: class Inner: def __init__(self, outer): self.spam = outer.quarantine def method(self): return self.Inner(self) Now all instances of Inner are actually instances of the same type. It's still local to the class, just not local to that method. None of this explains your aversion to creating functions in a loop at class scope, while still being perfectly happy doing so at function scope. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Steven D'Aprano : > On Sun, 19 Aug 2018 11:43:44 +0300, Marko Rauhamaa wrote: >> At least some of the methods of inner classes are closures (or there >> would be no point to an inner class). > > [...] > > (2) Whether or not the methods of an inner class are closures depends on > the methods, not the fact that it is an inner class. There are no > closures here: > > class Outer: > class Inner: >... > > no matter what methods Inner has. Nor is this a closure: > > class Outer: > def method(self): > class Inner: > def spam(self): > return self.eggs > return Inner The most useful use of inner classes is something like this: class Outer: def method(self): outer = self class Inner: def spam(self, a, b): outer.quarantine(a, b) return Inner() > You made a vague comment about inner classes being equivalent to > closures in some unknown fashion, but inner classes are not themselves > closures, and the methods of inner classes are not necessarily > closures. I hope the above outline removes the vagueness. populating an object with fields (methods) in a loop is very rarely a good idea. >>> >>> Of course it is *rarely* a good idea >> >> So no dispute then. > > Isn't there? Then why are you disagreeing with me about the > exceptional cases where it *is* a good idea? I don't know which statement of mine you are referring to exactly now. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Sun, 19 Aug 2018 11:43:44 +0300, Marko Rauhamaa wrote: > Steven D'Aprano : > >> On Sun, 19 Aug 2018 00:11:30 +0300, Marko Rauhamaa wrote: >> >>> In Python programming, I mostly run into closures through inner >>> classes (as in Java). >> >> Inner classes aren't closures. > > At least some of the methods of inner classes are closures (or there > would be no point to an inner class). (1) Ironically, the only times I've used an inner class, its methods were not closures. So yes, there are sometimes uses for inner classes that don't include closures. There's an example in the argparse module in the standard library, and it too has no closures. (2) Whether or not the methods of an inner class are closures depends on the methods, not the fact that it is an inner class. There are no closures here: class Outer: class Inner: ... no matter what methods Inner has. Nor is this a closure: class Outer: def method(self): class Inner: def spam(self): return self.eggs return Inner since the spam method doesn't close over any of the variables in method. You made a vague comment about inner classes being equivalent to closures in some unknown fashion, but inner classes are not themselves closures, and the methods of inner classes are not necessarily closures. >> Its also quite expensive to be populating your application with lots of >> classes used only once each, which is a common pitfall when using inner >> classes. Memory is cheap, but it's not so cheap that we ought to just >> profligately waste it needlessly. > > That is a completely separate question. It wasn't a question, it was an observation. > There's is no a-priori reason for inner classes to be wasteful; Not in languages where classes are declared statically and built at compile-time, no. But in a language like Python where classes are executable statements that are built at run time, like constructing any other mutable object, it is very easy to use them badly and waste memory. This doesn't look harmful: def func(x): class Record: def __init__(self, a): self.a = a return Record(x) but it is. You might not like that design, but it is part of Python's execution model and whether you like it or not you have to deal with the consequences :-) > they > have been part and parcel of Java programming from its early days, and > Java is widely used for high-performance applications. https://dirtsimple.org/2004/12/python-is-not-java.html > CPython does use memory quite liberally. I don't mind that as > expressivity beats performance in 99% of programming tasks. Fair enough, but in the example I showed above, the practical effect is to increase the de facto size of the objects returned by func() twenty times. And fragment memory as well. In a long-lived application where you are calling func() a lot, and saving the objects, it all adds up. >>> populating an object with fields (methods) in a loop is very rarely a >>> good idea. >> >> Of course it is *rarely* a good idea > > So no dispute then. Isn't there? Then why are you disagreeing with me about the exceptional cases where it *is* a good idea? -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Chris Angelico : > On Sun, Aug 19, 2018 at 9:03 AM, Marko Rauhamaa wrote: >> Chris Angelico : >> >>> *headscratch* >>> >>> So this is okay: >>> >>> def f(): >>> for i in range(5): >>> def g(): ... >>> >>> But this isn't: >>> >>> class C: >>> for i in range(5): >>> def m(self): ... >>> >>> I've missed something here. >> >> No, you got it right. > > Then I've completely missed the problem. Why is one of them acceptable > and the other not? In the def-def case, you will do something mundane with g. For example, you will register it as a callback. In the class-def case, you are defining the method m five times in the same namespace and overwriting all but one of the definitions, which probably isn't what you are after. In order to populate the class with methods of different names, you will need to manipulate the namespace programmatically. If you find yourself needing to do something like that, you need to take a couple of steps back and ask yourself if there might be a more conventional way to solve the problem at hand. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Steven D'Aprano : > On Sun, 19 Aug 2018 00:11:30 +0300, Marko Rauhamaa wrote: > >> In Python programming, I mostly run into closures through inner classes >> (as in Java). > > Inner classes aren't closures. At least some of the methods of inner classes are closures (or there would be no point to an inner class). > Its also quite expensive to be populating your application with lots > of classes used only once each, which is a common pitfall when using > inner classes. Memory is cheap, but it's not so cheap that we ought to > just profligately waste it needlessly. That is a completely separate question. There's is no a-priori reason for inner classes to be wasteful; they have been part and parcel of Java programming from its early days, and Java is widely used for high-performance applications. CPython does use memory quite liberally. I don't mind that as expressivity beats performance in 99% of programming tasks. >> populating an object with fields (methods) in a loop is very rarely a >> good idea. > > Of course it is *rarely* a good idea So no dispute then. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Sun, Aug 19, 2018 at 9:03 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> *headscratch* >> >> So this is okay: >> >> def f(): >> for i in range(5): >> def g(): ... >> >> But this isn't: >> >> class C: >> for i in range(5): >> def m(self): ... >> >> I've missed something here. > > No, you got it right. > Then I've completely missed the problem. Why is one of them acceptable and the other not? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Chris Angelico : > *headscratch* > > So this is okay: > > def f(): > for i in range(5): > def g(): ... > > But this isn't: > > class C: > for i in range(5): > def m(self): ... > > I've missed something here. No, you got it right. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Sun, Aug 19, 2018 at 11:13 AM, Steven D'Aprano wrote: > Obviously there is some (small) complexity cost to automating it. I > didn't specify what a fair number of methods would be (my example showed > four, but that was just an illustration, not real code). In practice I > wouldn't even consider this for three methods. Six or eight seems like a > reasonable cut-of point for me, but it depends on the specifics of the > code and who I was writing it for. > > (Note that this makes me much more conservative than the usual advice > given by system admins, when you need to do the same thing for the third > time, write a script to automate it.) The boundary definitely varies. I've often gone to a dozen almost-identical blocks of code before turning them into a loop (when the "almost" makes it a lot harder to collapse them usefully), and sometimes, just two copies is enough to refactor. But six to eight does seem like a reasonable point for tiny (maybe stub) functions. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Sun, 19 Aug 2018 00:11:30 +0300, Marko Rauhamaa wrote:
> In Python programming, I mostly run into closures through inner classes
> (as in Java).
Inner classes aren't closures.
Its also quite expensive to be populating your application with lots of
classes used only once each, which is a common pitfall when using inner
classes. Memory is cheap, but it's not so cheap that we ought to just
profligately waste it needlessly.
> populating an object with fields (methods) in a loop is very
> rarely a good idea.
Of course it is *rarely* a good idea, because it is rare for the fields
to be either identical (except for the name) or algebraically derived
from the loop counter. Using a dict in place of an object, it's hard to
see any elegant way to move this into a loop:
{'a': 10, 'B': -2, 'c': 97, 'd': None, 'h': 'surprise!', 'm': []}
and so we should not. Any such loop would surely be complex, complicated,
obscure, even obfuscated compared to writing out the dict/object
assignments manually.
But in context, we're not discussing the millions of cases were the
methods/fields are naturally written out manually.
So give me credit for not being a total idiot. Not once in this thread
have I suggested that we ought to run through all our projects, changing
every class and putting all methods inside factories. It goes without
saying that under usual, common circumstances we write out our methods
manually. I was speaking about one very specific case:
* You have a fair number of identical methods in a single class.
Our choices are, (1):
- write a large block of mindless boilerplate;
- even worse, have that same boilerplate but split it up,
scattering the individual methods all around the class;
- either way, it is repetitious and error-prone, with
obvious reliability and maintenance problems:
def foo(self):
return NotImplemented
def bar(self):
return NotImplemented
def baz(self):
return NonImplemented
or, (2):
- automate the repetitious code by moving the method
definitions into a loop.
Obviously there is some (small) complexity cost to automating it. I
didn't specify what a fair number of methods would be (my example showed
four, but that was just an illustration, not real code). In practice I
wouldn't even consider this for three methods. Six or eight seems like a
reasonable cut-of point for me, but it depends on the specifics of the
code and who I was writing it for.
(Note that this makes me much more conservative than the usual advice
given by system admins, when you need to do the same thing for the third
time, write a script to automate it.)
--
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson
--
https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Sun, Aug 19, 2018 at 7:11 AM, Marko Rauhamaa wrote: > Chris Angelico : >> Your acceptance of closures is a perfect proof of how magic stops >> looking like magic once you get accustomed to it. > > Actually, that's a very good observation. You should stick with a > smallish kernel of primitives and derive the universe from them. > > Anyway, functions as first-class objects are truly foundational in all > high-level programming. In Python programming, I mostly run into > closures through inner classes (as in Java). > >> If you can accept closures because they just DTRT, why not accept a >> much simpler and more obvious operation like putting a 'def' statement >> in a loop? > > Nothing wrong or extraordinary with putting a def statement in a loop, > but populating an object with fields (methods) in a loop is very rarely > a good idea. > *headscratch* So this is okay: def f(): for i in range(5): def g(): ... But this isn't: class C: for i in range(5): def m(self): ... I've missed something here. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Chris Angelico : > Your acceptance of closures is a perfect proof of how magic stops > looking like magic once you get accustomed to it. Actually, that's a very good observation. You should stick with a smallish kernel of primitives and derive the universe from them. Anyway, functions as first-class objects are truly foundational in all high-level programming. In Python programming, I mostly run into closures through inner classes (as in Java). > If you can accept closures because they just DTRT, why not accept a > much simpler and more obvious operation like putting a 'def' statement > in a loop? Nothing wrong or extraordinary with putting a def statement in a loop, but populating an object with fields (methods) in a loop is very rarely a good idea. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Sun, Aug 19, 2018 at 6:28 AM, Marko Rauhamaa wrote:
> Steven D'Aprano :
>>> In a word, steer clear of metaprogramming.
>>
>> [...]
>> (2) if you mean what you say, that means no decorators,
>
> Correct. I don't find decorators all that useful or tasteful.
>
>> no closures,
>
> Closures I consider ordinary programming. Nothing meta there.
So it's okay to have a function that can be created more than once,
then? Because closures usually will be.
>> no introspection ("reflection" in Java terms),
>
> Introspection is suspect in general.
So, no calling help() or using any sort of automated signature display.
>> no source-code generators.
>
> Lisp-style macros (or scheme syntax rules) are rather a clean way to do
> that, but even that mechanism should be used very sparingly and
> tastefully.
I posted something from an active project of mine earlier. Please
explain how to do it better, in your no-metaprogramming world. Show me
how my code would be improved by avoiding code generation.
>> No namedtuples, Enums, or data-classes.
>
> They don't seem all that meta to me, but coincidentally I never found
> uses for them in my Python code.
They all involve metaclasses, which you have forbidden. Ergo you
should not use them. (At least, I think namedtuple uses a metaclass.
Might be wrong there. It certainly uses exec().)
Your acceptance of closures is a perfect proof of how magic stops
looking like magic once you get accustomed to it. How do closures
actually work? Do you have any idea, or do you just accept that stuff
works the way you expect it to?
If you can accept closures because they just DTRT, why not accept a
much simpler and more obvious operation like putting a 'def' statement
in a loop?
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Steven D'Aprano :
>> In a word, steer clear of metaprogramming.
>
> [...]
> (2) if you mean what you say, that means no decorators,
Correct. I don't find decorators all that useful or tasteful.
> no closures,
Closures I consider ordinary programming. Nothing meta there.
> no introspection ("reflection" in Java terms),
Introspection is suspect in general.
> no metaclasses (other than type),
Correct.
> no use of descriptors (other than the built-in ones),
Haven't used (or at least defined) them.
> no template-based programming,
Please, no.
> no source-code generators.
Lisp-style macros (or scheme syntax rules) are rather a clean way to do
that, but even that mechanism should be used very sparingly and
tastefully.
> No namedtuples, Enums, or data-classes.
They don't seem all that meta to me, but coincidentally I never found
uses for them in my Python code.
Marko
--
https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Sat, 18 Aug 2018 00:33:26 +0300, Marko Rauhamaa wrote:
> Chris Angelico :
>> Programming is heavily about avoiding duplicated work.
>
> That is one aspect, but overcondensing and overabstracting programming
> logic usually makes code less obvious to its maintainer.
That may very well be true, but we're not talking about those evils here.
We're talking about a simple factory technique for creating a number of
identical objects in a loop.
[...]
> I would guess such techniques could come in handy in some framework
> development but virtually never in ordinary application development. In
> a word, steer clear of metaprogramming.
Depending on your definition of metaprogramming, either:
(1) this either isn't metaprogramming at all, merely programming and no
more scary than populating a dict at runtime; or
(2) if you mean what you say, that means no decorators, no closures, no
introspection ("reflection" in Java terms), no metaclasses (other than
type), no use of descriptors (other than the built-in ones), no template-
based programming, no source-code generators. No namedtuples, Enums, or
data-classes.
--
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson
--
https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Sat, Aug 18, 2018 at 7:33 AM, Marko Rauhamaa wrote:
> Chris Angelico :
>> Programming is heavily about avoiding duplicated work.
>
> That is one aspect, but overcondensing and overabstracting programming
> logic usually makes code less obvious to its maintainer. It is essential
> to find coding idioms that communicate ideas as clearly as possible. In
> some situations boilerplate and redundancy can help make the code more
> robust, as not every line of code becomes a clever brainteaser.
As Steven already pointed out:
> [W]hat states the intention "These methods are identical except
> in their name" more strongly than creating them in a loop?
What better way is there to communicate the idea "we now create ten
methods according to this template" than a loop?
>> Creating methods is work. Creating many identical (or similar) methods
>> is duplicated work. What's wrong with using a loop to create
>> functions?
>
> I would guess such techniques could come in handy in some framework
> development but virtually never in ordinary application development. In
> a word, steer clear of metaprogramming.
Ah yes, because app development never has situations where lots of
similar code that could benefit from metaprogramming. Blub Paradox
strikes again: since you are not comfortable with metaprogramming, you
never see a need for it, and when you're brought face-to-face with it,
you consider it worse than the alternatives. Me, I consider it a vital
feature, to the extent that I will metaprogram externally if I need
to.
https://github.com/Rosuav/TF2BuffBot/blob/master/drzed.sp#L23
https://github.com/Rosuav/TF2BuffBot/blob/master/gen_effects_tables.py#L118
The Python script generates a bunch of lines of code like this:
sm_drzed_max_hitpoints = CreateConVar("sm_drzed_max_hitpoints", "0",
"Number of hitpoints a normal character has (w/o Assault Suit) - 0 to
leave at default", 0, true, 0.0);
Since SourcePawn has no metaprogramming facilities other than C-style
#define, I have to use a Python script that reads specially-formatted
comments and generates an include file which is then compiled in.
Which is better - to do that, or to have a loop that generates
functions *right in the class that needs them*?
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Chris Angelico : > Programming is heavily about avoiding duplicated work. That is one aspect, but overcondensing and overabstracting programming logic usually makes code less obvious to its maintainer. It is essential to find coding idioms that communicate ideas as clearly as possible. In some situations boilerplate and redundancy can help make the code more robust, as not every line of code becomes a clever brainteaser. > Creating methods is work. Creating many identical (or similar) methods > is duplicated work. What's wrong with using a loop to create > functions? I would guess such techniques could come in handy in some framework development but virtually never in ordinary application development. In a word, steer clear of metaprogramming. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Fri, 17 Aug 2018 15:19:05 +0200, Peter Otten wrote: > You usually do not want many identical (or very similar) methods because > invoking the right one is then errorprone, too, and you end up with an > interface that is hard to maintain. At some point you may need to > introduce subtle changes to one out of ten methods, These hypotheticals are fairly tedious. "At some point you might..." yeah, you might, but you probably won't, and in the meantime, YAGNI. And if you do, it is easy to do: pull the special method out of the loop. Or add code to modify it after the loop. Or make the changes in a subclass. We do these things *all the time* for data objects, creating them in a loop then modifying those that need modifying. There is *no difference* here: methods can be treated as data objects too. > and later someone > else may overlook that specific angle in the documentation... You say that as if people never failed to read the documentation about "regular" methods that are made by hand in the conventional way. > If you have many similar methods you should spend your time on reducing > their number rather than to find shortcuts to automate their creation. The assumption here is that the basic design is sound. Why do you assume it isn't? According to the OP Frank, this design has been in production for many years and works well. While I personally have some reservations that using subclasses is the best solution, I'm willing to give him the benefit of the doubt rather than insult his competence by assuming it is a broken design without ever seeing the code. > Programming is not only about avoiding duplication, it is also about > stating your intents clearly. Indeed. And what states the intention "These methods are identical except in their name" more strongly than creating them in a loop? -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Fri, 17 Aug 2018 11:49:01 +, Jon Ribbens wrote: > On 2018-08-17, Steven D'Aprano > wrote: >> On the other hand, your objection to the following three idioms is as >> good an example of the Blurb Paradox as I've ever seen. > > Do you mean the Blub Paradox? If so, you're misunderstanding or at least > misapplying it. Yes, that was a simple typo, and no, I'm not misunderstanding it. You're looking up the ladder to a more powerful technique available in Python (methods as first-class values capable of being manipulated like any other object) and dismissing it in favour of mindless boilerplate containing duplicated code, and requiring oodles of copy-and-paste programming to maintain. Graham used the "Blub Paradox" to describe programmers' failure to understand more powerful features available in languages they didn't use, but there's no reason why this failure applies only to comparisons between languages. It also applies to arguments about idioms within a single language. That's the Blub Paradox too, even though only a single language is involved. >>> * code running directly under the class definition >>> * creating a method then changing its name with foo.__name__ >>> * poking things into to the class namespace with locals() >> >> Each of these are standard Python techniques, utterly unexceptional. > > I guess we'll have to agree to disagree there. >> "Code running directly under the class" describes every use of the >> class keyword (except those with an empty body). If you write: >> >> class Spam: >> x = 1 >> >> you are running code under the class. This is not just a pedantic >> technicality, > > Yes, it absolutely is, in this context. Having code other than > assignments and function definitions under the class statement is > extremely rare. Its rare because it isn't needed often, not because it is broken or dangerous or illegal or fattening. [...] >> You might be thinking of the warning in the docs: >> >> "Dynamically adding abstract methods to a class, [...] [is] not >> supported." >> >> but that is talking about the case where you add the method to the >> class after the class is created, from the outside: > > Yes, I was referring to that. You may well be right about what it means > to say, but it's not what it actually says. *shrug* It was obvious to me that it wasn't talking about methods dynamically inserted inside the class body since ALL methods are dynamically inserted inside the class body. If that was what it meant, it would be saying that abstractmethod never works. Clearly that's absurd, since it does work. So why interpret it as saying something absurd instead of using a bit of common sense and knowledge of how Python words to interpret it correctly? Inside a class (or at the global scope) there is no meaningful difference between these: spam = eggs locals()['spam'] = eggs >>> (Not to mention your code means the methods cannot have meaningful >>> docstrings.) >> >> Of course they can, provided they're all identical, give or take some >> simple string substitutions. > > Hence "meaningful". They can still be meaningful even if identical. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Frank Millman wrote: > I find that using a separate method per subclass does exactly what I want, > and that part of my project has been working stably for some time. I think that approach is fine. Do not let a tool have you bend over backwards. Do not give in to pylint's nagging when you are convinced that your solution is a good one. Personally I often prefer code with a score of eight or nine to a "perfect" ten achieved by mechanically working around pylint's complaints. PS: In this case I'd probably start with a noop check that allows everything rather than draw in the abc machinery, but /chacun/ /à/ /son/ /goût/. -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
Chris Angelico wrote: > On Fri, Aug 17, 2018 at 9:49 PM, Jon Ribbens > wrote: >>> "Code running directly under the class" describes every use of the class >>> keyword (except those with an empty body). If you write: >>> >>> class Spam: >>> x = 1 >>> >>> you are running code under the class. This is not just a pedantic >>> technicality, >> >> Yes, it absolutely is, in this context. Having code other than >> assignments and function definitions under the class statement >> is extremely rare. > > Programming is heavily about avoiding duplicated work. Creating > methods is work. Creating many identical (or similar) methods is > duplicated work. What's wrong with using a loop to create functions? You usually do not want many identical (or very similar) methods because invoking the right one is then errorprone, too, and you end up with an interface that is hard to maintain. At some point you may need to introduce subtle changes to one out of ten methods, and later someone else may overlook that specific angle in the documentation... If you have many similar methods you should spend your time on reducing their number rather than to find shortcuts to automate their creation. Programming is not only about avoiding duplication, it is also about stating your intents clearly. -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Fri, Aug 17, 2018 at 9:49 PM, Jon Ribbens wrote: >> "Code running directly under the class" describes every use of the class >> keyword (except those with an empty body). If you write: >> >> class Spam: >> x = 1 >> >> you are running code under the class. This is not just a pedantic >> technicality, > > Yes, it absolutely is, in this context. Having code other than > assignments and function definitions under the class statement > is extremely rare. Programming is heavily about avoiding duplicated work. Creating methods is work. Creating many identical (or similar) methods is duplicated work. What's wrong with using a loop to create functions? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On 2018-08-17, Steven D'Aprano wrote: > On the other hand, your objection to the following three idioms is as > good an example of the Blurb Paradox as I've ever seen. Do you mean the Blub Paradox? If so, you're misunderstanding or at least misapplying it. >> * code running directly under the class definition >> * creating a method then changing its name with foo.__name__ >> * poking things into to the class namespace with locals() > > Each of these are standard Python techniques, utterly unexceptional. I guess we'll have to agree to disagree there. > "Code running directly under the class" describes every use of the class > keyword (except those with an empty body). If you write: > > class Spam: > x = 1 > > you are running code under the class. This is not just a pedantic > technicality, Yes, it absolutely is, in this context. Having code other than assignments and function definitions under the class statement is extremely rare. >> * dynamically adding @abstractmethod methods to a class > > I simply don't get this objection at all. All methods are added > dynamically to classes (that's how Python's execution model works, def is > an executable statement not a declaration). Making them abstract doesn't > change this. > > You might be thinking of the warning in the docs: > > "Dynamically adding abstract methods to a class, [...] > [is] not supported." > > but that is talking about the case where you add the method to the class > after the class is created, from the outside: Yes, I was referring to that. You may well be right about what it means to say, but it's not what it actually says. >> (Not to mention your code means the methods cannot have meaningful >> docstrings.) > > Of course they can, provided they're all identical, give or take some > simple string substitutions. Hence "meaningful". -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
"Dan Sommers" wrote in message news:pl622a$a1f$1...@blaine.gmane.org... On Fri, 17 Aug 2018 09:46:01 +0200, Frank Millman wrote: > It is just a slight annoyance (to me) that pylint complains about the > subclass methods when they are called from the Field class. I don't > mind adding 10 stub methods to the Field class to keep it happy, but I > do not get the feeling that it is improving my code. I do grant, > however, that it may be of benefit to someone reading my code for the > first time. What about the next subclass, the one that doesn't exist today? If you write it, then you know to create those ten methods. If I write it, though, then those ten stubs tell me that I have to override them, and their docstrings tell me how. That is a huge improvement in long term maintainability. Can't argue with that. I guess it shows that I am still stuck in a 'one-man project' mentality. Frank -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Fri, 17 Aug 2018 09:46:01 +0200, Frank Millman wrote: > It is just a slight annoyance (to me) that pylint complains about the > subclass methods when they are called from the Field class. I don't > mind adding 10 stub methods to the Field class to keep it happy, but I > do not get the feeling that it is improving my code. I do grant, > however, that it may be of benefit to someone reading my code for the > first time. What about the next subclass, the one that doesn't exist today? If you write it, then you know to create those ten methods. If I write it, though, then those ten stubs tell me that I have to override them, and their docstrings tell me how. That is a huge improvement in long term maintainability. Dan -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
"Steven D'Aprano" wrote in message news:pl5qbk$r7k$5...@blaine.gmane.org... On Fri, 17 Aug 2018 08:14:02 +0200, Frank Millman wrote: > I find that using a separate method per subclass does exactly what I > want, and that part of my project has been working stably for some time. You might consider using single dispatch instead: https://docs.python.org/3/library/functools.html#functools.singledispatch Thanks. I had a quick look, but I could not see an immediate improvement over my current setup. I just did a quick inventory of my Field class, which is an abstract class. I have 13 datatypes, therefore 13 subclasses. I have 26 methods which are common to all subclasses, so they are defined on the Field class. Each subclass has between 5 and 10 methods declared on that subclass. Some of them do not exist on the Field class, as they are all different. Some of them do exist on the Field class, as only some of the subclasses require an override, the rest can use the generic method. This arrangement has evolved over time, but has now settled down and is stable. It is just a slight annoyance (to me) that pylint complains about the subclass methods when they are called from the Field class. I don't mind adding 10 stub methods to the Field class to keep it happy, but I do not get the feeling that it is improving my code. I do grant, however, that it may be of benefit to someone reading my code for the first time. Frank -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Fri, 17 Aug 2018 08:14:02 +0200, Frank Millman wrote: > How would you extend it without a long chain of > if isinstance(v, str): > [perform checks for str] > elif isinstance(v, int) > [perform checks for int] > etc > etc > > I find that using a separate method per subclass does exactly what I > want, and that part of my project has been working stably for some time. You might consider using single dispatch instead: https://docs.python.org/3/library/functools.html#functools.singledispatch -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Wed, 15 Aug 2018 10:44:57 +, Jon Ribbens wrote:
> Obviously what I'm objecting to is not the presence of a loop, it's the
> presence of many obscure, bodgey and/or broken features all lumped
> together:
> * using @abstractmethod without metaclass=ABCMeta
I'll cop to that one -- it was a bug in my made-up pseudo-code. I simply
assumed that people would know you need to use ABCMeta.
Meap culpa.
On the other hand, your objection to the following three idioms is as
good an example of the Blurb Paradox as I've ever seen. Python is
designed to do these sorts of things. Failing to use them when it
simplifies your code is like hiring a fork-lift to lift and turn your car
around because using reverse gear is "obscure, bodgey and/or broken".
> * code running directly under the class definition
> * creating a method then changing its name with foo.__name__
> * poking things into to the class namespace with locals()
Each of these are standard Python techniques, utterly unexceptional.
"Code running directly under the class" describes every use of the class
keyword (except those with an empty body). If you write:
class Spam:
x = 1
you are running code under the class. This is not just a pedantic
technicality, it is fundamental to Python's execution model. If someone
does not understand this fact, they don't understand Python.
The Python interpreter goes to great lengths to insert the class
namespace into the execution scope while executing code under the class
body, and has done so since Python 1.5 or older, even before it had
lexical scoping!
[steve@ando ~]$ python1.5
Python 1.5.2 (#1, Aug 27 2012, 09:09:18) [GCC 4.1.2 20080704 (Red Hat
4.1.2-52)] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> x = 23
>>> class Spam:
... x = 42
... y = x + 1
...
>>> print x, Spam.x, Spam.y
23 42 43
"Creating a method then changing its name" -- that's precisely what we do
anytime we create a closure and modify it using functools.wraps. The only
difference is that wraps hides that behind a decorator. Some of us have
been using Python long enough to remember when the standard idiom for
decorating a function was to manually adjust its name, docstring etc:
# Something like this.
def decorate(func):
def inner(arg):
preprocess()
x = func(arg)
postprocess()
return x
inner.__name__ = func.__name__
inner.__doc__ = func.__doc__
inner.__dict__.update(func.__dict__)
return inner
Now functools.wraps hides all that behind a convenient decorator
interface, but we're still changing the name.
If you *don't* change the function's name, you're effectively populating
your code with functions that might as well be anonymous, or at least
have generic names like "inner". Why would you not give your functions
meaningful names, just because they came out of a factory?
"Poking things into the class namespace with locals()" -- I'll admit that
this one is a little rarer. But again, the interpreter goes to deliberate
lengths to ensure that locals in a class body is available for writing.
If we're not supposed to use locals, why do you think Python makes it
available? Its not like writing to locals inside a class is any more
bizarre than using getattr, its just less common.
Not one of those three techniques is "bodgey and/or broken", and if you
think they're obscure, well, I'm sorry but that says more about the
breadth of your Python knowledge than the feature itself.
Nobody calling themselves an experienced Python programmer ought to have
any trouble reading and maintaining code like that I showed. If they do,
they have a serious hole in their knowledge of the language, like a C
programmer who doesn't get pointers.
> * dynamically adding @abstractmethod methods to a class
I simply don't get this objection at all. All methods are added
dynamically to classes (that's how Python's execution model works, def is
an executable statement not a declaration). Making them abstract doesn't
change this.
You might be thinking of the warning in the docs:
"Dynamically adding abstract methods to a class, [...]
[is] not supported."
but that is talking about the case where you add the method to the class
after the class is created, from the outside:
class X(metaclass=ABCMeta):
def method(self):
pass
# Doesn't work.
X.method = abstractmethod(X.method)
But this works exactly as you would expect:
py> class Foo(metaclass=abc.ABCMeta):
... for name in ('spam', 'eggs'):
... @abc.abstractmethod
... def inner(self): pass
... inner.__name__ = name
... locals()[name] = inner
... del inner
...
py> class Bar(Foo):
... pass
...
py> Bar()
Traceback (most recent call last):
File "", line 1, in
TypeError: Can't instantiate abstract class Bar with abstract
methods eggs, spam
> (Not to mention your code
Re: Pylint false positives
"D'Arcy Cain" wrote in message
news:6b4b8587-46c0-19b0-c538-efdf396f0...@vybenetworks.com...
On 2018-08-14 04:58 AM, Frank Millman wrote:
> As an example, I have a master class defining a unit of data (i.e. the
> value of a column) retrieved from a database. I have separate
> sub-classes for each data type - text, integer, date, etc. To ensure
> that a value is valid before storing it as an instance attribute, I call
> a method called 'check_value'. The details of check_value vary according
> to the data type, but that is transparent to the piece of code that
> calls check_value().
class classA:
DATATYPE = None # Or default type
def check_value(self, v)
if not isinstance(v, self.DATATYPE):
raise RuntimeError("Invalid data type for '%s'" % v)
class classB(classA):
DATATYPE = int
Very simplistic and untested but does that give you any ideas?
Hopefully your email client doesn't mess up the formatting. You can
fill out check_value to do more than simply check the the type matches
and you can also do further checks based on the type. Also, you can
have more than one sub-class doing the same check without having to cut
and paste code from another class.
Thanks, D'Arcy. That is a neat idea if all you want to do is check the data
type, but I do a lot more than that.
How would you extend it without a long chain of
if isinstance(v, str):
[perform checks for str]
elif isinstance(v, int)
[perform checks for int]
etc
etc
I find that using a separate method per subclass does exactly what I want,
and that part of my project has been working stably for some time.
The only thing that has changed is that I recently started using pylint (as
a result of switching my editor to VS Code).
My main class has the following method (simplified) -
class Field:
def setval(self, value): # handle value received from external source
"""
checkval is a method defined in each subclass
it does a bit of typecasting, so value is replaced on return
it can raise an exception, which is caught elsewhere
"""
value = self.checkval(value) # this is the line that pylint
complains about
[various other checks]
if all checks are passed:
self._value = value
Frank
--
https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On 2018-08-14 04:58 AM, Frank Millman wrote:
> "D'Arcy Cain" wrote in message
>> I am also getting a funny smell from your description. Are you sure
>> that you need to redefine the methods? Perhaps you just need to define
>> some class variables and use one method. You can also define your own
>> method and call the classA method inside it for common functionality.
>
> As an example, I have a master class defining a unit of data (i.e. the
> value of a column) retrieved from a database. I have separate
> sub-classes for each data type - text, integer, date, etc. To ensure
> that a value is valid before storing it as an instance attribute, I call
> a method called 'check_value'. The details of check_value vary according
> to the data type, but that is transparent to the piece of code that
> calls check_value().
class classA:
DATATYPE = None # Or default type
def check_value(self, v)
if not isinstance(v, self.DATATYPE):
raise RuntimeError("Invalid data type for '%s'" % v)
class classB(classA):
DATATYPE = int
Very simplistic and untested but does that give you any ideas?
Hopefully your email client doesn't mess up the formatting. You can
fill out check_value to do more than simply check the the type matches
and you can also do further checks based on the type. Also, you can
have more than one sub-class doing the same check without having to cut
and paste code from another class.
--
D'Arcy J.M. Cain
Vybe Networks Inc.
http://www.VybeNetworks.com/
IM:da...@vex.net VoIP: sip:da...@vybenetworks.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On 2018-08-15, Steven D'Aprano wrote: > On Tue, 14 Aug 2018 15:18:13 +, Jon Ribbens wrote: >> On 2018-08-14, Steven D'Aprano >> wrote: >>> # === process abstract methods en masse === >>> for name in "method_a method_b method_c method_d".split(): >>> @abstractmethod >>> def inner(self): >>> raise NotImplementedError >>> inner.__name__ = name >>> # This is okay, writing to locals works inside the class body. >>> locals()[name] = inner >>> >>> del inner, name # Clean up the class namespace. >> >> You have a peculiar idea of "good style"... > > Yes, very peculiar. It's called "factor out common operations" and "Don't > Repeat Yourself" :-) > > In a world full of people who write: > > d[1] = None > d[2] = None > d[3] = None > d[4] = None > > I prefer to write: > > for i in range(1, 5): >d[i] = None > > Shocking, I know. Obviously what I'm objecting to is not the presence of a loop, it's the presence of many obscure, bodgey and/or broken features all lumped together: * code running directly under the class definition * creating a method then changing its name with foo.__name__ * poking things into to the class namespace with locals() * using @abstractmethod without metaclass=ABCMeta * dynamically adding @abstractmethod methods to a class (Not to mention your code means the methods cannot have meaningful docstrings.) I would refuse a pull request containing code such as the above, unless the number of methods being dynamically created was much larger than 4, in which case I would refuse it because the design of a class requiring huge numbers of dynamically created methods is almost certainly fundamentally broken. >> No - if you think about it, there's no way Pylint could possibly know >> that the above class has methods method_a, method_b, etc. > > Well, if a human reader can do it, a sufficiently advanced source-code > analyser could do it too... *wink* If we get there, we won't need to do computer programming since we'll have strong AI and we'll just say "computer, obey my wishes" :-) > Yes, of course you are right, in practical terms I think it is extremely > unlikely that PyLint or any other linter is smart enough to recognise > that locals()[name] = inner is equivalent to setting attributes method_a > etc. I actually knew that... "although to be honest I'm not sure" is an > understated way of saying "It isn't" :-) It's not so much the locals()[name] thing as much as that Pylint would have to know what the computed values of `name` would be. >> It also doesn't like the `del inner, name` because theoretically >> neither of those names might be defined, if the loop executed zero >> times. > > That's a limitation of the linter. Don't blame me if it is too stupid to > recognise that looping over a non-empty string literal cannot possibly > loop zero times :-) Well, it's not looping over a string literal, it's looping over the output of a method call on a string literal. You could *maybe* argue that "literal words here".split() is sufficiently idiomatic that Pylint should understand it, and the Pylint issues page would be an excellent place to do so ;-) -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Tue, 14 Aug 2018 15:18:13 +, Jon Ribbens wrote:
> On 2018-08-14, Steven D'Aprano
> wrote:
>> If there really are a lot of such missing methods, I'd consider writing
>> something like this:
>>
>> class A:
>> def __init__(self, ...):
>> ...
>>
>> # === process abstract methods en masse ===
>> for name in "method_a method_b method_c method_d".split():
>> @abstractmethod
>> def inner(self):
>> raise NotImplementedError
>> inner.__name__ = name
>> # This is okay, writing to locals works inside the class body.
>> locals()[name] = inner
>>
>> del inner, name # Clean up the class namespace.
>
> You have a peculiar idea of "good style"...
Yes, very peculiar. It's called "factor out common operations" and "Don't
Repeat Yourself" :-)
In a world full of people who write:
d[1] = None
d[2] = None
d[3] = None
d[4] = None
I prefer to write:
for i in range(1, 5):
d[i] = None
Shocking, I know.
Literally my first professional programming job was working on a
Hypercard project written by a professional programmer. (He was paid for
it, so he was professional.) The first time I looked at his code, as a
fresh-out-of-uni naive coder, I was surprised to read his GUI set-up
code. By memory, it was something like this:
set the name of button 1 to "Wibble 1"
set the name of button 2 to "Wibble 2"
set the name of button 3 to "Wibble 3"
set the name of button 4 to "Wibble 4"
# and so on...
set the name of button 100 to "Wibble 100"
(using "Wibble" as a placeholder for the actual name, which I don't
recall). The first thing I did was replace that with a loop:
for i = 1 to 100 do
set the name of button 100 to ("Wibble " & i)
end for
Hypertalk uses & for string concatenation. That one change cut startup
time from something like 90 seconds to about 30, and a few more equally
trivial changes got it down to about 15 seconds.
Hypertalk in 1988 was not the fastest language in the world, but it was
fun to work with.
>> although to be honest I'm not sure if that would be enough to stop
>> PyLint from complaining.
>
> No - if you think about it, there's no way Pylint could possibly know
> that the above class has methods method_a, method_b, etc.
Well, if a human reader can do it, a sufficiently advanced source-code
analyser could do it too... *wink*
Yes, of course you are right, in practical terms I think it is extremely
unlikely that PyLint or any other linter is smart enough to recognise
that locals()[name] = inner is equivalent to setting attributes method_a
etc. I actually knew that... "although to be honest I'm not sure" is an
understated way of saying "It isn't" :-)
https://en.wikipedia.org/wiki/Litotes
> It also
> doesn't like the `del inner, name` because theoretically neither of
> those names might be defined, if the loop executed zero times.
That's a limitation of the linter. Don't blame me if it is too stupid to
recognise that looping over a non-empty string literal cannot possibly
loop zero times :-)
--
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson
--
https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
"Frank Millman" wrote in message news:pku0qd$ua5$1...@blaine.gmane.org... Pylint is flagging a lot of lines as errors that I would consider to be acceptable. I have an abstract class ClassA with a number of concrete sub-classes. ClassA has a method which invokes 'self.method_b()' which is defined separately on each sub-class. Pylint complains that "Instance of 'ClassA' has no 'method_b' member". Thanks for all the responses. They have helped to clarify my thinking. To summarise - 1. If I want my project to be taken seriously (which I do) I should define all methods in ClassA, with docstrings. 2. Pythons abc module, with its @abstractmethod decorator, seems the ideal fit for my situation, so I should use that. Frank -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On 8/14/2018 5:05 AM, Thomas Jollans wrote: On 2018-08-14 09:38, Frank Millman wrote: Hi all Pylint is flagging a lot of lines as errors that I would consider to be acceptable. I have an abstract class ClassA with a number of concrete sub-classes. ClassA has a method which invokes 'self.method_b()' which is defined separately on each sub-class. Pylint complains that "Instance of 'ClassA' has no 'method_b' member". First question - as a matter of style, is Pylint correct? If so, I could define 'method_b' in ClassA and raise NotImplementedError. Is this considered more pythonic? The downside is that I have quite a few of them, so it would add some clutter. If I were reading you class, I would like to see all methods defined there. If you went the old NotImplemented(Error) route, you could avoid cluttter with method_c = method_d = method_e = method_b but I would also want docstrings. At that point, I would consider abstractmethod. But I have not used that, not seen an stdlib class that does. I wouldn't say it's unpythonic per se, but if your ClassA is logically an abstract base class that requires certain methods to be implemented in subclasses, then it's probably clearer to use Python's abc [1] facilities, and declare your abstract methods as actual abstractmethods. [1]: https://docs.python.org/3/library/abc.html i.e. from abc import ABC, abstractmethod class ClassA(ABC): def do_stuff(self): return self.method_b(42)**3 @abstractmethod def method_b(self, answer): """ This is a great place to put a docstring """ You *can* raise NotImplementedError in your abstractmethods, but I don't think it's really necessary. You need a statement in the method body of course, but since you're going to put a docstring there anyway (right?), that's already taken care of. -- Thomas Second question - if my present code is not unpythonic, is there an easy way to suppress the error messages, without disabling 'no-member' altogether? -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On 2018-08-14, Steven D'Aprano wrote: > If there really are a lot of such missing methods, I'd consider writing > something like this: > > class A: > def __init__(self, ...): > ... > > # === process abstract methods en masse === > for name in "method_a method_b method_c method_d".split(): > @abstractmethod > def inner(self): > raise NotImplementedError > inner.__name__ = name > # This is okay, writing to locals works inside the class body. > locals()[name] = inner > > del inner, name # Clean up the class namespace. You have a peculiar idea of "good style"... > although to be honest I'm not sure if that would be enough to stop PyLint > from complaining. No - if you think about it, there's no way Pylint could possibly know that the above class has methods method_a, method_b, etc. It also doesn't like the `del inner, name` because theoretically neither of those names might be defined, if the loop executed zero times. -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On Tue, 14 Aug 2018 10:58:17 +0200, Frank Millman wrote: >> > I have an abstract class ClassA with a number of concrete >> > sub-classes. ClassA has a method which invokes 'self.method_b()' >> > which is defined separately on each sub-class. Pylint complains that >> > "Instance of 'ClassA' has no 'method_b' member". [...] > I do mean a lot of methods, not classes. I don't have any problem adding > the lines. It is just that, before I starting using pylint, it had not > occurred to me that there was any problem with my approach. If an > experienced python programmer was reviewing my code, would they flag it > as 'bad style'? *shrug* I wouldn't necessarily call it *bad*, but perhaps *not-quite good* style. I think its fine for a small projects and quick scripts, especially if they're written and maintained by a single person for their own use. Perhaps not so much for large projects intended for long-term use with continual development. If there really are a lot of such missing methods, I'd consider writing something like this: class A: def __init__(self, ...): ... # === process abstract methods en masse === for name in "method_a method_b method_c method_d".split(): @abstractmethod def inner(self): raise NotImplementedError inner.__name__ = name # This is okay, writing to locals works inside the class body. locals()[name] = inner del inner, name # Clean up the class namespace. def concrete_method_a(self): ... although to be honest I'm not sure if that would be enough to stop PyLint from complaining. -- Steven D'Aprano "Ever since I learned about confirmation bias, I've been seeing it everywhere." -- Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
"Thomas Jollans" wrote in message news:53faf0ef-4054-53fa-6179-a862495ea...@tjol.eu... On 2018-08-14 09:38, Frank Millman wrote: > Hi all > > Pylint is flagging a lot of lines as errors that I would consider to be > acceptable. > > I have an abstract class ClassA with a number of concrete sub-classes. > ClassA has a method which invokes 'self.method_b()' which is defined > separately on each sub-class. Pylint complains that "Instance of > 'ClassA' has no 'method_b' member". > > First question - as a matter of style, is Pylint correct? If so, I could > define 'method_b' in ClassA and raise NotImplementedError. Is this > considered more pythonic? The downside is that I have quite a few of > them, so it would add some clutter. I wouldn't say it's unpythonic per se, but if your ClassA is logically an abstract base class that requires certain methods to be implemented in subclasses, then it's probably clearer to use Python's abc [1] facilities, and declare your abstract methods as actual abstractmethods. [1]: https://docs.python.org/3/library/abc.html Thanks for the pointer - I will look into that alternative. [...] You *can* raise NotImplementedError in your abstractmethods, but I don't think it's really necessary. You need a statement in the method body of course, but since you're going to put a docstring there anyway (right?), that's already taken care of. At first I thought that it would be necessary, but then I saw in the docs that an ABC class "cannot be instantiated unless all of its abstract methods and properties are overridden", so there is no danger of forgetting to add it to the subclass. Frank -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On 2018-08-14 09:38, Frank Millman wrote: > Hi all > > Pylint is flagging a lot of lines as errors that I would consider to be > acceptable. > > I have an abstract class ClassA with a number of concrete sub-classes. > ClassA has a method which invokes 'self.method_b()' which is defined > separately on each sub-class. Pylint complains that "Instance of > 'ClassA' has no 'method_b' member". > > First question - as a matter of style, is Pylint correct? If so, I could > define 'method_b' in ClassA and raise NotImplementedError. Is this > considered more pythonic? The downside is that I have quite a few of > them, so it would add some clutter. I wouldn't say it's unpythonic per se, but if your ClassA is logically an abstract base class that requires certain methods to be implemented in subclasses, then it's probably clearer to use Python's abc [1] facilities, and declare your abstract methods as actual abstractmethods. [1]: https://docs.python.org/3/library/abc.html i.e. from abc import ABC, abstractmethod class ClassA(ABC): def do_stuff(self): return self.method_b(42)**3 @abstractmethod def method_b(self, answer): """ This is a great place to put a docstring """ You *can* raise NotImplementedError in your abstractmethods, but I don't think it's really necessary. You need a statement in the method body of course, but since you're going to put a docstring there anyway (right?), that's already taken care of. -- Thomas > Second question - if my present code is not unpythonic, is there an easy > way to suppress the error messages, without disabling 'no-member' > altogether? -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
"D'Arcy Cain" wrote in message news:865ed61a-cf1d-959f-f77e-dc586fe6e...@vybenetworks.com... On 2018-08-14 03:38 AM, Frank Millman wrote: > Hi all > > Pylint is flagging a lot of lines as errors that I would consider to be > acceptable. > > I have an abstract class ClassA with a number of concrete sub-classes. > ClassA has a method which invokes 'self.method_b()' which is defined > separately on each sub-class. Pylint complains that "Instance of > 'ClassA' has no 'method_b' member". > > First question - as a matter of style, is Pylint correct? If so, I could > define 'method_b' in ClassA and raise NotImplementedError. Is this > considered more pythonic? The downside is that I have quite a few of > them, so it would add some clutter. I would add the method. It's one line: def method_b(self): raise NotImplementedError When you say that you have quite a lot of them, what is "them"? Many master classes or many methods in the class? If the latter, one line each isn't so bad. If the former I wonder if a master, master class is called for. I do mean a lot of methods, not classes. I don't have any problem adding the lines. It is just that, before I starting using pylint, it had not occurred to me that there was any problem with my approach. If an experienced python programmer was reviewing my code, would they flag it as 'bad style'? I am also getting a funny smell from your description. Are you sure that you need to redefine the methods? Perhaps you just need to define some class variables and use one method. You can also define your own method and call the classA method inside it for common functionality. As an example, I have a master class defining a unit of data (i.e. the value of a column) retrieved from a database. I have separate sub-classes for each data type - text, integer, date, etc. To ensure that a value is valid before storing it as an instance attribute, I call a method called 'check_value'. The details of check_value vary according to the data type, but that is transparent to the piece of code that calls check_value(). Frank -- https://mail.python.org/mailman/listinfo/python-list
Re: Pylint false positives
On 2018-08-14 03:38 AM, Frank Millman wrote: > Hi all > > Pylint is flagging a lot of lines as errors that I would consider to be > acceptable. > > I have an abstract class ClassA with a number of concrete sub-classes. > ClassA has a method which invokes 'self.method_b()' which is defined > separately on each sub-class. Pylint complains that "Instance of > 'ClassA' has no 'method_b' member". > > First question - as a matter of style, is Pylint correct? If so, I could > define 'method_b' in ClassA and raise NotImplementedError. Is this > considered more pythonic? The downside is that I have quite a few of > them, so it would add some clutter. I would add the method. It's one line: def method_b(self): raise NotImplementedError When you say that you have quite a lot of them, what is "them"? Many master classes or many methods in the class? If the latter, one line each isn't so bad. If the former I wonder if a master, master class is called for. I am also getting a funny smell from your description. Are you sure that you need to redefine the methods? Perhaps you just need to define some class variables and use one method. You can also define your own method and call the classA method inside it for common functionality. Just my 2¢ based on no information about your actual application. -- D'Arcy J.M. Cain Vybe Networks Inc. http://www.VybeNetworks.com/ IM:da...@vex.net VoIP: sip:da...@vybenetworks.com -- https://mail.python.org/mailman/listinfo/python-list
