[issue1683368] object.__init__ shouldn't allow args/kwds
Changes by Terry J. Reedy tjre...@udel.edu: -- stage: - resolved type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Raymond Hettinger added the comment: Jason, I made some recommendations on this subject in my blog post a few years ago: http://rhettinger.wordpress.com/2011/05/26/super-considered-super/ ''' A more flexible approach is to have every method in the ancestor tree cooperatively designed to accept keyword arguments and a keyword-arguments dictionary, to remove any arguments that it needs, and to forward the remaining arguments using **kwds, eventually leaving the dictionary empty for the final call in the chain. Each level strips-off the keyword arguments that it needs so that the final empty dict can be sent to a method that expects no arguments at all (for example, object.__init__ expects zero arguments): class Shape: def __init__(self, shapename, **kwds): self.shapename = shapename super().__init__(**kwds) class ColoredShape(Shape): def __init__(self, color, **kwds): self.color = color super().__init__(**kwds) cs = ColoredShape(color='red', shapename='circle') ''' -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Changes by Guido van Rossum gu...@python.org: -- Removed message: http://bugs.python.org/msg219253 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Changes by Terry J. Reedy tjre...@udel.edu: -- Removed message: http://bugs.python.org/msg219255 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Guido van Rossum added the comment: If you don't know enough about the base class you shouldn't be subclassing it. In this particular case you should be overriding __init__, not __new__. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Terry J. Reedy added the comment: From what I see, you do not need to change either __new__ or __init__, just add __enter__ and __exit__ , and you only need to do that in 2.6. Since Zipfile is written in Python, you could monkey-patch instead of subclassing, if that is easier in your particular case. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Eric Snow added the comment: If you don't know enough about the base class you shouldn't be subclassing it. That's important when overriding any API in subclass and absolutely always essential when it comes to __new__ and __init__! That's something that isn't very obvious at first. :( In this particular case you should be overriding __init__, not __new__. Jason's code is doing something like OSError.__new__ does now, which returns an instance of a subclass depending on the errno. However, while the language supports it, I see that as a viable hack only when backward-compatibilty is a big concern. Otherwise I find factory classmethods to be a much better solution for discoverability and clarity of implementation. -- nosy: +eric.snow ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Guido van Rossum added the comment: Sorry, I didn't realize why __new__ was being used. But what Jason's code is doing isn't any cleaner than monkey-patching. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Jason R. Coombs added the comment: Maybe I should have focused on a more trivial example to demonstrate the place where my expectation was violated. The use of a real-world example is distracting from my intended point. Consider instead this abstract example: class SomeClass(SomeParentClass): def __new__(cls, *args, **kwargs): return super(SomeClass, cls).__new__(cls, *args, **kwargs) def __init__(self, *args, **kwargs): super(SomeClass, self).__init__(*args, **kwargs) Ignoring for a moment the incongruity of the invocation of __new__ with 'cls' due to __new__ being a staticmethod, the naive programmer expects the above SomeClass to work exactly like SomeParentClass because both overrides are implemented as a trivial pass-through. And indeed that technique will work just fine if the parent class implements both __init__ and __new__, but if the parent class (or one of its parents) does not implement either of those methods, the technique will fail, because the fall through to 'object' class. I believe this incongruity stems from the fact that __new__ and __init__ are special-cased not to be called if they aren't implemented on the class. Therefore, to write SomeClass without knowledge of the SomeParentClass implementation, one could write this instead: class SomeClass(SomeParentClass): def __new__(cls, *args, **kwargs): super_new = super(SomeClass, cls).__new__ if super_new is object.__new__: return super_new(cls) return super_new(cls, *args, **kwargs) def __init__(self, *args, **kwargs): super_init = super(SomeClass, self).__init__ if super_init.__objclass__ is object: return super_init(*args, **kwargs) Now that implementation is somewhat ugly and perhaps a bit brittle (particularly around use of __objclass__). Ignoring that for now, it does have the property that regardless of the class from which it derives, it will work, including: SomeParentClass = datetime.datetime # implements only __new__ SomeParentClass = zipfile.ZipFile # implements only __init__ class SomeParentClass: pass # implements neither __init__ nor __new__ While I would prefer a language construct that didn't require this dance for special casing (or similarly require the programmer to hard-code the dance to a specific implementation of a specific parent class as Guido recommends), at the very least I would suggest that the documentation better reflect this somewhat surprising behavior. Currently, the documentation states [https://docs.python.org/2/reference/datamodel.html#object.__new__] effectively Typical implementations of __new__ invoke the superclass’ __new__() method with appropriate arguments. It's left as an exercise to the reader to ascertain what 'appropriate arguments' are, and doesn't communicate that the introduction or omission of __new__ or __init__ to a class hierarchy affects the process by which a class is constructed/initialized. Greg Smith's blog demonstrates some even more dangerous cases. I don't understand why his concerns weren't addressed, because they seem legitimate, and I agree with his conclusion that the older behavior is more desirable, despite the concerns raised by the OP. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Jason R. Coombs added the comment: Based on the example above, I've created a blog post to publish my recommendation for overriding these special methods in a way that's safe regardless of the parent implementation, given the status quo: http://blog.jaraco.com/2014/05/how-to-safely-override-init-or-new-in.html -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Guido van Rossum added the comment: Hrm. I've always thought that the key point of cooperative MI was the term *cooperative*. Consider a regular (non-constructor) method. You must have a common base class that defines this method, and *that* method shouldn't be calling the super-method (because there isn't one). All cooperative classes extending this method must derive from that base class. It's the same for __init__ and __new__, except that you may treat each (keyword) argument as a separate method. But you must still have a point in the tree to eat that argument, and that point must not pass it up the super call chain. If in a particular framework you want unrecognized keyword arguments to the constructor to be ignored, you should define a common base class from which all your cooperative subclasses inherit. But given the prevalence of *single* inheritance, 'object' shouldn't be that common base class. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Jason R. Coombs added the comment: I recently ran into this error again. I was writing this class to provide backward-compatible context manager support for zipfile.ZipFile on Python 2.6 and 3.1: class ContextualZipFile(zipfile.ZipFile): Supplement ZipFile class to support context manager for Python 2.6 def __enter__(self): return self def __exit__(self, type, value, traceback): self.close() def __new__(cls, *args, **kwargs): Construct a ZipFile or ContextualZipFile as appropriate if hasattr(zipfile.ZipFile, '__exit__'): return zipfile.ZipFile(*args, **kwargs) return super(ContextualZipFile, cls).__new__(cls, *args, **kwargs) At the point where super is called, the author is unaware of the details of the function signature for zipfile.ZipFile.__new__, so simply passes the same arguments as were received by the derived class. However, this behavior raises a DeprecationWarning on Python 2.6 and 3.1 (and would raise an error on Python 3.2 if the code allowed it). What's surprising is that the one cannot simply override a constructor or initializer without knowing in advance which of those methods are implemented (and with what signature) on the parent class. It seems like the construction (calling of __new__) is special-cased for classes that don't implement __new__. What is the proper implementation of ContextualZipFile.__new__? Should it use super but omit the args and kwargs? Should it call object.__new__ directly? Should it check for the existence of __new__ on the parent class (or compare it to object.__new__)? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Jason R. Coombs added the comment: I recently ran into this error again. I was writing this class to provide backward-compatible context manager support for zipfile.ZipFile on Python 2.6 and 3.1: class ContextualZipFile(zipfile.ZipFile): Supplement ZipFile class to support context manager for Python 2.6 def __enter__(self): return self def __exit__(self, type, value, traceback): self.close() def __new__(cls, *args, **kwargs): Construct a ZipFile or ContextualZipFile as appropriate if hasattr(zipfile.ZipFile, '__exit__'): return zipfile.ZipFile(*args, **kwargs) return super(ContextualZipFile, cls).__new__(cls, *args, **kwargs) At the point where super is called, the author is unaware of the details of the function signature for zipfile.ZipFile.__new__, so simply passes the same arguments as were received by the derived class. However, this behavior raises a DeprecationWarning on Python 2.6 and 3.1 (and would raise an error on Python 3.2 if the code allowed it). What's surprising is that the one cannot simply override a constructor or initializer without knowing in advance which of those methods are implemented (and with what signature) on the parent class. It seems like the construction (calling of __new__) is special-cased for classes that don't implement __new__. What is the proper implementation of ContextualZipFile.__new__? Should it use super but omit the args and kwargs? Should it call object.__new__ directly? Should it check for the existence of __new__ on the parent class (or compare it to object.__new__)? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Jason R. Coombs added the comment: For reference, I encountered an issue due to this change and didn't quite understand what was going on. I distilled the problem down and posted a question on stack overflow: http://stackoverflow.com/questions/14300153/why-does-this-datetime-subclass-fail-on-python-3/14324503#14324503 The answer led me here, so now I understand. I wanted to share this use-case for posterity. I didn't find anything in the what's new documents for Python 3.3 or 3.0. Was this fundamental signature change to all objects documented anywhere? Any objection if I draft a change to the docs? -- nosy: +jason.coombs ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Terry J. Reedy added the comment: First, What's New explains the new features in Python. This issue is a bugfix. AFAIK, object() has always been documented as having no parameters. The fact that passing extra args should raise a TypeError is no secret. Second, this *is* documented. The third sentence of http://docs.python.org/3/whatsnew/3.3.html is For full details, see the changelog. We really mean that ;-). The changelog is derived from Misc/NEWS in the repository. It says Issue #1683368: object.__new__ and object.__init__ raise a TypeError if they are passed arguments and their complementary method is not overridden. That is prefixed by Issue #1683368:, which links to this issue. This entry is easily found by searching for 'object.__init__' (or a sufficient prefix thereof). For 3.2, the What's New sentence was For full details, see the Misc/NEWS file and the link went to the raw repository file. http://hg.python.org/cpython/file/3.2/Misc/NEWS My impression is that this issue played a role in including the prettified version, instead of just the repository link, in the on-line version of the docs. What's New for 2.7 does not even have the link. In any case, *any* bugfix breaks code that depends on the bug. Hence the decision to make the full changelog more available and more readable. I realize that the change to the header for What's New is hard to miss. But what are we to do? Add a new What's New in What's New doc for one release? Put the change in flashing red type? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Jason R. Coombs added the comment: Aah. Indeed, that's where I should have looked. Thanks for the pointer. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 25b71858cb14 by Benjamin Peterson in branch 'default': make extra arguments to object.__init__/__new__ to errors in most cases (finishes #1683368) http://hg.python.org/cpython/rev/25b71858cb14 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Guido van Rossum gu...@python.org added the comment: Please don't add python-...@python.org to the nosy list. -- nosy: -python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Benjamin Peterson benja...@python.org added the comment: python-dev is just the name of the robot which notes records changesets. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
dauerbaustelle jo...@lophus.org added the comment: What exactly is the correct solution with Python 2.6 to avoid this warning? My use case is something like class myunicode(unicode): def __init__(self, *args, **kwargs): unicode.__init__(self, *args, **kwargs) self.someattribute = calculate_attribute_once() Shall I overwrite __new__ rather than __init__? Or what :-) -- nosy: +dauerbaustelle ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Terry J. Reedy tjre...@udel.edu added the comment: @dauerbaustelle I believe your question is a separate issue and that it should have been asked on Python list. However, yes, subclasses of immutables must override __new__. For more, do ask on the list, not here. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Gregory P. Smith g...@krypto.org added the comment: FYI - A discussion on why this change may have been a bad idea and breaks peoples existing code: http://freshfoo.com/blog/object__init__takes_no_parameters -- nosy: +gregory.p.smith ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Changes by Kirit Sælensminde ki...@felspar.com: -- nosy: +KayEss ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Changes by Jesús Cea Avión [EMAIL PROTECTED]: -- nosy: +jcea ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue1683368 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Changes by Guido van Rossum [EMAIL PROTECTED]: -- status: open - closed _ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1683368 _ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Benjamin Peterson [EMAIL PROTECTED] added the comment: Can this be closed? -- nosy: +benjamin.peterson _ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1683368 _ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1683368] object.__init__ shouldn't allow args/kwds
Changes by Guido van Rossum: -- versions: +Python 3.0 _ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1683368 _ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
