Re: [Python-Dev] subclassing builtin data structures

[Steven D'Aprano]

On Fri, Feb 13, 2015 at 06:03:35PM -0500, Neil Girdhar wrote:
 I personally don't think this is a big enough issue to warrant any changes,
 but I think Serhiy's solution would be the ideal best with one additional
 parameter: the caller's type.  Something like
 
 def __make_me__(self, cls, *args, **kwargs)
 
 and the idea is that any time you want to construct a type, instead of
 
 self.__class__(assumed arguments…)
 
 where you are not sure that the derived class' constructor knows the right
 argument types, you do
 
 def SomeCls:
  def some_method(self, ...):
return self.__make_me__(SomeCls, assumed arguments…)
 
 Now the derived class knows who is asking for a copy.

What if you wish to return an instance from a classmethod? You don't 
have a `self` available.

class SomeCls:
def __init__(self, x, y, z):
...
@classmethod
def from_spam(cls, spam):
x, y, z = process(spam)
return cls.__make_me__(self, cls, x, y, z)  # oops, no self


Even if you are calling from an instance method, and self is available, 
you cannot assume that the information needed for the subclass 
constructor is still available. Perhaps that information is used in the 
constructor and then discarded.

The problem we wish to solve is that when subclassing, methods of some
base class blindly return instances of itself, instead of self's type:


py class MyInt(int):
... pass
...
py n = MyInt(23)
py assert isinstance(n, MyInt)
py assert isinstance(n+1, MyInt)
Traceback (most recent call last):
  File stdin, line 1, in ?
AssertionError


The means that subclasses often have to override all the parent's 
methods, just to ensure the type is correct:

class MyInt(int):
def __add__(self, other):
o = super().__add__(other)
if o is not NotImplemented:
o = type(self)(o)
return o


Something like that, repeated for all the int methods, should work:

py n = MyInt(23)
py type(n+1)
class '__main__.MyInt'


This is tedious and error prone, but at least once it is done, 
subclasses of MyInt will Just Work:


py class MyOtherInt(MyInt):
... pass
...
py a = MyOtherInt(42)
py type(a + 1000)
class '__main__.MyOtherInt'


(At least, *in general* they will work. See below.)

So, why not have int's methods use type(self) instead of hard coding 
int? The answer is that *some* subclasses might override the 
constructor, which would cause the __add__ method to fail:

# this will fail if the constructor has a different signature
o = type(self)(o)


Okay, but changing the constructor signature is quite unusual. Mostly, 
people subclass to add new methods or attributes, or to override a 
specific method. The dict/defaultdict situation is relatively uncommon.

Instead of requiring *every* subclass to override all the methods, 
couldn't we require the base classes (like int) to assume that the 
signature is unchanged and call type(self), and leave it up to the 
subclass to override all the methods *only* if the signature has 
changed? (Which they probably would have to do anyway.)

As the MyInt example above shows, or datetime in the standard library, 
this actually works fine in practice:

py from datetime import datetime
py class MySpecialDateTime(datetime):
... pass
...
py t = MySpecialDateTime.today()
py type(t)
class '__main__.MySpecialDateTime'


Why can't int, str, list, tuple etc. be more like datetime?



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Nick Coghlan]

On 14 Feb 2015 08:57, Alexander Belopolsky alexander.belopol...@gmail.com
wrote:


 On Fri, Feb 13, 2015 at 4:44 PM, Neil Girdhar mistersh...@gmail.com
wrote:

 Interesting:
http://stackoverflow.com/questions/5490824/should-constructors-comply-with-the-liskov-substitution-principle


 Let me humbly conjecture that the people who wrote the top answers have
background in less capable languages than Python.

 Not every language allows you to call self.__class__().  In the languages
that don't you can get away with incompatible constructor signatures.

 However, let me try to focus the discussion on a specific issue before we
go deep into OOP theory.

 With python's standard datetime.date we have:

  from datetime import *
  class Date(date):
 ... pass
 ...
  Date.today()
 Date(2015, 2, 13)
  Date.fromordinal(1)
 Date(1, 1, 1)

 Both .today() and .fromordinal(1) will break in a subclass that redefines
__new__ as follows:

  class Date2(date):
 ... def __new__(cls, ymd):
 ... return date.__new__(cls, *ymd)
 ...
  Date2.today()
 Traceback (most recent call last):
   File stdin, line 1, in module
 TypeError: __new__() takes 2 positional arguments but 4 were given
  Date2.fromordinal(1)
 Traceback (most recent call last):
   File stdin, line 1, in module
 TypeError: __new__() takes 2 positional arguments but 4 were given

 Why is this acceptable, but we have to sacrifice the convenience of
having Date + timedelta
 return Date to make it work  with Date2:

  Date2((1,1,1)) + timedelta(1)
 datetime.date(1, 1, 2)

Coupling alternative constructors to the default constructor signature is
pretty normal - it just means that if you override the signature of the
default constructor, you may need to update the alternative ones
accordingly.

Cheers,
Nick.






 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Nick Coghlan]

On 14 Feb 2015 07:39, Isaac Schwabacher ischwabac...@wisc.edu wrote:

 On 15-02-13, Guido van Rossum  wrote:
  Are you willing to wait 10 days for an answer? I#39;m out of round
tuits for a while.

 IIUC, the argument is that the Liskov Substitution Principle is a
statement about how objects of a subtype behave relative to objects of a
supertype, and it doesn't apply to constructors because they aren't
behaviors of existing objects. So other overriding methods *should* be able
to handle the same inputs that the respective overridden methods do, but
constructors don't need to. Even though __init__ is written as an instance
method, it seems like it's morally a part of the class method __new__
that's only split off for convenience.

A potentially helpful example is to consider a type system where Square is
a subclass of Rectangle.

To specify a Rectangle takes a height and a width, but a Square only needs
the length of one side, and letting the height and width be specified
independently would be outright wrong.

Many possible operations on a Square also *should* formally return a
Rectangle, as doing something like doubling the height gives you a result
that isn't a square any more. (That's also why shapes must be immutable for
this particular type hierarchy to make any sense)

(You can construct similar examples for Circle  Ellipse, and Python's
numeric hierarchy is a real-world example of some of the complexity that
can arise)

There's simply no sensible default behaviour other than the status quo in
the face of scenarios like that - whether or not there's a better answer
than return an instance of the parent class for any given inherited
method depends entirely on the invariants of the subclass and how they
differ from those of the parent class.

It's certainly possible to write methods that return a new instance of the
current type (that's common in alternative constructors, for example), but
it usually involves placing additional expectations on the developers of
subclasses. That's most commonly seen within the confines of a single
project, or when defining a development framework, rather than in libraries
that choose to expose some public types and also supports their use as base
types.

Cheers,
Nick.


 If this message is unclear, it's because I don't really understand this
myself and I'm trying to articulate my best understanding of what's been
said on this thread and those it links to.

 ijs

  On Fri, Feb 13, 2015 at 10:22 AM, Alexander Belopolsky 
alexander.belopol...@gmail.com(javascript:main.compose() wrote:
 
  
   On Fri, Feb 13, 2015 at 1:19 PM, Alexander Belopolsky 
alexander.belopol...@gmail.com(javascript:main.compose() wrote:
   
FWIW you#39;re wrong when you claim that a constructor is no
different from any other method. Someone else should probably explain this
(it#39;s an old argument that#39;s been thoroughly settled).
   
   
Well, the best answer I#39;ve got in the past [1] was ask on
python-dev since Guido called the operator overriding expectation. :-)
  
  
   And let me repost this bit of history [1]:
  
   Here is the annotated pre-r82065 code:
  
   39876 gvanrossum def __add__(self, other):
   39876 gvanrossum if isinstance(other, timedelta):
   39928 gvanrossum return self.__class__(self.__days + other.__days,
   39876 gvanrossum self.__seconds + other.__seconds,
   39876 gvanrossum self.__microseconds + other.__microseconds)
   40207 tim_one return NotImplemented
   39876 gvanrossum
  
  
  
   [1] http://bugs.python.org/issue2267#msg125979
  
  
 
 
 
 
  --
  --Guido van Rossum (python.org/~guido(http://python.org/~guido))
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Steve Dower]

Instead of requiring *every* subclass to override all the methods,
couldn't we require the base classes (like int) to assume that the
signature is unchanged and call type(self), and leave it up to the
subclass to override all the methods *only* if the signature has
changed?

I assumed everyone was just saying this point over and over, so I haven't been 
following the thread closely. This is precisely how inheritance works: 
subclasses are constrained by the base class. If you want to play, you *must* 
play by its rules. (Else, use composition.)

It's fine for base classes to assume a compatible constructor, and if builtins 
can do it without hurting performance for the non-subclassed case, I don't see 
why not.

Cheers,
Steve

Top-posted from my Windows Phone

From: Steven D'Apranomailto:st...@pearwood.info
Sent: ‎2/‎14/‎2015 4:24
To: python-dev@python.orgmailto:python-dev@python.org
Subject: Re: [Python-Dev] subclassing builtin data structures

On Fri, Feb 13, 2015 at 06:03:35PM -0500, Neil Girdhar wrote:
 I personally don't think this is a big enough issue to warrant any changes,
 but I think Serhiy's solution would be the ideal best with one additional
 parameter: the caller's type.  Something like

 def __make_me__(self, cls, *args, **kwargs)

 and the idea is that any time you want to construct a type, instead of

 self.__class__(assumed arguments…)

 where you are not sure that the derived class' constructor knows the right
 argument types, you do

 def SomeCls:
  def some_method(self, ...):
return self.__make_me__(SomeCls, assumed arguments…)

 Now the derived class knows who is asking for a copy.

What if you wish to return an instance from a classmethod? You don't
have a `self` available.

class SomeCls:
def __init__(self, x, y, z):
...
@classmethod
def from_spam(cls, spam):
x, y, z = process(spam)
return cls.__make_me__(self, cls, x, y, z)  # oops, no self


Even if you are calling from an instance method, and self is available,
you cannot assume that the information needed for the subclass
constructor is still available. Perhaps that information is used in the
constructor and then discarded.

The problem we wish to solve is that when subclassing, methods of some
base class blindly return instances of itself, instead of self's type:


py class MyInt(int):
... pass
...
py n = MyInt(23)
py assert isinstance(n, MyInt)
py assert isinstance(n+1, MyInt)
Traceback (most recent call last):
  File stdin, line 1, in ?
AssertionError


The means that subclasses often have to override all the parent's
methods, just to ensure the type is correct:

class MyInt(int):
def __add__(self, other):
o = super().__add__(other)
if o is not NotImplemented:
o = type(self)(o)
return o


Something like that, repeated for all the int methods, should work:

py n = MyInt(23)
py type(n+1)
class '__main__.MyInt'


This is tedious and error prone, but at least once it is done,
subclasses of MyInt will Just Work:


py class MyOtherInt(MyInt):
... pass
...
py a = MyOtherInt(42)
py type(a + 1000)
class '__main__.MyOtherInt'


(At least, *in general* they will work. See below.)

So, why not have int's methods use type(self) instead of hard coding
int? The answer is that *some* subclasses might override the
constructor, which would cause the __add__ method to fail:

# this will fail if the constructor has a different signature
o = type(self)(o)


Okay, but changing the constructor signature is quite unusual. Mostly,
people subclass to add new methods or attributes, or to override a
specific method. The dict/defaultdict situation is relatively uncommon.

Instead of requiring *every* subclass to override all the methods,
couldn't we require the base classes (like int) to assume that the
signature is unchanged and call type(self), and leave it up to the
subclass to override all the methods *only* if the signature has
changed? (Which they probably would have to do anyway.)

As the MyInt example above shows, or datetime in the standard library,
this actually works fine in practice:

py from datetime import datetime
py class MySpecialDateTime(datetime):
... pass
...
py t = MySpecialDateTime.today()
py type(t)
class '__main__.MySpecialDateTime'


Why can't int, str, list, tuple etc. be more like datetime?



--
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Alexander Belopolsky]

On Sat, Feb 14, 2015 at 7:23 AM, Steven D'Aprano st...@pearwood.info
wrote:

 Why can't int, str, list, tuple etc. be more like datetime?


They are.  In all these types, class methods call subclass constructors but
instance methods don't.

 class Int(int):
... pass
...
 Int.from_bytes(bytes([1,2,3]), 'big')
66051
 type(_)
class '__main__.Int'

 Int(1) + 1
2
 type(_)
class 'int'

In the case of int, there is a good reason for this behavior - bool.  In
python, we want True + True == 2.  In numpy, where binary operations
preserve subclasses, you have

 import numpy
 numpy.bool_(1) + numpy.bool_(1)
True

I don't see a similar argument for the date class, however.  Given
date.{to|from}ordinal(), date subclasses are pretty much bound to have
timedelta addition satisfy (d + td).toordinal() == d.toordinal() +
td.days.  Any other definition would be fighting the baseclass design and
would be better implemented via containment.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Georg Brandl]

On 02/14/2015 07:26 PM, Alexander Belopolsky wrote:

 In the case of int, there is a good reason for this behavior - bool.  In 
 python,
 we want True + True == 2.  In numpy, where binary operations preserve
 subclasses, you have
 
 import numpy
 numpy.bool_(1) + numpy.bool_(1)
 True

I don't think numpy.bool_ subclasses some class like numpy.int_.

Georg

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Neil Girdhar]

I think the __make_me__ pattern discussed earlier is still the most generic
cooperative solution.  Here it is with a classmethod version too:

class C(D, E):
def some_method(self):
return __make_me__(self, C)

def __make_me__(self, arg_cls, *args, **kwargs):
if arg_cls is C:
pass
elif issubclass(D, arg_cls):
args, kwargs = modified_args_for_D(args, kwargs)
elif issubclass(E, arg_cls):
args, kwargs = modified_args_for_D(args, kwargs)
else:
raise ValueError

if self.__class__ == C:
return C(*args, **kwargs)
return self.__make_me__(C, *args, **kwargs)

@classmethod
def __make_me_cls__(cls, arg_cls, *args, **kwargs):
if arg_cls is C:
pass
elif issubclass(D, arg_cls):
args, kwargs = modified_args_for_D(args, kwargs)
elif issubclass(E, arg_cls):
args, kwargs = modified_args_for_D(args, kwargs)
else:
raise ValueError

if cls == C:
return C(*args, **kwargs)
return cls.__make_me_cls__(C, *args, **kwargs)


On Sat, Feb 14, 2015 at 7:23 AM, Steven D'Aprano st...@pearwood.info
wrote:

 On Fri, Feb 13, 2015 at 06:03:35PM -0500, Neil Girdhar wrote:
  I personally don't think this is a big enough issue to warrant any
 changes,
  but I think Serhiy's solution would be the ideal best with one additional
  parameter: the caller's type.  Something like
 
  def __make_me__(self, cls, *args, **kwargs)
 
  and the idea is that any time you want to construct a type, instead of
 
  self.__class__(assumed arguments…)
 
  where you are not sure that the derived class' constructor knows the
 right
  argument types, you do
 
  def SomeCls:
   def some_method(self, ...):
 return self.__make_me__(SomeCls, assumed arguments…)
 
  Now the derived class knows who is asking for a copy.

 What if you wish to return an instance from a classmethod? You don't
 have a `self` available.

 class SomeCls:
 def __init__(self, x, y, z):
 ...
 @classmethod
 def from_spam(cls, spam):
 x, y, z = process(spam)
 return cls.__make_me__(self, cls, x, y, z)  # oops, no self


 Even if you are calling from an instance method, and self is available,
 you cannot assume that the information needed for the subclass
 constructor is still available. Perhaps that information is used in the
 constructor and then discarded.

 The problem we wish to solve is that when subclassing, methods of some
 base class blindly return instances of itself, instead of self's type:


 py class MyInt(int):
 ... pass
 ...
 py n = MyInt(23)
 py assert isinstance(n, MyInt)
 py assert isinstance(n+1, MyInt)
 Traceback (most recent call last):
   File stdin, line 1, in ?
 AssertionError


 The means that subclasses often have to override all the parent's
 methods, just to ensure the type is correct:

 class MyInt(int):
 def __add__(self, other):
 o = super().__add__(other)
 if o is not NotImplemented:
 o = type(self)(o)
 return o


 Something like that, repeated for all the int methods, should work:

 py n = MyInt(23)
 py type(n+1)
 class '__main__.MyInt'


 This is tedious and error prone, but at least once it is done,
 subclasses of MyInt will Just Work:


 py class MyOtherInt(MyInt):
 ... pass
 ...
 py a = MyOtherInt(42)
 py type(a + 1000)
 class '__main__.MyOtherInt'


 (At least, *in general* they will work. See below.)

 So, why not have int's methods use type(self) instead of hard coding
 int? The answer is that *some* subclasses might override the
 constructor, which would cause the __add__ method to fail:

 # this will fail if the constructor has a different signature
 o = type(self)(o)


 Okay, but changing the constructor signature is quite unusual. Mostly,
 people subclass to add new methods or attributes, or to override a
 specific method. The dict/defaultdict situation is relatively uncommon.

 Instead of requiring *every* subclass to override all the methods,
 couldn't we require the base classes (like int) to assume that the
 signature is unchanged and call type(self), and leave it up to the
 subclass to override all the methods *only* if the signature has
 changed? (Which they probably would have to do anyway.)

 As the MyInt example above shows, or datetime in the standard library,
 this actually works fine in practice:

 py from datetime import datetime
 py class MySpecialDateTime(datetime):
 ... pass
 ...
 py t = MySpecialDateTime.today()
 py type(t)
 class '__main__.MySpecialDateTime'


 Why can't int, str, list, tuple etc. be more like datetime?



 --
 Steve
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/mistersheik%40gmail.com

Re: [Python-Dev] subclassing builtin data structures

[Neil Girdhar]

Oops, I meant to call super if necessary:

@classmethod
def __make_me_cls__(cls, arg_cls, *args, **kwargs):
if arg_cls is C:
pass
elif arg_cls is D:
args, kwargs = modified_args_for_D(args, kwargs)
elif arg_cls is E:
args, kwargs = modified_args_for_D(args, kwargs)
else:
return super().__make_me_cls__(arg_cls, args, kwargs)

if cls is C:
return C(*args, **kwargs)
return cls.__make_me_cls__(C, *args, **kwargs)


On Sat, Feb 14, 2015 at 3:15 PM, Neil Girdhar mistersh...@gmail.com wrote:

 I think the __make_me__ pattern discussed earlier is still the most
 generic cooperative solution.  Here it is with a classmethod version too:

 class C(D, E):
 def some_method(self):
 return __make_me__(self, C)

 def __make_me__(self, arg_cls, *args, **kwargs):
 if arg_cls is C:
 pass
 elif issubclass(D, arg_cls):
 args, kwargs = modified_args_for_D(args, kwargs)
 elif issubclass(E, arg_cls):
 args, kwargs = modified_args_for_D(args, kwargs)
 else:
 raise ValueError

 if self.__class__ == C:
 return C(*args, **kwargs)
 return self.__make_me__(C, *args, **kwargs)

 @classmethod
 def __make_me_cls__(cls, arg_cls, *args, **kwargs):
 if arg_cls is C:
 pass
 elif issubclass(D, arg_cls):
 args, kwargs = modified_args_for_D(args, kwargs)
 elif issubclass(E, arg_cls):
 args, kwargs = modified_args_for_D(args, kwargs)
 else:
 raise ValueError

 if cls == C:
 return C(*args, **kwargs)
 return cls.__make_me_cls__(C, *args, **kwargs)


 On Sat, Feb 14, 2015 at 7:23 AM, Steven D'Aprano st...@pearwood.info
 wrote:

 On Fri, Feb 13, 2015 at 06:03:35PM -0500, Neil Girdhar wrote:
  I personally don't think this is a big enough issue to warrant any
 changes,
  but I think Serhiy's solution would be the ideal best with one
 additional
  parameter: the caller's type.  Something like
 
  def __make_me__(self, cls, *args, **kwargs)
 
  and the idea is that any time you want to construct a type, instead of
 
  self.__class__(assumed arguments…)
 
  where you are not sure that the derived class' constructor knows the
 right
  argument types, you do
 
  def SomeCls:
   def some_method(self, ...):
 return self.__make_me__(SomeCls, assumed arguments…)
 
  Now the derived class knows who is asking for a copy.

 What if you wish to return an instance from a classmethod? You don't
 have a `self` available.

 class SomeCls:
 def __init__(self, x, y, z):
 ...
 @classmethod
 def from_spam(cls, spam):
 x, y, z = process(spam)
 return cls.__make_me__(self, cls, x, y, z)  # oops, no self


 Even if you are calling from an instance method, and self is available,
 you cannot assume that the information needed for the subclass
 constructor is still available. Perhaps that information is used in the
 constructor and then discarded.

 The problem we wish to solve is that when subclassing, methods of some
 base class blindly return instances of itself, instead of self's type:


 py class MyInt(int):
 ... pass
 ...
 py n = MyInt(23)
 py assert isinstance(n, MyInt)
 py assert isinstance(n+1, MyInt)
 Traceback (most recent call last):
   File stdin, line 1, in ?
 AssertionError


 The means that subclasses often have to override all the parent's
 methods, just to ensure the type is correct:

 class MyInt(int):
 def __add__(self, other):
 o = super().__add__(other)
 if o is not NotImplemented:
 o = type(self)(o)
 return o


 Something like that, repeated for all the int methods, should work:

 py n = MyInt(23)
 py type(n+1)
 class '__main__.MyInt'


 This is tedious and error prone, but at least once it is done,
 subclasses of MyInt will Just Work:


 py class MyOtherInt(MyInt):
 ... pass
 ...
 py a = MyOtherInt(42)
 py type(a + 1000)
 class '__main__.MyOtherInt'


 (At least, *in general* they will work. See below.)

 So, why not have int's methods use type(self) instead of hard coding
 int? The answer is that *some* subclasses might override the
 constructor, which would cause the __add__ method to fail:

 # this will fail if the constructor has a different signature
 o = type(self)(o)


 Okay, but changing the constructor signature is quite unusual. Mostly,
 people subclass to add new methods or attributes, or to override a
 specific method. The dict/defaultdict situation is relatively uncommon.

 Instead of requiring *every* subclass to override all the methods,
 couldn't we require the base classes (like int) to assume that the
 signature is unchanged and call type(self), and leave it up to the
 subclass to override all the methods *only* if the signature has
 changed? (Which they probably would have to do 

Re: [Python-Dev] subclassing builtin data structures

[Alexander Belopolsky]

On Sat, Feb 14, 2015 at 2:36 PM, Georg Brandl g.bra...@gmx.net wrote:

  In the case of int, there is a good reason for this behavior - bool.  In
 python,
  we want True + True == 2.  In numpy, where binary operations preserve
  subclasses, you have
 
  import numpy
  numpy.bool_(1) + numpy.bool_(1)
  True

 I don't think numpy.bool_ subclasses some class like numpy.int_.


And  numpy.bool_ subclasses don't preserve type in addition:

 import numpy
 class Bool(numpy.bool_):
... pass
...
 numpy.bool_.mro()
[class 'numpy.bool_', class 'numpy.generic', class 'object']
 Bool(1) + Bool(1)
True
 type(_)
class 'numpy.bool_'

So there goes my theory. :-)

I think all these examples just highlight the need for a clear guidance
when self.__class__() can be called in base classes to construct instances
of derived classes.

Apparently numpy has it both ways.  One way for scalars (see above) and the
other for arrays:

 class Array(numpy.ndarray):
... pass
...
 a = Array(1)
 a[0] = 1
 a+a
Array([ 2.])
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Steven D'Aprano]

On Sat, Feb 14, 2015 at 01:26:36PM -0500, Alexander Belopolsky wrote:
 On Sat, Feb 14, 2015 at 7:23 AM, Steven D'Aprano st...@pearwood.info
 wrote:
 
  Why can't int, str, list, tuple etc. be more like datetime?
 
 
 They are.  In all these types, class methods call subclass constructors but
 instance methods don't.

But in datetime, instance methods *do*.

Sorry that my example with .today() was misleading.

py from datetime import datetime
py class MyDatetime(datetime):
... pass
...
py MyDatetime.today()
MyDatetime(2015, 2, 15, 12, 45, 38, 429269)
py MyDatetime.today().replace(day=20)
MyDatetime(2015, 2, 20, 12, 45, 53, 405889)


 In the case of int, there is a good reason for this behavior - bool.  In
 python, we want True + True == 2.

Sure. But bool is only one subclass. I expect that it should be bool's 
responsibility to override __add__ etc. to return an instance of the 
parent class (int) rather have nearly all subclasses have to override 
__add__ etc. to return instances of themselves.



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Jonas Wielicki]

If I may humbly chime in this, with a hint...

On 13.02.2015 05:01, Guido van Rossum wrote:
 On Thu, Feb 12, 2015 at 7:41 PM, Ethan Furman et...@stoneleaf.us wrote:
 [snip]
 2) always use the type of self when creating new instances

pros:
  - subclasses automatically maintain type
  - much less code in the simple cases [1]

cons:
  - if constructor signatures change, must override all methods which
create new objects

 Unless there are powerful reasons against number 2 (such as performance,
 or the effort to affect the change), it sure
 seems like the nicer way to go.

 So back to my original question: what other concerns are there, and has
 anybody done any benchmarks?

 
 Con for #2 is a showstopper. Forget about it.

I would like to mention that there is another language out there which
knows about virtual constructors (virtual like in virtual methods, with
signature match requirements and such), which is FreePascal (and Delphi,
and I think original Object Pascal too).

It is actually a feature I liked about these languages, compared to
C++03 and others, that constructors could be virtual and that classes
were first-class citizens.

Of course, Python cannot check the signature at compile time. But I
think as long as it is documented, there should be no reason not to
allow and support it. It really is analogous to other methods which need
to have a matching signature.

just my two cents,
jwi






signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Neil Girdhar]

With Python's cooperative inheritance, I think you want to do everything
through one constructor sending keyword arguments up the chain.  The
keyword arguments are popped off as needed.  With this setup I don't think
you need overloaded constructors.

Best,
Neil

On Fri, Feb 13, 2015 at 4:44 AM, Jonas Wielicki j.wieli...@sotecware.net
wrote:

 If I may humbly chime in this, with a hint...

 On 13.02.2015 05:01, Guido van Rossum wrote:
  On Thu, Feb 12, 2015 at 7:41 PM, Ethan Furman et...@stoneleaf.us
 wrote:
  [snip]
  2) always use the type of self when creating new instances
 
 pros:
   - subclasses automatically maintain type
   - much less code in the simple cases [1]
 
 cons:
   - if constructor signatures change, must override all methods which
 create new objects
 
  Unless there are powerful reasons against number 2 (such as performance,
  or the effort to affect the change), it sure
  seems like the nicer way to go.
 
  So back to my original question: what other concerns are there, and has
  anybody done any benchmarks?
 
 
  Con for #2 is a showstopper. Forget about it.

 I would like to mention that there is another language out there which
 knows about virtual constructors (virtual like in virtual methods, with
 signature match requirements and such), which is FreePascal (and Delphi,
 and I think original Object Pascal too).

 It is actually a feature I liked about these languages, compared to
 C++03 and others, that constructors could be virtual and that classes
 were first-class citizens.

 Of course, Python cannot check the signature at compile time. But I
 think as long as it is documented, there should be no reason not to
 allow and support it. It really is analogous to other methods which need
 to have a matching signature.

 just my two cents,
 jwi





 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/mistersheik%40gmail.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Jonas Wielicki]

If I may humbly chime in this, with a hint...

On 13.02.2015 05:01, Guido van Rossum wrote:
 On Thu, Feb 12, 2015 at 7:41 PM, Ethan Furman et...@stoneleaf.us wrote:
 [snip]
 2) always use the type of self when creating new instances

pros:
  - subclasses automatically maintain type
  - much less code in the simple cases [1]

cons:
  - if constructor signatures change, must override all methods which
create new objects

 Unless there are powerful reasons against number 2 (such as performance,
 or the effort to affect the change), it sure
 seems like the nicer way to go.

 So back to my original question: what other concerns are there, and has
 anybody done any benchmarks?

 
 Con for #2 is a showstopper. Forget about it.

I would like to mention that there is another language out there which
knows about virtual constructors (virtual like in virtual methods, with
signature match requirements and such), which is FreePascal (and Delphi,
and I think original Object Pascal too).

It is actually a feature I liked about these languages, compared to
C++03 and others, that constructors could be virtual and that classes
were first-class citizens.

Of course, Python cannot check the signature at compile time. But I
think as long as it is documented, there should be no reason not to
allow and support it. It really is analogous to other methods which need
to have a matching signature.

just my two cents,
jwi




signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Alexander Belopolsky]

On Thu, Feb 12, 2015 at 11:01 PM, Guido van Rossum gu...@python.org wrote:


 2) always use the type of self when creating new instances
 ..
cons:
  - if constructor signatures change, must override all methods which
create new objects


 Con for #2 is a showstopper. Forget about it.


Sorry if I am missing something obvious, but I still don't understand why
the same logic does not apply to class methods that create new instances:

 from datetime import *
 date.today()
datetime.date(2015, 2, 13)
 datetime.today()
datetime.datetime(2015, 2, 13, 11, 37, 23, 678680)
 class Date(date):
... pass
...
 Date.today()
Date(2015, 2, 13)

(I actually find datetime.today() returning a datetime rather than a date a
questionable design decision, but probably the datetime type should not
have been a subclass of the date to begin with.)

Are there any date subclasses in the wild that don't accept year, month,
day in the constructor?  If you create such a class, wouldn't you want to
override __add__ and friends anyways?  We already know that you will have
to override today().
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Guido van Rossum]

On Fri, Feb 13, 2015 at 10:02 AM, Alexander Belopolsky 
alexander.belopol...@gmail.com wrote:


 On Fri, Feb 13, 2015 at 12:35 PM, Guido van Rossum gu...@python.org
 wrote:

 IIUC you're proposing that the base class should *try* to construct an
 instance of the subclass by calling the type with an argument, and fail if
 it doesn't work. But that makes the whole thing brittle in the light of
 changes to the subclass constructor. Also, what should the argument be? The
 only answer I can think of is an instance of the base class.


 No.  The arguments should be whatever arguments are appropriate for the
 baseclass's __init__ or __new__.  In the case of datetime.date that would
 be year, month, day.


Agreed. (I was thinking of the case that Ethan brought up, which used int
as an example.)


 Note that the original pure python prototype of the datetime module had
 date.__add__ and friends call self.__class__(year, month, day).
 Unfortunately, it looks like the original sandbox did not survive the the
 hg conversion, so I cannot provide a link to the relevant history.


FWIW you're wrong when you claim that a constructor is no different from
any other method. Someone else should probably explain this (it's an old
argument that's been thoroughly settled).

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Alexander Belopolsky]

On Fri, Feb 13, 2015 at 1:19 PM, Alexander Belopolsky 
alexander.belopol...@gmail.com wrote:

 FWIW you're wrong when you claim that a constructor is no different
from any other method. Someone else should probably explain this (it's an
old argument that's been thoroughly settled).


 Well, the best answer I've got in the past [1] was ask on python-dev
since Guido called the operator overriding expectation. :-)


And let me repost this bit of history [1]:

Here is the annotated pre-r82065 code:

 39876 gvanrossum def __add__(self, other):
 39876 gvanrossum if isinstance(other, timedelta):
 39928 gvanrossum return self.__class__(self.__days +
other.__days,
 39876 gvanrossum   self.__seconds +
other.__seconds,
 39876 gvanrossum   self.__microseconds +
other.__microseconds)
 40207tim_one return NotImplemented
 39876 gvanrossum


[1] http://bugs.python.org/issue2267#msg125979
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Guido van Rossum]

Are you willing to wait 10 days for an answer? I'm out of round tuits for a
while.

On Fri, Feb 13, 2015 at 10:22 AM, Alexander Belopolsky 
alexander.belopol...@gmail.com wrote:


 On Fri, Feb 13, 2015 at 1:19 PM, Alexander Belopolsky 
 alexander.belopol...@gmail.com wrote:
 
  FWIW you're wrong when you claim that a constructor is no different
 from any other method. Someone else should probably explain this (it's an
 old argument that's been thoroughly settled).
 
 
  Well, the best answer I've got in the past [1] was ask on python-dev
 since Guido called the operator overriding expectation. :-)


 And let me repost this bit of history [1]:

 Here is the annotated pre-r82065 code:

  39876 gvanrossum def __add__(self, other):
  39876 gvanrossum if isinstance(other, timedelta):
  39928 gvanrossum return self.__class__(self.__days +
 other.__days,
  39876 gvanrossum   self.__seconds +
 other.__seconds,
  39876 gvanrossum   self.__microseconds +
 other.__microseconds)
  40207tim_one return NotImplemented
  39876 gvanrossum


 [1] http://bugs.python.org/issue2267#msg125979




-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Alexander Belopolsky]

On Fri, Feb 13, 2015 at 1:11 PM, Guido van Rossum gu...@python.org wrote:



 Note that the original pure python prototype of the datetime module had
 date.__add__ and friends call self.__class__(year, month, day).
 Unfortunately, it looks like the original sandbox did not survive the the
 hg conversion, so I cannot provide a link to the relevant history.


 FWIW you're wrong when you claim that a constructor is no different from
 any other method. Someone else should probably explain this (it's an old
 argument that's been thoroughly settled).


Well, the best answer I've got in the past [1] was ask on python-dev since
Guido called the operator overriding expectation. :-)

[1] http://bugs.python.org/issue2267#msg108060
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Ionel Cristian Mărieș]

Can we at least make it use the constructor (if there's a custom one)?
Seems like a reasonable compromise to me (let whoever implements a custom
__new__ deal with argument variance).

Eg, make it use a __new__ like this:

 class FancyInt(int):
... def __new__(self, value):
... return int.__new__(FancyInt, value)
...
... def __repr__(self):
... return FancyInt(%s) % super().__repr__()
...
 x = FancyInt(1)

 x
FancyInt(1)
 x += 1
 x   # it should be FancyInt(2)
2



Thanks,
-- Ionel Cristian Mărieș, blog.ionelmc.ro

On Fri, Feb 13, 2015 at 6:01 AM, Guido van Rossum gu...@python.org wrote:

 On Thu, Feb 12, 2015 at 7:41 PM, Ethan Furman et...@stoneleaf.us wrote:

 On 02/12/2015 06:39 PM, Alexander Belopolsky wrote:

  In my view, a constructor is no different from any other method.  If
 the designers of the subclass decided to change the
  signature in an incompatible way, they should either override all
 methods that create new objects or live with tracebacks.

  On the other hand, if all I want in my Date class is a better
 __format__ method, I am forced to override all operators
  or have my objects silently degrade [...]

 So there are basically two choices:

 1) always use the type of the most-base class when creating new instances

pros:
  - easy
  - speedy code
  - no possible tracebacks on new object instantiation

cons:
  - a subclass that needs/wants to maintain itself must override all
methods that create new instances, even if the only change is to
the type of object returned

 2) always use the type of self when creating new instances

pros:
  - subclasses automatically maintain type
  - much less code in the simple cases [1]

cons:
  - if constructor signatures change, must override all methods which
create new objects

 Unless there are powerful reasons against number 2 (such as performance,
 or the effort to affect the change), it sure
 seems like the nicer way to go.

 So back to my original question: what other concerns are there, and has
 anybody done any benchmarks?


 Con for #2 is a showstopper. Forget about it.

 --
 --Guido van Rossum (python.org/~guido)

 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/contact%40ionelmc.ro


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Guido van Rossum]

On Thu, Feb 12, 2015 at 8:58 PM, Ethan Furman et...@stoneleaf.us wrote:

 On 02/12/2015 08:01 PM, Guido van Rossum wrote:
  On Thu, Feb 12, 2015 at 7:41 PM, Ethan Furman wrote:
 
  2) always use the type of self when creating new instances
 
 cons:
   - if constructor signatures change, must override all methods
 which
 create new objects
 
  Con for #2 is a showstopper. Forget about it.

 Happy to, but can you explain why requiring the programmer to override the
 necessary methods, or get tracebacks, is a
 showstopper?  Is there a previous email thread I can read that discusses
 it?


IIUC you're proposing that the base class should *try* to construct an
instance of the subclass by calling the type with an argument, and fail if
it doesn't work. But that makes the whole thing brittle in the light of
changes to the subclass constructor. Also, what should the argument be? The
only answer I can think of is an instance of the base class. Finally, this
would require more special-casing in every built-in class (at least every
built-in class that sometimes returns instances of itself).

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Alexander Belopolsky]

On Fri, Feb 13, 2015 at 12:35 PM, Guido van Rossum gu...@python.org wrote:

 IIUC you're proposing that the base class should *try* to construct an
 instance of the subclass by calling the type with an argument, and fail if
 it doesn't work. But that makes the whole thing brittle in the light of
 changes to the subclass constructor. Also, what should the argument be? The
 only answer I can think of is an instance of the base class.


No.  The arguments should be whatever arguments are appropriate for the
baseclass's __init__ or __new__.  In the case of datetime.date that would
be year, month, day.

Note that the original pure python prototype of the datetime module had
date.__add__ and friends call self.__class__(year, month, day).
Unfortunately, it looks like the original sandbox did not survive the the
hg conversion, so I cannot provide a link to the relevant history.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Alexander Belopolsky]

On Fri, Feb 13, 2015 at 4:44 PM, Neil Girdhar mistersh...@gmail.com wrote:

 Interesting:
 http://stackoverflow.com/questions/5490824/should-constructors-comply-with-the-liskov-substitution-principle


Let me humbly conjecture that the people who wrote the top answers have
background in less capable languages than Python.

Not every language allows you to call self.__class__().  In the languages
that don't you can get away with incompatible constructor signatures.

However, let me try to focus the discussion on a specific issue before we
go deep into OOP theory.

With python's standard datetime.date we have:

 from datetime import *
 class Date(date):
... pass
...
 Date.today()
Date(2015, 2, 13)
 Date.fromordinal(1)
Date(1, 1, 1)

Both .today() and .fromordinal(1) will break in a subclass that redefines
__new__ as follows:

 class Date2(date):
... def __new__(cls, ymd):
... return date.__new__(cls, *ymd)
...
 Date2.today()
Traceback (most recent call last):
  File stdin, line 1, in module
TypeError: __new__() takes 2 positional arguments but 4 were given
 Date2.fromordinal(1)
Traceback (most recent call last):
  File stdin, line 1, in module
TypeError: __new__() takes 2 positional arguments but 4 were given

Why is this acceptable, but we have to sacrifice the convenience of having
Date + timedelta
return Date to make it work  with Date2:

 Date2((1,1,1)) + timedelta(1)
datetime.date(1, 1, 2)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Serhiy Storchaka]

On 13.02.15 05:41, Ethan Furman wrote:

So there are basically two choices:

1) always use the type of the most-base class when creating new instances

pros:
  - easy
  - speedy code
  - no possible tracebacks on new object instantiation

cons:
  - a subclass that needs/wants to maintain itself must override all
methods that create new instances, even if the only change is to
the type of object returned

2) always use the type of self when creating new instances

pros:
  - subclasses automatically maintain type
  - much less code in the simple cases [1]

cons:
  - if constructor signatures change, must override all methods which
create new objects


And switching to (2) would break existing code which uses subclasses 
with constructors with different signature (e.g. defaultdict).


The third choice is to use different specially designed constructor.

class A(int):


class A(int):
... def __add__(self, other): 

... return self.__make_me__(int(self) + int(other)) 



... def __repr__(self): 


... return 'A(%d)' % self
...

A.__make_me__ = A
A(2) + 3

A(5)

class B(A):

... def __repr__(self):
... return 'B(%d)' % self
...

B.__make_me__ = B
B(2) + 3

B(5)

We can add special attribute used to creating results of operations to 
all basic classes. By default it would be equal to the base class 
constructor.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Neil Girdhar]

I personally don't think this is a big enough issue to warrant any changes,
but I think Serhiy's solution would be the ideal best with one additional
parameter: the caller's type.  Something like

def __make_me__(self, cls, *args, **kwargs)

and the idea is that any time you want to construct a type, instead of

self.__class__(assumed arguments…)

where you are not sure that the derived class' constructor knows the right
argument types, you do

def SomeCls:
 def some_method(self, ...):
   return self.__make_me__(SomeCls, assumed arguments…)

Now the derived class knows who is asking for a copy.  In the case of
defaultdict, for example, he can implement __make_me__ as follows:

def __make_me__(self, cls, *args, **kwargs):
if cls is dict: return default_dict(self.default_factory, *args,
**kwargs)
return default_dict(*args, **kwargs)

essentially the caller is identifying himself so that the receiver knows
how to interpret the arguments.

Best,

Neil

On Fri, Feb 13, 2015 at 5:55 PM, Alexander Belopolsky 
alexander.belopol...@gmail.com wrote:


 On Fri, Feb 13, 2015 at 4:44 PM, Neil Girdhar mistersh...@gmail.com
 wrote:

 Interesting:
 http://stackoverflow.com/questions/5490824/should-constructors-comply-with-the-liskov-substitution-principle


 Let me humbly conjecture that the people who wrote the top answers have
 background in less capable languages than Python.

 Not every language allows you to call self.__class__().  In the languages
 that don't you can get away with incompatible constructor signatures.

 However, let me try to focus the discussion on a specific issue before we
 go deep into OOP theory.

 With python's standard datetime.date we have:

  from datetime import *
  class Date(date):
 ... pass
 ...
  Date.today()
 Date(2015, 2, 13)
  Date.fromordinal(1)
 Date(1, 1, 1)

 Both .today() and .fromordinal(1) will break in a subclass that redefines
 __new__ as follows:

  class Date2(date):
 ... def __new__(cls, ymd):
 ... return date.__new__(cls, *ymd)
 ...
  Date2.today()
 Traceback (most recent call last):
   File stdin, line 1, in module
 TypeError: __new__() takes 2 positional arguments but 4 were given
  Date2.fromordinal(1)
 Traceback (most recent call last):
   File stdin, line 1, in module
 TypeError: __new__() takes 2 positional arguments but 4 were given

 Why is this acceptable, but we have to sacrifice the convenience of having
 Date + timedelta
 return Date to make it work  with Date2:

  Date2((1,1,1)) + timedelta(1)
 datetime.date(1, 1, 2)



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Neil Girdhar]

Interesting:
http://stackoverflow.com/questions/5490824/should-constructors-comply-with-the-liskov-substitution-principle

On Fri, Feb 13, 2015 at 3:37 PM, Isaac Schwabacher ischwabac...@wisc.edu
wrote:

 On 15-02-13, Guido van Rossum  wrote:
  Are you willing to wait 10 days for an answer? I#39;m out of round
 tuits for a while.

 IIUC, the argument is that the Liskov Substitution Principle is a
 statement about how objects of a subtype behave relative to objects of a
 supertype, and it doesn't apply to constructors because they aren't
 behaviors of existing objects. So other overriding methods *should* be able
 to handle the same inputs that the respective overridden methods do, but
 constructors don't need to. Even though __init__ is written as an instance
 method, it seems like it's morally a part of the class method __new__
 that's only split off for convenience.

 If this message is unclear, it's because I don't really understand this
 myself and I'm trying to articulate my best understanding of what's been
 said on this thread and those it links to.

 ijs

  On Fri, Feb 13, 2015 at 10:22 AM, Alexander Belopolsky 
 alexander.belopol...@gmail.com(javascript:main.compose() wrote:
 
  
   On Fri, Feb 13, 2015 at 1:19 PM, Alexander Belopolsky 
 alexander.belopol...@gmail.com(javascript:main.compose() wrote:
   
FWIW you#39;re wrong when you claim that a constructor is no
 different from any other method. Someone else should probably explain this
 (it#39;s an old argument that#39;s been thoroughly settled).
   
   
Well, the best answer I#39;ve got in the past [1] was ask on
 python-dev since Guido called the operator overriding expectation. :-)
  
  
   And let me repost this bit of history [1]:
  
   Here is the annotated pre-r82065 code:
  
   39876 gvanrossum def __add__(self, other):
   39876 gvanrossum if isinstance(other, timedelta):
   39928 gvanrossum return self.__class__(self.__days + other.__days,
   39876 gvanrossum self.__seconds + other.__seconds,
   39876 gvanrossum self.__microseconds + other.__microseconds)
   40207 tim_one return NotImplemented
   39876 gvanrossum
  
  
  
   [1] http://bugs.python.org/issue2267#msg125979
  
  
 
 
 
 
  --
  --Guido van Rossum (python.org/~guido(http://python.org/~guido))
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/mistersheik%40gmail.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Isaac Schwabacher]

On 15-02-13, Guido van Rossum  wrote:
 Are you willing to wait 10 days for an answer? I#39;m out of round tuits for 
 a while.

IIUC, the argument is that the Liskov Substitution Principle is a statement 
about how objects of a subtype behave relative to objects of a supertype, and 
it doesn't apply to constructors because they aren't behaviors of existing 
objects. So other overriding methods *should* be able to handle the same inputs 
that the respective overridden methods do, but constructors don't need to. Even 
though __init__ is written as an instance method, it seems like it's morally 
a part of the class method __new__ that's only split off for convenience.

If this message is unclear, it's because I don't really understand this myself 
and I'm trying to articulate my best understanding of what's been said on this 
thread and those it links to.

ijs

 On Fri, Feb 13, 2015 at 10:22 AM, Alexander Belopolsky 
 alexander.belopol...@gmail.com(javascript:main.compose() wrote:
 
  
  On Fri, Feb 13, 2015 at 1:19 PM, Alexander Belopolsky 
  alexander.belopol...@gmail.com(javascript:main.compose() wrote:
  
   FWIW you#39;re wrong when you claim that a constructor is no different 
   from any other method. Someone else should probably explain this 
   (it#39;s an old argument that#39;s been thoroughly settled).
  
  
   Well, the best answer I#39;ve got in the past [1] was ask on python-dev 
   since Guido called the operator overriding expectation. :-)
  
  
  And let me repost this bit of history [1]:
  
  Here is the annotated pre-r82065 code:
  
  39876 gvanrossum def __add__(self, other):
  39876 gvanrossum if isinstance(other, timedelta):
  39928 gvanrossum return self.__class__(self.__days + other.__days,
  39876 gvanrossum self.__seconds + other.__seconds,
  39876 gvanrossum self.__microseconds + other.__microseconds)
  40207 tim_one return NotImplemented
  39876 gvanrossum
  
  
  
  [1] http://bugs.python.org/issue2267#msg125979
  
  
 
 
 
 
 -- 
 --Guido van Rossum (python.org/~guido(http://python.org/~guido))
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Isaac Schwabacher]

On 15-02-13, Neil Girdhar  wrote:
 Interesting: 
 http://stackoverflow.com/questions/5490824/should-constructors-comply-with-the-liskov-substitution-principle

You'd better believe I read that thread not 20 minutes ago. :)

 On Fri, Feb 13, 2015 at 3:37 PM, Isaac Schwabacher 
 alexander.belopol...@gmail.com ischwabac...@wisc.edu') 
 target=1ischwabac...@wisc.edu wrote:
 
  On 15-02-13, Guido van Rossum wrote:
   Are you willing to wait 10 days for an answer? I#39;m out of round tuits 
   for a while.
  
  IIUC, the argument is that the Liskov Substitution Principle is a statement 
  about how objects of a subtype behave relative to objects of a supertype, 
  and it doesn#39;t apply to constructors because they aren#39;t behaviors 
  of existing objects. So other overriding methods *should* be able to handle 
  the same inputs that the respective overridden methods do, but constructors 
  don#39;t need to. Even though __init__ is written as an instance method, 
  it seems like it#39;s morally a part of the class method __new__ 
  that#39;s only split off for convenience.
  
  If this message is unclear, it#39;s because I don#39;t really understand 
  this myself and I#39;m trying to articulate my best understanding of 
  what#39;s been said on this thread and those it links to.
  
  ijs
  
   On Fri, Feb 13, 2015 at 10:22 AM, Alexander Belopolsky 
   http://bugs.python.org/issue2267#msg125979(javascript:main.compose('new',
't=alexander.belopol...@gmail.com(java_script:main.compose() wrote:
  
   
On Fri, Feb 13, 2015 at 1:19 PM, Alexander Belopolsky 
alexander.belopol...@gmail.com 
alexander.belopol...@gmail.com(java_script:main.compose() wrote:

 FWIW you#39;re wrong when you claim that a constructor is no 
 different from any other method. Someone else should probably 
 explain this (it#39;s an old argument that#39;s been thoroughly 
 settled).


 Well, the best answer I#39;ve got in the past [1] was ask on 
 python-dev since Guido called the operator overriding expectation. 
 :-)
   
   
And let me repost this bit of history [1]:
   
Here is the annotated pre-r82065 code:
   
39876 gvanrossum def __add__(self, other):
39876 gvanrossum if isinstance(other, timedelta):
39928 gvanrossum return self.__class__(self.__days + other.__days,
39876 gvanrossum self.__seconds + other.__seconds,
39876 gvanrossum self.__microseconds + other.__microseconds)
40207 tim_one return NotImplemented
39876 gvanrossum
   
   
   
[1]  --Guido van Rossum 
(python.org/~guido(http://python.org/~guido)(http://python.org/~guido(http://python.org/~guido)))
  ___
  Python-Dev mailing list
  Python-Dev@python.org Python-Dev@python.org
  https://mail.python.org/mailman/listinfo/python-dev
  Unsubscribe: 
  https://mail.python.org/mailman/options/python-dev/mistersheik%40gmail.com
 
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Isaac Schwabacher]

On 15-02-13, Neil Girdhar  wrote:
 Unlike a regular method, you would never need to call super since you should 
 know everyone that could be calling you. Typically, when you call super, you 
 have something like this:
 
 A  B, C
 
 
 B  D
 
 
 so you end up with 
 
 
 mro: A, B, C, D
 
 
 And then when A calls super and B calls super it gets C which it doesn#39;t 
 know about.

But C calls super and gets D. The scenario I'm concerned with is that A knows 
how to mimic B's constructor and B knows how to mimic D's, but A doesn't know 
about D. So D asks A if it knows how to mimic D's constructor, and it says no. 
Via super, B gets a shot, and it does know, so it translates the arguments to 
D's constructor into arguments to B's constructor, and again asks A if it knows 
how to handle them. Then A says yes, translates the args, and constructs an A. 
If C ever gets consulted, it responds I don't know a thing and calls super.

 But in the case of make_me, it#39;s someone like C who is calling make_me. 
 If it gets a method in B, then that#39;s a straight-up bug. make_me needs to 
 be reimplemented in A as well, and A would never delegate up since other 
 classes in the mro chain (like B) might not know about C.

This scheme (as I've written it) depends strongly on all the classes in the MRO 
having __make_me__ methods with this very precisely defined structure: test 
base against yourself, then any superclasses you care to mimic, then call 
super. Any antisocial superclass ruins everyone's party.

 Best,
 Neil
 
 
 On Fri, Feb 13, 2015 at 7:00 PM, Isaac Schwabacher 
 alexander.belopol...@gmail.com ischwabac...@wisc.edu') 
 target=1ischwabac...@wisc.edu wrote:
 
  On 15-02-13, Neil Girdhar wrote:
   I personally don#39;t think this is a big enough issue to warrant any 
   changes, but I think Serhiy#39;s solution would be the ideal best with 
   one additional parameter: the caller#39;s type. Something like
  
   def __make_me__(self, cls, *args, **kwargs)
  
  
   and the idea is that any time you want to construct a type, instead of
  
  
   self.__class__(assumed arguments…)
  
  
   where you are not sure that the derived class#39; constructor knows the 
   right argument types, you do
  
  
   def SomeCls:
   def some_method(self, ...):
   return self.__make_me__(SomeCls, assumed arguments…)
  
  
   Now the derived class knows who is asking for a copy. In the case of 
   defaultdict, for example, he can implement __make_me__ as follows:
  
  
   def __make_me__(self, cls, *args, **kwargs):
   if cls is dict: return default_dict(self.default_factory, *args, **kwargs)
   return default_dict(*args, **kwargs)
  
  
   essentially the caller is identifying himself so that the receiver knows 
   how to interpret the arguments.
  
  
   Best,
  
  
   Neil
  
  Such a method necessarily involves explicit switching on classes... ew.
  Also, to make this work, a class needs to have a relationship with its 
  superclass#39;s superclasses. So in order for DefaultDict#39;s subclasses 
  not to need to know about dict, it would need to look like this:
  
  class DefaultDict(dict):
  @classmethod # instance method doesn#39;t make sense here
  def __make_me__(cls, base, *args, **kwargs): # make something like 
  base(*args, **kwargs)
  # when we get here, nothing in cls.__mro__ above DefaultDict knows 
  how to construct an equivalent to base(*args, **kwargs) using its own 
  constructor
  if base is DefaultDict:
  return DefaultDict(*args, **kwargs) # if DefaultDict is the 
  best we can do, do it
  elif base is dict:
  return cls.__make_me__(DefaultDict, None, *args, **kwargs) # 
  subclasses that know about DefaultDict but not dict will intercept this
  else:
  super(DefaultDict, cls).__make_me__(base, *args, **kwargs) # we 
  don#39;t know how to make an equivalent to base.__new__(*args, **kwargs), 
  so keep looking
  
  I don#39;t even think this is guaranteed to construct an object of class 
  cls corresponding to a base(*args, **kwargs) even if it were possible, 
  since multiple inheritance can screw things up. You might need to have an 
  explicit list of these are the superclasses whose constructors I can 
  imitate, and have the interpreter find an optimal path for you.
  
   On Fri, Feb 13, 2015 at 5:55 PM, Alexander Belopolsky 
   http://stackoverflow.com/questions/5490824/should-constructors-comply-with-the-liskov-substitution-principle(javascript:main.compose('new',
't=alexander.belopol...@gmail.com(java_script:main.compose() wrote:
  
   
On Fri, Feb 13, 2015 at 4:44 PM, Neil Girdhar mistersh...@gmail.com 
mistersh...@gmail.com(java_script:main.compose() wrote:
   
 Interesting:   Not every language allows you to call 
 self.__class__(). In the languages that don#39;t you can get away 
 with incompatible constructor signatures.
   
   
However, let me try to focus the discussion on a specific issue 

Re: [Python-Dev] subclassing builtin data structures

[Neil Girdhar]

I think it works as Isaac explained if __make_me__ is an instance method
that also accepts the calling class type.

On Fri, Feb 13, 2015 at 8:12 PM, Ethan Furman et...@stoneleaf.us wrote:

 On 02/13/2015 02:31 PM, Serhiy Storchaka wrote:
  On 13.02.15 05:41, Ethan Furman wrote:
  So there are basically two choices:
 
  1) always use the type of the most-base class when creating new
 instances
 
  pros:
- easy
- speedy code
- no possible tracebacks on new object instantiation
 
  cons:
- a subclass that needs/wants to maintain itself must override all
  methods that create new instances, even if the only change is to
  the type of object returned
 
  2) always use the type of self when creating new instances
 
  pros:
- subclasses automatically maintain type
- much less code in the simple cases [1]
 
  cons:
- if constructor signatures change, must override all methods
 which
  create new objects
 
  And switching to (2) would break existing code which uses subclasses
 with constructors with different signature (e.g.
  defaultdict).

 I don't think defaultdict is a good example -- I don't see any methods on
 it that return a new dict, default or
 otherwise. So if this change happened, defaultdict would have to have its
 own __add__ and not rely on dict's __add__.


  The third choice is to use different specially designed constructor.
 
  class A(int):
 
  -- class A(int):
  ... def __add__(self, other):
  ... return self.__make_me__(int(self) + int(other))
 
  ... def __repr__(self):
  ... return 'A(%d)' % self

 How would this help in the case of defaultdict?  __make_me__ is a class
 method, but it needs instance info to properly
 create a new dict with the same default factory.

 --
 ~Ethan~


 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/mistersheik%40gmail.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Isaac Schwabacher]

Hmm... super *is* a problem, because if we super, we can end up asking 
superclasses that we then won't know how to mimic. So what we really want is to 
move the superclasses we can mimic to the front of the MRO. If none of those 
can indirectly mimic the base class, then we try the other classes in the MRO 
to see if we can get a partial upgrade.

On 15-02-13, Neil Girdhar  wrote:
 I think it works as Isaac explained if __make_me__ is an instance method that 
 also accepts the calling class type.
 
 On Fri, Feb 13, 2015 at 8:12 PM, Ethan Furman Python-Dev@python.org 
 et...@stoneleaf.us') target=1et...@stoneleaf.us wrote:
 
  On 02/13/2015 02:31 PM, Serhiy Storchaka wrote:
   On 13.02.15 05:41, Ethan Furman wrote:
   So there are basically two choices:
  
   1) always use the type of the most-base class when creating new instances
  
   pros:
   - easy
   - speedy code
   - no possible tracebacks on new object instantiation
  
   cons:
   - a subclass that needs/wants to maintain itself must override all
   methods that create new instances, even if the only change is to
   the type of object returned
  
   2) always use the type of self when creating new instances
  
   pros:
   - subclasses automatically maintain type
   - much less code in the simple cases [1]
  
   cons:
   - if constructor signatures change, must override all methods which
   create new objects
  
   And switching to (2) would break existing code which uses subclasses with 
   constructors with different signature (e.g.
   defaultdict).
  
  I don#39;t think defaultdict is a good example -- I don#39;t see any 
  methods on it that return a new dict, default or
  otherwise. So if this change happened, defaultdict would have to have its 
  own __add__ and not rely on dict#39;s __add__.
  
  
   The third choice is to use different specially designed constructor.
  
   class A(int):
  
   -- class A(int):
   ... def __add__(self, other):
   ... return self.__make_me__(int(self) + int(other))
  
   ... def __repr__(self):
   ... return #39;A(%d)#39; % self
  
  How would this help in the case of defaultdict? __make_me__ is a class 
  method, but it needs instance info to properly
  create a new dict with the same default factory.
  
  --
  ~Ethan~
  
  
  ___
  Python-Dev mailing list
  https://mail.python.org/mailman/listinfo/python-dev(javascript:main.compose('new',
   't=Python-Dev@python.org
  a href=)
  Unsubscribe: 
  https://mail.python.org/mailman/options/python-dev/mistersheik%40gmail.com
  
 
import inspect

class A:
def __init__(self, bar):
self.bar = bar
def foo(self):
return self.__class__.__make_me__(A, self.bar)
@classmethod
def __make_me__(cls, base, *args, **kwargs):
if base is A:
return A(*args, **kwargs)
else:
return super(A, cls).__make_me__(base, *args, **kwargs)

class B(A):
def __init__(self, bar, baz):
self.bar = bar
self.baz = baz
@classmethod
def __make_me__(cls, base, *args, **kwargs):
if base is B:
return B(*args, **kwargs)
elif base is A:
bound = inspect.signature(A).bind(*args, **kwargs)
return cls.__make_me__(B, bound.arguments['bar'], None)
else:
super(A, cls).__make_me__(base, *args, **kwargs)

class C(A):
def __init__(self):
pass
@classmethod
def __make_me__(cls, base, *args, **kwargs):
if base is C:
return C(*args, **kwargs)
elif base is A:
bound = inspect.signature(A).bind(*args, **kwargs)
return cls.__make_me__(C)
else:
return super(C, cls).__make_me__(base, *args, **kwargs)

class D(C, B):
def __init__(self, bar, baz, spam):
self.bar = bar
self.baz = baz
self.spam = spam
@classmethod
def __make_me__(cls, base, *args, **kwargs):
if base is D:
return D(*args, **kwargs)
elif base is B:
bound = inspect.signature(B).bind(*args, **kwargs)
return cls.__make_me__(D, bound.arguments['bar'],
   bound.arguments['baz'], 'hello')
else:
return super(D, cls).__make_me__(base, *args, **kwargs)

if __name__ == '__main__':
assert D('a', 'b', 'c').foo().__class__ is D
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Greg Ewing]

Isaac Schwabacher wrote:

IIUC, the argument is that the Liskov Substitution Principle is a statement
about how objects of a subtype behave relative to objects of a supertype, and
it doesn't apply to constructors because they aren't behaviors of existing
objects.


Another way to say that is that constructors are class
methods, not instance methods.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Neil Girdhar]

Unlike a regular method, you would never need to call super since you
should know everyone that could be calling you.  Typically, when you call
super, you have something like this:

A  B, C

B  D

so you end up with

mro: A, B, C, D

And then when A calls super and B calls super it gets C which it doesn't
know about.

But in the case of make_me, it's someone like C who  is calling make_me.
If it gets a method in B, then that's a straight-up bug.  make_me needs to
be reimplemented in A as well, and A would never delegate up since other
classes in the mro chain (like B) might not know about C.

Best,
Neil

On Fri, Feb 13, 2015 at 7:00 PM, Isaac Schwabacher ischwabac...@wisc.edu
wrote:

 On 15-02-13, Neil Girdhar  wrote:
  I personally don#39;t think this is a big enough issue to warrant any
 changes, but I think Serhiy#39;s solution would be the ideal best with one
 additional parameter: the caller#39;s type. Something like
 
  def __make_me__(self, cls, *args, **kwargs)
 
 
  and the idea is that any time you want to construct a type, instead of
 
 
  self.__class__(assumed arguments…)
 
 
  where you are not sure that the derived class#39; constructor knows the
 right argument types, you do
 
 
  def SomeCls:
  def some_method(self, ...):
  return self.__make_me__(SomeCls, assumed arguments…)
 
 
  Now the derived class knows who is asking for a copy. In the case of
 defaultdict, for example, he can implement __make_me__ as follows:
 
 
  def __make_me__(self, cls, *args, **kwargs):
  if cls is dict: return default_dict(self.default_factory, *args,
 **kwargs)
  return default_dict(*args, **kwargs)
 
 
  essentially the caller is identifying himself so that the receiver knows
 how to interpret the arguments.
 
 
  Best,
 
 
  Neil

 Such a method necessarily involves explicit switching on classes... ew.
 Also, to make this work, a class needs to have a relationship with its
 superclass's superclasses. So in order for DefaultDict's subclasses not to
 need to know about dict, it would need to look like this:

 class DefaultDict(dict):
 @classmethod # instance method doesn't make sense here
 def __make_me__(cls, base, *args, **kwargs): # make something like
 base(*args, **kwargs)
 # when we get here, nothing in cls.__mro__ above DefaultDict knows
 how to construct an equivalent to base(*args, **kwargs) using its own
 constructor
 if base is DefaultDict:
 return DefaultDict(*args, **kwargs) # if DefaultDict is the
 best we can do, do it
 elif base is dict:
 return cls.__make_me__(DefaultDict, None, *args, **kwargs) #
 subclasses that know about DefaultDict but not dict will intercept this
 else:
 super(DefaultDict, cls).__make_me__(base, *args, **kwargs) #
 we don't know how to make an equivalent to base.__new__(*args, **kwargs),
 so keep looking

 I don't even think this is guaranteed to construct an object of class cls
 corresponding to a base(*args, **kwargs) even if it were possible, since
 multiple inheritance can screw things up. You might need to have an
 explicit list of these are the superclasses whose constructors I can
 imitate, and have the interpreter find an optimal path for you.

  On Fri, Feb 13, 2015 at 5:55 PM, Alexander Belopolsky 
 alexander.belopol...@gmail.com(javascript:main.compose() wrote:
 
  
   On Fri, Feb 13, 2015 at 4:44 PM, Neil Girdhar 
   mistersh...@gmail.com(javascript:main.compose()
 wrote:
  
Interesting:
 http://stackoverflow.com/questions/5490824/should-constructors-comply-with-the-liskov-substitution-principle
   
  
  
   Let me humbly conjecture that the people who wrote the top answers
 have background in less capable languages than Python.
  
  
   Not every language allows you to call self.__class__(). In the
 languages that don#39;t you can get away with incompatible constructor
 signatures.
  
  
   However, let me try to focus the discussion on a specific issue before
 we go deep into OOP theory.
  
  
   With python#39;s standard datetime.date we have:
  
  
from datetime import *
class Date(date):
   ... pass
   ...
Date.today()
   Date(2015, 2, 13)
Date.fromordinal(1)
   Date(1, 1, 1)
  
  
   Both .today() and .fromordinal(1) will break in a subclass that
 redefines __new__ as follows:
  
  
class Date2(date):
   ... def __new__(cls, ymd):
   ... return date.__new__(cls, *ymd)
   ...
Date2.today()
   Traceback (most recent call last):
   File stdin, line 1, in module
   TypeError: __new__() takes 2 positional arguments but 4 were given
Date2.fromordinal(1)
   Traceback (most recent call last):
   File stdin, line 1, in module
   TypeError: __new__() takes 2 positional arguments but 4 were given
  
  
  
  
   Why is this acceptable, but we have to sacrifice the convenience of
 having Date + timedelta
   return Date to make it work with Date2:
  
  
Date2((1,1,1)) + timedelta(1)
   datetime.date(1, 1, 2)
  
  
  
  
  
  
  
  
  
  

Re: [Python-Dev] subclassing builtin data structures

[Neil Girdhar]

You're right.

On Fri, Feb 13, 2015 at 7:55 PM, Isaac Schwabacher ischwabac...@wisc.edu
wrote:

 On 15-02-13, Neil Girdhar  wrote:
  Unlike a regular method, you would never need to call super since you
 should know everyone that could be calling you. Typically, when you call
 super, you have something like this:
 
  A  B, C
 
 
  B  D
 
 
  so you end up with
 
 
  mro: A, B, C, D
 
 
  And then when A calls super and B calls super it gets C which it
 doesn#39;t know about.

 But C calls super and gets D. The scenario I'm concerned with is that A
 knows how to mimic B's constructor and B knows how to mimic D's, but A
 doesn't know about D. So D asks A if it knows how to mimic D's constructor,
 and it says no. Via super, B gets a shot, and it does know, so it
 translates the arguments to D's constructor into arguments to B's
 constructor, and again asks A if it knows how to handle them. Then A says
 yes, translates the args, and constructs an A. If C ever gets consulted, it
 responds I don't know a thing and calls super.

  But in the case of make_me, it#39;s someone like C who is calling
 make_me. If it gets a method in B, then that#39;s a straight-up bug.
 make_me needs to be reimplemented in A as well, and A would never delegate
 up since other classes in the mro chain (like B) might not know about C.

 This scheme (as I've written it) depends strongly on all the classes in
 the MRO having __make_me__ methods with this very precisely defined
 structure: test base against yourself, then any superclasses you care to
 mimic, then call super. Any antisocial superclass ruins everyone's party.

  Best,
  Neil
 
 
  On Fri, Feb 13, 2015 at 7:00 PM, Isaac Schwabacher 
 alexander.belopol...@gmail.com ischwabac...@wisc.edu') target=1
 ischwabac...@wisc.edu wrote:
 
   On 15-02-13, Neil Girdhar wrote:
I personally don#39;t think this is a big enough issue to warrant
 any changes, but I think Serhiy#39;s solution would be the ideal best with
 one additional parameter: the caller#39;s type. Something like
   
def __make_me__(self, cls, *args, **kwargs)
   
   
and the idea is that any time you want to construct a type, instead
 of
   
   
self.__class__(assumed arguments…)
   
   
where you are not sure that the derived class#39; constructor knows
 the right argument types, you do
   
   
def SomeCls:
def some_method(self, ...):
return self.__make_me__(SomeCls, assumed arguments…)
   
   
Now the derived class knows who is asking for a copy. In the case of
 defaultdict, for example, he can implement __make_me__ as follows:
   
   
def __make_me__(self, cls, *args, **kwargs):
if cls is dict: return default_dict(self.default_factory, *args,
 **kwargs)
return default_dict(*args, **kwargs)
   
   
essentially the caller is identifying himself so that the receiver
 knows how to interpret the arguments.
   
   
Best,
   
   
Neil
  
   Such a method necessarily involves explicit switching on classes... ew.
   Also, to make this work, a class needs to have a relationship with its
 superclass#39;s superclasses. So in order for DefaultDict#39;s subclasses
 not to need to know about dict, it would need to look like this:
  
   class DefaultDict(dict):
   @classmethod # instance method doesn#39;t make sense here
   def __make_me__(cls, base, *args, **kwargs): # make something like
 base(*args, **kwargs)
   # when we get here, nothing in cls.__mro__ above DefaultDict
 knows how to construct an equivalent to base(*args, **kwargs) using its own
 constructor
   if base is DefaultDict:
   return DefaultDict(*args, **kwargs) # if DefaultDict is
 the best we can do, do it
   elif base is dict:
   return cls.__make_me__(DefaultDict, None, *args, **kwargs)
 # subclasses that know about DefaultDict but not dict will intercept this
   else:
   super(DefaultDict, cls).__make_me__(base, *args, **kwargs)
 # we don#39;t know how to make an equivalent to base.__new__(*args,
 **kwargs), so keep looking
  
   I don#39;t even think this is guaranteed to construct an object of
 class cls corresponding to a base(*args, **kwargs) even if it were
 possible, since multiple inheritance can screw things up. You might need to
 have an explicit list of these are the superclasses whose constructors I
 can imitate, and have the interpreter find an optimal path for you.
  
On Fri, Feb 13, 2015 at 5:55 PM, Alexander Belopolsky 
 http://stackoverflow.com/questions/5490824/should-constructors-comply-with-the-liskov-substitution-principle(javascript:main.compose('new',
 't=alexander.belopol...@gmail.com(java_script:main.compose() wrote:
   

 On Fri, Feb 13, 2015 at 4:44 PM, Neil Girdhar 
 mistersh...@gmail.com mistersh...@gmail.com(java_script:main.compose()
 wrote:

  Interesting:   Not every language allows you to call
 self.__class__(). In the languages that don#39;t you can get away with
 incompatible 

Re: [Python-Dev] subclassing builtin data structures

[Ethan Furman]

On 02/13/2015 02:31 PM, Serhiy Storchaka wrote:
 On 13.02.15 05:41, Ethan Furman wrote:
 So there are basically two choices:

 1) always use the type of the most-base class when creating new instances

 pros:
   - easy
   - speedy code
   - no possible tracebacks on new object instantiation

 cons:
   - a subclass that needs/wants to maintain itself must override all
 methods that create new instances, even if the only change is to
 the type of object returned

 2) always use the type of self when creating new instances

 pros:
   - subclasses automatically maintain type
   - much less code in the simple cases [1]

 cons:
   - if constructor signatures change, must override all methods which
 create new objects
 
 And switching to (2) would break existing code which uses subclasses with 
 constructors with different signature (e.g.
 defaultdict).

I don't think defaultdict is a good example -- I don't see any methods on it 
that return a new dict, default or
otherwise. So if this change happened, defaultdict would have to have its own 
__add__ and not rely on dict's __add__.


 The third choice is to use different specially designed constructor.
 
 class A(int):
 
 -- class A(int):
 ... def __add__(self, other):
 ... return self.__make_me__(int(self) + int(other))
 
 ... def __repr__(self):
 ... return 'A(%d)' % self

How would this help in the case of defaultdict?  __make_me__ is a class method, 
but it needs instance info to properly
create a new dict with the same default factory.

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Isaac Schwabacher]

On 15-02-13, Neil Girdhar  wrote:
 I personally don#39;t think this is a big enough issue to warrant any 
 changes, but I think Serhiy#39;s solution would be the ideal best with one 
 additional parameter: the caller#39;s type. Something like
 
 def __make_me__(self, cls, *args, **kwargs)
 
 
 and the idea is that any time you want to construct a type, instead of 
 
 
 self.__class__(assumed arguments…)
 
 
 where you are not sure that the derived class#39; constructor knows the 
 right argument types, you do
 
 
 def SomeCls:
 def some_method(self, ...):
 return self.__make_me__(SomeCls, assumed arguments…)
 
 
 Now the derived class knows who is asking for a copy. In the case of 
 defaultdict, for example, he can implement __make_me__ as follows:
 
 
 def __make_me__(self, cls, *args, **kwargs):
 if cls is dict: return default_dict(self.default_factory, *args, **kwargs)
 return default_dict(*args, **kwargs)
 
 
 essentially the caller is identifying himself so that the receiver knows how 
 to interpret the arguments.
 
 
 Best,
 
 
 Neil

Such a method necessarily involves explicit switching on classes... ew.
Also, to make this work, a class needs to have a relationship with its 
superclass's superclasses. So in order for DefaultDict's subclasses not to need 
to know about dict, it would need to look like this:

class DefaultDict(dict):
@classmethod # instance method doesn't make sense here
def __make_me__(cls, base, *args, **kwargs): # make something like 
base(*args, **kwargs)
# when we get here, nothing in cls.__mro__ above DefaultDict knows how 
to construct an equivalent to base(*args, **kwargs) using its own constructor
if base is DefaultDict:
return DefaultDict(*args, **kwargs) # if DefaultDict is the best we 
can do, do it
elif base is dict:
return cls.__make_me__(DefaultDict, None, *args, **kwargs) # 
subclasses that know about DefaultDict but not dict will intercept this
else:
super(DefaultDict, cls).__make_me__(base, *args, **kwargs) # we 
don't know how to make an equivalent to base.__new__(*args, **kwargs), so keep 
looking

I don't even think this is guaranteed to construct an object of class cls 
corresponding to a base(*args, **kwargs) even if it were possible, since 
multiple inheritance can screw things up. You might need to have an explicit 
list of these are the superclasses whose constructors I can imitate, and have 
the interpreter find an optimal path for you.

 On Fri, Feb 13, 2015 at 5:55 PM, Alexander Belopolsky 
 alexander.belopol...@gmail.com(javascript:main.compose() wrote:
 
  
  On Fri, Feb 13, 2015 at 4:44 PM, Neil Girdhar 
  mistersh...@gmail.com(javascript:main.compose() wrote:
  
   Interesting: 
   http://stackoverflow.com/questions/5490824/should-constructors-comply-with-the-liskov-substitution-principle
   
  
  
  Let me humbly conjecture that the people who wrote the top answers have 
  background in less capable languages than Python.
  
  
  Not every language allows you to call self.__class__(). In the languages 
  that don#39;t you can get away with incompatible constructor signatures.
  
  
  However, let me try to focus the discussion on a specific issue before we 
  go deep into OOP theory.
  
  
  With python#39;s standard datetime.date we have:
  
  
   from datetime import *
   class Date(date):
  ... pass
  ...
   Date.today()
  Date(2015, 2, 13)
   Date.fromordinal(1)
  Date(1, 1, 1)
  
  
  Both .today() and .fromordinal(1) will break in a subclass that redefines 
  __new__ as follows:
  
  
   class Date2(date):
  ... def __new__(cls, ymd):
  ... return date.__new__(cls, *ymd)
  ...
   Date2.today()
  Traceback (most recent call last):
  File stdin, line 1, in module
  TypeError: __new__() takes 2 positional arguments but 4 were given
   Date2.fromordinal(1)
  Traceback (most recent call last):
  File stdin, line 1, in module
  TypeError: __new__() takes 2 positional arguments but 4 were given
  
  
  
  
  Why is this acceptable, but we have to sacrifice the convenience of having 
  Date + timedelta
  return Date to make it work with Date2:
  
  
   Date2((1,1,1)) + timedelta(1)
  datetime.date(1, 1, 2)
  
  
  
  
  
  
  
  
  
 
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Serhiy Storchaka]

On 14.02.15 03:12, Ethan Furman wrote:

The third choice is to use different specially designed constructor.

class A(int):

-- class A(int):
... def __add__(self, other):
... return self.__make_me__(int(self) + int(other))

... def __repr__(self):
... return 'A(%d)' % self


How would this help in the case of defaultdict?  __make_me__ is a class method, 
but it needs instance info to properly
create a new dict with the same default factory.


In case of defaultdict (when dict would have to have __add__ and like) 
either __make_me__ == dict (then defaultdict's methods will return 
dicts) or it will be instance method.


def __make_me__(self, other):
return defaultdict(self.default_factory, other)


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Serhiy Storchaka]

On 14.02.15 01:03, Neil Girdhar wrote:

Now the derived class knows who is asking for a copy.  In the case of
defaultdict, for example, he can implement __make_me__ as follows:

def __make_me__(self, cls, *args, **kwargs):
 if cls is dict: return default_dict(self.default_factory, *args,
**kwargs)
 return default_dict(*args, **kwargs)

essentially the caller is identifying himself so that the receiver knows
how to interpret the arguments.


No, my idea was that __make_me__ has the same signature in all 
subclasses. It takes exactly one argument and creates an instance of 
concrete class, so it never fails. If you want to create an instance of 
different class in the derived class, you should explicitly override 
__make_me__.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Guido van Rossum]

On Thu, Feb 12, 2015 at 4:36 PM, Ethan Furman et...@stoneleaf.us wrote:

 I suspect the last big hurdle to making built-in data structures nicely
 subclassable is the insistence of such types to
 return new instances as the base class instead of the derived class.

 In case that wasn't clear ;)

 -- class MyInt(int):
 ...   def __repr__(self):
 ... return 'MyInt(%d)' % self
 ...
 -- m = MyInt(42)
 -- m
 MyInt(42)
 -- m + 1
 43
 -- type(m+1)
 class 'int'

 Besides the work it would take to rectify this, I imagine the biggest
 hurdle would be the performance hit in always
 looking up the type of self.  Has anyone done any preliminary
 benchmarking?  Are there other concerns?


Actually, the problem is that the base class (e.g. int) doesn't know how to
construct an instance of the subclass -- there is no reason (in general)
why the signature of a subclass constructor should match the base class
constructor, and it often doesn't.

So this is pretty much a no-go. It's not unique to Python -- it's a basic
issue with OO.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Ethan Furman]

On 02/12/2015 05:46 PM, MRAB wrote:
 On 2015-02-13 00:55, Guido van Rossum wrote:
 On Thu, Feb 12, 2015 at 4:36 PM, Ethan Furman et...@stoneleaf.us
 mailto:et...@stoneleaf.us wrote:

 I suspect the last big hurdle to making built-in data structures
 nicely subclassable is the insistence of such types to
 return new instances as the base class instead of the derived class.

 In case that wasn't clear ;)

 -- class MyInt(int):
 ...   def __repr__(self):
 ... return 'MyInt(%d)' % self
 ...
 -- m = MyInt(42)
 -- m
 MyInt(42)
 -- m + 1
 43
 -- type(m+1)
 class 'int'

 Besides the work it would take to rectify this, I imagine the
 biggest hurdle would be the performance hit in always
 looking up the type of self.  Has anyone done any preliminary
 benchmarking?  Are there other concerns?


 Actually, the problem is that the base class (e.g. int) doesn't know how
 to construct an instance of the subclass -- there is no reason (in
 general) why the signature of a subclass constructor should match the
 base class constructor, and it often doesn't.

 So this is pretty much a no-go. It's not unique to Python -- it's a
 basic issue with OO.

 Really?

What I was asking about, and Guido responded to, was not having to specifically 
override __add__, __mul__, __sub__, and
all the others; if we do override them then there is no problem.

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Ethan Furman]

On 02/12/2015 04:55 PM, Guido van Rossum wrote:
 On Thu, Feb 12, 2015 at 4:36 PM, Ethan Furman et...@stoneleaf.us 
 mailto:et...@stoneleaf.us wrote:
 
 I suspect the last big hurdle to making built-in data structures nicely 
 subclassable is the insistence of such types to
 return new instances as the base class instead of the derived class.
 
 In case that wasn't clear ;)
 
 -- class MyInt(int):
 ...   def __repr__(self):
 ... return 'MyInt(%d)' % self
 ...
 -- m = MyInt(42)
 -- m
 MyInt(42)
 -- m + 1
 43
 -- type(m+1)
 class 'int'
 
 Besides the work it would take to rectify this, I imagine the biggest 
 hurdle would be the performance hit in always
 looking up the type of self.  Has anyone done any preliminary 
 benchmarking?  Are there other concerns?
 
 
 Actually, the problem is that the base class (e.g. int) doesn't know how to 
 construct an instance of the subclass --
 there is no reason (in general) why the signature of a subclass constructor 
 should match the base class constructor, and
 it often doesn't.
 
 So this is pretty much a no-go. It's not unique to Python -- it's a basic 
 issue with OO.

Thank you.

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[MRAB]

On 2015-02-13 00:55, Guido van Rossum wrote:

On Thu, Feb 12, 2015 at 4:36 PM, Ethan Furman et...@stoneleaf.us
mailto:et...@stoneleaf.us wrote:

I suspect the last big hurdle to making built-in data structures
nicely subclassable is the insistence of such types to
return new instances as the base class instead of the derived class.

In case that wasn't clear ;)

-- class MyInt(int):
...   def __repr__(self):
... return 'MyInt(%d)' % self
...
-- m = MyInt(42)
-- m
MyInt(42)
-- m + 1
43
-- type(m+1)
class 'int'

Besides the work it would take to rectify this, I imagine the
biggest hurdle would be the performance hit in always
looking up the type of self.  Has anyone done any preliminary
benchmarking?  Are there other concerns?


Actually, the problem is that the base class (e.g. int) doesn't know how
to construct an instance of the subclass -- there is no reason (in
general) why the signature of a subclass constructor should match the
base class constructor, and it often doesn't.


 So this is pretty much a no-go. It's not unique to Python -- it's a
 basic issue with OO.

Really?


 class BaseInt:
... def __init__(self, value):
... self._value = value
... def __add__(self, other):
... return type(self)(self._value + other)
... def __repr__(self):
... return '%s(%s)' % (type(self), self._value)
...
 class MyInt(BaseInt):
... pass
...

 m = BaseInt(42)
 m
class '__main__.BaseInt'(42)
 m + 1
class '__main__.BaseInt'(43)
 type(m + 1)
class '__main__.BaseInt'

 m = MyInt(42)
 m
class '__main__.MyInt'(42)
 m + 1
class '__main__.MyInt'(43)
 type(m + 1)
class '__main__.MyInt'


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Chris Angelico]

On Fri, Feb 13, 2015 at 12:46 PM, MRAB pyt...@mrabarnett.plus.com wrote:
 class BaseInt:
 ... def __init__(self, value):
 ... self._value = value
 ... def __add__(self, other):
 ... return type(self)(self._value + other)

On Fri, Feb 13, 2015 at 11:55 AM, Guido van Rossum gu...@python.org wrote:
 ... there is no reason (in general) why
 the signature of a subclass constructor should match the base class
 constructor, and it often doesn't.

You're requiring that any subclass of BaseInt be instantiable with one
argument, namely its value. That's requiring that the signature of the
subclass constructor match the base class constructor.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Alexander Belopolsky]

On Thu, Feb 12, 2015 at 7:55 PM, Guido van Rossum gu...@python.org wrote:

 the problem is that the base class (e.g. int) doesn't know how to
 construct an instance of the subclass -- there is no reason (in general)
 why the signature of a subclass constructor should match the base class
 constructor, and it often doesn't.


I hear this explanation every time we have a discussion about subclassing
of datetime types and I don't really buy this.

Consider this simple subclass:

 from datetime import date
 class Date(date):
... pass
...

What do you think Date.today() should return?  Since I did not override
todat() in my Date class, it has to be datetime.date instance, right?

However:

 Date.today().__class__
class '__main__.Date'

Wait, Date doesn't know how to construct an instance of the subclass ..

Indeed, if I change the constructor signature, Date.today() won't work:

 class Date(date):
... def __init__(self, extra):
... pass
...
 Date.today()
Traceback (most recent call last):
  File stdin, line 1, in module
TypeError: __init__() takes exactly 2 arguments (4 given)

In my view, a constructor is no different from any other method.  If the
designers of the subclass decided to change the signature in an
incompatible way, they should either override all methods that create new
objects or live with tracebacks.

On the other hand, if all I want in my Date class is a better __format__
method, I am forced to override all operators or have my objects silently
degrade in situations like this:

 d = Date.today()
 d.__class__
class '__main__.Date'
 d += timedelta(1)
 d.__class__
type 'datetime.date'

Having binary operations return subclass instances is not without
precedent.  For example, in numpy,

 from numpy import ndarray
 class Array(ndarray):
... pass
...
 a = Array(1)
 a[0] = 42
 a
Array([ 42.])
 a + a
Array([ 84.])

I believe numpy had this behavior since types became subclassable in
Python, so this design is definitely not a no-go.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Mark Roberts]

 On Feb 12, 2015, at 18:40, Chris Angelico ros...@gmail.com wrote:
 
 On Fri, Feb 13, 2015 at 12:46 PM, MRAB pyt...@mrabarnett.plus.com wrote:
 class BaseInt:
 ... def __init__(self, value):
 ... self._value = value
 ... def __add__(self, other):
 ... return type(self)(self._value + other)
 
 On Fri, Feb 13, 2015 at 11:55 AM, Guido van Rossum gu...@python.org wrote:
 ... there is no reason (in general) why
 the signature of a subclass constructor should match the base class
 constructor, and it often doesn't.
 
 You're requiring that any subclass of BaseInt be instantiable with one
 argument, namely its value. That's requiring that the signature of the
 subclass constructor match the base class constructor.
 
 ChrisA
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe: 
 https://mail.python.org/mailman/options/python-dev/wizzat%40gmail.com

No, it seems like he's asking that the type return a new object of the same 
type instead of one of the superclass. In effect, making the Date class call 
type(self)(*args) instead of datetime.date(*args). He seems completely willing 
to accept the consequences of changing the constructor (namely that he will 
have to override all the methods that call the constructor).

It seems like good object oriented design to me.

-Mark
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Ethan Furman]

On 02/12/2015 06:39 PM, Alexander Belopolsky wrote:

 In my view, a constructor is no different from any other method.  If the 
 designers of the subclass decided to change the
 signature in an incompatible way, they should either override all methods 
 that create new objects or live with tracebacks.

 On the other hand, if all I want in my Date class is a better __format__ 
 method, I am forced to override all operators
 or have my objects silently degrade [...]

So there are basically two choices:

1) always use the type of the most-base class when creating new instances

   pros:
 - easy
 - speedy code
 - no possible tracebacks on new object instantiation

   cons:
 - a subclass that needs/wants to maintain itself must override all
   methods that create new instances, even if the only change is to
   the type of object returned

2) always use the type of self when creating new instances

   pros:
 - subclasses automatically maintain type
 - much less code in the simple cases [1]

   cons:
 - if constructor signatures change, must override all methods which
   create new objects

Unless there are powerful reasons against number 2 (such as performance, or the 
effort to affect the change), it sure
seems like the nicer way to go.

So back to my original question: what other concerns are there, and has anybody 
done any benchmarks?

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Ethan Furman]

On 02/12/2015 08:01 PM, Guido van Rossum wrote:
 On Thu, Feb 12, 2015 at 7:41 PM, Ethan Furman wrote:
 
 2) always use the type of self when creating new instances
 
cons:
  - if constructor signatures change, must override all methods which
create new objects
 
 Con for #2 is a showstopper. Forget about it.

Happy to, but can you explain why requiring the programmer to override the 
necessary methods, or get tracebacks, is a
showstopper?  Is there a previous email thread I can read that discusses it?

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Ethan Furman]

On 02/12/2015 06:57 PM, Steven D'Aprano wrote:
 On Thu, Feb 12, 2015 at 06:14:22PM -0800, Ethan Furman wrote:
 On 02/12/2015 05:46 PM, MRAB wrote:
 On 2015-02-13 00:55, Guido van Rossum wrote:
 
 Actually, the problem is that the base class (e.g. int) doesn't know how
 to construct an instance of the subclass -- there is no reason (in
 general) why the signature of a subclass constructor should match the
 base class constructor, and it often doesn't.

 So this is pretty much a no-go. It's not unique to Python -- it's a
 basic issue with OO.

 Really?

 What I was asking about, and Guido responded to, was not having to 
 specifically override __add__, __mul__, __sub__, and
 all the others; if we do override them then there is no problem.
 
 I think you have misunderstood MRAB's comment. My interpretation is 
 that MRAB is suggesting that methods in the base classes should use 
 type(self) rather than hard-coding their own type.

That makes more sense, thanks.

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Guido van Rossum]

On Thu, Feb 12, 2015 at 7:41 PM, Ethan Furman et...@stoneleaf.us wrote:

 On 02/12/2015 06:39 PM, Alexander Belopolsky wrote:

  In my view, a constructor is no different from any other method.  If the
 designers of the subclass decided to change the
  signature in an incompatible way, they should either override all
 methods that create new objects or live with tracebacks.

  On the other hand, if all I want in my Date class is a better __format__
 method, I am forced to override all operators
  or have my objects silently degrade [...]

 So there are basically two choices:

 1) always use the type of the most-base class when creating new instances

pros:
  - easy
  - speedy code
  - no possible tracebacks on new object instantiation

cons:
  - a subclass that needs/wants to maintain itself must override all
methods that create new instances, even if the only change is to
the type of object returned

 2) always use the type of self when creating new instances

pros:
  - subclasses automatically maintain type
  - much less code in the simple cases [1]

cons:
  - if constructor signatures change, must override all methods which
create new objects

 Unless there are powerful reasons against number 2 (such as performance,
 or the effort to affect the change), it sure
 seems like the nicer way to go.

 So back to my original question: what other concerns are there, and has
 anybody done any benchmarks?


Con for #2 is a showstopper. Forget about it.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[Steven D'Aprano]

On Thu, Feb 12, 2015 at 06:14:22PM -0800, Ethan Furman wrote:
 On 02/12/2015 05:46 PM, MRAB wrote:
  On 2015-02-13 00:55, Guido van Rossum wrote:

  Actually, the problem is that the base class (e.g. int) doesn't know how
  to construct an instance of the subclass -- there is no reason (in
  general) why the signature of a subclass constructor should match the
  base class constructor, and it often doesn't.
 
  So this is pretty much a no-go. It's not unique to Python -- it's a
  basic issue with OO.
 
  Really?
 
 What I was asking about, and Guido responded to, was not having to 
 specifically override __add__, __mul__, __sub__, and
 all the others; if we do override them then there is no problem.

I think you have misunderstood MRAB's comment. My interpretation is 
that MRAB is suggesting that methods in the base classes should use 
type(self) rather than hard-coding their own type.

E.g. if int were written in pure Python, it might look something like 
this:

class int(object):
def __new__(cls, arg):
...

def __add__(self, other):
return int(self, other)

(figuratively, rather than literally). But if it looked like this:

def __add__(self, other):
return type(self)(self, other)


then sub-classing would just work without the sub-class having to 
override each and every method.



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com