Re: [Python-ideas] divmod(): fallback to __floordiv__ and __mod__?

[Steven D'Aprano]

On Sat, Sep 17, 2016 at 09:01:53AM +, Spencer Brown wrote:

> Currently, calling divmod() on a class with __floordiv__ and __mod__ 
> defined, but not __divmod__ raises a TypeError. Is there any reason 
> why it doesn't fallback to (self // x, self % x)? 

Because things get really complex, fast. Does the added complexity pay 
its own way?

I may have some of the details wrong, but I understand the 
implementation of divmod() is something like this:

# pseudo-code for divmod
def divmod(a, b):
A, B = type(a), type(b)
if issubclass(B, A):
# Give priority to the reverse dunder version
if hasattr(B, '__rdivmod__'):
result = B.__rdivmod__(b, a)
if result is not NotImplemented:
return result
if hasattr(A, '__divmod__'):
result = A.__divmod__(a, b)
if result is not NotImplemented:
return result
raise TypeError
if hasattr(A, '__divmod__'):
result = A.__divmod__(a, b)
if result is not NotImplemented:
return result
if hasattr(B, '__rdivmod__'):
result = B.__rdivmod__(b, a)
if result is not NotImplemented:
return result
raise TypeError


only it will be in C, so probably three or five times as much code :-)

Now consider adding a fallback to __floordiv__ and __mod__ as suggested. 
That adds a lot more complexity:

- what if the object has __floordiv__, but not __mod__? Do you try 
  the reversed method, __rmod__, instead?

- likewise for NotImplemented?

- what if they're None?


So let's look at the complexity of the fallback:


# reverse dunder version, for when b is a subclass of a
rdiv = getattr(B, '__rfloordiv__, None)
rmod = getattr(B, '__rmod__, None)
if rdiv is not None and rmod is not None:
x, y = rdiv(b, a), rmod(b, a)
if x is NotImplemented:
div = getattr(A, '__floordiv__', None)
if div is not None:
x = div(a, b)
if x is NotImplemented:
raise TypeError
if y is NotImplemented:
mod = getattr(A, '__mod__', None)
if mod is not None:
y = mod(a, b)
if y is NotImplemented:
raise TypeError
assert NotImplemented not in (x, y)
else:
# try the __floordiv__ and __mod__ versions, without falling 
# back to reversed versions
...
return (x, y)


And then more or less the same for the "standard" case where a has 
priority over b (i.e. isn't a subclass). So, my estimate is that this 
will roughly triple the complexity of the divmod() function. Perhaps 
worse.

Oh, and I forgot the case where __divmod__ exists but is None, but I'm 
not going back to add it in because I'm not even sure where that ought 
to be tested.

Now I daresay we can refactor my naive pseudo-code above, but however 
you look at it, there's going to be a big jump in complexity for not a 
lot of benefit. In the case of __ne__ falling back on not __eq__, there 
is at least an utterly clear and obvious user-expectation that the two 
methods are linked, and there are no reversed __req__ and __rne__ 
methods to care about. That's not the case here.

Going the other way (fall back to __divmod__ if __floordiv__ or __mod__ 
is not defined) will probably be not quite as complex, but it will still 
add a fair amount of complexity.

So, *emotionally* I actually do quite like the idea of having divmod 
fall back to // and %, *and* the other way, but *intellectually* when I 
think about the complexity and the tests that will be needed to cover 
all the cases, I have to say "Uh uh, no way!"

It may be that the actual C implementation is simpler than I think, in 
which case maybe this is a reasonable idea, but on the face of it, I 
think that it will be really hard to get right, add a lot of code, and 
provide very little benefit.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] divmod(): fallback to __floordiv__ and __mod__?

[Ethan Furman]

On 09/17/2016 03:14 PM, Chris Angelico wrote:

On Sun, Sep 18, 2016 at 8:06 AM, Ethan Furman wrote:



Just like Python will use the defined __ne__ if
it's present, or fall back to negating the result of __eq__ if __ne__ is
not present, I see __divmod__ working the same way:

- is __mod__ present? use it
- is __floordiv__ present? use it
- otherwise, use __divmod__ and return the needed piece

I'm pretty sure __div__ should not fall back to __divmod__.


How does __mod__ fall back to __floordiv__? I'm lost.


Oops, sorry.  Got my directions reversed when thinking about how __div__ should 
fit in.

Bird's eye view: if the exact method needed is present, use it; otherwise if a 
fallback method is available, use that.

Currently this is done for __ne__ --> not __eq__, and I seem to remember 
another case or two that was talked about but I don't remember what they were and 
I'm not sure if they got implemented to follow the fallback pattern.

--
~Ethan~
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] divmod(): fallback to __floordiv__ and __mod__?

[Chris Angelico]

On Sun, Sep 18, 2016 at 8:06 AM, Ethan Furman  wrote:
> Just like Python will use the defined __ne__ if
> it's present, or fall back to negating the result of __eq__ if __ne__ is
> not present, I see __divmod__ working the same way:
>
> - is __mod__ present? use it
> - is __floordiv__ present? use it
> - otherwise, use __divmod__ and return the needed piece
>
> I'm pretty sure __div__ should not fall back to __divmod__.

How does __mod__ fall back to __floordiv__? I'm lost.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] divmod(): fallback to __floordiv__ and __mod__?

[Chris Angelico]

On Sun, Sep 18, 2016 at 3:57 AM, David Mertz  wrote:
> For example, '%' is fairly widely (ab)used for meanings other than modulo.
> E.g. string formatting.  Probably not that many classes that respond to '%'
> to do something non-modulo simultaneously implement `.__divmod__()` ... but
> maybe some use case is not obvious to me.  If those exist, your change would
> break that (well, depending whether methods of ancestors are used or not).

So there'll be classes that define __mod__ but not __divmod__ - that's
fine. Are there any that go the other way around?

It might technically be a breaking change, but it's unlikely to cause
very much breakage. So it'd be inappropriate for 3.6.1, but okay for
3.7.

+1 on the fallback exactly as Ethan described. The One Obvious Way to
implement arithmetic division would be to define __divmod__. Defining
__truediv__ or __mod__ would be for times when you can implement it
more efficiently by not calculating the other half, and of course the
times when they're not implementing division (cf pathlib.Path and str,
for / and % respectively).

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] divmod(): fallback to __floordiv__ and __mod__?

[David Mertz]

It seems like this could be something similar to `functools.total_ordering`
and decorate a class.  In principle that transformation could go in either
direction, but only if the decorator is used.

On Sat, Sep 17, 2016 at 3:56 AM, Mark Dickinson  wrote:

> On Sat, Sep 17, 2016 at 10:01 AM, Spencer Brown 
> wrote:
> > Currently, calling divmod() on a class with __floordiv__ and __mod__
> > defined, but not __divmod__ raises a TypeError. Is there any reason why
> it
> > doesn't fallback to (self // x, self % x)?
>
> It's an interesting idea. I wonder whether the falling back shouldn't
> be in the other direction, though: that is, if a class defines
> `__divmod__` but not `__floordiv__` or `__mod__`, perhaps the `//` and
> `%` operations should use `__divmod__`? That way, if you're writing a
> class that intends to support all three operations, you only have to
> write one method. And it might make sense from an efficiency
> perspective, too; it's common for a `divmod` computation to be cheaper
> than doing separate computations of the quotient and remainder.
>
> For the builtin int type, for example, in nontrivial cases Python
> computes both the quotient and remainder when asked to do a % or //
> operation, and then discards whichever part isn't needed. So in that
> case it would be wasteful to build up the divmod result from two
> separate % and // calls.
>
> --
> Mark
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/