[Python-Dev] == on object tests identity in 3.x - list delegation to members?
Am 13.07.2014 18:23, schrieb Steven D'Aprano: On Sun, Jul 13, 2014 at 05:13:20PM +0200, Andreas Maier wrote: Second, if not by delegation to equality of its elements, how would the equality of sequences defined otherwise? Wow. I'm impressed by the amount of detailed effort you've put into investigating this. (Too much detail to absorb, I'm afraid.) But perhaps you might have just asked on the python-l...@python.org mailing list, or here, where we would have told you the answer: list __eq__ first checks element identity before going on to check element equality. I apologize for not asking. It seems I was looking at the trees (behaviors of specific cases) without seeing the wood (identity goes first). If you can read C, you might like to check the list source code: http://hg.python.org/cpython/file/22e5a85ba840/Objects/listobject.c I can read (and write) C fluently, but (1) I don't have a build environment on my Windows system so I cannot debug it, and (2) I find it hard to judge from just looking at the C code which C function is invoked when the Python code enters the C code. (Quoting Raymond H. from his blog: Unless you know where to look, searching the source for an answer can be a time consuming intellectual investment.) So thanks for clarifying this. I guess I am arriving (slowly and still partly reluctantly, and I'm not alone with that feeling, it seems ...) at the bottom line of all this, which is that reflexivity is an important goal in Python, that self-written non-reflexive classes are not intended nor well supported, and that the non-reflexive NaN is considered an exception that cannot be expected to be treated consistently non-reflexive. This was discussed to death some time ago, both on python-dev and python-ideas. If you're interested, you can start here: https://mail.python.org/pipermail/python-list/2012-October/633992.html which is in the middle of one of the threads, but at least it gets you to the right time period. I read a number of posts in that thread by now. Sorry for not reading it earlier, but the mailing list archive just does not lend itself to searching the past. Of course, one can google it ;-) Andy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] == on object tests identity in 3.x - list delegation to members?
Am 13.07.2014 22:05, schrieb Akira Li: Nick Coghlan ncogh...@gmail.com writes: ... definition of floats and the definition of container invariants like assert x in [x]) The current approach means that the lack of reflexivity of NaN's stays confined to floats and similar types - it doesn't leak out and infect the behaviour of the container types. What we've never figured out is a good place to *document* it. I thought there was an open bug for that, but I can't find it right now. There was related issue Tuple comparisons with NaNs are broken http://bugs.python.org/issue21873 but it was closed as not a bug despite the corresponding behavior is *not documented* anywhere. I currently know about these two issues related to fixing the docs: http://bugs.python.org/11945 - about NaN values in containers http://bugs.python.org/12067 - comparisons I am working on the latter, currently. The patch only targets the comparisons chapter in the Language Reference, there is another comparisons chapter in the Library Reference, and one in the Tutorial. I will need to update the patch to issue 12067 as a result of this discussion. Andy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] == on object tests identity in 3.x - list delegation to members?
Am 14.07.2014 04:55, schrieb Ethan Furman: On 07/13/2014 08:13 AM, Andreas Maier wrote: Test #8: Same object of class C (C.__eq__() implemented with equality of x, C.__ne__() returning NotImplemented): obj1: type=class '__main__.C', str=C(256), id=39406504 obj2: type=class '__main__.C', str=C(256), id=39406504 a) obj1 is obj2: True C.__eq__(): self=39406504, other=39406504, returning True This is interesting/weird/odd -- why is __eq__ being called for an 'is' test? The debug messages are printed before the result is printed. So this is the debug message for the next case, 8.b). Sorry for not explaining it. Andy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] == on object tests identity in 3.x - list delegation to members?
Am 11.07.2014 22:54, schrieb Ethan Furman:
On 07/11/2014 07:04 AM, Andreas Maier wrote:
Am 09.07.2014 03:48, schrieb Raymond Hettinger:
Personally, I see no need to make the same mistake by removing
the identity-implies-equality rule from the built-in containers.
There's no need to upset the apple cart for nearly zero benefit.
Containers delegate the equal comparison on the container to their
elements; they do not apply identity-based comparison
to their elements. At least that is the externally visible behavior.
If that were true, then [NaN] == [NaN] would be False, and it is not.
Here is the externally visible behavior:
Python 3.5.0a0 (default:34881ee3eec5, Jun 16 2014, 11:31:20)
[GCC 4.7.3] on linux
Type help, copyright, credits or license for more information.
-- NaN = float('nan')
-- NaN == NaN
False
-- [NaN] == [NaN]
True
Ouch, that hurts ;-)
First, the delegation of sequence equality to element equality is not
something I have come up with during my doc patch. It has always been in
5.9 Comparisons of the Language Reference (copied from Python 3.4):
Tuples and lists are compared lexicographically using comparison of
corresponding elements. This means that to compare equal, each element
must compare equal and the two sequences must be of the same type and
have the same length.
Second, if not by delegation to equality of its elements, how would the
equality of sequences defined otherwise?
But your test is definitely worth having a closer look at. I have
broadened the test somewhat and that brings up further questions. Here
is the test output, and a discussion of the results (test program
try_eq.py and its output test_eq.out are attached to issue #12067):
Test #1: Different equal int objects:
obj1: type=class 'int', str=257, id=39305936
obj2: type=class 'int', str=257, id=39306160
a) obj1 is obj2: False
b) obj1 == obj2: True
c) [obj1] == [obj2]: True
d) {obj1:'v'} == {obj2:'v'}: True
e) {'k':obj1} == {'k':obj2}: True
f) obj1 == obj2: True
Discussion:
Case 1.c) can be interpreted that the list delegates its == to the == on
its elements. It cannot be interpreted to delegate to identity
comparison. That is consistent with how everyone (I hope ;-) would
expect int objects to behave, or lists or dicts of them.
The motivation for case f) is explained further down, it has to do with
caching.
Test #2: Same int object:
obj1: type=class 'int', str=257, id=39305936
obj2: type=class 'int', str=257, id=39305936
a) obj1 is obj2: True
b) obj1 == obj2: True
c) [obj1] == [obj2]: True
d) {obj1:'v'} == {obj2:'v'}: True
e) {'k':obj1} == {'k':obj2}: True
f) obj1 == obj2: True
- No surprises (I hope).
Test #3: Different equal float objects:
obj1: type=class 'float', str=257.0, id=5734664
obj2: type=class 'float', str=257.0, id=5734640
a) obj1 is obj2: False
b) obj1 == obj2: True
c) [obj1] == [obj2]: True
d) {obj1:'v'} == {obj2:'v'}: True
e) {'k':obj1} == {'k':obj2}: True
f) obj1 == obj2: True
Discussion:
I added this test only to show that float NaN is a special case, and
that this test for float objects - that are not NaN - behaves like test
#1 for int objects.
Test #4: Same float object:
obj1: type=class 'float', str=257.0, id=5734664
obj2: type=class 'float', str=257.0, id=5734664
a) obj1 is obj2: True
b) obj1 == obj2: True
c) [obj1] == [obj2]: True
d) {obj1:'v'} == {obj2:'v'}: True
e) {'k':obj1} == {'k':obj2}: True
f) obj1 == obj2: True
- Same as test #2, hopefully no surprises.
Test #5: Different float NaN objects:
obj1: type=class 'float', str=nan, id=5734784
obj2: type=class 'float', str=nan, id=5734976
a) obj1 is obj2: False
b) obj1 == obj2: False
c) [obj1] == [obj2]: False
d) {obj1:'v'} == {obj2:'v'}: False
e) {'k':obj1} == {'k':obj2}: False
f) obj1 == obj2: False
Discussion:
Here, the list behaves as I would expect under the rule that it
delegates equality to its elements. Case c) allows that interpretation.
However, an interpretation based on identity would also be possible.
Test #6: Same float NaN object:
obj1: type=class 'float', str=nan, id=5734784
obj2: type=class 'float', str=nan, id=5734784
a) obj1 is obj2: True
b) obj1 == obj2: False
c) [obj1] == [obj2]: True
d) {obj1:'v'} == {obj2:'v'}: True
e) {'k':obj1} == {'k':obj2}: True
f) obj1 == obj2: False
Discussion (this is Ethan's example):
Case 6.b) shows the special behavior of float NaN that is documented: a
float NaN object is the same as itself but unequal to itself.
Case 6.c) is the surprising case. It could be interpreted in two ways
(at least that's what I found):
1) The comparison is based on identity of the float objects. But that is
inconsistent with test #4. And why would the list special-case NaN
comparison in such a way that it ends up being inconsistent with the
special definition of NaN (outside of the list)?
2) The list does not always delegate to element equality, but
Re: [Python-Dev] == on object tests identity in 3.x - list delegation to members?
On Sun, Jul 13, 2014 at 05:13:20PM +0200, Andreas Maier wrote: Second, if not by delegation to equality of its elements, how would the equality of sequences defined otherwise? Wow. I'm impressed by the amount of detailed effort you've put into investigating this. (Too much detail to absorb, I'm afraid.) But perhaps you might have just asked on the python-l...@python.org mailing list, or here, where we would have told you the answer: list __eq__ first checks element identity before going on to check element equality. If you can read C, you might like to check the list source code: http://hg.python.org/cpython/file/22e5a85ba840/Objects/listobject.c but if I'm reading it correctly, list.__eq__ conceptually looks something like this: def __eq__(self, other): if not isinstance(other, list): return NotImplemented if len(other) != len(self): return False for a, b in zip(self, other): if not (a is b or a == b): return False return True (The actual code is a bit more complex than that, since there is a single function, list_richcompare, which handles all the rich comparisons.) The critical test is PyObject_RichCompareBool here: http://hg.python.org/cpython/file/22e5a85ba840/Objects/object.c which explicitly says: /* Quick result when objects are the same. Guarantees that identity implies equality. */ [...] I added this test only to show that float NaN is a special case, NANs are not a special case. List __eq__ treats all object types identically (pun intended): py class X: ... def __eq__(self, other): return False ... py x = X() py x == x False py [x] == [X()] False py [x] == [x] True [...] Case 6.c) is the surprising case. It could be interpreted in two ways (at least that's what I found): 1) The comparison is based on identity of the float objects. But that is inconsistent with test #4. And why would the list special-case NaN comparison in such a way that it ends up being inconsistent with the special definition of NaN (outside of the list)? It doesn't. NANs are not special cased in any way. This was discussed to death some time ago, both on python-dev and python-ideas. If you're interested, you can start here: https://mail.python.org/pipermail/python-list/2012-October/633992.html which is in the middle of one of the threads, but at least it gets you to the right time period. 2) The list does not always delegate to element equality, but attempts to optimize if the objects are the same (same identity). Right! It's not just lists -- I believe that tuples, dicts and sets behave the same way. We will see later that that happens. Further, when comparing float NaNs of the same identity, the list implementation forgot to special-case NaNs. Which would be a bug, IMHO. Forgot? I don't think the behaviour of list comparisons is an accident. NAN equality is non-reflexive. Very few other things are the same. It would be seriously weird if alist == alist could return False. You'll note that the IEEE-754 standard has nothing to say about the behaviour of Python lists containing NANs, so we're free to pick whatever behaviour makes the most sense for Python, and that is to minimise the Gotcha! factor. NANs are a gotcha to anyone who doesn't know IEEE-754, and possibly even some who do. I will go to the barricades to fight to keep the non-reflexivity of NANs *in isolation*, but I believe that Python has made the right decision to treat lists containing NANs the same as everything else. NAN == NAN # obeys IEEE-754 semantics and returns False [NAN] == [NAN] # obeys standard expectation that equality is reflexive This behaviour is not a bug, it is a feature. As far as I am concerned, this only needs documenting. If anyone needs list equality to honour the special behaviour of NANs, write a subclass or an equal() function. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] == on object tests identity in 3.x - list delegation to members?
On Mon, Jul 14, 2014 at 2:23 AM, Steven D'Aprano st...@pearwood.info wrote: We will see later that that happens. Further, when comparing float NaNs of the same identity, the list implementation forgot to special-case NaNs. Which would be a bug, IMHO. Forgot? I don't think the behaviour of list comparisons is an accident. Well, forgot is on the basis that the identity check is intended to be a mere optimization. If that were the case (don't actually call __eq__ when you reckon it'll return True), then yes, failing to special-case NaN would be a bug. But since it's intended behaviour, as explained further down, it's not a bug and not the result of forgetfulness. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] == on object tests identity in 3.x - list delegation to members?
On 13 July 2014 11:34, Chris Angelico ros...@gmail.com wrote: On Mon, Jul 14, 2014 at 2:23 AM, Steven D'Aprano st...@pearwood.info wrote: We will see later that that happens. Further, when comparing float NaNs of the same identity, the list implementation forgot to special-case NaNs. Which would be a bug, IMHO. Forgot? I don't think the behaviour of list comparisons is an accident. Well, forgot is on the basis that the identity check is intended to be a mere optimization. If that were the case (don't actually call __eq__ when you reckon it'll return True), then yes, failing to special-case NaN would be a bug. But since it's intended behaviour, as explained further down, it's not a bug and not the result of forgetfulness. Right, it's not a mere optimisation - it's the only way to get containers to behave sensibly. Otherwise we'd end up with nonsense like: x = float(nan) x in [x] False That currently returns True because of the identity check - it would return False if we delegated the check to float.__eq__ because the defined IEEE754 behaviour for NaN's breaks the mathematical definition of an equivalence class as a transitive, reflexive and commutative operation. (It breaks it for *good reasons*, but we still need to figure out a way of dealing with the impedance mismatch between the definition of floats and the definition of container invariants like assert x in [x]) The current approach means that the lack of reflexivity of NaN's stays confined to floats and similar types - it doesn't leak out and infect the behaviour of the container types. What we've never figured out is a good place to *document* it. I thought there was an open bug for that, but I can't find it right now. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] == on object tests identity in 3.x - list delegation to members?
On Mon, Jul 14, 2014 at 4:11 AM, Nick Coghlan ncogh...@gmail.com wrote: What we've never figured out is a good place to *document* it. I thought there was an open bug for that, but I can't find it right now. Yeah. The Py3 docs explain why x in [x] is True, but I haven't found a parallel explanation of sequence equality. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] == on object tests identity in 3.x - list delegation to members?
On 13 July 2014 13:16, Chris Angelico ros...@gmail.com wrote: On Mon, Jul 14, 2014 at 4:11 AM, Nick Coghlan ncogh...@gmail.com wrote: What we've never figured out is a good place to *document* it. I thought there was an open bug for that, but I can't find it right now. Yeah. The Py3 docs explain why x in [x] is True, but I haven't found a parallel explanation of sequence equality. We might need to expand the tables of sequence operations to cover equality and inequality checks - those are currently missing. Cheers, Nick. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] == on object tests identity in 3.x - list delegation to members?
Nick Coghlan ncogh...@gmail.com: Right, it's not a mere optimisation - it's the only way to get containers to behave sensibly. Otherwise we'd end up with nonsense like: x = float(nan) x in [x] False Why is that nonsense? I mean, why is it any more nonsense than x == x False Anyway, personally, I'm perfectly happy to live with the choices of past generations, regardless of whether they were good or not. What you absolutely don't want to do is correct the choices of past generations. Marko ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] == on object tests identity in 3.x - list delegation to members?
Nick Coghlan ncogh...@gmail.com writes: ... definition of floats and the definition of container invariants like assert x in [x]) The current approach means that the lack of reflexivity of NaN's stays confined to floats and similar types - it doesn't leak out and infect the behaviour of the container types. What we've never figured out is a good place to *document* it. I thought there was an open bug for that, but I can't find it right now. There was related issue Tuple comparisons with NaNs are broken http://bugs.python.org/issue21873 but it was closed as not a bug despite the corresponding behavior is *not documented* anywhere. -- Akira ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] == on object tests identity in 3.x - list delegation to members?
On 07/13/2014 08:13 AM, Andreas Maier wrote:
Am 11.07.2014 22:54, schrieb Ethan Furman:
Here is the externally visible behavior:
Python 3.5.0a0 (default:34881ee3eec5, Jun 16 2014, 11:31:20)
[GCC 4.7.3] on linux
Type help, copyright, credits or license for more information.
-- NaN = float('nan')
-- NaN == NaN
False
-- [NaN] == [NaN]
True
Ouch, that hurts ;-)
Yeah, I've been bitten enough times that now I try to always test code before I
post. ;)
Test #8: Same object of class C
(C.__eq__() implemented with equality of x,
C.__ne__() returning NotImplemented):
obj1: type=class '__main__.C', str=C(256), id=39406504
obj2: type=class '__main__.C', str=C(256), id=39406504
a) obj1 is obj2: True
C.__eq__(): self=39406504, other=39406504, returning True
This is interesting/weird/odd -- why is __eq__ being called for an 'is' test?
--- test_eq.py
class TestEqTrue:
def __eq__(self, other):
print('Test.__eq__ returning True')
return True
class TestEqFalse:
def __eq__(self, other):
print('Test.__eq__ returning False')
return False
tet = TestEqTrue()
print(tet is tet)
print(tet in [tet])
tef = TestEqFalse()
print(tef is tef)
print(tef in [tef])
---
When I run this all I get is four Trues, never any messages about being in
__eq__.
How did you get that result?
--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] == on object tests identity in 3.x - list delegation to members?
On 07/13/2014 10:33 PM, Andreas Maier wrote: Am 14.07.2014 04:55, schrieb Ethan Furman: On 07/13/2014 08:13 AM, Andreas Maier wrote: Test #8: Same object of class C (C.__eq__() implemented with equality of x, C.__ne__() returning NotImplemented): obj1: type=class '__main__.C', str=C(256), id=39406504 obj2: type=class '__main__.C', str=C(256), id=39406504 a) obj1 is obj2: True C.__eq__(): self=39406504, other=39406504, returning True This is interesting/weird/odd -- why is __eq__ being called for an 'is' test? The debug messages are printed before the result is printed. So this is the debug message for the next case, 8.b). Ah, whew! That's a relief. Sorry for not explaining it. Had I been reading more closely I would (hopefully) have noticed that, but I was headed out the door at the time. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
