[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Steven D'Aprano]

On Sun, Jul 26, 2020 at 08:22:47PM -0700, Guido van Rossum wrote:

> Regarding `__hash__`, it is a very bad idea to call `super().__hash__()`!

Today I learned. Thank you.

-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LSXQMDWABS7HLRIOSVNUTDUN3EO6Y3KO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Guido van Rossum]

I am really surprised at the resistance against defining `__eq__` on the
target objects. Every time this problem has cropped up in code I was
working on (including code part of very large corporate code bases) the
obvious solution was to define `__eq__`. The only reason I can think of why
you are so resistant to this would be due to poor development practices,
e.g. adding tests long after the "main" code has already been deployed, or
having a separate team write tests.

Regarding `__hash__`, it is a very bad idea to call `super().__hash__()`!
Unless your `__eq__` also just calls `super().__eq__(other)` (and what
would be the point of that?), defining `__hash__` that way will cause
irreproducible behavior where *sometimes* an object that is equal to a dict
key will not be found in the dict even though it is already present,
because the two objects have different hash values. Defining `__hash__` as
`id(self)` is no better. In fact, defining `__hash__` as returning the
constant `42` is better, because it is fine if two objects that *don't*
compare equal still have the same hash value (but not the other way around).

The right way to define `__hash__` is to construct a tuple of all the
attributes that are considered by `__eq__` and return the `hash()` of that
tuple. (In some cases you can make it faster by leaving some expensive
attribute out of the tuple -- again, that's fine, but don't consider
anything that's not used by `__eq__`.)

Finally, dataclasses get you all this for free, and they are the future.

On Sun, Jul 26, 2020 at 7:48 PM Henry Lin  wrote:

> @Steven D'Aprano  All good ideas ☺ I'm in agreement
> that we should be building solutions which are generalizable.
>
> Are there more concerns people would like to bring up when considering the
> problem of object equality?
>
> On Sun, Jul 26, 2020 at 9:25 PM Steven D'Aprano 
> wrote:
>
>> On Sun, Jul 26, 2020 at 11:12:39PM +0200, Alex Hall wrote:
>>
>> > There's another reason people might find this useful - if the objects
>> have
>> > differing attributes, the assertion can show exactly which ones,
>> instead of
>> > just saying that the objects are not equal.
>>
>> That's a good point.
>>
>> I sat down to start an implementation, when a fundamental issue with
>> this came to mind. This proposed comparison is effectively something
>> close to:
>>
>> vars(actual) == vars(expected)
>>
>> only recursively and with provision for objects with `__slots__` and/or
>> no `__dict__`. And that observation lead me to the insight that as tests
>> go, this is a risky, unreliable test.
>>
>> A built-in example:
>>
>>
>> actual = lambda: 1  # simulate some complex object
>> expected = lambda: 2  # another complex object
>> vars(actual) == vars(expected)  # returns True
>>
>>
>> So this is a comparison that needs to be used with care. It is easy for
>> the test to pass while the objects are nevertheless not what you expect.
>>
>> Having said that, another perspective is that unittest already has a
>> smart test for comparing dicts, assertDictEqual, which is automatically
>> called by assertEqual.
>>
>>
>> https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertDictEqual
>>
>> So it may be sufficient to have a utility function that copies
>> an instance's slots and dict into a dict, and then compare dicts. Here's
>> a sketch:
>>
>> d1 = vars(actual).copy()
>> d1.update({key: value for key in actual.__slots__})
>> # Likewise for d2 from expected
>> self.assertEqual(d1, d2)
>>
>> Make that handle the corner cases where objects have no instance dict or
>> slots, and we're done.
>>
>> Thinking aloud here I see this as a kind of copy operation, and
>> think this would be useful outside of testing. I've written code to copy
>> attributes from instances on multiple occasions. So how about a new
>> function in the `copy` module to do so:
>>
>> copy.getattrs(obj, deep=False)
>>
>> that returns a dict. Then the desired comparison could be a thin
>> wrapper:
>>
>> def assertEqualAttrs(self, actual, expected, msg=None):
>> self.assertEqual(getattrs(actual), getattrs(expected))
>>
>>
>> I'm not keen on a specialist test function, but I'm warming to the idea
>> of exposing this functionality in a more general, and hence more useful,
>> form.
>>
>>
>> --
>> Steven
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/MLRFS6RO7WF2UAEOS4YMH2FXRQHJUGWU/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Henry Lin]

@Steven D'Aprano  All good ideas ☺ I'm in agreement
that we should be building solutions which are generalizable.

Are there more concerns people would like to bring up when considering the
problem of object equality?

On Sun, Jul 26, 2020 at 9:25 PM Steven D'Aprano  wrote:

> On Sun, Jul 26, 2020 at 11:12:39PM +0200, Alex Hall wrote:
>
> > There's another reason people might find this useful - if the objects
> have
> > differing attributes, the assertion can show exactly which ones, instead
> of
> > just saying that the objects are not equal.
>
> That's a good point.
>
> I sat down to start an implementation, when a fundamental issue with
> this came to mind. This proposed comparison is effectively something
> close to:
>
> vars(actual) == vars(expected)
>
> only recursively and with provision for objects with `__slots__` and/or
> no `__dict__`. And that observation lead me to the insight that as tests
> go, this is a risky, unreliable test.
>
> A built-in example:
>
>
> actual = lambda: 1  # simulate some complex object
> expected = lambda: 2  # another complex object
> vars(actual) == vars(expected)  # returns True
>
>
> So this is a comparison that needs to be used with care. It is easy for
> the test to pass while the objects are nevertheless not what you expect.
>
> Having said that, another perspective is that unittest already has a
> smart test for comparing dicts, assertDictEqual, which is automatically
> called by assertEqual.
>
>
> https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertDictEqual
>
> So it may be sufficient to have a utility function that copies
> an instance's slots and dict into a dict, and then compare dicts. Here's
> a sketch:
>
> d1 = vars(actual).copy()
> d1.update({key: value for key in actual.__slots__})
> # Likewise for d2 from expected
> self.assertEqual(d1, d2)
>
> Make that handle the corner cases where objects have no instance dict or
> slots, and we're done.
>
> Thinking aloud here I see this as a kind of copy operation, and
> think this would be useful outside of testing. I've written code to copy
> attributes from instances on multiple occasions. So how about a new
> function in the `copy` module to do so:
>
> copy.getattrs(obj, deep=False)
>
> that returns a dict. Then the desired comparison could be a thin
> wrapper:
>
> def assertEqualAttrs(self, actual, expected, msg=None):
> self.assertEqual(getattrs(actual), getattrs(expected))
>
>
> I'm not keen on a specialist test function, but I'm warming to the idea
> of exposing this functionality in a more general, and hence more useful,
> form.
>
>
> --
> Steven
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/MLRFS6RO7WF2UAEOS4YMH2FXRQHJUGWU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/U6RQ6GA7RXYHTEORTD3VBZ4KMJMXMB5O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Steven D'Aprano]

On Sun, Jul 26, 2020 at 11:12:39PM +0200, Alex Hall wrote:

> There's another reason people might find this useful - if the objects have
> differing attributes, the assertion can show exactly which ones, instead of
> just saying that the objects are not equal. 

That's a good point.

I sat down to start an implementation, when a fundamental issue with 
this came to mind. This proposed comparison is effectively something 
close to:

vars(actual) == vars(expected)

only recursively and with provision for objects with `__slots__` and/or 
no `__dict__`. And that observation lead me to the insight that as tests 
go, this is a risky, unreliable test.

A built-in example:


actual = lambda: 1  # simulate some complex object
expected = lambda: 2  # another complex object
vars(actual) == vars(expected)  # returns True


So this is a comparison that needs to be used with care. It is easy for 
the test to pass while the objects are nevertheless not what you expect.

Having said that, another perspective is that unittest already has a 
smart test for comparing dicts, assertDictEqual, which is automatically 
called by assertEqual.

https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertDictEqual

So it may be sufficient to have a utility function that copies 
an instance's slots and dict into a dict, and then compare dicts. Here's 
a sketch:

d1 = vars(actual).copy()
d1.update({key: value for key in actual.__slots__})
# Likewise for d2 from expected
self.assertEqual(d1, d2)

Make that handle the corner cases where objects have no instance dict or 
slots, and we're done.

Thinking aloud here I see this as a kind of copy operation, and 
think this would be useful outside of testing. I've written code to copy 
attributes from instances on multiple occasions. So how about a new 
function in the `copy` module to do so:

copy.getattrs(obj, deep=False)

that returns a dict. Then the desired comparison could be a thin 
wrapper:

def assertEqualAttrs(self, actual, expected, msg=None):
self.assertEqual(getattrs(actual), getattrs(expected))


I'm not keen on a specialist test function, but I'm warming to the idea 
of exposing this functionality in a more general, and hence more useful, 
form.


-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MLRFS6RO7WF2UAEOS4YMH2FXRQHJUGWU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Henry Lin]

@Steven D'Aprano 

>- Developers might not want to leak the `__eq__` function to other
> >developers; I wouldn't want to invade the implementation of my class
> just
> >for testing.
> That seems odd to me. You are *literally* comparing two instances for
> equality, just calling it something different from `==`. Why would you
> not be happy to expose it?


My thinking is by default, the `==` operator checks whether two objects
have the same reference. So implementing `__eq__` is actually a breaking
change for developers. It seems by consensus of people here, people do tend
to implement `__eq__` anyways, so maybe this point is minor.

I do appreciate the suggestion of adding this feature into functools
though.

Let's assume we commit to doing something like this. Thinking how this
feature can be extended, let's suppose for testing purposes, I want to
highlight which attributes of two objects are mismatching. Would we have to
implement something different to find the delta between two objects, or
could components of the functools solution be reused? (Would we want a
feature like this to exist in the standard library?)

On Sun, Jul 26, 2020 at 8:29 PM Steven D'Aprano  wrote:

> On Sun, Jul 26, 2020 at 07:47:39PM +0200, Marco Sulla wrote:
> > On Sun, 26 Jul 2020 at 19:33, Henry Lin  wrote:
> >
> > >
> > >- Any class implementing the `__eq__` operator is no longer hashable
> > >
> > >
> > You can use:
> >
> > def __hash__(self):
> > return id(self)
>
> Don't do that. It's a horrible hash function.
>
> The `object` superclass already knows how to do a good, reliable hash
> function. Use it.
>
> def __hash__(self):
> return super().__hash__()
>
>
> --
> Steven
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/7N3GMYKC46MRA4DUNS2C5R2CA4CJGMOG/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7MC27A465I5ZB6PV4S64C2XTAVNLTKIF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Steven D'Aprano]

On Sun, Jul 26, 2020 at 07:47:39PM +0200, Marco Sulla wrote:
> On Sun, 26 Jul 2020 at 19:33, Henry Lin  wrote:
> 
> >
> >- Any class implementing the `__eq__` operator is no longer hashable
> >
> >
> You can use:
> 
> def __hash__(self):
> return id(self)

Don't do that. It's a horrible hash function.

The `object` superclass already knows how to do a good, reliable hash 
function. Use it.

def __hash__(self):
return super().__hash__()


-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7N3GMYKC46MRA4DUNS2C5R2CA4CJGMOG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Steven D'Aprano]

On Sun, Jul 26, 2020 at 12:31:17PM -0500, Henry Lin wrote:
> Hi Steven,
> 
> You're right, declaring `__eq__` for the class we want to compare would
> solve this issue. However, we have the tradeoff that
> 
>- All classes need to implement the `__eq__` method to compare two
>instances;

One argument in favour of a standard solution would be to avoid 
duplicated implementations. Perhaps we should add something, not as a 
unittest method, but in functools:

def compare(a, b):
if a is b:
return True
# Simplified version.
return vars(a) == vars(b)

The actual implementation would be more complex, of course. Then classes 
could optionally implement equality:


def __eq__(self, other):
if isinstance(other, type(self):
return functools.compare(self, other)
return NotImplemented

or if you prefer, you could call the function directly in your unit 
tests:

self.assertTrue(functools.compare(actual, other))



>- Any class implementing the `__eq__` operator is no longer hashable

Easy enough to add back in:

def __hash__(self):
return super().__hash__()


>- Developers might not want to leak the `__eq__` function to other
>developers; I wouldn't want to invade the implementation of my class just
>for testing.

That seems odd to me. You are *literally* comparing two instances for 
equality, just calling it something different from `==`. Why would you 
not be happy to expose it?


> In terms of the "popularity" of this potential feature, from what I
> understand (and through my own development), there are testing libraries
> built with this feature. For example, testfixtures.compare
> 
> can compare two objects recursively, and I am using it in my development
> for this purpose.

That's a good example of what we should *not* do, and why trying to 
create a single standard solution for every imaginable scenario can only 
end up with an over-engineered, complex, complicated, confusing API:

testfixtures.compare(
x, y,
prefix=None, 
suffix=None, 
raises=True, 
recursive=True, 
strict=False, 
comparers=None, 
**kw)

Not shown in the function signature are additional keyword arguments:

actual, expected # alternative spelling for x, y
x_label,
y_label,
ignore_eq

That is literally thirteen optional parameters, plus arbitrary keyword 
parameters, for something that just compares two objects.

But a simple comparison function, possibly in functools, that simply 
compares attributes, might be worthwhile.



-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/56GD3DDO2PLNOB4TIIO7PBJCUPFLGA3V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Henry Lin]

+1 to Alex Hall.

In general I think there are a lot of questions regarding whether using the
__eq__ operator is sufficient. It seems from people's feedback that it will
essentially get the job done, but like Alex says, if we want to understand
which field is leading to a test breaking, we wouldn't have the ability to
easily check.

On Sun, Jul 26, 2020 at 4:13 PM Alex Hall  wrote:

> On Sun, Jul 26, 2020 at 11:01 PM Ethan Furman  wrote:
>
>> On 7/26/20 10:31 AM, Henry Lin wrote:
>>
>> > You're right, declaring `__eq__` for the class we want to compare would
>> > solve this issue. However, we have the tradeoff that
>> >
>> >   * All classes need to implement the `__eq__` method to compare two
>> > instances;
>>
>> I usually implement __eq__ sooner or later anyway -- even if just for
>> testing.
>>
>> >   * Any class implementing the `__eq__` operator is no longer hashable
>>
>> One just needs to define a __hash__ method that behaves properly.
>>
>
> This is quite a significant change in behaviour which may break
> compatibility. Equality and hashing based only on identity can be quite a
> useful property which I often rely on.
>
> There's another reason people might find this useful - if the objects have
> differing attributes, the assertion can show exactly which ones, instead of
> just saying that the objects are not equal. Even if all the involved
> classes implement a matching repr, which is yet more work, the reprs will
> likely be on a single line and the diff will be difficult to read.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/ZJYGN42PCO4J73AAM7ZZSVTOFHPBADWT/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YFGS3DVFNLZ34BPCPTRBXC7H7WLCZ5FK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] PEP8 update: using typing.TypeVar before function defenition

[Nikita Serba via Python-ideas]

Hello!


According to PEP8, there should be 2 blank lines before the function 
definition. But,
I think in case of defining new typing.TypeVar, that will be used only in one 
function
definition, it would be better to put it right before the function definition, 
i.e. with no
blank lines. So I suggest to update PEP8, so it'd recommend skipping blank 
lines in this
case.


Thank you for your attention and sorry for my bad English.


Sincerely yours,
Nikita Serba.___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UVAQNA5S2SL3XAF2Z2F2RBCNJNYTPTXV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Alex Hall]

On Sun, Jul 26, 2020 at 11:01 PM Ethan Furman  wrote:

> On 7/26/20 10:31 AM, Henry Lin wrote:
>
> > You're right, declaring `__eq__` for the class we want to compare would
> > solve this issue. However, we have the tradeoff that
> >
> >   * All classes need to implement the `__eq__` method to compare two
> > instances;
>
> I usually implement __eq__ sooner or later anyway -- even if just for
> testing.
>
> >   * Any class implementing the `__eq__` operator is no longer hashable
>
> One just needs to define a __hash__ method that behaves properly.
>

This is quite a significant change in behaviour which may break
compatibility. Equality and hashing based only on identity can be quite a
useful property which I often rely on.

There's another reason people might find this useful - if the objects have
differing attributes, the assertion can show exactly which ones, instead of
just saying that the objects are not equal. Even if all the involved
classes implement a matching repr, which is yet more work, the reprs will
likely be on a single line and the diff will be difficult to read.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZJYGN42PCO4J73AAM7ZZSVTOFHPBADWT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Ethan Furman]

On 7/26/20 10:31 AM, Henry Lin wrote:

You're right, declaring `__eq__` for the class we want to compare would 
solve this issue. However, we have the tradeoff that


  * All classes need to implement the `__eq__` method to compare two
instances;


I usually implement __eq__ sooner or later anyway -- even if just for 
testing.



  * Any class implementing the `__eq__` operator is no longer hashable


One just needs to define a __hash__ method that behaves properly.


  * Developers might not want to leak the `__eq__` function to other
developers; I wouldn't want to invade the implementation of my class
just for testing.


And yet that's exactly what you are proposing with your object compare. 
If two objects are, in fact, equal, why is it bad for == to say so?


--
~Ethan~
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XFX6SXJ3QULMW3N5GZFA7CGDVXWF3KQM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Richard Damon]

On 7/26/20 4:09 PM, Marco Sulla wrote:
> You're quite right, but if you don't implement __eq__, the hash of an
> object is simply a random integer (I suppose generated from the
> address of the object).
>
> Alternatively, if you want a quick hash, you can use hash(str(obj))
> (if you implemented __str__ or __repr__).
>
And if you don't implement __eq__, I thought that the default equal was
same id(), (which is what the hash is based on too).

The idea was (I thought) that if you implement an __eq__, so that two
different object could compare equal, then you needed to come up with
some hash function for that object that matched that equality function,
or the object is considered unhashable.

-- 
Richard Damon
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NWVR4JJPJYJLJDVINNYTZSR4AAPKNHXK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Marco Sulla]

You're quite right, but if you don't implement __eq__, the hash of an
object is simply a random integer (I suppose generated from the address of
the object).

Alternatively, if you want a quick hash, you can use hash(str(obj)) (if you
implemented __str__ or __repr__).
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QDH4JUM547J6AGEEQ6XXM5RVLKAXC5JE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Richard Damon]

On 7/26/20 1:47 PM, Marco Sulla wrote:
> On Sun, 26 Jul 2020 at 19:33, Henry Lin  > wrote:
>
>   * Any class implementing the `__eq__` operator is no longer hashable
>
>
> You can use:
>
> def __hash__(self):
>     return id(self)
I thought that there was an assumption that if two objects are equal
(via __eq__) then their hashes (via __hash__) should be equal? Which
wouldn't hold for this definition, and thus dictionaries wouldn't behave
as expected.

-- 
Richard Damon
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/T5DLPPPMWLGYAYNZ7LXG2IZJJWD5RWF6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Marco Sulla]

On Sun, 26 Jul 2020 at 19:33, Henry Lin  wrote:

>
>- Any class implementing the `__eq__` operator is no longer hashable
>
>
You can use:

def __hash__(self):
return id(self)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AQESXZVDMQHY367WVYNH2N46R67CD5OF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Henry Lin]

Hi Steven,

You're right, declaring `__eq__` for the class we want to compare would
solve this issue. However, we have the tradeoff that

   - All classes need to implement the `__eq__` method to compare two
   instances;
   - Any class implementing the `__eq__` operator is no longer hashable
   - Developers might not want to leak the `__eq__` function to other
   developers; I wouldn't want to invade the implementation of my class just
   for testing.

In terms of the "popularity" of this potential feature, from what I
understand (and through my own development), there are testing libraries
built with this feature. For example, testfixtures.compare

can compare two objects recursively, and I am using it in my development
for this purpose.

On Sun, Jul 26, 2020 at 4:56 AM Steven D'Aprano  wrote:

> On Sat, Jul 25, 2020 at 10:15:16PM -0500, Henry Lin wrote:
> > Hey all,
> >
> > What are thoughts about implementing an object-compare function in the
> > unittest package? (Compare two objects recursively, attribute by
> > attribute.)
>
> Why not just ask the objects to compare themselves?
>
> assertEqual(actual, expected)
>
> will work if actual and expected define a sensible `__eq__` and are the
> same type. If they aren't the same type, why not?
>
> actual = MyObject(spam=1, eggs=2, cheese=3)
> expected = DifferentObject(spam=1, eggs=2, cheese=3)
>
>
> > This seems like a common use case in many testing scenarios,
>
> I've never come across it. Can you give an example where defining an
> `__eq__` method won't be the right solution?
>
>
>
>
> --
> Steven
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/TDLFBURVX4N4JJP4ELIRLKULR775VNOY/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LZYZIDMFBRIHHPWITSGZT6ITIA2P2ZUW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: New clause in FOR and WHILE instead of ELSE

[Rob Cliffe via Python-ideas]

On 25/07/2020 20:11, Christopher Barker wrote:
On Sat, Jul 25, 2020 at 4:39 AM Eric V. Smith > wrote:


On 7/25/2020 7:28 AM, Chris Angelico wrote:
> Do you REALLY think that everyone will automatically understand it
> just because it's spelled "for... then" instead of "for... else"?

I wouldn't find "for ... then" any less confusing than "for ...
else". I
do find "for ... else if not break" easier to understand,


I'm pretty sure that the OP on this proposed "then" as a placeholder, 
with the final choice of word to be determined. But I do think this is 
pretty darn illustrative --'cause I would find "then" even LESS 
intuitive, and it's a whole new keyword to boot!


I'm one of those folks that have been around Python for a long time -- 
starting with 1.5 -- and I still find for-else a bit confusing, but I 
remember that it exists, so go figure it out when I need it. And that 
is not very often. Frankly, it's a nifty feature, but it's still not a 
very common use case -- so It's just not a very clear win to spell it 
better. And as the "then" example shows, it's really hard to find a 
better spelling -- it's just not that common a construct in 
programming languages, and it's not used that often in Python -- if it 
were, I"d probably remember what it meant :-)


but I'm not
convinced its worth the hassle over "for ... else # if not break".
Four
keywords in a row!


well, there are two sides to usability:

1) Remembering it's there, and how to use it when you are writing 
code, which I think is not such a big deal -- if you remember its 
there, you can figure out how to use it. And if you don't remember 
it's there, then a better spelling isn't going to help.


2) understanding what it means when you are reading code -- and that, 
a better spelling would help with. But I don't get folks that would 
reject it in a code review -- I'd just suggest that you add a # if not 
break comment -- or better yet a comment that describes why it's there 
in the context of that problem -- as comments are supposed to do.


Would you really as someone to replace:

for dodad in all_the_things:
    if dodad == something_special:
found_special_dodad = True
    break
do_things_with(dodad)
else: # the special one wasn't there -- need to process the stuff
    process(stuff)

with this?:

found_special_dodad = False
for dodad in all_the_things:
    if dodad == something_special:
found_special_dodad = True
    break
do_things_with(dodad)
    ...
if not found_special_dodad:
    # the special one wasn't there -- need to process the stuff
    process(stuff)

The for--else construct exists for a reason: it's much cleaner than 
creating a flag and checking it later. Sure it would be nicer if it 
had a better spelling, but else  # not break is pretty easy if you 
want to make your code more readable and there isn't need for an 
explanatory comment otherwise.


I'm not advocating for any change (Obviously :-) ) -- but if there 
were to be one, I"d strongly vote for some version of else not break 
or else if not break because it introduces no keywords, and it's a lot 
more compatible documentation-wise than a different word -- it's 
almost like a built-in commnet :-) By that I mean that folks that see 
the two different spellings in two different tutorials or SO answers 
,or whatever, are more likely to realize that they mean the same thing.


But all of these always make me wonder about a return inside the
suite.
Does the "else" part execute (answer: no). So as a rule, I avoid the
construct entirely,


really? does an if-else clause execute if there's a return in the if? 
no. why would this be any more confusing?


not that it comes up very often, anyway.


This is the really key point. I wanted to use a real code example 
above, but I could not remember when I'd ever used it -- and it's hard 
to search for, so I couldn't find one.


So: churn to make something rarely needed a little bit better -- I 
don't see the point.
Naturally there is *less* benefit from improving something that is used 
rarely than from improving something that is used often.

That doesn't mean *no* benefit, or that we shouldn't bother.


-CHB

--
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SJJZZ5JVU77ACZSEDLSN7OORNPGIPLOT/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@py

[Python-ideas] Re: Thoughts about implementing object-compare in unittest package?

[Steven D'Aprano]

On Sat, Jul 25, 2020 at 10:15:16PM -0500, Henry Lin wrote:
> Hey all,
> 
> What are thoughts about implementing an object-compare function in the
> unittest package? (Compare two objects recursively, attribute by
> attribute.)

Why not just ask the objects to compare themselves?

assertEqual(actual, expected)

will work if actual and expected define a sensible `__eq__` and are the 
same type. If they aren't the same type, why not?

actual = MyObject(spam=1, eggs=2, cheese=3)
expected = DifferentObject(spam=1, eggs=2, cheese=3)


> This seems like a common use case in many testing scenarios,

I've never come across it. Can you give an example where defining an 
`__eq__` method won't be the right solution?




-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TDLFBURVX4N4JJP4ELIRLKULR775VNOY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] How to prevent shared memory from being corrupted ?

[Vinay Sharma via Python-ideas]

Problem:
Currently, let’s say I create a shared_memory segment using 
mulitprocessing.shared_memory.SharedMemory 
 in 
Process 1 and open the same in Process 2.
Then, I try to write some data to the shared memory segment using both the 
processes, so for me to prevent any race condition (data corruption), either 
these operations must be atomic, or I should be able to lock / unlock shared 
memory segment, which I cannot at the moment.

I earlier posted a solution 

 to this problem, which received positive response, but there weren’t many 
responses to it, despite the fact this problem makes shared_memory practically 
unusable if there are simultaneous writes.
So, the purpose of this post is to have discussion about the solution of the 
same.

Some Solutions:

1. Support for shared semaphores across unrelated processes to lock/unlock 
shared memory segment.
--> More Details 


2. Let the first bit in the shared memory segment be the synchronisation bit 
used for locking/unlocking.
--> A process can only lock the shared memory segment if this bit is unset, 
and will set this bit after acquiring lock, and similarly unset this bit
after unlocking. Although set/unset operations must be atomic. Therefore, the 
following tools can be used:
type __sync_add_and_fetch (type *ptr, type value, ...)
type __sync_sub_and_fetch (type *ptr, type value, ...)
type __sync_or_and_fetch (type *ptr, type value, ...)

Documentation of these can be found at below links:
https://gcc.gnu.org/onlinedocs/gcc/_005f_005fsync-Builtins.html#g_t_005f_005fsync-Builtins
 

https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html#g_t_005f_005fatomic-Builtins
 


Any other ideas/solutions are very welcome.___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7C3TART5SPHLJZZHZIY43KUEIB2IUX75/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Faster object representation for UIs

[Serhiy Storchaka]

26.07.20 01:41, Greg Ewing пише:

On 26/07/20 1:34 am, Elizabeth Shashkova wrote:
1. We need this lazy `__repr__` calculation inside our debugger, where 
we work with different user's objects. Usually it isn't some specific 
type, for which you know that it'll be big and its `__repr__` 
calculation will be slow


Seems to me it would be better for the debugger to take a
conservative approach here, and assume that the repr *will*
be big and/or slow unless it's a type that it *does* know
about. All other objects should be displayed using something
similar to object.__repr__ unless the user requests otherwise.


Concur.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GKLPYVVMJ3YP5EK7J7NMFLTZBUVJEEF3/
Code of Conduct: http://python.org/psf/codeofconduct/