[issue38769] generators are currently hashable

[Steven D'Aprano]

Steven D'Aprano  added the comment:

I can see that making generators unhashable would have found your bug earlier, 
but it is otherwise unjustified: generators are just objects that are compared 
by identity. It would break other people's code, which is a very invasive 
change to make just so that a small subset of "forgot to call tuple" bugs might 
sometimes be found earlier.

Personally, I think that Serhiy's comment that there is "a chance" that this 
will be accepted is over-optimistic, but either way this would need to have a 
PEP.

I'm going to close this as "Not a bug" but that doesn't mean discussion has to 
stop. If you feel strongly enough about this issue to write a PEP proposing 
this change, please take the discussion to the Python-Dev mailing list with a 
summary and see if you can get somebody to sponsor the PEP. If you do, feel 
free to re-open as an Enhancement.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38769] generators are currently hashable

[Tim Peters]

Tim Peters  added the comment:

Yes, what Steven said.  All kinds of functions (including, but not limited to, 
generator-iterators) are compared by object identity, nothing at all about 
internal state.  The contract of hash() is that if a == b, then we must have 
that hash(a) == hash(b) too.  That's got nothing to do with internal state 
either, _only_ with how __eq__ is implemented.

There is no sense in which any kind of function object is "mutable" that has 
any effect on its object identity, so also no sense in which it can affect 
__eq__ results, so no reason at all for __hash__ to care.  It's all working as 
designed and as intended.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38769] generators are currently hashable

[Ammar Askar]

Ammar Askar  added the comment:

An easier way to understand might be to just write out the generator expression 
"manually":

  def generator_expression():
for k, v in sorted(self.__dict__.items()):
  yield k
  return hash(my_generator_expression())

Would you say that `generator_expression` here is mutable or immutable? The 
code that underpins how it generates values is clearly immutable just like 
functions but the values it closes over can be changed.

Another minimal example:

>>> def f():
...   yield from some_list
...
>>> some_list = [1,2,3]
>>> list(f())
[1, 2, 3]
>>> some_list = [1,2,3,4,5]
>>> list(f())
[1, 2, 3, 4, 5]

Would you say that `f` is immutable or mutable in this case?

--
nosy: +ammar2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38769] generators are currently hashable

[Steven D'Aprano]

Steven D'Aprano  added the comment:

Anthony, can you please explain what you mean when you describe generators as 
"mutable"? I don't understand what you mean.

To *me*, the value of a generator, in so far as comparisons goes, is its 
identity, not its invisible internal state. You can test that by noting that 
two generators with the same state compare unequal:

py> def gen():
... yield 1
...
py> a = gen()
py> b = gen()
py> a == b
False

Furthermore, the hash of the generator doesn't change when its internal state 
changes.

py> hash(a)
192456114
py> next(a)
1
py> hash(a)
192456114


And as for functions being "immutable", you can attach arbitrary attributes to 
them, so their object state can change. Does that make them mutable? Either 
way, it doesn't matter, since functions are also compared by identity. Their 
value is their identity, not their state. (Two functions with the same internal 
state are compared as unequal.)

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38769] generators are currently hashable

[Anthony Sottile]

Anthony Sottile  added the comment:

I think I'm missing what you're saying, apologies I'm probably confused :(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38769] generators are currently hashable

[Tim Peters]

Tim Peters  added the comment:

What makes you think generator-iterator objects are mutable?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38769] generators are currently hashable

[Anthony Sottile]

Anthony Sottile  added the comment:

function objects are immutable though

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38769] generators are currently hashable

[Tim Peters]

Tim Peters  added the comment:

Function objects can be used as dict keys too.  Given that, it would be _very_ 
surprising if generator-iterators weren't.  I don't really see a good point to 
leaving this open.  Live with it - it's a feature ;-)

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38769] generators are currently hashable

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I agree with Raymond. By default all objects are compared by identity and are 
hashable. User classes are hashable, files are hashable, iterators are 
hashable. Generator objects are just not special enough to be non-hashable.

We can reconsider this. But it a part of much larger Python language change. It 
should be discussed on Python-Dev. We should estimate the benefit and the 
amount of possible breakage. I think there is a chance of changing this, but we 
need more information.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38769] generators are currently hashable

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> there are other such mutable objects in python which do define
> __hash__ = None such as list and dict

Those objects compare on their contents, not on object identity.

Take a look at how hashing works in dataclasses.  You can have a mix of mutable 
and immutable fields as long as both __eq__ and __hash__ depend on only the 
immutable fields.

Also, take a look at database cursor objects.  Both __eq__ and __hash__ use 
object identity even though fetchone() advances the internal state (just like 
an iterator).

Anyway, the breaking of existing working code makes this a non-starter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38769] generators are currently hashable

[Anthony Sottile]

Anthony Sottile  added the comment:

:shrugs: it did go through a round of code review but was simply missed -- the 
testing bit is fair

there are other such mutable objects in python which do define __hash__ = None 
such as list and dict

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38769] generators are currently hashable

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> the author had forgotten the `tuple(...)` call (or incorrectly 
> assumed a parenthesized generator was a tuple comprehension)

Presumably that will bite the author in many ways, not just hashability.  There 
are many other places where substituting a generator for a tuple would result 
in a downstream error.  ISTM, this user error would have been caught with even 
minimal testing or code review.


> it seems wrong that generators are currently hashable as they are mutable

It is perfectly reasonable given that generators compare based on object 
identity.


> Thoughts on `__hash__ = None` for generators?

That would break currently working code that depends on hashability.  I have 
seen such code in production more than once (using distinct generator instances 
as dictionary keys for example).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38769] generators are currently hashable

[Ammar Askar]

Change by Ammar Askar :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38769] generators are currently hashable

[Anthony Sottile]

New submission from Anthony Sottile :

We recently found a bug in one of our codebases that looked ~roughly like this:

class C:
...

def __hash__(self):
return hash((v for k, v in sorted(self.__dict__.items(

which resulted in a production bug

The *intention* was to hash a tuple of those elements but the author had 
forgotten the `tuple(...)` call (or incorrectly assumed a parenthesized 
generator was a tuple comprehension) -- either way it seems wrong that 
generators are currently hashable as they are mutable

Thoughts on `__hash__ = None` for generators?

--
components: Interpreter Core
messages: 356382
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: generators are currently hashable
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com