[issue4701] range objects becomes hashable after attribute access

[Jesús Cea Avión]

Changes by Jesús Cea Avión :


--
nosy: +jcea

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Nick Coghlan]

Nick Coghlan  added the comment:

Forward port to 3.x:

3.1: r68058
3.0: r68060

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Nick Coghlan]

Nick Coghlan  added the comment:

Fixed using a lazy call to PyType_Ready in PyObject_Hash:

2.7: r68051
2.6: r68052

Forward-port to Py3k still to come.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Eric Smith]

Eric Smith  added the comment:

> Perhaps the path of least resistance is to change PyObject_Hash to be
> yet another place where PyType_Ready will be called implicitly if it
> hasn't been called already?

I think that's the best thing to do. It would bring PyObject_Hash in
line with PyObject_Format, for example.

If we pick some other solution (instead of modifying PyObject_Hash),
then we should find all of the lazy calls to PyType_Ready (that exist to
solve this problem) and remove them. IOW, it should be handled the same
everywhere.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Nick Coghlan]

Nick Coghlan  added the comment:

Copied from python-dev post:

Perhaps the path of least resistance is to change PyObject_Hash to be
yet another place where PyType_Ready will be called implicitly if it
hasn't been called already?

That approach would get us back to the Python 2.x status quo where
calling PyType_Ready was only absolutely essential if you wanted to
correctly inherit a slot from a type other than object itself.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Nick Coghlan]

Nick Coghlan  added the comment:

enumerate can be added to the list of builtin types which isn't
initialised correctly, as can the callable+sentinel iterator return from
the 2-argument version of iter() and the default sequence iterator
returned by iter() when given a type with both __len__ and __getitem__
methods, but no __iter__ method.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Nick Coghlan]

Nick Coghlan  added the comment:

Hagen Fürstenau wrote:
> Hagen Fürstenau  added the comment:
> 
>> I don't believe there is any driving reason for them not to be hashable.
> 
> On the other hand, what is the use case for hashing objects whose
> equality is defined as object identity?

Fast membership testing in sets to track which objects you have and
haven't seen, mapping from objects to arbitrary metadata about those
objects without having to explicitly redirect through id(), that kind of
thing. Generally speaking, the idea is that objects should be hashable
if their concept of "equality" cannot change over the life time of the
object. Identity based equality meets that criteria, which is why
objects (including range/xrange) are hashable by default.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Hagen Fürstenau]

Hagen Fürstenau  added the comment:

> I don't believe there is any driving reason for them not to be hashable.

On the other hand, what is the use case for hashing objects whose
equality is defined as object identity?

Python 3.0 (r30:67503, Dec  4 2008, 06:47:38)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> range(10).__hash__

>>> {range(10), range(10)}
{range(0, 10), range(0, 10)}

I can see only confusion arising from that.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Nick Coghlan]

Nick Coghlan  added the comment:

Ah, I think I figured it out - in 2.x, PyObject_Hash itself includes the
fallback to _PyHash_Pointer if none of tp_hash, tp_compare or the
tp_richcompare slots have been implemented on the type.

So long as a type is only trying to inherit object.__hash__ (as is the
case with xrange), then this fallback will do the right thing if
PyType_Ready hasn't been called yet.

In 3.0, on the other hand, PyObject_Hash has no fallback - if tp_hash
isn't filled in, the type isn't considered hashable. This means that for
a type to properly inherit hashability in Py3k, PyType_Ready *must* be
called on it.

Probably the best thing to do is to add xrange and range to the list of
types initialised in _Py_ReadyTypes in 2.x and 3.x respectively.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Nick Coghlan]

Nick Coghlan  added the comment:

Bumping to release blocker for 3.0.1 - until I understand this better,
I'm not sure if the xrange->range hashing behaviour change between 2.x
and 3.x is a type specific problem or a general issue in the
implementation of the new approach to tp_hash inheritance for Py3k.

--
priority:  -> release blocker
resolution: accepted -> 
versions:  -Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Nick Coghlan]

Nick Coghlan  added the comment:

It has been pointed out to me that xrange() objects are hashable in 2.x,
and since these range objects are immutable, I don't believe there is
any driving reason for them not to be hashable.

At that point the question becomes one of why xrange() is being
initialised correctly in 2.x while something is going wrong with the
tp_hash slot initialisation for range() in 3.x that doesn't get fixed
until PyType_Ready is called to populate tp_dict.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Raymond Hettinger]

Changes by Raymond Hettinger :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Raymond Hettinger]

Raymond Hettinger  added the comment:

FWIW, xrange() is hashable in Py2.6.  I believe it was intended that
that carry over to Py3.0's range() objects.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc  added the comment:

Patch looks good to me.

--
nosy: +amaury.forgeotdarc
resolution:  -> accepted

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Nick Coghlan]

Nick Coghlan  added the comment:

OK, the discrepancy with bytearray turns out to be fairly
straightforward: bytearray overrides the comparison operations, so
inheritance of the default object.__hash__ is automatically blocked.
range() objects don't support comparison, so they inherit __hash__ when
PyType_Ready is called.

Which then begs the question of why range() instances are unhashable
only until something happens to invoke the tp_getattro slot on the type
object... and it turns out that PyType_Ready isn't called on the type
during interpreter startup. Instead, it only happens lazily when one of
the operations that needs the tp_dict to be filled in calls PyType_Ready
(the default dir() retrieves __dict__ from the type object, and this
attribute access causes PyType_Ready to be called).

Only at this point is the slot inheritance on the range() type
calculated correctly.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Nick Coghlan]

Nick Coghlan  added the comment:

The origin of the unnecessary hashfunc casts is just me following some
of the more specific examples of filling in the tp_hash slot too closely
without checking if the cast was still needed.

I'll apply and backport Hagen's patches to 3.0 soon (as well as fixing
some other non-hashable types such as slice() to use
PyHash_NotImplemented), but first I want to understand why range()
exhibits this behaviour, while other classes with a superficially
similar implementation (such as bytearray) do not.

--
assignee:  -> ncoghlan
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Hagen Fürstenau]

Changes by Hagen Fürstenau :


Removed file: http://bugs.python.org/file12401/rangehash2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Hagen Fürstenau]

Changes by Hagen Fürstenau :


Added file: http://bugs.python.org/file12403/rangehash3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Eric Smith]

Changes by Eric Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Hagen Fürstenau]

Changes by Hagen Fürstenau :


Removed file: http://bugs.python.org/file12400/rangehash.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Hagen Fürstenau]

Changes by Hagen Fürstenau :


Added file: http://bugs.python.org/file12402/remove_casts.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Hagen Fürstenau]

Hagen Fürstenau  added the comment:

Here's an updated patch without the cast and a separate patch for
removing the other casts.

Added file: http://bugs.python.org/file12401/rangehash2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Guilherme Polo]

Guilherme Polo  added the comment:

On Fri, Dec 19, 2008 at 2:14 PM, Hagen Fürstenau  wrote:
>
> Hagen Fürstenau  added the comment:
>
> I'm talking about places like these:
>
> [hag...@chage py3k]$ grep -R "(hashfunc)PyObject_HashNotImplemented"
> Objects/*.c Modules/*.c
> Objects/dictobject.c:   (hashfunc)PyObject_HashNotImplemented,  /*
> tp_hash */
> Objects/listobject.c:   (hashfunc)PyObject_HashNotImplemented,  /*
> tp_hash */
> Objects/setobject.c:(hashfunc)PyObject_HashNotImplemented,  /*
> tp_hash */
> Modules/_collectionsmodule.c:   (hashfunc)PyObject_HashNotImplemented,
> /* tp_hash */

I have checked some of the examples you gave and noticed they were
re-added (or changed) recently, but all these casts could be removed.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Hagen Fürstenau]

Hagen Fürstenau  added the comment:

I'm talking about places like these:

[hag...@chage py3k]$ grep -R "(hashfunc)PyObject_HashNotImplemented"
Objects/*.c Modules/*.c
Objects/dictobject.c:   (hashfunc)PyObject_HashNotImplemented,  /*
tp_hash */
Objects/listobject.c:   (hashfunc)PyObject_HashNotImplemented,  /*
tp_hash */
Objects/setobject.c:(hashfunc)PyObject_HashNotImplemented,  /*
tp_hash */
Modules/_collectionsmodule.c:   (hashfunc)PyObject_HashNotImplemented, 
/* tp_hash */

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Guilherme Polo]

Guilherme Polo  added the comment:

On Fri, Dec 19, 2008 at 2:02 PM, Hagen Fürstenau  wrote:
>
> Hagen Fürstenau  added the comment:
>
> Why does every other place seem to do the cast? Historical reasons?
>

No, if you look at the functions being casted you will notice them do
not take a pointer to PyObject as the first argument, if you are
talking about tp_hash specifically.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Hagen Fürstenau]

Hagen Fürstenau  added the comment:

Why does every other place seem to do the cast? Historical reasons?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Guilherme Polo]

Guilherme Polo  added the comment:

You don't need to cast PyObject_HashNotImplemented to hashfunc

--
nosy: +gpolo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Hagen Fürstenau]

Changes by Hagen Fürstenau :


--
keywords: +patch
Added file: http://bugs.python.org/file12400/rangehash.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4701] range objects becomes hashable after attribute access

[Hagen Fürstenau]

New submission from Hagen Fürstenau :

As reported by Dmitry Vasiliev on python-dev, a range object suddenly
becomes hash()able after an attribute access, e.g. by dir().

If I understand correctly, then the reason is that PyRange_Type doesn't
set tp_hash and PyType_Ready is not normally called on the type, but
only e.g. in PyObject_GenericGetAttr.

I don't see any use for range objects being hashable, as they don't even
have meaningful equality defined on them. So I'd recommend making them
unhashable. The attached patch does this and adds a test.

--
components: Interpreter Core
messages: 78059
nosy: hagen
severity: normal
status: open
title: range objects becomes hashable after attribute access
type: behavior
versions: Python 3.0, Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com