[issue17816] Weak*Dictionary KeyErrors during callbacks

[Irit Katriel]

Irit Katriel  added the comment:

This was fixed in Python 3 and Python 2 is past its EOL.

--
nosy: +iritkatriel
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17816] Weak*Dictionary KeyErrors during callbacks

[Nils Bruin]

Nils Bruin added the comment:

One solution is to patch both WeakValueDictionary and WeakKeyDictionary with 
their own clear methods where we first store the strong links (to keys, resp. 
values) in a list, then clear the underlying dictionaries (this will now 
trigger the deletion of the weakrefs, so all callbacks are neutralized), and 
then delete the list. It does use more storage that way, but it gets rid of the 
ignored key errors.

This is a different problem from issue7105, which deals with the (much more 
complicated) scenario of avoiding dictionary reshaping due to GC when iterators 
are still (potentially) active.

--
keywords: +patch
Added file: http://bugs.python.org/file30101/17816_custom_clear.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17816] Weak*Dictionary KeyErrors during callbacks

[Nils Bruin]

New submission from Nils Bruin:

The following program is a little dependent on memory layout but will usually 
generate lots of

Exception KeyError: (A(9996),) in function remove at 0xa47050 ignored

messages in Python 2.7.

import weakref

class A(object):
def __init__(self,n):
self.n=n
def __repr__(self):
return A(%d)%self.n

def mess(n):
D=weakref.WeakValueDictionary()
L=[A(i) for i in range(n)]
for i in range(n-1):
j=(i+10)%n
D[L[i]]=L[j]
return D

D=mess(1)
D.clear()

The reason is that on D.clear() all entries are removed from D before actually 
deleting those entries. Once the entries are deleted one-by-one, sometimes the 
removal of a key will result in deallocation of that key, which may be a 
not-yet-deleted ex-value of the dictionary as well. The callback triggers on 
the weakref, but the dict itself was already emptied, so nothing is found.

I've checked and on Python 3.2.3 this problem does not seem to occur. I haven't 
checked the Python source to see how Python 3 behaves differently and whether 
that behaviour would be easy to backport to fix this bug in 2.7.

--
components: Library (Lib)
messages: 187570
nosy: Nils.Bruin
priority: normal
severity: normal
status: open
title: Weak*Dictionary KeyErrors during callbacks
versions: Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17816] Weak*Dictionary KeyErrors during callbacks

[Volker Braun]

Volker Braun added the comment:

This is http://bugs.python.org/issue7105. The patch from there could easily be 
backported, I think.

--
nosy: +vbraun

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17816] Weak*Dictionary KeyErrors during callbacks

[Nils Bruin]

Nils Bruin added the comment:

Have you tried if the fix at issue7105 solves the problem? I don't see the 
patch there introduce a `clear` method override for WeakValueDictionary or 
WeakKeyDictionary. The one for WeakSet still calls self.data.clear(), which for 
dictionaries would still result in the problem in this ticket (but not for 
WeakSet, because clearing a WeakSet shouldn't decref anything other than the 
weak references stored in the underlying set).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17816] Weak*Dictionary KeyErrors during callbacks

[Nils Bruin]

Nils Bruin added the comment:

I think the difference in behaviour between Py3 and Py2 is coming from:

http://hg.python.org/cpython/file/a26df2d03989/Objects/dictobject.c#l1275

which first clears all values before removing any keys. For a 
WeakValueDictionary that means all the weakrefs are neutralized before the can 
be activated. I don't quite understand how Py3 manages to avoid problems for a 
WeakKeyDictionary, but apparently it does.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17816
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com