Hi Guido,
I'm sorry for being unclear! I just try actually to learn what thoses
consequences for theses 'unattended' mutations in dictionary key lookups
could be, :-)
however, it seems now that I might have touch a nerve without realizing
it. I would therefore appreciate more light on this "issue" if you like
to enlighten us all. :D
Regards,
Etienne
On 03/30/2012 10:47 AM, Guido van Rossum wrote:
Etienne, I have not understood either of your messages in this thread.
They just did not make sense to me. Do you actually understand the issue
at hand?
--Guido
On Friday, March 30, 2012, Etienne Robillard wrote:
On 03/29/2012 06:07 PM, R. David Murray wrote:
On Thu, 29 Mar 2012 23:00:20 +0200, Stefan
Behnel<stefan...@behnel.de> wrote:
R. David Murray, 29.03.2012 22:31:
On Thu, 29 Mar 2012 13:09:17 -0700, Guido van Rossum wrote:
On Thu, Mar 29, 2012 at 12:58 PM, R. David Murray wrote:
Some of us have expressed uneasiness about the
consequences of dict
raising an error on lookup if the dict has been
modified, the fix Victor
made to solve one of the crashers.
I don't know if I speak for the others, but
(assuming that I understand
the change correctly) my concern is that there
is probably a significant
amount of threading code out there that assumes
that dict *lookup* is
a thread-safe operation. Much of that code
will, if moved to Python
3.3, now be subject to random runtime errors for
which it will not
be prepared. Further, code which appears safe
can suddenly become
unsafe if a refactoring of the code causes an
object to be stored in
the dictionary that has a Python equality method.
My original assessment was that this only affects
dicts whose keys
have a user-implemented __hash__ or __eq__
implementation, and that
the number of apps that use this *and* assume the
threadsafe property
would be pretty small. This is just intuition, I
don't have hard
facts. But I do want to stress that not all dict
lookups automatically
become thread-unsafe, only those that need to run
user code as part of
the key lookup.
You are probably correct, but the thing is that one
still has to do the
code audit to be sure...and then make sure that no one
later introduces
such an object type as a dict key.
The thing is: the assumption that arbitrary dict lookups are
GIL-atomic has
*always* been false. Only those that do not involve Python
code execution
for the hash key calculation or the object comparison are.
That includes
the built-in strings and numbers (and tuples of them), which
are by far the
most common dict keys. Looking up arbitrary user provided
objects is
definitely not guaranteed to be atomic.
Well, I'm afraid I was using the term 'thread safety' rather too
loosely
there. What I mean is that if you do a dict lookup, the lookup
either
returns a value or a KeyError, and that if you get back an
object that
object has internally consistent state. The problem this fix
introduces
is that the lookup may fail with a RuntimeError rather than a
KeyError,
which it has never done before.
I think that is what Guido means by code that uses objects with
python
eq/hash *and* assumes threadsafe lookup. If mutation of the objects
or dict during the lookup is a concern, then the code would use
locks
and wouldn't have the problem. But there are certainly situations
where it doesn't matter if the dictionary mutates during the lookup,
as long as you get either an object or a KeyError, and thus no
locks are
(currently) needed.
Maybe I'm being paranoid about breakage here, but as with most
backward
compatibility concerns, there are probably more bits of code
that will
be affected than our intuition indicates.
--David
______________________________ _________________
what this suppose to mean exactly? To "mutate" is a bit odd concept
for a programming language I suppose. Also I suppose I must be
missing something which makes you feel like this is an OT post when
the problem seem most likely to be exclusively in python 3.3,
another reason I guess to not upgrade yet all that massively using
2to3. :-)
cheers,
Etienne
______________________________ _________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/ mailman/listinfo/python-dev
<http://mail.python.org/mailman/listinfo/python-dev>
Unsubscribe: http://mail.python.org/ mailman/options/python-dev/
guido%40python.org
<http://mail.python.org/mailman/options/python-dev/guido%40python.org>
--
--Guido van Rossum (python.org/~guido <http://python.org/~guido>)
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
