Josh Rosenberg <shadowranger+pyt...@gmail.com> added the comment:
As Karthikeyan, this is an inevitable, and documented, consequence of the
proxies not being aware of in-place modification of their contents.
As your own example demonstrates, any approach that provides that information
to the shared dict proxy will work; |= and += are almost the same as .update
and .extend, but implemented such that the left hand side is always reassigned,
even when the result of __ior__/__iadd__ is the same object it was called on.
Thus, |=/+= work, while add/append/update/extend do not.
Note that as of 3.6, there is another option: Nested shared objects:
> Changed in version 3.6: Shared objects are capable of being nested. For
> example, a shared container object such as a shared list can contain other
> shared objects which will all be managed and synchronized by the SyncManager.
So the alternative solution in your case (assuming you're on 3.6 or later,
which your bug version tags say you are) is to make the sub-lists
manager.lists, or replace use of a set with manager.dict (as dicts with all
values set to True, are largely compatible with set anyway, especially with the
advent of dict views):
manager = Manager()
shared_dict = manager.dict()
shared_dict['test'] = manager.dict() # or shared_dict['test'] = manager.list()
shared_dict['test'][1234] = True # or shared_dict['test'].append(1234)
Downside: The repr of shared dicts/lists doesn't display the contents, so your
example code won't make it obvious that the problem is fixed, but it does in
fact work. I wrote a terrible JSON one-liner to check the contents, and it
demonstrates that the shared dict/list work just fine:
import json
from multiprocessing import Manager
from multiprocessing.managers import DictProxy, ListProxy
manager = Manager()
shared_dict = manager.dict()
shared_dict['testset'] = set()
shared_dict['testlist'] = []
shared_dict['testshareddict'] = manager.dict()
shared_dict['testsharedlist'] = manager.list()
shared_dict['testset'].add(1234)
shared_dict['testlist'].append(1234)
shared_dict['testshareddict'][1234] = True
shared_dict['testsharedlist'].append(1234)
print(json.dumps(shared_dict, default=lambda x: dict(x) if isinstance(x,
DictProxy) else
list(x) if isinstance(x,
ListProxy) else
dict.fromkeys(x, True) if
isinstance(x, (set, frozenset)) else
x))
The dump shows that the changes to the shared inner dict and list are reflected
in the result directly, even with no assignment back to the keys of the outer
dict (while, as you note, the plain set and list show no changes).
Closing as not a bug, since this is fully documented, with multiple workarounds
available.
----------
nosy: +josh.r
resolution: -> not a bug
stage: -> resolved
status: open -> closed
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue36119>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
