[issue35786] get_lock() method is not present for Values created using multiprocessing.Manager()

[Dávid Nemeskey]

Dávid Nemeskey  added the comment:

Nothing in the documentation says that multiprocessing.Value and the object 
returned by manager.Value() is any different. Nor is it clear why they should 
be.

It is perfectly understandable to expect that manager.Value() is actually of 
type multiprocessing.Value. It is also a valid assumption that the "canonical" 
way to acquire such objects are from a Manager, not directly. An example that 
reinforces this assumption is that of the Queue class, which HAS to be created 
through a Manager, lest we get "RuntimeError: Queue objects should only be 
shared between processes through inheritance".

In conclusion, I think this is definitely a valid issue. What I am not so sure 
about is if it is (just) an issue in the documentation or the API itself.

--
nosy: +nemeskeyd
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35786] get_lock() method is not present for Values created using multiprocessing.Manager()

[Dávid Nemeskey]

Dávid Nemeskey  added the comment:

OK, actually: trying to create a multiprocessing.Value object and sharing it 
between a Pool of processes results in "RuntimeError: Synchronized objects 
should only be shared between processes through inheritance". So the only way 
seems to be through a Manager, but its Value() is of a different class?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35786] get_lock() method is not present for Values created using multiprocessing.Manager()

[Josh Rosenberg]

Josh Rosenberg  added the comment:

Reading the docs, I'd definitely expect multiprocessing.Manager().Value to obey 
the same interface as multiprocessing.Value. The SyncManager docs say:

> Its methods create and return Proxy Objects for a number of commonly used 
> data types to be synchronized across processes.

It does also say (under the linked definition for Proxy Objects):

> A proxy object has methods which invoke corresponding methods of its referent 
> (although not every method of the referent will necessarily be available 
> through the proxy).

which implies the proxy might be more limited, but in fact the proxy is 
wrapping a completely different underlying class, 
multiprocessing.managers.Value (that has nothing to do with the class you'd get 
from calling multiprocessing.Value or multiprocessing.sharedctypes.Value).

It looks like there was, at some point, an attempt to make certain interfaces 
match; multiprocessing.managers.Value's initializer has a prototype of:

def __init__(self, typecode, value, lock=True):

(documented prototype excludes the lock argument), but AFAICT, lock is 
completely ignored, and typecode is stored as _typecode on the underlying 
Value, but otherwise ignored; the multiprocessing.managers.ValueProxy object 
returned by manager.Value() doesn't expose it (except insofar as you can call 
._getvalue() on it to retrieve a copy of the underlying Value), and even if you 
get the unwrapped object, all _typecode does is change the repr; it's not used 
anywhere else.

It seems like SyncManager.Value is just the worst of all possible worlds; it 
has a prototype that (roughly) matches 
multiprocessing.Value/multiprocessing.sharedctypes.Value, and the limited 
documentation implies it serves the same function, but:

1. It effectively ignores every argument aside from value (which is the second 
argument, so you're stuck passing the first argument even though nothing uses 
it)
2. It doesn't actually use ctypes to achieve efficient storage/IPC 
communication (updating value involves pickling it, not just sending the raw 
data of a C array/struct)
3. It doesn't provide any of the interface of the other Value, so you can't use 
get_lock (or use a with statement to lock it, or manually call acquire or 
release) or get_obj.

I'm not sure what to do about it though; the manager version of Value is much 
more flexible, and I'm sure there is existing code that takes advantage of 
that, so we can't rewrite to make it ctypes backed. The 
multiprocessing.managers code is too new/complex for me to determine if any 
easy solution exists to expand multiprocessing.managers.ValueProxy to support 
context management/get_lock/get_obj to match the behavior of the class returned 
by multiprocessing.Value/multiprocessing.sharedctypes.Value, but it seems like, 
if we're going to let the docs imply a relationship, we ought to at least try 
to make the API of the two classes roughly match.

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35786] get_lock() method is not present for Values created using multiprocessing.Manager()

[Lorenzo Persichetti]

New submission from Lorenzo Persichetti :

According to the documentation of the multiprocessing.Value() class available 
here 
https://docs.python.org/3.6/library/multiprocessing.html#multiprocessing.Value

Operations like += which involve a read and write are not atomic. So if, for 
instance, you want to atomically increment a shared value it is insufficient to 
just do

counter.value += 1
Assuming the associated lock is recursive (which it is by default) you can 
instead do

with counter.get_lock():
counter.value += 1

What happens is that when running the following snippet

import multiprocessing
manager = multiprocessing.Manager()
value = manager.Value('i', 0)
value.get_lock()

the result is 
AttributeError: 'ValueProxy' object has no attribute 'get_lock'

--
assignee: docs@python
components: Documentation, Library (Lib), Windows
messages: 334070
nosy: Lorenzo Persichetti, docs@python, paul.moore, steve.dower, tim.golden, 
zach.ware
priority: normal
severity: normal
status: open
title: get_lock() method is not present for Values created using 
multiprocessing.Manager()
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35786] get_lock() method is not present for Values created using multiprocessing.Manager()

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

You are not using multiprocessing.Value:

>>> import multiprocessing
>>> x = multiprocessing.Value("i", 0)
>>> x.get_lock()


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35786] get_lock() method is not present for Values created using multiprocessing.Manager()

[Ian Jacob Bertolacci]

Ian Jacob Bertolacci  added the comment:

What's being done about this?
I would say this is less "misleading documentation" and more "incorrect 
implementation"

There is also not an obvious temporary work-around.

--
nosy: +IanBertolacci

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35786] get_lock() method is not present for Values created using multiprocessing.Manager()

[Steve Dower]

Change by Steve Dower :


--
components:  -Windows
nosy:  -steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35786] get_lock() method is not present for Values created using multiprocessing.Manager()

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
nosy:  -pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35786] get_lock() method is not present for Values created using multiprocessing.Manager()

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
nosy: +Jeffrey.Kintscher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35786] get_lock() method is not present for Values created using multiprocessing.Manager()

[Ned Deily]

Change by Ned Deily :


--
nosy: +davin, pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com