New submission from Jeremy <[email protected]>:
While writing a program using the multiprocessing library I stumbled upon what
appears to be a bug with how different platforms deal with private methods.
When a class has a private method which is the target for a multiprocessing
process, this name is correctly resolved on Linux (20.04.1-Ubuntu running
Python 3.8.10) but fails to be resolved correctly on MacOS (Python 3.8.2 and
3.8.8) or Windows 10 (Python 3.9.6).
import multiprocessing
class Test(object):
def __init__(self):
self.a = 1
self._b = 2
self.__c = 3
self.run1()
self.run2()
def _test1(self, conn):
conn.send(self._b)
def __test2(self, conn):
conn.send(self.__c)
def run1(self):
print("Running self._test1()")
parent, child = multiprocessing.Pipe()
process = multiprocessing.Process(target=self._test1, args=(child, ))
process.start()
print(parent.recv())
process.join()
def run2(self):
print("Running self.__test2()")
parent, child = multiprocessing.Pipe()
process = multiprocessing.Process(target=self.__test2, args=(child, ))
process.start()
print(parent.recv())
process.join()
if __name__ == "__main__":
t = Test()
On Linux, this has the intended behavior of printing:
Running self._test1()
2
Running self.__test2()
3
However, on Windows 10, this results in an Exception being raised:
Running self._test1()
2
Running self.__test2()
Traceback (most recent call last):
File "<string>", line 1, in <module>
File
"C:\Users\<USER>\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py",
line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File
"C:\Users\<USER>\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py",
line 126, in _main
self = reduction.pickle.load(from_parent)
AttributeError: 'Test' object has no attribute '__test2'
A similar exception is also raised on MacOS for this code.
It would therefore appear that there is different behavior for resolving class
attributes starting with `__` on different platforms (at least within
multiprocessing). It is my understanding that because multiprocessing.Process
is called within the class, the private method should be within scope and so
should resolve correctly.
I'm aware that Python doesn't have strict private methods, and instead renames
them (Test.__test2 becomes Test._Test__test2) - explaining why on Windows it
cannot find the attribute with that name.
My question really is, which platform is correct here, and is the inconsistency
intentional? I'd suggest Linux is most correct here as the process is spawned
from within the object so the method should be in scope, but either way, the
inconsistency between platforms may cause some unintended issues.
----------
components: Library (Lib), Windows, macOS
messages: 397810
nosy: ned.deily, paul.moore, ronaldoussoren, steve.dower, tim.golden, ymerej,
zach.ware
priority: normal
severity: normal
status: open
title: Cross-platform issues with private methods and multiprocessing
type: behavior
versions: Python 3.8, Python 3.9
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue44675>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
