TristonC edited a comment on issue #20959:
URL:
https://github.com/apache/incubator-mxnet/issues/20959#issuecomment-1074580675
I wonder you might be interested in digger little deeper @ann-qin-lu. The it
seems current gluon dataloader using fork to start a worker process in
multiprocessing.Pool(..) function call (as it is default in Unix-like system).
It might be a problem for this issue as the child process inherit everything
from its parent process. It might be a good idea to use spawn instead of using
fork this function. Unfortunately, I ran into a issue that blocks my test of
multiprocessing.get_context('spawn').Pool(...) .
```bash
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "/usr/lib/python3.8/multiprocessing/spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
File "/opt/mxnet/python/mxnet/gluon/data/dataloader.py", line 58, in
rebuild_ndarray
return nd.NDArray(nd.ndarray._new_from_shared_mem(pid, fd, shape, dtype))
File "/opt/mxnet/python/mxnet/ndarray/ndarray.py", line 193, in
_new_from_shared_mem
check_call(_LIB.MXNDArrayCreateFromSharedMemEx(
File "/opt/mxnet/python/mxnet/base.py", line 246, in check_call
raise get_last_ffi_error()
mxnet.base.MXNetError: Traceback (most recent call last):
File "../src/storage/./cpu_shared_storage_manager.h", line 179
MXNetError: Check failed: ptr != ((void *) -1) (0xffffffffffffffff vs.
0xffffffffffffffff) : Failed to map shared memory. mmap failed with error
Permission denied
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
