New submission from Justin Ting:
Multiprocessing is throwing this error when dealing with large amounts of data
(all floating points an integers), but none of which exceeds the number
boundaries in the error that it throws:
File "/root/anaconda3/lib/python3.5/multiprocessing/pool.py", line 268, in
starmap
return self._map_async(func, iterable, starmapstar, chunksize).get()
File "/root/anaconda3/lib/python3.5/multiprocessing/pool.py", line 608, in get
raise self._value
File "/root/anaconda3/lib/python3.5/multiprocessing/pool.py", line 385, in
_handle_tasks
put(task)
File "/root/anaconda3/lib/python3.5/multiprocessing/connection.py", line 206,
in send
self._send_bytes(ForkingPickler.dumps(obj))
File "/root/anaconda3/lib/python3.5/multiprocessing/connection.py", line 393,
in _send_bytes
header = struct.pack("!i", n)
struct.error: 'i' format requires -2147483648 <= number <= 2147483647
> /root/anaconda3/lib/python3.5/multiprocessing/connection.py(393)_send_bytes()
-> header = struct.pack("!i", n)
It works fine on any number of subsets of this data, but not when put together.
----------
components: Library (Lib)
messages: 279200
nosy: Justin Ting
priority: normal
severity: normal
status: open
title: Multiprocessing Pool starmap - struct.error: 'i' format requires
-2e10<=n<=2e10
type: behavior
versions: Python 3.5
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue28506>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
