Elliot Hallmark <permafact...@gmail.com> wrote: > Strula, this sounds brilliant! To be clear, you're talking about > serializing the numpy array and reconstructing it in a way that's faster > than pickle?
Yes. We know the binary format of NumPy arrays. We don't need to invoke the machinery of pickle to serialize an array and write the bytes to some IPC mechanism (pipe, tcp socket, unix socket, shared memory). The choise of IPC mechanism might not even be relevant, and could even be deferred to a library like ZeroMQ. The point is that if multiple peocesses are to cooperate efficiently, we need a way to let them communicate NumPy arrays quickly. That is where using multiprocessing hurts today, and shared memory does not help here. Sturla _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
