Hi Lev, Thanks a lot for the example! It really helps!
Just to make sure I understand the example in the right way: The actual p2p data transfer happens when executing the following line? x_gpu = gpuarray.GPUArray(shape, dtype, gpudata=drv.IPCMemoryHandle(h)) So, if I use different devices in proc1 and proc2, then x_gpu = gpuarray.GPUArray(shape, dtype, gpudata=drv.IPCMemoryHandle(h)) would actually transfer the data from device 1 to device 2 through p2p? My GPU server has 3 gpus, and only device 1 and 2 can do p2p. What I observed is that the example works when 1. proc1 and proc2 using the same device 2. proc1 and proc2 using device 1 and 2 respectively or vice versa it doesn't work when proc1 using device 0 and proc2 using 1 or 2 and vice versa it also returns the error of "LogicError: cuIpcOpenMemHandle failed: invalid/unknown error code" So I guess that's the desired behaviour? and IPCMemoryHandle only works when p2p (or within same GPU) is possible? Thanks a lot! On Tue, Oct 7, 2014 at 12:21 AM, Lev Givon <l...@columbia.edu> wrote: > Received from Gavin Weiguang Ding on Mon, Oct 06, 2014 at 11:55:49PM EDT: > > Hi Lev, > > > > Thanks for the reply! > > My GPUs do support GPUDirect. And I've tested using the "simpleP2P" from > > cuda samples. > > > > I've been trying a little bit on that but without success. I'm new to > > pycuda and multiprocessing, excuse me if I ask dumb questions. > > > > If I understand it right, I need to do pycuda.driver.init() and > > make_context() inside each process. > > > > But to use pycuda.driver.memcpy_peer, I need to pass the context defined > in > > one process to another. But when I try to pass the context with Pipe or > > Queue from multiprocessing, it returns pickling error. > > Is this the right way of doing it? assuming pickling error can be solved. > > Since CUDA contexts are private, you can't use the context set up in one > process > in another. In recent versions of CUDA, you can use its IPC API to > transfer a > GPU memory address from one GPU to another. See > https://gist.github.com/lebedov/6408165 for an example of how to use the > API > (requires pyzmq). > -- > Lev Givon > Bionet Group | Neurokernel Project > http://www.columbia.edu/~lev/ > http://lebedov.github.io/ > http://neurokernel.github.io/ > >
_______________________________________________ PyCUDA mailing list PyCUDA@tiker.net http://lists.tiker.net/listinfo/pycuda
