sunce4t commented on issue #3102: URL: https://github.com/apache/brpc/issues/3102#issuecomment-3333067648
> > > [@sunce4t](https://github.com/sunce4t) 请教一下，训练使用GPU Direct RDMA的时候，需要精细化管理显存，让每个RPC的数据写入指定的显存吗？还是直接从池子里取一块显存写入？ > > > > > > [@yanglimingcn](https://github.com/yanglimingcn) [@chenBright](https://github.com/chenBright) hi，如果我们attachment里存放显存地址，在CutFromIOBufList中解析这个attachment，这是否修改难度较小 > > CutFromIOBufList是在发送消息的时候使用的，解析这块在ProcessRpcRequest这里面。你是打算attachment都直接使用显存的内存吗？ 是的，void * device_ptr = InitGPUMemoryForRDMA(attachment_size); mr = ibv_reg_mr(brpc::rdma::GetRdmaPd(), device_ptr, attachment_size, IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_ATOMIC | IBV_ACCESS_REMOTE_READ | IBV_ACCESS_REMOTE_WRITE); _attachment.append_user_data_with_meta(device_ptr, attachment_size, [](void*){}, mr->lkey); 在发送方，我们将attachment内的数据替换成显存中的数据，接收方postrecv时也可以类似这样去替换成显存 现在问题是收到消息进入ProcessNewMessage后，这里需要加一些从显存拷贝 协议头部+meta+body 到内存的逻辑 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
