On Nov 13, 2012, at 11:36 PM, Christopher Mitchell <christop...@cemetech.net> wrote:
> Hi, > > I am working on building an Infiniband application with a server that > can handle many simultaneous clients. The server exposes a chunk of > memory that each of the clients can read via RDMA. I was previously > creating a new MR on the server for each client (and of course in that > connection's PD). However, under stress testing, I realized that > ibv_reg_mr() started failing after I simultaneously MRed the same area > enough times to cover 20.0 GB. I presume that the problem is reaching > some pinning limit, although ulimit reports "unlimited" for all > relevant possibilities. I tried creating a single global PD and a > single MR to be shared among the multiple connections, but > rdma_create_qp() fails with an invalid argument when I try to do that. > I therefore deduce that the PD specified in rdma_create_qp() must > correspond to an active connection, not simply be created by opening a > device. > > Long question short: is there any way I can share the same MR among > multiple clients, so that my shared memory region is limited to N > bytes instead of N/C (clients) bytes? Christopher, Yes, it is possible. You have to use the same PD for all QPs/connections. We do this in CCI when using the Verbs transport. Scott-- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
