Hi,
We are working to create a RDMA client within the kernel which will
connect to a working RDMA userspace server.
We are using the userspace code for testing purpose , in the final
version all the communication will be done within the kernel space.
We rely on softiwarp so far for dev purpose.
We have achieved a connection but are now having trouble prior to the
communication stage.
We are trying to register a user space created buffer using the
device->reg_user_mr . However the kernel crash and the crash dump
shows a page fault after siw_reg_user_mr.
What we would like to know:
If this is the correct approach to register memory for RDMA
communication ? If yes what are we doing wrong ?
Is there a possibility to use kernel allocated memory rather than
using userspace created and then passing the address back to the
kernel through IOCTL ? ( if yes if there is any code example floating
around?)
basically the question boils down to how to allcoate and registr
buffer for RDMA communication from inside a kernel module?
This is the code.
######################################################################################
struct rdma_cm_id *id = rdma_create_id(my_rdma_cm_event_handler
event_handler, NULL, RDMA_PS_TCP, IB_QPT_RC);
struct ib_pd *pd = ib_alloc_pd(id->device);
struct ib_cq *cq = ib_create_cq(id->device, NULL, NULL, NULL, 2, 0);
// Userspace buffer addresses passed via IOCTl
unsigned long user_buffer_send;
unsigned long user_buffer_recv;
int my_rdma_cm_event_handler event_handler(struct rdma_cm_id *id,
struct rdma_cm_event *event)
{
switch (event)
{
case RDMA_CM_EVENT_ADDR_RESOLVED:
rdma_resolve_route(id, 2000);
break;
case RDMA_CM_EVENT_ROUTE_RESOLVED:
{
struct ib_qp_init_attr attr;
memset(&attr, 0, sizeof attr);
attr.send_cq = cq;
attr.recv_cq = cq;
attr.sq_sig_type = IB_SIGNAL_ALL_WR;
attr.cap.max_send_wr = 2;
attr.cap.max_recv_wr = 2;
attr.cap.max_send_sge = 1;
attr.cap.max_recv_sge = 1;
attr.qp_type = IB_QPT_RC;
attr.port_num = id->port_num;
attr.event_handler = my_qp_event_handler;
rdma_create_qp(id, pd, &attr)
rdma_connect(id, &conn_param);
break;
}
case RDMA_CM_EVENT_ESTABLISHED:
{
struct ib_mr *send_mr = id->device->reg_user_mr(pd,
(u64)
user_buffer_send, sizeof *user_buffer_send, NULL,
IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_READ |
IB_ACCESS_REMOTE_WRITE, NULL);
struct ib_mr *recv_mr = id->device->reg_user_mr(pd,
(u64)
user_buffer_recv, sizeof *user_buffer_recv, NULL,
IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_READ |
IB_ACCESS_REMOTE_WRITE, NULL);
break;
}
}
...
}
######################################################################################
--
" The production of too many useful things results in too many useless people"
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
