Hello Manoj, Responding to your questions. 1. the 510 is HW max for QP length for NE020 card so it cannot be increased.
2. The NE020 driver keeps posted buffers on the FIFO-like queue which is 510 entries long in your application. There are maintained tail and head pointers that does not allow for QP overflow. The head pointer is updated during post_send and tail is updated during poll_cq on CQ assigned to your QP. When you make post_sends without checking cq the tail pointer is not updated so after 510 post_send calls the QP looks being full for driver and in effect you cannot send more data using that QP. Regards, Mirek -----Original Message----- From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Manoj Nambiar Sent: Wednesday, January 19, 2011 12:21 PM To: linux-rdma@vger.kernel.org Subject: Problems with ibv_post_send and completion queues Hi, I am writing some rdma based programs using the Intel NetEffect iWARP NICs. I am running into the following problems with my code - 1. I can only set maximum work requests to 510 using rdma_create_qp, otherwise it gives me an error – “libnes: nes_ucreate_qp Bad sq attr parameters max_send_wr=511 max_send_sge=1. Is there a way to increase this?” 2. Is there a way to do RDMA writes without using a completion queue? I use an alternative channel to determine if my work requests were correctly executed or not. When I tried to do so I could send 510 (may be related to the previous question) work requests successfully. After that ibv_post_send returns 22. Repeatedly retrying ibv_post_send doesn’t seem to clear the problem. It returns the same error code. Checked up the error code which tells me invalid arguments.? Unable to make sense of this. Is there a way to clean up the work requests in the system? I am creating the queue pair with sq_sig_all = 0 in struct ibv_qp_init_attr and am not setting setting IBV_SEND_SIGNALED in the send_flags member of struct ibv_send_wr. This means I do not get any completion events. When calling rdma_create_qp I initialize the send_cq and recv_cq of the struct ibv_qp_init_attr with the completion queue created using ibv_create_cq (with cge parameter same as cap.max_send_wr in struct ibv_qp_init_attr. (I think that creating a completion queue is a must for creating an rdma queue pair - pls correct me if I am wrong.) Pls note – I do not get this problem when I poll completion queues (when rdma queue pair is created with sq_sig_all = 0 && IBV_SEND_SIGNALED set in the flag of the work requests to ibv_post_send) Thanks, Manoj Nambiar -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
