Thanks for the clarifications Mark. That settles it.
Manoj Nambiar
-----Original Message-----
From: Walukiewicz, Miroslaw
Sent: Wednesday, January 19, 2011 6:51 PM
To: Manoj Nambiar ; linux-rdma@vger.kernel.org
Subject: RE: Problems with ibv_post_send and completion queues
Hello Manoj,
Responding to your questions.
1. the 510 is HW max for QP length for NE020 card so it cannot be increased.
2. The NE020 driver keeps posted buffers on the FIFO-like queue which is 510
entries long in your application.
There are maintained tail and head pointers that does not allow for QP
overflow. The head pointer is updated during post_send and tail is updated
during poll_cq on CQ assigned to your QP. When you make post_sends without
checking cq the tail pointer is not updated so after 510 post_send calls the
QP looks being full for driver and in effect you cannot send more data using
that QP.
Regards,
Mirek
-----Original Message-----
From: linux-rdma-ow...@vger.kernel.org
[mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Manoj Nambiar
Sent: Wednesday, January 19, 2011 12:21 PM
To: linux-rdma@vger.kernel.org
Subject: Problems with ibv_post_send and completion queues
Hi,
I am writing some rdma based programs using the Intel NetEffect iWARP NICs.
I am running into the following problems with my code -
1.
I can only set maximum work requests to 510 using rdma_create_qp, otherwise
it gives me an error – “libnes: nes_ucreate_qp Bad sq attr parameters
max_send_wr=511 max_send_sge=1.
Is there a way to increase this?”
2.
Is there a way to do RDMA writes without using a completion queue? I use an
alternative channel to determine if my work requests were correctly executed
or not. When I tried to do so I could send 510 (may be related to the
previous question) work requests successfully. After that ibv_post_send
returns 22. Repeatedly retrying ibv_post_send doesn’t seem to clear the
problem. It returns the same error code. Checked up the error code which
tells me invalid arguments.? Unable to make sense of this. Is there a way
to clean up the work requests in the system?
I am creating the queue pair with sq_sig_all = 0 in struct ibv_qp_init_attr
and am not setting setting IBV_SEND_SIGNALED in the send_flags member of
struct ibv_send_wr. This means I do not get any completion events. When
calling rdma_create_qp I initialize the send_cq and recv_cq of the struct
ibv_qp_init_attr with the completion queue created using ibv_create_cq (with
cge parameter same as cap.max_send_wr in struct ibv_qp_init_attr. (I think
that creating a completion queue is a must for creating an rdma queue pair -
pls correct me if I am wrong.)
Pls note – I do not get this problem when I poll completion queues (when
rdma queue pair is created with sq_sig_all = 0 && IBV_SEND_SIGNALED set in
the flag of the work requests to ibv_post_send)
Thanks,
Manoj Nambiar
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
N���r�y��زX��vؖ)�{nlj��{�ي{y�ʇ��j�f����z��w����j:v��wjض����zZ�����j��
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
