So if I have 16K QP for a node, the QPs will use 16K*64K = 1G. It is fairly large memory.
If I change max_send_wr=16, inline_data=16, the memory per QP will be 16*16 = 256 bytes ? I am asking Mellanox engineer, is there any document or will document the formula ? --CQ > -----Original Message----- > From: Roland Dreier [mailto:[EMAIL PROTECTED] > Sent: Tuesday, October 30, 2007 4:14 PM > To: Tang, Changqing > Cc: [email protected] > Subject: Re: [ofa-general] How much pinned memory each QP needs ? > > > We are running a big system with some memory > issue(16K QPs), I want to know how much > > pinned memory each QP needs, the settings when creating QP > is follows: > > > > qp_init_attr.cap.max_send_wr = 136; > > qp_init_attr.cap.max_recv_wr = 1; > > qp_init_attr.cap.max_send_sge = 1; > > qp_init_attr.cap.max_recv_sge = 1; > > qp_init_attr.cap.max_inline_data = 128; > > Not sure really without tracing through the code. The > easiest way to find out would probably be to add a print > statement into the low-level driver library to find out how > big a buffer it allocates for the QP. > > Naively, the max_send_wr is going to be rounded up to a power > of 2, so 256, and so will the work request size, which has to > be over the inline data size of 128, so that will be 256 > bytes. So I would guess you'll end up using about 256 * 256 > or 64 KB per QP. > > - R. > _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
