Jason,
It is always acceptable to use a lkey MR instead of the local dma lkey, but ULPs should prefer to use the local dma lkey if possible, for performance reasons.
I don't necessarily agree with this statement (at least with the second part of it), the world is not always perfect. For RDMA READs, a HCA will perform the memory scatters when on the RX path, when receiving the read responses containing the data. This means that the HCA needs to perform a lookup of the relevant scatter entries upon each read response. Due to that, modern HCAs keep a dedicate cache for this type of RX-path lookup (which is limited in size naturally). So, having *small* sgls for rdma_read is much (much) more friendly to the HCA caches which means that for large transfers, even for IB, it might be better to register memory than having endless sgls. I was able to see that under some workloads. This is yet another non-trivial consideration that ULPs need to be aware of... We really are better off putting this stuff in the core... Sagi. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
