Re: [PATCH WIP 00/43] New fast registration API
On Wed, Jul 22, 2015 at 11:27:02AM -0600, Jason Gunthorpe wrote: What is SRP trying to accomplish with that? The only reason that springs to mind is to emulate IB_MR_MAP_ARB_SG ? It's not emulating IB_MR_MAP_ARB_SG, it simply allows muliple memory registrations per I/O request. Be that to support gappy SGLs in a generic way, or to allow larger I/O sizes than the HCA MR size. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH WIP 00/43] New fast registration API
On Wed, Jul 22, 2015 at 08:42:32PM +0300, Sagi Grimberg wrote: We can do that, but I'd prefer not to pollute the API just for this single use case. What we can do, is add a pool API that would take care of that. But even then we might end up with different strategies as not all ULPs can use it the same way (protocol constraints)... Today SRP has this logic that registers multiple SG aligned partials. We can just have it pass a partial SG list to what we have today instead of building the page vectors... Or if we can come up with something that will keep the API trivial, we can take care of that too. Supporting an array or list of MRs seems pretty easy. If you ignore the weird fallback to physical DMA case when a MR fails case the SRP memory registration code isn't significanly more complex than that in iSER for example. And I think NFS needs the same support as well, as it allows using additional MRs when detecting a gap. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH WIP 00/43] New fast registration API
On 7/23/2015 12:28 PM, Christoph Hellwig wrote: On Wed, Jul 22, 2015 at 08:42:32PM +0300, Sagi Grimberg wrote: We can do that, but I'd prefer not to pollute the API just for this single use case. What we can do, is add a pool API that would take care of that. But even then we might end up with different strategies as not all ULPs can use it the same way (protocol constraints)... Today SRP has this logic that registers multiple SG aligned partials. We can just have it pass a partial SG list to what we have today instead of building the page vectors... Or if we can come up with something that will keep the API trivial, we can take care of that too. Supporting an array or list of MRs seems pretty easy. I'm missing the simplicity here... If you ignore the weird fallback to physical DMA case when a MR fails case the SRP memory registration code isn't significanly more complex than that in iSER for example. And I think NFS needs the same support as well, as it allows using additional MRs when detecting a gap. This kinda changing the semantics a bit. With this we need to return a value of how many MRs used to register. It will also make it a bit sloppy as the actual mapping is driven from the drivers (which use their internal buffers). Don't you think that a separate pool API is better for addressing this? -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH WIP 00/43] New fast registration API
On Wed, Jul 22, 2015 at 10:10:23AM -0700, Christoph Hellwig wrote: The one thing I'm curious about is how we can support SRP with it's multiple MR support without too much boilerplate code. One option would be that pass an array of MRs to the map routines, and while most callers would just pass in one it would handle multiple for those drivers that supply them. What is SRP trying to accomplish with that? The only reason that springs to mind is to emulate IB_MR_MAP_ARB_SG ? Jason -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH WIP 00/43] New fast registration API
On 7/22/2015 8:10 PM, Christoph Hellwig wrote: Thanks Sagi, this looks pretty good in general, various nitpicks nonwithstanding. The one thing I'm curious about is how we can support SRP with it's multiple MR support without too much boilerplate code. One option would be that pass an array of MRs to the map routines, and while most callers would just pass in one it would handle multiple for those drivers that supply them. We can do that, but I'd prefer not to pollute the API just for this single use case. What we can do, is add a pool API that would take care of that. But even then we might end up with different strategies as not all ULPs can use it the same way (protocol constraints)... Today SRP has this logic that registers multiple SG aligned partials. We can just have it pass a partial SG list to what we have today instead of building the page vectors... Or if we can come up with something that will keep the API trivial, we can take care of that too. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH WIP 00/43] New fast registration API
Thanks Sagi, this looks pretty good in general, various nitpicks nonwithstanding. The one thing I'm curious about is how we can support SRP with it's multiple MR support without too much boilerplate code. One option would be that pass an array of MRs to the map routines, and while most callers would just pass in one it would handle multiple for those drivers that supply them. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
