On Tue, 2011-02-22 at 15:38 -0800, Roland Dreier wrote: > On Tue, Feb 22, 2011 at 6:49 AM, David Dillow <dillo...@ornl.gov> wrote: > > Thanks for the note Or -- you confirmed my suspensions after testing > > last night. Looks like multiple MRs per request, then. > > Multiple MRs per request, or one fast reg MR per request?
Multiple fast reg MRs per request -- on mlx4, they max out at 511 entries, so I'd need 5 FRMRs to get to 8 MB requests. I'm scaling based on max_sect and the device attributes, so if one doesn't want more than 1 MB requests, they don't pay for the extra MRs -- or if other cards support larger/smaller fast reg MRs, then the code does the right thing. > This really should be roughly the same as old-style FMRs -- in both > cases, we need one "registration" per request that might be in flight. > Old-style FMRs and new IB-spec fast reg MRs should be roughly > the same in terms of resources consumed... Yeah, it seems pretty close overall. Fast reg MRs put more of it in view of SRP, where as old-style FMRs hid the bookkeeping in ib_pool_fmr. As an aside, is there any guarantee that ib_post_send() will not modify the list of work requests? I currently make no assumptions about that, but it would avoid some work if I could re-use a work request, and just change the fields I need instead of fully re-initializing it every time. -- Dave Dillow National Center for Computational Science Oak Ridge National Laboratory (865) 241-6602 office -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
