I think Jason is the only one who is remaining at least somewhat on-
topic here.
My goal for this thread was to open discussion about solving the
broken memory management model for OpenFabrics. Specifically, I'm
looking for a software solution for today's hardware (both IB and
iWARP).
While MPI is currently the biggest victim, this broken memory
management model is also an enormous roadblock for any other
application or ULP to write to verbs.
On-demand paging, while the desired end result sounds great, will
definitely require new hardware and will likely be a very complex
design conversation that takes a long, long time. That's great if my
proposal [finally] seriously inspires people to tackle this problem
(and/or other hardware-assisted ideas), but I consider those to be
longer-term solutions.
I'm looking for a) a much shorter-term solution that is b) workable on
current hardware.
Thanks.
On Apr 29, 2009, at 6:44 PM, Jason Gunthorpe wrote:
On Wed, Apr 29, 2009 at 03:28:00PM -0700, Ralph Campbell wrote:
> Besides, mmap() only allocates a virtual address range in the user's
> address space. It doesn't fault in all the pages into physical
memory.
> That happens when the application tries to read or write memory in
> the VA range of the mmap. The IB memory registrations need physical
> addresses and it would be impractical to do this for every mmap or
> brk.
If your goal is to keep the mr consistent then you only need to fault
and pin pages from the new mmap that intersect with pre-existing
memory registrations.
I chucked out 3 things to consider:
- Pin and register all process memory (no swap!)
- Keep the MR consistent by pinning and registering new mmaps
that intersect with pre-existing memory registrations
- Keep the MR consistent by preventing the kernel from returning
new mmaps that overlap existing memory registrations.
Jason
--
Jeff Squyres
Cisco Systems
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
