On Aug 21, 2006, at 5:58 PM, Pete Wyckoff wrote:
[EMAIL PROTECTED] wrote on Mon, 21 Aug 2006 14:32 -0500:
Sam and I raised this question for the following reason (motivated
during the discussion of
BMI-MX's design with Scott)
Suppose a BMI method wants to keep around a pre-allocated pool of
unexpected buffers, it seems wasteful to copy them into a user
buffer and
then free it again.
So we were kind of wondering if a test_unexpected() returns one of
those
preallocated buffers (and mark them internally as being used) and
then
the caller can then call BMI_memfree() which will return it to the
pool of
unexpected buffers (by marking them as free again).
But if this does not appeal to you guys, I suspect the
unexpected_free()
callback function would also be better than free()..
I think doing what Phil suggested with BMI_unexpected_free() is the
best way to go. This is the same thing you're describing except it
makes explicit the separate functions of: 1) freeing
network-registered memory (BMI_memfree), and 2) acknowledging an
unexpected message has been serviced (BMI_unexpected_free).
You'll just have to go through the code and audit all users of
testunexpected() to make sure they call your new function instead of
normal free(); it'll have to be mandatory from now on. And add
stubs in bmi_* that implement this by calling free().
Don't expect to see huge performance gains by doing this, though, as
unexpected messages tend to be pretty small and everything will sit
in L2 anyawy. But saving the malloc/memcpy/free is probably worth
the expansion of the API, I'll agree. It'll also clear up any
confusion about the lifetime of returned buffers, and bring
responsibility for flow control of unexpected messages up to the
application where it belongs. Once you toss in these changes, I'll
switch bmi_ib over to be clever with unexpected buffers too, as you
describe.
-- Pete
Sam and Murali brought this up based on my questions about
BMI_memalloc() and BMI_memfree(). Unlike GM, MX does not have a
memory registration API. It handles registration within the library
automatically. The user always passes in plain malloc'd buffers for
both send and receive. Since the unexpected buffer for a MX BMI would
simply be a malloc'd buffer, I did not think it made much sense to
malloc another buffer, copy the data and pass that back to BMI.
Given that, I could live with BMI_memalloc() and BMI_memfree() that
are simply malloc() and free().
Scott
_______________________________________________
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
