On Aug 18, 2006, at 8:33 PM, Pete Wyckoff wrote:
[EMAIL PROTECTED] wrote on Fri, 18 Aug 2006 18:44 -0500:
The BMI_testeunexpected call returns a list of BMI_unexpected_info
structs. Each of these has a buffer that's been allocated by the bmi
method internally. It looks like that buffer gets freed in
server_state_machine_complete (pvfs2-server.c) with a call to free
(also gets freed on an error). Two questions:
1. Should BMI_memfree be used instead of free, since the allocation
is done in the specific bmi method?
The unexpected message is received into network-registered memory,
but then copied to hand back to the user. This copied buf is just
malloc()-ed, not BMI_memalloc()-ed, because it won't ever be used
for BMI operations (okay, unlikely at least). So free is right.
Under the hood, the network receive buffer is recycled back into
service before the upper layers look at it. In IB, this is done at
the end of a successful testunexpected to provide some flow control
on unex messages. In GM, the memcpy happens earlier, so that the
network receive buffer gets put back into service as soon as the
unex message comes off the network.
Hi Pete,
Thanks for the method details. My concern is mostly with the
interface semantics for future bmi method implementations, as well as
other users of bmi (possibly outside of pvfs). Could a BMI method
potentially return unexpected message buffers not allocated with malloc?
Also, from a consistency perspective, I would prefer to have bmi
methods return malloced buffers, and not expect the user to know to
call free on that buffer. If only that bmi may at some point be a
separate library, allocating memory in one library and freeing it in
another is usually considered bad practice.
-sam
2. Could the free of that buffer occur in
server_state_machine_start, right after the call to PINT_decode? It
doesn't look like the buffer is used after that, and it might allow
the bmi method to reuse buffers better if it handles unexpected
messages that way. At the least, it might reduce the server's
footprint a little under heavy load.
Sure. It's your buffer. Do with it what you will, as long as you
free it when you're done. Might reduce a bit of RSS, but unex
messages are small to start with and we don't expect too many of
them on each connection. Still, earlier is better; I guess right
after the PINT_decode() should work.
-- Pete
_______________________________________________
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
