Hey,
For the CALLBACK option, you would use that to have the individual
methods filling in things at the "generic BMI" layer (for lack of the
right terminology), but the overall user API would be the same?
I don't think that the CONTEXT option is appropriate. I don't want to
expose the specifics of the underlying networks any more than we have
already.
There should be relevant research in the MPI space related to the
POLL_PLAN option.
Do we consider this to be a problem for both clients and servers, or
is it really a server-specific issue? If this is something we think
will solely (or mostly) a server thing, we could consider throwing a
thread at the issue. One option might be to kick off a thread to wait
on the TCP side of things, since the kernel is doing most of the work
for us anyway, and put completed TCP events into the completion list
asynchronously (for servers only)?
Rob
On Jan 7, 2009, at 4:06 PM, Sam Lang wrote:
Hi All,
Right now if multiple methods are enabled in BMI, we tend to get
poor performance from the "fast" network, because BMI_testcontext
iterates through all the active methods calling testcontext for each
one. It tries to be smart about which methods get scheduled ;-) to
prevent starvation, but it treats all the methods fairly, which
tends to make tcp (the slow one) hog the time spent in testcontext.
I have a few ideas for this, so I'll go ahead and propose them and
let you all shoot them down or propose others.
Option CALLBACK: Instead of returning completion as a list in
testcontext, we allow a BMI context to be constructed with a
callback, and on completion of operations, the callback is called.
This allows each method to drive its own operations, and notify the
consumer of completion immediately. There would still need to be a
testcontext call for methods that only service operations during
that call. The changes might not be that significant, the
BMI_open_context call could just take an extra parameter that was
the callback function. If the parameter is null, we just use the
completion list as before.
Option CONTEXT: Require separate contexts for separate methods.
This pushes the problem up to the application, probably not where it
belongs, since active methods are opaque from the BMI api.
Option POLL_PLAN: Modify the construct_poll_plan function in bmi
that already tries to be fair, so that its aware of the performance
discrepancy between methods. Maybe it can just skip tcp every other
time for example. This is probably the easiest, since it doesn't
require API changes and the like.
-sam
_______________________________________________
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
