sl...@mcs.anl.gov wrote on Wed, 07 Jan 2009 16:06 -0600:
> Right now if multiple methods are enabled in BMI, we tend to get poor
> performance from the "fast" network, because BMI_testcontext iterates
> through all the active methods calling testcontext for each one. It
> tries to be smart about which methods get scheduled ;-) to prevent
> starvation, but it treats all the methods fairly, which tends to make
> tcp (the slow one) hog the time spent in testcontext. I have a few
> ideas for this, so I'll go ahead and propose them and let you all shoot
> them down or propose others.
I've always been fond of a third Option: CENTRALIZED_POLLING. All
BMI methods are changed to hand back an fd to some core BMI routine.
Individual BMI methods do not poll their devices. The core BMI
routine sticks all fds in a single select() or epoll(), and when it
gets one that triggers, calls back into the appropriate BMI method
to do its business. No need to balance across all the methods.
This can work today with TCP obviously, and IB with some minor
manipulation. GM cannot fit in such a framework, I believe, being
completely poll-driven. MX should work however, I think. If a
method wants to poll for a bit after getting an fd trigger, it can
get away with that.
This is how pretty much all externally driven server applications
work today. Lots of threads are still not as good a way to manage
concurrency.
Tweaking construct_poll_plan() is doomed to fail. I've tried.
Maybe I misunderstood your CALLBACK option and you're thinking like
this too.
-- Pete
_______________________________________________
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
