Raghu Angadi wrote:
Right now Namenode does not know about the cluster size before starting
IPC server.
Sounds like perhaps we should make the handler count, queue size, etc.
dynamically adjustable, e.g., by adding Server methods for
setHandlerCount(), setQueueSize(), etc. There's been talk of trying to
automatically adjust these within Server.java, based on load, and that
would be better yet, but short of that, we might adjust them
heuristically based on cluster size.
The urgent thing, since we expect the best settings for large clusters
to change, is to make it so that folks don't need to adjust these
manually, even if the automation is an ill-understood heuristic. I
think we can easily get some workable heuristics into 0.15, but we might
not get be able to implement async responses or figure out how to adjust
it automatically in Server.java or whatever in that timeframe. Perhaps
we should just change the defaults to be big enough for 2000 nodes, but
that seems like too big of a hammer.
Doug
- Re: [Lucene-hadoop Wiki] Update of "FAQ" by ... Doug Cutting
-
