On Mon, Jul 21, 2003 at 08:37:14AM -0400, Ed Tomlinson wrote: > On July 20, 2003 03:53 pm, Ian Clarke wrote: > > A thought just occurred to me. It is quite possible, that from the > > perspective of the NG routing algorithm - that it will be preferable to > > send a request to a poorly suited node to whom a connection is already > > open, than to a well suited node to which we would need to establish a > > connection. > > Yes. The current code does two things to limit the effect of this. First > when sorting the NG weight it does not complete the sort, so it randomly > (in a quick sort sense) chooses one of the better nodes to route to. The > second thing that I do is open a connect when we get a new reference > to a node (this caused the exeception at startup). > > > Why is this bad? Well, NG would be making a perfectly sensible and > > rational decision given that its goal is to minimize the time required > > to retrieve the data for *this* request, but it doesn't account for the > > fact that the establishment of a connection will benefit future > > requests. > > agreed. > > > How do we fix it? Well, the hard way would be to try to factor in the > > future benefits of opening a connection to this node into this routing > > request. This would make things a bit messy as much of NG's elegance > > comes from the simplicity of the thing it is trying to minimize (how > > long will it take to get the data), but it would probably be doable > > (perhaps subtract, from the cost of establishing the connection, the > > estimated future time savings based on the average number of messages > > per connection). > > > > Fortunately, a much easier way would be to simply say that all nodes in > > the RT must already have connections opened to them the moment they > > enter the routing table. If such a connection cannot be established, > > then the node in question is ignored in routing decisions and perhaps > > removed from the datastore altogether. > > This is basicily what the current code tries to do. Quite probably I should > make nodes that do not have connection have a very low CP. Note about > the only thing CP is used for the NG code to to drop nodes (and give the > sort order for the display servlet).
Drop nodes? We don't drop nodes based on CP, do we? > > > There are several things that make this a better idea in NG than it > > might have been in the past: > > > > Firstly, NG should be able to get by with much fewer references (30-50 > > at a guess, but we will need to experiment). The reason is that the > > fewer references, the faster our node will collect information on each > > individual reference - so too many references could actually be bad for > > NG routing. Secondly, with NIO the cost of idle connections has been > > greatly reduced in-terms of threads (and therefore CPU and memory > > usage). > > Seems that 50 works fine here. > > > This will also serve to decrease any performance slowness for the user's > > first few requests which will otherwise be delayed as we establish > > connections. > > > > So hopefully that is problem solved for that issue, although it does > > raise a more general question of whether there are other areas in which > > NG Routing's lack of a "big picture" could be problematic. I can't > > think of any, and perhaps there aren't any - but we should be vigilant. > > > > Ian. -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so.
pgp00000.pgp
Description: PGP signature
