On July 20, 2003 03:53 pm, Ian Clarke wrote: > A thought just occurred to me. It is quite possible, that from the > perspective of the NG routing algorithm - that it will be preferable to > send a request to a poorly suited node to whom a connection is already > open, than to a well suited node to which we would need to establish a > connection.
Yes. The current code does two things to limit the effect of this. First when sorting the NG weight it does not complete the sort, so it randomly (in a quick sort sense) chooses one of the better nodes to route to. The second thing that I do is open a connect when we get a new reference to a node (this caused the exeception at startup). > Why is this bad? Well, NG would be making a perfectly sensible and > rational decision given that its goal is to minimize the time required > to retrieve the data for *this* request, but it doesn't account for the > fact that the establishment of a connection will benefit future > requests. agreed. > How do we fix it? Well, the hard way would be to try to factor in the > future benefits of opening a connection to this node into this routing > request. This would make things a bit messy as much of NG's elegance > comes from the simplicity of the thing it is trying to minimize (how > long will it take to get the data), but it would probably be doable > (perhaps subtract, from the cost of establishing the connection, the > estimated future time savings based on the average number of messages > per connection). > > Fortunately, a much easier way would be to simply say that all nodes in > the RT must already have connections opened to them the moment they > enter the routing table. If such a connection cannot be established, > then the node in question is ignored in routing decisions and perhaps > removed from the datastore altogether. This is basicily what the current code tries to do. Quite probably I should make nodes that do not have connection have a very low CP. Note about the only thing CP is used for the NG code to to drop nodes (and give the sort order for the display servlet). > There are several things that make this a better idea in NG than it > might have been in the past: > > Firstly, NG should be able to get by with much fewer references (30-50 > at a guess, but we will need to experiment). The reason is that the > fewer references, the faster our node will collect information on each > individual reference - so too many references could actually be bad for > NG routing. Secondly, with NIO the cost of idle connections has been > greatly reduced in-terms of threads (and therefore CPU and memory > usage). Seems that 50 works fine here. > This will also serve to decrease any performance slowness for the user's > first few requests which will otherwise be delayed as we establish > connections. > > So hopefully that is problem solved for that issue, although it does > raise a more general question of whether there are other areas in which > NG Routing's lack of a "big picture" could be problematic. I can't > think of any, and perhaps there aren't any - but we should be vigilant. > > Ian. Ed _______________________________________________ devl mailing list [EMAIL PROTECTED] http://hawk.freenetproject.org:8080/cgi-bin/mailman/listinfo/devl
