On Wed, 03 Dec 2003 01:20:50 +0000, Toad wrote: > On Wed, Dec 03, 2003 at 12:37:29AM +0200, Jusa Saari wrote: >> I'm running stable build 5046. Currently, I have 49 nodes in my routing >> table, 45 which are contacted. Of these 45, only 4 a not backed off >> currently. At no time have I observed more than 6 nodes being not backed >> up. >> >> Now then. If I understood correctly, this means that my node only has 4 >> other nodes it can route all requests to. I would imagine this doing nasty >> things to routing (if it doesn't, then why have such a large routing table >> to begin with ?), and of course those 4 nodes aren't likely to like the >> flood of requests either. >> >> Also, my own node can only handle about 2000 requests per hour before >> going to rejecting all state because of thread limit. With NGRouting, the >> same node got 10000+ QPH (and was in "reject all" or "reject most" all the >> time. >> >> So... As far as I can tell, the biggest problem right now is load. Not >> load balancing, but the total amount of load. Freenet simply cannot >> currently handle the amount of requests made. >> >> I wonder if most nodes rejecting most queries isn't at least partially to >> blame for NGRoutings failure - after all, if we can't route to the "best" >> node, but have to take whatever node happens to let us through, then >> routing will be essentially random, with just a small bias towards the >> "better" nodes (since they will be tried first - but backing up will take >> away even that, since they won't be even tried). >> >> So, please, optimize ! Make a binary message format (faster to parse, I'd >> assume), better NIO, multiplexing connections, _anything_ that will let >> nodes handle more requests. And for pity's sake, remove the "backing off >> on QueryReject" -feature; at least give the node a chance to relay to the >> best node ! Premature optimization might be the root of all evil, but it >> seems to me it's either optimize or get everyone an AMD64 ;). >> >> Anyway, it's completely pointless to search for better routing methods, >> untill a node can actually send it's messages to the node chosen by that >> routing method, as opposed to having that node QueryReject and the >> messsage forwarded to a more or less random node... >> >> And no, load balancing won't help; it's a matter of total load exceeding >> total resources. You can't divide 10 tons to be carried by 10 people, no >> matter how hard you try. > > Where do you think the load comes from? And where are you going to send > it? Why do you think we implemented NIO? Why do you think we implemented > backoff? The purpose of backoff is primarily to REDUCE TOTAL LOAD. > Please make a consistent argument.
Well, maybe I've misunderstood something. Lets see now... My node, X, has two nodes, A and B, in it's routing table. X gets a request for which A is the supreme choice, and B a terrible one. However, A is backed of, so X routes the request to B. Because B is bad at finding this sort of data, the request is likely to take longer (more hops) to complete than if it had been routed to A, and is more likely to fail with DNF, which will cause a retry. In short, while load has been balanced (moved from A to B), the total load on the network has increased, since it takes more hops to find the data. Also the likelihood of the original requester getting the data has been reduced, so he's more likely to retry, further increasing the load. Please explain which part of this is incorrect ? Coming to think about it, isn't the whole concept of load-balancing incompatible with routing ? We want a request to be routed to the node that is most likely to return the data fastest (whatever that means...), but the very definition of load-balancing means we give the request to the node which has lowest load. If these don't happen to be the same, then either routing will break or load-balancing will break. A possible solution might be to take (for each routing decision) the best five nodes in the routing table, and give the request to the least loaded of them. That should result in a reasonable compromise between optimal routing and optimal load-balancing... This would require nodes to have real-time information about each others load situation to work well. Perhaps a new message, ReportLoadStatus ? _______________________________________________ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
