On Sat, Oct 04, 2003 at 11:12:07AM -0400, Ed Tomlinson wrote: > On October 04, 2003 10:21 am, Toad wrote: > > On Fri, Oct 03, 2003 at 11:40:25PM +0100, Toad wrote: > > > On Fri, Oct 03, 2003 at 12:40:03PM -0700, Ian Clarke wrote: > > > > Toad wrote: > > > > >On Sun, Sep 28, 2003 at 12:21:05AM -0500, Brandon Low wrote: > > > > >>Also, it is possible that inserts are __slow__ due (indirectly) to > > > > >> the asyncronizing of trailer sends. I've tried to explain my view > > > > >> of this to toad, but lets see if mentioning it here gets me > > > > >> anywhere: With trailers async, there is no local limit on how many > > > > >> trailers we can start, that is we will keep queueing up trailers > > > > >> till we are blue in the face, causing tremendous slowness in the > > > > >> actual transfer of said trailers. IT IS MY __HUMBLE__ opinion that > > > > >> this behavior is less desirable than having threads block on trailer > > > > >> sends forcing the node to limit itself on how many trailers it can > > > > >> start. > > > > > > > > I agree with Brandon. > > > > > > > > Would you rather go to a restaurant where they just kept accepting more > > > > customers, slowing the service afforded to their existing customers, or > > > > would you prefer that a restaurant did its best to serve its existing > > > > customers, and didn't accept new customers if existing customers would > > > > suffer? > > > > > > > > I would prefer the latter, and would point out that the latter > > > > restaurant may well end up serving as many if not more customers > > > > overall than the former. > > > > > > The solution as I hammered out with edt recently, is to make the send > > > queue length accurate (by only including data we actually have in cache > > > ready to send, this defeats some DoS attacks and makes the whole thing > > > much more accurate when dealing with stream-through requests), and to > > > reject all queries when the send queue exceeds some factor multiplied by > > > the bandwidth limit. He said he'd have a go at implementing it; I am > > > still working on PeerHandler. Do we have your approval? My main concern > > > was that a simple limit on the number of trailers moving was going to be > > > a major problem, but the above scheme should work. > > > > There is a major problem with this. > > > > We have no control over how fast data is sent out from the send queue if > > we limit the send queue - so we could end up using very low bandwidth > > and still rejecting all incoming requests, for no good reason. > > No. This does not have to be the case. > > > The right solution is exactly what we have now - if the output bandwidth > > usage is over some fraction of the limit, reject all queries. > > What we have now works as designed. Its lets us start far more work than > we have the resources to do. > > > What we have now is apparently not working, this is most likely due to > > some problem with the exact ratios, bugs in the bandwidth limiter or > > similar problems. You could try increasing lowLevelBWLimitMultiplier > > (default currently 1.2, used to be 1.5, maybe it is too low), disabling > > the hard limiter completely (doLowLevelOutputLimiting=false), or > > decreasing the outLimitCutoff (currently 0.9, try 0.75). This should > > produce similar results to what you have with your other limiting > > schemes - in theory - modulo bugs. > > What I have do is setup a feedback loop that finds the smallest sendQueueSize > such that we are using our bandwidth. I we do not use the bandwidth it > increases the window and the bandwidth usage goes up. The converse > is also true.
You are using all sorts of voodoo that you have not explained to me. Please explain it. And what is wrong with my suggestion anyway? If we are using all our available resources, we won't accept any more requests. It's just that the current implementation has a few problems - you yourself said that it was sending 8kB/sec of a 10kB/sec limit. The rejection code only kicks in at 9kB/sec (90%), and the hard limit is 120% - but the hard limit may be too conservative, and limiting it to 80% or so, with the result that it never actually hits the QueryReject limit. What is wrong with this theory? It fits the facts. > > Ed -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so.
signature.asc
Description: Digital signature
_______________________________________________ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
