On Tuesday 20 October 2009 05:01:50 Ian Clarke wrote: > It would be interesting to try to build up a picture of what exactly > is happening while we wait for a data request to come back with a > response. We could presumably do this by collecting statistics on one > or more nodes, tracking stuff like: > > - Time from a request entering to exiting > - Time for a request existing to generate a response relative to its > TTL and anything else that might indicate where it is in its request > chain > > >From this we could narrow down where delays are coming from, is it > throughput, or perhaps just long request chains? > > Has anything like this been done yet?
Yes, some work has been done. We played around with various load management policies in February (?) to try to improve latency, and ran up against a nasty throughput/latency tradeoff: Because most requests fail (even most CHK requests), we have to accept a reasonable number of requests to ensure that we use up our bandwidth. But if we accept too many, the transfers will be very slow, and if by some chance they all succeed, they may become *really* slow. So we target a specific timeout (currently 90 seconds) and accept requests up to the point where if they all succeeded, they would still be able to transfer all their data within that period; this is called bandwidth liability limiting. We could reduce the timeout, but then if fewer requests succeeded we might not use all available bandwidth, resulting in lower throughput. IMHO the best medium-term solution is to have a flag on each request to either prefer throughput or latency. We have implemented some hacks so that if a request is slow, we receive the data on one node and kill the request downstream, and then use ULPRs to offer the data to the nodes that requested it. IMHO this is 0.9 stuff: as evanbd says, targeting data retrievability (which right now is surprisingly bad) will help more than working on latency. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20091022/c352a2b6/attachment.pgp>
