On Friday 04 January 2008 21:32, Robert Hailey wrote: > > On Jan 4, 2008, at 2:30 PM, Matthew Toseland wrote: > > > On Friday 04 January 2008 18:32, Robert Hailey wrote: > >> > >> Apparently until this revision 16886, (so long as any one node does > >> not timeout) a node will take as long as necessary to exhaust > >> routable > >> peers. Even long after the original requestor has given up on that > >> node. > > > > Yes. Is this bad? Obviously there are limits - if it gets a post- > > accepted > > timeout on any one node it will finish the request. > > > > Generally I think this is probably a good thing - the data is > > wanted, so why > > not find it? It will be cached and will be transferred later. With > > ULPRs it > > will even be transferred when we complete, despite the timeout. > > So in the best case, the timeout value was just-wrong-enough for > overall network topology towards this address. In the worst case, the > data does not exist (e.g. looking for a newer USK, or the next frost > message)... which might actually be the common case.
In which case, it will run out of HTL reasonably quickly. I don't see a problem. > > In either case, resuming a request after we know that the upstream > peer has forgotten about it could be very bad. Assuming 20 peers (ala > opennet), the theoretical worst-case-per-node is that the last new > request will leave the node about 40 minutes from when it entered the > node. Not possible because of HTL. We will DNF (or timeout) before we've visited all 20 nodes. We don't allow RNFs to increase the HTL or change the nearest-loc-found, but we do allow them to decrease the HTL. And we decrement the HTL unless nearestLoc improves (the HTL decrement/reset code really needs to be looked at, it's far from clear and may be wrong in places). One caveat: at HTL=10 or HTL=1, whether or not the node decrements is determined probabilistically (the decision is made once per source node). > To the best of my knowledge, all of the upstream nodes will not > respond with the LOOP rejection before then. And even well before the > worst case, this effect can accrue across many nodes in the path. If the same request is routed to a node which is already running it, it rejects it with RejectedLoop. If it's routed to a node which has recently ran it, it again rejects it. If it is a different request for the same key, it may be coalesced. > > >> I suspect good and bad from this. On the one hand, all nodes will > >> become much less busy (increase throughput, decrease latency), but > >> the > >> data may be not be found at all (as presently the node may continue > >> to > >> search all its peers and it be cached for next time). > > > > Right, the advantage of the current system is that the data will > > probably be > > found eventually, and the next time the request is made it will be > > available > > before it times out. But if it's hard to find, if we timeout as you > > suggest, > > it may not be found. > > I think the more-correct solution to that problem is to route less to > nodes which take so long, or choose them last. Which is to say, if a > certain node (or path to a node) is making us timeout, avoid it; > benefiting the general network health and responsiveness. Maybe. Maybe the key is in the wrong place on the network, the topology is perverse, but the request does eventually return the data. In which case it gets cached and propagated. > > > The only real reason I can think of to timeout as you do below would > > be for > > load balancing fairness: If you make a request, you had better be > > prepared > > (bandwidth wise) to accept the resulting data, if you're not, that's > > a denial > > of service attack. Load management relies on propagating the load > > back to the > > requestor. But the requestor can have only a very limited influence > > on timing > > out here so I don't consider it to be valid. > > Perhaps related to this is the furthestStoreSuccess stat, which will > usually near-instantly stick to ~0.5. Sometimes exactly 0.5... which > means that (wherever the request came from), my node was the absolute- > farthest node possible, last on his list to call, and all my peers > where closer. But weirdest of all, that the data was actually in the > store. More evidence (IMHO) for store-position-bias :) Intuitively, we > should be near the data the network sends us, and not search the whole > network for the data we want. More evidence that we have too much location churn, presumably caused by connection churn, node churn, and old perverse topologies. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20080108/d1e5a540/attachment.pgp>
