On Sunday 30 January 2011 03:20:24 Robert Hailey wrote: > > On 2011/01/26 (Jan), at 1:15 PM, Matthew Toseland wrote: > > > The following is relevant to Robert's thread. It's close to what he > > was talking about. TheSeeker is the other big contributor. > > > > On a node which does a lot of downloads, the peers approach a flat > > circle: They are more or less evenly distributed. > > > > On a node which just routes we get a very strong small world effect. > > Wow, that makes sense. I think that you have found the direct cause, > > > PRO: > > - Routing is no longer disproportionately optimised for local > > requests. Hopefully higher overall performance. > > - Prevents the trivial opennet path-folding-originator-sampling > > attack. > > > > CON: > > - No direct theoretical backing at present. We need to talk to a > > theoretician. > > - We should not deploy this while there are other big network things > > happening. > > Theoretical backing? > > Facts... > > (1) Path folding presently operates across local & remote requests > > Axioms... > > (1) local requests tend to be evenly distributed, > (2) remote requests tend to be highly specialized, > (3) some nodes have disproportionately high local requests (downloaders) > (4) some nodes have disproportionately high remote requests (browsers)
"browsers" ??? Confusing terminology. IMHO "downloaders" vs "routers" is far less confusing. Downloaders actually use Freenet. Routers largely run their nodes for the benefit of others. > > I would expect... > > (5) downloaders to have a routable view of the network, No. Downloaders will have a highly un-specialised view of the network, to the point that routing the last-few-hops is impossible. They don't have the nearest-nodes-on-each-side that the Kleinberg simulations assume: They can only make big jumps, not small jumps, even for keys close to their specialisation. Therefore they are a big problem for routing. We are approaching this from opposite angles. > (6) browsers will have a routable view of there local network & > *might* over specialize IMHO over-specialisation is a red herring. > (7) when browsers "start browsing again" the node might quickly latch > onto the nearest downloader b/c it can satisfy the requests Now your confusing terminology is coming home to roost. :) When routing-only nodes start downloading, their connections will tend to broaden. > (8) when browsers go idle, any long connections or connections to > downloaders will start dropping off Right, because they are then routing the incoming requests, which are highly specialised. Which is not a problem - not ALL of their incoming requests are highly specialised, just the vast majority. If all of them were highly specialised we wouldn't receive anything, because it would never reach us. > > The FOAF routing scheme probably makes the #7 & #8 behaviour much more > real. The FOAF scheme was proposed in a paper applying to greedy-routed networks afaik. > > I recall the whole 'run a spider and freenet starts working' argument > and now wonder if it was more routing than caching. Well, on opennet, we only path fold when we have successful requests, so maybe there's an element of that ... > > As an aside, my understanding is that opennet routing would > theoretically work even if every node simply held onto it's two > nearest connections (left & right; neglecting load concerns entirely > for the moment); this is the circular klienberg model which the > simulators use. Right. But over-unspecialising *eliminates these side links* and therefore routing degenerates into close to random routing. Especially given we don't back out when we go too far away from the ideal. > > In my former investigation (which I appreciate you bringing back up > for its relevance), it seemed like my 'browser' nodes were becoming > way too specialized; e.g. 5-7 fold redundant links to nearby nodes. It > was this 'downloader' / 'browser' role separation is what I was > refering to when I said the network might be behaving like a scale > free network (maybe wrong terminology). Definitely wrong terminology, scale free refers to the distribution of link counts, not of link lengths. The correct terminology is that the "routers" are specialised (adequately so provided they have a range of link lengths so can get external requests out reasonably quickly), and the "downloaders" are broken. An interesting security side-effect is that on a well specialised node almost all local requests will go to our one or two furthest links. > > My original thought on improving this situation (if it does have such > a negative effect), was to only accept a path folding request for > local requests (i.e. eliminate the causing fact #1). I seriously doubt that the problem is over-specialisation. Specialisation is GOOD. I would require evidence, and probably simulations. > This would surely > cut down on the number of path folds accepted by browser-nodes, but > they would be 'healthier'.... or at least become the same as the > 'downloaders' path folding, so the network would be more homogenous > (if nothing else). It would be interesting to hear theories and > opinions on this change. More homogenous is bad. Like you said above, we need the short side-links that the Kleinberg models just put in and assume exist. We need specialisation!! > > > Should we only start path folding once HTL drops below the caching > > threshold? Security-wise the case is pretty interesting: We don't write to caches while we are at high HTLs (although we do check them); we are only likely to want the data again if it is very popular; arguably encrypting it (if useful tunnels could be constructed without significantly increasing hops used, which is an open question) would have very low performance impact; and path folding directly from the request originator is very dangerous. > > I don't totally understand the implications of that, but it sounds > like you're looking for a solution on the origination side of path > folding, and creating a new tunable. I would consider modifications on > the acceptance side of path folding first, and nail down what it means > (in code) to 'want' a new opennet peer enough to grab a path folding > request. We have lots of heuristics, but they are all designed to avoid the big theoretical issues i.e. to avoid things like keyspace biases. So they are basically a set of churn limits. IMHO anything that could have serious theoretical implications MUST be simulated and tested extensively. That includes the proposal to not path fold until we are out of the no-caching area. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20110203/5b163e7d/attachment.pgp>
