On Tue, Jun 29, 2010 at 5:15 PM, Matthew Toseland <toad at amphibian.dyndns.org> wrote: > On Tuesday 29 June 2010 16:54:42 Evan Daniel wrote: >> On Tue, Jun 29, 2010 at 11:45 AM, Robert Hailey >> <robert at freenetproject.org> wrote: >> > >> > On Jun 29, 2010, at 8:26 AM, Matthew Toseland wrote: >> > >> >> Tests showed ages ago that triple inserting the same block gives 90%+ >> >> persistence after a week instead of 70%+. There is a (probably >> >> statistically >> >> insignificant) improvement relative even to inserting 3 separate blocks. >> >> I had thought it was due to not forking on cacheable, but we fork on >> >> cacheable now and the numbers are the same. >> >> [...] >> >> >> >> With backoff: >> >> >> >> IMHO rejections are more likely to be the culprit. There just isn't that >> >> much backoff any more. However, we could allow an insert to be routed to a >> >> backed off peer provided the backoff-time-remaining is under some >> >> arbitrary >> >> threshold. >> >> >> >> Now, can we test these proposals? Yes. >> >> >> >> We need a new MHK tester to get more data, and determine whether triple >> >> insertion still helps a lot. IMHO there is no obvious reason why it would >> >> have degenerated. We need to insert and request a larger number of blocks >> >> (rather than 3+1 per day), and we need to test with fork on cacheable vs >> >> without it. We should probably also use a 2 week period rather than a 1 >> >> week >> >> period, to get more detailed numbers. However, we can add two more >> >> per-insert flags which we could test: >> >> - Ignore low backoff: If enabled, route inserts to nodes with backoff time >> >> remaining under some threshold. This is easy to implement. >> >> - Prefer inserts: If enabled, target a 1/1/3/3 ratio rather than a 1/1/1/1 >> >> ratio. To implement this using the current kludge, we would need to deduct >> >> the space used by 2 inserts of each type from the space used, when we are >> >> considering whether to accept an insert. However IMHO the current kludge >> >> probably doesn't work very well. It would likely be better to change it as >> >> above, then we could just have a different target ratio. But for testing >> >> purposes we could reasonably just try the kludge. >> >> >> >> Of course, the real solution is probably to rework load management so we >> >> don't misroute, or misroute much less (especially on inserts). >> > >> > About persistence... it logically must be confined to these areas. >> > >> > 1) insertion logic >> > 2) network change over time >> > 3) fetch logic >> > >> > If there is a major issue with 2 or 3, then beefing up 1 may not be a >> > "good" >> > solution. Then again, I like your ideas more than just chalking it up to >> > "bad network topology"... >> >> Bad topology is not confined to those areas. ?The insert / fetch logic >> can be locally correct, and the network static, and bad topology will >> still produce poor performance. > > True but opennet should produce good topology shouldn't it? Generally the > stats page seems to suggest routing is working? >
True in theory. Stats page suggests routing basically works, and is not inconsistent with good overall topology. I have enough data from probe requests to do serious topology analysis, but have not yet done so. At this point I would say that the topology is assumed to be good, but that we aren't completely certain. Evan Daniel
