On Tue, Jun 29, 2010 at 11:45 AM, Robert Hailey <robert at freenetproject.org> wrote: > > On Jun 29, 2010, at 8:26 AM, Matthew Toseland wrote: > >> Tests showed ages ago that triple inserting the same block gives 90%+ >> persistence after a week instead of 70%+. There is a (probably statistically >> insignificant) improvement relative even to inserting 3 separate blocks. >> I had thought it was due to not forking on cacheable, but we fork on >> cacheable now and the numbers are the same. >> [...] >> >> With backoff: >> >> IMHO rejections are more likely to be the culprit. There just isn't that >> much backoff any more. However, we could allow an insert to be routed to a >> backed off peer provided the backoff-time-remaining is under some arbitrary >> threshold. >> >> Now, can we test these proposals? Yes. >> >> We need a new MHK tester to get more data, and determine whether triple >> insertion still helps a lot. IMHO there is no obvious reason why it would >> have degenerated. We need to insert and request a larger number of blocks >> (rather than 3+1 per day), and we need to test with fork on cacheable vs >> without it. We should probably also use a 2 week period rather than a 1 week >> period, to get more detailed numbers. However, we can add two more >> per-insert flags which we could test: >> - Ignore low backoff: If enabled, route inserts to nodes with backoff time >> remaining under some threshold. This is easy to implement. >> - Prefer inserts: If enabled, target a 1/1/3/3 ratio rather than a 1/1/1/1 >> ratio. To implement this using the current kludge, we would need to deduct >> the space used by 2 inserts of each type from the space used, when we are >> considering whether to accept an insert. However IMHO the current kludge >> probably doesn't work very well. It would likely be better to change it as >> above, then we could just have a different target ratio. But for testing >> purposes we could reasonably just try the kludge. >> >> Of course, the real solution is probably to rework load management so we >> don't misroute, or misroute much less (especially on inserts). > > About persistence... it logically must be confined to these areas. > > 1) insertion logic > 2) network change over time > 3) fetch logic > > If there is a major issue with 2 or 3, then beefing up 1 may not be a "good" > solution. Then again, I like your ideas more than just chalking it up to > "bad network topology"...
Bad topology is not confined to those areas. The insert / fetch logic can be locally correct, and the network static, and bad topology will still produce poor performance. Evan Daniel
