On Fri, 2003-03-28 at 23:20, Matthew Toseland wrote: > On Fri, Mar 28, 2003 at 10:10:55PM -0600, Edgar Friendly wrote: > > Ian Clarke <ian at locut.us> writes: > > > > > We need to get probabilistic caching in the code ASAP - the current > > > level of load most nodes are getting is totally disrupting the > > > clustering effect from what I have seen in my node's diagnostics. > > > Freenet simply won't work properly without this clustering. > > > > > If there's a problem that's caused by nodes not being able to handle > > the load put on them, it seems logical to me to work on making the > > nodes more efficient, so they're able to handle the increased load. > > (at least until bandwidth is the limiting factor, which I'm pretty > > sure it's not.) > > > > > I propose that we have a fall-off parameter which adjusts how the cache > > > probabilty "falls off" as the reply gets further from its source. This > > > can be configurable. We can put this functionality in unstable, and > > > encourage people to experiment with different parameters. We should see > > > increased clustering on those nodes irrespective of whether the rest of > > > the network contains that code. > > > > > Disk space is cheap, I don't see much reason for nodes not to cache > > everything they can get their hands on. If they're being asked for > > the data, it does no good for them to forward the request somewhere > > else because at one time they thought that they weren't responsible > > for that data. > > You are assuming that all nodes have infinite storage. This is not the > case. Most nodes have full datastores. This means that caching one item > means throwing another one out.
What about just enhancing the algorithm that deletes content out of a DataStore so that it improves specialization? Maybe LRU is not the best way to determine what content gets deleted first. It tends to break apart specialization for widely-requested content. Another way of doing it might be by storing request keys, and deleting keys out of the data store that are not near close groups of request keys. > > > > > I propose making this the highest priority at this time. > > > > > > Ian. > > > > > I see the biggest improvement that can be made is NIO, with bugfixes > > and usability improvements coming second. Of course I could be wrong. > > > > Thelema > > -- > > E-mail: thelema at swbell.net Raabu and Piisu > > GPG 1024D/36352AAB fpr:756D F615 B4F3 BFFC 02C7 84B7 D8D7 6ECE 3635 2AAB > -- > Matthew Toseland > toad at amphibian.dyndns.org/amphibian at users.sourceforge.net > Full time freenet hacker. > http://freenetproject.org/ > Freenet Distribution Node (temporary) at > ICTHUS. _______________________________________________ devl mailing list devl at freenetproject.org http://hawk.freenetproject.org:8080/cgi-bin/mailman/listinfo/devl
