On Thursday 14 October 2010 18:41:50 Robert Hailey wrote: > > It routinely surprises me at how I can fetch a freesite one day, and > the next I have to wait for it to refetch the whole site again.
Are you sure it is the same version of the same freesite? > Particularly knowing how much of the freenet code base is dedicated to > caches! How big is your client-cache? Is it in-ram-only? > > Concerning remote fetches, I remember that Matthew said something to > the effect of: "general network performance is related to how easily > the data is found" (b/c then *much* network traffic is trying to get > to the 2-3 nodes which have the data we need). Unpopular data will only be on 3 or so nodes yes, but in theory those nodes should be "the right" nodes. Unfortunately because of load management sometimes those 3 nodes will all reject the insert. This appears to be why if we insert the same key 3 times it lasts for *vastly* longer, even though it goes to the same series of nodes each time. > > I've been a out-of-the loop for a while, so if something here is now > technically wrong please correct it... > > Theory: > ? local/remote CHK fetches should be nearly 100% (mine are at 74% & 9%) Only if no data ever fell out of the network. But they are pretty good here. Look at the breakdown by HTL. It's true that psuccess on average overall is relatively low, but at HTL 18 or 16 it is very high. > ? most node's datastores "are not filling up" (long-running, my is at > 40% of 50GB) Do you have any evidence of this assertion? The code will cache everything that goes through the node, except for stuff which is requested at HTL over 16 or inserted at HTL over 15. This is for privacy reasons. > ? a node's "cache" should: > - serve requests of the node (with the view to avoid remote requests) The client cache caches locally requested stuff. The cache/store does not, as described above. > - not reveal what the node "views" (i.e. don't service remotes from > the cache if far from network location) The cache/store serve *anything that is in them*. This is necessary for slashdot resistance e.g. if a file becomes really popular you don't want to have to go to the "ideal" node location-wise. > - serve requests of the network (e.g. to move displaced data back > into the correct "store") > > As best I can reason, either some part of the node's cache (as > described above) is *broken*, or we need to make inserts effect *many- > more* data stores. > > One idea I had was to rather than depositing the datum in the 2-3 > "closest" nodes, was to deposit the datum in every node within > "epsilon" of the target address (e.g. try to store it in an adjustable > pie-wedge sliver, than a constant number of nodes). It must be a constant or the number of nodes touched will get huge and it will not scale. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20101015/86dd7b43/attachment.pgp>
