On Dec 8, 2007, at 7:14 AM, Matthew Toseland wrote: > On Friday 07 December 2007 22:47, Robert Hailey wrote: >> * The contents of the datastore(s) are not given weight when >> considering a swap request. > > Indeed. This is entirely deliberate. We have no simulations of what > would > happen if we did bias by datastore contents, and we are not going to > implement unsimulated changes to core routing, certainly not to > swapping.
Yeah, it seems like the swapping/routing is presently the best it has been thus far; but I do think the 0% store hit the Ed mentions may be a factor in freenet's slowness. I see that in my nodes as well. > [...] >> Consider the following patch (completed to the furthest point that my >> current freenet understand can take it). >> - If a CHKBlock is evicted from the datastore (not cache), AND we are >> not (as best we can tell) "close" to where it should be, THEN re- >> insert that block. > > This is going to generate an insane amount of traffic, and sabotage > the > natural decay of data that happens to be in the wrong place. In the present network, it probably would; but in theory I think that the patch is correct (or some variant thereto). The problem is... it presumes that the network location is stable. Now, if a nodes location in the network was presently "stable-enough", this entire issue would not be brought up. Whenever a request would be made, the same node that would receive it as in insert would respond from the datastore, even for potentially really-old data, as there is a lot of storage available on the network. In fact, if the network was as rock-solid, this patch would amount to a no-op, as we are close to anything that we put in our store, then we are close to anything that we evict and the 'closer' condition would always be false. In the worst case, if two nodes have data stores exactly specialized for there present location, and they swap locations, then they would ~eventually~ (one block at at time) swap data stores too. At the end of which, they would be in the same LRU-order too. What's more, in the case where a new node is introduced to a stable network, any errant inserts it stores while it roams the network for it's location would be saved. I imagine that insert traffic is a small percentage of total traffic at large, and the secondary insert could be tuned differently than a standard insert (probably in HTL). As presented, this patch would yield an increasing rate of transfer as more inserts come in (throttled by the thread limit). I suppose the real issue remains network-location being incompatible with store values. >> Also, I could not figure out how to revive a SSKBlock from the >> datastore (or find out if it is even possible). > > An SSK requires the pubkey from one datastore and the actual SSK > from another. So implementing SSK healing-at-eviction-time would require several extra database hits and an extra store-block-read, but would be possible in theory. > Not true. We have mechanisms to get popular data back into the > datastore. The > main one is 1 in every 100 successful requests gets turned into an > insert > post-completion. That is intriguing. I wonder how much of my 0.3% store-hit-successes (my present value) are actually blocks recently-inserted from the 1% healing (i.e. bleed-over from the cache). -- Robert Hailey -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20071210/657b66fd/attachment.html>
