On Dec 8, 2007, at 7:14 AM, Matthew Toseland wrote:
On Friday 07 December 2007 22:47, Robert Hailey wrote:
* The contents of the datastore(s) are not given weight when
considering a swap request.
Indeed. This is entirely deliberate. We have no simulations of what
would
happen if we did bias by datastore contents, and we are not going to
implement unsimulated changes to core routing, certainly not to
swapping.
Yeah, it seems like the swapping/routing is presently the best it has
been thus far; but I do think the 0% store hit the Ed mentions may be
a factor in freenet's slowness. I see that in my nodes as well.
[...]
Consider the following patch (completed to the furthest point that my
current freenet understand can take it).
- If a CHKBlock is evicted from the datastore (not cache), AND we are
not (as best we can tell) "close" to where it should be, THEN re-
insert that block.
This is going to generate an insane amount of traffic, and sabotage
the
natural decay of data that happens to be in the wrong place.
In the present network, it probably would; but in theory I think that
the patch is correct (or some variant thereto).
The problem is... it presumes that the network location is stable.
Now, if a nodes location in the network was presently "stable-enough",
this entire issue would not be brought up. Whenever a request would be
made, the same node that would receive it as in insert would respond
from the datastore, even for potentially really-old data, as there is
a lot of storage available on the network.
In fact, if the network was as rock-solid, this patch would amount to
a no-op, as we are close to anything that we put in our store, then we
are close to anything that we evict and the 'closer' condition would
always be false.
In the worst case, if two nodes have data stores exactly specialized
for there present location, and they swap locations, then they would
~eventually~ (one block at at time) swap data stores too. At the end
of which, they would be in the same LRU-order too.
What's more, in the case where a new node is introduced to a stable
network, any errant inserts it stores while it roams the network for
it's location would be saved.
I imagine that insert traffic is a small percentage of total traffic
at large, and the secondary insert could be tuned differently than a
standard insert (probably in HTL). As presented, this patch would
yield an increasing rate of transfer as more inserts come in
(throttled by the thread limit).
I suppose the real issue remains network-location being incompatible
with store values.
Also, I could not figure out how to revive a SSKBlock from the
datastore (or find out if it is even possible).
An SSK requires the pubkey from one datastore and the actual SSK
from another.
So implementing SSK healing-at-eviction-time would require several
extra database hits and an extra store-block-read, but would be
possible in theory.
Not true. We have mechanisms to get popular data back into the
datastore. The
main one is 1 in every 100 successful requests gets turned into an
insert
post-completion.
That is intriguing. I wonder how much of my 0.3% store-hit-successes
(my present value) are actually blocks recently-inserted from the 1%
healing (i.e. bleed-over from the cache).
--
Robert Hailey
_______________________________________________
Devl mailing list
[email protected]
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
