> FROM: Theodore Hong
> DATE: 04/19/2000 04:46:42
> SUBJECT: RE: [Freenet-dev] Proposal for the Near Future (Searching, CHKs and
> encryption, ..., oh my!)
>
>
> Indexing under KHK:CHK is a good thought. A big problem we`ve
> been having
> previously is how to eliminate bogus KHK->CHK pointers where the
> CHK data
> doesn`t have anything to do with the nominal KHK. Here, the
> pointers ARE
> the data -- constructed dynamically from the data in your store.
> If no one
> retrieves a particular KHK:CHK, the data dies and automatically
> takes down
> the "pointer" with it.
The cpu load of all these CHK generations is the only thing that may be a
problem if the CH algorithm is too complex. However, it doesn't need to be
overly complex since more uniqueness will be
provided by hashing it wil the KHK. Also, generating the hash once on insert
and then having sentry nodes verify the hash should ease some of the load.
>
> Some kind of unrequest or whatever is still required to
> deprecate the data
> if you were somehow tricked into retrieving it, but at least you
> don`t have
> to deprecate the pointers as well.
Agreed ... that is why the subject line is not "Proposal for the Near Future
(Unrequests, Searching, CHKs and encryption, ..., oh my!)" ;] There are several
other concepts that are needed but I
think that this proposal "plays nice" with them and the current implementation
of freenet.
>
> One difficulty: how can you index data under multiple KHK names,
> without
> inserting it multiple times?
Just insert metadata references that point to the KHK/CHK pair with the data
... if you want to be friendly to the freenet, that is. There is nothing
stopping people from inserting the same data
over and over under different KHKs (as is the case right now). These will be
retrievable via general searching and they will not block others from using
that same KHK. Since the "pure-metadata"
reference will have a CHK as well, it can be voted for with the current system
with a H(KHK:CHK) request. At this point the metadata is freeform and would be
solely up to the clients to generate
and interpret.
Key squatting would be a thing of the past since the data final indexing
wouldn't be up to the inserter but instead up to the data that is being
inserted and multiple entries under the same KHK are
allowed.
Road Map (to get their from here):
Client:
- introduce the CHK value in the message header of inserts (not strictly
necessary but distributes cpu load) and requests (won't break current node)
- introduce data header tacked onto the front of data inserts (won't break
current node)
- introduce "NO-CACHE" message header to avoid general search messages from
being cached along the request route (won't break current node)
Node:
- routing on just KHK while collisions are by H(KHK:CHK)
- modification of data store to generate (if not inserted by the client in the
original insert) and include CHK and index everything under H(KHK:CHK) (old
nodes and old clients will only get
metadata headers from old style requests... incentive to upgrade)
- interpret "NO-CACHE" (or request including no CHK) data request message
- as a general search
- interpret "found CHKs" included from downstream nodes
- data header extraction mechanism
- data header list compilation (reply mechanism could heavily use
current data reply code with just the addition of a message header that
indicates, "This data returned with some HTL left
... wait for the rest")
- HTL decremented on a "hits returned" rather than node hop basis
(hits/hops-to-live)
- place some reasonable limits on the size of the metadata to prevent abuse
(although this could be part of the client general request ... "max metadata
per KHK" value in request message header.
- requests including CHK are treated like requests are treated now
- "sentry nodes" that verify passed data CHKs to keep freenet clean of cancer
nodes and communication errors. if someone has a node that has heaps of CPU
they can turn on this option and it will
enable a chance for the node to verify the CHK of any given forwarded specific
data reply.
This looks incrementally doable.
Mike
_______________________________________________
Freenet-dev mailing list
Freenet-dev at lists.sourceforge.net
http://lists.sourceforge.net/mailman/listinfo/freenet-dev
