Suppose:
- We have a separate store for SSKs. They have 1kB of payload, plus the
headers.
- SSKs are versioned, with a timestamp. I.e. they become TUKs. Requests
can optionally propagate far enough to fetch all values. There can be
any number of SSKs with the same key and different version numbers /
datestamps. Requests can ask for either a single specific version, the
most recent version, or everything since a given revision. Only the
data blocks which are actually returned are promoted in the cache.
- To subscribe to a stream, I send a request out for the key I want,
with the minimum acceptable version number/datestamp, plus three flags:
- Passive request - If the data is not found, keep a tag on each node
on the chain, and if it ever passes through any of the nodes on the
chain, pass it back to me.
- TUK - Return the most recent version of the SSK.
- Subscribe - Requires the other two. Means the tag persists.
Obviously it will be deleted if the originator rejects the data, or
unsubscribes.
Now, how would this actually work?
Each node we are connected to can have up to 1000 subscriptions. Each
subscription is for a key - this can be an SSK or a CHK - and if it is
an SSK, it can have the "persist" flag. If it does not have this flag,
or this flag is false, then when the data is found, the subscription is
removed. If we no do not have enough subscription quota left for a given
subscription, for that node, or for ourselves with another node, we
reject the request attempting to add the subscription with a
RejectOverload. Nodes can unsubscribe. Each subscription can have an
expiry date (actually, perhaps we should impose one).
Passive requests can only be created by actual requests with the passive
request flag. The request will be routed as normal, until HTL runs out
(or it finds the data if the TUK flag is not enabled).
Now, how do we make this scale?:
1. If the TUK flag is off, and the request reaches a node which has the
data, we return it, and the request ends. No passive requests are
installed.
2. If any request for a TUK, even if not passive, reaches a node which
already has a valid subscription at a higher or equal HTL for the key,
then the request ends at that point, and returns the data that node has.
If the passive-request flag is enabled, then passive request tags are
added up to that node, and that node adds one for the node connecting to
it if necessary.
3. What to do about looping and coalescing? One option is to not
coalesce at all with running requests, and either forward the request
as-is or reject it... more in my next email...
--
Matthew J Toseland - toad at amphibian.dyndns.org
Freenet Project Official Codemonkey - http://freenetproject.org/
ICTHUS - Nothing is impossible. Our Boss says so.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL:
<https://emu.freenetproject.org/pipermail/tech/attachments/20051020/2b573701/attachment.pgp>
