On Thu, Oct 20, 2005 at 07:46:46PM +0100, Matthew Toseland wrote: > On Thu, Oct 20, 2005 at 06:54:42PM +0100, Matthew Toseland wrote: > > Suppose: > > - We have a separate store for SSKs. They have 1kB of payload, plus the > > headers. > > - SSKs are versioned, with a timestamp. I.e. they become TUKs. Requests > > can optionally propagate far enough to fetch all values. There can be > > any number of SSKs with the same key and different version numbers / > > datestamps. Requests can ask for either a single specific version, the > > most recent version, or everything since a given revision. Only the > > data blocks which are actually returned are promoted in the cache. > > - To subscribe to a stream, I send a request out for the key I want, > > with the minimum acceptable version number/datestamp, plus three flags: > > - Passive request - If the data is not found, keep a tag on each node > > on the chain, and if it ever passes through any of the nodes on the > > chain, pass it back to me. > > - TUK - Return the most recent version of the SSK. > > - Subscribe - Requires the other two. Means the tag persists. > > Obviously it will be deleted if the originator rejects the data, or > > unsubscribes. > > > > Now, how would this actually work? > > > > Each node we are connected to can have up to 1000 subscriptions. Each > > subscription is for a key - this can be an SSK or a CHK - and if it is > > an SSK, it can have the "persist" flag. If it does not have this flag, > > or this flag is false, then when the data is found, the subscription is > > removed. If we no do not have enough subscription quota left for a given > > subscription, for that node, or for ourselves with another node, we > > reject the request attempting to add the subscription with a > > RejectOverload. Nodes can unsubscribe. Each subscription can have an > > expiry date (actually, perhaps we should impose one). > > > > Passive requests can only be created by actual requests with the passive > > request flag. The request will be routed as normal, until HTL runs out > > (or it finds the data if the TUK flag is not enabled). > > > > Now, how do we make this scale?: > > 1. If the TUK flag is off, and the request reaches a node which has the > > data, we return it, and the request ends. No passive requests are > > installed. > > 2. If any request for a TUK, even if not passive, reaches a node which > > already has a valid subscription at a higher or equal HTL for the key, > > then the request ends at that point, and returns the data that node has. > > If the passive-request flag is enabled, then passive request tags are > > added up to that node, and that node adds one for the node connecting to > > it if necessary. > > 3. What to do about looping and coalescing? One option is to not > > coalesce at all with running requests, and either forward the request > > as-is or reject it... more in my next email... > > As far as looping and coalescing goes: We can coalesce the requests > exactly as proposed for regular requests. So the request will not get > stuck by eating its own tail. As long as the network isn't totally > degenerate this shouldn't be a big problem. Also, if we do hit our own > tail, we could have a "lightweight subscribe" which isn't refcounted, > but passes packets both ways to strengthen the network. > > As far as resubscription and renewals go, there are two options really: > 1. Try to get the network to maintain the subscriptions itself; > automatic resubscription etc. > 2. Make the client resubscribe itself every so often. We would have to > let some of their requests through... > 3. Don't bother. Switch streams from time to time at the client level. > The problem with this is that we will still have to deal with situations > such as the parent of a node being disconnected.
Which we can do by simply telling our clients/dependants to resubscribe! -- Matthew J Toseland - toad at amphibian.dyndns.org Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: <https://emu.freenetproject.org/pipermail/tech/attachments/20051020/43eeb502/attachment.pgp>
