On Thu, Oct 20, 2005 at 07:56:56PM +0100, Matthew Toseland wrote: > On Thu, Oct 20, 2005 at 07:46:46PM +0100, Matthew Toseland wrote: > > On Thu, Oct 20, 2005 at 06:54:42PM +0100, Matthew Toseland wrote: > > > Suppose: > > > - We have a separate store for SSKs. They have 1kB of payload, plus the > > > headers. > > > - SSKs are versioned, with a timestamp. I.e. they become TUKs. Requests > > > can optionally propagate far enough to fetch all values. There can be > > > any number of SSKs with the same key and different version numbers / > > > datestamps. Requests can ask for either a single specific version, the > > > most recent version, or everything since a given revision. Only the > > > data blocks which are actually returned are promoted in the cache. > > > - To subscribe to a stream, I send a request out for the key I want, > > > with the minimum acceptable version number/datestamp, plus three flags: > > > - Passive request - If the data is not found, keep a tag on each node > > > on the chain, and if it ever passes through any of the nodes on the > > > chain, pass it back to me. > > > - TUK - Return the most recent version of the SSK. > > > - Subscribe - Requires the other two. Means the tag persists. > > > Obviously it will be deleted if the originator rejects the data, or > > > unsubscribes. > > > > > > Now, how would this actually work? > > > > > > Each node we are connected to can have up to 1000 subscriptions. Each > > > subscription is for a key - this can be an SSK or a CHK - and if it is > > > an SSK, it can have the "persist" flag. If it does not have this flag, > > > or this flag is false, then when the data is found, the subscription is > > > removed. If we no do not have enough subscription quota left for a given > > > subscription, for that node, or for ourselves with another node, we > > > reject the request attempting to add the subscription with a > > > RejectOverload. Nodes can unsubscribe. Each subscription can have an > > > expiry date (actually, perhaps we should impose one). > > > > > > Passive requests can only be created by actual requests with the passive > > > request flag. The request will be routed as normal, until HTL runs out > > > (or it finds the data if the TUK flag is not enabled). > > > > > > Now, how do we make this scale?: > > > 1. If the TUK flag is off, and the request reaches a node which has the > > > data, we return it, and the request ends. No passive requests are > > > installed. > > > 2. If any request for a TUK, even if not passive, reaches a node which > > > already has a valid subscription at a higher or equal HTL for the key, > > > then the request ends at that point, and returns the data that node has. > > > If the passive-request flag is enabled, then passive request tags are > > > added up to that node, and that node adds one for the node connecting to > > > it if necessary. > > > 3. What to do about looping and coalescing? One option is to not > > > coalesce at all with running requests, and either forward the request > > > as-is or reject it... more in my next email... > > > > As far as looping and coalescing goes: We can coalesce the requests > > exactly as proposed for regular requests. So the request will not get > > stuck by eating its own tail. As long as the network isn't totally > > degenerate this shouldn't be a big problem. Also, if we do hit our own > > tail, we could have a "lightweight subscribe" which isn't refcounted, > > but passes packets both ways to strengthen the network. > > > > As far as resubscription and renewals go, there are two options really: > > 1. Try to get the network to maintain the subscriptions itself; > > automatic resubscription etc. > > 2. Make the client resubscribe itself every so often. We would have to > > let some of their requests through... > > 3. Don't bother. Switch streams from time to time at the client level. > > The problem with this is that we will still have to deal with situations > > such as the parent of a node being disconnected. > > Which we can do by simply telling our clients/dependants to resubscribe!
So the choice is this: either a) When something breaks, everyone has to resubscribe, or b) When something breaks, we tell all dependant nodes to let our requests through, and resubscribe ourself. Either way there has to be a concept of "tree" - the node which we expect to get messages from (the node we routed to). We don't have to adhere strictly to this in passing data packets onwards, but it does have to exist. Therefore, we will have a concept of "root", and of "root location". So a certain amount of what has been planned has to be kept. A lot of it, in fact, but the data storage is different, and we can have very many streams. So: If we lose our parent: - Simple way: We forcibly unsubscribe all our dependants. Their clients will resubscribe. - Hard way: We send a SubscribeRestarted to all our dependants. Then we send a request out, similar to the one we have previously received, seeking the key. It has a key-must-be-better-than property, as do regular requests (they use it in combination with HTL); this is unset, or rather it is set far away from the target. -- Matthew J Toseland - toad at amphibian.dyndns.org Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: <https://emu.freenetproject.org/pipermail/tech/attachments/20051020/330e00b6/attachment.pgp>
