Here's an interesting issue with inserts I ran into recently: Currently, we have a 2 minute timeout on inserts, after the Accepted has been received and we have sent the DataInsert and started the transfer. We are waiting for an InsertReply, a RouteNotFound etc. Even nodes which return RouteNotFound will get the data, so we may well end up sending it to several nodes.
Because we need to be able to detect downstream overload, we need to detect when a transfer has failed (this is usually due to overload). So at present, we wait for all transfers to complete before relaying the status. Unfortunately, this means the search phase of the insert can be very slow, because we go to one node, get an RNF, wait for the transfers to complete, return RNF, go to next node, start transfer, wait for transfer to complete, RNF... and we go over the 2 minute time limit. Solution: (simplification: A inserting locally, only knows B) A -> B: InsertRequest B -> A: Accepted A -> B: DataInsert /// searches - 2 minute time limit /// B -> A: RouteNotFound/InsertReply/RejectedOverload/etc /// wait for transfers - 4?? minute time limit /// B -> A: InsertTransfersComplete/InsertTransfersFailed So we do the search as fast as possible, and then wait for the transfers to complete, and for downstream nodes to tell us that *their* transfers have completed. We then propagate the status to our parent node. If all the nodes we contacted returned InsertTransfersComplete, we return InsertTransfersComplete. If any returned InsertTransfersFailed, we return InsertTransfersFailed. If some don't return it at all, we return InsertTransfersFailed after timing out. If we originated the insert, then if we get a timeout or an InsertTransfersFailed, we treat this the same way as getting a RejectedOverload, i.e. we reduce our insert send rate. Note that InsertTransfersFailed does *not* trigger any sanctions against the node (i.e. backoff), unlike a timeout, or a RejectedOverload with the local flag set. It is purely to ensure that we are told if the network is so overloaded that a transfer times out, or doesn't complete in a reasonable time, while still having a fast search, and not unnecessarily serializing transfers. -- Matthew J Toseland - toad at amphibian.dyndns.org Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: <https://emu.freenetproject.org/pipermail/tech/attachments/20051203/a973c5b6/attachment.pgp>
