Hello, Here is some try from Sich and myself to think about the design for some insert on demand application based on the WoT plugin.
----------------------------------------------------------------------------------------------- key structure : ssk/filesharing/updates/inserts/x/inserts ssk/filesharing/updates/requests/y/requests ssk/filesharing/z/index/ ssk/filesharing/filehash/n/index inserts file structure: <filehash>xxx</filehash> <chunk>all or chunk number</chunk> requests file structure: <filehash>xxx</filehash> <chunk>all or chunk number</chunk> index file structure: <file> <name>xxx</name> <size>xxx</size> <hash>xxx</hash> <content type>xxx</content type> </file> filehash index structure: <comment>comment on this file</comment> #one per chunk <chunk> <hash>xxx</hash> <ssk>xxx</ssk> #if available <chunk number>xxx</chunk number> #necessary if all the chunks aren't listed </chunk> Each file is splited into several chunks. Each chunks is inserted in is own SSK. We suscribe to every keys needed to know about updates (that is : ssk/filesharing/upadtes/inserts and ssk/filesharing/upadtes/requests) Search : Each people publish all the files that he is sharing in the ssk/filesharing/index file. When you search a file, you will look in each identity to find your file. Then you have a list of filename and corresponding filehash. When you have this you can choose to download a specific filehash. #Each people who have this file can begin to insert some chunk of the file and telling that they are currently inserting a part of the file. The chunk are randomly choose, then multiple people can insert #the whole file more faster. #When the chunk is inserted the SSK key of the chunk is published, then we can begin to download it. Download: Once you found the file you want, you search for shared chunks in the keys of the identities sharing the file. If several ssk are available for one chunk, choose the one that appears the more. (during the search, when you see sskY for chunkXXX, just add 1 to the priority of sskY) If no ssk is available for one chunk, add a request in ssk/filesharing/requests We suscribe to the key : ssk/filesharing/filehash/ to know about all new chunks available, ... Insertion: If someone is requesting a chunk (see Download), we start inserting it. We publish that in the key under : ssk/filesharing/inserts, so other identities won't insert it. When it's finished, we indicate it under the same file. Healing : When you try to download a file who was already inserted, and if you can't download a specific chunk, a request is publish to ask for this chunk. People who have the original file can begin to insert this chunk and tell the other that they have begin to insert it in order to avoid multiple insert of the same chunk. Then the new SSK is published to download the missing chunk. WoT: If someone give us a wrong chunk or some fake, we mark it as bad. One question however : how can we detect a wrong chunk ? If we have multiple source for the file we can try to compare the chunk filehash index (ssk/filesharing/filehash/index). advantage to split the file : - We can have multiple source to insert the whole file - If some chunk are no more available we can only ask to reinsert this chunk. This will limit the datastore use (no need to reinsert the whole file on a new SSK) - We can begin to download the file before all the chunks are inserted. This is a little more faster for the people who are downloading. - Using some preview system ? WoT-based : - Using WOT can limit the problem for the bootstrap when you have just install the filesharing program. - Using the security of the WOT can help to find people who are publishing fake files. -------------------------------------------------------------------------------------------------------- Please don't hesitate to comment. Are we missing some points, is that feasable, will it be too slow, etc...
