(There really should be a freenet-apps list. Maybe rename freenet-client?)
Applications based off Freenet will probably have some common design
patterns, and I've been thinking about one of them as a way of implementing
a filesystem-ish system over Freenet. You could for example use very
similar code to create a email based system on top Freenet (using
SMTP->Freenet and Freenet->IMAP apps), but with the unfortunate property
that you could read every single email sent to any person (as long as you
know their "email".
===========================================================
We assume an unlimited number of "filesystems", by tacking on an arbitrary
prefix to keys representing directory structure. In other words, I can
choose "/filesystems/areallylongprefix376/" as a prefix, and then when I
read the file "/directory/dir2/" I use the key
"/filesystems/areallylongprefix376/directory/dir2/" as the Freenet KHK for
the file.
The problem with storing directory information in a Freenet doc is that we
cannot Update it every time we change the contents of the directory.
Consider -
1. Alice loads directory listing, adds a file, and updates the directory
listing with her file added.
2. Bob does the same at the same time.
If Bob loads the directory listing before Alice updated it, after Bob's
update Alice's new file will not be in the directory listing.
Therefore, instead of using (unimplemented) update mechanism, we use Inserts
and Updates. Let's say we have a directory '/dir1' - in this directory we
will have two special files, '.listing' and '.approxrevision'.
'.listing' is not in fact a single file. Instead, we first create
'.listing0', and then every time we change the contents of the directory we
insert a new '.listing' file with an added 1 to it's old number. So, we
have '.listingN' where N = 0, 1, 2, 3, 4, ....
'.approxrevision' contains the approximate revision number that '.listingN'
has reached. Since we usually only want the latest revision of '.listing',
we don't want to start checking if 'listiing0' exists, then '.listing1' etc.
till we reach the latest one, especially since older ones might have
expired. Instead, we start searching from the number listed in
'.approxrevision', which we should Update every 10 or so revisions of
'.listing'.
Why is storing the directory contents in numbered "files" a good idea?
Consider our original scenario - Bob loads directory listing - say,
'.listing31', Alice does the same, adds new file and inserts an updated
'.listing32'.
Now, Bob adds a file, and tries to insert '.listing32'. Since it already
exists, the Freenet *returns it to him* - and now he sees that, refreshes
his own directory listing to match the newer version, and then inserts
'.listing33' that also contains the changes Alice made.
Now that we know how to store the data, we have to decide what to store.
The simplest info we can store is a list of filenames, and then the key for
the filenames can be matched to a Freenet doc by appending the filename to
the "directory path" - the directory's key with a '/' tacked on. So if the
directory is '/dir/a', the file is 'readme.txt' and the prefix is
'/fs/ac432' then the directory's "key" is '/fs/ac432/dir/a (and it's actual
info is in '/fs/ac432/dir/a/.listingN' and
'/fs/ac432/dir/a/.approxrevision') and the file's key is
'/fs/ac432/dir/a/readme.txt'.
In fact, we should not only store the list of files (and whatever metadata
we fell necessary - e.g. we decide we want a way to differentiate between
files and directories) but also the list of changes made between each
revision. That is, '.listing3' might say (in a more terse format, of
course):
Files in this directory:
readme1.txt
Changes:
Revision 1 - added file test.txt
Revision 2 - deleted file test.txt
Revision 3 - added file readme1.txt
Every 50 or so revisions we can drop the changelog for the previous 50
revisions, so for example '.listing100' will only list changes from 99 to
100, and if we need them the changes from 50 -99 are im '.listing99' and
from revision 0 - 49 in '.listing49'.
Why is this changelog necessary? Since we have a few people working at once
on a directory, and it might take different amount of time for their changes
to propagate, we might have different versions of the same revision N
'.listingN' on the Freenet at the same time, etc.. This way, it's much
easier to resynchronize so that everyone has the same version of the
filesystem.
--
Itamar S.T. itamar at maxnm.com
_______________________________________________
Freenet-dev mailing list
Freenet-dev at lists.sourceforge.net
http://lists.sourceforge.net/mailman/listinfo/freenet-dev
