An important issue that has not been discussed yet is scalability. I
just realized when I was hacking on nfreenetd that datastore formats
(especially datastore index formats) are actually hard to get right.
It is easy to make such an index which *works*, but it is quite hard
to make it *scalable*. Several considerations are:
1. How do you efficiently search indices (remember that datastore
indices are like the flat filesystems on old bitty boxen)?
2. How do you efficiently add items to datastore indices (you can't
go and write an entire half gigabyte index to file each time you
add a single file (yes, this is an extreme example, but it *will*
happen))?
3. How do you efficiently look for files to displace from the
datastore when you add new files (yeah, like the other stuff in
this list, it doesn't matter when you have a few hundred files in
the datastore, but it soon becomes very significant when you have
half a million files)?
4. How do you break up files in the datastore and the datastore
indices to get around file size limits imposed by filesystems?
5. Binary formats make datastores *easier* to work with (you don't
have to rewrite the entire datastore each time you change the
status of a file in the datastore), but you have to watch out for
stuff like endianness.
All of these issues have not been addressed yet, and they will be
significant problems once Freenet reaches primetime. I'm planning on
not procrastinating on solving these problems, but instead solving
them from the first version of nfreenetd. All of this stuff makes
writing a Freenet node daemon far more complex and difficult than it
appears at face value.
--
Travis Bemann
Sendmail is still screwed up on my box.
My email address is really bemann at execpc.com.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 1654 bytes
Desc: not available
URL:
<https://emu.freenetproject.org/pipermail/devl/attachments/20000803/1352a79d/attachment.pgp>
