On Thu, Jan 23, 2003 at 11:31:25AM +0000, Gordan Bobic wrote: > Hi. > > Is there such a thing as chunk size in the way Freenet deals with storing > and transfering the data? Freenet key contents are normally some fields plus a power of two size data. Minimum 1kB, I believe. > > By chunk size, I mean a minimal unit of data that is worked with. For > example, for disk access, this would be the size of the inode. The other interesting parameter here is the default datastore size. Freenet nodes should generally not have a store less than a little over 200MB. The default is 256MB. The maximum size chunk that a datastore will store is 1/200th of the store size. Hence, a file split into 1MB chunks (or less) will be cacheable on all nodes it goes through. > > I am trying to determine what is the optimal size to split data into. The > size I am looking for is the one that implies that (file size) == (block > size), so that if a block gets lost, the whole file (not just a part of > it) is gone. Hmm. Use a power of two. A file and a block are no different on freenet; we delete the files when they reach the end of the LRU on a full datastore and we need more space. However if you need to store chunks of more than a meg, you need to use redundant (FEC) splitfiles. > > The reason for this is that I am trying to design a database application > that uses Frenet as the storage medium (yes, I know about FreeSQL, and it > doesn't do what I want in the way I want it done). Files going missing are > an obvious problem that needs to be tackled. I'd like to know what the > block size is in order to implement redundancy padding in the data by > exploiting the overheads produced by the block size, when a single item of > data is smaller than the block that contains it. Cool. See above. > > This could be optimized out in run-time to make no impact on execution > speed (e.g. skip downloads of blocks that we can reconstruct from already > downloaded segments). Hmm. Not sure I follow. > > Thanks. > > Gordan >
-- Matthew Toseland [EMAIL PROTECTED][EMAIL PROTECTED] Full time freenet hacker. http://freenetproject.org/ Freenet Distribution Node (temporary) at http://amphibian.dyndns.org:8889/I3mGXPd6zTA/ ICTHUS.
msg01047/pgp00000.pgp
Description: PGP signature
