Paul Libbrecht skrev  den 13-02-2007 15:36:

hello List,

maybe this should be directed to some other list... I am in the quest of a library to store, key-based, byte-streams of very varying sizes in large quantities. I need to read and write, multithreaded, and I have big and small byte-streams, from 50 bytes till 3 mega-bytes.

Files work, of course, but there are very many small fragments there, like 100 bytes or so. This is the solution I am using, I have a directory of about 27000 files right now... berk.
Perhaps you could elaborate a bit on which key you use to look up the byte stream and how these are generated.

If the filenames are evenly distributed, you could have a K/KE/KEYOFDATA.xml two-level hierarchy to easen the load off the filesystem.

You would most likely also look into using an appropriate filesystem for the task. Linux e.g. has several, with different properties.

So, do tell more :)

--
 Thorbjørn

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature

Reply via email to