On Apr 24, 2007, at 11:26 AM, David Brown wrote:
I know there's been lots of research into investigating the
possibilities of putting some form of processing on archives with
massive amounts of disks (active-storage). I was wondering if anyone
has approached the pvfs developers with ideas of integrating an
active-storage like system to the servers to allow them to perform
preprocessing on the data before it gets sent over the network?
One could implement something very quick using fuse and have the
servers work on top of the fuse file system. However, that would
depend on how the raw data of the files are stored on each of the
storage servers. Could someone give me a place to start looking as to
where the raw data of the files are located? algorithms used for
distributing files among the storage servers? block sizes available?
Hi David,
The file's stripe on an IO server (we call it a byte-stream or
bstream) is located at:
/pvfs-storage/<hex-fs-id>/bstreams/<bucket-hash>/<hex-handle>.bstream
The storage of files is all done in the trove dbpf component, namely
dbpf-bstream.c.
The default algorithm (we call them 'distributions') for distributing
files is our simple-stripe. There's other distributions you can
specify on a per-file basis. The implementations of them are in src/
io/description/.
Let us know if you have other questions.
Thanks,
-sam
Any information would be helpful.
Thanks,
- David Brown
_______________________________________________
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
_______________________________________________
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
