On Thu, Apr 23, 2009 at 5:43 AM, Rufus Pollock <[email protected]> wrote: > 2009/4/22 Michael Chelen <[email protected]>: >> tahoe is good for distributed filesystem, all the nodes must be trusted >> though > > I don't think that would be big problem for us -- we would only be > hosting open material and my understanding of tahoe was that it can be > configured to replicate chunks a given number of times in classic p2p > fashion. > >> what order of magnitude are the storage requirements? > > Somewhere in the region 100 GB - 5 TM at the moment -- the large > uncertainty is due to the fact that we are currently not doing a whole > bunch of things because we don't have the capacity and it is not > certain in advance how much capacity they will exactly require (e.g. > hosting datasets from CKAN0. > >> another good option might be public p2p distribution, through bittorrent >> hosting sites like vipeers.com or other software derivatives like wuala.com > > We though about a bittorrent option originally. Does the BT option > guarantee good persistence (we want stuff to stay around), and how > does it deal with replication, chunking etc > >> anyone can contribute storage space & bw by seeding the OKFN torrents, which >> are easy to share or post on sites > > Sounds interesting. How does the system control what a given node will store?
What are you mostly storing right now? Are we talking about "static data", data with updates that you put into svn? Do you need a mirror of svn as well? If it is a static huge files, then bittorrent would sound interesting. You could host it on your side, and allow mirrors to join in. If we are talking about small files <1mb then torrent might not be a solution, but http/ftp mirrors might. I guess the question would be: Could you describe the type of data you currently have. (percentage of space, downloads, changes) Thanks, Lucas _______________________________________________ okfn-discuss mailing list [email protected] http://lists.okfn.org/cgi-bin/mailman/listinfo/okfn-discuss
