a recent meditation from a mailing list on the issue of media bandwidth for big data. note especially the claim that the time needed to migrate the data to another medium exceeds the lifetime of the current medium.
> Exactly what I was referring to - bandwidth needed for data integrity & > migration. Oh, and read-never is not a myth at all - at least in the minds > of many datacenters and the folks who run them. They are under either legal > mandate and/or company policy to retain data, read regardless. They hope to > never read it at all. Still, they must prove in a court of law that they have > retained it. > > Which brings us back to data integrity and long-term preservation. If you > think it's a problem today, just wait...this is the 8 bazillion pound gorilla > that faces all institutions who plan on storing exabytes of data. FB is one > of those. > > To your point about large tape farms (disclaimer: I used to work for > StorageTek) I already know several HPC sites who are 'stuck' - i.e. they > cannot (or will not pay for) the necessary infrastructure to correctly > maintain and migrate exascale data collections. It would take them longer to > migrate the collection to new tape than the useful lifetime of the media. > And they are too cheap to buy and maintain the needed infrastructure to > perform such a migration in parallel, to reduce the time needed. > > Just you wait. 5 years from now, the scheist will hit the (exabyte) fan. > Storing data today is one thing, preserving it for decades is quite another. > HIPAA, anyone? ----------------------- Andrew Hume 949-707-1964 (VO and best) 732-420-2275 (NJ) [email protected]
_______________________________________________ Tech mailing list [email protected] https://lists.lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
