Like you've correctly stated below, hardware is the least of the problems in this case.
Making the data readable after 25 years is the issue that needs to be resolved. There are several ways of going at this, but TSM on it's own is not the solution for retention over decades since TSM itself has no way of storing and converting the information so that is readable several decades into the future. We have a few customers that has had these issues, and in their cases we're talking 50-100 years (depending on information and regulations. Some state agencys are required to keep information for well over 25 years and so are insurance companies). IBM has several tools to handle this, you either go the packaged way with IBM DIAS (http://www-05.ibm.com/nl/dias/) which uses a Virtual PC technology amongst other technologies to make sure the information is gonna be readble way into the futer. DIAS is, like option number 2 below, based off Content Manager but comes in a pre-packaged configuration. Or, you go the non-packaged way and base it of Content Manager. In this case, you will need to build the routines for viewing and converting the information yourself. I'm sure you could also do all of this by yourself, but a long time ago I learned that in these cases, building the app all by yourself is not the way to go. I'd bet my last cent that 25 years from now when someone tries to read a document out of that home-built app, they're not gonna be able to read it, and there is probably not gonna be anyone around that was there when the app was written ;) Best Regards Daniel Sparrman From: "Kauffman, Tom" <kauffm...@nibco.com> To: ADSM-L@VM.MARIST.EDU Date: 2009-05-06 15:37 Subject: Re: LTO for long term archiving Sent by: "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> Soapbox time. The media is not important - any sane retention policy will require that the information be copied at least annually, with enough copies for redundancy. BUT - to be able to PROCESS the data 25 years from now imposes additional requirements. First and formost -- the data will NOT be in any format except unloaded flat-file; ASCII, UTF-8, or UTF-16 encoding. You will NOT be able to process proprietary data formats 25 years from now. Look at the Domesday Book - the version written on parchment or vellum in 1086 is still readable. The BBC digitized it in 1986 and found it almost impossible to find systems to read the digitized version in 2002 - 16 years later. You're trying for 25 years. Tom Kauffman NIBCO, Inc > -----Original Message----- > From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf > Of Evans, Bill > Sent: Tuesday, May 05, 2009 5:34 PM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: LTO for long term archiving > > I truly doubt that archiving drives, servers and tapes for 25 years > each > time the technology updates will let you read the tapes because the > drive and server will probably not even boot up and run. > > You will have to update the data every two LTO cycles or so. LTO will > read two generations back and 25 years from now we will be on LTO14 or > 15, so LTO4 is toast. Or, more probably, we will be storing into some > kind of flash drive at Petabyte capacities. > > I think that Blu-Ray DVD will meet your 25 year mark without having to > retrieve and update to new media. I know that my 1986 CD's (those not > seriously scratched or warped from laying on the dash ) still work on > today's systems. Properly stored DVD's would need to have players > stored also, but, these are mechanically simpler than LTO drives and > servers and would most likely still run. They are also much cheaper, > so > having a new DVD in storage every 5 years is no big expense. > > The bigger issue is where and how do you keep track of all of this? I > think TSM's HSM is probably capable, however, I'm not real comfortable > recommending it. We have had several years of problems running HSM on > Solaris and have finally turned it off. > > What is needed is a good archiving tool that can keep an updated DB of > content and storage location that users can browse. > > We recently restored a power point file written by Office version (?) > on > an OS 9 Mac. This could NOT be read by Office XP, 2003, 2007 (PC) or > Office 2004 or 2008 (Mac). We had to find an old Mac OS/9 that still > had a copy of Office 2000, read it, write it back to the 2000 version > .ppt file before any 2004-2009 software could read it. If it had been > 18 years instead of 9 years, then we never would have been able to read > it at all, that old OS 9 Mac would never have been saved. > > This will happen more and more as our programs become more complex and > require significant changes in the file formats. So the real problem > is > not just how to archive the data for 25 years, it's how to archive the > applications for 25 years so we can access that data! > > Actually, stone tablets are, so far, the best archive media... > > > Thanks, > > Bill Evans > Research Computing Support > FRED HUTCHINSON CANCER RESEARCH CENTER > 206.667.4194 ~ bev...@fhcrc.org > > > -----Original Message----- > From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf > Of > Kelly Lipp > Sent: Tuesday, May 05, 2009 1:42 PM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: [ADSM-L] LTO for long term archiving > > I like the implication, but I'm pretty sure somebody actually thought > being able to read the information would have been a good idea. > > Kelly Lipp > CTO > STORServer, Inc. > 485-B Elkton Drive > Colorado Springs, CO 80907 > 719-266-8777 x7105 > www.storserver.com > > > -----Original Message----- > From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On Behalf > Of > Remco Post > Sent: Tuesday, May 05, 2009 2:39 PM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: [ADSM-L] LTO for long term archiving > > I do agree, having the tapedrives around _could_ be important. I know > of at least one environment that was able to produce the media that > stores the data, but no drives. But then again, they only had to > retain the data, not the infra to access it. > > On May 5, 2009, at 22:35 , Kelly Lipp wrote: > > > To me the problem is having the drives around and more importantly, > > the interfaces to the drives. I think that probably the best bet is > > to plan on "archiving" a TSM server with a drive along with the > > media periodically. Snap off the last database backup, restore it > > on the to be archived server (a good test in itself), and store the > > whole kit together. If one needs to retrieve an archive, fire up > > the archived server, query the database to determine what tape is > > required, get it, retrieve the data and put the whole mess away. > > > > The other way to do this would be to migrate the archived data to > > new tape media as you march through time. I like this approach as > > that will have the double advantage of refreshing and verifying the > > data on those tapes. One could shelve the media in the archive > > pools and do this on a very controller basis when the media > > changes. Lots of data movement potentially, but it would become a > > nicely verified process that your auditors could look at to help > > ensure compliance. It's one thing to say we're doing and quite > > another to show we're doing it. Having the archive data more > > readily retrievable has obvious benefits as well. > > > > Kelly Lipp > > CTO > > STORServer, Inc. > > 485-B Elkton Drive > > Colorado Springs, CO 80907 > > 719-266-8777 x7105 > > www.storserver.com > > > > > > -----Original Message----- > > From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On > > Behalf Of Huebschman, George J. > > Sent: Tuesday, May 05, 2009 2:25 PM > > To: ADSM-L@VM.MARIST.EDU > > Subject: Re: [ADSM-L] LTO for long term archiving > > > > Does anyone have 25 year old tape media or tape drives around? > > Will you stil be able to use LTOx media in 25 years? > > > > -----Original Message----- > > From: ADSM: Dist Stor Manager [mailto:ads...@vm.marist.edu] On > > Behalf Of > > Thomas Denier > > Sent: Tuesday, May 05, 2009 4:11 PM > > To: ADSM-L@VM.MARIST.EDU > > Subject: [ADSM-L] LTO for long term archiving > > > > I work for a large hospital. I have been asked to investigate > possible > > configurations for archiving something between a few hundred > terabytes > > and a petabyte of data for 25 years. This would be clinical records > > that > > we need to keep in case of a malpractice suit. The retention period > is > > 25 years because there are two ways we can get sued for alleged > > malpractice involving a pediatric patient. The parents or guardians > > have > > a seven year window of opportunity to file suit, starting at the > > time of > > the alleged malpractice. The patient has a seven year window of > > opportunity, starting at his or her 18th birthday. In principle, the > > retention period should vary depending on patient age, but nobody I > > have > > talked to so far thinks it is practical to sort records in this way; > > they want a uniform retention period that covers the worst case > > scenario > > (a patient allegedly harmed as a newborn suing just before the end of > > his or her seven year window). > > > > As far as I can tell, the most expensive part of such a > > configuration is > > the media, and LTO media will cost about a third as much as the most > > economical MagStar media (extended length 3592 volumes read and > > written > > with TS1130 drives). With the sort of workload described above I > don't > > expect any difficulty staying within the recommended limit on the > > number > > of times an LTO volume passes over the tape heads. Are there any > other > > reasons to be nervous about using LTO for long term archives? > > > > IMPORTANT: E-mail sent through the Internet is not secure. Legg > > Mason therefore recommends that you do not send any confidential or > > sensitive information to us via electronic mail, including social > > security numbers, account numbers, or personal identification > > numbers. Delivery, and or timely delivery of Internet mail is not > > guaranteed. Legg Mason therefore recommends that you do not send > > time sensitive > > or action-oriented messages to us via electronic mail. > > > > This message is intended for the addressee only and may contain > > privileged or confidential information. Unless you are the intended > > recipient, you may not use, copy or disclose to anyone any > > information contained in this message. If you have received this > > message in error, please notify the author by replying to this > > message and then kindly delete the message. Thank you. > > -- > Met vriendelijke groeten, > > Remco Post > r.p...@plcs.nl > +31 6 248 21 622 CONFIDENTIALITY NOTICE: This email and any attachments are for the exclusive and confidential use of the intended recipient. If you are not the intended recipient, please do not read, distribute or take action in reliance upon this message. If you have received this in error, please notify us immediately by return email and promptly delete this message and its attachments from your computer system. We do not waive attorney-client or work product privilege by the transmission of this message.