Re: Data offload to DVDs or external drives
David O'Brien posts: The following has been posed by one of our mainframe users. QUESTION: What options (if any) are available for migrating these old study files and contents of accounts to storage media such as DVDs and external hard drives that could be securely held (off-line) by the agencies? Looking at the user id provided I find that most of these files are VSAM. Is there any way to use VSAM in a non-mainframe environment? Is there a way to write directly to a dvd from a mainframe? There have been a lot of replies already, but I think there's also a lot of guessing going on. Let's not guess too much. The most reasonable question to ask at this point is What business problem(s) are you trying to solve? Without that context, it's difficult to answer these questions. To answer the questions to the degree possible for now: 1. There are myriad options for copying data. Mainframe-accessible data are also ones and zeros. Several copy methods have been mentioned, and I could probably come up with a dozen more. (IND$FILE? Kermit? :-)) What would you prefer? 2. Maybe there is a way to use VSAM (data) in a non-mainframe environment. The data are ones and zeros, regardless of platform. Interpreting the data is another question. What programs interpret and process the data today? How were those programs created, and how are they maintained? Is NIH's requirement to preserve the ability to run those programs upon demand for some period of time and to preserve the associated data for the same period of time? And to do that in a way that reliably reproduces original study results (and potential new results based on older data), with the integrity associated with careful medical research? For how long? Or are there some other (or additional) requirements? Is that an ongoing requirement for current and future studies, to have a computing infrastructure that supports long-term retention of data *and the ability to interpret those data*? That's exactly what mainframes are designed to do. There are programs written in the mid-1960s (and perhaps even earlier) that are still running today, still processing and interpreting their data. Medical research goes back at least that far, including groundbreaking studies on smoking and cancer (for example). This is something NIH really ought to be thinking about, carefully, and at senior levels. The central design premise of mainframes is avoid breaking programs if at all possible. In contrast, our PCs (and Macs) break programs practically every year that passes. Archivists are warning that society is rushing headlong into creating a big digital hole in the historical record, because we simply won't be able to process and interpret older data (even if we have it) even a few years from now. Mainframes are a very notable exception, precisely because many businesses have the same requirements. Many insurance companies, for example, need to retain policies and the processing rules associated with those policies for 100 years or more (the lifetime of a life insurance policyholder and his/her heirs). Mainframes do that -- and support running brand new programs written 5 minutes ago. 3. Yes, actually. (There's at least one vendor that sells hardware to do that.) To what purpose? Many/most mainframes have tape available, often HSM-managed, which works beautifully for archiving programs and data -- and for managing the ability to carry those data forward for decades through technology changes, if that's the retention policy. (And I could see why that might be the retention policy for certain NIH programs and data. Cancer studies need to be long-running, for example. Same with research into chemicals that mimic hormones, which are very subtle and gradual but extremely serious, to pick another example.) If the business problem is to save money, bear in mind that programs that don't run consume zero CPU and data stored on tape consumes (a part of) a tape. That's it. Mainframes are exceptionally talented at (centralized cloud) archiving, because that's what businesses (and governments) need to do quite often, and those are the systems they buy to do it. I'm hard pressed to think of another option that could be less expensive in the real world. If somebody is charging someone else within the same federal government a price that bears no relation to that reality, then that's the business problem to solve -- certainly for the sake of this taxpayer and millions of others. Timothy Sipples Resident Enterprise Architect (Based in Singapore) E-Mail: timothy.sipp...@us.ibm.com -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
Backing up to paper (and restoring from it) is already possible... http://www.ollydbg.de/Paperbak/ The author claims about 3 MB per A4 page, so 1 TB could be backed up to about 700 reams. I think the paper would last longer than the ink though. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
Responding to Ed's question: New media usually means new tape number ranges. Once the new media is in use for HSM it is a simple matter to recycle using: RECYCLE EXECUTE ML2 PERCENTVALID(99) - SELECT(INCLUDE(RANGE(E2:E20999))) -Original Message- From: Mike Schwab [mailto:mike.a.sch...@gmail.com] Sent: Wednesday, October 19, 2011 8:21 PM To: IBM-MAIN@bama.ua.edu Subject: Re: Data offload to DVDs or external drives You can also specify volsers. Very useful if one is damaged. On Wed, Oct 19, 2011 at 5:27 PM, Ed Gould ps2...@yahoo.com wrote: David, Great reply. Thank you for pointing this out. As a small side question (I have never had to do this) how does one identify tapes that need recycling? I have just done recycles on percent valid and was never worried about doing it based on device type. Ed -- Mike A Schwab, Springfield IL USA Where do Forest Rangers go to get away from it all? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
-Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of Ed Gould Sent: Wednesday, October 19, 2011 5:34 PM To: IBM-MAIN@bama.ua.edu Subject: Re: Data offload to DVDs or external drives John, Come on egypt tech??? And certain types of paper can last with the proper conditions can last a few hundred years but when we are talking IT we are talking machine readable. Ed Ps OCR is not an option. I guess that I should never rely on the implied smiley in what I consider to be obvious foolishness on my part. silliness type=more How about MICR ink and bar coding? There may be an ASR-33 around somewhere which can read/write Mylar tape. I wonder how long that would last? Hum, how about metallic tape? /silliness -- John McKown Systems Engineer IV IT Administrative Services Group HealthMarkets(r) 9151 Boulevard 26 * N. Richland Hills * TX 76010 (817) 255-3225 phone * john.mck...@healthmarkets.com * www.HealthMarkets.com Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
W dniu 2011-10-19 17:08, O'Brien, David W. (NIH/CIT) [C] pisze: The following has been posed by one of our mainframe users. QUESTION: What options (if any) are available for migrating these old study files and contents of accounts to storage media such as DVDs and external hard drives that could be securely held (off-line) by the agencies? A lot of. Quite many at least. Looking at the user id provided I find that most of these files are VSAM. Is there any way to use VSAM in a non-mainframe environment? NFS, SMB, ftp. For convenience I would use REPRO, DUMP or other option before the above. Is there a way to write directly to a dvd from a mainframe? Directly ? What does it mean? How much directly is directly enough? Hint: Most DVDs in PC world are recorded INdirectly, by using some CD/DVD Burn program, like Nero, Roxio, CDBurn, etc. -- Radoslaw Skorupka Lodz, Poland -- Tre tej wiadomoci moe zawiera informacje prawnie chronione Banku przeznaczone wycznie do uytku subowego adresata. Odbiorc moe by jedynie jej adresat z wyczeniem dostpu osób trzecich. Jeeli nie jeste adresatem niniejszej wiadomoci lub pracownikiem upowanionym do jej przekazania adresatowi, informujemy, e jej rozpowszechnianie, kopiowanie, rozprowadzanie lub inne dziaanie o podobnym charakterze jest prawnie zabronione i moe by karalne. Jeeli otrzymae t wiadomo omykowo, prosimy niezwocznie zawiadomi nadawc wysyajc odpowied oraz trwale usun t wiadomo wczajc w to wszelkie jej kopie wydrukowane lub zapisane na dysku. This e-mail may contain legally privileged information of the Bank and is intended solely for business use of the addressee. This e-mail may only be received by the addressee and may not be disclosed to any third parties. If you are not the intended addressee of this e-mail or the employee authorised to forward it to the addressee, be advised that any dissemination, copying, distribution or any other similar activity is legally prohibited and may be punishable. If you received this e-mail by mistake please advise the sender immediately by using the reply facility in your e-mail software and delete permanently this e-mail including any copies of it either printed or saved to hard drive. BRE Bank SA, 00-950 Warszawa, ul. Senatorska 18, tel. +48 (22) 829 00 00, fax +48 (22) 829 00 33, e-mail: i...@brebank.pl Sd Rejonowy dla m. st. Warszawy XII Wydzia Gospodarczy Krajowego Rejestru Sdowego, nr rejestru przedsibiorców KRS 025237, NIP: 526-021-50-88. Wedug stanu na dzie 01.01.2011 r. kapita zakadowy BRE Banku SA (w caoci wpacony) wynosi 168.346.696 zotych. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
Dave, This is a data migration problem, something very common in mainframe migration/modernization projects. There are ETL tools (Extract-Transform-Load) to help with doing such projects. But first you need to establish the goal of migrating the data. From what you describe, I could see two possible goals: (1) Get the data off the mainframe and onto cheap commodity storage so that it can be accessed easily using non-mainframe tools (SQL queries, etc.), OR (2) Get the data off the mainframe and onto cheap storage. If ever needed in the future it could be recalled back to the mainframe for access. Option (1) is a lot more work. I will explain in a separate note what I did at another client a few years back to preserve a bunch of old mainframe datasets for SQL access. Option (2) is pretty easy. You just write something to dump your records as external hexadecimal, FTP the files to a Windows/Linux server, and then burn to DVD (or move to a cheap disk array). You then build some programs to reverse the process when needed to bring back data to the mainframe. John -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
Dave, In 2006 I did a project in Ohio to offload a bunch of data files to MS SQL. They had completed a SAP implementation, but had not bothered to deal with all the historical data left behind on the old mainframe. They had VSAM files, QSAM disk files, thousands of tapes, and even a few IMS databases. Altogether they had nearly a terabyte of data to preserve. The goal was to make the migrated data accessible in the MS SQL environment. There would be no going back to the mainframe. It was thought that only a handful of IT staff would ever have cause to look at the preserved data, but they were reluctant to just scrap it all. A big problem was that the data records contained a mix of text and binary fields (COMP and COMP-3). This implied that I needed to do field level transformations. I had budget constraints. The client would not pay for any special ETL tools (e.g. Informatica), and the budget for the SQL server was under $10K. This is how I performed the project: (1) Had the client purchase a modest Windows Server, a few SATA hard disks of 750GB each, and installed MS SQL Server 2005 Workgroup Edition. (2) Took a survey of all the mainframe datasets within scope. (3) Loaded information about the mainframe datasets into an MS Access database. (4) Analyzed the Access DB to try to identify dataset families that share the same record layouts. (5) Matched COPYBOOKS against the dataset families (a mostly manual task). (6) Wrote a bunch of little data extract programs for each of the COPYBOOKS. These programs would transform the data from mainframe format to comma-separated-value (CSV) format. The programs also created SQL table DDL, SQL BCP format files, FTP commands, etc. (7) Ran the extract programs. (8) FTP'ed the extracted data to the SQL Server. Also FTPed the DDL, BCP Formats, etc. (9) Ran the DDL to define the SQL tables. (10) Ran the BCP scripts to load the tables. (11) Wrote some DOC so future IT staff could relate old mainframe DSNAMES to the new SQL Tables. John -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
-Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of O'Brien, David W. (NIH/CIT) [C] Sent: Wednesday, October 19, 2011 10:08 AM To: IBM-MAIN@bama.ua.edu Subject: Data offload to DVDs or external drives The following has been posed by one of our mainframe users. QUESTION: What options (if any) are available for migrating these old study files and contents of accounts to storage media such as DVDs and external hard drives that could be securely held (off-line) by the agencies? Looking at the user id provided I find that most of these files are VSAM. Is there any way to use VSAM in a non-mainframe environment? Not really. And which VSAM do you mean? There are 5 flavors: ESDS (sequential), KSDS (keyed), RRDS (relative record - fixed length records), VRRDS (Relative record - variable length records), and LINEAR (no logical records at all, more like memory since access is via DIV and byte addressing). I assume you mean KSDS. I know of no way to directly access a KSDS except via z/OS. Now, there are equivalent indexed sequential access methods available on UNIX and Windows platforms. But that is a file conversion. Is there a way to write directly to a dvd from a mainframe? I've never heard of one. DVDs are generally WORM devices. And you normally premaster the DVD image and write it in a single session. There are devices on the market which emulate mainframe tapes, storing the tape image in a disk file. You could then put those images on a DVD using UNIX/Windows software. That would be for off-site storage. And some of those devices have software subroutines so that a UNIX/Windows program can read the data from the tape image. Assuming the program knows how to handle the z/OS data (EBCDIC character coding, z/OS packed decimal, z/OS binary integers, z/OS HFP and BFP floating point formats). I think MicroFocus COBOL has the capability to do this latter in its I/O routines. What is your bottom line objective? To just have the data on DVDs in some format so that it can be restored to the mainframe? Or in a format so that it can be post processed by a non-z/OS system such as UNIX or Windows? In the former case, I would backup the dataset using ADRDSSU or FDR, then BINary ftp that dataset to a Linux/Windows platform so that it can be burned to DVD. In the latter case, you either need to ASCII ftp a sequential unload of the data which has converted everything to printable (translatable) characters. You could then burn a DVD with those files. TIA, Dave O'Brien NIH Contractor -- John McKown Systems Engineer IV IT Administrative Services Group HealthMarkets(r) 9151 Boulevard 26 * N. Richland Hills * TX 76010 (817) 255-3225 phone * john.mck...@healthmarkets.com * www.HealthMarkets.com Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
John, Thank you to all who have responded. Looking at the supplied HLQ, all of the VSAM appear to be KSDS. I believe the intent is to remove the data from the mainframe and subsequently access it using PC SAS, if necessary. Dave O'Brien -Original Message- From: McKown, John [mailto:john.mck...@healthmarkets.com] Sent: Wednesday, October 19, 2011 1:57 PM To: IBM-MAIN@bama.ua.edu Subject: Re: Data offload to DVDs or external drives -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of O'Brien, David W. (NIH/CIT) [C] Sent: Wednesday, October 19, 2011 10:08 AM To: IBM-MAIN@bama.ua.edu Subject: Data offload to DVDs or external drives The following has been posed by one of our mainframe users. QUESTION: What options (if any) are available for migrating these old study files and contents of accounts to storage media such as DVDs and external hard drives that could be securely held (off-line) by the agencies? Looking at the user id provided I find that most of these files are VSAM. Is there any way to use VSAM in a non-mainframe environment? Not really. And which VSAM do you mean? There are 5 flavors: ESDS (sequential), KSDS (keyed), RRDS (relative record - fixed length records), VRRDS (Relative record - variable length records), and LINEAR (no logical records at all, more like memory since access is via DIV and byte addressing). I assume you mean KSDS. I know of no way to directly access a KSDS except via z/OS. Now, there are equivalent indexed sequential access methods available on UNIX and Windows platforms. But that is a file conversion. Is there a way to write directly to a dvd from a mainframe? I've never heard of one. DVDs are generally WORM devices. And you normally premaster the DVD image and write it in a single session. There are devices on the market which emulate mainframe tapes, storing the tape image in a disk file. You could then put those images on a DVD using UNIX/Windows software. That would be for off-site storage. And some of those devices have software subroutines so that a UNIX/Windows program can read the data from the tape image. Assuming the program knows how to handle the z/OS data (EBCDIC character coding, z/OS packed decimal, z/OS binary integers, z/OS HFP and BFP floating point formats). I think MicroFocus COBOL has the capability to do this latter in its I/O routines. What is your bottom line objective? To just have the data on DVDs in some format so that it can be restored to the mainframe? Or in a format so that it can be post processed by a non-z/OS system such as UNIX or Windows? In the former case, I would backup the dataset using ADRDSSU or FDR, then BINary ftp that dataset to a Linux/Windows platform so that it can be burned to DVD. In the latter case, you either need to ASCII ftp a sequential unload of the data which has converted everything to printable (translatable) characters. You could then burn a DVD with those files. TIA, Dave O'Brien NIH Contractor -- John McKown Systems Engineer IV IT Administrative Services Group HealthMarkets(r) 9151 Boulevard 26 * N. Richland Hills * TX 76010 (817) 255-3225 phone * john.mck...@healthmarkets.com * www.HealthMarkets.com Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
On Wed, 19 Oct 2011 11:08:09 -0400, O'Brien, David W. wrote: What options (if any) are available for migrating these old study files and contents of accounts to storage media such as DVDs and external hard drives that could be securely held (off-line) by the agencies? If you really mean DVD, I'd suggest caution. IBM DVD media is not very reliable. If you are going to move the data to a PC to create the DVD, I think it would be better to write it to a hard drive. Even at that, I think I'd want to create two copies on two hard drives. You might want to consider what Shai Hess offers. -- Tom Marchant -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
Tom: What ever media is decided on remember this. The life expectancy (readability) is at most 10 years for DVD and CD's . I had a long discussion with an expert and he just does not have a satisfactory answer (IMO) for anything long term. I suspect tape (even IBM's tape drives) to be iffy at 10 years as well. Technology changes so fast that there isn't a really great long term storage option. Ed' - Original Message - From: Tom Marchant m42tom-ibmm...@yahoo.com To: IBM-MAIN@bama.ua.edu Cc: Sent: Wednesday, October 19, 2011 1:21 PM Subject: Re: Data offload to DVDs or external drives On Wed, 19 Oct 2011 11:08:09 -0400, O'Brien, David W. wrote: What options (if any) are available for migrating these old study files and contents of accounts to storage media such as DVDs and external hard drives that could be securely held (off-line) by the agencies? If you really mean DVD, I'd suggest caution. IBM DVD media is not very reliable. If you are going to move the data to a PC to create the DVD, I think it would be better to write it to a hard drive. Even at that, I think I'd want to create two copies on two hard drives. You might want to consider what Shai Hess offers. -- Tom Marchant -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
-Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of Ed Gould Sent: Wednesday, October 19, 2011 1:44 PM To: IBM-MAIN@bama.ua.edu Subject: Re: Data offload to DVDs or external drives Tom: What ever media is decided on remember this. The life expectancy (readability) is at most 10 years for DVD and CD's . I had a long discussion with an expert and he just does not have a satisfactory answer (IMO) for anything long term. I suspect tape (even IBM's tape drives) to be iffy at 10 years as well. Technology changes so fast that there isn't a really great long term storage option. Ed' Egypt tech. Chisled granite kept in a desert has a proven long life time. Clay tablets aren't too bad either. Both are difficult to write to from a PC or even a mainframe. Of course, the information density is poor. -- John McKown Systems Engineer IV IT Administrative Services Group HealthMarkets® 9151 Boulevard 26 . N. Richland Hills . TX 76010 (817) 255-3225 phone . john.mck...@healthmarkets.com . www.HealthMarkets.com Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets® is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company®, Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
Responding to Ed's comments: At least if the data is under HSM control, it (the data) gets recycled to newer media every time the tape media changes. The same cannot be said for the private media be it tape or DVD that my user seems to have in mind. -Original Message- From: McKown, John [mailto:john.mck...@healthmarkets.com] Sent: Wednesday, October 19, 2011 2:55 PM To: IBM-MAIN@bama.ua.edu Subject: Re: Data offload to DVDs or external drives -Original Message- From: IBM Mainframe Discussion List [mailto:IBM-MAIN@bama.ua.edu] On Behalf Of Ed Gould Sent: Wednesday, October 19, 2011 1:44 PM To: IBM-MAIN@bama.ua.edu Subject: Re: Data offload to DVDs or external drives Tom: What ever media is decided on remember this. The life expectancy (readability) is at most 10 years for DVD and CD's . I had a long discussion with an expert and he just does not have a satisfactory answer (IMO) for anything long term. I suspect tape (even IBM's tape drives) to be iffy at 10 years as well. Technology changes so fast that there isn't a really great long term storage option. Ed' Egypt tech. Chisled granite kept in a desert has a proven long life time. Clay tablets aren't too bad either. Both are difficult to write to from a PC or even a mainframe. Of course, the information density is poor. -- John McKown Systems Engineer IV IT Administrative Services Group HealthMarkets® 9151 Boulevard 26 . N. Richland Hills . TX 76010 (817) 255-3225 phone . john.mck...@healthmarkets.com . www.HealthMarkets.com Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets® is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company®, Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
On Wed, 19 Oct 2011 13:21:14 -0500, Tom Marchant wrote: IBM DVD media is not very reliable. I meant to write IMO DVD media is not very reliable. I don't know how my fingers wrote IBM or why my eyes didn't catch the error. And when I say that it is not very reliable, I'm not just talking about longevity. I create rather a lot of DVDs of Linux distros and the like. The number of bad DVDs created is probably only a few percent, but I consider it unacceptably high. And the errors are not usually detected at write time. -- Tom Marchant -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
The life expectancy (readability) is at most 10 years for DVD and CD's . Whenever you are archiving data, you need to ask the question: what is the expected Standard of Care. IMO, data archival is often a CYA proposition. You may think that the data is a bunch of old junk, but you are unsure about the legal requirements for retention. So you just take the easy way out and copy the whole lot to tapes or optical media. Then you send it away to IronMountain.Com and forget about it. If ever an issue arises (like a court order), you just recall the media and tell the investigators to knock themselves out going thru the whole mess. If a few of the media are damaged or unreadable, then it's just bad luck - so sorry. You didn't overtly do anything to ruin any of the records, so it's not your fault. An organization can probably get away with this in most circumstances. For example, if one of Dave's NIH Study Files concerned a failed drug that never made it beyond animal testing, then I doubt anyone much cares what happens to these records. But if another of the NIH Study Files involves a drug implicated in many deaths (e.g. Avandia), then those records should be proactively preserved, even if that means copying to new media every 5 years. John -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
With the reference to moving data files from z/OS to an ASCII platform and intending to read that archived data, for any binary file, that is, a file that can contain non-pure-text data, the PGM=FTP on z/OS will delete ALL of the BDWs and RDWs from any dataset with RECFM=V or VB or VBS, and you end up with a serial file of bytes of binary data, on your PC, with no way to know where a record started or ended. You must use specialized JCL around your PGM=FTP to preserve the BDWs and RDWs Example i., below), or use PGM=IEBGENER to create a block-for-block copy of the z/OS file, but with RECFM=U (Example iv. below), since RECFM=U prevents PGM=FTP from mucking about with the BDWs and RDWs). i. ftp instructions for sending data to the MXG ftp site Using the IBM ftp program, you can ftp any MVS data file to the MXG ftp site with this syntax; do NOT change the DCB attributes of the //SMFFILE DD: even though it points to a VBS DSNAME, the RECFM=U,BLKSIZE=32760 must be used to include BDW and RDWs. //FTP EXEC PGM=FTP,PARM='(EXIT=4' //SYSPRINT DD SYSOUT=* //OUTPUT DD SYSOUT=* //SMFFILE DD DSN=YOUR.SMF.DATA,DCB=RECFM=U,BLKSIZE=32760,DISP=SHR //INPUT DD * ftp.mxg.com userid password quote PASV bin put //DD:SMFFILE yourname.smf close quit /* IMPORTANT: userid/password may be mixed case do NOT change the DCB attributes of the //SMFFILE DD. This ftp can be used for MVS files with RECFM=U,F,FB,V,VB, or VBS, but for F/FB files, please email us the LRECL value of each file. The SYSPRINT output will tell you if the ftp transfer succeeded or not; in a production environment, you probably want a Return Code set if there was any error; the syntax for the IBM ftp program to set Return Code 12 on any error is //FTP EXEC PGM=FTP,PARM='FTP.MXG.COM (EXIT=12' iv. If you simply try to ftp a V/VB/VBS file, it will be unreadable because when PGM=FTP sees those RECFMs, it shows how smart it is and sends only the data, stripping the BDW and RDW. The previous PGM=FTP with its special references solves that problem. But an alternative is to create a RECFM=U copy of the V/VB/VBS file, and then that copy can be ftp'd and the BDW/RDW will be preserved. // EXEC PGM=IEBGENER //SYSUT1 DD DSN=YOUR.VB.FILE,DISP=SHR,RECFM=U,BLKSIZE=32760 //SYSUT2 DD DSN=YOUR.NEW.RECFMU,DISP=(,CATLG),RECFM=U, // BLKSIZE=32760,UNIT=SYSDA,SPACE=(CYL,(50,50)) //SYSIN DD DUMMY -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
David, Great reply. Thank you for pointing this out. As a small side question (I have never had to do this) how does one identify tapes that need recycling? I have just done recycles on percent valid and was never worried about doing it based on device type. Ed -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
John, Come on egypt tech??? And certain types of paper can last with the proper conditions can last a few hundred years but when we are talking IT we are talking machine readable. Ed Ps OCR is not an option. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
Why not print it all out? Then get an army of Monks to transcribe to archival parchment scrolls using quill pens and an ink formula from about 2000 years ago. Seal the results into ceramic pottery vessels and place in a cave, somewhere near the Dead Sea. Roll a big rock in front of the cave and don't tell anyone the location. I seem to recall that this worked for some people around 200AD. John -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
You can also specify volsers. Very useful if one is damaged. On Wed, Oct 19, 2011 at 5:27 PM, Ed Gould ps2...@yahoo.com wrote: David, Great reply. Thank you for pointing this out. As a small side question (I have never had to do this) how does one identify tapes that need recycling? I have just done recycles on percent valid and was never worried about doing it based on device type. Ed -- Mike A Schwab, Springfield IL USA Where do Forest Rangers go to get away from it all? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: Data offload to DVDs or external drives
John. I know you are kidding, right? Have you seen some pictures f the scrolls? They are not easily readable cracked with age and the only thing that helped save them was the lack of humidity in the desert. Now I realize. There are people that have partially that deciphered them but we are talking about bight trained people Do you expect people in 2000 years to attempt to read a payroll file and to what End? IRS would not care, a historian wouldn#39;t care but a business might. Ed -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
