FTP error checking and recovery
How reliable is FTP across the internet ? I have a large number of DFDSS dumps to move to the IBM Remote Development Facility and the most convenient way is via FTP. Can I be confident that if the FTP completes without error then the DFDSS dump file will get there completely intact and without errors. Or should I look for alternate methods of delivery such as copying the dumps to DVD. Jim McAlpine -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: FTP error checking and recovery
Jim FTP uses TCP so it's the reliability of TCP that matters here. Of what underlying facilities happen to consist is very probably of no relevance. Thus, assuming the underlying facilities are inherently *less* reliable that the reliability offered by TCP, you get the same level of reliability whether you are passing data within your local infranet or over the Internet. So, if you consider TCP offers sufficient reliability, you don't need to go looking for another technique for moving your dumps. I suspect that, for dumps, TCP is quite adequate. Chris Mason On Fri, 11 Sep 2009 09:42:05 +0100, Jim McAlpine jim.mcalp...@gmail.com wrote: How reliable is FTP across the internet ? I have a large number of DFDSS dumps to move to the IBM Remote Development Facility and the most convenient way is via FTP. Can I be confident that if the FTP completes without error then the DFDSS dump file will get there completely intact and without errors. Or should I look for alternate methods of delivery such as copying the dumps to DVD. Jim McAlpine -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: FTP error checking and recovery
On Fri, Sep 11, 2009 at 9:56 AM, Chris Mason chrisma...@belgacom.netwrote: Jim FTP uses TCP so it's the reliability of TCP that matters here. Of what underlying facilities happen to consist is very probably of no relevance. Thus, assuming the underlying facilities are inherently *less* reliable that the reliability offered by TCP, you get the same level of reliability whether you are passing data within your local infranet or over the Internet. So, if you consider TCP offers sufficient reliability, you don't need to go looking for another technique for moving your dumps. I suspect that, for dumps, TCP is quite adequate. Chris Mason Chris, thanks, that's good to hear. However I've just tried to FTP my first dump which was approx 1.4 GB and it failed about one third the way through with - Netout : Connection reset by peer 451 Transfer aborted due to receive error which doesn't bode well. Jim McAlpine -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: FTP error checking and recovery
Jim Being reliable includes giving up - and saying so - when slings and arrows intervene. Perhaps I picked up the wrong emphasis from your original post. I took it to be a concern over FTP rather than a concern over the Internet. As you have demonstrated to yourself, the Internet - as they say these days - has reliability issues! At some point, TCP will give up retrying to compensate for the unreliability of the underlying TCP connection path. Chris Mason On Fri, 11 Sep 2009 10:26:20 +0100, Jim McAlpine jim.mcalp...@gmail.com wrote: On Fri, Sep 11, 2009 at 9:56 AM, Chris Mason chrisma...@belgacom.netwrote: Jim FTP uses TCP so it's the reliability of TCP that matters here. Of what underlying facilities happen to consist is very probably of no relevance. Thus, assuming the underlying facilities are inherently *less* reliable that the reliability offered by TCP, you get the same level of reliability whether you are passing data within your local infranet or over the Internet. So, if you consider TCP offers sufficient reliability, you don't need to go looking for another technique for moving your dumps. I suspect that, for dumps, TCP is quite adequate. Chris Mason Chris, thanks, that's good to hear. However I've just tried to FTP my first dump which was approx 1.4 GB and it failed about one third the way through with - Netout : Connection reset by peer 451 Transfer aborted due to receive error which doesn't bode well. Jim McAlpine -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: FTP error checking and recovery
-Original Message- From: IBM Mainframe Discussion List On Behalf Of Jim McAlpine How reliable is FTP across the internet ? I have a large number of DFDSS dumps to move to the IBM Remote Development Facility and the most convenient way is via FTP. Can I be confident that if the FTP completes without error then the DFDSS dump file will get there completely intact and without errors. Or should I look for alternate methods of delivery such as copying the dumps to DVD. How reliable is the Internet itself? The only FTP problem we've ever experienced, where a file received was NOT identical to the file sent, was (finally) traced to a defective switch. At random, bytes were trans-positioned within a block such that the checksum for the block remained unchanged. That was a real head-scratcher. If the files you need to move are sufficiently important to you, do both. -jc- -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: FTP error checking and recovery
Jim, How reliable is FTP across the internet ? I have a large number of DFDSS dumps to move to the IBM Remote Development Facility and the most convenient way is via FTP. Can I be confident that if the FTP completes without error then the DFDSS dump file will get there completely intact and without errors. Or should I look for alternate methods of delivery such as copying the dumps to DVD. I always had problems sending DSS dumps via ftp unless I tersed it. DSS expects a short block at the beginning of the file and my ftp, especially to a non-mainframe server always seemed to cause the resulting file to be reblocked. The result was DSS would state it was not a valid backup. It seems to me that you could use struct=r or something like that if you were ftp'ing from a z/os directly to another z/os. As for the Internet, you get what you get. There are ftp clients capable of restarting the ftp at the point in the file where it failed. That would allow you to avoid resending data that has already been received. Good luck, John -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: FTP error checking and recovery
On Fri, 11 Sep 2009 09:42:05 +0100, Jim McAlpine wrote: How reliable is FTP across the internet ? ... (That's how it is most commonly used.) How reliable is FTP across the internet ? I have a large number of DFDSS dumps to move to the IBM Remote Development Facility and the most convenient way is via FTP. Can I be confident that if the FTP completes without error then the DFDSS dump file will get there completely intact and without errors. Or should I look for alternate methods of delivery such as copying the dumps to DVD. With either technique, you will probably need to protect the record structure with an envlope. IBM usually recommends AMATERSE. Do TERSE envelopes use any sort checksum for verification? You will need to investigate: o Does the RDF provide support for data on DVD? o Does the RDF provide support for verifying checksums? The base z/OS provides the POSIX cksum(1), and ICSF supports SHA-1, perhaps others, and MD5 is widely available as source in an RFC and as executables for various platforms. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: FTP error checking and recovery
On Fri, 11 Sep 2009 10:26:20 +0100, Jim McAlpine wrote: On Fri, Sep 11, 2009 at 9:56 AM, Chris Mason wrote: FTP uses TCP so it's the reliability of TCP that matters here. Of what underlying facilities happen to consist is very probably of no relevance. Thus, Chris, thanks, that's good to hear. However I've just tried to FTP my first dump which was approx 1.4 GB and it failed about one third the way through with - Netout : Connection reset by peer 451 Transfer aborted due to receive error which doesn't bode well. In days of yore, transmitting binary files (SMP/E sysmods) from desktop systems to MVS FTP servers, I have observed massive unreported failures such as dropouts of entire 8KiByte blocks. I don't know whether z/OS is better nowadays. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: FTP error checking and recovery
FWIW, SSH protocol 2 includes MAC (message authentication code) checking (as well as compression and encryption). SSH/SFTP is therefore verified, compressed, and encrypted automatically. Kirk Wolf Dovetailed Technologies http://dovetail.com On Fri, Sep 11, 2009 at 8:44 AM, Paul Gilmartin paulgboul...@aim.comwrote: On Fri, 11 Sep 2009 10:26:20 +0100, Jim McAlpine wrote: On Fri, Sep 11, 2009 at 9:56 AM, Chris Mason wrote: FTP uses TCP so it's the reliability of TCP that matters here. Of what underlying facilities happen to consist is very probably of no relevance. Thus, Chris, thanks, that's good to hear. However I've just tried to FTP my first dump which was approx 1.4 GB and it failed about one third the way through with - Netout : Connection reset by peer 451 Transfer aborted due to receive error which doesn't bode well. In days of yore, transmitting binary files (SMP/E sysmods) from desktop systems to MVS FTP servers, I have observed massive unreported failures such as dropouts of entire 8KiByte blocks. I don't know whether z/OS is better nowadays. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
