Same code on two different source hosts going to the same destination fails on
occasion. I've never had an issue with any other destination hosts using this
code (5 years?).
I had to add a bit to get the example code to work (pbsz and prot, as well as
site commands), but I was able to successfully get the file transferred.
A little background on my issue, the mainframe I'm sending to is fairly
saturated, and I suspected the issue was caused by that.
I started playing around with the control keepalive timeout's, which my
understanding is, that a NOOP gets sent over the control connection every
keepalive timeout period.
In my case, I override it by default to 4:30. I thought maybe the control
connection was closing (timed out), and I wasn't receiving the successful
reply, so I dropped it to 60 seconds.
I have one 2GB file which I can get to fail consistently using my code. When I
set the keepAliveTimout (-k 60) using the FTPClientExample, it also fails in
the same way.
It's odd that the protocol listener doesn't print the result, even though the
file transmission completed.
Command:
java -cp commons-net-3.5.jar;. FTPClientExample -p false -# -s -k 60
mvsp.xxx:2121 C836PTS password 'A836.ORG.CAMA2.FULL' d:\LargeExtracts\full.txt
FTPClientExample Log Snippet:
#####################java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at sun.security.ssl.InputRecord.readFully(Unknown Source)
at sun.security.ssl.InputRecord.read(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readDataRecord(Unknown Source)
at sun.security.ssl.AppInputStream.read(Unknown Source)
at sun.nio.cs.StreamDecoder.readBytes(Unknown Source)
at sun.nio.cs.StreamDecoder.implRead(Unknown Source)
at sun.nio.cs.StreamDecoder.read(Unknown Source)
at java.io.InputStreamReader.read(Unknown Source)
at java.io.BufferedReader.fill(Unknown Source)
at java.io.BufferedReader.readLine(Unknown Source)
at java.io.BufferedReader.readLine(Unknown Source)
at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:313)
at org.apache.commons.net.ftp.FTP.__getReplyNoReport(FTP.java:303)
at org.apache.commons.net.ftp.FTPClient$CSL.cleanUp(FTPClient.java:3839)
at org.apache.commons.net.ftp.FTPClient._storeFile(FTPClient.java:695)
at org.apache.commons.net.ftp.FTPClient.__storeFile(FTPClient.java:643)
at org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:2034)
at FTPClientExample.main(FTPClientExample.java:415)
-----Original Message-----
From: sebb [mailto:[email protected]]
Sent: Thursday, November 17, 2016 2:49 PM
To: Commons Users List
Subject: Re: [net] FTPClient Transfer succeeds but fails in cleanUp
Does the same code fail when sending the same file to a different host?
Have you tried using the example utility?
http://commons.apache.org/proper/commons-net/#Examples
This exercises most of the FTP methods.
On 17 November 2016 at 18:01, Seganti, Michael (DOF) <[email protected]>
wrote:
> I just updated to 3.5, and this file is consistently failing.
>
> I no longer receive the
>
> 250 Transfer completed successfully.
> 200 OK
>
> Replies.
>
> Log Snippet:
>
> 2016-11-17 12:56:33.870 - 100 Percent Complete, Total Bytes
> Transferred: 2101359588
>
> File Transfer Failed at: 2016-11-17 12:56:38.885 Error
> Occurred Transmitting File to Remote System, aborting...
>
> Read timed out
> java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(Unknown Source)
> at java.net.SocketInputStream.read(Unknown Source)
> at sun.security.ssl.InputRecord.readFully(Unknown Source)
> at sun.security.ssl.InputRecord.read(Unknown Source)
> at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
> at sun.security.ssl.SSLSocketImpl.readDataRecord(Unknown Source)
> at sun.security.ssl.AppInputStream.read(Unknown Source)
> at sun.nio.cs.StreamDecoder.readBytes(Unknown Source)
> at sun.nio.cs.StreamDecoder.implRead(Unknown Source)
> at sun.nio.cs.StreamDecoder.read(Unknown Source)
> at java.io.InputStreamReader.read(Unknown Source)
> at java.io.BufferedReader.fill(Unknown Source)
> at java.io.BufferedReader.readLine(Unknown Source)
> at java.io.BufferedReader.readLine(Unknown Source)
> at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:313)
> at org.apache.commons.net.ftp.FTP.__getReplyNoReport(FTP.java:303)
> at
> org.apache.commons.net.ftp.FTPClient$CSL.cleanUp(FTPClient.java:3839)
> at org.apache.commons.net.ftp.FTPClient._storeFile(FTPClient.java:695)
> at
> org.apache.commons.net.ftp.FTPClient.__storeFile(FTPClient.java:643)
> at org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:2034)
> at gov.nyc.ftputils.FTPConnection.sendFile(FTPConnection.java:695)
> at
> gov.nyc.ftputils.FTPPropertyProcessor.putFile(FTPPropertyProcessor.java:54)
> at
> gov.nyc.ftputils.FTPPropertyProcessor.putFile(FTPPropertyProcessor.java:96)
> at
> gov.nyc.ftputils.FTPPropertyProcessor.processFileProps(FTPPropertyProcessor.java:126)
> at gov.nyc.ftputils.FTPClientUtil.main(FTPClientUtil.java:378)
> FTPClientUtil has Failed
> QUIT
> 221 Quit command received. Goodbye.
>
>
> -----Original Message-----
> From: Seganti, Michael (DOF) [mailto:[email protected]]
> Sent: Thursday, November 17, 2016 12:34 PM
> To: Commons Users List
> Subject: RE: [net] FTPClient Transfer succeeds but fails in cleanUp
>
> I need to do a quick test prior to pushing to production, but yes, that's my
> next step.
>
> I didn't see anything that stood out in the release notes. It would make
> life easier if this was reproduceable...
>
> -----Original Message-----
> From: Gary Gregory [mailto:[email protected]]
> Sent: Thursday, November 17, 2016 12:31 PM
> To: Commons Users List
> Subject: Re: [net] FTPClient Transfer succeeds but fails in cleanUp
>
> Can you try version 3.5?
>
> Gary
>
> On Thu, Nov 17, 2016 at 9:19 AM, Seganti, Michael (DOF) <
> [email protected]> wrote:
>
>> I'm having a sporadic issue transferring files to a Mainframe, was
>> wondering if anyone else has run into this. I'm trying to figure out
>> if this is a Mainframe issue, or a net commons issue.
>>
>> I'm using Java 1.7 and commons-net-3.3.jar
>>
>> The file transfer completes, but it looks like either the 200 OK is
>> sent over the wrong connection, or is not properly being parsed and
>> is causing the cleanup to fail.
>>
>> According to the reply: IBM FTP CS V2R2 is the version where the
>> issue occurs.
>>
>> I have a second mainframe that is running IBM FTP CS V1R13 that
>> doesn't seem to have this issue
>>
>> In a normal transfer, you would see:
>>
>> 250 Transfer completed successfully.
>> File Transfer Completed Successfully at: 2016-11-17
>> 04:30:51.569 QUIT
>> 221 Quit command received. Goodbye.
>>
>> This is what we are getting on the failures:
>>
>> 250 Transfer completed successfully.
>> 200 OK
>>
>> File Transfer Failed at: 2016-11-17 04:36:50.246 Error
>> Occurred Transmitting File to Remote System, aborting...
>>
>> Read timed out
>>
>> Here's the full log from the failed transfer:
>>
>> Using FTPS Connection with Protocol: TLS and Explicit Security
>> Request
>> 220-FTPSECR1 IBM FTP CS V2R2 at MVSP.XXX, 09:27:29 on 2016-11-17.
>> 220 Connection will close if idle for more than 5 minutes.
>> AUTH TLS
>> 234 Security environment established - ready for negotiation
>> Connected to mvsp.xxx on xxxx PBSZ 0
>> 200 Protection buffer size accepted
>> PROT P
>> 200 Data connection protection set to private USER *******
>> 331 Send password please.
>> PASS *******
>> 230 C836PTS is logged on. Working directory is "C836PTS.".
>> PWD
>> 257 "'C836PTS.'" is working directory.
>>
>> Using a Packet Size of: 4096
>> Control KeepAliveTimeOut: 60
>> Control KeepAliveReplyTimeOut: 5000
>> FEAT
>> 211- Extensions supported
>> AUTH TLS
>> PBSZ
>> PROT
>> 211 End
>>
>> ...FTPClientUtil Connected Successfully!
>> SITE TRAIL
>> 200 SITE command was accepted
>> SITE CYLINDERS PRIMARY=2100 SECONDARY=250
>> 200 SITE command was accepted
>> SITE RECFM=FB LRECL=1915 BLKSIZE=30640
>> 200 SITE command was accepted
>> SITE CONDDISP=delete
>> 200 SITE command was accepted
>>
>>
>> Sending:
>> Local: d:\LargeExtracts\full.txt
>> Remote: 'A836.ORG.CAMA2.FULL'
>> Starting at: 2016-11-17 09:27:34.798 Total Bytes To
>> Send: 2101359588 PASV
>> 227 Entering Passive Mode (xx,xx,x,xx,xx,xx) STOR
>> 'A836.ORG.CAMA2.FULL'
>> 125 Storing data set A836.ORG.CAMA2.FULL
>> 2016-11-17 09:27:47.721 - 5 Percent Complete, Total Bytes Transferred:
>> 105070592
>> 2016-11-17 09:28:01.190 - 10 Percent Complete, Total Bytes Transferred:
>> 210137088
>> ...
>> 2016-11-17 09:32:21.127 - 100 Percent Complete, Total Bytes Transferred:
>> 2101359588
>> 250 Transfer completed successfully.
>> 200 OK
>>
>> File Transfer Failed at: 2016-11-17 09:32:26.127
>> Error Occurred Transmitting File to Remote System, aborting...
>>
>> Read timed out
>> java.net.SocketTimeoutException: Read timed out
>> at java.net.SocketInputStream.socketRead0(Native Method)
>> at java.net.SocketInputStream.read(Unknown Source)
>> at java.net.SocketInputStream.read(Unknown Source)
>> at sun.security.ssl.InputRecord.readFully(Unknown Source)
>> at sun.security.ssl.InputRecord.read(Unknown Source)
>> at sun.security.ssl.SSLSocketImpl.readRecord(Unknown
>> Source)
>> at
>> sun.security.ssl.SSLSocketImpl.readDataRecord(Unknown
>> Source)
>> at sun.security.ssl.AppInputStream.read(Unknown Source)
>> at sun.nio.cs.StreamDecoder.readBytes(Unknown Source)
>> at sun.nio.cs.StreamDecoder.implRead(Unknown Source)
>> at sun.nio.cs.StreamDecoder.read(Unknown Source)
>> at java.io.InputStreamReader.read(Unknown Source)
>> at java.io.BufferedReader.fill(Unknown Source)
>> at java.io.BufferedReader.readLine(Unknown Source)
>> at java.io.BufferedReader.readLine(Unknown Source)
>> at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:314)
>> at org.apache.commons.net.ftp.FTP.__getReplyNoReport(FTP.
>> java:304)
>> at org.apache.commons.net.ftp.FTPClient$CSL.cleanUp(
>> FTPClient.java:3697)
>> at org.apache.commons.net.ftp.FTPClient._storeFile(
>> FTPClient.java:669)
>> at org.apache.commons.net.ftp.FTPClient.__storeFile(
>> FTPClient.java:624)
>> at org.apache.commons.net.ftp.
>> FTPClient.storeFile(FTPClient.java:1976)
>> at gov.nyc.ftputils.FTPConnection.sendFile(
>> FTPConnection.java:695)
>> at gov.nyc.ftputils.FTPPropertyProcessor.putFile(
>> FTPPropertyProcessor.java:54)
>> at gov.nyc.ftputils.FTPPropertyProcessor.putFile(
>> FTPPropertyProcessor.java:96)
>> at
>> gov.nyc.ftputils.FTPPropertyProcessor.processFileProps(
>> FTPPropertyProcessor.java:126)
>> at gov.nyc.ftputils.FTPClientUtil.main(
>> FTPClientUtil.java:378)
>> FTPClientUtil has Failed
>> QUIT
>> 221 Quit command received. Goodbye.
>>
>>
>> Here's a code snippet of the transfer code:
>>
>> try{
>> if (trace){
>>
>> System.out.println("Total Bytes To Send: " + localFile.length());
>> CopyStreamListener
>> listener = new ProgressStreamListener(localFile.length());
>>
>> ftp.setCopyStreamListener( listener);
>> }
>>
>> InputStream input = new FileInputStream(localFileName);
>> boolean result = ftp.storeFile(tempRemoteFileName, input);
>> input.close();
>>
>> if (result){
>> System.out.println("\tFile Transfer
>> Completed Successfully at: " + sdf.format(Calendar.
>> getInstance().getTime()));
>> ...
>> }
>> else
>> System.out.println("\tFile Transfer
>> Failed
>> at: " + sdf.format(Calendar.getInstance().getTime()));
>>
>> return result;
>> }catch (CopyStreamException cse){
>> System.err.println("\n\tFile Transfer Failed at: " +
>> sdf.format(Calendar.getInstance().getTime()) + ", Failure occurred
>> after " + cse.getTotalBytesTransferred() + " Bytes Transferred");
>> System.err.println("Error Occurred Transmitting File
>> to Remote System, aborting...\n");
>> cse.printStackTrace(System.err);
>> System.err.println("\n\nIOException Stack Trace that
>> Caused the Error:\n");
>> cse.getIOException().printStackTrace(System.err);
>> return false;
>> }catch (IOException ioe){
>> System.err.println("\n\tFile Transfer Failed at: " + sdf.format(Calendar.
>> getInstance().getTime()));
>> System.err.println("Error Occurred Transmitting File
>> to Remote System, aborting...\n");
>> System.err.println(ioe.getMessage());
>> ioe.printStackTrace(System.err);
>> return false;
>> }catch (Exception e){
>> System.err.println("\n\tFile Transfer Failed at: " +
>> sdf.format(Calendar.getInstance().getTime()));
>> System.err.println("Error Occurred Transmitting File
>> to Remote System, aborting...\n");
>> e.printStackTrace(System.err);
>> return false;
>> }
>> }
>>
>>
>> Thanks,
>>
>> Mike...
>>
>
>
>
> --
> E-Mail: [email protected] | [email protected] Java Persistence
> with Hibernate, Second Edition
> <https://www.amazon.com/gp/product/1617290459/ref=as_li_tl?ie=UTF8&cam
> p=1789&creative=9325&creativeASIN=1617290459&linkCode=as2&tag=garygreg
> ory-20&linkId=cadb800f39946ec62ea2b1af9fe6a2b8>
>
> <http:////ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=am2&o=1&a=
> 1617290459>
> JUnit in Action, Second Edition
> <https://www.amazon.com/gp/product/1935182021/ref=as_li_tl?ie=UTF8&cam
> p=1789&creative=9325&creativeASIN=1935182021&linkCode=as2&tag=garygreg
> ory-20&linkId=31ecd1f6b6d1eaf8886ac902a24de418%22>
>
> <http:////ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=am2&o=1&a=
> 1935182021>
> Spring Batch in Action
> <https://www.amazon.com/gp/product/1935182951/ref=as_li_tl?ie=UTF8&cam
> p=1789&creative=9325&creativeASIN=1935182951&linkCode=%7B%7BlinkCode%7
> D%7D&tag=garygregory-20&linkId=%7B%7Blink_id%7D%7D%22%3ESpring+Batch+i
> n+Action>
> <http:////ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=am2&o=1&a=
> 1935182951>
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
