Reed Bertolotti created NET-739:
-----------------------------------
Summary: FTPSClient storeFile() fails with "451 Transfer aborted"
on TLS 1.3 (with low latency and pure-ftpd)
Key: NET-739
URL: https://issues.apache.org/jira/browse/NET-739
Project: Commons Net
Issue Type: Bug
Reporter: Reed Bertolotti
*Summary:*
{{FTPSClient.storeFile()}} intermittently fails to complete file uploads when
using *TLSv1.3* against a strict FTP server (Pure-FTPd). The file data is
successfully transmitted, but the transfer ends with a {{451 Transfer aborted}}
error from the server instead of the expected {{{}226 Transfer complete{}}}.
This appears to be a timing race condition in the data channel closure
sequence. The issue is highly sensitive to network latency: it is reproducible
in *low-latency environments* (e.g. FTP client and FTP server in same AWS Same
Region), but disappears if network latency is increased (e.g. different AWS
regions or adding latency with tc command)
----
*Environment:*
* *Library:* Apache Commons Net 3.9.0 (also reproduced on 3.10.0, 3.11.0,
3.12.0)
* *Java:* JDK 17
* *OS:* Amazon Linux 2023
* *Server:* Pure-FTPd 1.0.52 (TLS 1.3 enabled)
** Relevant Build Args:
*** {{--with-tls}} (Standard TLS support)
** Relevant Runtime Args:
*** {{-Y 3}} (Enforce TLS for Control & Data)
*** {{{}--tlsciphersuite=HIGH:MEDIUM:!TLSv1:!TLSv1.1:!aNULL{}}}{{{}{}}}
*Comparison:*
* *Fails:* Commons Net {{storeFile()}} + TLS 1.3 + low latency -> Returns
{{false}} (451).
* *Works:* Commons Net {{storeFile()}} + TLS 1.2 -> Returns {{true}} (226).
* *Works:* Commons Net {{storeFile()}} + TLS 1.3 + high latency -> Returns
{{true}} (226).
* *Works:* FileZilla FTP Client + TLS 1.3
* *Works:* Manual implementation using {{storeFileStream()}}
* {{}}
|*Scenario*|*Client/Server Location*|*Protocol*|*Latency*|*Result*|
|*AWS Intra-Region*|*Same Region (e.g. us-west-2)*|*TLSv1.3*|*Low* |*FAILS
(451)*|
|AWS Inter-Region|Different Regions|TLSv1.3|Not low|WORKS (226)|
|Artificial Delay|Same Region + {{tc}} delay|TLSv1.3|Not low (artificial)|WORKS
(226)|
|TLS 1.2 Downgrade|Same Region|TLSv1.2|Any (high or low)|WORKS (226)|
{_}Note: Attempts to reproduce this using {{localhost}} failed. Reproduction
seems to require a public IP / two servers{_}{{{}{}}}
*Reproduction Code (Simplified):*
{code:java}
FTPSClient ftpClient = new FTPSClient(false);
ftpClient.setEnabledProtocols(new String[]{"TLSv1.3"}); // Issue specific to
TLS 1.3
ftpClient.addProtocolCommandListener(new PrintCommandListener(new
PrintWriter(System.out), true));
// Setup
ftpClient.configure(new FTPClientConfig(FTPClientConfig.SYST_UNIX));
ftpClient.setTrustManager(TrustManagerUtils.getAcceptAllTrustManager());
// IMPORTANT: Reproduction requires a real network path (cannot be localhost)
String serverPublicIp = "18.98.23.15";
ftpClient.connect(serverPublicIp, 21);
ftpClient.login(user, pass);
ftpClient.execPROT("P");
ftpClient.enterLocalPassiveMode();
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
// Upload
// Use a dummy 10KB payload in memory so this is runnable without external files
try (InputStream input = new ByteArrayInputStream(new byte[10240])) {
String remoteFilePath = "privateFileUpload_10KB.txt";
// Returns false. Server replies with 451.
boolean success = ftpClient.storeFile(remoteFilePath, input);
System.out.println("Success status: " + success);
}{code}
*Workaround / Additional Context:*
We found that we could avoid the error by avoiding {{storeFile()}} and manually
implementing the transfer using {{{}storeFileStream(){}}}. Specifically, using
{{input.transferTo(output)}} inside a try-with-resources block (to ensure
{{{}close(){}}}), followed by {{{}completePendingCommand(){}}}, results in a
successful transfer (226) in the same TLSv1.3/low latency environment where
{{storeFile()}} fails.
----
*Logs:*
*1. Client Protocol Log (Commons Net):* The client sends the data, but receives
a 451 failure immediately after the transfer.
{code:java}
...
PASV
227 Entering Passive Mode (18,98,23,15,218,245)
STOR //privateFileUpload_10KB.txt
150 Accepted data connection
451-Transfer aborted
451 0.084 seconds (measured here), 119.71 Kbytes per second
...{code}
*2. Server Log (Pure-FTPd):* The server acknowledges the upload size (10240
bytes) but flags the transfer as aborted, likely due to a socket closure race
condition.
{code:java}
[INFO] TLS: Enabled TLSv1.3 with TLS_AES_256_GCM_SHA384, 256 secret bits cipher
[DEBUG] 150 Accepted data connection
[NOTICE] privateFileUpload_10KB.txt uploaded (10240 bytes, 119.71KB/sec)
[DEBUG] 451-Transfer aborted {code}
----
It appears {{commons-net}} is closing the underlying socket or data stream too
aggressively for TLS 1.3 strictness. If {{commons-net}} closes the TCP socket
before the server has processed the TLS closure, strict servers like
{{pure-ftpd}} interpret this as a premature disconnection ("Transfer aborted").
Since this works on TLS 1.2 and high-latency connections, it suggests a timing
race between the Java client's TCP FIN and the TLS protocol shutdown.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
