Croway opened a new pull request, #30087:
URL: https://github.com/apache/camel/pull/30087

   [CAMEL-26999](https://issues.apache.org/jira/browse/CAMEL-26999) and 
[CAMEL-27000](https://issues.apache.org/jira/browse/CAMEL-27000): an FTP 
server, or the network path to it, that stops answering on an open connection 
(a hung server, a connection dropped by a firewall without a reset) costs 
`soTimeout` (5 minutes by default) again and again. Found with fault-injection 
tests; reproduced with an in-process fake FTP server (no Docker).
   
   ### CAMEL-26999: disconnect waits up to soTimeout for the reply to QUIT
   
   `FtpOperations.doDisconnect()` calls `client.logout()`, which in commons-net 
3.13.0 sends QUIT and reads the reply with the SO_TIMEOUT of the control 
connection, i.e. `soTimeout`. So:
   
   * `RemoteFileConsumer.doStop()` / `RemoteFileProducer.doStop()` hold 
`CamelContext.stop()` for `soTimeout` per connected endpoint, also with nothing 
in flight (the shutdown timeout does not bound it: the second stop from the 
caller waits on the lock of `BaseService.stop()`).
   * `forceDisconnect()` called the same `doDisconnect()`, so after a failed 
poll `RemoteFilePollingConsumerPollStrategy.rollback()` sent QUIT on the dead 
connection and waited `soTimeout` once more before the next poll could connect.
   
   Fix (in `FtpOperations` only):
   * `forceDisconnect()` closes the connection without QUIT: it is the hard 
disconnect of the `RemoteFileOperations` contract, used to recover from a 
failure (`SftpOperations.forceDisconnect()` already closes the session without 
waiting).
   * `doDisconnect()` still sends QUIT, but waits for the reply at most 2 s 
(`soTimeout` when it is shorter, also when `soTimeout=0`). A healthy server 
answers QUIT at once; the connection is closed either way.
   
   ### CAMEL-27000: after a read timeout, the rest of the batch runs on the 
dead connection
   
   A failed download does not leave `GenericFileConsumer.processBatch()`, and 
commons-net leaves the timed-out connection open (with the reply to RETR 
unread, so a late reply would be read as the reply to the next command). Each 
remaining file sent its PWD (`stepwise=true`) on it and waited `soTimeout`: 
recovery took (remaining files + 2) x `soTimeout` and one WARN per file. The 
next poll's NOOP waited once more, then `connect()` replaced the socket without 
closing it.
   
   Fix:
   * `FtpConsumer.handleException()` (camel-ftp; no change to the shared 
`RemoteFileConsumer`): when a file failed because the server did not answer in 
time (a `SocketTimeoutException` in the cause: `soTimeout` on the control 
connection, `timeout` on the data connection), force-disconnect (now without 
QUIT) and end the batch through `isBatchAllowed()`. `processBatch()` releases 
the remaining files, as when the consumer stops, and the next poll connects 
again and picks them up.
   * `FtpOperations.doConnect()` closes a client that is still connected before 
connecting again, so a connection whose NOOP timed out is not left open.
   
   Only a timeout ends the batch. Other I/O failures keep the behaviour of 
CAMEL-25915 (the NOOP before the next poll decides): a reset connection fails 
fast, and a local error such as a full disk does not drop a good connection 
(`RemoteFileConsumerSendNoopTest` covers both and is unchanged).
   
   The two fixes are in one PR because the batch fix relies on the forced 
disconnect not sending QUIT (otherwise it would still wait `soTimeout` for that 
reply), and both tests use the same new fake FTP server. Two commits, one per 
JIRA.
   
   ### Tests
   
   New unit tests with `FakeFtpServer`, an in-process FTP server (passive mode) 
that can stop answering on an open connection while it serves new ones:
   
   * `FtpDisconnectFromSilentServerTest` (CAMEL-26999): `CamelContext.stop()` 
with an idle consumer / producer whose server stopped answering (`soTimeout` 15 
s) returns in less than 10 s and sends QUIT; a failed poll (LIST timed out) 
does not send QUIT on the dead connection.
   * `FtpConsumerBatchAfterReadTimeoutTest` (CAMEL-27000): 10 files, the server 
stops answering at the RETR of the 3rd, before the reply or in the middle of 
the transfer: only `f-03` fails, nothing more is sent on the dead connection, 
it is closed, and all 10 files are delivered; a connection that stops answering 
between two polls is closed when the consumer connects again.
   
   Before the fix (main): all 6 fail, deterministically:
   * `stopIsNotHeldByAnIdleConsumerWhoseServerStoppedAnswering` / 
`...Producer...`: `CamelContext.stop() took 15002 ms with soTimeout 15000 ms`
   * `failedPollDoesNotSendQuitOnTheConnectionThatStoppedAnswering`: `expected: 
<[]> but was: <[QUIT]>`
   * `readTimeoutFailsOnlyTheFileBeingTransferred` (both stall modes): 
`expected: <[f-03.txt]> but was: <[f-03.txt, f-04.txt, ..., f-10.txt]>`
   * `connectionThatStoppedAnsweringIsClosedBeforeConnectingAgain`: the 
connection that stopped answering is still open 5 s after the reconnect
   
   After the fix: the 6 new tests pass (about 5 s per class), and all camel-ftp 
unit tests (188) and ITs (500, embedded FTP/SFTP servers) pass but one, 
`FtpProducerHealthCheckIT`: closing the old connection before connecting resets 
the reply string of commons-net, so a refused connect got the reason `null 
Connection refused`. The connect failure now uses an empty reply string, as 
before; `FtpProducerHealthCheckIT` and the connect, reconnect, disconnect and 
login-leak ITs pass again.
   
   ### Docs and upgrade guide
   
   * `ftp-component.adoc`: a new section "A server that stops answering", after 
"Concurrency". It is not in "About timeouts" because #30062 rewrites that 
section; it can move there once both are merged.
   * `camel-4x-upgrade-guide-4_24.adoc`: the wait for the reply to QUIT is 
bounded to 2 s, the forced disconnect sends no QUIT, and a timeout while 
retrieving a file ends the batch.
   
   _Claude Code on behalf of Croway_
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to