[
https://issues.apache.org/jira/browse/CAMEL-25896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-25896:
--------------------------------
Fix Version/s: 4.24.0
> camel-ftp - Let sendNoop=false also skip the consumer's pre-poll noop (SFTP
> sends [email protected] on every poll)
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25896
> URL: https://issues.apache.org/jira/browse/CAMEL-25896
> Project: Camel
> Issue Type: Improvement
> Components: camel-ftp, camel-mina-sftp, camel-sftp
> Affects Versions: 4.18.5, 4.23.0
> Reporter: Johan Almqvist
> Priority: Minor
> Fix For: 4.24.0
>
>
> h3. Problem
> Before every poll, {{RemoteFileConsumer.connectIfNecessary()}} calls
> {{getOperations().sendNoop()}}. For the JSch-based sftp component,
> {{SftpOperations.sendNoop()}} calls {{Session.sendKeepAliveMsg()}}, which
> sends a {{[email protected]}} global request. The {{sendNoop}} option
> (CAMEL-8807) is read only by {{RemoteFileProducer}}, so a consumer cannot
> turn this off. The code is the same in 4.18.5, in 4.23.0 and on main, where
> {{RemoteFileConsumer}} now lives in camel-ftp-common.
> ProFTPD's mod_sftp does not support that request. It replies with a failure
> and writes one SFTPLog line for each one:
> {noformat}
> <timestamp> mod_sftp/<version>[<pid>]: client sent GLOBAL_REQUEST for
> '[email protected]', denying
> {noformat}
> With the default {{delay=500}}, that is about two lines a second, roughly
> 170,000 a day, for each route polling the server. The volume grows with every
> route, and it buries authentication failures and other real errors in the
> same log. mod_sftp writes the line for every global request it does not
> handle (the same code in 1.3.7 to 1.3.9, 1.3.10rc4 and master), so the server
> cannot silence it without turning SFTPLog off entirely.
> h3. For the JSch-based sftp component the keepalive does not change the
> outcome
> * {{Session.sendKeepAliveMsg()}} writes the request and returns without
> waiting for the reply, so it cannot tell a live server from a dead one.
> * {{SftpOperations.sendNoop()}} only sends it when {{isConnected()}} is true.
> Otherwise it returns false without sending anything.
> * If the write throws, {{sendNoop()}} returns false and
> {{connectIfNecessary()}} calls {{connect()}}. That returns at once while
> {{isConnected()}} is still true, so nothing reconnects at that point. If JSch
> has already marked the session disconnected, {{isConnected()}} alone triggers
> the same reconnect.
> * When the server closes the socket (the CAMEL-6591 case), JSch's reader
> thread marks the session disconnected, so {{isConnected()}} already returns
> false and {{sendNoop()}} sends nothing.
> * A session that dies without the client noticing is recovered as it is
> today: the poll fails, {{RemoteFilePollingConsumerPollStrategy.rollback()}}
> force-disconnects, and the next poll reconnects.
> So for an sftp consumer the result is the same as calling {{isConnected()}},
> plus one packet per poll.
> h3. Proposal
> Let {{sendNoop}} also apply to the consumer's pre-poll check. With
> {{sendNoop=false}}, the consumer relies on {{isConnected()}}:
> {code:java}
> protected void connectIfNecessary() {
> boolean isConnected = false;
> try {
> if (getEndpoint().getConfiguration().isSendNoop()) {
> isConnected = getOperations().sendNoop();
> } else {
> isConnected = getOperations().isConnected();
> }
> } catch (Exception ex) {
> // unchanged
> }
> // unchanged from here
> }
> {code}
> * The default stays {{true}}. The only users affected are those who already
> set {{sendNoop=false}} on a consumer URI, where it is ignored today (for
> example a URI shared between {{from()}} and {{to()}}). That is worth a line
> in the upgrade guide.
> * {{RemoteFileConsumer}} is shared, so this also applies to ftp, ftps and
> mina-sftp. There {{sendNoop()}} is a real round trip (FTP {{NOOP}}, MINA
> {{stat("/")}}). With {{sendNoop=false}}, a connection the server has closed
> is found only when the poll fails, and the poll strategy then reconnects on
> the next poll. That is the same trade-off {{sendNoop=false}} already makes
> for producers.
> * The {{@UriParam}} label changes from {{producer,advanced}} to
> {{common,advanced}} (as {{fastExistsCheck}} uses). The description and the
> {{setSendNoop()}} Javadoc cover the pre-poll check as well as the pre-write
> check. Regenerating updates the component JSON, the catalog and the endpoint
> DSL.
> * Tests: a consumer counterpart of the existing producer {{sendNoop=false}}
> test for ftp and sftp, and a unit test that with {{sendNoop=false}} the
> consumer calls {{isConnected()}} instead of {{sendNoop()}} and still
> reconnects when it returns false.
> h3. Alternatives considered
> * Going back to {{sendIgnore()}}: CAMEL-10149 moved away from it because
> ProFTPD closed the connection on {{SSH_MSG_IGNORE}}.
> * {{disconnect=true}}: no keepalive, but a full SSH handshake and login on
> every poll.
> * camel-mina-sftp: its {{sendNoop()}} is an SFTP {{stat("/")}} with no global
> request, but switching components is not always an option.
> * Changing {{SftpOperations.sendNoop()}} for everyone, either to return
> {{isConnected()}} or to make a real round trip. That would change the default
> for every SFTP user, so this issue proposes an opt-in that matches
> CAMEL-8807, but either would also solve it.
> h3. Workaround today
> The only way to stop the keepalive today is to subclass {{SftpOperations}},
> {{SftpEndpoint}} and {{SftpComponent}} and register the subclass as {{sftp}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
