[
https://issues.apache.org/jira/browse/NIFI-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16676263#comment-16676263
]
Koji Kawamura commented on NIFI-5764:
-------------------------------------
[~alfredo.deluca] When I tested SFTP processors, connections are properly
closed after execution. And I confirmed the number of connection stayed
reasonable, didn't increased.
We need more info. Would you be able to get the number of connections to the
SFTP server created by NiFi? That can be retrieved by a command like:
{code}
# 2222 is the SFTP server port number
$ netstat -an |grep 2222
# then you'd see something like
tcp4 0 0 127.0.0.1.64709 127.0.0.1.2222 ESTABLISHED
# and you can check if the client is a nifi process
$ lsof -i tcp:64709
{code}
If there are many connections established by NiFi process, then the next step
would be taking thread dump by:
{code}
$ ./bin/nifi.sh dump
{code}
> Allow ListSftp connection parameter
> -----------------------------------
>
> Key: NIFI-5764
> URL: https://issues.apache.org/jira/browse/NIFI-5764
> Project: Apache NiFi
> Issue Type: Wish
> Components: Extensions
> Affects Versions: 1.6.0
> Reporter: dav
> Priority: Critical
> Labels: SFTP, customization, sftp
>
> ListSftp and other Sftp processors should be able to add parameters
> (like [-B buffer_size] [-b batchfile] [-c cipher]
> [-D sftp_server_path] [-F ssh_config] [-i identity_file] [-l limit]
> [-o ssh_option] [-P port] [-R num_requests] [-S program]
> [-s subsystem | sftp_server] host
> sftp [user@]host[:file ...]
> sftp [user@]host[:dir[/]]
> sftp -b batchfile [user@]host)
> in order to edit the type of connection on Sftp Server.
> For instance, I have this error on nifi:
> 2018-10-29 11:06:09,462 ERROR [Timer-Driven Process Thread-5]
> SimpleProcessLogger.java:254
> ListSFTP[id=766ac418-27ce-335a-5b13-52abe3495592] Failed to perform listing
> on remote host due to java.io.IOException: Failed to obtain connection to
> remote host due to com.jcraft.jsch.JSchException: Auth fail: {}
> java.io.IOException: Failed to obtain connection to remote host due to
> com.jcraft.jsch.JSchException: Auth fail
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getChannel(SFTPTransfer.java:468)
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:192)
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:156)
> at
> org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:105)
> at
> org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:401)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1147)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:175)
> at
> org.apache.nifi.controller.scheduling.QuartzSchedulingAgent$2.run(QuartzSchedulingAgent.java:140)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: com.jcraft.jsch.JSchException: Auth fail
> at com.jcraft.jsch.Session.connect(Session.java:519)
> at com.jcraft.jsch.Session.connect(Session.java:183)
> at
> org.apache.nifi.processors.standard.util.SFTPTransfer.getChannel(SFTPTransfer.java:448)
> ... 15 common frames omitted
> This can be avoided by connect to Sftp server with this string:
> *sftp -o “controlmaster auto” username@sftp_server*
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
