Check state managent <https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.3.0/org.apache.nifi.processors.standard.ListSFTP/index.html> :
After performing a listing of files, the timestamp of the newest file is stored. This allows the Processor to list only files that have been added or modified after this date the next time that the Processor is run. State is stored across the cluster so that this Processor can be run on Primary Node only and if a new Primary Node is selected, the new node will not duplicate the data that was listed by the previous Primary Node. Try to copy & paste the component. It will be fetch again. On Mon, 2 Oct 2017 at 16:02 Mark Payne <marka...@hotmail.com> wrote: > Hey Jeremy, > > I was looking into this recently. I created a ListSFTP -> FetchSFTP flow. > Intentionally entered the wrong > password for FetchSFTP and started it. It did give the expected "Auth > failed" error message. When I changed > the password to the correct password and started it again, it did continue > on successfully. I am also definitely > seeing logs about the processor being stopped & started. > > Any other details that may be of interest in helping me to reproduce the > behavior you're seeing? > > Thanks > -Mark > > > On Aug 24, 2017, at 2:04 PM, Jeremy Farbota <jfarb...@payoff.com> wrote: > > I have a ListSFTP process that I've been using for a few months where a > partner posts some large files, I turn it on, then ListSFTP gets all the > filenames so a FetchSFTP can download them prior to other steps and > notification events later on. > > I had an issue this month initially because a password was changed. The > processor was unable to access the server (however it did not alert me to > that issue). Once I fixed the password, the primary node where this was > running still was not finding the new files. I switched to "All Nodes" and > one of the nodes was able to access the server. Unfortunately I had not yet > updated the password on the FetchSFTP processor so it failed to download. > When I fixed the password on that FetchSFTP (with 20 files still in the > queue), it would not run to download the files. > > Due to time constraints, I built another standalone service, moved the > template there, and it ran fine on that new, standalone NiFi on a different > server. > > I am able to SSH and SFTP into this SFTP server from the NiFi nodes in > question in the command line. I can list files from the command line as > well. I'm not being blacklisted by the host SFTP. > > I've restarted the NiFi nodes to see if that would help. It did not. > > I do not see the FetchSFTP service starting and stopping in nifi-app.log > when I turn it on and off. > > Another issue is that there is no failure or alert with the bad password. > If it fails, there is no response. Shouldn't there be an error if I put a > bad host or an "access denied" if I put in the wrong password? I ended up > figuring out the password was wrong by using a different client to figure > out why the NiFi processors were not responding. > > Any advice on how to get my ListSFTP and FetchSFTP responding again on my > main cluster? I'm hoping to continue using ListSFTP for other things on > prod but the processors are not able to be configured or ran at this time > on that cluster. When I try an create new processes, they do not respond. > > Thanks in advance! > > > [image: Payoff, Inc.] > *Jeremy Farbota* > Software Engineer, Data > Payoff, Inc. > > jfarb...@payoff.com > > > >
