[ https://issues.apache.org/jira/browse/NIFI-8483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513974#comment-17513974 ]
Josef Zahner commented on NIFI-8483: ------------------------------------ We have upgraded to NiFi 1.15.3 and it seems that the issue is gone. > Restart NiFi - duplicate SFTP flowfiles > --------------------------------------- > > Key: NIFI-8483 > URL: https://issues.apache.org/jira/browse/NIFI-8483 > Project: Apache NiFi > Issue Type: Bug > Affects Versions: 1.13.2 > Environment: Java 1.8.0_282, CentOS 7, 8-Node Cluster > Reporter: Josef Zahner > Priority: Major > Attachments: SFTP_failure.png > > > Since the upgrade from NiFi 1.11.4 to 1.13.2 we faced an issue with the > FetchSFTP & PutSFTP processors. We have a 8-Node NiFi cluster. Pattern is > always ListSFTP (tracking timestamp) - FetchSFTP (and delete) and PutSFTP. > If we do a restart of NiFi and NiFi comes back, we sometimes see flowfiles > for FetchSFTP (not found) and PutSFTP (already present on disk) which have > been processed successfully and have been stored already. So in fact we see > flowfiles in a failure queue which have been save to disk with PutSFTP, which > should never happen. The files are always small (a few MBs) and the network > connectivity is insanely fast. The cluster shutdown is always before the > grace period runs out. The attached screeshot shows an example where the > FetchSFTP and the PutSFTP failure queue has files. Especially for the > FetchSFTP this shouldn't be possible and if I do a restart with the command > below, I would expect that within the grace period the processor has been > stopped and it can't be processed twice. > {code:java} > /opt/nifi/bin/nifi.sh restart > {code} > At the moment we have no clue where the issue comes from and why it happens, > so I can't provide an exact scenario to reproduce it. I only know that it > sometimes happens after a restart of our 8-node cluster. -- This message was sent by Atlassian Jira (v8.20.1#820001)