Ravi Please use the 'File Filter' property of ListFile to control ignoring filenames until they no longer end in 'part'.
Thanks On Wed, Dec 6, 2017 at 2:14 AM, Ravi Papisetti (rpapiset) <rpapi...@cisco.com> wrote: > Hi, > > > > We are using Apache NiFi 1.3.0 > > > > We have a process flow to copy files from NFS to HDFS (with processors > ListFile, FetchFile and PutHDFS) > > > > In the NiFi process flow, ListFile is configured to listen to a directory on > NFS. When a file (ex: x.csv) is being copied from a windows machine to NFS > (while transfer is in the middle), a part file(x.csv.part) is created at NFS > until transfer is complete. > > > > ListFile has picked up this x.csv.part file and fetchFile picked up this to > transfer to HDFS, didn’t update the file name back to x.csv in HDFS when > transfer is complete. > > > > But, in case a file from linux file system, while file copy to NFS is in > progress it created (.x.csv) and when transfer is complete, at both NFS and > HDFS, filename is updated to x.csv (from .x.csv). > > > > Any thought how we can configure ListFile not to pickup these part files or > any configurations in NiFi that fixes file names for these windows part > files? > > > > Appreciate your help. > > > > Thanks, > > Ravi Papisetti
