+1 to Mark's idea. Having both attributes might be nice and probably
doesn't hurt anything.
The convenience of having a relative path is that you can munge it or add
prefixes to it more easily via the expression language. An absolute path
would somewhat be more difficult to work with, if you wanted to manipulate
the root directory part.
On Wed, Nov 25, 2015 at 2:00 PM, Mark Payne <marka...@hotmail.com> wrote:
> Tony,
>
> I would recommend that ListFile add both 'path' and 'absolute.path'. The
> 'path' would be relative to the base directory being listed.
> For example, if ListFile is configured to list files from /data/nifi/in
> and recurse subdirectories, and it finds a file named:
> /data/nifi/in/123/myfile.txt
> then i would expect the following attributes:
>
> absolute.path = /data/nifi/in/123
> path = ./123
> filename = myfile.txt
>
> Thanks
> -Mark
>
>
> > On Nov 25, 2015, at 1:56 PM, Tony Kurc <trk...@gmail.com> wrote:
> >
> > All,
> > Joe and I commented on NIFI-631 that it didn't "just work" when wiring
> the
> > processors together. ListFile was populating the attributes as
> > described in CoreAttributes.java
> > [1] (path being relative to the input directory, and absolute being the
> > full path). FetchFile was using ${path}/${filename} as the default, which
> > wouldn't grab the directory. I'm puzzled as to what the correct behavior
> > should be. The description of path said it is relative ... relative to
> > what? ListHDFS appears to state path is absolute [2] [3], and I expect we
> > should have consistent behavior between ListHDFS and ListFile.
> >
> > So, I guess I'm not sure what guidance to give on a review of NIFI-631.
> > Should the default of FetchFile be changed to
> ${absolute.path}/${filename}
> > (which may be inconsistent with other List/Fetch processor combos), or
> > should ListFile be changed to have path be absolute?
> >
> > [1]
> >
> https://github.com/apache/nifi/blob/master/nifi-commons/nifi-utils/src/main/java/org/apache/nifi/flowfile/attributes/CoreAttributes.java
> > [2]
> >
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/ListHDFS.java#L79
> > [3]
> >
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/ListHDFS.java#L442
>
>
