MergeContent with a fragment strategy will bundle the fragmented flowfiles (children) together based on parent attributes (directory). Each fragment will have a unique fragment.index and common fragment.count ( total siblings). The fragments are bundled together based on the directory (fragment.identifier/segment.original.filename). Once all fragments are processed and accounted for at MergeContent, a single flowfile will be sent as 'merged' and will indicate that you have successfully processed all of the files in the directory. This merged flowfile can initiate another task to apply the per-directory image tags to each file.
If a file fails the test and does not arrive at the MergeContent processor, the fragments will not merge and will be routed to failure. The splitText processor is used in conjunction with the MergeContent Processor, and will fill out the fragment.index/count/identifier attributes automatically. -Lee On Sat, Jan 28, 2017 at 2:55 PM, Wshrdryr Corp <[email protected]> wrote: > Hello all. > > I'm new to Nifi so please forgive me if this is an easily answered > question. > > I have an ETL job which has to process directories full of image files. I > have figured out how to use ListFile to find directories as they are added, > but I need to sanity-check each file in a directory by checking its > attributes and if and only if each file passes this test, I need to apply > some per-directory image tags to each file. > > How do I know when I have processed every file in a directory? As part of > processing I can create a new FlowFile for each file in a directory, but > how can I annotate the 'parent' flowfile that for example only 11 of 12 > 'children' passed so it can be routed as failed? > > >
