Ok. Certainly simplifies it but likely makes it applicable to larger flowfiles only. The format is meant to allow appending and result in large sets of flowfiles for io efficiency and specifically for storage as the small files/tons of files thing can cause poor performance pretty quickly (10s of thousands of files in a single directory).
But maybe that simplicity is fine and we just link to the MergeContent packaging option if users need more. On Fri, Sep 8, 2023 at 7:06 AM Michael Moser <[email protected]> wrote: > I was thinking 1 file in -> 1 flowfile-v3 file out. No merging of multiple > files at all. Probably change the mime.type attribute. It might not even > have any config properties at all if we only support flowfile-v3 and not v1 > or v2. > > -- Mike > > > On Fri, Sep 8, 2023 at 9:56 AM Joe Witt <[email protected]> wrote: > > > Mike > > > > In user terms this makes sense to me. Id only bother with v3 or whatever > is > > latest. We want to dump the old code. And if there are seriously older > > versions v1,v2 then nifi 1.x can be used. > > > > The challenge is that you end up needing some of the same complexity in > > implementation and config of merge content i think. What did you have in > > mind for that? > > > > Thanks > > > > On Fri, Sep 8, 2023 at 6:53 AM Michael Moser <[email protected]> wrote: > > > > > Devs, > > > > > > I can't find if this was suggested before, so here goes. With the > demise > > > of PostHTTP in NiFi 2.0, the recommended alternative is to > MergeContent 1 > > > file into FlowFile-v3 format then InvokeHTTP. What does the community > > > think about supporting a new PackageFlowFile processor that is simple > to > > > configure (compared to MergeContent!) and simply packages flowfile > > > attributes + content into a FlowFile-v[1,2,3] format? This would also > > > offer a simple way to export flowfiles from NiFi that could later be > > > re-ingested and recovered using UnpackContent. I don't want to submit > a > > PR > > > for such a processor without first asking the community whether this > > would > > > be acceptable. > > > > > > Thanks, > > > -- Mike > > > > > >
