+1 from me.
I’ve experimented with both methods. The simplicity of a PackageFlowfile
straight up 1:1 is convenient and straightforward.
MergeContent on the other hand can be difficult to understand and tweak
appropriately to gain desired results/throughput.
On Sep 8, 2023 at 10:14 AM -0400, Joe Witt <[email protected]>, wrote:
> Ok. Certainly simplifies it but likely makes it applicable to larger
> flowfiles only. The format is meant to allow appending and result in large
> sets of flowfiles for io efficiency and specifically for storage as the
> small files/tons of files thing can cause poor performance pretty quickly
> (10s of thousands of files in a single directory).
>
> But maybe that simplicity is fine and we just link to the MergeContent
> packaging option if users need more.
>
> On Fri, Sep 8, 2023 at 7:06 AM Michael Moser <[email protected]> wrote:
>
> > I was thinking 1 file in -> 1 flowfile-v3 file out. No merging of multiple
> > files at all. Probably change the mime.type attribute. It might not even
> > have any config properties at all if we only support flowfile-v3 and not v1
> > or v2.
> >
> > -- Mike
> >
> >
> > On Fri, Sep 8, 2023 at 9:56 AM Joe Witt <[email protected]> wrote:
> >
> > > Mike
> > >
> > > In user terms this makes sense to me. Id only bother with v3 or whatever
> > is
> > > latest. We want to dump the old code. And if there are seriously older
> > > versions v1,v2 then nifi 1.x can be used.
> > >
> > > The challenge is that you end up needing some of the same complexity in
> > > implementation and config of merge content i think. What did you have in
> > > mind for that?
> > >
> > > Thanks
> > >
> > > On Fri, Sep 8, 2023 at 6:53 AM Michael Moser <[email protected]> wrote:
> > >
> > > > Devs,
> > > >
> > > > I can't find if this was suggested before, so here goes. With the
> > demise
> > > > of PostHTTP in NiFi 2.0, the recommended alternative is to
> > MergeContent 1
> > > > file into FlowFile-v3 format then InvokeHTTP. What does the community
> > > > think about supporting a new PackageFlowFile processor that is simple
> > to
> > > > configure (compared to MergeContent!) and simply packages flowfile
> > > > attributes + content into a FlowFile-v[1,2,3] format? This would also
> > > > offer a simple way to export flowfiles from NiFi that could later be
> > > > re-ingested and recovered using UnpackContent. I don't want to submit
> > a
> > > PR
> > > > for such a processor without first asking the community whether this
> > > would
> > > > be acceptable.
> > > >
> > > > Thanks,
> > > > -- Mike
> > > >
> > >
> >
