: dev@nifi.apache.org
Subject: Re: SplitRecord behaviour
If you increase the concurrent tasks on PublishKafka then you are
right that you could publish multiple records at the same time, but I
suspect that the overhead of doing the split will cancel out any gains
from publishing in parallel
t;
> -Original Message-
> From: Bryan Bende
> Sent: Friday, March 1, 2019 7:52 PM
> To: dev@nifi.apache.org
> Subject: Re: SplitRecord behaviour
>
> Hello,
>
> Flow files are not transferred until the session they came form is committed.
> So imagine we periodic
one shot instead of sending one
record at a time.
Thanks,
Hemantha
-Original Message-
From: Bryan Bende
Sent: Friday, March 1, 2019 7:52 PM
To: dev@nifi.apache.org
Subject: Re: SplitRecord behaviour
Hello,
Flow files are not transferred until the session they came form is committed
You can call transfer for each segment while processing the incoming
stream, its just that the real transfer won't actually happen until
commit is called.
Most processors extend AbstractProcessor so commit is called for you
at the end, but you could choose to manage the session yourself and
call c
Bryan,
So the best practice when segmenting is to
- build your segments as a list while processing the incoming stream
- then after send them all to the relationship
right?
On March 1, 2019 at 09:21:46, Bryan Bende (bbe...@gmail.com) wrote:
Hello,
Flow files are not transferred until the sess
Hello,
Flow files are not transferred until the session they came form is
committed. So imagine we periodically commit and some of the splits
are transferred, then half way through a failure is encountered, the
entire original flow file will be reprocessed, producing some of the
same splits that w
