Hi Koji, thanks for the explanation, I checked the NIFI documentation you provided, do you mean I should use the FIFO prioritizer in my case? Because as you mentioned the FlowFiles would be put back into their original positions, so as I understand using FIFO should make the FlowFiles in consistent order.
Regards, Ben 2017-07-10 17:06 GMT+08:00 Koji Kawamura <ijokaruma...@gmail.com>: > Hi, > > I think it puts back a FlowFile to its original position but update > queued date as implemented here: > https://github.com/apache/nifi/blob/master/nifi-nar- > bundles/nifi-framework-bundle/nifi-framework/nifi-framework- > core/src/main/java/org/apache/nifi/controller/repository/ > StandardProcessSession.java#L1851 > > In order to pull FlowFiles from a queue in consistent order, you need > to specify a prioritizer. > https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#prioritization > > I'm just curious about the functionality you added. Wait processor has > 'Releasable FlowFile Count' and it could be used to make a batch of > FlowFiles wait and go. Or Notify's 'Signal Counter Delta' could be > useful, too. > > Regards, > Koji > > On Mon, Jul 10, 2017 at 4:43 PM, 尹文才 <batman...@gmail.com> wrote: > > Hi guys, I have written a customized processor whose functionality is > > similar to the NIFI's Wait processor, the difference is my processor > needs > > to wait a batch of data and when the batch end flag is found, it will > > transfer the batch of data to destinations. > > > > I checked the source code of Wait processor and also transferred the > > flowfiles to Relationship.SELF which is the incoming queue when the batch > > of data is not yet complete. The problem I found was sometimes I could > see > > the sequence of the FlowFiles transferred from my processor to > destinations > > were not in order. > > I then added sequence attribute(number starting from 1) to all FlowFiles > > coming into my processor and I could verify that this problem happen from > > time to time, but I couldn't find the stable way to reproduce it. > > > > My question is how does NIFI handle the FlowFile when it's being > > transferred to Relationship.SELF, does it put back to its original > position > > in the incoming queue? Thanks. > > > > Regards, > > Ben >
