Hello all, I'm kind of new to developing Nifi, though I've been doing some pretty in depth stuff and some advanced database queries. My question is in regarding the queues between processor, I want to limit a queue to... say 2000, how would I go about doing that? Or better yet, how would I tell the processor generating the queue to only put a max of 2000 files into the queue?
Allow me to explain with a scenario: We are doing data migration from one database to another. -Processor A is generating a queue consumed by Processor B -Processor A is taking configuration and generating SQL queries in 1000 record chunks so that Processor B can insert them into a new database. Given the size of the source database, Processor A can potentially generate hundreds of thousands of files. Is there a way for Processor A to check it's down stream queue for the queue size? How would I get Processor A to only put 2000 files into the queue at any given time, so that Processor A can continue running but wait for room in the queue? Thank you in advance. -- *Shaine Berube*
