My reading of the manual on this suggests that the batch size is determined by the contents of the queue upstream of the input node, not the downstream queue.
"... the Input Ports will continue to ingest all data until all of the queues feeding the Input Ports have been emptied." Perhaps putting a ControlRate processor further upstream can be used to only pass flow files in batches of a particular size or number at regular intervals to the input port? Regards Steve Hindmarch From: Alexis Sarda-Espinosa <[email protected]> Sent: Monday, June 3, 2024 10:41 AM To: [email protected] Subject: Single Batch Per Node and back pressure You don't often get email from [email protected]<mailto:[email protected]>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> Hello, I am using NiFi 1.26.0. I created a processor group and configured its FlowFile Concurrency as "Single Batch Per Node". The group has a single input port that connects to only one processor. I configured the queue between the input port and the processor to generate back pressure, hoping that if the Back Pressure Object Threshold is X, the maximum batch size per node would be equal to X, but based on my experiments it seems that the input port will consume as much data as it can without considering the back pressure from the queues connected to it. Is this expected? Regards, Alexis.
