Hi Koji, Thanks for the quick response. I have set the batch size to 1 indeed, and the flow you describe works, but my problem is a bit more complex. I'll try to show it with an example:
In this case Node 1 is parsing a flow file (indicated by the X in the connection between FetchS3Object and Parse). Both connections have a backpressure threshold of 1, but because the object is already fetched, the first connection is empty and can thus be filled. This means that, if a new item becomes available in the queue, both of the following cases can happen with equal probability: I'd like to force the second case to happen, because node 2 has more resources available. I hope this explains the situation a bit better. So basically I want the backpressure to occur based on a threshold on the whole flow, not an individual connection. I haven't found a way to do this up to this point. Hopefully you have an idea how to achieve this. Regards, Bas -- View this message in context: http://apache-nifi-users-list.2361937.n4.nabble.com/Problem-when-using-backpressure-to-distribute-load-over-nodes-in-a-cluster-tp863p877.html Sent from the Apache NiFi Users List mailing list archive at Nabble.com.
