Jim, There's a better solution in NiFi. Right click on the connection between ListFile and FetchFile and select a cluster distribution strategy in options. That's it :)
Andrew On Tue, Jul 2, 2019, 7:37 AM James McMahon <jsmcmah...@gmail.com> wrote: > We would like to employ a DistributeLoad processor, restricted to run on > the primary node of our cluster. Is there a recommended approach employed > to efficiently distribute across nodes in the cluster? > > As I understand it, and using a FetchFile running in "all nodes" as the > first processor following the DistributeLoad, I can have it distribute by > round robin, next available, or load distribution service. Can anyone > provide a link to an example that employs the load distribution service? Is > that the recommended distribution approach when running in clustered mode? > > I am interested in maintaining load balance across my cluster nodes when > running at high flowfile volumes. Flow files will vary greatly in contents, > so I'd like to design with an approach that helps me balance processing > distribution. > > Thanks very much in advance. -Jim >
