CaptureChangeMySQLSelf is from CaptureChangeMySQL and I fix a small bug about binlog parsing. But i am sure there's no change that will affect the backpressure mechanism.
Actually I don't quite understand it. It is related with the implementation of the processor? In order to use the backpressure mechanism, the processor must do something? In my case, what it will happen if the queue continue to grow(for example to 100G)? Something bad will happen? Thanks, Lei wangl...@geekplus.com.cn From: josef.zahn...@swisscom.com Date: 2019-10-16 14:17 To: us...@nifi.apache.org; dev@nifi.apache.org Subject: Re: NiFi backpressure not work Hi Lei I assume it’s the same as for the ListSFTP. As I don’t know the CaptureChangeMySQLSelf processor I try to explain it for the ListSFTP. There with one single execution of the processor you get as much flowfiles as you have files on your disk. If it finds 1000 it sends out 1000 flowfiles. If it finds 1’000’000 it sends out 1’000’000 flowfiles. The processor can’t hold back some flows and knows nothing about your queue, so it sends out everything in one “batch” per execution – even if it is more than the queue size. Hope this explains the behavior a bit. Cheers From: "wangl...@geekplus.com.cn" <wangl...@geekplus.com.cn> Reply to: "us...@nifi.apache.org" <us...@nifi.apache.org> Date: Wednesday, 16 October 2019 at 08:10 To: users <us...@nifi.apache.org>, dev <dev@nifi.apache.org> Subject: NiFi backpressure not work As the picture shows. I have set the backpressure object threshold to 10000, but picture shows there's more than 1,000,000. Why this happens? Thanks, Lei wangl...@geekplus.com.cn
