Thanks Gordon. Don't worry, I'll be careful to not have empty partitions until the next release. Also, I'll keep an eye to FLINK-5479 and if at some point I see that there is a fix and the issue bothers us too much I'll try to apply the patch myself to the latest stable release.
Gerard On Wed, Dec 13, 2017 at 10:31 AM, Tzu-Li (Gordon) Tai <tzuli...@apache.org> wrote: > Hi, > > I've just elevated FLINK-5479 to BLOCKER for 1.5. > > Unfortunately, AFAIK there is no easy workaround solution for this issue > yet in the releases so far. > The min watermark logic that controls per-partition watermark emission is > hidden inside the consumer, making it hard to work around it. > > One possible solution I can imagine, but perhaps not that trivial, is to > inject some special marker event into all partitions periodically. > The watermark assigner should be able to recognize this special marker and > try to provide some watermark for it. > Another option is that I can provide some patch you can apply for a custom > build of the Kafka connector that handles partition idleness properly. > However, given that we're aiming for a faster release cycle for Flink 1.5 > (proposed release date is Feb. 2018), it might not be worth the extra > maintenance effort on your side of a custom build. > > Best, > Gordon > > > On Tue, Dec 12, 2017 at 9:28 PM, gerardg <ger...@talaia.io> wrote: > >> I'm also affected by this behavior. There are no updates in FLINK-5479 but >> did you manage to find a way to workaround this? >> >> Thanks, >> >> Gerard >> >> >> >> -- >> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4. >> nabble.com/ >> > >
