Hi Sohi! On Wed, Jan 23, 2019 at 9:01 PM sohimankotia <sohimanko...@gmail.com> wrote:
> Hi, > > Let's say I have flink Kafka consumer read from 3 topics , [ T-1 ,T-2,T-3 > ] > . > > - T1 and T2 are having partitions equal to 100 > - T3 is having partitions equal to 60 > - Flink Task (parallelism is 50) > There isn't any means of prioritizing some partitions / topics to be read first before others in the Flink Kafka Consumer. > > How flink will prioritize Kafka topic ? > > If T-3 has more lag than other topics will flink give higher priority to > T-3 > No. However, there is some relevant discussion that would be helpful for these situations. Please see [1]. Long story short, if sources are aligned by event-time, the partitions that are lagging behind (in event-time) will slow down other source subtasks until the lagging partition has caught up. This is a feature that is still in discussion. Cheers, Gordon [1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-27-Refactor-Source-Interface-td24952.html > ? > > > Thanks and Regards > Sohi > > > > -- > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ >
