Re: [External] : Re: Use FlinkKafkaConsumer to synchronize multiple Kafka topics

2021-09-10 Thread Yan Wang
that FlinkKafkaConsumer will fail hard and stop retrieving other two live topics. Thank you very much! Thanks, Yan From: Arvid Heise Date: Thursday, September 2, 2021 at 1:27 PM To: Yan Wang Cc: user@flink.apache.org Subject: [External] : Re: Use FlinkKafkaConsumer to synchronize multiple Kafka

Re: Use FlinkKafkaConsumer to synchronize multiple Kafka topics

2021-09-02 Thread Arvid Heise
Hi Yan, Afaik this is not directly supported and would be surprising to other users since it's a rather specific requirement. In fact, Flink delegates reading the topics to Kafka consumer API and I suspect that the warning you received is also coming from Kafka consumer (I have not found a respect

Use FlinkKafkaConsumer to synchronize multiple Kafka topics

2021-09-01 Thread Yan Wang
Hi, We are currently using a single FlinkKafkaConsumer to consume multiple Kafka topics, however, we find that if one of the Kafka topics goes down at run time(like rebooting one of the topics), the FlinkKafkaConsumer will keep throwing warning message of the dead Kafka topic, and will also con