Hi, Madan. This error seems like that there are some problems when the consumer tries to read the topic metadata. If you use the same source for these topics, the kafka connector cannot skip one of them. As you say, you need to modify the connector's default behavior. Maybe you should read the code in KafkaSourceEnumerator to skip this error.
Best, Hang Junrui Lee <jrlee....@gmail.com> 于2023年11月6日周一 14:30写道： > Hi Madan, > > Do you mean you want to restart only the failed tasks, rather than > restarting the entire pipeline region? As far as I know, currently Flink > does not support task-level restart, but requires restarting the pipeline > region. > > Best, > Junrui > > Madan D via user <user@flink.apache.org> 于2023年10月11日周三 12:37写道： > >> Hello Team, >> We are running the Flink pipeline by consuming data from multiple topics, >> but we recently encountered that if there's one topic having issues with >> participation, etc., the whole Flink pipeline is failing, which is >> affecting topics. Is there a way we can make Flink Piplein keep running >> even after one of the topics has an issue? We tried to handle exceptions to >> make sure the job wouldn't fail, but it didn't help out. >> >> Caused by: java.lang.RuntimeException: Failed to get metadata for topics >> >> >> Can you please provide any insights? >> >> >> Regards, >> Madan >> >
