[
https://issues.apache.org/jira/browse/FLINK-11848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Anderson updated FLINK-11848:
-----------------------------------
Description:
Recently we are doing some streaming jobs with apache flink. There are multiple
KAFKA topics with a format as xxxxxx_yy-mm-dd. We used a topic regex pattern to
let a consumer to consume those topics. However, if we delete some older
topics, it seems that the metadata in consumer does not update properly so It
still remember those outdated topic in its topic list, which leads to
*UNKNOWN_TOPIC_EXCEPTION*. We must restart the consumer job to recovery. It
seems to occur in producer as well. Any idea to solve this problem? Thank you
very much!
was:
Recently we are doing some streaming jobs with apache flink. There are multiple
KAFKA topics with a format as xxxxxx_yy-mm-dd. We used a topic regex pattern to
let a consumer to consume those topics. However, if we delete some older
topics, it seems that the metadata in consumer does not update properly so It
still remember those outdated topic in its topic list, which leads to
*UNKNOWN_TOPIC_EXCEPTIOIN*. We must restart the consumer job to recovery. It
seems to occur in producer as well. Any idea to solve this problem? Thank you
very much!
> Delete outdated kafka topics caused UNKNOWN_TOPIC_EXCEPTIION
> ------------------------------------------------------------
>
> Key: FLINK-11848
> URL: https://issues.apache.org/jira/browse/FLINK-11848
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Affects Versions: 1.6.4
> Reporter: Shengnan YU
> Priority: Major
>
> Recently we are doing some streaming jobs with apache flink. There are
> multiple KAFKA topics with a format as xxxxxx_yy-mm-dd. We used a topic regex
> pattern to let a consumer to consume those topics. However, if we delete some
> older topics, it seems that the metadata in consumer does not update properly
> so It still remember those outdated topic in its topic list, which leads to
> *UNKNOWN_TOPIC_EXCEPTION*. We must restart the consumer job to recovery. It
> seems to occur in producer as well. Any idea to solve this problem? Thank you
> very much!
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
